Science.gov

Sample records for lipreading

  1. Script activation in lipreading.

    PubMed

    Samuelsson, S; Rönnberg, J

    1991-01-01

    In three experiments the roles of script, basic and low-level contexts in lipreading (Abbott et al., 1985) were studied. Three scripts were used: a restaurant, a railway station, and a clothing store. Presence or absence of script headers constituted a between-subjects factor. Basic and low-level context, as well as test-sentences (referring to basic and low-level of abstraction), were within-subjects factors. Also, different kinds of working memory tests were employed to assess potentially important information-processing skills for lipreading. The results show that (a) lipreaders benefit from a compatibility between context level and sentence type, (b) the directions of inferences were asymmetric such that the basic context prevented inferences at the low-level of abstraction, but not vice versa, (c) low-level context in general was most useful for skilled lipreaders in the script group, (d) skilled lipreaders in the non-script group took most advantage of basic level sentences, and (e) working memory capacity did not prove to be important for lipreading skill. Guessing and pure lipreading control experiments established that each of these potentially confounding factors only accounted for about 5% of the lipreading performance, and thus, can not explain the obtained interactive effects between visual information conveyed by lip movements and context. Theorizing departing from prototypicality vs. script accounts of inference-making was critically evaluated, and a predictive script activation mechanism, relevant to lipreading skill, was suggested.

  2. Voice pitch as an aid to lipreading.

    PubMed

    Rosen, S M; Fourcin, A J; Moore, B C

    1981-05-14

    The totally deafened adult, unable to make use of a hearing aid, has no alternative to lipreading for everyday communication. Lipreading, however, is no substitute for hearing speech. Many lipreaders have great difficulty in ideal conditions and even the best lipreaders find the task demanding and tiring. Prosthetic attempts to substitute for lost hearing have centred on three distinct types of intervention, visual, tactile, and electrocochlear. As none of these is likely to yield a good understanding of a speech independent of lipreading in the near future, we have attempted to isolate relatively simple patterns of stimulation that, although not intelligible in themselves, well aid lipreading. From this point of view, the fundamental frequency or 'pitch' of the voice is the most important pattern element because if provides both segmental and suprasegmental information and is practically invisible. It thus complements the visual information already available on the face. As we show here, with the voice pitch presented acoustically, normal listeners can lipread a speaker reading continuous text at up to two and a half times the rate possible on the basis of lipreading alone. The pitch signal by itself, of course, is completely unintelligible. Although our work is primarily concerned with methods of electrical stimulation of the cochlea, it has implications for other sensory substitution techniques, the design of special purpose hearing aids and current theories of speech perception.

  3. Effects of Consonantal Context on Vowel Lipreading.

    ERIC Educational Resources Information Center

    Montgomery, Allen A.; And Others

    1987-01-01

    The effects of consonantal context on vowel lipreading were assessed for 30 adult males with mild-to-moderate sensorineural hearing loss who lipread videotape recordings of two female talkers. Results indicated that vowel intelligibility was significantly poorer in most contexts involving highly visible consonants. (Author/DB)

  4. Lip-reading enhancement for law enforcement

    NASA Astrophysics Data System (ADS)

    Theobald, Barry J.; Harvey, Richard; Cox, Stephen J.; Lewis, Colin; Owen, Gari P.

    2006-09-01

    Accurate lip-reading techniques would be of enormous benefit for agencies involved in counter-terrorism and other law-enforcement areas. Unfortunately, there are very few skilled lip-readers, and it is apparently a difficult skill to transmit, so the area is under-resourced. In this paper we investigate the possibility of making the lip-reading task more amenable to a wider range of operators by enhancing lip movements in video sequences using active appearance models. These are generative, parametric models commonly used to track faces in images and video sequences. The parametric nature of the model allows a face in an image to be encoded in terms of a few tens of parameters, while the generative nature allows faces to be re-synthesised using the parameters. The aim of this study is to determine if exaggerating lip-motions in video sequences by amplifying the parameters of the model improves lip-reading ability. We also present results of lip-reading tests undertaken by experienced (but non-expert) adult subjects who claim to use lip-reading in their speech recognition process. The results, which are comparisons of word error-rates on unprocessed and processed video, are mixed. We find that there appears to be the potential to improve the word error rate but, for the method to improve the intelligibility there is need for more sophisticated tracking and visual modelling. Our technique can also act as an expression or visual gesture amplifier and so has applications to animation and the presentation of information via avatars or synthetic humans.

  5. Speech Perception Results: Audition and Lipreading Enhancement.

    ERIC Educational Resources Information Center

    Geers, Ann; Brenner, Chris

    1994-01-01

    This paper describes changes in speech perception performance of deaf children using cochlear implants, tactile aids, or conventional hearing aids over a three-year period. Eleven of the 13 children with cochlear implants were able to identify words on the basis of auditory consonant cues. Significant lipreading enhancement was also achieved with…

  6. Linking Visual and Kinesthetic Imagery in Lipreading Instruction.

    ERIC Educational Resources Information Center

    De Filippo, Carol Lee; And Others

    1995-01-01

    Four groups of 12 young adult deaf subjects evaluated 2 aspects of lipreading training: source of video feedback (self or trainer) and timing of feedback (during or after speech production). Results substantiate the beneficial effects of multisensory feedback by practicing lipreading of one's own speech production. (Author/JDD)

  7. The Development of Generative Lipreading Skills in Deaf Persons Using Cued Speech Training.

    ERIC Educational Resources Information Center

    Neef, Nancy A.; Iwata, Brian A

    1985-01-01

    Evaluation of the effects of cued speech on lipreading performance of two deaf males indicated that Ss were able to accurately lipread cued stimuli after cued speech training and that generalization of lipreading skills to novel nonsense syllables occurred. Cued speech training also appeared to facilitate lipreading performance with noncued…

  8. Lipreading in School-Age Children: The Roles of Age, Hearing Status, and Cognitive Ability

    ERIC Educational Resources Information Center

    Tye-Murray, Nancy; Hale, Sandra; Spehar, Brent; Myerson, Joel; Sommers, Mitchell S.

    2014-01-01

    Purpose: The study addressed three research questions: Does lipreading improve between the ages of 7 and 14 years? Does hearing loss affect the development of lipreading? How do individual differences in lipreading relate to other abilities? Method: Forty children with normal hearing (NH) and 24 with hearing loss (HL) were tested using 4…

  9. Lipreading in School-Age Children: The Roles of Age, Hearing Status, and Cognitive Ability

    ERIC Educational Resources Information Center

    Tye-Murray, Nancy; Hale, Sandra; Spehar, Brent; Myerson, Joel; Sommers, Mitchell S.

    2014-01-01

    Purpose: The study addressed three research questions: Does lipreading improve between the ages of 7 and 14 years? Does hearing loss affect the development of lipreading? How do individual differences in lipreading relate to other abilities? Method: Forty children with normal hearing (NH) and 24 with hearing loss (HL) were tested using 4…

  10. Recalibration of Phonetic Categories by Lipread Speech versus Lexical Information

    ERIC Educational Resources Information Center

    van Linden, Sabine; Vroomen, Jean

    2007-01-01

    Listeners hearing an ambiguous phoneme flexibly adjust their phonetic categories in accordance with information telling what the phoneme should be (i.e., recalibration). Here the authors compared recalibration induced by lipread versus lexical information. Listeners were exposed to an ambiguous phoneme halfway between /t/ and /p/ dubbed onto a…

  11. Tactual display of consonant voicing as a supplement to lipreading.

    PubMed

    Yuan, Hanfeng; Reed, Charlotte M; Durlach, Nathaniel I

    2005-08-01

    This research is concerned with the development and evaluation of a tactual display of consonant voicing to supplement the information available through lipreading for persons with profound hearing impairment. The voicing cue selected is based on the envelope onset asynchrony derived from two different filtered bands (a low-pass band and a high-pass band) of speech. The amplitude envelope of each of the two bands was used to modulate a different carrier frequency which in turn was delivered to one of the two fingers of a tactual stimulating device. Perceptual evaluations of speech reception through this tactual display included the pairwise discrimination of consonants contrasting voicing and identification of a set of 16 consonants under conditions of the tactual cue alone (T), lipreading alone (L), and the combined condition (L + T). The tactual display was highly effective for discriminating voicing at the segmental level and provided a substantial benefit to lipreading on the consonant-identification task. No such benefits of the tactual cue were observed, however, for lipreading of words in sentences due perhaps to difficulties in integrating the tactual and visual cues and to insufficient training on the more difficult task of connected-speech reception.

  12. The effects of age and gender on lipreading abilities.

    PubMed

    Tye-Murray, Nancy; Sommers, Mitchell S; Spehar, Brent

    2007-01-01

    Age-related declines for many sensory and cognitive abilities are greater for males than for females. The primary purpose of the present investigation was to consider whether age-related changes in lipreading abilities are similar for men and women by comparing the lipreading abilities of separate groups of younger and older adults. Older females, older males, younger females and younger males completed vision-only speech recognition tests of: (1) 13 consonants in a vocalic /i/-C-/i/ environment; (2) words in a carrier phrase; and (3) meaningful sentences. In addition to percent correct performance, consonant data were analyzed for performance within viseme categories. The results suggest that while older adults do not lipread as well as younger adults, the difference between older and younger participants was comparable across gender. We also found no differences in the lipreading abilities of males and females, regardless of stimulus type (i.e., consonants, words, sentences), a finding that differs from some reports by previous investigators (e.g., Dancer, Krain, Thompson, Davis, & Glenn, 1994).

  13. Lip-reading abilities in a subject with congenital prosopagnosia.

    PubMed

    Wathour, J; Decat, M; Vander Linden, F; Deggouj, N

    2015-01-01

    We present the case of an individual with congenital prosopagnosia or "face blindness", a disorder where the ability to recognize faces is impaired. We studied the lip-reading ability and audiovisual perception of this subject using a DVD with four conditions (audiovisual congruent, auditory, visual, and audiovisual incongruent) and compared results with a normal patient cohort. The patient had no correct responses in the visual lip-reading task; whereas, he improved in the audiovisual congruent task. In the audiovisual incongruent task, the patient provided one response; thus, he was able to lip-read. (He was able to use lip-reading/to use labial informations) This patient perceived only global dynamic facial movements, not the fine ones. He had a sufficient complementary use of lip-reading in audiovisual tasks, but not visual ones. These data are consistent with abnormal development of the pathways used for visual speech perception and associated with second-order face processing disorders and normal development of the audiovisual network for speech perception.

  14. Lip-Reading by Deaf and Hearing Children

    ERIC Educational Resources Information Center

    Conradm, R.

    1977-01-01

    A group of profoundly deaf 15-year-old subjects with no other handicap and of average non-verbal intelligence were given a lip-reading test. The same test was given to comparable hearing subjects "deafened" by white noise masking. The difference between the groups was not significant. (Editor)

  15. A UMACE Filter Approach to Lipreading in Biometric Authentication System

    NASA Astrophysics Data System (ADS)

    Athiar Ramli, Dzati; Samad, Salina Abdul; Hussain, Aini

    Visual speech information, for example the appearance and the movement of lip during speech utterance can characterize a person`s identity and therefore it can be used in personal authentication systems. In this study, we propose a novel approach by using lipreading data i.e., the sequence of entire region of mouth area produced during speech and the implementation of the Unconstrained Minimum Average Correlation Energy (UMACE) filter as a classifier for biometric authentication. The system performance is also enhanced by the implementation of multi sample fusion scheme using average operator. The results obtained from using a Digit Database shows that the use of lipreading information and UMACE filter has good potentials and is highly effective in reducing false acceptance and false rejection rates for speaker verification system performance.

  16. Recalibration of phonetic categories by lipread speech versus lexical information.

    PubMed

    van Linden, Sabine; Vroomen, Jean

    2007-12-01

    Listeners hearing an ambiguous phoneme flexibly adjust their phonetic categories in accordance with information telling what the phoneme should be (i.e., recalibration). Here the authors compared recalibration induced by lipread versus lexical information. Listeners were exposed to an ambiguous phoneme halfway between /t/ and /p/ dubbed onto a face articulating /t/ or /p/ or embedded in a Dutch word ending in /t/ (e.g., groot [big]) or /p/ (knoop [button]). In a posttest, participants then categorized auditory tokens as /t/ or /p/. Lipread and lexical aftereffects were comparable in size (Experiment 1), dissipated about equally fast (Experiment 2), were enhanced by exposure to a contrast phoneme (Experiment 3), and were not affected by a 3-min silence interval (Experiment 4). Exposing participants to 1 instead of both phoneme categories did not make the phenomenon more robust (Experiment 5). Despite the difference in nature (bottom-up vs. top-down information), lipread and lexical information thus appear to serve a similar role in phonetic adjustments.

  17. Neural responses to silent lipreading in normal hearing male and female subjects.

    PubMed

    Ruytjens, Liesbet; Albers, Frans; van Dijk, Pim; Wit, Hero; Willemsen, Antoon

    2006-09-01

    In the past, researchers investigated silent lipreading in normal hearing subjects with functional neuroimaging tools and showed how the brain processes visual stimuli that are normally accompanied by an auditory counterpart. Previously, we showed activation differences between males and females in primary auditory cortex during silent lipreading, i.e. only the female group significantly activated the primary auditory region during lipreading. Here we report and discuss the overall activation pattern in males and females. We used positron emission tomography to study silent lipreading in 19 normal hearing subjects (nine females). Prior to scanning, subjects were tested on their lipreading ability and only good lipreaders were included in the study. Silent lipreading was compared with a static image. In the whole group, activations were found mainly in the left hemisphere with major clusters in superior temporal, inferior parietal, inferior frontal and precentral regions. The female group showed more clusters and these clusters were larger than in the male group. Sex differences were found mainly in right inferior frontal and left inferior parietal regions and to a lesser extent in bilateral angular and precentral gyri. The sex differences in the parietal multimodal region support our previous hypothesis that the male and female brain process visual speech stimuli differently without differences in overt lipreading ability. Specifically, females associate the visual speech image with the corresponding auditory speech sound whereas males focus more on the visual image itself.

  18. Lipreading Ability and Its Cognitive Correlates in Typically Developing Children and Children with Specific Language Impairment

    ERIC Educational Resources Information Center

    Heikkilä, Jenni; Lonka, Eila; Ahola, Sanna; Meronen, Auli; Tiippana, Kaisa

    2017-01-01

    Purpose: Lipreading and its cognitive correlates were studied in school-age children with typical language development and delayed language development due to specific language impairment (SLI). Method: Forty-two children with typical language development and 20 children with SLI were tested by using a word-level lipreading test and an extensive…

  19. The effect of lip-reading on primary stream segregation.

    PubMed

    Devergie, Aymeric; Grimault, Nicolas; Gaudrain, Etienne; Healy, Eric W; Berthommier, Frédéric

    2011-07-01

    Lip-reading has been shown to improve the intelligibility of speech in multitalker situations, where auditory stream segregation naturally takes place. This study investigated whether the benefit of lip-reading is a result of a primary audiovisual interaction that enhances the obligatory streaming mechanism. Two behavioral experiments were conducted involving sequences of French vowels that alternated in fundamental frequency. In Experiment 1, subjects attempted to identify the order of items in a sequence. In Experiment 2, subjects attempted to detect a disruption to temporal isochrony across alternate items. Both tasks are disrupted by streaming, thus providing a measure of primary or obligatory streaming. Visual lip gestures articulating alternate vowels were synchronized with the auditory sequence. Overall, the results were consistent with the hypothesis that visual lip gestures enhance segregation by affecting primary auditory streaming. Moreover, increases in the naturalness of visual lip gestures and auditory vowels, and corresponding increases in audiovisual congruence may potentially lead to increases in the effect of visual lip gestures on streaming. © 2011 Acoustical Society of America

  20. Experience with a talker can transfer across modalities to facilitate lipreading.

    PubMed

    Sanchez, Kauyumari; Dias, James W; Rosenblum, Lawrence D

    2013-10-01

    Rosenblum, Miller, and Sanchez (Psychological Science, 18, 392-396, 2007) found that subjects first trained to lip-read a particular talker were then better able to perceive the auditory speech of that same talker, as compared with that of a novel talker. This suggests that the talker experience a perceiver gains in one sensory modality can be transferred to another modality to make that speech easier to perceive. An experiment was conducted to examine whether this cross-sensory transfer of talker experience could occur (1) from auditory to lip-read speech, (2) with subjects not screened for adequate lipreading skill, (3) when both a familiar and an unfamiliar talker are presented during lipreading, and (4) for both old (presentation set) and new words. Subjects were first asked to identify a set of words from a talker. They were then asked to perform a lipreading task from two faces, one of which was of the same talker they heard in the first phase of the experiment. Results revealed that subjects who lip-read from the same talker they had heard performed better than those who lip-read a different talker, regardless of whether the words were old or new. These results add further evidence that learning of amodal talker information can facilitate speech perception across modalities and also suggest that this information is not restricted to previously heard words.

  1. Experience with a Talker Can Transfer Across Modalities to Facilitate Lipreading

    PubMed Central

    Sanchez, Kauyumari; Dias, James W.; Rosenblum, Lawrence D.

    2013-01-01

    Rosenblum, Miller, and Sanchez (2007) found that participants first trained to lipread a particular talker were then better able to perceive the auditory speech of that same talker, compared to that of a novel talker. This suggests that the talker experience a perceiver gains in one sensory modality can be transferred to another modality to make that speech easier to perceive. An experiment was conducted to examine whether this cross-sensory transfer of talker experience could occur: 1) from auditory to lipread speech; 2) with subjects not screened for adequate lipreading skill; 3) when both a familiar and unfamiliar talker are presented during lipreading; and 4) for both old (presentation set) and new words. Subjects were first asked to identify a set of words from a talker. They were then asked to perform a lipreading task from two faces, one of which was of the same talker they heard in the first phase of the experiment. Results revealed that subjects who lipread from the same talker they had heard performed better than those who lipead a different talker, regardless of whether the words were old or new. These results add further evidence that learning of amodal talker information can facilitate speech perception across modalities and also suggest that this information is not restricted to previously heard words. PMID:23955059

  2. The Neural Basis of Speech Perception through Lipreading and Manual Cues: Evidence from Deaf Native Users of Cued Speech

    PubMed Central

    Aparicio, Mario; Peigneux, Philippe; Charlier, Brigitte; Balériaux, Danielle; Kavec, Martin; Leybaert, Jacqueline

    2017-01-01

    We present here the first neuroimaging data for perception of Cued Speech (CS) by deaf adults who are native users of CS. CS is a visual mode of communicating a spoken language through a set of manual cues which accompany lipreading and disambiguate it. With CS, sublexical units of the oral language are conveyed clearly and completely through the visual modality without requiring hearing. The comparison of neural processing of CS in deaf individuals with processing of audiovisual (AV) speech in normally hearing individuals represents a unique opportunity to explore the similarities and differences in neural processing of an oral language delivered in a visuo-manual vs. an AV modality. The study included deaf adult participants who were early CS users and native hearing users of French who process speech audiovisually. Words were presented in an event-related fMRI design. Three conditions were presented to each group of participants. The deaf participants saw CS words (manual + lipread), words presented as manual cues alone, and words presented to be lipread without manual cues. The hearing group saw AV spoken words, audio-alone and lipread-alone. Three findings are highlighted. First, the middle and superior temporal gyrus (excluding Heschl’s gyrus) and left inferior frontal gyrus pars triangularis constituted a common, amodal neural basis for AV and CS perception. Second, integration was inferred in posterior parts of superior temporal sulcus for audio and lipread information in AV speech, but in the occipito-temporal junction, including MT/V5, for the manual cues and lipreading in CS. Third, the perception of manual cues showed a much greater overlap with the regions activated by CS (manual + lipreading) than lipreading alone did. This supports the notion that manual cues play a larger role than lipreading for CS processing. The present study contributes to a better understanding of the role of manual cues as support of visual speech perception in the framework

  3. The Neural Basis of Speech Perception through Lipreading and Manual Cues: Evidence from Deaf Native Users of Cued Speech.

    PubMed

    Aparicio, Mario; Peigneux, Philippe; Charlier, Brigitte; Balériaux, Danielle; Kavec, Martin; Leybaert, Jacqueline

    2017-01-01

    We present here the first neuroimaging data for perception of Cued Speech (CS) by deaf adults who are native users of CS. CS is a visual mode of communicating a spoken language through a set of manual cues which accompany lipreading and disambiguate it. With CS, sublexical units of the oral language are conveyed clearly and completely through the visual modality without requiring hearing. The comparison of neural processing of CS in deaf individuals with processing of audiovisual (AV) speech in normally hearing individuals represents a unique opportunity to explore the similarities and differences in neural processing of an oral language delivered in a visuo-manual vs. an AV modality. The study included deaf adult participants who were early CS users and native hearing users of French who process speech audiovisually. Words were presented in an event-related fMRI design. Three conditions were presented to each group of participants. The deaf participants saw CS words (manual + lipread), words presented as manual cues alone, and words presented to be lipread without manual cues. The hearing group saw AV spoken words, audio-alone and lipread-alone. Three findings are highlighted. First, the middle and superior temporal gyrus (excluding Heschl's gyrus) and left inferior frontal gyrus pars triangularis constituted a common, amodal neural basis for AV and CS perception. Second, integration was inferred in posterior parts of superior temporal sulcus for audio and lipread information in AV speech, but in the occipito-temporal junction, including MT/V5, for the manual cues and lipreading in CS. Third, the perception of manual cues showed a much greater overlap with the regions activated by CS (manual + lipreading) than lipreading alone did. This supports the notion that manual cues play a larger role than lipreading for CS processing. The present study contributes to a better understanding of the role of manual cues as support of visual speech perception in the framework

  4. Silent lipreading and covert speech production suppress processing of non-linguistic sounds in auditory cortex.

    PubMed

    Balk, Marja H; Kari, Heini; Kauramäki, Jaakko; Ahveninen, Jyrki; Sams, Mikko; Autti, Taina; Jääskeläinen, Iiro P

    2013-02-06

    Previous studies have suggested that speech motor system mediates suppression by silent lipreading of electromagnetic auditory cortex responses to pure tones at about 100 ms from sound onset. We used sparse sampling functional magnetic resonance imaging (fMRI) at 3 Tesla to map auditory-cortex foci of suppressant effects during silent lipreading and covert self-production. Streams of video clips were presented simultaneously with 1/3 octave noise bursts centered at 250 Hz (low frequency, LF) or 2000 Hz (mid-frequency, MF), or during no auditory stimulation. In different conditions, the subjects were a) to press a button whenever they lipread the face articulate the same consecutive Finnish vowels /a/, /i/, /o/, and /y/, b) covertly selfproducing vowels while viewing still face image, or c) to press a button whenever a circle pictured on top of the lips expanded into oval shape of the same orientation twice in a row. The regions of interest (ROIs) within the superior temporal lobes of each hemisphere were defined by contrasting MF and LF stimulation against silence. Contrasting the nonlinguistic (i.e., expanding circle) vs. linguistic (i.e., lipreading and covert self-production) conditions within these ROIs showed significant suppression of hemodynamic activity to MF sounds in the linguistic condition in left hemisphere first transverse sulcus (FTS) and right hemisphere superior temporal gyrus (STG) lateral to Heschl's sulcus (HS). These findings suggest that the speech motor system mediates suppression of auditory-cortex processing of non-linguistic sounds during silent lipreading and covert self-production in left hemisphere FST and right hemisphere STG lateral to HS.

  5. Lipreading and covert speech production similarly modulate human auditory-cortex responses to pure tones

    PubMed Central

    Kauramäki, Jaakko; Jääskeläinen, Iiro P.; Hari, Riitta; Möttönen, Riikka; Rauschecker, Josef P.; Sams, Mikko

    2010-01-01

    Watching the lips of a speaker enhances speech perception. At the same time, the 100-ms response to speech sounds is suppressed in the observer’s auditory cortex. Here, we used whole-scalp 306-channel magnetoencephalography (MEG) to study whether lipreading modulates human auditory processing already at the level of the most elementary sound features, i.e., pure tones. We further envisioned the temporal dynamics of the suppression to tell whether the effect is driven by top-down influences. Nineteen subjects were presented with 50-ms tones spanning six octaves (125–8000 Hz) (1) during “lipreading”, i.e. when they watched video clips of silent articulations of Finnish vowels /a/, /i/, /o/, /y/, and reacted to vowels presented twice in a row, (2) during a visual control task, (3) during a still-face passive control condition, and, in a separate experiment with a subset of nine subjects, (4) during covert production of the same vowels. Auditory-cortex 100-ms responses (N100m) were equally suppressed in the lipreading and covert speech-production tasks compared with the visual control and baseline tasks; the effects involved all frequencies and were most prominent in the left hemisphere. Responses to tones presented at different times with respect to the onset of the visual articulation showed significantly increased N100m suppression immediately after the articulatory gesture. These findings suggest that the lipreading-related suppression in the auditory cortex is caused by top-down influences, possibly by an efference copy from the speech-production system, generated during both own speech and lipreading. PMID:20107058

  6. Lip-read me now, hear me better later: cross-modal transfer of talker-familiarity effects.

    PubMed

    Rosenblum, Lawrence D; Miller, Rachel M; Sanchez, Kauyumari

    2007-05-01

    There is evidence that for both auditory and visual speech perception, familiarity with the talker facilitates speech recognition. Explanations of these effects have concentrated on the retention of talker information specific to each of these modalities. It could be, however, that some amodal, talker-specific articulatory-style information facilitates speech perception in both modalities. If this is true, then experience with a talker in one modality should facilitate perception of speech from that talker in the other modality. In a test of this prediction, subjects were given about 1 hr of experience lipreading a talker and were then asked to recover speech in noise from either this same talker or a different talker. Results revealed that subjects who lip-read and heard speech from the same talker performed better on the speech-in-noise task than did subjects who lip-read from one talker and then heard speech from a different talker.

  7. Visual Cortical Entrainment to Motion and Categorical Speech Features during Silent Lipreading

    PubMed Central

    O’Sullivan, Aisling E.; Crosse, Michael J.; Di Liberto, Giovanni M.; Lalor, Edmund C.

    2017-01-01

    Speech is a multisensory percept, comprising an auditory and visual component. While the content and processing pathways of audio speech have been well characterized, the visual component is less well understood. In this work, we expand current methodologies using system identification to introduce a framework that facilitates the study of visual speech in its natural, continuous form. Specifically, we use models based on the unheard acoustic envelope (E), the motion signal (M) and categorical visual speech features (V) to predict EEG activity during silent lipreading. Our results show that each of these models performs similarly at predicting EEG in visual regions and that respective combinations of the individual models (EV, MV, EM and EMV) provide an improved prediction of the neural activity over their constituent models. In comparing these different combinations, we find that the model incorporating all three types of features (EMV) outperforms the individual models, as well as both the EV and MV models, while it performs similarly to the EM model. Importantly, EM does not outperform EV and MV, which, considering the higher dimensionality of the V model, suggests that more data is needed to clarify this finding. Nevertheless, the performance of EMV, and comparisons of the subject performances for the three individual models, provides further evidence to suggest that visual regions are involved in both low-level processing of stimulus dynamics and categorical speech perception. This framework may prove useful for investigating modality-specific processing of visual speech under naturalistic conditions. PMID:28123363

  8. Fast 3D NIR systems for facial measurement and lip-reading

    NASA Astrophysics Data System (ADS)

    Brahm, Anika; Ramm, Roland; Heist, Stefan; Rulff, Christian; Kühmstedt, Peter; Notni, Gunther

    2017-05-01

    Structured-light projection is a well-established optical method for the non-destructive contactless three-dimensional (3D) measurement of object surfaces. In particular, there is a great demand for accurate and fast 3D scans of human faces or facial regions of interest in medicine, safety, face modeling, games, virtual life, or entertainment. New developments of facial expression detection and machine lip-reading can be used for communication tasks, future machine control, or human-machine interactions. In such cases, 3D information may offer more detailed information than 2D images which can help to increase the power of current facial analysis algorithms. In this contribution, we present new 3D sensor technologies based on three different methods of near-infrared projection technologies in combination with a stereo vision setup of two cameras. We explain the optical principles of an NIR GOBO projector, an array projector and a modified multi-aperture projection method and compare their performance parameters to each other. Further, we show some experimental measurement results of applications where we realized fast, accurate, and irritation-free measurements of human faces.

  9. Audiovisual Speech Integration and Lipreading in Autism

    ERIC Educational Resources Information Center

    Smith, Elizabeth G.; Bennetto, Loisa

    2007-01-01

    Background: During speech perception, the ability to integrate auditory and visual information causes speech to sound louder and be more intelligible, and leads to quicker processing. This integration is important in early language development, and also continues to affect speech comprehension throughout the lifespan. Previous research shows that…

  10. Audiovisual Speech Integration and Lipreading in Autism

    ERIC Educational Resources Information Center

    Smith, Elizabeth G.; Bennetto, Loisa

    2007-01-01

    Background: During speech perception, the ability to integrate auditory and visual information causes speech to sound louder and be more intelligible, and leads to quicker processing. This integration is important in early language development, and also continues to affect speech comprehension throughout the lifespan. Previous research shows that…

  11. Visual words for lip-reading

    NASA Astrophysics Data System (ADS)

    Hassanat, Ahmad B. A.; Jassim, Sabah

    2010-04-01

    In this paper, the automatic lip reading problem is investigated, and an innovative approach to providing solutions to this problem has been proposed. This new VSR approach is dependent on the signature of the word itself, which is obtained from a hybrid feature extraction method dependent on geometric, appearance, and image transform features. The proposed VSR approach is termed "visual words". The visual words approach consists of two main parts, 1) Feature extraction/selection, and 2) Visual speech feature recognition. After localizing face and lips, several visual features for the lips where extracted. Such as the height and width of the mouth, mutual information and the quality measurement between the DWT of the current ROI and the DWT of the previous ROI, the ratio of vertical to horizontal features taken from DWT of ROI, The ratio of vertical edges to horizontal edges of ROI, the appearance of the tongue and the appearance of teeth. Each spoken word is represented by 8 signals, one of each feature. Those signals maintain the dynamic of the spoken word, which contains a good portion of information. The system is then trained on these features using the KNN and DTW. This approach has been evaluated using a large database for different people, and large experiment sets. The evaluation has proved the visual words efficiency, and shown that the VSR is a speaker dependent problem.

  12. Study of lip-reading detecting and locating technique

    NASA Astrophysics Data System (ADS)

    Wang, Lirong; Li, Jie; Zhao, Yanyan

    2008-03-01

    With the development of human computer interaction, lip reading technology has become a topic focus in the multimode technologic field. However, detecting and locating lip accurately are very difficult because lip contours of different people, varied illuminant conditions, head movements and other factors. Based on the methods of detecting and locating lip we proposed the methods which are based on the lips color extracted lip contour using the adaptive chromatic filter from the facial images. It is not sensitive to illumination, but appropriate chromatic lip filter is given by analyzing the entire face color and clustering statistics of lip color. It is proposed the combinable method which is preprocessing the face image including rotating the angle of face and improving image contrast in this paper and the lip region is analyzed clustering characteristics for the skin color and lip color, obtained adaptive chromatic filter which can prominent lips from the facial image. This method overcomes the varied illuminate, incline face. The experiments showed that it enhanced detection and location accurately through rough detecting lip region. It lay a good foundation for extraction the lip feature and tracking lip subsequently.

  13. Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing

    ERIC Educational Resources Information Center

    Spehar, Brent; Goebel, Stacey; Tye-Murray, Nancy

    2015-01-01

    Purpose: This study compared the use of 2 different types of contextual cues (sentence based and situation based) in 2 different modalities (visual only and auditory only). Method: Twenty young adults were tested with the Illustrated Sentence Test (Tye-Murray, Hale, Spehar, Myerson, & Sommers, 2014) and the Speech Perception in Noise Test…

  14. Lipreading and audiovisual speech recognition across the adult lifespan: Implications for audiovisual integration.

    PubMed

    Tye-Murray, Nancy; Spehar, Brent; Myerson, Joel; Hale, Sandra; Sommers, Mitchell

    2016-06-01

    In this study of visual (V-only) and audiovisual (AV) speech recognition in adults aged 22-92 years, the rate of age-related decrease in V-only performance was more than twice that in AV performance. Both auditory-only (A-only) and V-only performance were significant predictors of AV speech recognition, but age did not account for additional (unique) variance. Blurring the visual speech signal decreased speech recognition, and in AV conditions involving stimuli associated with equivalent unimodal performance for each participant, speech recognition remained constant from 22 to 92 years of age. Finally, principal components analysis revealed separate visual and auditory factors, but no evidence of an AV integration factor. Taken together, these results suggest that the benefit that comes from being able to see as well as hear a talker remains constant throughout adulthood and that changes in this AV advantage are entirely driven by age-related changes in unimodal visual and auditory speech recognition. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

  15. Lipreading and Audiovisual Speech Recognition across the Adult Lifespan: Implications for Audiovisual Integration

    PubMed Central

    Tye-Murray, Nancy; Spehar, Brent; Myerson, Joel; Hale, Sandra; Sommers, Mitchell

    2016-01-01

    In this study of visual (V-only) and audiovisual (AV) speech recognition in adults aged 22-92 years, the rate of age-related decrease in V-only performance was more than twice that in AV performance. Both auditory-only (A-only) and V-only performance were significant predictors of AV speech recognition, but age did not account for additional (unique) variance. Blurring the visual speech signal decreased speech recognition, and in AV conditions involving stimuli associated with equivalent unimodal performance for each participant, speech recognition remained constant from 22 to 92 years of age. Finally, principal components analysis revealed separate visual and auditory factors, but no evidence of an AV integration factor. Taken together, these results suggest that the benefit that comes from being able to see as well as hear a talker remains constant throughout adulthood, and that changes in this AV advantage are entirely driven by age-related changes in unimodal visual and auditory speech recognition. PMID:27294718

  16. Impact of Audio-Visual Asynchrony on Lip-Reading Effects -Neuromagnetic and Psychophysical Study-

    PubMed Central

    Yahata, Izumi; Kanno, Akitake; Sakamoto, Shuichi; Takanashi, Yoshitaka; Takata, Shiho; Nakasato, Nobukazu; Kawashima, Ryuta; Katori, Yukio

    2016-01-01

    The effects of asynchrony between audio and visual (A/V) stimuli on the N100m responses of magnetoencephalography in the left hemisphere were compared with those on the psychophysical responses in 11 participants. The latency and amplitude of N100m were significantly shortened and reduced in the left hemisphere by the presentation of visual speech as long as the temporal asynchrony between A/V stimuli was within 100 ms, but were not significantly affected with audio lags of -500 and +500 ms. However, some small effects were still preserved on average with audio lags of 500 ms, suggesting similar asymmetry of the temporal window to that observed in psychophysical measurements, which tended to be more robust (wider) for audio lags; i.e., the pattern of visual-speech effects as a function of A/V lag observed in the N100m in the left hemisphere grossly resembled that in psychophysical measurements on average, although the individual responses were somewhat varied. The present results suggest that the basic configuration of the temporal window of visual effects on auditory-speech perception could be observed from the early auditory processing stage. PMID:28030631

  17. Techniques for Assessing Auditory Speech Perception and Lipreading Enhancement in Young Deaf Children.

    ERIC Educational Resources Information Center

    Geers, Ann

    1994-01-01

    This paper examines the special considerations involved in selecting a speech perception test battery for young deaf children. The auditory-only tests consisted of closed-set word identification tasks and minimal-pairs syllable tasks. Additional tests included identification of words in sentences, open-set word recognition, and evaluation of…

  18. Impact of Audio-Visual Asynchrony on Lip-Reading Effects -Neuromagnetic and Psychophysical Study.

    PubMed

    Kawase, Tetsuaki; Yahata, Izumi; Kanno, Akitake; Sakamoto, Shuichi; Takanashi, Yoshitaka; Takata, Shiho; Nakasato, Nobukazu; Kawashima, Ryuta; Katori, Yukio

    2016-01-01

    The effects of asynchrony between audio and visual (A/V) stimuli on the N100m responses of magnetoencephalography in the left hemisphere were compared with those on the psychophysical responses in 11 participants. The latency and amplitude of N100m were significantly shortened and reduced in the left hemisphere by the presentation of visual speech as long as the temporal asynchrony between A/V stimuli was within 100 ms, but were not significantly affected with audio lags of -500 and +500 ms. However, some small effects were still preserved on average with audio lags of 500 ms, suggesting similar asymmetry of the temporal window to that observed in psychophysical measurements, which tended to be more robust (wider) for audio lags; i.e., the pattern of visual-speech effects as a function of A/V lag observed in the N100m in the left hemisphere grossly resembled that in psychophysical measurements on average, although the individual responses were somewhat varied. The present results suggest that the basic configuration of the temporal window of visual effects on auditory-speech perception could be observed from the early auditory processing stage.

  19. Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing

    ERIC Educational Resources Information Center

    Spehar, Brent; Goebel, Stacey; Tye-Murray, Nancy

    2015-01-01

    Purpose: This study compared the use of 2 different types of contextual cues (sentence based and situation based) in 2 different modalities (visual only and auditory only). Method: Twenty young adults were tested with the Illustrated Sentence Test (Tye-Murray, Hale, Spehar, Myerson, & Sommers, 2014) and the Speech Perception in Noise Test…

  20. Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing

    PubMed Central

    Goebel, Stacey; Tye-Murray, Nancy

    2015-01-01

    Purpose This study compared the use of 2 different types of contextual cues (sentence based and situation based) in 2 different modalities (visual only and auditory only). Method Twenty young adults were tested with the Illustrated Sentence Test (Tye-Murray, Hale, Spehar, Myerson, & Sommers, 2014) and the Speech Perception in Noise Test (Bilger, Nuetzel, Rabinowitz, & Rzeczkowski, 1984; Kalikow, Stevens, & Elliott, 1977) in the 2 modalities. The Illustrated Sentences Test presents sentences with no context and sentences accompanied by picture-based situational context cues. The Speech Perception in Noise Test presents sentences with low sentence-based context and sentences with high sentence-based context. Results Participants benefited from both types of context and received more benefit when testing occurred in the visual-only modality than when it occurred in the auditory-only modality. Participants' use of sentence-based context did not correlate with use of situation-based context. Cue usage did not correlate between the 2 modalities. Conclusions The ability to use contextual cues appears to be dependent on the type of cue and the presentation modality of the target word(s). In a theoretical sense, the results suggest that models of word recognition and sentence processing should incorporate the influence of multiple sources of information and recognize that the 2 types of context have different influences on speech perception. In a clinical sense, the results suggest that aural rehabilitation programs might provide training to optimize use of both kinds of contextual cues. PMID:25863923

  1. 77 FR 24554 - Culturally Significant Objects Imported for Exhibition; Determinations: “Quay Brothers: On...

    Federal Register 2010, 2011, 2012, 2013, 2014

    2012-04-24

    ... Pharmacist's Prescription for Lip-Reading Puppets'' AGENCY: State Department. ACTION: Notice. SUMMARY: Notice... Lip-Reading Puppets'' imported from abroad for temporary exhibition within the United States, are of...

  2. A Study of the Combined Use of a Hearing Aid and Tactual Aid in an Adult with Profound Hearing Loss

    ERIC Educational Resources Information Center

    Reed, Charlotte M.; Delhorne, Lorraine A.

    2006-01-01

    This study examined the benefits of the combined used of a hearing aid and tactual aid to supplement lip-reading in the reception of speech and for the recognition of environmental sounds in an adult with profound hearing loss. Speech conditions included lip-reading alone (L), lip-reading + tactual aid (L+TA) lip-reading + hearing aid (L+HA) and…

  3. Computer Aided Lip Reading Training Tool

    ERIC Educational Resources Information Center

    Sarmasik, Gamze; Dalkilic, Gokhan; Kut, Alp; Cebi, Yalcin; Serbetcioglu, Bulent

    2007-01-01

    Worldwide auditory-verbal education is becoming widespread for deaf children. But many prelingually, late-diagnosed deaf children and adults may utilize neither hearing aids nor cochlear implants and needed the support of lip-reading. Therefore, lip-reading skill remains to be important for oral education programmes of hearing impaired. The…

  4. Effects of a Wearable, Tactile Aid on Language Comprehension of Prelingual Profoundly Deaf Children.

    ERIC Educational Resources Information Center

    Proctor, Adele

    Factors influencing the use of nonacoustic aids (such as visual displays and tactile devices) with the hearing impaired are reviewed. The benefits of tactile devices in improving speech reading/lipreading and speech are pointed out. Tactile aids which provide information on rhythm, rate, intensity, and duration of speech increase lipreading and…

  5. Phonetic Recalibration Only Occurs in Speech Mode

    ERIC Educational Resources Information Center

    Vroomen, Jean; Baart, Martijn

    2009-01-01

    Upon hearing an ambiguous speech sound dubbed onto lipread speech, listeners adjust their phonetic categories in accordance with the lipread information (recalibration) that tells what the phoneme should be. Here we used sine wave speech (SWS) to show that this tuning effect occurs if the SWS sounds are perceived as speech, but not if the sounds…

  6. Can Explicit Training in Cued Speech Improve Phoneme Identification?

    ERIC Educational Resources Information Center

    Rees, R.; Fitzpatrick, C.; Foulkes, J.; Peterson, H.; Newton, C.

    2017-01-01

    When identifying phonemes in new spoken words, lipreading is an important source of information for many deaf people. Because many groups of phonemes are virtually indistinguishable by sight, deaf people are able to identify about 30% of phonemes when lipreading non-words. Cued speech (CS) is a system of hand shapes and hand positions used…

  7. Tracking Human Faces in Real-Time,

    DTIC Science & Technology

    1995-11-01

    human-computer interactive applications such as lip-reading and gaze tracking. The principle in developing this system can be extended to other tracking problems such as tracking the human hand for gesture recognition .

  8. Development of a speech autocuer

    NASA Technical Reports Server (NTRS)

    Bedles, R. L.; Kizakvich, P. N.; Lawson, D. T.; Mccartney, M. L.

    1980-01-01

    A wearable, visually based prosthesis for the deaf based upon the proven method for removing lipreading ambiguity known as cued speech was fabricated and tested. Both software and hardware developments are described, including a microcomputer, display, and speech preprocessor.

  9. Modalities of memory: is reading lips like hearing voices?

    PubMed

    Maidment, David W; Macken, Bill; Jones, Dylan M

    2013-12-01

    Functional similarities in verbal memory performance across presentation modalities (written, heard, lipread) are often taken to point to a common underlying representational form upon which the modalities converge. We show here instead that the pattern of performance depends critically on presentation modality and different mechanisms give rise to superficially similar effects across modalities. Lipread recency is underpinned by different mechanisms to auditory recency, and while the effect of an auditory suffix on an auditory list is due to the perceptual grouping of the suffix with the list, the corresponding effect with lipread speech is due to misidentification of the lexical content of the lipread suffix. Further, while a lipread suffix does not disrupt auditory recency, an auditory suffix does disrupt recency for lipread lists. However, this effect is due to attentional capture ensuing from the presentation of an unexpected auditory event, and is evident both with verbal and nonverbal auditory suffixes. These findings add to a growing body of evidence that short-term verbal memory performance is determined by modality-specific perceptual and motor processes, rather than by the storage and manipulation of phonological representations. Copyright © 2013 Elsevier B.V. All rights reserved.

  10. READING YOUR OWN LIPS: COMMON CODING THEORY AND VISUAL SPEECH PERCEPTION

    PubMed Central

    Tye-Murray, Nancy; Spehar, Brent P.; Myerson, Joel; Hale, Sandra; Sommers, Mitchell S.

    2012-01-01

    Common coding theory posits: 1) perceiving an action activates the same representations of motor plans that are activated by actually performing that action; 2) because of individual differences in the way actions are performed, observing recordings of one’s own previous behavior activates motor plans to an even greater degree than observing someone else’s behavior. We hypothesized that if observing oneself activates motor plans to a greater degree than observing others, and these activated plans contribute to perception, then people should be able to lipread silent video clips of their own previous utterances more accurately than they can lipread video clips of other talkers. As predicted, two groups of participants were able to lipread video clips of themselves recorded more than two weeks earlier significantly more accurately than video clips of others. These results suggest that visual input activates speech motor activity that links to word representations in the mental lexicon. PMID:23132604

  11. Perception of the auditory-visual illusion in speech perception by children with phonological disorders.

    PubMed

    Dodd, Barbara; McIntosh, Beth; Erdener, Dogu; Burnham, Denis

    2008-01-01

    An example of the auditory-visual illusion in speech perception, first described by McGurk and MacDonald, is the perception of [ta] when listeners hear [pa] in synchrony with the lip movements for [ka]. One account of the illusion is that lip-read and heard speech are combined in an articulatory code since people who mispronounce words respond differently from controls on lip-reading tasks. A same-different judgment task assessing perception of the illusion showed no difference in performance between controls and children with speech difficulties. Another experiment compared children with delayed and disordered speech on perception of the illusion. While neither group perceived many illusions, a significant interaction indicated that children with disordered phonology were strongly biased to the auditory component while the delayed group's response was more evenly split between the auditory and visual components of the illusion. These findings suggest that phonological processing, rather than articulation, supports lip-reading ability.

  12. Is automated conversion of video to text a reality?

    NASA Astrophysics Data System (ADS)

    Bowden, Richard; Cox, Stephen J.; Harvey, Richard W.; Lan, Yuxuan; Ong, Eng-Jon; Owen, Gari; Theobald, Barry-John

    2012-10-01

    A recent trend in law enforcement has been the use of Forensic lip-readers. Criminal activities are often recorded on CCTV or other video gathering systems. Knowledge of what suspects are saying enriches the evidence gathered but lip-readers, by their own admission, are fallible so, based on long term studies of automated lip-reading, we are investigating the possibilities and limitations of applying this technique under realistic conditions. We have adopted a step-by-step approach and are developing a capability when prior video information is available for the suspect of interest. We use the terminology video-to-text (V2T) for this technique by analogy with speech-to-text (S2T) which also has applications in security and law-enforcement.

  13. How to improve communication with deaf children in the dental clinic.

    PubMed

    Alsmark, Silvia San Bernardino; García, Joaquín; Martínez, María Rosa Mourelle; López, Nuria Esther Gallardo

    2007-12-01

    It may be difficult for hearing-impaired people to communicate with people who hear. In the health care area, there is often little awareness of the communication barriers faced by the deaf and, in dentistry, the attitude adopted towards the deaf is not always correct. A review is given of the basic rules and advice given for communicating with the hearing-impaired. The latter are classified in three groups - lip-readers, sign language users and those with hearing aids. The advice given varies for the different groups although the different methods of communication are often combined (e.g. sign language plus lip-reading, hearing-aids plus lip-reading). Treatment of hearing-impaired children in the dental clinic must be personalised. Each child is different, depending on the education received, the communication skills possessed, family factors (degree of parental protection, etc.), the existence of associated problems (learning difficulties), degree of loss of hearing, age, etc.

  14. "All Methods--and Wedded to None": The Deaf Education Methods Debate and Progressive Educational Reform in Toronto, Canada, 1922-1945

    ERIC Educational Resources Information Center

    Ellis, Jason A.

    2014-01-01

    This article is about the deaf education methods debate in the public schools of Toronto, Canada. The author demonstrates how pure oralism (lip-reading and speech instruction to the complete exclusion of sign language) and day school classes for deaf schoolchildren were introduced as a progressive school reform in 1922. Plans for further oralist…

  15. "All Methods--and Wedded to None": The Deaf Education Methods Debate and Progressive Educational Reform in Toronto, Canada, 1922-1945

    ERIC Educational Resources Information Center

    Ellis, Jason A.

    2014-01-01

    This article is about the deaf education methods debate in the public schools of Toronto, Canada. The author demonstrates how pure oralism (lip-reading and speech instruction to the complete exclusion of sign language) and day school classes for deaf schoolchildren were introduced as a progressive school reform in 1922. Plans for further oralist…

  16. Speech imagery recalibrates speech-perception boundaries.

    PubMed

    Scott, Mark

    2016-07-01

    The perceptual boundaries between speech sounds are malleable and can shift after repeated exposure to contextual information. This shift is known as recalibration. To date, the known inducers of recalibration are lexical (including phonotactic) information, lip-read information and reading. The experiments reported here are a proof-of-effect demonstration that speech imagery can also induce recalibration.

  17. Visual Cues and Listening Effort: Individual Variability

    ERIC Educational Resources Information Center

    Picou, Erin M.; Ricketts, Todd A; Hornsby, Benjamin W. Y.

    2011-01-01

    Purpose: To investigate the effect of visual cues on listening effort as well as whether predictive variables such as working memory capacity (WMC) and lipreading ability affect the magnitude of listening effort. Method: Twenty participants with normal hearing were tested using a paired-associates recall task in 2 conditions (quiet and noise) and…

  18. A LINGUISTIC APPROACH TO THE EDUCATION OF AURALLY-HANDICAPPED CHILDREN.

    ERIC Educational Resources Information Center

    LOWELL, EDGAR L.; WOODWARD, MARY F.

    IT IS HELD THAT ALTHOUGH PRESENT TUTORING TECHNIQUES IN LIPREADING FREQUENTLY ACHIEVE REMARKABLE RESULTS, THEY MIGHT BE BETTER UNDERSTOOD AND MADE MORE EFFECTIVE BY THE SYSTEMATIC APPLICATION OF LINGUISTIC CONCEPTS AND METHODS TO THE PROBLEMS INVOLVED. THE INVESTIGATION HAS BEEN CONCENTRATED ON A STUDY OF THE VISUAL PERCEPTIBILITY OF ENGLISH…

  19. Visual Cues and Listening Effort: Individual Variability

    ERIC Educational Resources Information Center

    Picou, Erin M.; Ricketts, Todd A; Hornsby, Benjamin W. Y.

    2011-01-01

    Purpose: To investigate the effect of visual cues on listening effort as well as whether predictive variables such as working memory capacity (WMC) and lipreading ability affect the magnitude of listening effort. Method: Twenty participants with normal hearing were tested using a paired-associates recall task in 2 conditions (quiet and noise) and…

  20. A SYNTHETIC APPROACH TO LIP READING, MATERIALS FOR THE CHILD OF GRADE SCHOOL AGE.

    ERIC Educational Resources Information Center

    HASPIEL, GEORGE S.

    LESSONS DESIGNED TO ASSIST TEACHERS AND PARENTS IN TEACHING SYNTHETIC LIPREADING TO HARD-OF-HEARING ELEMENTARY AGE CHILDREN ARE GROUPED INTO PRIMARY AND INTERMEDIATE LEVELS. EACH LESSON PLAN IS BASED ON THE PREVIOUS LESSON AND INCLUDES SEVERAL RELATED SPECIFIC ACTIVITIES. INSTRUCTIONS ABOUT MATERIALS, PROCEDURE, AND THE EXACT SENTENCES TO BE USED…

  1. The What? When? and How? of Teaching Language to Deaf Children - Preschool and Primary Grades.

    ERIC Educational Resources Information Center

    Hogan, Sister James Lorene

    Three levels of work in language development for preschool and primary age deaf children are presented, along with suggested daily schedules and yearly programs. Skills covered are speech, lipreading, auditory training, and language. Instructions are given for teaching activities in the areas of the various parts of speech and types of sentences.…

  2. [Cochlear implants].

    PubMed

    Lehnhardt, E; Battmer, R D; Nakahodo, K; Laszig, R

    1986-07-01

    Since the middle of 1984, the HNO-Klinik der Medizinischen Hochschule Hannover has provided deaf adults with a 22-channel cochlear implant (CI) device of Clark-NUCLEUS. The digital working system consists of an implantable stimulator/receiver and an externally worn speech processor. Energy and signals are transmitted transcutaneously via a transmitter coil. During the prevailing 26 operations (April 1986) the electrode array could be inserted at least 17 mm into the cochlea. The threshold and comfort levels of all patients were adjusted very quickly; the dynamic range usually grows during the first postoperative weeks. The individual rehabilitation results vary greatly, but all patients show a significant increase of vowel and consonant comprehension while using the speech processor and an improvement of words understood per minute in speech tracking from lip-reading alone to lip-reading with speech processor. Four months after surgery seven of 17 patients (group I) are able to understand on average 42.7 words per minute by speech tracking without lip-reading. Six patients (group II) recognise 69.2% of vowels and 42.5% of consonants by speech processor alone. Four patients (group III) can correctly repeat only vowels (52.3%) without lip-reading, but using the speech processor together with lip reading they have an improvement in consonant understanding of 37.9% and under freefield conditions they are able to understand up to 17.8% numbers of the Freiburg speech test.

  3. TEACHING DEAF CHILDREN TO TALK.

    ERIC Educational Resources Information Center

    EWING, ALEXANDER; EWING, ETHEL C.

    DESIGNED AS A TEXT FOR AUDIOLOGISTS AND TEACHERS OF HEARING IMPAIRED CHILDREN, THIS BOOK PRESENTS BASIC INFORMATION ABOUT SPOKEN LANGUAGE, HEARING, AND LIPREADING. METHODS AND RESULTS OF EVALUATING SPOKEN LANGUAGE OF AURALLY HANDICAPPED CHILDREN WITHOUT USING READING OR WRITING ARE REPORTED. VARIOUS TYPES OF INDIVIDUAL AND GROUP HEARING AIDS ARE…

  4. "The Business of Life": Educating Catholic Deaf Children in Late Nineteenth-Century England

    ERIC Educational Resources Information Center

    Mangion, Carmen M.

    2012-01-01

    Much of the debates in late nineteenth-century Britain regarding the education of deaf children revolved around communication. For many Victorians, sign language was unacceptable; many proponents of oralism attempted to "normalise" the hearing impaired by replacing deaf methods of communication with spoken language and lipreading. While…

  5. Perception of the Auditory-Visual Illusion in Speech Perception by Children with Phonological Disorders

    ERIC Educational Resources Information Center

    Dodd, Barbara; McIntosh, Beth; Erdener, Dogu; Burnham, Denis

    2008-01-01

    An example of the auditory-visual illusion in speech perception, first described by McGurk and MacDonald, is the perception of [ta] when listeners hear [pa] in synchrony with the lip movements for [ka]. One account of the illusion is that lip-read and heard speech are combined in an articulatory code since people who mispronounce words respond…

  6. A FEASIBILITY STUDY TO INVESTIGATE THE INSTRUMENTATION, ESTABLISHMENT, AND OPERATION OF A LEARNING LABORATORY FOR HARD-OF-HEARING CHILDREN, FINAL REPORT.

    ERIC Educational Resources Information Center

    STEPP, ROBERT E.

    TEN CHILDREN AGED 5-8 WERE SELECTED TO TEST A SELF-INSTRUCTIONAL, SELF-OPERATING SYSTEM TO DEVELOP LIPREADING SKILLS. THEIR HEARING DEFICIENCY RANGED FROM HARD OF HEARING TO PROFOUNDLY DEAF. THE SYSTEM CONSISTED OF THREE STUDY CARRELS, AN 8-MM CARTRIDGE-LOADING SOUND MOTION PICTURE PROJECTOR, AND AN OBSERVATION BOOTH UTILIZING A ONE-WAY MIRROR.…

  7. Effects of English Cued Speech on Speech Perception, Phonological Awareness and Literacy: A Case Study of a 9-Year-Old Deaf Boy Using a Cochlear Implant

    ERIC Educational Resources Information Center

    Rees, Rachel; Bladel, Judith

    2013-01-01

    Many studies have shown that French Cued Speech (CS) can enhance lipreading and the development of phonological awareness and literacy in deaf children but, as yet, there is little evidence that these findings can be generalized to English CS. This study investigated the possible effects of English CS on the speech perception, phonological…

  8. "The Business of Life": Educating Catholic Deaf Children in Late Nineteenth-Century England

    ERIC Educational Resources Information Center

    Mangion, Carmen M.

    2012-01-01

    Much of the debates in late nineteenth-century Britain regarding the education of deaf children revolved around communication. For many Victorians, sign language was unacceptable; many proponents of oralism attempted to "normalise" the hearing impaired by replacing deaf methods of communication with spoken language and lipreading. While…

  9. Effects of English Cued Speech on Speech Perception, Phonological Awareness and Literacy: A Case Study of a 9-Year-Old Deaf Boy Using a Cochlear Implant

    ERIC Educational Resources Information Center

    Rees, Rachel; Bladel, Judith

    2013-01-01

    Many studies have shown that French Cued Speech (CS) can enhance lipreading and the development of phonological awareness and literacy in deaf children but, as yet, there is little evidence that these findings can be generalized to English CS. This study investigated the possible effects of English CS on the speech perception, phonological…

  10. Breaking the Sound Barrier.

    ERIC Educational Resources Information Center

    Garmon, Linda

    1981-01-01

    Reviews various methods of communication for hearing-impaired individuals, including American Sign Language (ASL) and a computer system which analyzes speech and flashes appropriate symbols onto a wearer's eyeglass lenses to aid in lipreading. Illustrates how an ASL sign can be changed to create a new word. (Author/JN)

  11. TEACHING DEAF CHILDREN TO TALK.

    ERIC Educational Resources Information Center

    EWING, ALEXANDER; EWING, ETHEL C.

    DESIGNED AS A TEXT FOR AUDIOLOGISTS AND TEACHERS OF HEARING IMPAIRED CHILDREN, THIS BOOK PRESENTS BASIC INFORMATION ABOUT SPOKEN LANGUAGE, HEARING, AND LIPREADING. METHODS AND RESULTS OF EVALUATING SPOKEN LANGUAGE OF AURALLY HANDICAPPED CHILDREN WITHOUT USING READING OR WRITING ARE REPORTED. VARIOUS TYPES OF INDIVIDUAL AND GROUP HEARING AIDS ARE…

  12. Tones for Profoundly Deaf Tone-Language Speakers.

    ERIC Educational Resources Information Center

    Ching, Teresa

    A study assessed the practical use of the simplified speech pattern approach to teaching lipreading in a tone language by comparing performance using an acoustic hearing-aid and a Sivo-aid in a tone labelling task. After initial assessment, subjects were given training to enhance perception of lexically contrastive tones, then post-tested. The…

  13. Deafness and Interpreting.

    ERIC Educational Resources Information Center

    New Jersey State Dept. of Labor, Trenton. Div. of the Deaf.

    This paper explains how the hearing loss of deaf persons affects communication, describes methods deaf individuals use to communicate, and addresses the role of interpreters in the communication process. The volume covers: communication methods such as speechreading or lipreading, written notes, gestures, or sign language (American Sign Language,…

  14. Breaking the Sound Barrier.

    ERIC Educational Resources Information Center

    Garmon, Linda

    1981-01-01

    Reviews various methods of communication for hearing-impaired individuals, including American Sign Language (ASL) and a computer system which analyzes speech and flashes appropriate symbols onto a wearer's eyeglass lenses to aid in lipreading. Illustrates how an ASL sign can be changed to create a new word. (Author/JN)

  15. Audio-visual speech in noise perception in dyslexia.

    PubMed

    van Laarhoven, Thijs; Keetels, Mirjam; Schakel, Lemmy; Vroomen, Jean

    2016-12-18

    Individuals with developmental dyslexia (DD) may experience, besides reading problems, other speech-related processing deficits. Here, we examined the influence of visual articulatory information (lip-read speech) at various levels of background noise on auditory word recognition in children and adults with DD. We found that children with a documented history of DD have deficits in their ability to gain benefit from lip-read information that disambiguates noise-masked speech. We show with another group of adult individuals with DD that these deficits persist into adulthood. These deficits could not be attributed to impairments in unisensory auditory word recognition. Rather, the results indicate a specific deficit in audio-visual speech processing and suggest that impaired multisensory integration might be an important aspect of DD.

  16. Bimodal bilingualism as multisensory training?: Evidence for improved audiovisual speech perception after sign language exposure.

    PubMed

    Williams, Joshua T; Darcy, Isabelle; Newman, Sharlene D

    2016-02-15

    The aim of the present study was to characterize effects of learning a sign language on the processing of a spoken language. Specifically, audiovisual phoneme comprehension was assessed before and after 13 weeks of sign language exposure. L2 ASL learners performed this task in the fMRI scanner. Results indicated that L2 American Sign Language (ASL) learners' behavioral classification of the speech sounds improved with time compared to hearing nonsigners. Results indicated increased activation in the supramarginal gyrus (SMG) after sign language exposure, which suggests concomitant increased phonological processing of speech. A multiple regression analysis indicated that learner's rating on co-sign speech use and lipreading ability was correlated with SMG activation. This pattern of results indicates that the increased use of mouthing and possibly lipreading during sign language acquisition may concurrently improve audiovisual speech processing in budding hearing bimodal bilinguals.

  17. Visual speech influences speech perception immediately but not automatically.

    PubMed

    Mitterer, Holger; Reinisch, Eva

    2017-02-01

    Two experiments examined the time course of the use of auditory and visual speech cues to spoken word recognition using an eye-tracking paradigm. Results of the first experiment showed that the use of visual speech cues from lipreading is reduced if concurrently presented pictures require a division of attentional resources. This reduction was evident even when listeners' eye gaze was on the speaker rather than the (static) pictures. Experiment 2 used a deictic hand gesture to foster attention to the speaker. At the same time, the visual processing load was reduced by keeping the visual display constant over a fixed number of successive trials. Under these conditions, the visual speech cues from lipreading were used. Moreover, the eye-tracking data indicated that visual information was used immediately and even earlier than auditory information. In combination, these data indicate that visual speech cues are not used automatically, but if they are used, they are used immediately.

  18. Electrophysiological evidence for speech-specific audiovisual integration.

    PubMed

    Baart, Martijn; Stekelenburg, Jeroen J; Vroomen, Jean

    2014-01-01

    Lip-read speech is integrated with heard speech at various neural levels. Here, we investigated the extent to which lip-read induced modulations of the auditory N1 and P2 (measured with EEG) are indicative of speech-specific audiovisual integration, and we explored to what extent the ERPs were modulated by phonetic audiovisual congruency. In order to disentangle speech-specific (phonetic) integration from non-speech integration, we used Sine-Wave Speech (SWS) that was perceived as speech by half of the participants (they were in speech-mode), while the other half was in non-speech mode. Results showed that the N1 obtained with audiovisual stimuli peaked earlier than the N1 evoked by auditory-only stimuli. This lip-read induced speeding up of the N1 occurred for listeners in speech and non-speech mode. In contrast, if listeners were in speech-mode, lip-read speech also modulated the auditory P2, but not if listeners were in non-speech mode, thus revealing speech-specific audiovisual binding. Comparing ERPs for phonetically congruent audiovisual stimuli with ERPs for incongruent stimuli revealed an effect of phonetic stimulus congruency that started at ~200 ms after (in)congruence became apparent. Critically, akin to the P2 suppression, congruency effects were only observed if listeners were in speech mode, and not if they were in non-speech mode. Using identical stimuli, we thus confirm that audiovisual binding involves (partially) different neural mechanisms for sound processing in speech and non-speech mode. © 2013 Published by Elsevier Ltd.

  19. Two cortical mechanisms support the integration of visual and auditory speech: a hypothesis and preliminary data.

    PubMed

    Okada, Kayoko; Hickok, Gregory

    2009-03-20

    Visual speech (lip-reading) influences the perception of heard speech. The literature suggests at least two possible mechanisms for this influence: "direct" sensory-sensory interaction, whereby sensory signals from auditory and visual modalities are integrated directly, likely in the superior temporal sulcus, and "indirect" sensory-motor interaction, whereby visual speech is first mapped onto motor-speech representations in the frontal lobe, which in turn influences sensory perception via sensory-motor integration networks. We hypothesize that both mechanisms exist, and further that previous demonstrations of lip-reading functional activations in Broca's region and the posterior planum temporale reflect the sensory-motor mechanism. We tested one prediction of this hypothesis using fMRI. We assessed whether viewing visual speech (contrasted with facial gestures) activates the same network as a speech sensory-motor integration task (listen to and then silently rehearse speech). Both tasks activated locations within Broca's area, dorsal premotor cortex, and the posterior planum temporal (Spt), and focal regions of the STS, all of which have previously been implicated in sensory-motor integration for speech. This finding is consistent with the view that visual speech influences heard speech via sensory-motor networks. Lip-reading also activated a much wider network in the superior temporal lobe than the sensory-motor task, possibly reflecting a more direct cross-sensory integration network.

  20. Normal-hearing and hearing-impaired subjects' ability to just follow conversation in competing speech, reversed speech, and noise backgrounds.

    PubMed

    Hygge, S; Rönnberg, J; Larsby, B; Arlinger, S

    1992-02-01

    The performance on a conversation-following task by 24 hearing-impaired persons was compared with that of 24 matched controls with normal hearing in the presence of three background noises: (a) speech-spectrum random noise, (b) a male voice, and (c) the male voice played in reverse. The subjects' task was to readjust the sound level of a female voice (signal), every time the signal voice was attenuated, to the subjective level at which it was just possible to understand what was being said. To assess the benefit of lipreading, half of the material was presented audiovisually and half auditorily only. It was predicted that background speech would have a greater masking effect than reversed speech, which would in turn have a lesser masking effect than random noise. It was predicted that hearing-impaired subjects would perform more poorly than the normal-hearing controls in a background of speech. The influence of lipreading was expected to be constant across groups and conditions. The results showed that the hearing-impaired subjects were equally affected by the three background noises and that normal-hearing persons were less affected by the background speech than by noise. The performance of the normal-hearing persons was superior to that of the hearing-impaired subjects. The prediction about lipreading was confirmed. The results were explained in terms of the reduced temporal resolution by the hearing-impaired subjects.

  1. How hearing aids, background noise, and visual cues influence objective listening effort.

    PubMed

    Picou, Erin M; Ricketts, Todd A; Hornsby, Benjamin W Y

    2013-09-01

    The purpose of this article was to evaluate factors that influence the listening effort experienced when processing speech for people with hearing loss. Specifically, the change in listening effort resulting from introducing hearing aids, visual cues, and background noise was evaluated. An additional exploratory aim was to investigate the possible relationships between the magnitude of listening effort change and individual listeners' working memory capacity, verbal processing speed, or lipreading skill. Twenty-seven participants with bilateral sensorineural hearing loss were fitted with linear behind-the-ear hearing aids and tested using a dual-task paradigm designed to evaluate listening effort. The primary task was monosyllable word recognition and the secondary task was a visual reaction time task. The test conditions varied by hearing aids (unaided, aided), visual cues (auditory-only, auditory-visual), and background noise (present, absent). For all participants, the signal to noise ratio was set individually so that speech recognition performance in noise was approximately 60% in both the auditory-only and auditory-visual conditions. In addition to measures of listening effort, working memory capacity, verbal processing speed, and lipreading ability were measured using the Automated Operational Span Task, a Lexical Decision Task, and the Revised Shortened Utley Lipreading Test, respectively. In general, the effects measured using the objective measure of listening effort were small (~10 msec). Results indicated that background noise increased listening effort, and hearing aids reduced listening effort, while visual cues did not influence listening effort. With regard to the individual variables, verbal processing speed was negatively correlated with hearing aid benefit for listening effort; faster processors were less likely to derive benefit. Working memory capacity, verbal processing speed, and lipreading ability were related to benefit from visual cues. No

  2. Impact of visual cues on directional benefit and preference: Part I--laboratory tests.

    PubMed

    Wu, Yu-Hsiang; Bentler, Ruth A

    2010-02-01

    The purpose of the laboratory tests of the current investigation was to examine how visual cues impact directional (DIR) benefit and preference for the DIR microphone hearing aid (re: the omnidirectional [OMNI] microphone). Specifically, three hypotheses were examined: (1) the presence of visual cues would improve OMNI-aided performance to ceiling levels and therefore reduce DIR benefit and preference, (2) DIR benefit measured in the audiovisual (AV) condition could not be predicted by that measured using auditory-only (AO) testing, and (3) with visual cues, listeners with greater lipreading skills would perceive less DIR benefit than did listeners with lesser lipreading skills. Twenty-four adults with sensorineural hearing loss were recruited. Their speech recognition performances were measured in two hearing aid microphone modes (DIR and OMNI), at various signal-to-noise ratios (SNR, -10 to +10 dB in 4-dB steps) and under two presentation conditions (AV and AO) by using the AV version of the Connected Speech Test. Microphone preference (DIR versus OMNI) was also assessed with and without visual cues at each of the SNRs by using the same Connected Speech Test sentences. Lipreading skills were measured using the Utley test. The speech recognition data revealed that the participants obtained significantly less DIR benefit in the AV condition because their AV performances were at the ceiling level. Consistent with this, the likelihood of preferring DIR processing was significantly reduced when visual cues were available to the listeners. Further, DIR benefit measured in the AV condition was not correlated with that measured in the AO condition while being significantly and negatively correlated with lipreading skill. These results suggest that AO laboratory testing overestimates the DIR benefit and preference for DIR processing that hearing aid users may have in most face-to-face conversations in typical SNR, real-world environments. Additionally, because the DIR

  3. Visual abilities are important for auditory-only speech recognition: evidence from autism spectrum disorder.

    PubMed

    Schelinski, Stefanie; Riedel, Philipp; von Kriegstein, Katharina

    2014-12-01

    In auditory-only conditions, for example when we listen to someone on the phone, it is essential to fast and accurately recognize what is said (speech recognition). Previous studies have shown that speech recognition performance in auditory-only conditions is better if the speaker is known not only by voice, but also by face. Here, we tested the hypothesis that such an improvement in auditory-only speech recognition depends on the ability to lip-read. To test this we recruited a group of adults with autism spectrum disorder (ASD), a condition associated with difficulties in lip-reading, and typically developed controls. All participants were trained to identify six speakers by name and voice. Three speakers were learned by a video showing their face and three others were learned in a matched control condition without face. After training, participants performed an auditory-only speech recognition test that consisted of sentences spoken by the trained speakers. As a control condition, the test also included speaker identity recognition on the same auditory material. The results showed that, in the control group, performance in speech recognition was improved for speakers known by face in comparison to speakers learned in the matched control condition without face. The ASD group lacked such a performance benefit. For the ASD group auditory-only speech recognition was even worse for speakers known by face compared to speakers not known by face. In speaker identity recognition, the ASD group performed worse than the control group independent of whether the speakers were learned with or without face. Two additional visual experiments showed that the ASD group performed worse in lip-reading whereas face identity recognition was within the normal range. The findings support the view that auditory-only communication involves specific visual mechanisms. Further, they indicate that in ASD, speaker-specific dynamic visual information is not available to optimize auditory

  4. Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training

    PubMed Central

    Eberhardt, Silvio P.; Auer Jr., Edward T.; Bernstein, Lynne E.

    2014-01-01

    In a series of studies we have been investigating how multisensory training affects unisensory perceptual learning with speech stimuli. Previously, we reported that audiovisual (AV) training with speech stimuli can promote auditory-only (AO) perceptual learning in normal-hearing adults but can impede learning in congenitally deaf adults with late-acquired cochlear implants. Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training. In Experiment 1, visual-only (VO) training on paired associations between CVCVC nonsense word videos and nonsense pictures demonstrated that VO words could be learned to a high level of accuracy even by poor lipreaders. In Experiment 2, visual-auditory (VA) training in the same paradigm but with the addition of synchronous vocoded acoustic speech impeded VO learning of the stimuli in the paired-associates paradigm. In Experiment 3, the vocoded AO stimuli were shown to be less informative than the VO speech. Experiment 4 combined vibrotactile speech stimuli with the visual stimuli during training. Vibrotactile stimuli were shown to promote visual perceptual learning. In Experiment 5, no-training controls were used to show that training with visual speech carried over to consonant identification of untrained CVCVC stimuli but not to lipreading words in sentences. Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training. Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee’s primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee’s lower rank perceptual pathway will promote learning by a higher-rank pathway. The mechanisms supporting these principles are discussed in light of multisensory reverse hierarchy theory (RHT). PMID:25400566

  5. Parametric Representation of the Speaker's Lips for Multimodal Sign Language and Speech Recognition

    NASA Astrophysics Data System (ADS)

    Ryumin, D.; Karpov, A. A.

    2017-05-01

    In this article, we propose a new method for parametric representation of human's lips region. The functional diagram of the method is described and implementation details with the explanation of its key stages and features are given. The results of automatic detection of the regions of interest are illustrated. A speed of the method work using several computers with different performances is reported. This universal method allows applying parametrical representation of the speaker's lipsfor the tasks of biometrics, computer vision, machine learning, and automatic recognition of face, elements of sign languages, and audio-visual speech, including lip-reading.

  6. Phonetic recalibration does not depend on working memory

    PubMed Central

    Baart, Martijn

    2010-01-01

    Listeners use lipread information to adjust the phonetic boundary between two speech categories (phonetic recalibration, Bertelson et al. 2003). Here, we examined phonetic recalibration while listeners were engaged in a visuospatial or verbal memory working memory task under different memory load conditions. Phonetic recalibration was—like selective speech adaptation—not affected by a concurrent verbal or visuospatial memory task. This result indicates that phonetic recalibration is a low-level process not critically depending on processes used in verbal- or visuospatial working memory. PMID:20437168

  7. Phi-square Lexical Competition Database (Phi-Lex): an online tool for quantifying auditory and visual lexical competition.

    PubMed

    Strand, Julia F

    2014-03-01

    A widely agreed-upon feature of spoken word recognition is that multiple lexical candidates in memory are simultaneously activated in parallel when a listener hears a word, and that those candidates compete for recognition (Luce, Goldinger, Auer, & Vitevitch, Perception 62:615-625, 2000; Luce & Pisoni, Ear and Hearing 19:1-36, 1998; McClelland & Elman, Cognitive Psychology 18:1-86, 1986). Because the presence of those competitors influences word recognition, much research has sought to quantify the processes of lexical competition. Metrics that quantify lexical competition continuously are more effective predictors of auditory and visual (lipread) spoken word recognition than are the categorical metrics traditionally used (Feld & Sommers, Speech Communication 53:220-228, 2011; Strand & Sommers, Journal of the Acoustical Society of America 130:1663-1672, 2011). A limitation of the continuous metrics is that they are somewhat computationally cumbersome and require access to existing speech databases. This article describes the Phi-square Lexical Competition Database (Phi-Lex): an online, searchable database that provides access to multiple metrics of auditory and visual (lipread) lexical competition for English words, available at www.juliastrand.com/phi-lex .

  8. A hand held two-channel vibro-tactile speech communication aid for the deaf: characteristics and results.

    PubMed

    Spens, K E; Huss, C; Dahlqvist, M; Agelfors, E

    1997-01-01

    The OSCAR-project (Optimal Speech Communication Assistance for Residual Abilities) within the TIDE frame is aiming at processing the speech signal in such a way so that it can be more effectively aurally perceived than with an ordinary hearing aid in case of a very small residual hearing. In case of total deafness the code will be tactile and in some cases a multi modal approach will be tried. One of the sub-results of the OSCAR-project is a hand held vibro-tactile speech communication aid for profoundly hard of hearing and deaf persons. The main idea of the aid is that it should not be a general tactile hearing aid, but rather give optimum tactile support to lip-reading in a close communication situation. It should be used when speech communication is necessary and other methods of communication i.e. sign language, lip-reading, reading and writing etc. does not work. The aid conveys two types of information via its two vibro-tactile transducers; the syllabic rhythm and frication. This paper describes some characteristics of the aid and some results.

  9. [Hemianopsia and agnosia].

    PubMed

    Landis, T; Regard, M

    1988-05-01

    Hemianopsia and agnosia, the coexistence of a disorder of perception and a disorder of recognition, is exemplified by two patients chosen for their clinical similarity. These two patients each had a stroke, one of the left and the other of the right posterior cerebral artery, resulting right and left hemianopsia respectively. Primary visual functions such as visual acuity and contrast sensitivity were intact. The differences were only apparent as regards the severity of the agnostic disorders. The patient with the lesion on the left failed to recognize stylized drawings, was alexic, could not lipread and was color anomic, but could easily identify handwriting, faces, and voices. The patient with the right posterior lesion could not recognize handwriting, was prosopagnosic and topographagnosic, but had no difficulty in reading, lipreading, or in recognizing stylized drawings. The apparent double dissociation of function demonstrates the presence of two different visual processing mechanisms which are very probably peculiar to the visual system of the left hemisphere in the one case and to that of the right hemisphere in the other. The authors' findings suggest that the two cerebral hemispheres differ fundamentally in the way they process visual information.

  10. Video analysis using spatiotemporal descriptor and kernel extreme learning machine for lip reading

    NASA Astrophysics Data System (ADS)

    Lu, Longbin; Zhang, Xinman; Xu, Xuebin; Shang, Dongpeng

    2015-09-01

    Lip-reading techniques have shown bright prospects for speech recognition under noisy environments and for hearing-impaired listeners. We aim to solve two important issues regarding lip reading: (1) how to extract discriminative lip motion features and (2) how to establish a classifier that can provide promising recognition accuracy for lip reading. For the first issue, a projection local spatiotemporal descriptor, which considers the lip appearance and motion information at the same time, is utilized to provide an efficient representation of a video sequence. For the second issue, a kernel extreme learning machine (KELM) based on the single-hidden-layer feedforward neural network is presented to distinguish all kinds of utterances. In general, this method has fast learning speed and great robustness to nonlinear data. Furthermore, quantum-behaved particle swarm optimization with binary encoding is introduced to select the appropriate feature subset and parameters for KELM training. Experiments conducted on the AVLetters and OuluVS databases show that the proposed lip-reading method achieves a superior recognition accuracy compared with two previous methods.

  11. The role of visual speech cues in reducing energetic and informational masking

    NASA Astrophysics Data System (ADS)

    Helfer, Karen S.; Freyman, Richard L.

    2005-02-01

    Two experiments compared the effect of supplying visual speech information (e.g., lipreading cues) on the ability to hear one female talker's voice in the presence of steady-state noise or a masking complex consisting of two other female voices. In the first experiment intelligibility of sentences was measured in the presence of the two types of maskers with and without perceived spatial separation of target and masker. The second study tested detection of sentences in the same experimental conditions. Results showed that visual cues provided more benefit for both recognition and detection of speech when the masker consisted of other voices (versus steady-state noise). Moreover, visual cues provided greater benefit when the target speech and masker were spatially coincident versus when they appeared to arise from different spatial locations. The data obtained here are consistent with the hypothesis that lipreading cues help to segregate a target voice from competing voices, in addition to the established benefit of supplementing masked phonetic information. .

  12. Effects of aging on audio-visual speech integration.

    PubMed

    Huyse, Aurélie; Leybaert, Jacqueline; Berthommier, Frédéric

    2014-10-01

    This study investigated the impact of aging on audio-visual speech integration. A syllable identification task was presented in auditory-only, visual-only, and audio-visual congruent and incongruent conditions. Visual cues were either degraded or unmodified. Stimuli were embedded in stationary noise alternating with modulated noise. Fifteen young adults and 15 older adults participated in this study. Results showed that older adults had preserved lipreading abilities when the visual input was clear but not when it was degraded. The impact of aging on audio-visual integration also depended on the quality of the visual cues. In the visual clear condition, the audio-visual gain was similar in both groups and analyses in the framework of the fuzzy-logical model of perception confirmed that older adults did not differ from younger adults in their audio-visual integration abilities. In the visual reduction condition, the audio-visual gain was reduced in the older group, but only when the noise was stationary, suggesting that older participants could compensate for the loss of lipreading abilities by using the auditory information available in the valleys of the noise. The fuzzy-logical model of perception confirmed the significant impact of aging on audio-visual integration by showing an increased weight of audition in the older group.

  13. Cochlear implants in the management of hearing loss in Neurofibromatosis Type 2.

    PubMed

    Harris, Frances; Tysome, James R; Donnelly, Neil; Durie-Gair, Juliette; Crundwell, Gemma; Tam, Yu Chuen; Knight, Richard D; Vanat, Zebunnisa H; Folland, Nicola; Axon, Patrick

    2017-05-01

    Review of cochlear implant (CI) outcomes in patients with Neurofibromatosis Type 2 (NF2), implanted in the presence of an ipsilateral vestibular schwannoma (VS). Hearing restoration was combined in some cases with a Bevacizumab regime. Retrospective review of 12 patients, managed over the period 2009-2016, at a tertiary referral multidisciplinary NF2 clinic. The patients are grouped by hearing outcomes to explore likely protective factors, and to generate a proposed decision-making tool for the selection of either CI or Auditory Brainstem Implant (ABI). Four of the 12 patients achieved speech discrimination without lip-reading. In these individuals there is reason to think that the mechanism of their hearing loss was cochlear dysfunction. A further four patients received benefit to lip-reading and awareness of environmental sound. For such patients their hearing loss may have been due to both cochlear and neural dysfunction. Two patients gained access to environmental sound only from their CI. Two patients derived no benefit from their CIs, which were subsequently explanted. Both these latter patients had had prior ipsilateral tumour surgery, one just before the CI insertion. Cochlear implantation can lead to open set speech discrimination in patients with NF2 in the presence of a stable VS. Use of promontory stimulation and intraoperative electrically evoked auditory brainstem response testing, along with case history, can inform the decision whether to implant an ABI or CI.

  14. Phonetic matching of auditory and visual speech develops during childhood: evidence from sine-wave speech.

    PubMed

    Baart, Martijn; Bortfeld, Heather; Vroomen, Jean

    2015-01-01

    The correspondence between auditory speech and lip-read information can be detected based on a combination of temporal and phonetic cross-modal cues. Here, we determined the point in developmental time at which children start to effectively use phonetic information to match a speech sound with one of two articulating faces. We presented 4- to 11-year-olds (N=77) with three-syllabic sine-wave speech replicas of two pseudo-words that were perceived as non-speech and asked them to match the sounds with the corresponding lip-read video. At first, children had no phonetic knowledge about the sounds, and matching was thus based on the temporal cues that are fully retained in sine-wave speech. Next, we trained all children to perceive the phonetic identity of the sine-wave speech and repeated the audiovisual (AV) matching task. Only at around 6.5 years of age did the benefit of having phonetic knowledge about the stimuli become apparent, thereby indicating that AV matching based on phonetic cues presumably develops more slowly than AV matching based on temporal cues. Copyright © 2014 Elsevier Inc. All rights reserved.

  15. New developments in speech pattern element hearing aids for the profoundly deaf.

    PubMed

    Faulkner, A; Walliker, J R; Howard, I S; Ball, V; Fourcin, A J

    1993-01-01

    Two new developments in speech pattern processing hearing aids will be described. The first development is the use of compound speech pattern coding. Speech information which is invisible to the lipreader was encoded in terms of three acoustic speech factors; the voice fundamental frequency pattern, coded as a sinusoid, the presence of aperiodic excitation, coded as a low-frequency noise, and the wide-band amplitude envelope, coded by amplitude modulation of the sinusoid and noise signals. Each element of the compound stimulus was individually matched in frequency and intensity to the listener's receptive range. Audio-visual speech receptive assessments in five profoundly hearing-impaired listeners were performed to examine the contributions of adding voiceless and amplitude information to the voice fundamental frequency pattern, and to compare these codings to amplified speech. In both consonant recognition and connected discourse tracking (CDT), all five subjects showed an advantage from the addition of amplitude information to the fundamental frequency pattern. In consonant identification, all five subjects showed further improvements in performance when voiceless speech excitation was additionally encoded together with amplitude information, but this effect was not found in CDT. The addition of voiceless information to voice fundamental frequency information did not improve performance in the absence of amplitude information. Three of the subjects performed significantly better in at least one of the compound speech pattern conditions than with amplified speech, while the other two performed similarly with amplified speech and the best compound speech pattern condition. The three speech pattern elements encoded here may represent a near-optimal basis for an acoustic aid to lipreading for this group of listeners. The second development is the use of a trained multi-layer-perceptron (MLP) pattern classification algorithm as the basis for a robust real-time voice

  16. Speech Analysis Based On Image Information from Lip Movement

    NASA Astrophysics Data System (ADS)

    Talha, Kamil S.; Wan, Khairunizam; Za'ba, S. K.; Mohamad Razlan, Zuradzman; B, Shahriman A.

    2013-12-01

    Deaf and hard of hearing people often have problems being able to understand and lip read other people. Usually deaf and hard of hearing people feel left out of conversation and sometimes they are actually ignored by other people. There are a variety of ways hearing-impaired person can communicate and gain accsss to the information. Communication support includes both technical and human aids. Human aids include interpreters, lip-readers and note-takers. Interpreters translate the Sign Language and must therefore be qualified. In this paper, vision system is used to track movements of the lip. In the experiment, the proposed system succesfully can differentiate 11 type of phonemes and then classified it to the respective viseme group. By using the proposed system the hearing-impaired persons could practise pronaunciations by themselve without support from the instructor.

  17. Don't Listen With Your Mouth Full: The Role of Facial Motor Action in Visual Speech Perception.

    PubMed

    Turner, Angela C; McIntosh, Daniel N; Moody, Eric J

    2015-06-01

    Theories of speech perception agree that visual input enhances the understanding of speech but disagree on whether physically mimicking the speaker improves understanding. This study investigated whether facial motor mimicry facilitates visual speech perception by testing whether blocking facial motor action impairs speechreading performance. Thirty-five typically developing children (19 boys; 16 girls; M age = 7 years) completed the Revised Craig Lipreading Inventory under two conditions. While observing silent videos of 15 words being spoken, participants either held a tongue depressor horizontally with their teeth (blocking facial motor action) or squeezed a ball with one hand (allowing facial motor action). As hypothesized, blocking motor action resulted in fewer correctly understood words than that of the control task. The results suggest that facial mimicry or other methods of facial action support visual speech perception in children. Future studies on the impact of motor action on the typical and atypical development of speech perception are warranted.

  18. Cochlear implants for congenitally deaf adolescents: is open-set speech perception a realistic expectation?

    PubMed

    Sarant, J Z; Cowan, R S; Blamey, P J; Galvin, K L; Clark, G M

    1994-10-01

    The prognosis for benefit from use of cochlear implants in congenitally deaf adolescents, who have a long duration of profound deafness prior to implantation, has typically been low. Speech perception results for two congenitally deaf patients implanted as adolescents at the University of Melbourne/Royal Victorian Eye and Ear Hospital Clinic show that, after 12 months of experience, both patients had significant open-set speech discrimination scores without lipreading. These results suggest that although benefits may in general be low for congenitally deaf adolescents, individuals may attain significant benefits to speech perception after a short period of experience. Prospective patients from this group should therefore be considered on an individual basis with regard to prognosis for benefit from cochlear implantation.

  19. Phonetic recalibration of speech by text.

    PubMed

    Keetels, Mirjam; Schakel, Lemmy; Bonte, Milene; Vroomen, Jean

    2016-04-01

    Listeners adjust their phonetic categories to cope with variations in the speech signal (phonetic recalibration). Previous studies have shown that lipread speech (and word knowledge) can adjust the perception of ambiguous speech and can induce phonetic adjustments (Bertelson, Vroomen, & de Gelder in Psychological Science, 14(6), 592-597, 2003; Norris, McQueen, & Cutler in Cognitive Psychology, 47(2), 204-238, 2003). We examined whether orthographic information (text) also can induce phonetic recalibration. Experiment 1 showed that after exposure to ambiguous speech sounds halfway between /b/ and /d/ that were combined with text (b or d) participants were more likely to categorize auditory-only test sounds in accordance with the exposed letters. Experiment 2 replicated this effect with a very short exposure phase. These results show that listeners adjust their phonetic boundaries in accordance with disambiguating orthographic information and that these adjustments show a rapid build-up.

  20. Deafness

    PubMed Central

    Weston, T E T

    1965-01-01

    Dr T E T Weston describes his research into the effect of noise on hearing acuity and of deafness in the aged. He found that presbyacusis is associated with a multiplicity of factors, e.g. smoking, circulatory disturbance, urban domicile, heredity and occupational acoustic trauma. Miss W Galbraith describes the social implications of various degrees of deafness and the ways in which they can be overcome by such measures as lipreading, hearing aids and rehabilitation. Sir Terence Cawthorne discusses otosclerosis, nearly 1% of the population being affected by this type of deafness. He describes the modern operation of insertion of an artificial piston through the stapes and states that 90% of cases submitted to this operation will show immediate improvement, whilst 85% should still have retained this improvement at the end of two years. PMID:14341856

  1. Adaptation of neuromagnetic N1 responses to phonetic stimuli by visual speech in humans.

    PubMed

    Jääskeläinen, Iiro P; Ojanen, Ville; Ahveninen, Jyrki; Auranen, Toni; Levänen, Sari; Möttönen, Riikka; Tarnanen, Iina; Sams, Mikko

    2004-12-22

    The technique of 306-channel magnetoencephalogaphy (MEG) was used in eight healthy volunteers to test whether silent lip-reading modulates auditory-cortex processing of phonetic sounds. Auditory test stimuli (either Finnish vowel /ae/ or /ø/) were preceded by a 500 ms lag by either another auditory stimulus (/ae/, /ø/ or the second-formant midpoint between /ae/ and /ø/), or silent movie of a person articulating /ae/ or /ø/. Compared with N1 responses to auditory /ae/ and /ø/ when presented without a preceding stimulus, the amplitudes of left-hemisphere N1 responses to the test stimuli were significantly suppressed both when preceded by auditory and visual stimuli, this effect being significantly stronger with preceding auditory stimuli. This suggests that seeing articulatory gestures of a speaker influences auditory speech perception by modulating the responsiveness of auditory-cortex neurons.

  2. Non-invasive electrical stimulation of the ear canal as a communication aid in acquired total deafness.

    PubMed

    Bochenek, W; Chorzempa, A; Hazell, J W; Kiciak, J; Kukwa, A

    1989-11-01

    Some profoundly deafened patients, who cannot be helped by sound amplification, claim to perceive auditory sensations when alternating currents up to 100 Hz are passed through electrodes applied to the skin of the external ear canal. A portable speech processor has been developed which supplies this current. The signal is a balanced square wave, the frequency of which is proportional to the first formant frequency. The amplitude is proportional to the intensity of voiced sounds. In order to fit the narrow frequency and dynamic range of the electrical stimulus, the speech processor produces a downward frequency transposition and strong limitation of the dynamic range. The device has been tested for (1) discrimination of environmental sounds; (2) question/statement discrimination; (3) identification of vowels and consonants in vowel/consonant/vowel context; (4) lip-reading with and without the prosthesis.

  3. PERVALE-S: a new cognitive task to assess deaf people’s ability to perceive basic and social emotions

    PubMed Central

    Mestre, José M.; Larrán, Cristina; Herrero, Joaquín; Guil, Rocío; de la Torre, Gabriel G.

    2015-01-01

    A poorly understood aspect of deaf people (DP) is how their emotional information is processed. Verbal ability is key to improve emotional knowledge in people. Nevertheless, DP are unable to distinguish intonation, intensity, and the rhythm of language due to lack of hearing. Some DP have acquired both lip-reading abilities and sign language, but others have developed only sign language. PERVALE-S was developed to assess the ability of DP to perceive both social and basic emotions. PERVALE-S presents different sets of visual images of a real deaf person expressing both basic and social emotions, according to the normative standard of emotional expressions in Spanish Sign Language. Emotional expression stimuli were presented at two different levels of intensity (1: low; and 2: high) because DP do not distinguish an object in the same way as hearing people (HP) do. Then, participants had to click on the more suitable emotional expression. PERVALE-S contains video instructions (given by a sign language interpreter) to improve DP’s understanding about how to use the software. DP had to watch the videos before answering the items. To test PERVALE-S, a sample of 56 individuals was recruited (18 signers, 8 lip-readers, and 30 HP). Participants also performed a personality test (High School Personality Questionnaire adapted) and a fluid intelligence (Gf) measure (RAPM). Moreover, all deaf participants were rated by four teachers for the deaf. Results: there were no significant differences between deaf and HP in performance in PERVALE-S. Confusion matrices revealed that embarrassment, envy, and jealousy were worse perceived. Age was just related to social-emotional tasks (but not in basic emotional tasks). Emotional perception ability was related mainly to warmth and consciousness, but negatively related to tension. Meanwhile, Gf was related to only social-emotional tasks. There were no gender differences. PMID:26300828

  4. Cross-cultural adaptation and validation of the Nijmegen Cochlear Implant Questionnaire into Italian.

    PubMed

    Ottaviani, F; Iacona, E; Sykopetrites, V; Schindler, A; Mozzanica, F

    2016-08-01

    The NCIQ is a quantifiable self-assessment health-related quality of life instrument specific for cochlear implant users. The aim of this study was to culturally adapt the NCIQ into Italian (I-NCIQ). A prospective instrument validation study was conducted. Cross-cultural adaptation and validation were accomplished. Cronbach α was used to test internal consistency in 51 CI users and in a control group composed by 38 post-lingual deaf adult on a waiting list for a CI. ICC test was used for test-retest reliability analysis. Kruskal-Wallis test with Mann-Whitney post hoc were used to compare the I-NCIQ scores in CI users before and after the cochlear implantation and in control patients. I-NCIQ scores obtained in CI users were compared with the results of Italian version of disyllabic testing without lip-reading and without masking. Good internal consistency and good test-retest reliability were found. I-NCIQ scores obtained in the 51 CI users after implantation were consistently higher than those obtained before implantation and in the control group. Moreover, no differences were found in the results of I-NCIQ obtained in the group of 51 CI users before implantation and in the group of control patients on post hoc Mann-Whitney analysis. Positive correlations between I-NCIQ scores and the results of disyllabic testing without lip-reading and without masking were found. The I-NCIQ is a reliable, valid, self-administered questionnaire for the measurement of QOL in CI users; its application is recommended.

  5. PERVALE-S: a new cognitive task to assess deaf people's ability to perceive basic and social emotions.

    PubMed

    Mestre, José M; Larrán, Cristina; Herrero, Joaquín; Guil, Rocío; de la Torre, Gabriel G

    2015-01-01

    A poorly understood aspect of deaf people (DP) is how their emotional information is processed. Verbal ability is key to improve emotional knowledge in people. Nevertheless, DP are unable to distinguish intonation, intensity, and the rhythm of language due to lack of hearing. Some DP have acquired both lip-reading abilities and sign language, but others have developed only sign language. PERVALE-S was developed to assess the ability of DP to perceive both social and basic emotions. PERVALE-S presents different sets of visual images of a real deaf person expressing both basic and social emotions, according to the normative standard of emotional expressions in Spanish Sign Language. Emotional expression stimuli were presented at two different levels of intensity (1: low; and 2: high) because DP do not distinguish an object in the same way as hearing people (HP) do. Then, participants had to click on the more suitable emotional expression. PERVALE-S contains video instructions (given by a sign language interpreter) to improve DP's understanding about how to use the software. DP had to watch the videos before answering the items. To test PERVALE-S, a sample of 56 individuals was recruited (18 signers, 8 lip-readers, and 30 HP). Participants also performed a personality test (High School Personality Questionnaire adapted) and a fluid intelligence (Gf) measure (RAPM). Moreover, all deaf participants were rated by four teachers for the deaf. there were no significant differences between deaf and HP in performance in PERVALE-S. Confusion matrices revealed that embarrassment, envy, and jealousy were worse perceived. Age was just related to social-emotional tasks (but not in basic emotional tasks). Emotional perception ability was related mainly to warmth and consciousness, but negatively related to tension. Meanwhile, Gf was related to only social-emotional tasks. There were no gender differences.

  6. Speech perception with the Vienna extra-cochlear single-channel implant: a comparison of two approaches to speech coding.

    PubMed

    Rosen, S; Ball, V

    1986-02-01

    Although it is generally accepted that single-channel electrical stimulation can significantly improve a deafened patient's speech perceptual ability, there is still much controversy surrounding the choice of speech processing schemes. We have compared, in the same patients, two different approaches: (1) The speech pattern extraction technique of the EPI group, London (Fourcin et al., British Journal of Audiology, 1979,13,85-107) in which voice fundamental frequency is extracted and presented in an appropriate way, and (2) The analogue 'whole speech' approach of Hochmair and Hochmair-Desoyer (Annals of the New York Academy of Sciences, 1983, 405, 268-279) of Vienna, in which the microphone-sensed acoustic signal is frequency-equalized and amplitude-compressed before being presented to the electrode. With the 'whole-speech' coding scheme (which they used daily), all three patients showed an improvement in lipreading when they used the device. No patient was able to understand speech without lipreading. Reasonable ability to distinguish voicing contrasts and voice pitch contours was displayed. One patient was able to detect and make appropriate use of the presence of voiceless frication in certain situations. Little sensitivity to spectral features in natural speech was noted, although two patients could detect changes in the frequency of the first formant of synthesised vowels. Presentation of the fundamental frequency only generally led to improved perception of features associated with it (voicing and intonation). Only one patient consistently showed any advantage (and that not in all tests) of coding more than the fundamental alone.

  7. The neural correlates of cross-modal interaction in speech perception during a semantic decision task on sentences: a PET study.

    PubMed

    Kang, Eunjoo; Lee, Dong Soo; Kang, Hyejin; Hwang, Chan Ho; Oh, Seung-Ha; Kim, Chong-Sun; Chung, June-Key; Lee, Myung Chul

    2006-08-01

    Speech perception in face-to-face conversation involves processing of speech sounds (auditory) and speech-associated mouth/lip movements (visual) from a speaker. Using PET where no scanner noise was present, brain regions involved in speech cue processing were investigated with the normal hearing subjects with no previous lip-reading training (N = 17) carrying out a semantic plausibility decision on spoken sentences delivered in a movie file. Multimodality was ensured at the sensory level in all four conditions. Sensory-specific speech cue of one sensory modality, i.e., auditory speech (A condition) or mouth movement (V condition), was delivered with a control stimulus of the other modality whereas speech cues of both sensory modalities (AV condition) were delivered during bimodal condition. In comparison to the control condition, extensive activations in the superior temporal regions were observed bilaterally during the A condition but these activations were reduced in extent and left lateralized during the AV condition. Polymodal region such as left posterior superior temporal sulcus (pSTS) involved in cross-modal interaction/integration of audiovisual speech was found to be activated during the A and more so during the AV conditions but not during the V condition. Activations were observed in Broca's (BA 44), medial frontal (BA 8), and anterior ventrolateral prefrontal (BA 47) regions in the left during the V condition, where lip-reading performance was less successful. Results indicated that the speech-associated lip movements (visual speech cue) rendered suppression on the activity in the right auditory temporal regions. Overadditivity (AV > A + V) observed in the right postcentral region during the bimodal condition relative to the sum of unimodal speech conditions was also associated with reduced activity during the V condition. These findings suggested that visual speech cue could exert an inhibitory modulatory effect on the brain activities in the right

  8. The effect of audiovisual and binaural listening on the acceptable noise level (ANL): establishing an ANL conceptual model.

    PubMed

    Wu, Yu-Hsiang; Stangl, Elizabeth; Pang, Carol; Zhang, Xuyang

    2014-02-01

    Little is known regarding the acoustic features of a stimulus used by listeners to determine the acceptable noise level (ANL). Features suggested by previous research include speech intelligibility (noise is unacceptable when it degrades speech intelligibility to a certain degree; the intelligibility hypothesis) and loudness (noise is unacceptable when the speech-to-noise loudness ratio is poorer than a certain level; the loudness hypothesis). The purpose of the study was to investigate if speech intelligibility or loudness is the criterion feature that determines ANL. To achieve this, test conditions were chosen so that the intelligibility and loudness hypotheses would predict different results. In Experiment 1, the effect of audiovisual (AV) and binaural listening on ANL was investigated; in Experiment 2, the effect of interaural correlation (ρ) on ANL was examined. A single-blinded, repeated-measures design was used. Thirty-two and twenty-five younger adults with normal hearing participated in Experiments 1 and 2, respectively. In Experiment 1, both ANL and speech recognition performance were measured using the AV version of the Connected Speech Test (CST) in three conditions: AV-binaural, auditory only (AO)-binaural, and AO-monaural. Lipreading skill was assessed using the Utley lipreading test. In Experiment 2, ANL and speech recognition performance were measured using the Hearing in Noise Test (HINT) in three binaural conditions, wherein the interaural correlation of noise was varied: ρ = 1 (N(o)S(o) [a listening condition wherein both speech and noise signals are identical across two ears]), -1 (NπS(o) [a listening condition wherein speech signals are identical across two ears whereas the noise signals of two ears are 180 degrees out of phase]), and 0 (N(u)S(o) [a listening condition wherein speech signals are identical across two ears whereas noise signals are uncorrelated across ears]). The results were compared to the predictions made based on the

  9. Audibility and visual biasing in speech perception

    NASA Astrophysics Data System (ADS)

    Clement, Bart Richard

    Although speech perception has been considered a predominantly auditory phenomenon, large benefits from vision in degraded acoustic conditions suggest integration of audition and vision. More direct evidence of this comes from studies of audiovisual disparity that demonstrate vision can bias and even dominate perception (McGurk & MacDonald, 1976). It has been observed that hearing-impaired listeners demonstrate more visual biasing than normally hearing listeners (Walden et al., 1990). It is argued here that stimulus audibility must be equated across groups before true differences can be established. In the present investigation, effects of visual biasing on perception were examined as audibility was degraded for 12 young normally hearing listeners. Biasing was determined by quantifying the degree to which listener identification functions for a single synthetic auditory /ba-da-ga/ continuum changed across two conditions: (1)an auditory-only listening condition; and (2)an auditory-visual condition in which every item of the continuum was synchronized with visual articulations of the consonant-vowel (CV) tokens /ba/ and /ga/, as spoken by each of two talkers. Audibility was altered by presenting the conditions in quiet and in noise at each of three signal-to- noise (S/N) ratios. For the visual-/ba/ context, large effects of audibility were found. As audibility decreased, visual biasing increased. A large talker effect also was found, with one talker eliciting more biasing than the other. An independent lipreading measure demonstrated that this talker was more visually intelligible than the other. For the visual-/ga/ context, audibility and talker effects were less robust, possibly obscured by strong listener effects, which were characterized by marked differences in perceptual processing patterns among participants. Some demonstrated substantial biasing whereas others demonstrated little, indicating a strong reliance on audition even in severely degraded acoustic

  10. Degradation of labial information modifies audiovisual speech perception in cochlear-implanted children.

    PubMed

    Huyse, Aurélie; Berthommier, Frédéric; Leybaert, Jacqueline

    2013-01-01

    The aim of the present study was to examine audiovisual speech integration in cochlear-implanted children and in normally hearing children exposed to degraded auditory stimuli. Previous studies have shown that speech perception in cochlear-implanted users is biased toward the visual modality when audition and vision provide conflicting information. Our main question was whether an experimentally designed degradation of the visual speech cue would increase the importance of audition in the response pattern. The impact of auditory proficiency was also investigated. A group of 31 children with cochlear implants and a group of 31 normally hearing children matched for chronological age were recruited. All children with cochlear implants had profound congenital deafness and had used their implants for at least 2 years. Participants had to perform an /aCa/ consonant-identification task in which stimuli were presented randomly in three conditions: auditory only, visual only, and audiovisual (congruent and incongruent McGurk stimuli). In half of the experiment, the visual speech cue was normal; in the other half (visual reduction) a degraded visual signal was presented, aimed at preventing lipreading of good quality. The normally hearing children received a spectrally reduced speech signal (simulating the input delivered by the cochlear implant). First, performance in visual-only and in congruent audiovisual modalities were decreased, showing that the visual reduction technique used here was efficient at degrading lipreading. Second, in the incongruent audiovisual trials, visual reduction led to a major increase in the number of auditory based responses in both groups. Differences between proficient and nonproficient children were found in both groups, with nonproficient children's responses being more visual and less auditory than those of proficient children. Further analysis revealed that differences between visually clear and visually reduced conditions and between

  11. Automatic lip reading by using multimodal visual features

    NASA Astrophysics Data System (ADS)

    Takahashi, Shohei; Ohya, Jun

    2013-12-01

    Since long time ago, speech recognition has been researched, though it does not work well in noisy places such as in the car or in the train. In addition, people with hearing-impaired or difficulties in hearing cannot receive benefits from speech recognition. To recognize the speech automatically, visual information is also important. People understand speeches from not only audio information, but also visual information such as temporal changes in the lip shape. A vision based speech recognition method could work well in noisy places, and could be useful also for people with hearing disabilities. In this paper, we propose an automatic lip-reading method for recognizing the speech by using multimodal visual information without using any audio information such as speech recognition. First, the ASM (Active Shape Model) is used to track and detect the face and lip in a video sequence. Second, the shape, optical flow and spatial frequencies of the lip features are extracted from the lip detected by ASM. Next, the extracted multimodal features are ordered chronologically so that Support Vector Machine is performed in order to learn and classify the spoken words. Experiments for classifying several words show promising results of this proposed method.

  12. Electrically evoked hearing perception by functional neurostimulation of the central auditory system.

    PubMed

    Tatagiba, M; Gharabaghi, A

    2005-01-01

    Perceptional benefits and potential risks of electrical stimulation of the central auditory system are constantly changing due to ongoing developments and technical modifications. Therefore, we would like to introduce current treatment protocols and strategies that might have an impact on functional results of auditory brainstem implants (ABI) in profoundly deaf patients. Patients with bilateral tumours as a result of neurofibromatosis type 2 with complete dysfunction of the eighth cranial nerves are the most frequent candidates for auditory brainstem implants. Worldwide, about 300 patients have already received an ABI through a translabyrinthine or suboccipital approach supported by multimodality electrophysiological monitoring. Patient selection is based on disease course, clinical signs, audiological, radiological and psycho-social criteria. The ABI provides the patients with access to auditory information such as environmental sound awareness together with distinct hearing cues in speech. In addition, this device markedly improves speech reception in combination with lip-reading. Nonetheless, there is only limited open-set speech understanding. Results of hearing function are correlated with electrode design, number of activated electrodes, speech processing strategies, duration of pre-existing deafness and extent of brainstem deformation. Functional neurostimulation of the central auditory system by a brainstem implant is a safe and beneficial procedure, which may considerably improve the quality of life in patients suffering from deafness due to bilateral retrocochlear lesions. The auditory outcome may be improved by a new generation of microelectrodes capable of penetrating the surface of the brainstem to access more directly the auditory neurons.

  13. Speechreading development in deaf and hearing children: introducing a new Test of Child Speechreading (ToCS)

    PubMed Central

    Kyle, Fiona Elizabeth; Campbell, Ruth; Mohammed, Tara; Coleman, Mike; MacSweeney, Mairéad

    2016-01-01

    Purpose We describe the development of a new Test of Child Speechreading (ToCS) specifically designed for use with deaf and hearing children. Speechreading is a skill which is required for deaf children to access the language of the hearing community. ToCS is a deaf-friendly, computer-based test that measures child speechreading (silent lipreading) at three psycholinguistic levels: words, sentences and short stories. The aims of the study were to standardize ToCS with deaf and hearing children and investigate the effects of hearing status, age and linguistic complexity on speechreading ability. Method 86 severely and profoundly deaf and 91 hearing children aged between 5 and 14 years participated. The deaf children were from a range of language and communication backgrounds and their preferred mode of communication varied. Results: Speechreading skills significantly improved with age for both deaf and hearing children. There was no effect of hearing status on speechreading ability and deaf and hearing showed similar performance across all subtests on ToCS. Conclusions The Test of Child Speechreading (ToCS) is a valid and reliable assessment of speechreading ability in school-aged children that can be used to measure individual differences in performance in speechreading ability. PMID:23275416

  14. Reading Faces for Information about Words and Emotions in Adolescents with Autism

    PubMed Central

    Grossman, Ruth B.; Tager-Flusberg, Helen

    2008-01-01

    Studies of explicit processing of facial expressions by individuals with autism spectrum disorder (ASD) have found a variety of deficits and preserved abilities compared to their typically developing (TD) peers. However, little attention has been paid to their implicit processing abilities for emotional facial expressions. The question has also been raised whether preferential attention to the mouth region of a speaker’s face by ASD individuals has resulted in a relative lipreading expertise. We present data on implicit processing of pseudo-dynamic facial emotions and visual speech in adolescents with autism. We compared 25 ASD and 25 TD participants on their ability to recreate the sequences of four dynamic emotional facial expressions (happy, sad, disgust, fear) as well as four spoken words (with, bath, thumb, watch) using six still images taken from a video sequence. Typical adolescents were significantly better at recreating the dynamic properties of emotional expressions than those of facial speech, while the autism group showed the reverse accuracy pattern. For Experiment 2 we obscured the eye region of the stimuli and found no significant difference between the 22 adolescents with ASD and 22 TD controls. Fearful faces achieved the highest accuracy results among the emotions in both groups. PMID:19809572

  15. Effects of phonetic context on audio-visual intelligibility of French.

    PubMed

    Benoît, C; Mohamadi, T; Kandel, S

    1994-10-01

    Bimodal perception leads to better speech understanding than auditory perception alone. We evaluated the overall benefit of lip-reading on natural utterances of French produced by a single speaker. Eighteen French subjects with good audition and vision were administered a closed set identification test of VCVCV nonsense words consisting of three vowels [i, a, y] and six consonants [b, v, z, 3, R, l]. Stimuli were presented under both auditory and audio-visual conditions with white noise added at various signal-to-noise ratios. Identification scores were higher in the bimodal condition than in the auditory-alone condition, especially in situations where acoustic information was reduced. The auditory and audio-visual intelligibility of the three vowels [i, a, y] averaged over the six consonantal contexts was evaluated as well. Two different hierarchies of intelligibility were found. Auditorily, [a] was most intelligible, followed by [i] and then by [y]; whereas visually [y] was most intelligible, followed by [a] and [i]. We also quantified the contextual effects of the three vowels on the auditory and audio-visual intelligibility of the consonants. Both the auditory and the audio-visual intelligibility of surrounding consonants was highest in the [a] context, followed by the [i] context and lastly the [y] context.

  16. Preparing for communication interactions: the value of anticipatory strategies for adults with hearing impairment.

    PubMed

    Tye-Murray, N

    1992-04-01

    Some people with hearing impairment may use anticipatory strategies to prepare for an upcoming communication interaction, such as a doctor's appointment. They may consider vocabulary and statements that might occur, and they may practice speechreading a partner saying the items. Experiment 1 evaluated the effectiveness of two types of anticipatory strategies: workbook activities and situation-specific lipreading practice. Two groups of normal-hearing subjects were asked to prepare for a communication interaction in a bank setting where they would be required to recognize speech using only the visual signal. Each group was assigned to one type of anticipatory strategy. A third group served as a control group. Experiment 2 evaluated whether multifaceted anticipatory practice improved cochlear implant users' ability to recognize statements and words audiovisually that might occur in a doctor's office, bank, movie theater, and gas station. One group of implanted subjects received 4 days of training, 1 day for each setting, and a second group served as a control group. In both experiments, subjects who used anticipatory strategies did not improve their performance on situation-specific sentence tests more than the control subjects.

  17. A comparison of three speech coding strategies using an acoustic model of a cochlear implant.

    PubMed

    Blamey, P J; Martin, L F; Clark, G M

    1985-01-01

    Three alternative speech coding strategies suitable for use with cochlear implants were compared in a study of three normally hearing subjects using an acoustic model of a multiple-channel cochlear implant. The first strategy (F2) presented the amplitude envelope of the speech and the second formant frequency. The second strategy (F0 F2) included the voice fundamental frequency, and the third strategy (F0 F1 F2) presented the first formant frequency as well. Discourse level testing with the speech tracking method showed a clear superiority of the F0 F1 F2 strategy when the auditory information was used to supplement lipreading. Tracking rates averaged over three subjects for nine 10-min sessions were 40 wpm for F2, 52 wpm for F0 F2, and 66 wpm for F0 F1 F2. Vowel and consonant confusion studies and a test of prosodic information were carried out with auditory information only. The vowel test showed a significant difference between the strategies, but no differences were found for the other tests. It was concluded that the amplitude and duration cues common to all three strategies accounted for the levels of consonant and prosodic information received by the subjects, while the different tracking rates were a consequence of the better vowel recognition and the more natural quality of the F0 F1 F2 strategy.

  18. Sizing up the competition: Quantifying the influence of the mental lexicon on auditory and visual spoken word recognition

    PubMed Central

    Strand, Julia F.; Sommers, Mitchell S.

    2011-01-01

    Much research has explored how spoken word recognition is influenced by the architecture and dynamics of the mental lexicon (e.g., Luce and Pisoni, 1998; McClelland and Elman, 1986). A more recent question is whether the processes underlying word recognition are unique to the auditory domain, or whether visually perceived (lipread) speech may also be sensitive to the structure of the mental lexicon (Auer, 2002; Mattys, Bernstein, and Auer, 2002). The current research was designed to test the hypothesis that both aurally and visually perceived spoken words are isolated in the mental lexicon as a function of their modality-specific perceptual similarity to other words. Lexical competition (the extent to which perceptually similar words influence recognition of a stimulus word) was quantified using metrics that are well-established in the literature, as well as a statistical method for calculating perceptual confusability based on the phi-square statistic. Both auditory and visual spoken word recognition were influenced by modality-specific lexical competition as well as stimulus word frequency. These findings extend the scope of activation-competition models of spoken word recognition and reinforce the hypothesis (Auer, 2002; Mattys et al., 2002) that perceptual and cognitive properties underlying spoken word recognition are not specific to the auditory domain. In addition, the results support the use of the phi-square statistic as a better predictor of lexical competition than metrics currently used in models of spoken word recognition. PMID:21895103

  19. Age-related hearing difficulties. I. Hearing impairment, disability, and handicap--a controlled study.

    PubMed

    Salomon, G; Vesterager, V; Jagd, M

    1988-01-01

    The study compares the audiological profile of a group of first-time applicants for hearing aids, a group of re-applicants and a group of non-complainers, aged 70-75 years (n = 71). In spite of overlap in range, a significant difference in thresholds and discrimination was found. The lip-reading capacity was well preserved in the elderly, but showed a significant correlation to the general health condition. The audiological benefit of hearing-aids did not increase with early fitting. General satisfaction with life was independent of satisfaction with hearing; two thirds of the patients were satisfied with their aids and used them regularly. The rest were dissatisfied and used them less than once a week. The aids were most systematically used to watch TV. Pure-tone average and handicap scaling were compared as guidelines for hearing-aid fitting. The most powerful tool to identify those in need of hearing-aids was handicap scaling based on interviews concerning self-assessed hearing difficulties.

  20. Auditory Midbrain Implant: Research and Development Towards a Second Clinical Trial

    PubMed Central

    Lim, Hubert H.; Lenarz, Thomas

    2015-01-01

    The cochlear implant is considered one of the most successful neural prostheses to date, which was made possible by visionaries who continued to develop the cochlear implant through multiple technological and clinical challenges. However, patients without a functional auditory nerve or implantable cochlea cannot benefit from a cochlear implant. The focus of the paper is to review the development and translation of a new type of central auditory prosthesis for this group of patients, which is known as the auditory midbrain implant (AMI) and is designed for electrical stimulation within the inferior colliculus. The rationale and results for the first AMI clinical study using a multi-site single-shank array will be presented initially. Although the AMI has achieved encouraging results in terms of safety and improvements in lip-reading capabilities and environmental awareness, it has not yet provided sufficient speech perception. Animal and human data will then be presented to show that a two-shank AMI array can potentially improve hearing performance by targeting specific neurons of the inferior colliculus. Modifications to the AMI array design, stimulation strategy, and surgical approach have been made that are expected to improve hearing performance in the patients implanted with a two-shank array in an upcoming clinical trial funded by the National Institutes of Health. Positive outcomes from this clinical trial will motivate new efforts and developments toward improving central auditory prostheses for those who cannot sufficiently benefit from cochlear implants. PMID:25613994

  1. Auditory Midbrain Implant: A Review

    PubMed Central

    Lim, Hubert H.; Lenarz, Minoo; Lenarz, Thomas

    2009-01-01

    The auditory midbrain implant (AMI) is a new hearing prosthesis designed for stimulation of the inferior colliculus in deaf patients who cannot sufficiently benefit from cochlear implants. The authors have begun clinical trials in which five patients have been implanted with a single shank AMI array (20 electrodes). The goal of this review is to summarize the development and research that has led to the translation of the AMI from a concept into the first patients. This study presents the rationale and design concept for the AMI as well a summary of the animal safety and feasibility studies that were required for clinical approval. The authors also present the initial surgical, psychophysical, and speech results from the first three implanted patients. Overall, the results have been encouraging in terms of the safety and functionality of the implant. All patients obtain improvements in hearing capabilities on a daily basis. However, performance varies dramatically across patients depending on the implant location within the midbrain with the best performer still not able to achieve open set speech perception without lip-reading cues. Stimulation of the auditory midbrain provides a wide range of level, spectral, and temporal cues, all of which are important for speech understanding, but they do not appear to sufficiently fuse together to enable open set speech perception with the currently used stimulation strategies. Finally, several issues and hypotheses for why current patients obtain limited speech perception along with several feasible solutions for improving AMI implementation are presented. PMID:19762428

  2. The Perceptual Characteristics of Voice-Hallucinations in Deaf People: Insights into the Nature of Subvocal Thought and Sensory Feedback Loops

    PubMed Central

    Atkinson, Joanna R.

    2006-01-01

    The study of voice-hallucinations in deaf individuals, who exploit the visuomotor rather than auditory modality for communication, provides rare insight into the relationship between sensory experience and how “voices” are perceived. Relatively little is known about the perceptual characteristics of voice-hallucinations in congenitally deaf people who use lip-reading or sign language as their preferred means of communication. The existing literature on hallucinations in deaf people is reviewed, alongside consideration of how such phenomena may fit into explanatory subvocal articulation hypotheses proposed for auditory verbal hallucinations in hearing people. It is suggested that a failure in subvocal articulation processes may account for voice-hallucinations in both hearing and deaf people but that the distinct way in which hallucinations are experienced may be due to differences in a sensory feedback component, which is influenced by both auditory deprivation and language modality. This article highlights how the study of deaf people may inform wider understanding of auditory verbal hallucinations and subvocal processes generally. PMID:16510696

  3. Segmentation of human face using gradient-based approach

    NASA Astrophysics Data System (ADS)

    Baskan, Selin; Bulut, M. Mete; Atalay, Volkan

    2001-04-01

    This paper describes a method for automatic segmentation of facial features such as eyebrows, eyes, nose, mouth and ears in color images. This work is an initial step for wide range of applications based on feature-based approaches, such as face recognition, lip-reading, gender estimation, facial expression analysis, etc. Human face can be characterized by its skin color and nearly elliptical shape. For this purpose, face detection is performed using color and shape information. Uniform illumination is assumed. No restrictions on glasses, make-up, beard, etc. are imposed. Facial features are extracted using the vertically and horizontally oriented gradient projections. The gradient of a minimum with respect to its neighbor maxima gives the boundaries of a facial feature. Each facial feature has a different horizontal characteristic. These characteristics are derived by extensive experimentation with many face images. Using fuzzy set theory, the similarity between the candidate and the feature characteristic under consideration is calculated. Gradient-based method is accompanied by the anthropometrical information, for robustness. Ear detection is performed using contour-based shape descriptors. This method detects the facial features and circumscribes each facial feature with the smallest rectangle possible. AR database is used for testing. The developed method is also suitable for real-time systems.

  4. Auditory-visual speech perception in three- and four-year-olds and its relationship to perceptual attunement and receptive vocabulary.

    PubMed

    Erdener, Doğu; Burnham, Denis

    2017-06-06

    Despite the body of research on auditory-visual speech perception in infants and schoolchildren, development in the early childhood period remains relatively uncharted. In this study, English-speaking children between three and four years of age were investigated for: (i) the development of visual speech perception - lip-reading and visual influence in auditory-visual integration; (ii) the development of auditory speech perception and native language perceptual attunement; and (iii) the relationship between these and a language skill relevant at this age, receptive vocabulary. Visual speech perception skills improved even over this relatively short time period. However, regression analyses revealed that vocabulary was predicted by auditory-only speech perception, and native language attunement, but not by visual speech perception ability. The results suggest that, in contrast to infants and schoolchildren, in three- to four-year-olds the relationship between speech perception and language ability is based on auditory and not visual or auditory-visual speech perception ability. Adding these results to existing findings allows elaboration of a more complete account of the developmental course of auditory-visual speech perception.

  5. Prototype to product—developing a commercially viable neural prosthesis

    NASA Astrophysics Data System (ADS)

    Seligman, Peter

    2009-12-01

    The Cochlear implant or 'Bionic ear' is a device that enables people who do not get sufficient benefit from a hearing aid to communicate with the hearing world. The Cochlear implant is not an amplifier, but a device that electrically stimulates the auditory nerve in a way that crudely mimics normal hearing, thus providing a hearing percept. Many recipients are able to understand running speech without the help of lipreading. Cochlear implants have reached a stage of maturity where there are now 170 000 recipients implanted worldwide. The commercial development of these devices has occurred over the last 30 years. This development has been multidisciplinary, including audiologists, engineers, both mechanical and electrical, histologists, materials scientists, physiologists, surgeons and speech pathologists. This paper will trace the development of the device we have today, from the engineering perspective. The special challenges of designing an active device that will work in the human body for a lifetime will be outlined. These challenges include biocompatibility, extreme reliability, safety, patient fitting and surgical issues. It is emphasized that the successful development of a neural prosthesis requires the partnership of academia and industry.

  6. Hearing Loss: Communicating With the Patient Who Is Deaf or Hard of Hearing.

    PubMed

    McKee, Michael M; Moreland, Christopher; Atcherson, Samuel R; Zazove, Philip

    2015-07-01

    Hearing loss impairs health care communication and adversely affects patient satisfaction, treatment adherence, and use of health services. Hearing loss is the third most common chronic health condition among older patients after hypertension and arthritis, but only 15% to 18% of older adults are screened for hearing loss during health maintenance examinations. Patients with hearing loss may be reluctant to disclose it because of fear of ageism, perceptions of disability, and vanity. Lipreading and note writing often are ineffective ways to communicate with deaf and hard of hearing (DHH) patients who use American Sign Language; use of medical sign language interpreters is preferred. A variety of strategies can improve the quality of health care communication for DHH patients, such as the physician facing the patient, listening attentively, and using visual tools. Physicians should learn what hearing loss means to the DHH patient. Deaf American Sign Language users may not perceive hearing loss as a disability but as a cultural identity. Patients' preferred communication strategies will vary. Relay services, electronic communication, and other telecommunications methods can be helpful, but family physicians and medical staff should learn from each DHH patient about which communication strategies will work best.

  7. Speech perception, production and intelligibility in French-speaking children with profound hearing loss and early cochlear implantation after congenital cytomegalovirus infection.

    PubMed

    Laccourreye, L; Ettienne, V; Prang, I; Couloigner, V; Garabedian, E-N; Loundon, N

    2015-12-01

    To analyze speech in children with profound hearing loss following congenital cytomegalovirus (cCMV) infection with cochlear implantation (CI) before the age of 3 years. In a cohort of 15 children with profound hearing loss, speech perception, production and intelligibility were assessed before and 3 years after CI; variables impacting results were explored. Post-CI, median word recognition was 74% on closed-list and 48% on open-list testing; 80% of children acquired speech production; and 60% were intelligible for all listeners or listeners attentive to lip-reading and/or aware of the child's hearing loss. Univariate analysis identified 3 variables (mean post-CI hearing threshold, bilateral vestibular areflexia, and brain abnormality on MRI) with significant negative impact on the development of speech perception, production and intelligibility. CI showed positive impact on hearing and speech in children with post-cCMV profound hearing loss. Our study demonstrated the key role of maximizing post-CI hearing gain. A few children had insufficient progress, especially in case of bilateral vestibular areflexia and/or brain abnormality on MRI. This led us to suggest that balance rehabilitation and speech therapy should be intensified in such cases. Copyright © 2015 Elsevier Masson SAS. All rights reserved.

  8. Cochlear implantation in children and adults in Switzerland.

    PubMed

    Brand, Yves; Senn, Pascal; Kompis, Martin; Dillier, Norbert; Allum, John H J

    2014-02-04

    The cochlear implant (CI) is one of the most successful neural prostheses developed to date. It offers artificial hearing to individuals with profound sensorineural hearing loss and with insufficient benefit from conventional hearing aids. The first implants available some 30 years ago provided a limited sensation of sound. The benefit for users of these early systems was mostly a facilitation of lip-reading based communication rather than an understanding of speech. Considerable progress has been made since then. Modern, multichannel implant systems feature complex speech processing strategies, high stimulation rates and multiple sites of stimulation in the cochlea. Equipped with such a state-of-the-art system, the majority of recipients today can communicate orally without visual cues and can even use the telephone. The impact of CIs on deaf individuals and on the deaf community has thus been exceptional. To date, more than 300,000 patients worldwide have received CIs. In Switzerland, the first implantation was performed in 1977 and, as of 2012, over 2,000 systems have been implanted with a current rate of around 150 CIs per year. The primary purpose of this article is to provide a contemporary overview of cochlear implantation, emphasising the situation in Switzerland.

  9. Language access and theory of mind reasoning: evidence from deaf children in bilingual and oralist environments.

    PubMed

    Meristo, Marek; Falkman, Kerstin W; Hjelmquist, Erland; Tedoldi, Mariantonia; Surian, Luca; Siegal, Michael

    2007-09-01

    This investigation examined whether access to sign language as a medium for instruction influences theory of mind (ToM) reasoning in deaf children with similar home language environments. Experiment 1 involved 97 deaf Italian children ages 4-12 years: 56 were from deaf families and had LIS (Italian Sign Language) as their native language, and 41 had acquired LIS as late signers following contact with signers outside their hearing families. Children receiving bimodal/bilingual instruction in LIS together with Sign-Supported and spoken Italian significantly outperformed children in oralist schools in which communication was in Italian and often relied on lipreading. Experiment 2 involved 61 deaf children in Estonia and Sweden ages 6-16 years. On a wide variety of ToM tasks, bilingually instructed native signers in Estonian Sign Language and spoken Estonian succeeded at a level similar to age-matched hearing children. They outperformed bilingually instructed late signers and native signers attending oralist schools. Particularly for native signers, access to sign language in a bilingual environment may facilitate conversational exchanges that promote the expression of ToM by enabling children to monitor others' mental states effectively.

  10. Nurse practitioner perceptions of barriers and facilitators in providing health care for deaf American Sign Language users: A qualitative socio-ecological approach.

    PubMed

    Pendergrass, Kathy M; Nemeth, Lynne; Newman, Susan D; Jenkins, Carolyn M; Jones, Elaine G

    2017-06-01

    Nurse practitioners (NPs), as well as all healthcare clinicians, have a legal and ethical responsibility to provide health care for deaf American Sign Language (ASL) users equal to that of other patients, including effective communication, autonomy, and confidentiality. However, very little is known about the feasibility to provide equitable health care. The purpose of this study was to examine NP perceptions of barriers and facilitators in providing health care for deaf ASL users. Semistructured interviews in a qualitative design using a socio-ecological model (SEM). Barriers were identified at all levels of the SEM. NPs preferred interpreters to facilitate the visit, but were unaware of their role in assuring effective communication is achieved. A professional sign language interpreter was considered a last resort when all other means of communication failed. Gesturing, note-writing, lip-reading, and use of a familial interpreter were all considered facilitators. Interventions are needed at all levels of the SEM. Resources are needed to provide awareness of deaf communication issues and legal requirements for caring for deaf signers for practicing and student NPs. Protocols need to be developed and present in all healthcare facilities for hiring interpreters as well as quick access to contact information for these interpreters. ©2017 American Association of Nurse Practitioners.

  11. Silent speechreading in the absence of scanner noise: an event-related fMRI study.

    PubMed

    MacSweeney, M; Amaro, E; Calvert, G A; Campbell, R; David, A S; McGuire, P; Williams, S C; Woll, B; Brammer, M J

    2000-06-05

    In a previous study we used functional magnetic resonance imaging (fMRI) to demonstrate activation in auditory cortex during silent speechreading. Since image acquisition during fMRI generates acoustic noise, this pattern of activation could have reflected an interaction between background scanner noise and the visual lip-read stimuli. In this study we employed an event-related fMRI design which allowed us to measure activation during speechreading in the absence of acoustic scanner noise. In the experimental condition, hearing subjects were required to speechread random numbers from a silent speaker. In the control condition subjects watched a static image of the same speaker with mouth closed and were required to subvocally count an intermittent visual cue. A single volume of images was collected to coincide with the estimated peak of the blood oxygen level dependent (BOLD) response to these stimuli across multiple baseline and experimental trials. Silent speechreading led to greater activation in lateral temporal cortex relative to the control condition. This indicates that activation of auditory areas during silent speechreading is not a function of acoustic scanner noise and confirms that silent speechreading engages similar regions of auditory cortex as listening to speech.

  12. Activation and Functional Connectivity of the Left Inferior Temporal Gyrus during Visual Speech Priming in Healthy Listeners and Listeners with Schizophrenia

    PubMed Central

    Wu, Chao; Zheng, Yingjun; Li, Juanhua; Zhang, Bei; Li, Ruikeng; Wu, Haibo; She, Shenglin; Liu, Sha; Peng, Hongjun; Ning, Yuping; Li, Liang

    2017-01-01

    Under a “cocktail-party” listening condition with multiple-people talking, compared to healthy people, people with schizophrenia benefit less from the use of visual-speech (lipreading) priming (VSP) cues to improve speech recognition. The neural mechanisms underlying the unmasking effect of VSP remain unknown. This study investigated the brain substrates underlying the unmasking effect of VSP in healthy listeners and the schizophrenia-induced changes in the brain substrates. Using functional magnetic resonance imaging, brain activation and functional connectivity for the contrasts of the VSP listening condition vs. the visual non-speech priming (VNSP) condition were examined in 16 healthy listeners (27.4 ± 8.6 years old, 9 females and 7 males) and 22 listeners with schizophrenia (29.0 ± 8.1 years old, 8 females and 14 males). The results showed that in healthy listeners, but not listeners with schizophrenia, the VSP-induced activation (against the VNSP condition) of the left posterior inferior temporal gyrus (pITG) was significantly correlated with the VSP-induced improvement in target-speech recognition against speech masking. Compared to healthy listeners, listeners with schizophrenia showed significantly lower VSP-induced activation of the left pITG and reduced functional connectivity of the left pITG with the bilateral Rolandic operculum, bilateral STG, and left insular. Thus, the left pITG and its functional connectivity may be the brain substrates related to the unmasking effect of VSP, assumedly through enhancing both the processing of target visual-speech signals and the inhibition of masking-speech signals. In people with schizophrenia, the reduced unmasking effect of VSP on speech recognition may be associated with a schizophrenia-related reduction of VSP-induced activation and functional connectivity of the left pITG. PMID:28360829

  13. From Mimicry to Language: A Neuroanatomically Based Evolutionary Model of the Emergence of Vocal Language

    PubMed Central

    Poliva, Oren

    2016-01-01

    The auditory cortex communicates with the frontal lobe via the middle temporal gyrus (auditory ventral stream; AVS) or the inferior parietal lobule (auditory dorsal stream; ADS). Whereas the AVS is ascribed only with sound recognition, the ADS is ascribed with sound localization, voice detection, prosodic perception/production, lip-speech integration, phoneme discrimination, articulation, repetition, phonological long-term memory and working memory. Previously, I interpreted the juxtaposition of sound localization, voice detection, audio-visual integration and prosodic analysis, as evidence that the behavioral precursor to human speech is the exchange of contact calls in non-human primates. Herein, I interpret the remaining ADS functions as evidence of additional stages in language evolution. According to this model, the role of the ADS in vocal control enabled early Homo (Hominans) to name objects using monosyllabic calls, and allowed children to learn their parents' calls by imitating their lip movements. Initially, the calls were forgotten quickly but gradually were remembered for longer periods. Once the representations of the calls became permanent, mimicry was limited to infancy, and older individuals encoded in the ADS a lexicon for the names of objects (phonological lexicon). Consequently, sound recognition in the AVS was sufficient for activating the phonological representations in the ADS and mimicry became independent of lip-reading. Later, by developing inhibitory connections between acoustic-syllabic representations in the AVS and phonological representations of subsequent syllables in the ADS, Hominans became capable of concatenating the monosyllabic calls for repeating polysyllabic words (i.e., developed working memory). Finally, due to strengthening of connections between phonological representations in the ADS, Hominans became capable of encoding several syllables as a single representation (chunking). Consequently, Hominans began vocalizing and

  14. Visual speech discrimination and identification of natural and synthetic consonant stimuli

    PubMed Central

    Files, Benjamin T.; Tjan, Bosco S.; Jiang, Jintao; Bernstein, Lynne E.

    2015-01-01

    From phonetic features to connected discourse, every level of psycholinguistic structure including prosody can be perceived through viewing the talking face. Yet a longstanding notion in the literature is that visual speech perceptual categories comprise groups of phonemes (referred to as visemes), such as /p, b, m/ and /f, v/, whose internal structure is not informative to the visual speech perceiver. This conclusion has not to our knowledge been evaluated using a psychophysical discrimination paradigm. We hypothesized that perceivers can discriminate the phonemes within typical viseme groups, and that discrimination measured with d-prime (d’) and response latency is related to visual stimulus dissimilarities between consonant segments. In Experiment 1, participants performed speeded discrimination for pairs of consonant-vowel spoken nonsense syllables that were predicted to be same, near, or far in their perceptual distances, and that were presented as natural or synthesized video. Near pairs were within-viseme consonants. Natural within-viseme stimulus pairs were discriminated significantly above chance (except for /k/-/h/). Sensitivity (d’) increased and response times decreased with distance. Discrimination and identification were superior with natural stimuli, which comprised more phonetic information. We suggest that the notion of the viseme as a unitary perceptual category is incorrect. Experiment 2 probed the perceptual basis for visual speech discrimination by inverting the stimuli. Overall reductions in d’ with inverted stimuli but a persistent pattern of larger d’ for far than for near stimulus pairs are interpreted as evidence that visual speech is represented by both its motion and configural attributes. The methods and results of this investigation open up avenues for understanding the neural and perceptual bases for visual and audiovisual speech perception and for development of practical applications such as visual lipreading

  15. Effects of Visual Speech on Early Auditory Evoked Fields - From the Viewpoint of Individual Variance.

    PubMed

    Yahata, Izumi; Kawase, Tetsuaki; Kanno, Akitake; Hidaka, Hiroshi; Sakamoto, Shuichi; Nakasato, Nobukazu; Kawashima, Ryuta; Katori, Yukio

    2017-01-01

    The effects of visual speech (the moving image of the speaker's face uttering speech sound) on early auditory evoked fields (AEFs) were examined using a helmet-shaped magnetoencephalography system in 12 healthy volunteers (9 males, mean age 35.5 years). AEFs (N100m) in response to the monosyllabic sound /be/ were recorded and analyzed under three different visual stimulus conditions, the moving image of the same speaker's face uttering /be/ (congruent visual stimuli) or uttering /ge/ (incongruent visual stimuli), and visual noise (still image processed from speaker's face using a strong Gaussian filter: control condition). On average, latency of N100m was significantly shortened in the bilateral hemispheres for both congruent and incongruent auditory/visual (A/V) stimuli, compared to the control A/V condition. However, the degree of N100m shortening was not significantly different between the congruent and incongruent A/V conditions, despite the significant differences in psychophysical responses between these two A/V conditions. Moreover, analysis of the magnitudes of these visual effects on AEFs in individuals showed that the lip-reading effects on AEFs tended to be well correlated between the two different audio-visual conditions (congruent vs. incongruent visual stimuli) in the bilateral hemispheres but were not significantly correlated between right and left hemisphere. On the other hand, no significant correlation was observed between the magnitudes of visual speech effects and psychophysical responses. These results may indicate that the auditory-visual interaction observed on the N100m is a fundamental process which does not depend on the congruency of the visual information.

  16. Brain Networks Engaged in Audiovisual Integration During Speech Perception Revealed by Persistent Homology-Based Network Filtration

    PubMed Central

    Kim, Heejung; Hahm, Jarang; Lee, Hyekyoung; Kang, Eunjoo

    2015-01-01

    Abstract The human brain naturally integrates audiovisual information to improve speech perception. However, in noisy environments, understanding speech is difficult and may require much effort. Although the brain network is supposed to be engaged in speech perception, it is unclear how speech-related brain regions are connected during natural bimodal audiovisual or unimodal speech perception with counterpart irrelevant noise. To investigate the topological changes of speech-related brain networks at all possible thresholds, we used a persistent homological framework through hierarchical clustering, such as single linkage distance, to analyze the connected component of the functional network during speech perception using functional magnetic resonance imaging. For speech perception, bimodal (audio-visual speech cue) or unimodal speech cues with counterpart irrelevant noise (auditory white-noise or visual gum-chewing) were delivered to 15 subjects. In terms of positive relationship, similar connected components were observed in bimodal and unimodal speech conditions during filtration. However, during speech perception by congruent audiovisual stimuli, the tighter couplings of left anterior temporal gyrus-anterior insula component and right premotor-visual components were observed than auditory or visual speech cue conditions, respectively. Interestingly, visual speech is perceived under white noise by tight negative coupling in the left inferior frontal region–right anterior cingulate, left anterior insula, and bilateral visual regions, including right middle temporal gyrus, right fusiform components. In conclusion, the speech brain network is tightly positively or negatively connected, and can reflect efficient or effortful processes during natural audiovisual integration or lip-reading, respectively, in speech perception. PMID:25495216

  17. Visual speech discrimination and identification of natural and synthetic consonant stimuli.

    PubMed

    Files, Benjamin T; Tjan, Bosco S; Jiang, Jintao; Bernstein, Lynne E

    2015-01-01

    From phonetic features to connected discourse, every level of psycholinguistic structure including prosody can be perceived through viewing the talking face. Yet a longstanding notion in the literature is that visual speech perceptual categories comprise groups of phonemes (referred to as visemes), such as /p, b, m/ and /f, v/, whose internal structure is not informative to the visual speech perceiver. This conclusion has not to our knowledge been evaluated using a psychophysical discrimination paradigm. We hypothesized that perceivers can discriminate the phonemes within typical viseme groups, and that discrimination measured with d-prime (d') and response latency is related to visual stimulus dissimilarities between consonant segments. In Experiment 1, participants performed speeded discrimination for pairs of consonant-vowel spoken nonsense syllables that were predicted to be same, near, or far in their perceptual distances, and that were presented as natural or synthesized video. Near pairs were within-viseme consonants. Natural within-viseme stimulus pairs were discriminated significantly above chance (except for /k/-/h/). Sensitivity (d') increased and response times decreased with distance. Discrimination and identification were superior with natural stimuli, which comprised more phonetic information. We suggest that the notion of the viseme as a unitary perceptual category is incorrect. Experiment 2 probed the perceptual basis for visual speech discrimination by inverting the stimuli. Overall reductions in d' with inverted stimuli but a persistent pattern of larger d' for far than for near stimulus pairs are interpreted as evidence that visual speech is represented by both its motion and configural attributes. The methods and results of this investigation open up avenues for understanding the neural and perceptual bases for visual and audiovisual speech perception and for development of practical applications such as visual lipreading/speechreading speech

  18. Brain networks engaged in audiovisual integration during speech perception revealed by persistent homology-based network filtration.

    PubMed

    Kim, Heejung; Hahm, Jarang; Lee, Hyekyoung; Kang, Eunjoo; Kang, Hyejin; Lee, Dong Soo

    2015-05-01

    The human brain naturally integrates audiovisual information to improve speech perception. However, in noisy environments, understanding speech is difficult and may require much effort. Although the brain network is supposed to be engaged in speech perception, it is unclear how speech-related brain regions are connected during natural bimodal audiovisual or unimodal speech perception with counterpart irrelevant noise. To investigate the topological changes of speech-related brain networks at all possible thresholds, we used a persistent homological framework through hierarchical clustering, such as single linkage distance, to analyze the connected component of the functional network during speech perception using functional magnetic resonance imaging. For speech perception, bimodal (audio-visual speech cue) or unimodal speech cues with counterpart irrelevant noise (auditory white-noise or visual gum-chewing) were delivered to 15 subjects. In terms of positive relationship, similar connected components were observed in bimodal and unimodal speech conditions during filtration. However, during speech perception by congruent audiovisual stimuli, the tighter couplings of left anterior temporal gyrus-anterior insula component and right premotor-visual components were observed than auditory or visual speech cue conditions, respectively. Interestingly, visual speech is perceived under white noise by tight negative coupling in the left inferior frontal region-right anterior cingulate, left anterior insula, and bilateral visual regions, including right middle temporal gyrus, right fusiform components. In conclusion, the speech brain network is tightly positively or negatively connected, and can reflect efficient or effortful processes during natural audiovisual integration or lip-reading, respectively, in speech perception.

  19. Auditory midbrain implant: research and development towards a second clinical trial.

    PubMed

    Lim, Hubert H; Lenarz, Thomas

    2015-04-01

    The cochlear implant is considered one of the most successful neural prostheses to date, which was made possible by visionaries who continued to develop the cochlear implant through multiple technological and clinical challenges. However, patients without a functional auditory nerve or implantable cochlea cannot benefit from a cochlear implant. The focus of the paper is to review the development and translation of a new type of central auditory prosthesis for this group of patients that is known as the auditory midbrain implant (AMI) and is designed for electrical stimulation within the inferior colliculus. The rationale and results for the first AMI clinical study using a multi-site single-shank array will be presented initially. Although the AMI has achieved encouraging results in terms of safety and improvements in lip-reading capabilities and environmental awareness, it has not yet provided sufficient speech perception. Animal and human data will then be presented to show that a two-shank AMI array can potentially improve hearing performance by targeting specific neurons of the inferior colliculus. A new two-shank array, stimulation strategy, and surgical approach are planned for the AMI that are expected to improve hearing performance in the patients who will be implanted in an upcoming clinical trial funded by the National Institutes of Health. Positive outcomes from this clinical trial will motivate new efforts and developments toward improving central auditory prostheses for those who cannot sufficiently benefit from cochlear implants. This article is part of a Special Issue entitled . Copyright © 2015 Elsevier B.V. All rights reserved.

  20. Neural representation in the auditory midbrain of the envelope of vocalizations based on a peripheral ear model

    PubMed Central

    Rode, Thilo; Hartmann, Tanja; Hubka, Peter; Scheper, Verena; Lenarz, Minoo; Lenarz, Thomas; Kral, Andrej; Lim, Hubert H.

    2013-01-01

    The auditory midbrain implant (AMI) consists of a single shank array (20 sites) for stimulation along the tonotopic axis of the central nucleus of the inferior colliculus (ICC) and has been safely implanted in deaf patients who cannot benefit from a cochlear implant (CI). The AMI improves lip-reading abilities and environmental awareness in the implanted patients. However, the AMI cannot achieve the high levels of speech perception possible with the CI. It appears the AMI can transmit sufficient spectral cues but with limited temporal cues required for speech understanding. Currently, the AMI uses a CI-based strategy, which was originally designed to stimulate each frequency region along the cochlea with amplitude-modulated pulse trains matching the envelope of the bandpass-filtered sound components. However, it is unclear if this type of stimulation with only a single site within each frequency lamina of the ICC can elicit sufficient temporal cues for speech perception. At least speech understanding in quiet is still possible with envelope cues as low as 50 Hz. Therefore, we investigated how ICC neurons follow the bandpass-filtered envelope structure of natural stimuli in ketamine-anesthetized guinea pigs. We identified a subset of ICC neurons that could closely follow the envelope structure (up to ß100 Hz) of a diverse set of species-specific calls, which was revealed by using a peripheral ear model to estimate the true bandpass-filtered envelopes observed by the brain. Although previous studies have suggested a complex neural transformation from the auditory nerve to the ICC, our data suggest that the brain maintains a robust temporal code in a subset of ICC neurons matching the envelope structure of natural stimuli. Clinically, these findings suggest that a CI-based strategy may still be effective for the AMI if the appropriate neurons are entrained to the envelope of the acoustic stimulus and can transmit sufficient temporal cues to higher centers. PMID

  1. Auditory cross-modal reorganization in cochlear implant users indicates audio-visual integration.

    PubMed

    Stropahl, Maren; Debener, Stefan

    2017-01-01

    There is clear evidence for cross-modal cortical reorganization in the auditory system of post-lingually deafened cochlear implant (CI) users. A recent report suggests that moderate sensori-neural hearing loss is already sufficient to initiate corresponding cortical changes. To what extend these changes are deprivation-induced or related to sensory recovery is still debated. Moreover, the influence of cross-modal reorganization on CI benefit is also still unclear. While reorganization during deafness may impede speech recovery, reorganization also has beneficial influences on face recognition and lip-reading. As CI users were observed to show differences in multisensory integration, the question arises if cross-modal reorganization is related to audio-visual integration skills. The current electroencephalography study investigated cortical reorganization in experienced post-lingually deafened CI users (n = 18), untreated mild to moderately hearing impaired individuals (n = 18) and normal hearing controls (n = 17). Cross-modal activation of the auditory cortex by means of EEG source localization in response to human faces and audio-visual integration, quantified with the McGurk illusion, were measured. CI users revealed stronger cross-modal activations compared to age-matched normal hearing individuals. Furthermore, CI users showed a relationship between cross-modal activation and audio-visual integration strength. This may further support a beneficial relationship between cross-modal activation and daily-life communication skills that may not be fully captured by laboratory-based speech perception tests. Interestingly, hearing impaired individuals showed behavioral and neurophysiological results that were numerically between the other two groups, and they showed a moderate relationship between cross-modal activation and the degree of hearing loss. This further supports the notion that auditory deprivation evokes a reorganization of the auditory system even at

  2. A Deficit in Movement-Derived Sentences in German-Speaking Hearing-Impaired Children

    PubMed Central

    Ruigendijk, Esther; Friedmann, Naama

    2017-01-01

    Children with hearing impairment (HI) show disorders in syntax and morphology. The question is whether and how these disorders are connected to problems in the auditory domain. The aim of this paper is to examine whether moderate to severe hearing loss at a young age affects the ability of German-speaking orally trained children to understand and produce sentences. We focused on sentence structures that are derived by syntactic movement, which have been identified as a sensitive marker for syntactic impairment in other languages and in other populations with syntactic impairment. Therefore, our study tested subject and object relatives, subject and object Wh-questions, passive sentences, and topicalized sentences, as well as sentences with verb movement to second sentential position. We tested 19 HI children aged 9;5–13;6 and compared their performance with hearing children using comprehension tasks of sentence-picture matching and sentence repetition tasks. For the comprehension tasks, we included HI children who passed an auditory discrimination task; for the sentence repetition tasks, we selected children who passed a screening task of simple sentence repetition without lip-reading; this made sure that they could perceive the words in the tests, so that we could test their grammatical abilities. The results clearly showed that most of the participants with HI had considerable difficulties in the comprehension and repetition of sentences with syntactic movement: they had significant difficulties understanding object relatives, Wh-questions, and topicalized sentences, and in the repetition of object who and which questions and subject relatives, as well as in sentences with verb movement to second sentential position. Repetition of passives was only problematic for some children. Object relatives were still difficult at this age for both HI and hearing children. An additional important outcome of the study is that not all sentence structures are impaired

  3. A Deficit in Movement-Derived Sentences in German-Speaking Hearing-Impaired Children.

    PubMed

    Ruigendijk, Esther; Friedmann, Naama

    2017-01-01

    Children with hearing impairment (HI) show disorders in syntax and morphology. The question is whether and how these disorders are connected to problems in the auditory domain. The aim of this paper is to examine whether moderate to severe hearing loss at a young age affects the ability of German-speaking orally trained children to understand and produce sentences. We focused on sentence structures that are derived by syntactic movement, which have been identified as a sensitive marker for syntactic impairment in other languages and in other populations with syntactic impairment. Therefore, our study tested subject and object relatives, subject and object Wh-questions, passive sentences, and topicalized sentences, as well as sentences with verb movement to second sentential position. We tested 19 HI children aged 9;5-13;6 and compared their performance with hearing children using comprehension tasks of sentence-picture matching and sentence repetition tasks. For the comprehension tasks, we included HI children who passed an auditory discrimination task; for the sentence repetition tasks, we selected children who passed a screening task of simple sentence repetition without lip-reading; this made sure that they could perceive the words in the tests, so that we could test their grammatical abilities. The results clearly showed that most of the participants with HI had considerable difficulties in the comprehension and repetition of sentences with syntactic movement: they had significant difficulties understanding object relatives, Wh-questions, and topicalized sentences, and in the repetition of object who and which questions and subject relatives, as well as in sentences with verb movement to second sentential position. Repetition of passives was only problematic for some children. Object relatives were still difficult at this age for both HI and hearing children. An additional important outcome of the study is that not all sentence structures are impaired

  4. A universal bias in adult vowel perception - By ear or by eye.

    PubMed

    Masapollo, Matthew; Polka, Linda; Ménard, Lucie

    2017-09-01

    Speech perceivers are universally biased toward "focal" vowels (i.e., vowels whose adjacent formants are close in frequency, which concentrates acoustic energy into a narrower spectral region). This bias is demonstrated in phonetic discrimination tasks as a directional asymmetry: a change from a relatively less to a relatively more focal vowel results in significantly better performance than a change in the reverse direction. We investigated whether the critical information for this directional effect is limited to the auditory modality, or whether visible articulatory information provided by the speaker's face also plays a role. Unimodal auditory and visual as well as bimodal (auditory-visual) vowel stimuli were created from video recordings of a speaker producing variants of /u/, differing in both their degree of focalization and visible lip rounding (i.e., lip compression and protrusion). In Experiment 1, we confirmed that subjects showed an asymmetry while discriminating the auditory vowel stimuli. We then found, in Experiment 2, a similar asymmetry when subjects lip-read those same vowels. In Experiment 3, we found asymmetries, comparable to those found for unimodal vowels, for bimodal vowels when the audio and visual channels were phonetically-congruent. In contrast, when the audio and visual channels were phonetically-incongruent (as in the "McGurk effect"), this asymmetry was disrupted. These findings collectively suggest that the perceptual processes underlying the "focal" vowel bias are sensitive to articulatory information available across sensory modalities, and raise foundational issues concerning the extent to which vowel perception derives from general-auditory or speech-gesture-specific processes. Copyright © 2017 Elsevier B.V. All rights reserved.

  5. Electrophysiological Validation of a Human Prototype Auditory Midbrain Implant in a Guinea Pig Model

    PubMed Central

    Lenarz, Minoo; Patrick, James F.; Anderson, David J.; Lenarz, Thomas

    2006-01-01

    The auditory midbrain implant (AMI) is a new treatment for hearing restoration in patients with neural deafness or surgically inaccessible cochleae who cannot benefit from cochlear implants (CI). This includes neurofibromatosis type II (NF2) patients who, due to development and/or removal of vestibular schwannomas, usually experience complete damage of their auditory nerves. Although the auditory brainstem implant (ABI) provides sound awareness and aids lip-reading capabilities for these NF2 patients, it generally only achieves hearing performance levels comparable with a single-channel CI. In collaboration with Cochlear Ltd. (Lane Cove, Australia), we developed a human prototype AMI, which is designed for electrical stimulation along the well-defined tonotopic gradient of the inferior colliculus central nucleus (ICC). Considering that better speech perception and hearing performance has been correlated with a greater number of discriminable frequency channels of information available, the ability of the AMI to effectively activate discrete frequency regions within the ICC may enable better hearing performance than achieved by the ABI. Therefore, the goal of this study was to investigate if our AMI array could achieve low-threshold, frequency-specific activation within the ICC, and whether the levels for ICC activation via AMI stimulation were within safe limits for human application. We electrically stimulated different frequency regions within the ICC via the AMI array and recorded the corresponding neural activity in the primary auditory cortex (A1) using a multisite silicon probe in ketamine-anesthetized guinea pigs. Based on our results, AMI stimulation achieves lower thresholds and more localized, frequency-specific activation than CI stimulation. Furthermore, AMI stimulation achieves cortical activation with current levels that are within safe limits for central nervous system stimulation. This study confirms that our AMI design is sufficient for ensuring

  6. Effects of Visual Speech on Early Auditory Evoked Fields - From the Viewpoint of Individual Variance

    PubMed Central

    Yahata, Izumi; Kanno, Akitake; Hidaka, Hiroshi; Sakamoto, Shuichi; Nakasato, Nobukazu; Kawashima, Ryuta; Katori, Yukio

    2017-01-01

    The effects of visual speech (the moving image of the speaker’s face uttering speech sound) on early auditory evoked fields (AEFs) were examined using a helmet-shaped magnetoencephalography system in 12 healthy volunteers (9 males, mean age 35.5 years). AEFs (N100m) in response to the monosyllabic sound /be/ were recorded and analyzed under three different visual stimulus conditions, the moving image of the same speaker’s face uttering /be/ (congruent visual stimuli) or uttering /ge/ (incongruent visual stimuli), and visual noise (still image processed from speaker’s face using a strong Gaussian filter: control condition). On average, latency of N100m was significantly shortened in the bilateral hemispheres for both congruent and incongruent auditory/visual (A/V) stimuli, compared to the control A/V condition. However, the degree of N100m shortening was not significantly different between the congruent and incongruent A/V conditions, despite the significant differences in psychophysical responses between these two A/V conditions. Moreover, analysis of the magnitudes of these visual effects on AEFs in individuals showed that the lip-reading effects on AEFs tended to be well correlated between the two different audio-visual conditions (congruent vs. incongruent visual stimuli) in the bilateral hemispheres but were not significantly correlated between right and left hemisphere. On the other hand, no significant correlation was observed between the magnitudes of visual speech effects and psychophysical responses. These results may indicate that the auditory-visual interaction observed on the N100m is a fundamental process which does not depend on the congruency of the visual information. PMID:28141836

  7. [Inpatient rehabilitation of adult CI users: Results in dependency of duration of deafness, CI experience and age].

    PubMed

    Zeh, R; Baumann, U

    2015-08-01

    Cochlear implants (CI) have proven to be a highly effective treatment for severe hearing loss or deafness. Inpatient rehabilitation therapy is frequently discussed as a means to increase the speech perception abilities achieved by CI. However, thus far there exists no quantitative evaluation of the effect of these therapies. A retrospective analysis of audiometric data obtained from 1355 CI users compared standardized and qualitative speech intelligibility tests conducted at two time points (admission to and discharge from inpatient hearing therapy, duration 3-5 weeks). The test battery comprised examination of vowel/consonant identification, the Freiburg numbers and monosyllabic test (65 and 80 dB sound pressure level, SPL, free-field sound level), the Hochmair-Schulz-Moser (HSM) sentence test in quiet and in noise (65 dB SPL speech level; 15 dB signal-to-noise ratio, SNR), and a speech tracking test with and without lip-reading. An average increase of 20 percentage points was scored at discharge compared to the admission tests. Patients of all ages and duration of deafness demonstrated the same amount of benefit from the rehabilitation treatment. After completion of inpatient rehabilitation treatment, patients with short duration of CI experience (below 4 months) achieved test scores comparable to experienced long-term users. The demonstrated benefit of the treatment was independent of age and duration of deafness or CI experience. The rehabilitative training program significantly improved hearing abilities and speech perception in CI users, thus promoting their professional and social inclusion. The present results support the efficacy of inpatient rehabilitation for CI recipients. Integration of this or similar therapeutic concepts in the German catalog of follow-up treatment measures appears justified.

  8. Home-based Early Intervention on Auditory and Speech Development in Mandarin-speaking Deaf Infants and Toddlers with Chronological Aged 7-24 Months.

    PubMed

    Yang, Ying; Liu, Yue-Hui; Fu, Ming-Fu; Li, Chun-Lin; Wang, Li-Yan; Wang, Qi; Sun, Xi-Bin

    2015-08-20

    Early auditory and speech development in home-based early intervention of infants and toddlers with hearing loss younger than 2 years are still spare in China. This study aimed to observe the development of auditory and speech in deaf infants and toddlers who were fitted with hearing aids and/or received cochlear implantation between the chronological ages of 7-24 months, and analyze the effect of chronological age and recovery time on auditory and speech development in the course of home-based early intervention. This longitudinal study included 55 hearing impaired children with severe and profound binaural deafness, who were divided into Group A (7-12 months), Group B (13-18 months) and Group C (19-24 months) based on the chronological age. Categories auditory performance (CAP) and speech intelligibility rating scale (SIR) were used to evaluate auditory and speech development at baseline and 3, 6, 9, 12, 18, and 24 months of habilitation. Descriptive statistics were used to describe demographic features and were analyzed by repeated measures analysis of variance. With 24 months of hearing intervention, 78% of the patients were able to understand common phrases and conversation without lip-reading, 96% of the patients were intelligible to a listener. In three groups, children showed the rapid growth of trend features in each period of habilitation. CAP and SIR scores have developed rapidly within 24 months after fitted auxiliary device in Group A, which performed much better auditory and speech abilities than Group B (P < 0.05) and Group C (P < 0.05). Group B achieved better results than Group C, whereas no significant differences were observed between Group B and Group C (P > 0.05). The data suggested the early hearing intervention and home-based habilitation benefit auditory and speech development. Chronological age and recovery time may be major factors for aural verbal outcomes in hearing impaired children. The development of auditory and speech in hearing

  9. Audiovisual asynchrony detection and speech intelligibility in noise with moderate to severe sensorineural hearing impairment.

    PubMed

    Başkent, Deniz; Bazo, Danny

    2011-01-01

    The objective of this study is to explore the sensitivity to intermodal asynchrony in audiovisual speech with moderate to severe sensorineural hearing loss. Based on previous studies, two opposing expectations were an increase in sensitivity, as hearing-impaired listeners heavily rely on lipreading in daily life, and a reduction in sensitivity, as hearing-impaired listeners tend to be elderly and advanced age could potentially impair audiovisual integration. Adults with normal (N = 11, ages between 23 and 50 yrs) and impaired hearing (N = 11, ages between 54 and 81 yrs, the pure-tone average between 42 and 67 dB HL) participated in two experiments. In the first experiment, the synchrony judgments were recorded for varying intermodal time differences in audiovisual sentence recordings. In the second experiment, the intelligibility of audiovisual and audio-only speech was measured in speech-shaped noise, and correlations were explored between the synchrony window and intelligibility scores for individual listeners. Similar to previous studies, a sensitivity window on the order of a few hundred milliseconds was observed with all listeners. The average window shapes did not differ between normal-hearing and hearing-impaired groups; however, there was large individual variability. Individual windows were quantified by Gaussian curve fitting. Point of subjective simultaneity, a measure of window peak shift from the actual synchrony point, and full-width at half-maximum, a measure of window duration, were not correlated with participant's age or the degree of hearing loss. Points of subjective simultaneity were also not correlated with speech intelligibility scores. A moderate negative correlation that was significant at most conditions was observed between the full-width at half-maximum values and intelligibility scores. Contrary to either expectation per se, there was no indication of an effect of hearing impairment or age on the sensitivity to intermodal asynchrony in

  10. An Eye Tracking Study on the Perception and Comprehension of Unimodal and Bimodal Linguistic Inputs by Deaf Adolescents.

    PubMed

    Mastrantuono, Eliana; Saldaña, David; Rodríguez-Ortiz, Isabel R

    2017-01-01

    An eye tracking experiment explored the gaze behavior of deaf individuals when perceiving language in spoken and sign language only, and in sign-supported speech (SSS). Participants were deaf (n = 25) and hearing (n = 25) Spanish adolescents. Deaf students were prelingually profoundly deaf individuals with cochlear implants (CIs) used by age 5 or earlier, or prelingually profoundly deaf native signers with deaf parents. The effectiveness of SSS has rarely been tested within the same group of children for discourse-level comprehension. Here, video-recorded texts, including spatial descriptions, were alternately transmitted in spoken language, sign language and SSS. The capacity of these communicative systems to equalize comprehension in deaf participants with that of spoken language in hearing participants was tested. Within-group analyses of deaf participants tested if the bimodal linguistic input of SSS favored discourse comprehension compared to unimodal languages. Deaf participants with CIs achieved equal comprehension to hearing controls in all communicative systems while deaf native signers with no CIs achieved equal comprehension to hearing participants if tested in their native sign language. Comprehension of SSS was not increased compared to spoken language, even when spatial information was communicated. Eye movements of deaf and hearing participants were tracked and data of dwell times spent looking at the face or body area of the sign model were analyzed. Within-group analyses focused on differences between native and non-native signers. Dwell times of hearing participants were equally distributed across upper and lower areas of the face while deaf participants mainly looked at the mouth area; this could enable information to be obtained from mouthings in sign language and from lip-reading in SSS and spoken language. Few fixations were directed toward the signs, although these were more frequent when spatial language was transmitted. Both native and non

  11. Sequential bilateral cochlear implantation in children: parents' perspective and device use.

    PubMed

    Sparreboom, Marloes; Leeuw, A Rens; Snik, Ad F M; Mylanus, Emmanuel A M

    2012-03-01

    The purpose of this study was (1) to measure parental expectations before surgery of a sequentially placed second cochlear implant and compare these results with parental observations postoperatively and (2) to measure device use of the second cochlear implant and compare to unilateral implant use. Thirty prelingually deaf children with a unilateral cochlear implant (mean age at first implant 1.8 years) received a second implant at a mean age of 5.3 years. To measure parental expectations and observations, parents completed the Parents' Perspective before surgery of the second implant and after 12 and 24 months. The questionnaire included 1 additional question on sound localization. Device use of both the first and second implants was assessed retrospectively after 6, 12 and 24 months of implant use. Device use of the study group was also compared to a reference group of 30 unilateral implant users matched for age at second implantation. Parental expectations with regard to sound localization were significantly higher than the observed changes within the first year of bilateral implant use. The observed changes in communication, listening to speech without lipreading, and speech and language skills met or surpassed parental expectations. Irrespective of age at second implantation, the second implant was significantly less worn than the first implant. No significant difference was observed between the use of the second implant of the study group and device use of the reference group. Second implant use was significantly correlated with the difference in speech recognition between the 2 implants alone. Preoperative parental expectations were too high with regard to the observed localization skills within the first year of bilateral implant use. The study showed that several of these sequentially implanted children had more difficulties in wearing the second implant than in wearing the first implant during the rehabilitation period. The present results suggest that

  12. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput.

    PubMed

    Novich, Scott D; Eagleman, David M

    2015-10-01

    Touch receptors in the skin can relay various forms of abstract information, such as words (Braille), haptic feedback (cell phones, game controllers, feedback for prosthetic control), and basic visual information such as edges and shape (sensory substitution devices). The skin can support such applications with ease: They are all low bandwidth and do not require a fine temporal acuity. But what of high-throughput applications? We use sound-to-touch conversion as a motivating example, though others abound (e.g., vision, stock market data). In the past, vibrotactile hearing aids have demonstrated improvement in speech perceptions in the deaf. However, a sound-to-touch sensory substitution device that works with high efficacy and without the aid of lipreading has yet to be developed. Is this because skin simply does not have the capacity to effectively relay high-throughput streams such as sound? Or is this because the spatial and temporal properties of skin have not been leveraged to full advantage? Here, we begin to address these questions with two experiments. First, we seek to determine the best method of relaying information through the skin using an identification task on the lower back. We find that vibrotactile patterns encoding information in both space and time yield the best overall information transfer estimate. Patterns encoded in space and time or "intensity" (the coupled coding of vibration frequency and force) both far exceed performance of only spatially encoded patterns. Next, we determine the vibrotactile two-tacton resolution on the lower back-the distance necessary for resolving two vibrotactile patterns. We find that our vibratory motors conservatively require at least 6 cm of separation to resolve two independent tactile patterns (>80 % correct), regardless of stimulus type (e.g., spatiotemporal "sweeps" versus single vibratory pulses). Six centimeter is a greater distance than the inter-motor distances used in Experiment 1 (2.5 cm), which

  13. An Eye Tracking Study on the Perception and Comprehension of Unimodal and Bimodal Linguistic Inputs by Deaf Adolescents

    PubMed Central

    Mastrantuono, Eliana; Saldaña, David; Rodríguez-Ortiz, Isabel R.

    2017-01-01

    An eye tracking experiment explored the gaze behavior of deaf individuals when perceiving language in spoken and sign language only, and in sign-supported speech (SSS). Participants were deaf (n = 25) and hearing (n = 25) Spanish adolescents. Deaf students were prelingually profoundly deaf individuals with cochlear implants (CIs) used by age 5 or earlier, or prelingually profoundly deaf native signers with deaf parents. The effectiveness of SSS has rarely been tested within the same group of children for discourse-level comprehension. Here, video-recorded texts, including spatial descriptions, were alternately transmitted in spoken language, sign language and SSS. The capacity of these communicative systems to equalize comprehension in deaf participants with that of spoken language in hearing participants was tested. Within-group analyses of deaf participants tested if the bimodal linguistic input of SSS favored discourse comprehension compared to unimodal languages. Deaf participants with CIs achieved equal comprehension to hearing controls in all communicative systems while deaf native signers with no CIs achieved equal comprehension to hearing participants if tested in their native sign language. Comprehension of SSS was not increased compared to spoken language, even when spatial information was communicated. Eye movements of deaf and hearing participants were tracked and data of dwell times spent looking at the face or body area of the sign model were analyzed. Within-group analyses focused on differences between native and non-native signers. Dwell times of hearing participants were equally distributed across upper and lower areas of the face while deaf participants mainly looked at the mouth area; this could enable information to be obtained from mouthings in sign language and from lip-reading in SSS and spoken language. Few fixations were directed toward the signs, although these were more frequent when spatial language was transmitted. Both native and non

  14. Long-Term Outcomes, Education, and Occupational Level in Cochlear Implant Recipients Who Were Implanted in Childhood.

    PubMed

    Illg, Angelika; Haack, Marius; Lesinski-Schiedat, Anke; Büchner, Andreas; Lenarz, Thomas

    skill level achieved was 2.24 (range 1 to 4; SD = 0.57) which was significantly poorer (t(127) = 4.886; p = 0.001) than the mean skill level of the respondents (mean = 2.54; SD = 0.85). Data collection up to 17.75 (SD = 3.08; range 13 to 28) years post implant demonstrated that the majority of participants who underwent implantation at an early age achieved discrimination of speech sounds without lipreading (CAP category 4.00). Educational, vocational, and occupational level achieved by this cohort were significantly poorer compared with the German and worldwide population average. Children implanted today who are younger at implantation, and with whom more advanced up-to-date CIs are used, are expected to exhibit better auditory performance and have enhanced educational and occupational opportunities. Compared with the circumstances immediately after World War II in the 20th century, children with hearing impairment who use these implants have improved prospects in this regard.