Science.gov

Sample records for lipreading

  1. Script activation in lipreading.

    PubMed

    Samuelsson, S; Rönnberg, J

    1991-01-01

    In three experiments the roles of script, basic and low-level contexts in lipreading (Abbott et al., 1985) were studied. Three scripts were used: a restaurant, a railway station, and a clothing store. Presence or absence of script headers constituted a between-subjects factor. Basic and low-level context, as well as test-sentences (referring to basic and low-level of abstraction), were within-subjects factors. Also, different kinds of working memory tests were employed to assess potentially important information-processing skills for lipreading. The results show that (a) lipreaders benefit from a compatibility between context level and sentence type, (b) the directions of inferences were asymmetric such that the basic context prevented inferences at the low-level of abstraction, but not vice versa, (c) low-level context in general was most useful for skilled lipreaders in the script group, (d) skilled lipreaders in the non-script group took most advantage of basic level sentences, and (e) working memory capacity did not prove to be important for lipreading skill. Guessing and pure lipreading control experiments established that each of these potentially confounding factors only accounted for about 5% of the lipreading performance, and thus, can not explain the obtained interactive effects between visual information conveyed by lip movements and context. Theorizing departing from prototypicality vs. script accounts of inference-making was critically evaluated, and a predictive script activation mechanism, relevant to lipreading skill, was suggested. PMID:2068548

  2. Lipreading for the Deaf and Hard of Hearing.

    ERIC Educational Resources Information Center

    New York City Board of Education, Brooklyn, NY. Bureau of Curriculum Development.

    The guide provides fundamental principles of lipreading instruction and 38 sequential formal lessons in lipreading for use at the junior high or older level. It stresses that lipreading training aims to develop the understanding of words, phrases, and sentences rather than the study of exact lip movements. The lipreading ability of the child is…

  3. Recent developments in automated lip-reading

    NASA Astrophysics Data System (ADS)

    Bowden, Richard; Cox, Stephen; Harvey, Richard; Lan, Yuxuan; Ong, Eng-Jon; Owen, Gari; Theobald, Barry-John

    2013-10-01

    Human lip-readers are increasingly being presented as useful in the gathering of forensic evidence but, like all humans, suffer from unreliability. Here we report the results of a long-term study in automatic lip-reading with the objective of converting video-to-text (V2T). The V2T problem is surprising in that some aspects that look tricky, such as real-time tracking of the lips on poor-quality interlaced video from hand-held cameras, but prove to be relatively tractable. Whereas the problem of speaker independent lip-reading is very demanding due to unpredictable variations between people. Here we review the problem of automatic lip-reading for crime fighting and identify the critical parts of the problem.

  4. Lipreading in School-Age Children: The Roles of Age, Hearing Status, and Cognitive Ability

    ERIC Educational Resources Information Center

    Tye-Murray, Nancy; Hale, Sandra; Spehar, Brent; Myerson, Joel; Sommers, Mitchell S.

    2014-01-01

    Purpose: The study addressed three research questions: Does lipreading improve between the ages of 7 and 14 years? Does hearing loss affect the development of lipreading? How do individual differences in lipreading relate to other abilities? Method: Forty children with normal hearing (NH) and 24 with hearing loss (HL) were tested using 4…

  5. Lip-reading abilities in a subject with congenital prosopagnosia.

    PubMed

    Wathour, J; Decat, M; Vander Linden, F; Deggouj, N

    2015-01-01

    We present the case of an individual with congenital prosopagnosia or "face blindness", a disorder where the ability to recognize faces is impaired. We studied the lip-reading ability and audiovisual perception of this subject using a DVD with four conditions (audiovisual congruent, auditory, visual, and audiovisual incongruent) and compared results with a normal patient cohort. The patient had no correct responses in the visual lip-reading task; whereas, he improved in the audiovisual congruent task. In the audiovisual incongruent task, the patient provided one response; thus, he was able to lip-read. (He was able to use lip-reading/to use labial informations) This patient perceived only global dynamic facial movements, not the fine ones. He had a sufficient complementary use of lip-reading in audiovisual tasks, but not visual ones. These data are consistent with abnormal development of the pathways used for visual speech perception and associated with second-order face processing disorders and normal development of the audiovisual network for speech perception. PMID:26513947

  6. Tactual display of consonant voicing as a supplement to lipreading

    NASA Astrophysics Data System (ADS)

    Yuan, Hanfeng; Reed, Charlotte M.; Durlach, Nathaniel I.

    2005-08-01

    This research is concerned with the development and evaluation of a tactual display of consonant voicing to supplement the information available through lipreading for persons with profound hearing impairment. The voicing cue selected is based on the envelope onset asynchrony derived from two different filtered bands (a low-pass band and a high-pass band) of speech. The amplitude envelope of each of the two bands was used to modulate a different carrier frequency which in turn was delivered to one of the two fingers of a tactual stimulating device. Perceptual evaluations of speech reception through this tactual display included the pairwise discrimination of consonants contrasting voicing and identification of a set of 16 consonants under conditions of the tactual cue alone (T), lipreading alone (L), and the combined condition (L+T). The tactual display was highly effective for discriminating voicing at the segmental level and provided a substantial benefit to lipreading on the consonant-identification task. No such benefits of the tactual cue were observed, however, for lipreading of words in sentences due perhaps to difficulties in integrating the tactual and visual cues and to insufficient training on the more difficult task of connected-speech reception.

  7. The Story of Lip-Reading; Its Genesis and Development.

    ERIC Educational Resources Information Center

    DeLand, Fred; Montague, Harriet Andrews

    The historical developments of the use of lipreading from 1500 A.D. to 1931 are described. Education of the deaf is traced from its beginnings in Spain to England, Belgium, Holland, and France with the use of quotations from literature and old documents. The lives and works of Charles Michel de l'Epee and Samuel Heinicke, the beginning of…

  8. Recalibration of Phonetic Categories by Lipread Speech: Measuring Aftereffects after a 24-Hour Delay

    ERIC Educational Resources Information Center

    Vroomen, Jean; Baart, Martijn

    2009-01-01

    Listeners hearing an ambiguous speech sound flexibly adjust their phonetic categories in accordance with lipread information telling what the phoneme should be (recalibration). Here, we tested the stability of lipread-induced recalibration over time. Listeners were exposed to an ambiguous sound halfway between /t/ and /p/ that was dubbed onto a…

  9. Cross-modal Informational Masking of Lipreading by Babble.

    PubMed

    Myerson, Joel; Spehar, Brent; Tye-Murray, Nancy; Van Engen, Kristin; Hale, Sandra; Sommers, Mitchell S

    2016-01-01

    Whereas the energetic and informational masking effects of unintelligible babble on auditory speech recognition are well established, the present study is the first to investigate its effects on visual speech recognition. Young and older adults performed two lipreading tasks while simultaneously experiencing either quiet, speech-shaped noise, or 6-talker background babble. Both words at the end of uninformative carrier sentences and key words in everyday sentences were harder to lipread in the presence of babble than in the presence of speech-shaped noise or quiet. Contrary to the inhibitory deficit hypothesis of cognitive aging, babble had equivalent effects on young and older adults. In a follow-up experiment, neither the babble nor the speech-shaped noise stimuli interfered with performance of a face-processing task, indicating that babble selectively interferes with visual speech recognition and not with visual perception tasks per se. The present results demonstrate that babble can produce cross-modal informational masking and suggest a breakdown in audiovisual scene analysis, either because of obligatory monitoring of even uninformative speech sounds or because of obligatory efforts to integrate speech sounds even with uncorrelated mouth movements. PMID:26474981

  10. Experience with a talker can transfer across modalities to facilitate lipreading.

    PubMed

    Sanchez, Kauyumari; Dias, James W; Rosenblum, Lawrence D

    2013-10-01

    Rosenblum, Miller, and Sanchez (Psychological Science, 18, 392-396, 2007) found that subjects first trained to lip-read a particular talker were then better able to perceive the auditory speech of that same talker, as compared with that of a novel talker. This suggests that the talker experience a perceiver gains in one sensory modality can be transferred to another modality to make that speech easier to perceive. An experiment was conducted to examine whether this cross-sensory transfer of talker experience could occur (1) from auditory to lip-read speech, (2) with subjects not screened for adequate lipreading skill, (3) when both a familiar and an unfamiliar talker are presented during lipreading, and (4) for both old (presentation set) and new words. Subjects were first asked to identify a set of words from a talker. They were then asked to perform a lipreading task from two faces, one of which was of the same talker they heard in the first phase of the experiment. Results revealed that subjects who lip-read from the same talker they had heard performed better than those who lip-read a different talker, regardless of whether the words were old or new. These results add further evidence that learning of amodal talker information can facilitate speech perception across modalities and also suggest that this information is not restricted to previously heard words. PMID:23955059

  11. Some observations on computer lip-reading: moving from the dream to the reality

    NASA Astrophysics Data System (ADS)

    Bear, Helen L.; Owen, Gari; Harvey, Richard; Theobald, Barry-John

    2014-10-01

    In the quest for greater computer lip-reading performance there are a number of tacit assumptions which are either present in the datasets (high resolution for example) or in the methods (recognition of spoken visual units called "visemes" for example). Here we review these and other assumptions and show the surprising result that computer lip-reading is not heavily constrained by video resolution, pose, lighting and other practical factors. However, the working assumption that visemes, which are the visual equivalent of phonemes, are the best unit for recognition does need further examination. We conclude that visemes, which were defined over a century ago, are unlikely to be optimal for a modern computer lip-reading system.

  12. Lipreading and covert speech production similarly modulate human auditory-cortex responses to pure tones.

    PubMed

    Kauramäki, Jaakko; Jääskeläinen, Iiro P; Hari, Riitta; Möttönen, Riikka; Rauschecker, Josef P; Sams, Mikko

    2010-01-27

    Watching the lips of a speaker enhances speech perception. At the same time, the 100 ms response to speech sounds is suppressed in the observer's auditory cortex. Here, we used whole-scalp 306-channel magnetoencephalography (MEG) to study whether lipreading modulates human auditory processing already at the level of the most elementary sound features, i.e., pure tones. We further envisioned the temporal dynamics of the suppression to tell whether the effect is driven by top-down influences. Nineteen subjects were presented with 50 ms tones spanning six octaves (125-8000 Hz) (1) during "lipreading," i.e., when they watched video clips of silent articulations of Finnish vowels /a/, /i/, /o/, and /y/, and reacted to vowels presented twice in a row; (2) during a visual control task; (3) during a still-face passive control condition; and (4) in a separate experiment with a subset of nine subjects, during covert production of the same vowels. Auditory-cortex 100 ms responses (N100m) were equally suppressed in the lipreading and covert-speech-production tasks compared with the visual control and baseline tasks; the effects involved all frequencies and were most prominent in the left hemisphere. Responses to tones presented at different times with respect to the onset of the visual articulation showed significantly increased N100m suppression immediately after the articulatory gesture. These findings suggest that the lipreading-related suppression in the auditory cortex is caused by top-down influences, possibly by an efference copy from the speech-production system, generated during both own speech and lipreading. PMID:20107058

  13. Lipreading sentences with vibrotactile vocoders: performance of normal-hearing and hearing-impaired subjects.

    PubMed

    Bernstein, L E; Demorest, M E; Coulter, D C; O'Connell, M P

    1991-12-01

    Three vibrotactile vocoders were compared in a training study involving several different speech perception tasks. Vocoders were: (1) the Central Institute for the Deaf version of the Queen's University vocoder, with 1/3-oct filter spacing and logarithmic output scaling (CIDLog) [Engebretson and O'Connell, IEEE Trans. Biomed. Eng. BME-33, 712-716 (1986)]; (2) the same vocoder with linear output scaling (CIDLin); and (3) the Gallaudet University vocoder designed with greater resolution in the second formant region, relative to the CID vocoders, and linear output scaling (GULin). Four normal-hearing subjects were assigned to either of two control groups, visual-only control and vocoder control, for which they received the CIDLog vocoder. Five normal-hearing and four hearing-impaired subjects were assigned to the linear vocoders. Results showed that the three vocoders provided equivalent information in word-initial and word-final tactile-only consonant identification. However, GULin was the only vocoder significantly effective in enhancing lipreading of isolated prerecorded sentences. Individual subject analyses showed significantly enhanced lipreading by the three normal-hearing and two hearing-impaired subjects who received the GULin vocoder. Over the entire training period of the experiment, the mean difference between aided and unaided lipreading of sentences by the GULin aided hearing-impaired subjects was approximately 6% words correct. Possible explanations for failure to confirm previous success with the CIDLog vocoder [Weisenberger et al., J. Acoust. Soc. Am. 86, 1764-1775 (1989)] are discussed. PMID:1838561

  14. Recalibration of auditory phonemes by lipread speech is ear-specific.

    PubMed

    Keetels, Mirjam; Pecoraro, Mauro; Vroomen, Jean

    2015-08-01

    Listeners quickly learn to label an ambiguous speech sound if there is lipread information that tells what the sound should be (i.e., phonetic recalibration Bertelson, Vroomen, & de Gelder (2003)). We report the counter-intuitive result that the same ambiguous sound can be simultaneously adapted to two opposing phonemic interpretations if presented in the left and right ear. This is strong evidence against the notion that phonetic recalibration involves an adjustment of abstract phoneme boundaries. It rather supports the idea that phonetic recalibration is closely tied to the sensory specifics of the learning context. PMID:25981732

  15. There Goes the Neighborhood: Lipreading and the Structure of the Mental Lexicon

    PubMed Central

    Feld, Julia; Sommers, Mitchell

    2010-01-01

    A central question in spoken word recognition research is whether words are recognized relationally, in the context of other words in the mental lexicon [1, 2]. The current research evaluated metrics for measuring the influence of the mental lexicon on visually perceived (lipread) spoken word recognition. Lexical competition (the extent to which perceptually similar words influence recognition of a stimulus word) was quantified using metrics that are well-established in the literature, as well as a novel statistical method for calculating perceptual confusability, based on the Phi-square statistic. The Phi-square statistic proved an effective measure for assessing lexical competition and explained significant variance in visual spoken word recognition beyond that accounted for by traditional metrics. Because these values include the influence of all words in the lexicon (rather than only perceptually very similar words), it suggests that even perceptually distant words may receive some activation, and therefore provide competition, during spoken word recognition. This work supports and extends earlier research [3] that proposed a common recognition system underlying auditory and visual spoken word recognition and provides support for the use of the Phi-square statistic for quantifying lexical competition. PMID:21170172

  16. Lip-reading the BKB sentence lists: corrections for list and practice effects.

    PubMed

    Foster, J R; Summerfield, A Q; Marshall, D H; Palmer, L; Ball, V; Rosen, S

    1993-08-01

    Two groups of 21 adult subjects with normal hearing viewed the video recordings of the Bamford-Kowal-Bench standard sentence lists issued by the EPI Group in 1986. Each subject viewed all of the 21 lists and attempted to write down the words contained in each sentence. One group lip-read the lists with no sound (the LR:alone condition). The other group also heard a sequence of acoustic pulses which were synchronized to the moments when the talker's vocal folds closed (the LR&Lx condition). Performance was assessed both by loose (KW(L)) and by tight (KW(T)) keyword scoring methods. Both scoring methods produced the same pattern of results: performance was better in the LR&Lx condition; performance in both conditions improved linearly with the logarithm of the list presentation order number; subjects who produced higher overall scores also improved more with experience of the lists. The data were described well by a logistic regression model which provided a formula which can be used to compensate for practice effects and for differences in difficulty between lists. Two simpler, but less accurate, methods for compensating for variation in inter-list difficulty are also described. A figure is provided which can be used to assess the significance of the difference between a pair of scores obtained from a single subject in any pair of presentation conditions. PMID:8312846

  17. Visual words for lip-reading

    NASA Astrophysics Data System (ADS)

    Hassanat, Ahmad B. A.; Jassim, Sabah

    2010-04-01

    In this paper, the automatic lip reading problem is investigated, and an innovative approach to providing solutions to this problem has been proposed. This new VSR approach is dependent on the signature of the word itself, which is obtained from a hybrid feature extraction method dependent on geometric, appearance, and image transform features. The proposed VSR approach is termed "visual words". The visual words approach consists of two main parts, 1) Feature extraction/selection, and 2) Visual speech feature recognition. After localizing face and lips, several visual features for the lips where extracted. Such as the height and width of the mouth, mutual information and the quality measurement between the DWT of the current ROI and the DWT of the previous ROI, the ratio of vertical to horizontal features taken from DWT of ROI, The ratio of vertical edges to horizontal edges of ROI, the appearance of the tongue and the appearance of teeth. Each spoken word is represented by 8 signals, one of each feature. Those signals maintain the dynamic of the spoken word, which contains a good portion of information. The system is then trained on these features using the KNN and DTW. This approach has been evaluated using a large database for different people, and large experiment sets. The evaluation has proved the visual words efficiency, and shown that the VSR is a speaker dependent problem.

  18. Study of lip-reading detecting and locating technique

    NASA Astrophysics Data System (ADS)

    Wang, Lirong; Li, Jie; Zhao, Yanyan

    2008-03-01

    With the development of human computer interaction, lip reading technology has become a topic focus in the multimode technologic field. However, detecting and locating lip accurately are very difficult because lip contours of different people, varied illuminant conditions, head movements and other factors. Based on the methods of detecting and locating lip we proposed the methods which are based on the lips color extracted lip contour using the adaptive chromatic filter from the facial images. It is not sensitive to illumination, but appropriate chromatic lip filter is given by analyzing the entire face color and clustering statistics of lip color. It is proposed the combinable method which is preprocessing the face image including rotating the angle of face and improving image contrast in this paper and the lip region is analyzed clustering characteristics for the skin color and lip color, obtained adaptive chromatic filter which can prominent lips from the facial image. This method overcomes the varied illuminate, incline face. The experiments showed that it enhanced detection and location accurately through rough detecting lip region. It lay a good foundation for extraction the lip feature and tracking lip subsequently.

  19. Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing

    PubMed Central

    Goebel, Stacey; Tye-Murray, Nancy

    2015-01-01

    Purpose This study compared the use of 2 different types of contextual cues (sentence based and situation based) in 2 different modalities (visual only and auditory only). Method Twenty young adults were tested with the Illustrated Sentence Test (Tye-Murray, Hale, Spehar, Myerson, & Sommers, 2014) and the Speech Perception in Noise Test (Bilger, Nuetzel, Rabinowitz, & Rzeczkowski, 1984; Kalikow, Stevens, & Elliott, 1977) in the 2 modalities. The Illustrated Sentences Test presents sentences with no context and sentences accompanied by picture-based situational context cues. The Speech Perception in Noise Test presents sentences with low sentence-based context and sentences with high sentence-based context. Results Participants benefited from both types of context and received more benefit when testing occurred in the visual-only modality than when it occurred in the auditory-only modality. Participants' use of sentence-based context did not correlate with use of situation-based context. Cue usage did not correlate between the 2 modalities. Conclusions The ability to use contextual cues appears to be dependent on the type of cue and the presentation modality of the target word(s). In a theoretical sense, the results suggest that models of word recognition and sentence processing should incorporate the influence of multiple sources of information and recognize that the 2 types of context have different influences on speech perception. In a clinical sense, the results suggest that aural rehabilitation programs might provide training to optimize use of both kinds of contextual cues. PMID:25863923

  20. Lipreading and audiovisual speech recognition across the adult lifespan: Implications for audiovisual integration.

    PubMed

    Tye-Murray, Nancy; Spehar, Brent; Myerson, Joel; Hale, Sandra; Sommers, Mitchell

    2016-06-01

    In this study of visual (V-only) and audiovisual (AV) speech recognition in adults aged 22-92 years, the rate of age-related decrease in V-only performance was more than twice that in AV performance. Both auditory-only (A-only) and V-only performance were significant predictors of AV speech recognition, but age did not account for additional (unique) variance. Blurring the visual speech signal decreased speech recognition, and in AV conditions involving stimuli associated with equivalent unimodal performance for each participant, speech recognition remained constant from 22 to 92 years of age. Finally, principal components analysis revealed separate visual and auditory factors, but no evidence of an AV integration factor. Taken together, these results suggest that the benefit that comes from being able to see as well as hear a talker remains constant throughout adulthood and that changes in this AV advantage are entirely driven by age-related changes in unimodal visual and auditory speech recognition. (PsycINFO Database Record PMID:27294718

  1. Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing

    ERIC Educational Resources Information Center

    Spehar, Brent; Goebel, Stacey; Tye-Murray, Nancy

    2015-01-01

    Purpose: This study compared the use of 2 different types of contextual cues (sentence based and situation based) in 2 different modalities (visual only and auditory only). Method: Twenty young adults were tested with the Illustrated Sentence Test (Tye-Murray, Hale, Spehar, Myerson, & Sommers, 2014) and the Speech Perception in Noise Test…

  2. Lip-Reading Aids Word Recognition Most in Moderate Noise: A Bayesian Explanation Using High-Dimensional Feature Space

    PubMed Central

    Ross, Lars A.; Foxe, John J.; Parra, Lucas C.

    2009-01-01

    Watching a speaker's facial movements can dramatically enhance our ability to comprehend words, especially in noisy environments. From a general doctrine of combining information from different sensory modalities (the principle of inverse effectiveness), one would expect that the visual signals would be most effective at the highest levels of auditory noise. In contrast, we find, in accord with a recent paper, that visual information improves performance more at intermediate levels of auditory noise than at the highest levels, and we show that a novel visual stimulus containing only temporal information does the same. We present a Bayesian model of optimal cue integration that can explain these conflicts. In this model, words are regarded as points in a multidimensional space and word recognition is a probabilistic inference process. When the dimensionality of the feature space is low, the Bayesian model predicts inverse effectiveness; when the dimensionality is high, the enhancement is maximal at intermediate auditory noise levels. When the auditory and visual stimuli differ slightly in high noise, the model makes a counterintuitive prediction: as sound quality increases, the proportion of reported words corresponding to the visual stimulus should first increase and then decrease. We confirm this prediction in a behavioral experiment. We conclude that auditory-visual speech perception obeys the same notion of optimality previously observed only for simple multisensory stimuli. PMID:19259259

  3. Multisensory narrative tracking by a profoundly deaf subject using an electrocutaneous vocoder and a vibrotactile aid.

    PubMed

    Lynch, M P; Eilers, R E; Oller, D K; Urbano, R C; Pero, P J

    1989-06-01

    A congenitally, profoundly deaf adult who had received 41 hours of tactual word recognition training in a previous study was assessed in tracking of connected discourse. This assessment was conducted in three phases. In the first phase, the subject used the Tacticon 1600 electrocutaneous vocoder to track a narrative in three conditions: (a) lipreading and aided hearing (L + H), (b) lipreading and tactual vocoder (L + TV), and (c) lipreading, tactual vocoder, and aided hearing (L + TV + H). Subject performance was significantly better in the L + TV + H condition than in the L + H condition, suggesting that the subject benefitted from the additional information provided by the tactual vocoder. In the second phase, the Tactaid II vibrotactile aid was used in three conditions: (a) lipreading alone, (b) lipreading and tactual aid (L + TA), and (c) lipreading, tactual aid, and aided hearing (L + TA + H). The subject was able to combine cues from the Tactaid II with those from lipreading and aided hearing. In the third phase, both tactual devices were used in six conditions: (a) lipreading alone (L), (b) lipreading and aided hearing (L + H), (c) lipreading and Tactaid II (L + TA), (d) lipreading and Tacticon 1600 (L + TV), (e) lipreading, Tactaid II, and aided hearing (L + TA + H), and (f) lipreading, Tacticon 1600, and aided hearing (L + TV + H). In this phase, only the Tactaid II significantly improved tracking performance over lipreading and aided hearing. Overall, improvement in tracking performance occurred within and across phases of this study. PMID:2739385

  4. 77 FR 24554 - Culturally Significant Objects Imported for Exhibition; Determinations: “Quay Brothers: On...

    Federal Register 2010, 2011, 2012, 2013, 2014

    2012-04-24

    ... Pharmacist's Prescription for Lip-Reading Puppets'' AGENCY: State Department. ACTION: Notice. SUMMARY: Notice... Lip-Reading Puppets'' imported from abroad for temporary exhibition within the United States, are...

  5. A 12-consonant confusion study on a multiple-channel cochlear implant patient.

    PubMed

    Dowell, R C; Martin, L F; Tong, Y C; Clark, G M; Seligman, P M; Patrick, J F

    1982-12-01

    A consonant confusion study was undertaken on a multiple-channel cochlear implant patient using a wearable speech processing device. This patient suffered from total bilateral deafness acquired postlingually. The consonants /b/, /p/, /m/, /v/, /f/, /d/, /t/, /n/, /z/, /s/, /g/, /k/ were presented in a VCV context with the vowel /a/ as in father by a male and female speaker under three conditions: lipreading alone; electrical stimulation alone using the wearable speech processor and multiple-channel cochlear implant; lipreading in conjunction with electrical stimulation. No significant difference was detected between the results for the male and female speakers. The percentage correct scores for the pooled results of both speakers were lipreading alone--30%; electrical stimulation alone--48%; lipreading with electrical stimulation--70%. Performance was significantly better for lipreading with electrical stimulation than for lipreading alone and for electrical stimulation alone than for lipreading alone. An information transmission analysis demonstrated the effective integration of visual and auditory information for lipreading with electrical stimulation. There was a significant improvement in performance for the electrical stimulation alone condition over the 2 months of the study in contrast to no such improvement for lipreading alone. PMID:6897661

  6. Effects of a Wearable, Tactile Aid on Language Comprehension of Prelingual Profoundly Deaf Children.

    ERIC Educational Resources Information Center

    Proctor, Adele

    Factors influencing the use of nonacoustic aids (such as visual displays and tactile devices) with the hearing impaired are reviewed. The benefits of tactile devices in improving speech reading/lipreading and speech are pointed out. Tactile aids which provide information on rhythm, rate, intensity, and duration of speech increase lipreading and…

  7. Phonetic Recalibration Only Occurs in Speech Mode

    ERIC Educational Resources Information Center

    Vroomen, Jean; Baart, Martijn

    2009-01-01

    Upon hearing an ambiguous speech sound dubbed onto lipread speech, listeners adjust their phonetic categories in accordance with the lipread information (recalibration) that tells what the phoneme should be. Here we used sine wave speech (SWS) to show that this tuning effect occurs if the SWS sounds are perceived as speech, but not if the sounds…

  8. Development of a speech autocuer

    NASA Technical Reports Server (NTRS)

    Bedles, R. L.; Kizakvich, P. N.; Lawson, D. T.; Mccartney, M. L.

    1980-01-01

    A wearable, visually based prosthesis for the deaf based upon the proven method for removing lipreading ambiguity known as cued speech was fabricated and tested. Both software and hardware developments are described, including a microcomputer, display, and speech preprocessor.

  9. What's Hearing Loss?

    MedlinePlus

    ... Medical Words En Español What Other Kids Are Reading Back-to-School Butterflies? Read This Chloe & Nurb ... like these to communicate: speechreading (also called lip-reading), which involves looking closely at a person's lips, ...

  10. Modalities of memory: is reading lips like hearing voices?

    PubMed

    Maidment, David W; Macken, Bill; Jones, Dylan M

    2013-12-01

    Functional similarities in verbal memory performance across presentation modalities (written, heard, lipread) are often taken to point to a common underlying representational form upon which the modalities converge. We show here instead that the pattern of performance depends critically on presentation modality and different mechanisms give rise to superficially similar effects across modalities. Lipread recency is underpinned by different mechanisms to auditory recency, and while the effect of an auditory suffix on an auditory list is due to the perceptual grouping of the suffix with the list, the corresponding effect with lipread speech is due to misidentification of the lexical content of the lipread suffix. Further, while a lipread suffix does not disrupt auditory recency, an auditory suffix does disrupt recency for lipread lists. However, this effect is due to attentional capture ensuing from the presentation of an unexpected auditory event, and is evident both with verbal and nonverbal auditory suffixes. These findings add to a growing body of evidence that short-term verbal memory performance is determined by modality-specific perceptual and motor processes, rather than by the storage and manipulation of phonological representations. PMID:24041834

  11. Clinical results with a hearing aid and a single-channel vibrotactile device for profoundly deaf adults.

    PubMed

    Blamey, P J; Dowell, R C; Brown, A M; Clark, G M

    1985-08-01

    The speech perception of a group of 19 adults with post-lingual profound to total hearing loss was tested with nine closed-set speech tests without lipreading, two open-set tests without lipreading and two open-set speech tests with lipreading. The subjects were all prospective cochlear implant patients participating in a clinical trial of the implant and the results reported here were obtained as part of the pre-operative assessment. They were divided into groups on the basis of their prior experience with the aid(s), their speech detection thresholds with the two aids and their personal preferences. Seven of the subjects used a hand-held single-channel vibrotactile device and the other 12 used a powerful conventional hearing aid. Subjects from each group scored significantly better than chance on the closed set tests without lipreading. Training or regular hearing aid use was correlated with good performance on the closed-set tests. No subject showed a significant improvement of the lipreading score when the aid was used as a supplement. The use of sophisticated wearable tactile devices and extensive training may allow a better result, but in this clinical program, neither a hearing aid nor a single-channel vibrotactile device greatly benefited the postlingually profoundly deaf adults. PMID:4063556

  12. Tracking skill of a deaf person with long-term tactile aid experience: a case study.

    PubMed

    Cholewiak, R W; Sherrick, C E

    1986-04-01

    This paper describes a case study of a single deaf individual who has been using a vibrotactile aid for approximately 13 years. He has acquired the ability to lip-read speakers in three languages, using the speech-analyzing device that he and his collaborators have developed. The report describes his communicative abilities with and without the aid in his native language, which is Russian, and in English and Hebrew. When he was tested with the De Filippo-Scott connected-discourse tracking technique, the aid produced a considerable improvement in performance over that for unaided lipreading. The amount of improvement was a function of several factors, in particular his unaided lipreading rates for the different languages. PMID:3723423

  13. Is automated conversion of video to text a reality?

    NASA Astrophysics Data System (ADS)

    Bowden, Richard; Cox, Stephen J.; Harvey, Richard W.; Lan, Yuxuan; Ong, Eng-Jon; Owen, Gari; Theobald, Barry-John

    2012-10-01

    A recent trend in law enforcement has been the use of Forensic lip-readers. Criminal activities are often recorded on CCTV or other video gathering systems. Knowledge of what suspects are saying enriches the evidence gathered but lip-readers, by their own admission, are fallible so, based on long term studies of automated lip-reading, we are investigating the possibilities and limitations of applying this technique under realistic conditions. We have adopted a step-by-step approach and are developing a capability when prior video information is available for the suspect of interest. We use the terminology video-to-text (V2T) for this technique by analogy with speech-to-text (S2T) which also has applications in security and law-enforcement.

  14. READING YOUR OWN LIPS: COMMON CODING THEORY AND VISUAL SPEECH PERCEPTION

    PubMed Central

    Tye-Murray, Nancy; Spehar, Brent P.; Myerson, Joel; Hale, Sandra; Sommers, Mitchell S.

    2012-01-01

    Common coding theory posits: 1) perceiving an action activates the same representations of motor plans that are activated by actually performing that action; 2) because of individual differences in the way actions are performed, observing recordings of one’s own previous behavior activates motor plans to an even greater degree than observing someone else’s behavior. We hypothesized that if observing oneself activates motor plans to a greater degree than observing others, and these activated plans contribute to perception, then people should be able to lipread silent video clips of their own previous utterances more accurately than they can lipread video clips of other talkers. As predicted, two groups of participants were able to lipread video clips of themselves recorded more than two weeks earlier significantly more accurately than video clips of others. These results suggest that visual input activates speech motor activity that links to word representations in the mental lexicon. PMID:23132604

  15. Psychoacoustic aspects of speech pattern coding for the deaf.

    PubMed

    Faulkner, A; Fourcin, A J; Moore, B C

    1990-01-01

    The SiVo aid, which provides a sinusoidal signal indicating voice fundamental frequency and voicing information, was compared as an aid to lipreading to a conventional hearing aid having extended low-frequency output. Speech perceptual measures of consonant confusions in lipreading and the identification of a simple intonation contrast were collected from a group of 8 profoundly deaf adults. Audiometric and psychoacoustic measures were also collected. In the intonation task, 6 of the 7 patients tested performed better with the SiVo aid and the seventh scored perfectly with both aids. Four patients (out of 8) preferred to use the SiVo aid rather than the conventional aid, and 2 of these showed a significant advantage with the SiVo aid in the aided lipreading of consonants. The remaining 4 patients preferred the conventional aid, although none of these patients showed a significant advantage with the conventional aid in the lipreading of consonants. The 4 patients who preferred the SiVo aid over the conventional aid had very profound hearing loss and minimal dynamic range at 500 Hz and above, and those receiving the greatest benefit showed no measurable frequency selectivity. In the 2 patients who preferred the SiVo aid, yet showed no significant advantage for the SiVo aid in consonant lipreading, consonant confusions were also measured for aided lipreading, using a prototype aid providing both voice fundamental frequency and voiceless excitation patterns. Both patients showed additional and significant benefit from inclusion of the voiceless pattern element. PMID:2356724

  16. Speech recognition for 40 patients receiving multichannel cochlear implants.

    PubMed

    Dowell, R C; Mecklenburg, D J; Clark, G M

    1986-10-01

    We collected data on 40 patients who received the Nucleus multichannel cochlear implant. Results were reviewed to determine if the coding strategy is effective in transmitting the intended speech features and to assess patient benefit in terms of communication skills. All patients demonstrated significant improvement over preoperative results with a hearing aid for both lipreading enhancement and speech recognition without lipreading. Of the patients, 50% demonstrated ability to understand connected discourse with auditory input only. For the 23 patients who were tested 12 months postoperatively, there was substantial improvement in open-set speech recognition. PMID:3755975

  17. Multisensory Narrative Tracking by a Profoundly Deaf Subject Using an Electrocutaneous Vocoder and a Vibrotactile Aid.

    ERIC Educational Resources Information Center

    Lynch, Michael P.; And Others

    1989-01-01

    The study assessed the ability to track connected discourse by a congenitally profoundly deaf adult using an electrocutaneous vocoder and/or a vibrotactile aid in conjunction with or without lipreading and aided hearing. Overall, improvement in tracking performance occurred within and across phases of the study. (Author/DB)

  18. Visual Cues and Listening Effort: Individual Variability

    ERIC Educational Resources Information Center

    Picou, Erin M.; Ricketts, Todd A; Hornsby, Benjamin W. Y.

    2011-01-01

    Purpose: To investigate the effect of visual cues on listening effort as well as whether predictive variables such as working memory capacity (WMC) and lipreading ability affect the magnitude of listening effort. Method: Twenty participants with normal hearing were tested using a paired-associates recall task in 2 conditions (quiet and noise) and…

  19. SELF TEACHING IN THE DEVELOPMENT OF SPEECHREADING IN DEAF CHILDREN.

    ERIC Educational Resources Information Center

    NEYHUS, ARTHUR I.

    THE EFFECTIVENESS OF MOTION PICTURE FILMS AS A TEACHING DEVICE IN THE DEVELOPMENT OF LIPREADING SKILLS AND THE USE OF A CARTRIDGE-LOAD, SELF-WINDING EIGHT MILLIMETER PROJECTOR AS A TEACHING TOOL WERE STUDIED. IT WAS HYPOTHESIZED THAT DEAF AND HARD OF HEARING CHILDREN WOULD LEARN PRESCRIBED VOCABULARY MORE QUICKLY BY AUTOINSTRUCTIONAL FILM METHODS…

  20. A FEASIBILITY STUDY TO INVESTIGATE THE INSTRUMENTATION, ESTABLISHMENT, AND OPERATION OF A LEARNING LABORATORY FOR HARD-OF-HEARING CHILDREN, FINAL REPORT.

    ERIC Educational Resources Information Center

    STEPP, ROBERT E.

    TEN CHILDREN AGED 5-8 WERE SELECTED TO TEST A SELF-INSTRUCTIONAL, SELF-OPERATING SYSTEM TO DEVELOP LIPREADING SKILLS. THEIR HEARING DEFICIENCY RANGED FROM HARD OF HEARING TO PROFOUNDLY DEAF. THE SYSTEM CONSISTED OF THREE STUDY CARRELS, AN 8-MM CARTRIDGE-LOADING SOUND MOTION PICTURE PROJECTOR, AND AN OBSERVATION BOOTH UTILIZING A ONE-WAY MIRROR.…

  1. TEACHING DEAF CHILDREN TO TALK.

    ERIC Educational Resources Information Center

    EWING, ALEXANDER; EWING, ETHEL C.

    DESIGNED AS A TEXT FOR AUDIOLOGISTS AND TEACHERS OF HEARING IMPAIRED CHILDREN, THIS BOOK PRESENTS BASIC INFORMATION ABOUT SPOKEN LANGUAGE, HEARING, AND LIPREADING. METHODS AND RESULTS OF EVALUATING SPOKEN LANGUAGE OF AURALLY HANDICAPPED CHILDREN WITHOUT USING READING OR WRITING ARE REPORTED. VARIOUS TYPES OF INDIVIDUAL AND GROUP HEARING AIDS ARE…

  2. Services for the Deaf: Check This Out.

    ERIC Educational Resources Information Center

    Lake County Public Library, Merrillville, IN.

    The Lake County (Indiana) Public Library provides a guide to materials for the deaf. The fiction list is annotated. Unannotated lists include materials on the deaf child, books on manual communication, resource books (law, education, directories, guides), poetry, lipreading materials, general information on deafness and the deaf, biographies,…

  3. Speechreading Instruction for Adults: Issues and Approaches.

    ERIC Educational Resources Information Center

    Cherry, Rochelle; Rubinstein, Adrienne

    1988-01-01

    Implementation of a speechreading program for hearing-impaired adults should consider: program clients; presentation of concepts such as differences between lipreading and speechreading and factors affecting speechreading; program structure including client needs assessments, individual/group therapy, length and termination of treatment, etc.;…

  4. Speech imagery recalibrates speech-perception boundaries.

    PubMed

    Scott, Mark

    2016-07-01

    The perceptual boundaries between speech sounds are malleable and can shift after repeated exposure to contextual information. This shift is known as recalibration. To date, the known inducers of recalibration are lexical (including phonotactic) information, lip-read information and reading. The experiments reported here are a proof-of-effect demonstration that speech imagery can also induce recalibration. PMID:27068050

  5. "The Business of Life": Educating Catholic Deaf Children in Late Nineteenth-Century England

    ERIC Educational Resources Information Center

    Mangion, Carmen M.

    2012-01-01

    Much of the debates in late nineteenth-century Britain regarding the education of deaf children revolved around communication. For many Victorians, sign language was unacceptable; many proponents of oralism attempted to "normalise" the hearing impaired by replacing deaf methods of communication with spoken language and lipreading. While debates on…

  6. "All Methods--and Wedded to None": The Deaf Education Methods Debate and Progressive Educational Reform in Toronto, Canada, 1922-1945

    ERIC Educational Resources Information Center

    Ellis, Jason A.

    2014-01-01

    This article is about the deaf education methods debate in the public schools of Toronto, Canada. The author demonstrates how pure oralism (lip-reading and speech instruction to the complete exclusion of sign language) and day school classes for deaf schoolchildren were introduced as a progressive school reform in 1922. Plans for further oralist…

  7. Tones for Profoundly Deaf Tone-Language Speakers.

    ERIC Educational Resources Information Center

    Ching, Teresa

    A study assessed the practical use of the simplified speech pattern approach to teaching lipreading in a tone language by comparing performance using an acoustic hearing-aid and a Sivo-aid in a tone labelling task. After initial assessment, subjects were given training to enhance perception of lexically contrastive tones, then post-tested. The…

  8. The What? When? and How? of Teaching Language to Deaf Children - Preschool and Primary Grades.

    ERIC Educational Resources Information Center

    Hogan, Sister James Lorene

    Three levels of work in language development for preschool and primary age deaf children are presented, along with suggested daily schedules and yearly programs. Skills covered are speech, lipreading, auditory training, and language. Instructions are given for teaching activities in the areas of the various parts of speech and types of sentences.…

  9. Comparison of benefit from vibrotactile aid and cochlear implant for postlinguistically deaf adults.

    PubMed

    Skinner, M W; Binzer, S M; Fredrickson, J M; Smith, P G; Holden, T A; Holden, L K; Juelich, M F; Turner, B A

    1988-10-01

    Four postlinguistically deaf adults were evaluated presurgically with a one- or two-channel vibrotactile aid and postsurgically with a multichannel, multielectrode, intracochlear implant. Although the vibrotactile aid provided awareness of sound and enhanced flow of conversation, benefit to lipreading was small on videotaped tests and speech tracking. Scores on recorded, sound-only speech tests were not significantly above chance except in discrimination of noise from voice. With the cochlear implant, benefit to lipreading was significantly greater than with the vibrotactile aid, and scores on sound-only tests were significantly above chance. Communication was markedly better with the implant than with the vibrotactile aid. In counseling those who get no benefit from a hearing aid, the results of this study provide data on the amount of benefit one- or two-channel vibrotactile aids provide postlinguistically deaf adults who are subsequently implanted. PMID:3172956

  10. Bimodal bilingualism as multisensory training?: Evidence for improved audiovisual speech perception after sign language exposure.

    PubMed

    Williams, Joshua T; Darcy, Isabelle; Newman, Sharlene D

    2016-02-15

    The aim of the present study was to characterize effects of learning a sign language on the processing of a spoken language. Specifically, audiovisual phoneme comprehension was assessed before and after 13 weeks of sign language exposure. L2 ASL learners performed this task in the fMRI scanner. Results indicated that L2 American Sign Language (ASL) learners' behavioral classification of the speech sounds improved with time compared to hearing nonsigners. Results indicated increased activation in the supramarginal gyrus (SMG) after sign language exposure, which suggests concomitant increased phonological processing of speech. A multiple regression analysis indicated that learner's rating on co-sign speech use and lipreading ability was correlated with SMG activation. This pattern of results indicates that the increased use of mouthing and possibly lipreading during sign language acquisition may concurrently improve audiovisual speech processing in budding hearing bimodal bilinguals. PMID:26740404

  11. A multiple-channel cochlear implant and wearable speech-processor: an audiological evaluation.

    PubMed

    Tong, Y C; Clark, G M; Dowell, R C; Martin, L F; Seligman, P M; Patrick, J F

    1981-01-01

    Stand audiological tests were administered to a totally deaf multiple-channel cochlear implant patient with a wearable speech-processor in a monitored sound field under the following conditions: a wearable unit activated alone (WA), lipreading with the wearable unit off (LA), and wearable unit activated in combination with lipreading (WL). Thresholds obtained for narrow-band noise signals indicated that the wearable unit allowed the patient to detect a variety of sounds at different frequencies. The results obtained in closed-set word tests and open-set word and sentence tests showed significant improvements in word and sentence scores from LA to WL. In the open-set (C.I.D.) sentence test, the patient scored 22% for LA and 76% for WL. The WL score of 76% correlates with a satisfactory performance in understanding connected speech. The patient also scored 30% correct in a test involving the recognition of environmental sounds. PMID:6895683

  12. A multiple-channel cochlear implant: an evaluation using open-set CID sentences.

    PubMed

    Clark, G M; Tong, Y C; Martin, L F

    1981-04-01

    A multiple-channel cochlear implant and speech processor have been used in two postlingually deaf adult patients with a total hearing loss, to enable them to perceive varying degrees of running speech. The results have veen confirmed with open-set CID everyday sentence tests. Using the implant alone, the patients obtained 8% and 14% scores with prerecorded material, and 34% and 36% scores for "live" presentations. This was equivalent to the perception of 35% of connected discourse. When the implant was used in conjuction with lipreading, improvements of 188% and 386% were obtained over lipreading alone, and the scores were 68% and 98% which were equivalent to the perception of 60% and 95% of connected discourse. PMID:7219008

  13. Initial findings with a wearable multichannel vibrotactile aid.

    PubMed

    Osberger, M J; Robbins, A M; Todd, S L; Brown, C J

    1991-01-01

    Data are presented on the speech perception performance of two profoundly hearing-impaired subjects while using a two-channel vibrotactile aid (Tactaid II) or a new, seven-channel instrument. Both subjects, one a profoundly hearing-impaired teenager and one a postlingually deaf adult, are experienced users of tactile aids. The data suggest better recognition of speech features, words, environmental sounds, and enhancement of lipreading skills with the new multichannel instrument than with the two-channel device. PMID:2069179

  14. How is the McGurk effect modulated by Cued Speech in deaf and hearing adults?

    PubMed Central

    Bayard, Clémence; Colin, Cécile; Leybaert, Jacqueline

    2014-01-01

    Speech perception for both hearing and deaf people involves an integrative process between auditory and lip-reading information. In order to disambiguate information from lips, manual cues from Cued Speech may be added. Cued Speech (CS) is a system of manual aids developed to help deaf people to clearly and completely understand speech visually (Cornett, 1967). Within this system, both labial and manual information, as lone input sources, remain ambiguous. Perceivers, therefore, have to combine both types of information in order to get one coherent percept. In this study, we examined how audio-visual (AV) integration is affected by the presence of manual cues and on which form of information (auditory, labial or manual) the CS receptors primarily rely. To address this issue, we designed a unique experiment that implemented the use of AV McGurk stimuli (audio /pa/ and lip-reading /ka/) which were produced with or without manual cues. The manual cue was congruent with either auditory information, lip information or the expected fusion. Participants were asked to repeat the perceived syllable aloud. Their responses were then classified into four categories: audio (when the response was /pa/), lip-reading (when the response was /ka/), fusion (when the response was /ta/) and other (when the response was something other than /pa/, /ka/ or /ta/). Data were collected from hearing impaired individuals who were experts in CS (all of which had either cochlear implants or binaural hearing aids; N = 8), hearing-individuals who were experts in CS (N = 14) and hearing-individuals who were completely naïve of CS (N = 15). Results confirmed that, like hearing-people, deaf people can merge auditory and lip-reading information into a single unified percept. Without manual cues, McGurk stimuli induced the same percentage of fusion responses in both groups. Results also suggest that manual cues can modify the AV integration and that their impact differs between hearing and deaf people

  15. Visual abilities are important for auditory-only speech recognition: evidence from autism spectrum disorder.

    PubMed

    Schelinski, Stefanie; Riedel, Philipp; von Kriegstein, Katharina

    2014-12-01

    In auditory-only conditions, for example when we listen to someone on the phone, it is essential to fast and accurately recognize what is said (speech recognition). Previous studies have shown that speech recognition performance in auditory-only conditions is better if the speaker is known not only by voice, but also by face. Here, we tested the hypothesis that such an improvement in auditory-only speech recognition depends on the ability to lip-read. To test this we recruited a group of adults with autism spectrum disorder (ASD), a condition associated with difficulties in lip-reading, and typically developed controls. All participants were trained to identify six speakers by name and voice. Three speakers were learned by a video showing their face and three others were learned in a matched control condition without face. After training, participants performed an auditory-only speech recognition test that consisted of sentences spoken by the trained speakers. As a control condition, the test also included speaker identity recognition on the same auditory material. The results showed that, in the control group, performance in speech recognition was improved for speakers known by face in comparison to speakers learned in the matched control condition without face. The ASD group lacked such a performance benefit. For the ASD group auditory-only speech recognition was even worse for speakers known by face compared to speakers not known by face. In speaker identity recognition, the ASD group performed worse than the control group independent of whether the speakers were learned with or without face. Two additional visual experiments showed that the ASD group performed worse in lip-reading whereas face identity recognition was within the normal range. The findings support the view that auditory-only communication involves specific visual mechanisms. Further, they indicate that in ASD, speaker-specific dynamic visual information is not available to optimize auditory

  16. Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training

    PubMed Central

    Eberhardt, Silvio P.; Auer Jr., Edward T.; Bernstein, Lynne E.

    2014-01-01

    In a series of studies we have been investigating how multisensory training affects unisensory perceptual learning with speech stimuli. Previously, we reported that audiovisual (AV) training with speech stimuli can promote auditory-only (AO) perceptual learning in normal-hearing adults but can impede learning in congenitally deaf adults with late-acquired cochlear implants. Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training. In Experiment 1, visual-only (VO) training on paired associations between CVCVC nonsense word videos and nonsense pictures demonstrated that VO words could be learned to a high level of accuracy even by poor lipreaders. In Experiment 2, visual-auditory (VA) training in the same paradigm but with the addition of synchronous vocoded acoustic speech impeded VO learning of the stimuli in the paired-associates paradigm. In Experiment 3, the vocoded AO stimuli were shown to be less informative than the VO speech. Experiment 4 combined vibrotactile speech stimuli with the visual stimuli during training. Vibrotactile stimuli were shown to promote visual perceptual learning. In Experiment 5, no-training controls were used to show that training with visual speech carried over to consonant identification of untrained CVCVC stimuli but not to lipreading words in sentences. Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training. Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee’s primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee’s lower rank perceptual pathway will promote learning by a higher-rank pathway. The mechanisms supporting these principles are discussed in light of multisensory reverse hierarchy theory (RHT). PMID:25400566

  17. The Self-Advantage in Visual Speech Processing Enhances Audiovisual Speech Recognition in Noise

    PubMed Central

    Tye-Murray, Nancy; Spehar, Brent P.; Myerson, Joel; Hale, Sandra; Sommers, Mitchell S.

    2014-01-01

    Individuals lipread themselves more accurately than they lipread others when only the visual speech signal is available (Tye-Murray, Spehar, Myerson, Hale, & Sommers, 2013). This self-advantage for vision-only speech recognition is consistent with the common-coding hypothesis (Prinz, 1997), which posits (1) that observing an action activates the same motor plan representation as actually performing that action and (2) that observing one’s own actions activates motor plan representations more than the others’ actions because of greater congruity between percepts and corresponding motor plans. The present study extends this line of research to audiovisual speech recognition by examining whether there is a self-advantage when the visual signal is added to the auditory signal under poor listening conditions. Participants were assigned to sub-groups for round-robin testing in which each participant was paired with every member of their subgroup, including themselves, serving as both talker and listener/observer. On average, the benefit participants obtained from the visual signal when they were the talker was greater than when the talker was someone else and also was greater than the benefit others obtained from observing as well as listening to them. Moreover, the self-advantage in audiovisual speech recognition was significant after statistically controlling for individual differences in both participants’ ability to benefit from a visual speech signal and the extent to which their own visual speech signal benefited others. These findings are consistent with our previous finding of a self-advantage in lipreading and with the hypothesis of a common code for action perception and motor plan representation. PMID:25421408

  18. Tactile perception by the profoundly deaf. Speech and environmental sounds.

    PubMed

    Plant, G L

    1982-11-01

    Four subjects fitted with single-channel vibrotactile aids and provided with training in their use took part in a testing programme aimed at assessing their aided and unaided lipreading performance, their ability to detect segmental and suprasegmental features of speech, and the discrimination of common environmental sounds. The results showed that the vibrotactile aid provided very useful information as to speech and non-speech stimuli with the subjects performing best on those tasks where time/intensity cues provided sufficient information to enable identification. The implications of the study are discussed and a comparison made with those results reported for subjects using cochlear implants. PMID:6897619

  19. Results of a preliminary clinical trial on a multiple channel cochlear prosthesis.

    PubMed

    Dowell, R C; Martin, L F; Clark, G M; Brown, A M

    1985-01-01

    Speech discrimination testing was carried out under clinical trial conditions for eight profoundly postlingually deaf adults to assess the efficacy of a newly developed 22-channel cochlear prosthesis and speech processor. Three months postoperatively, these patients showed significantly better results with the cochlear prosthesis than for preoperative testing with a conventional hearing aid or vibrotactile aid (following a 6-month trial with the aid) on each of a series of tests from the Minimal Auditory Capabilities battery. Assessment of lipreading enhancement using standard speech tests, consonant recognition studies, and speech tracking showed significant improvements for each patient when using the cochlear prosthesis. Six patients showed a significant amount of open set speech discrimination without lipreading at levels which have not been reported for single electrode cochlear prostheses. The two patients who performed poorly on these tests both had restricted multiple channel systems due to their disease, one patient being restricted to virtually a single channel system and the other to only ten of the 22 electrodes. These results indicate that this multiple channel cochlear prosthesis has potential as a treatment for profound postlingual deafness over a wide range of etiologies and ages. PMID:3160295

  20. Effects of aging on audio-visual speech integration.

    PubMed

    Huyse, Aurélie; Leybaert, Jacqueline; Berthommier, Frédéric

    2014-10-01

    This study investigated the impact of aging on audio-visual speech integration. A syllable identification task was presented in auditory-only, visual-only, and audio-visual congruent and incongruent conditions. Visual cues were either degraded or unmodified. Stimuli were embedded in stationary noise alternating with modulated noise. Fifteen young adults and 15 older adults participated in this study. Results showed that older adults had preserved lipreading abilities when the visual input was clear but not when it was degraded. The impact of aging on audio-visual integration also depended on the quality of the visual cues. In the visual clear condition, the audio-visual gain was similar in both groups and analyses in the framework of the fuzzy-logical model of perception confirmed that older adults did not differ from younger adults in their audio-visual integration abilities. In the visual reduction condition, the audio-visual gain was reduced in the older group, but only when the noise was stationary, suggesting that older participants could compensate for the loss of lipreading abilities by using the auditory information available in the valleys of the noise. The fuzzy-logical model of perception confirmed the significant impact of aging on audio-visual integration by showing an increased weight of audition in the older group. PMID:25324091

  1. Video analysis using spatiotemporal descriptor and kernel extreme learning machine for lip reading

    NASA Astrophysics Data System (ADS)

    Lu, Longbin; Zhang, Xinman; Xu, Xuebin; Shang, Dongpeng

    2015-09-01

    Lip-reading techniques have shown bright prospects for speech recognition under noisy environments and for hearing-impaired listeners. We aim to solve two important issues regarding lip reading: (1) how to extract discriminative lip motion features and (2) how to establish a classifier that can provide promising recognition accuracy for lip reading. For the first issue, a projection local spatiotemporal descriptor, which considers the lip appearance and motion information at the same time, is utilized to provide an efficient representation of a video sequence. For the second issue, a kernel extreme learning machine (KELM) based on the single-hidden-layer feedforward neural network is presented to distinguish all kinds of utterances. In general, this method has fast learning speed and great robustness to nonlinear data. Furthermore, quantum-behaved particle swarm optimization with binary encoding is introduced to select the appropriate feature subset and parameters for KELM training. Experiments conducted on the AVLetters and OuluVS databases show that the proposed lip-reading method achieves a superior recognition accuracy compared with two previous methods.

  2. Continuing evaluation of the Queen's University tactile vocoder. I: Identification of open set words.

    PubMed

    Brooks, P L; Frost, B J; Mason, J L; Gibson, D M

    1986-01-01

    Identification of open set words, by an experienced normal hearing subject using the tactile vocoder developed at Queen's University, was examined. The tactile vocoder filters and processes the acoustic waveform into 16 filter channels, each of which controls a vibrator on the skin surface. After acquiring a 250-word vocabulary through the tactile vocoder, the subject was presented with three sets of 1000 different open set words in three reception conditions. The percentages of words correctly identified in the tactile vocoder (TV), lipreading (L), and lipreading plus tactile vocoder (L + TV) conditions were 8.8, 39.4, and 68.7 percent respectively. Phonemic analysis of stimulus/response pairs revealed that 36.7, 64.9 and 85.5 percent of the phonemes were correctly identified in TV, L, and L + TV conditions, respectively, indicating that incorrect-response words often contained many correct phonemes. Also, syllabic stress of stimulus and response words was identical 88 percent of the time in the TV condition. Important information about speech was transmitted through the tactile vocoder. PMID:3958993

  3. Phonetic matching of auditory and visual speech develops during childhood: Evidence from sine-wave speech

    PubMed Central

    Baart, Martijn; Bortfeld, Heather; Vroomen, Jean

    2014-01-01

    The correspondence between auditory speech and lip-read information can be detected based on a combination of temporal and phonetic cross-modal cues. Here, we determined the point in developmental time at which children start to effectively use phonetic information to match a speech sound with one of two articulating faces. We presented 4- to 11-year old children (N = 77) with three-syllabic sine-wave speech replicas of two pseudo-words that were perceived as non-speech and asked them to match the sounds with the corresponding lip-read video. At first, children had no phonetic knowledge about the sounds and matching was thus based on temporal cues that are fully retained in sine-wave speech. Next, we trained all children to perceive the phonetic identity of the sine-wave speech and repeated the audiovisual matching task. Only at around 6.5 years of age, the benefit of having phonetic knowledge about the stimuli became apparent, thus indicating that AV-matching based on phonetic cues presumably develops more slowly than AV matching based on temporal cues. PMID:25258018

  4. The role of visual speech cues in reducing energetic and informational masking

    NASA Astrophysics Data System (ADS)

    Helfer, Karen S.; Freyman, Richard L.

    2005-02-01

    Two experiments compared the effect of supplying visual speech information (e.g., lipreading cues) on the ability to hear one female talker's voice in the presence of steady-state noise or a masking complex consisting of two other female voices. In the first experiment intelligibility of sentences was measured in the presence of the two types of maskers with and without perceived spatial separation of target and masker. The second study tested detection of sentences in the same experimental conditions. Results showed that visual cues provided more benefit for both recognition and detection of speech when the masker consisted of other voices (versus steady-state noise). Moreover, visual cues provided greater benefit when the target speech and masker were spatially coincident versus when they appeared to arise from different spatial locations. The data obtained here are consistent with the hypothesis that lipreading cues help to segregate a target voice from competing voices, in addition to the established benefit of supplementing masked phonetic information. .

  5. Phi-square Lexical Competition Database (Phi-Lex): an online tool for quantifying auditory and visual lexical competition.

    PubMed

    Strand, Julia F

    2014-03-01

    A widely agreed-upon feature of spoken word recognition is that multiple lexical candidates in memory are simultaneously activated in parallel when a listener hears a word, and that those candidates compete for recognition (Luce, Goldinger, Auer, & Vitevitch, Perception 62:615-625, 2000; Luce & Pisoni, Ear and Hearing 19:1-36, 1998; McClelland & Elman, Cognitive Psychology 18:1-86, 1986). Because the presence of those competitors influences word recognition, much research has sought to quantify the processes of lexical competition. Metrics that quantify lexical competition continuously are more effective predictors of auditory and visual (lipread) spoken word recognition than are the categorical metrics traditionally used (Feld & Sommers, Speech Communication 53:220-228, 2011; Strand & Sommers, Journal of the Acoustical Society of America 130:1663-1672, 2011). A limitation of the continuous metrics is that they are somewhat computationally cumbersome and require access to existing speech databases. This article describes the Phi-square Lexical Competition Database (Phi-Lex): an online, searchable database that provides access to multiple metrics of auditory and visual (lipread) lexical competition for English words, available at www.juliastrand.com/phi-lex . PMID:23754576

  6. New developments in speech pattern element hearing aids for the profoundly deaf.

    PubMed

    Faulkner, A; Walliker, J R; Howard, I S; Ball, V; Fourcin, A J

    1993-01-01

    Two new developments in speech pattern processing hearing aids will be described. The first development is the use of compound speech pattern coding. Speech information which is invisible to the lipreader was encoded in terms of three acoustic speech factors; the voice fundamental frequency pattern, coded as a sinusoid, the presence of aperiodic excitation, coded as a low-frequency noise, and the wide-band amplitude envelope, coded by amplitude modulation of the sinusoid and noise signals. Each element of the compound stimulus was individually matched in frequency and intensity to the listener's receptive range. Audio-visual speech receptive assessments in five profoundly hearing-impaired listeners were performed to examine the contributions of adding voiceless and amplitude information to the voice fundamental frequency pattern, and to compare these codings to amplified speech. In both consonant recognition and connected discourse tracking (CDT), all five subjects showed an advantage from the addition of amplitude information to the fundamental frequency pattern. In consonant identification, all five subjects showed further improvements in performance when voiceless speech excitation was additionally encoded together with amplitude information, but this effect was not found in CDT. The addition of voiceless information to voice fundamental frequency information did not improve performance in the absence of amplitude information. Three of the subjects performed significantly better in at least one of the compound speech pattern conditions than with amplified speech, while the other two performed similarly with amplified speech and the best compound speech pattern condition. The three speech pattern elements encoded here may represent a near-optimal basis for an acoustic aid to lipreading for this group of listeners. The second development is the use of a trained multi-layer-perceptron (MLP) pattern classification algorithm as the basis for a robust real-time voice

  7. Vibrotactile stimulation: case study with a profoundly deaf child.

    PubMed

    Geers, A E

    1986-01-01

    This case study reports results obtained from a young, profoundly deaf child, M, who was fitted with a single-channel vibrotactile device, the Tactaid I, at 29 months of age. Her progress in speech and language development was evaluated over a 14-month period. During this period, M learned to understand 101 words through lipreading and the Tactaid I, and to produce consistent approximations of 90 words. Her scores on language tests with hearing-impaired norms progressed from below average to above average for her age. M's scores on language tests with hearing norms also reflected significant progress, although she did not achieve normal language development. These results indicate that a single-channel vibrotactile aid may facilitate the acquisition of spoken language in a profoundly deaf child who is unable to benefit from a conventional hearing aid. PMID:3958992

  8. Encephalitis herpes simplex: aural rehabilitation following bilateral deafness.

    PubMed

    Montano, J J; Melley, C C; Karam, D B

    1983-10-01

    Aural rehabilitation is a critical and often neglected aspect of a hearing-impaired patient's total rehabilitation. This case description illustrates the need for implementation of aural rehabilitation services. A 59-year-old woman exhibited bilateral profound sensorineural hearing loss following the onset of encephalitis herpes simplex. Auditory amplification attempts were unsuccessful. Aural rehabilitation was initiated immediately, and she was seen for lipreading and vibrotactile stimulation training. Goals progressed from identification of single words within a category to phonemic recognition. Vibrotactile stimulation was used to facilitate environmental awareness. Therapy goals reflected the patient's increased motivation to communicate within her environment. This patient's communicative status is viewed on a continuum: from success in individual treatment goals, extending to successful communication within the structure of the entire rehabilitation setting, and finally to functional communication within her home environment. PMID:6625883

  9. Speech Analysis Based On Image Information from Lip Movement

    NASA Astrophysics Data System (ADS)

    Talha, Kamil S.; Wan, Khairunizam; Za'ba, S. K.; Mohamad Razlan, Zuradzman; B, Shahriman A.

    2013-12-01

    Deaf and hard of hearing people often have problems being able to understand and lip read other people. Usually deaf and hard of hearing people feel left out of conversation and sometimes they are actually ignored by other people. There are a variety of ways hearing-impaired person can communicate and gain accsss to the information. Communication support includes both technical and human aids. Human aids include interpreters, lip-readers and note-takers. Interpreters translate the Sign Language and must therefore be qualified. In this paper, vision system is used to track movements of the lip. In the experiment, the proposed system succesfully can differentiate 11 type of phonemes and then classified it to the respective viseme group. By using the proposed system the hearing-impaired persons could practise pronaunciations by themselve without support from the instructor.

  10. Surgical treatment and rehabilitation of prelingually and perilingually deafened children and adults with the nucleus multichannel cochlear implant.

    PubMed

    García, J M; Barón de Otero, C; García, J; Peñaranda, A; Niño, C; Campos, S

    1994-03-01

    We began our program in September 1992, using the Nucleus 22 Channel Cochlear Implant. To date, we have operated on four patients, one child with congenital hearing loss, two prelinguistically deaf adults and one perilingually deaf adult. Our results have shown a significant increase in auditory and speech reception and perception skills in the child. The perilingually deaf adult is able to understand speech in open set speech discrimination testing and, although we do not expect open set speech discrimination in the prelinguistically deaf adults, to date their results have been satisfactory. The two prelingually deaf adults are in an audiological rehabilitation program. Their response in prosodic aspects of speech and lipreading ability with sound have improved significantly. The only surgical complication was an infection of the flap in the child, but it was treated satisfactorily with i.v. penicillin. PMID:8205978

  11. Visual signal processing, speechreading, and related issues

    NASA Astrophysics Data System (ADS)

    Levitt, Harry

    2003-06-01

    People with hearing loss make use of visual speech cues to supplement the impoverished speech signal. This process, known as speechreading (or lipreading) can be very effective because of the complementary nature of auditory and visual speech cues. Despite the importance of visual speech cues (for both normal-hearing and hearing-impaired people) research on the visual characteristics of speech has lagged behind research on the acoustic characteristics of speech. The field of acoustic phonetics benefited substantially from the availability of powerful techniques for acoustic signal analysis. The substantial, recent advances in optical signal processing have opened up new vistas for visual speech analysis analogous to the way technological innovation revolutionized the field of acoustic phonetics. This paper describes several experiments in the emerging field of optic phonetics.

  12. Phonetic recalibration of speech by text.

    PubMed

    Keetels, Mirjam; Schakel, Lemmy; Bonte, Milene; Vroomen, Jean

    2016-04-01

    Listeners adjust their phonetic categories to cope with variations in the speech signal (phonetic recalibration). Previous studies have shown that lipread speech (and word knowledge) can adjust the perception of ambiguous speech and can induce phonetic adjustments (Bertelson, Vroomen, & de Gelder in Psychological Science, 14(6), 592-597, 2003; Norris, McQueen, & Cutler in Cognitive Psychology, 47(2), 204-238, 2003). We examined whether orthographic information (text) also can induce phonetic recalibration. Experiment 1 showed that after exposure to ambiguous speech sounds halfway between /b/ and /d/ that were combined with text (b or d) participants were more likely to categorize auditory-only test sounds in accordance with the exposed letters. Experiment 2 replicated this effect with a very short exposure phase. These results show that listeners adjust their phonetic boundaries in accordance with disambiguating orthographic information and that these adjustments show a rapid build-up. PMID:26704562

  13. Musicians have enhanced subcortical auditory and audiovisual processing of speech and music

    PubMed Central

    Musacchia, Gabriella; Sams, Mikko; Skoe, Erika; Kraus, Nina

    2007-01-01

    Musical training is known to modify cortical organization. Here, we show that such modifications extend to subcortical sensory structures and generalize to processing of speech. Musicians had earlier and larger brainstem responses than nonmusician controls to both speech and music stimuli presented in auditory and audiovisual conditions, evident as early as 10 ms after acoustic onset. Phase-locking to stimulus periodicity, which likely underlies perception of pitch, was enhanced in musicians and strongly correlated with length of musical practice. In addition, viewing videos of speech (lip-reading) and music (instrument being played) enhanced temporal and frequency encoding in the auditory brainstem, particularly in musicians. These findings demonstrate practice-related changes in the early sensory encoding of auditory and audiovisual information. PMID:17898180

  14. Learning one-to-many mapping functions for audio-visual integrated perception

    NASA Astrophysics Data System (ADS)

    Lim, Jung-Hui; Oh, Do-Kwan; Lee, Soo-Young

    2010-04-01

    In noisy environment the human speech perception utilizes visual lip-reading as well as audio phonetic classification. This audio-visual integration may be done by combining the two sensory features at the early stage. Also, the top-down attention may integrate the two modalities. For the sensory feature fusion we introduce mapping functions between the audio and visual manifolds. Especially, we present an algorithm to provide one-to-many mapping function for the videoto- audio mapping. The top-down attention is also presented to integrate both the sensory features and classification results of both modalities, which is able to explain McGurk effect. Each classifier is separately implemented by the Hidden-Markov Model (HMM), but the two classifiers are combined at the top level and interact by the top-down attention.

  15. PERVALE-S: a new cognitive task to assess deaf people’s ability to perceive basic and social emotions

    PubMed Central

    Mestre, José M.; Larrán, Cristina; Herrero, Joaquín; Guil, Rocío; de la Torre, Gabriel G.

    2015-01-01

    A poorly understood aspect of deaf people (DP) is how their emotional information is processed. Verbal ability is key to improve emotional knowledge in people. Nevertheless, DP are unable to distinguish intonation, intensity, and the rhythm of language due to lack of hearing. Some DP have acquired both lip-reading abilities and sign language, but others have developed only sign language. PERVALE-S was developed to assess the ability of DP to perceive both social and basic emotions. PERVALE-S presents different sets of visual images of a real deaf person expressing both basic and social emotions, according to the normative standard of emotional expressions in Spanish Sign Language. Emotional expression stimuli were presented at two different levels of intensity (1: low; and 2: high) because DP do not distinguish an object in the same way as hearing people (HP) do. Then, participants had to click on the more suitable emotional expression. PERVALE-S contains video instructions (given by a sign language interpreter) to improve DP’s understanding about how to use the software. DP had to watch the videos before answering the items. To test PERVALE-S, a sample of 56 individuals was recruited (18 signers, 8 lip-readers, and 30 HP). Participants also performed a personality test (High School Personality Questionnaire adapted) and a fluid intelligence (Gf) measure (RAPM). Moreover, all deaf participants were rated by four teachers for the deaf. Results: there were no significant differences between deaf and HP in performance in PERVALE-S. Confusion matrices revealed that embarrassment, envy, and jealousy were worse perceived. Age was just related to social-emotional tasks (but not in basic emotional tasks). Emotional perception ability was related mainly to warmth and consciousness, but negatively related to tension. Meanwhile, Gf was related to only social-emotional tasks. There were no gender differences. PMID:26300828

  16. Cross-cultural adaptation and validation of the Nijmegen Cochlear Implant Questionnaire into Italian.

    PubMed

    Ottaviani, F; Iacona, E; Sykopetrites, V; Schindler, A; Mozzanica, F

    2016-08-01

    The NCIQ is a quantifiable self-assessment health-related quality of life instrument specific for cochlear implant users. The aim of this study was to culturally adapt the NCIQ into Italian (I-NCIQ). A prospective instrument validation study was conducted. Cross-cultural adaptation and validation were accomplished. Cronbach α was used to test internal consistency in 51 CI users and in a control group composed by 38 post-lingual deaf adult on a waiting list for a CI. ICC test was used for test-retest reliability analysis. Kruskal-Wallis test with Mann-Whitney post hoc were used to compare the I-NCIQ scores in CI users before and after the cochlear implantation and in control patients. I-NCIQ scores obtained in CI users were compared with the results of Italian version of disyllabic testing without lip-reading and without masking. Good internal consistency and good test-retest reliability were found. I-NCIQ scores obtained in the 51 CI users after implantation were consistently higher than those obtained before implantation and in the control group. Moreover, no differences were found in the results of I-NCIQ obtained in the group of 51 CI users before implantation and in the group of control patients on post hoc Mann-Whitney analysis. Positive correlations between I-NCIQ scores and the results of disyllabic testing without lip-reading and without masking were found. The I-NCIQ is a reliable, valid, self-administered questionnaire for the measurement of QOL in CI users; its application is recommended. PMID:26324881

  17. Quantifying lip-read-induced suppression and facilitation of the auditory N1 and P2 reveals peak enhancements and delays.

    PubMed

    Baart, Martijn

    2016-09-01

    Lip-read speech suppresses and speeds up the auditory N1 and P2 peaks, but these effects are not always observed or reported. Here, the robustness of lip-read-induced N1/P2 suppression and facilitation in phonetically congruent audiovisual speech was assessed by analyzing peak values that were taken from published plots and individual data. To determine whether adhering to the additive model of AV integration (i.e., A+V ≠ AV, or AV-V ≠ A) is critical for correct characterization of lip-read-induced effects on the N1 and P2, auditory data was compared to AV and to AV-V. On average, the N1 and P2 were consistently suppressed and sped up by lip-read information, with no indication that AV integration effects were significantly modulated by whether or not V was subtracted from AV. To assess the possibility that variability in observed N1/P2 amplitudes and latencies may explain why N1/P2 suppression and facilitation are not always found, additional correlations between peak values and size of the AV integration effects were computed. These analyses showed that N1/P2 peak values correlated with the size of AV integration effects. However, it also became apparent that a portion of the AV integration effects was characterized by lip-read-induced peak enhancements and delays rather than suppressions and facilitations, which, for the individual data, seemed related to particularly small/early A-only peaks and large/late AV(-V) peaks. PMID:27295181

  18. Audibility and visual biasing in speech perception

    NASA Astrophysics Data System (ADS)

    Clement, Bart Richard

    Although speech perception has been considered a predominantly auditory phenomenon, large benefits from vision in degraded acoustic conditions suggest integration of audition and vision. More direct evidence of this comes from studies of audiovisual disparity that demonstrate vision can bias and even dominate perception (McGurk & MacDonald, 1976). It has been observed that hearing-impaired listeners demonstrate more visual biasing than normally hearing listeners (Walden et al., 1990). It is argued here that stimulus audibility must be equated across groups before true differences can be established. In the present investigation, effects of visual biasing on perception were examined as audibility was degraded for 12 young normally hearing listeners. Biasing was determined by quantifying the degree to which listener identification functions for a single synthetic auditory /ba-da-ga/ continuum changed across two conditions: (1)an auditory-only listening condition; and (2)an auditory-visual condition in which every item of the continuum was synchronized with visual articulations of the consonant-vowel (CV) tokens /ba/ and /ga/, as spoken by each of two talkers. Audibility was altered by presenting the conditions in quiet and in noise at each of three signal-to- noise (S/N) ratios. For the visual-/ba/ context, large effects of audibility were found. As audibility decreased, visual biasing increased. A large talker effect also was found, with one talker eliciting more biasing than the other. An independent lipreading measure demonstrated that this talker was more visually intelligible than the other. For the visual-/ga/ context, audibility and talker effects were less robust, possibly obscured by strong listener effects, which were characterized by marked differences in perceptual processing patterns among participants. Some demonstrated substantial biasing whereas others demonstrated little, indicating a strong reliance on audition even in severely degraded acoustic

  19. Automatic lip reading by using multimodal visual features

    NASA Astrophysics Data System (ADS)

    Takahashi, Shohei; Ohya, Jun

    2013-12-01

    Since long time ago, speech recognition has been researched, though it does not work well in noisy places such as in the car or in the train. In addition, people with hearing-impaired or difficulties in hearing cannot receive benefits from speech recognition. To recognize the speech automatically, visual information is also important. People understand speeches from not only audio information, but also visual information such as temporal changes in the lip shape. A vision based speech recognition method could work well in noisy places, and could be useful also for people with hearing disabilities. In this paper, we propose an automatic lip-reading method for recognizing the speech by using multimodal visual information without using any audio information such as speech recognition. First, the ASM (Active Shape Model) is used to track and detect the face and lip in a video sequence. Second, the shape, optical flow and spatial frequencies of the lip features are extracted from the lip detected by ASM. Next, the extracted multimodal features are ordered chronologically so that Support Vector Machine is performed in order to learn and classify the spoken words. Experiments for classifying several words show promising results of this proposed method.

  20. Speechreading development in deaf and hearing children: introducing a new Test of Child Speechreading (ToCS)

    PubMed Central

    Kyle, Fiona Elizabeth; Campbell, Ruth; Mohammed, Tara; Coleman, Mike; MacSweeney, Mairéad

    2016-01-01

    Purpose We describe the development of a new Test of Child Speechreading (ToCS) specifically designed for use with deaf and hearing children. Speechreading is a skill which is required for deaf children to access the language of the hearing community. ToCS is a deaf-friendly, computer-based test that measures child speechreading (silent lipreading) at three psycholinguistic levels: words, sentences and short stories. The aims of the study were to standardize ToCS with deaf and hearing children and investigate the effects of hearing status, age and linguistic complexity on speechreading ability. Method 86 severely and profoundly deaf and 91 hearing children aged between 5 and 14 years participated. The deaf children were from a range of language and communication backgrounds and their preferred mode of communication varied. Results: Speechreading skills significantly improved with age for both deaf and hearing children. There was no effect of hearing status on speechreading ability and deaf and hearing showed similar performance across all subtests on ToCS. Conclusions The Test of Child Speechreading (ToCS) is a valid and reliable assessment of speechreading ability in school-aged children that can be used to measure individual differences in performance in speechreading ability. PMID:23275416

  1. Auditory Midbrain Implant: Research and Development Towards a Second Clinical Trial

    PubMed Central

    Lim, Hubert H.; Lenarz, Thomas

    2015-01-01

    The cochlear implant is considered one of the most successful neural prostheses to date, which was made possible by visionaries who continued to develop the cochlear implant through multiple technological and clinical challenges. However, patients without a functional auditory nerve or implantable cochlea cannot benefit from a cochlear implant. The focus of the paper is to review the development and translation of a new type of central auditory prosthesis for this group of patients, which is known as the auditory midbrain implant (AMI) and is designed for electrical stimulation within the inferior colliculus. The rationale and results for the first AMI clinical study using a multi-site single-shank array will be presented initially. Although the AMI has achieved encouraging results in terms of safety and improvements in lip-reading capabilities and environmental awareness, it has not yet provided sufficient speech perception. Animal and human data will then be presented to show that a two-shank AMI array can potentially improve hearing performance by targeting specific neurons of the inferior colliculus. Modifications to the AMI array design, stimulation strategy, and surgical approach have been made that are expected to improve hearing performance in the patients implanted with a two-shank array in an upcoming clinical trial funded by the National Institutes of Health. Positive outcomes from this clinical trial will motivate new efforts and developments toward improving central auditory prostheses for those who cannot sufficiently benefit from cochlear implants. PMID:25613994

  2. Segmentation of human face using gradient-based approach

    NASA Astrophysics Data System (ADS)

    Baskan, Selin; Bulut, M. Mete; Atalay, Volkan

    2001-04-01

    This paper describes a method for automatic segmentation of facial features such as eyebrows, eyes, nose, mouth and ears in color images. This work is an initial step for wide range of applications based on feature-based approaches, such as face recognition, lip-reading, gender estimation, facial expression analysis, etc. Human face can be characterized by its skin color and nearly elliptical shape. For this purpose, face detection is performed using color and shape information. Uniform illumination is assumed. No restrictions on glasses, make-up, beard, etc. are imposed. Facial features are extracted using the vertically and horizontally oriented gradient projections. The gradient of a minimum with respect to its neighbor maxima gives the boundaries of a facial feature. Each facial feature has a different horizontal characteristic. These characteristics are derived by extensive experimentation with many face images. Using fuzzy set theory, the similarity between the candidate and the feature characteristic under consideration is calculated. Gradient-based method is accompanied by the anthropometrical information, for robustness. Ear detection is performed using contour-based shape descriptors. This method detects the facial features and circumscribes each facial feature with the smallest rectangle possible. AR database is used for testing. The developed method is also suitable for real-time systems.

  3. Language access and theory of mind reasoning: evidence from deaf children in bilingual and oralist environments.

    PubMed

    Meristo, Marek; Falkman, Kerstin W; Hjelmquist, Erland; Tedoldi, Mariantonia; Surian, Luca; Siegal, Michael

    2007-09-01

    This investigation examined whether access to sign language as a medium for instruction influences theory of mind (ToM) reasoning in deaf children with similar home language environments. Experiment 1 involved 97 deaf Italian children ages 4-12 years: 56 were from deaf families and had LIS (Italian Sign Language) as their native language, and 41 had acquired LIS as late signers following contact with signers outside their hearing families. Children receiving bimodal/bilingual instruction in LIS together with Sign-Supported and spoken Italian significantly outperformed children in oralist schools in which communication was in Italian and often relied on lipreading. Experiment 2 involved 61 deaf children in Estonia and Sweden ages 6-16 years. On a wide variety of ToM tasks, bilingually instructed native signers in Estonian Sign Language and spoken Estonian succeeded at a level similar to age-matched hearing children. They outperformed bilingually instructed late signers and native signers attending oralist schools. Particularly for native signers, access to sign language in a bilingual environment may facilitate conversational exchanges that promote the expression of ToM by enabling children to monitor others' mental states effectively. PMID:17723042

  4. Auditory midbrain implant: a review.

    PubMed

    Lim, Hubert H; Lenarz, Minoo; Lenarz, Thomas

    2009-09-01

    The auditory midbrain implant (AMI) is a new hearing prosthesis designed for stimulation of the inferior colliculus in deaf patients who cannot sufficiently benefit from cochlear implants. The authors have begun clinical trials in which five patients have been implanted with a single shank AMI array (20 electrodes). The goal of this review is to summarize the development and research that has led to the translation of the AMI from a concept into the first patients. This study presents the rationale and design concept for the AMI as well a summary of the animal safety and feasibility studies that were required for clinical approval. The authors also present the initial surgical, psychophysical, and speech results from the first three implanted patients. Overall, the results have been encouraging in terms of the safety and functionality of the implant. All patients obtain improvements in hearing capabilities on a daily basis. However, performance varies dramatically across patients depending on the implant location within the midbrain with the best performer still not able to achieve open set speech perception without lip-reading cues. Stimulation of the auditory midbrain provides a wide range of level, spectral, and temporal cues, all of which are important for speech understanding, but they do not appear to sufficiently fuse together to enable open set speech perception with the currently used stimulation strategies. Finally, several issues and hypotheses for why current patients obtain limited speech perception along with several feasible solutions for improving AMI implementation are presented. PMID:19762428

  5. Prototype to product—developing a commercially viable neural prosthesis

    NASA Astrophysics Data System (ADS)

    Seligman, Peter

    2009-12-01

    The Cochlear implant or 'Bionic ear' is a device that enables people who do not get sufficient benefit from a hearing aid to communicate with the hearing world. The Cochlear implant is not an amplifier, but a device that electrically stimulates the auditory nerve in a way that crudely mimics normal hearing, thus providing a hearing percept. Many recipients are able to understand running speech without the help of lipreading. Cochlear implants have reached a stage of maturity where there are now 170 000 recipients implanted worldwide. The commercial development of these devices has occurred over the last 30 years. This development has been multidisciplinary, including audiologists, engineers, both mechanical and electrical, histologists, materials scientists, physiologists, surgeons and speech pathologists. This paper will trace the development of the device we have today, from the engineering perspective. The special challenges of designing an active device that will work in the human body for a lifetime will be outlined. These challenges include biocompatibility, extreme reliability, safety, patient fitting and surgical issues. It is emphasized that the successful development of a neural prosthesis requires the partnership of academia and industry.

  6. Electrically evoked hearing perception by functional neurostimulation of the central auditory system.

    PubMed

    Tatagiba, M; Gharabaghi, A

    2005-01-01

    Perceptional benefits and potential risks of electrical stimulation of the central auditory system are constantly changing due to ongoing developments and technical modifications. Therefore, we would like to introduce current treatment protocols and strategies that might have an impact on functional results of auditory brainstem implants (ABI) in profoundly deaf patients. Patients with bilateral tumours as a result of neurofibromatosis type 2 with complete dysfunction of the eighth cranial nerves are the most frequent candidates for auditory brainstem implants. Worldwide, about 300 patients have already received an ABI through a translabyrinthine or suboccipital approach supported by multimodality electrophysiological monitoring. Patient selection is based on disease course, clinical signs, audiological, radiological and psycho-social criteria. The ABI provides the patients with access to auditory information such as environmental sound awareness together with distinct hearing cues in speech. In addition, this device markedly improves speech reception in combination with lip-reading. Nonetheless, there is only limited open-set speech understanding. Results of hearing function are correlated with electrode design, number of activated electrodes, speech processing strategies, duration of pre-existing deafness and extent of brainstem deformation. Functional neurostimulation of the central auditory system by a brainstem implant is a safe and beneficial procedure, which may considerably improve the quality of life in patients suffering from deafness due to bilateral retrocochlear lesions. The auditory outcome may be improved by a new generation of microelectrodes capable of penetrating the surface of the brainstem to access more directly the auditory neurons. PMID:15986735

  7. The Perceptual Characteristics of Voice-Hallucinations in Deaf People: Insights into the Nature of Subvocal Thought and Sensory Feedback Loops

    PubMed Central

    Atkinson, Joanna R.

    2006-01-01

    The study of voice-hallucinations in deaf individuals, who exploit the visuomotor rather than auditory modality for communication, provides rare insight into the relationship between sensory experience and how “voices” are perceived. Relatively little is known about the perceptual characteristics of voice-hallucinations in congenitally deaf people who use lip-reading or sign language as their preferred means of communication. The existing literature on hallucinations in deaf people is reviewed, alongside consideration of how such phenomena may fit into explanatory subvocal articulation hypotheses proposed for auditory verbal hallucinations in hearing people. It is suggested that a failure in subvocal articulation processes may account for voice-hallucinations in both hearing and deaf people but that the distinct way in which hallucinations are experienced may be due to differences in a sensory feedback component, which is influenced by both auditory deprivation and language modality. This article highlights how the study of deaf people may inform wider understanding of auditory verbal hallucinations and subvocal processes generally. PMID:16510696

  8. Separation of Audio-Visual Speech Sources: A New Approach Exploiting the Audio-Visual Coherence of Speech Stimuli

    NASA Astrophysics Data System (ADS)

    Sodoyer, David; Schwartz, Jean-Luc; Girin, Laurent; Klinkisch, Jacob; Jutten, Christian

    2002-12-01

    We present a new approach to the source separation problem in the case of multiple speech signals. The method is based on the use of automatic lipreading, the objective is to extract an acoustic speech signal from other acoustic signals by exploiting its coherence with the speaker's lip movements. We consider the case of an additive stationary mixture of decorrelated sources, with no further assumptions on independence or non-Gaussian character. Firstly, we present a theoretical framework showing that it is indeed possible to separate a source when some of its spectral characteristics are provided to the system. Then we address the case of audio-visual sources. We show how, if a statistical model of the joint probability of visual and spectral audio input is learnt to quantify the audio-visual coherence, separation can be achieved by maximizing this probability. Finally, we present a number of separation results on a corpus of vowel-plosive-vowel sequences uttered by a single speaker, embedded in a mixture of other voices. We show that separation can be quite good for mixtures of 2, 3, and 5 sources. These results, while very preliminary, are encouraging, and are discussed in respect to their potential complementarity with traditional pure audio separation or enhancement techniques.

  9. Auditory Midbrain Implant: A Review

    PubMed Central

    Lim, Hubert H.; Lenarz, Minoo; Lenarz, Thomas

    2009-01-01

    The auditory midbrain implant (AMI) is a new hearing prosthesis designed for stimulation of the inferior colliculus in deaf patients who cannot sufficiently benefit from cochlear implants. The authors have begun clinical trials in which five patients have been implanted with a single shank AMI array (20 electrodes). The goal of this review is to summarize the development and research that has led to the translation of the AMI from a concept into the first patients. This study presents the rationale and design concept for the AMI as well a summary of the animal safety and feasibility studies that were required for clinical approval. The authors also present the initial surgical, psychophysical, and speech results from the first three implanted patients. Overall, the results have been encouraging in terms of the safety and functionality of the implant. All patients obtain improvements in hearing capabilities on a daily basis. However, performance varies dramatically across patients depending on the implant location within the midbrain with the best performer still not able to achieve open set speech perception without lip-reading cues. Stimulation of the auditory midbrain provides a wide range of level, spectral, and temporal cues, all of which are important for speech understanding, but they do not appear to sufficiently fuse together to enable open set speech perception with the currently used stimulation strategies. Finally, several issues and hypotheses for why current patients obtain limited speech perception along with several feasible solutions for improving AMI implementation are presented. PMID:19762428

  10. Hearing Loss: Communicating With the Patient Who Is Deaf or Hard of Hearing.

    PubMed

    McKee, Michael M; Moreland, Christopher; Atcherson, Samuel R; Zazove, Philip

    2015-07-01

    Hearing loss impairs health care communication and adversely affects patient satisfaction, treatment adherence, and use of health services. Hearing loss is the third most common chronic health condition among older patients after hypertension and arthritis, but only 15% to 18% of older adults are screened for hearing loss during health maintenance examinations. Patients with hearing loss may be reluctant to disclose it because of fear of ageism, perceptions of disability, and vanity. Lipreading and note writing often are ineffective ways to communicate with deaf and hard of hearing (DHH) patients who use American Sign Language; use of medical sign language interpreters is preferred. A variety of strategies can improve the quality of health care communication for DHH patients, such as the physician facing the patient, listening attentively, and using visual tools. Physicians should learn what hearing loss means to the DHH patient. Deaf American Sign Language users may not perceive hearing loss as a disability but as a cultural identity. Patients' preferred communication strategies will vary. Relay services, electronic communication, and other telecommunications methods can be helpful, but family physicians and medical staff should learn from each DHH patient about which communication strategies will work best. PMID:26161525

  11. [Central auditory prosthesis].

    PubMed

    Lenarz, T; Lim, H; Joseph, G; Reuter, G; Lenarz, M

    2009-06-01

    Deaf patients with severe sensory hearing loss can benefit from a cochlear implant (CI), which stimulates the auditory nerve fibers. However, patients who do not have an intact auditory nerve cannot benefit from a CI. The majority of these patients are neurofibromatosis type 2 (NF2) patients who developed neural deafness due to growth or surgical removal of a bilateral acoustic neuroma. The only current solution is the auditory brainstem implant (ABI), which stimulates the surface of the cochlear nucleus in the brainstem. Although the ABI provides improvement in environmental awareness and lip-reading capabilities, only a few NF2 patients have achieved some limited open set speech perception. In the search for alternative procedures our research group in collaboration with Cochlear Ltd. (Australia) developed a human prototype auditory midbrain implant (AMI), which is designed to electrically stimulate the inferior colliculus (IC). The IC has the potential as a new target for an auditory prosthesis as it provides access to neural projections necessary for speech perception as well as a systematic map of spectral information. In this paper the present status of research and development in the field of central auditory prostheses is presented with respect to technology, surgical technique and hearing results as well as the background concepts of ABI and AMI. PMID:19517084

  12. Cochlear implantation in children and adults in Switzerland.

    PubMed

    Brand, Yves; Senn, Pascal; Kompis, Martin; Dillier, Norbert; Allum, John H J

    2014-01-01

    The cochlear implant (CI) is one of the most successful neural prostheses developed to date. It offers artificial hearing to individuals with profound sensorineural hearing loss and with insufficient benefit from conventional hearing aids. The first implants available some 30 years ago provided a limited sensation of sound. The benefit for users of these early systems was mostly a facilitation of lip-reading based communication rather than an understanding of speech. Considerable progress has been made since then. Modern, multichannel implant systems feature complex speech processing strategies, high stimulation rates and multiple sites of stimulation in the cochlea. Equipped with such a state-of-the-art system, the majority of recipients today can communicate orally without visual cues and can even use the telephone. The impact of CIs on deaf individuals and on the deaf community has thus been exceptional. To date, more than 300,000 patients worldwide have received CIs. In Switzerland, the first implantation was performed in 1977 and, as of 2012, over 2,000 systems have been implanted with a current rate of around 150 CIs per year. The primary purpose of this article is to provide a contemporary overview of cochlear implantation, emphasising the situation in Switzerland. PMID:24496729

  13. The deaf utilize phonological representations in visually presented verbal memory tasks.

    PubMed

    Okada, Rieko; Nakagawa, Jun; Takahashi, Muneyoshi; Kanaka, Noriko; Fukamauchi, Fumihiko; Watanabe, Katsumi; Namatame, Miki; Matsuda, Tetsuya

    2015-01-01

    The phonological abilities of congenitally deaf individuals are inferior to those of people who can hear. However, deaf individuals can acquire spoken languages by utilizing orthography and lip-reading. The present study used functional magnetic resonance imaging (fMRI) to show that deaf individuals utilize phonological representations via a mnemonic process. We compared the brain activation of deaf and hearing participants while they memorized serially visually presented Japanese kana letters (Kana), finger alphabets (Finger), and Arabic letters (Arabic). Hearing participants did not know which finger alphabets corresponded to which language sounds, whereas deaf participants did. All of the participants understood the correspondence between Kana and their language sounds. None of the participants knew the correspondence between Arabic and their language sounds, so this condition was used as a baseline. We found that the left superior temporal gyrus (STG) was activated by phonological representations in the deaf group when memorizing both Kana and Finger. Additionally, the brain areas associated with phonological representations for Finger in the deaf group were the same as the areas for Kana in the hearing group. Overall, despite the fact that they are superior in visual information processing, deaf individuals utilize phonological rather than visual representations in visually presented verbal memory. PMID:25498951

  14. From Mimicry to Language: A Neuroanatomically Based Evolutionary Model of the Emergence of Vocal Language

    PubMed Central

    Poliva, Oren

    2016-01-01

    The auditory cortex communicates with the frontal lobe via the middle temporal gyrus (auditory ventral stream; AVS) or the inferior parietal lobule (auditory dorsal stream; ADS). Whereas the AVS is ascribed only with sound recognition, the ADS is ascribed with sound localization, voice detection, prosodic perception/production, lip-speech integration, phoneme discrimination, articulation, repetition, phonological long-term memory and working memory. Previously, I interpreted the juxtaposition of sound localization, voice detection, audio-visual integration and prosodic analysis, as evidence that the behavioral precursor to human speech is the exchange of contact calls in non-human primates. Herein, I interpret the remaining ADS functions as evidence of additional stages in language evolution. According to this model, the role of the ADS in vocal control enabled early Homo (Hominans) to name objects using monosyllabic calls, and allowed children to learn their parents' calls by imitating their lip movements. Initially, the calls were forgotten quickly but gradually were remembered for longer periods. Once the representations of the calls became permanent, mimicry was limited to infancy, and older individuals encoded in the ADS a lexicon for the names of objects (phonological lexicon). Consequently, sound recognition in the AVS was sufficient for activating the phonological representations in the ADS and mimicry became independent of lip-reading. Later, by developing inhibitory connections between acoustic-syllabic representations in the AVS and phonological representations of subsequent syllables in the ADS, Hominans became capable of concatenating the monosyllabic calls for repeating polysyllabic words (i.e., developed working memory). Finally, due to strengthening of connections between phonological representations in the ADS, Hominans became capable of encoding several syllables as a single representation (chunking). Consequently, Hominans began vocalizing and

  15. How can audiovisual pathways enhance the temporal resolution of time-compressed speech in blind subjects?

    PubMed Central

    Hertrich, Ingo; Dietrich, Susanne; Ackermann, Hermann

    2013-01-01

    In blind people, the visual channel cannot assist face-to-face communication via lipreading or visual prosody. Nevertheless, the visual system may enhance the evaluation of auditory information due to its cross-links to (1) the auditory system, (2) supramodal representations, and (3) frontal action-related areas. Apart from feedback or top-down support of, for example, the processing of spatial or phonological representations, experimental data have shown that the visual system can impact auditory perception at more basic computational stages such as temporal signal resolution. For example, blind as compared to sighted subjects are more resistant against backward masking, and this ability appears to be associated with activity in visual cortex. Regarding the comprehension of continuous speech, blind subjects can learn to use accelerated text-to-speech systems for “reading” texts at ultra-fast speaking rates (>16 syllables/s), exceeding by far the normal range of 6 syllables/s. A functional magnetic resonance imaging study has shown that this ability, among other brain regions, significantly covaries with BOLD responses in bilateral pulvinar, right visual cortex, and left supplementary motor area. Furthermore, magnetoencephalographic measurements revealed a particular component in right occipital cortex phase-locked to the syllable onsets of accelerated speech. In sighted people, the “bottleneck” for understanding time-compressed speech seems related to higher demands for buffering phonological material and is, presumably, linked to frontal brain structures. On the other hand, the neurophysiological correlates of functions overcoming this bottleneck, seem to depend upon early visual cortex activity. The present Hypothesis and Theory paper outlines a model that aims at binding these data together, based on early cross-modal pathways that are already known from various audiovisual experiments on cross-modal adjustments during space, time, and object

  16. From Mimicry to Language: A Neuroanatomically Based Evolutionary Model of the Emergence of Vocal Language.

    PubMed

    Poliva, Oren

    2016-01-01

    The auditory cortex communicates with the frontal lobe via the middle temporal gyrus (auditory ventral stream; AVS) or the inferior parietal lobule (auditory dorsal stream; ADS). Whereas the AVS is ascribed only with sound recognition, the ADS is ascribed with sound localization, voice detection, prosodic perception/production, lip-speech integration, phoneme discrimination, articulation, repetition, phonological long-term memory and working memory. Previously, I interpreted the juxtaposition of sound localization, voice detection, audio-visual integration and prosodic analysis, as evidence that the behavioral precursor to human speech is the exchange of contact calls in non-human primates. Herein, I interpret the remaining ADS functions as evidence of additional stages in language evolution. According to this model, the role of the ADS in vocal control enabled early Homo (Hominans) to name objects using monosyllabic calls, and allowed children to learn their parents' calls by imitating their lip movements. Initially, the calls were forgotten quickly but gradually were remembered for longer periods. Once the representations of the calls became permanent, mimicry was limited to infancy, and older individuals encoded in the ADS a lexicon for the names of objects (phonological lexicon). Consequently, sound recognition in the AVS was sufficient for activating the phonological representations in the ADS and mimicry became independent of lip-reading. Later, by developing inhibitory connections between acoustic-syllabic representations in the AVS and phonological representations of subsequent syllables in the ADS, Hominans became capable of concatenating the monosyllabic calls for repeating polysyllabic words (i.e., developed working memory). Finally, due to strengthening of connections between phonological representations in the ADS, Hominans became capable of encoding several syllables as a single representation (chunking). Consequently, Hominans began vocalizing and

  17. Auditory midbrain implant: research and development towards a second clinical trial.

    PubMed

    Lim, Hubert H; Lenarz, Thomas

    2015-04-01

    The cochlear implant is considered one of the most successful neural prostheses to date, which was made possible by visionaries who continued to develop the cochlear implant through multiple technological and clinical challenges. However, patients without a functional auditory nerve or implantable cochlea cannot benefit from a cochlear implant. The focus of the paper is to review the development and translation of a new type of central auditory prosthesis for this group of patients that is known as the auditory midbrain implant (AMI) and is designed for electrical stimulation within the inferior colliculus. The rationale and results for the first AMI clinical study using a multi-site single-shank array will be presented initially. Although the AMI has achieved encouraging results in terms of safety and improvements in lip-reading capabilities and environmental awareness, it has not yet provided sufficient speech perception. Animal and human data will then be presented to show that a two-shank AMI array can potentially improve hearing performance by targeting specific neurons of the inferior colliculus. A new two-shank array, stimulation strategy, and surgical approach are planned for the AMI that are expected to improve hearing performance in the patients who will be implanted in an upcoming clinical trial funded by the National Institutes of Health. Positive outcomes from this clinical trial will motivate new efforts and developments toward improving central auditory prostheses for those who cannot sufficiently benefit from cochlear implants. This article is part of a Special Issue entitled . PMID:25613994

  18. [Extensive hearing loss and deafness in adults].

    PubMed

    Laszig, R

    1993-09-01

    Hearing and understanding are two related, yet different processes. Hearing is the perception of sound. It can be of enormous value to patients with severely impaired hearing, as it facilitates acoustic orientation. An understanding of speech, however, remains virtually impossible for most of these patients. Nevertheless, early habituation to their acoustic situation makes lip-reading much easier, thus enabling conversations to be possible in good listening environments. Severely impaired patients, however, are still not in a position to follow conversation in larger groups. Even with hearing aids and the deployment of the latest technology, sufficient help is not always given. Before making a decision on the use of these technical aids, the ENT specialist should discuss the needs of the particular individual with the hearing-aid specialist. Provided that residual hearing can be used to understand speech with the help of a hearing aid, intracochlear implantation of an electronic prosthesis is not indicated. A cochlear implant is indicated when there is a complete hearing loss on both sides. Such a profound loss means that sufficient understanding of speech is no longer possible, even with the assistance of the latest hearing aids. For most adults, deafness is a postlingual phenomenon. Adults who were born deaf or who lost their hearing in childhood tend to be unsuitable for cochlear implantation. Up to the age of six years, however, children born deaf can benefit considerably from a cochlear implant. Children who are provided with a cochlear implant shortly after becoming deaf also have a good chance of being capable of learning and understanding speech. PMID:8273025

  19. Visual speech discrimination and identification of natural and synthetic consonant stimuli

    PubMed Central

    Files, Benjamin T.; Tjan, Bosco S.; Jiang, Jintao; Bernstein, Lynne E.

    2015-01-01

    From phonetic features to connected discourse, every level of psycholinguistic structure including prosody can be perceived through viewing the talking face. Yet a longstanding notion in the literature is that visual speech perceptual categories comprise groups of phonemes (referred to as visemes), such as /p, b, m/ and /f, v/, whose internal structure is not informative to the visual speech perceiver. This conclusion has not to our knowledge been evaluated using a psychophysical discrimination paradigm. We hypothesized that perceivers can discriminate the phonemes within typical viseme groups, and that discrimination measured with d-prime (d’) and response latency is related to visual stimulus dissimilarities between consonant segments. In Experiment 1, participants performed speeded discrimination for pairs of consonant-vowel spoken nonsense syllables that were predicted to be same, near, or far in their perceptual distances, and that were presented as natural or synthesized video. Near pairs were within-viseme consonants. Natural within-viseme stimulus pairs were discriminated significantly above chance (except for /k/-/h/). Sensitivity (d’) increased and response times decreased with distance. Discrimination and identification were superior with natural stimuli, which comprised more phonetic information. We suggest that the notion of the viseme as a unitary perceptual category is incorrect. Experiment 2 probed the perceptual basis for visual speech discrimination by inverting the stimuli. Overall reductions in d’ with inverted stimuli but a persistent pattern of larger d’ for far than for near stimulus pairs are interpreted as evidence that visual speech is represented by both its motion and configural attributes. The methods and results of this investigation open up avenues for understanding the neural and perceptual bases for visual and audiovisual speech perception and for development of practical applications such as visual lipreading

  20. Brain networks engaged in audiovisual integration during speech perception revealed by persistent homology-based network filtration.

    PubMed

    Kim, Heejung; Hahm, Jarang; Lee, Hyekyoung; Kang, Eunjoo; Kang, Hyejin; Lee, Dong Soo

    2015-05-01

    The human brain naturally integrates audiovisual information to improve speech perception. However, in noisy environments, understanding speech is difficult and may require much effort. Although the brain network is supposed to be engaged in speech perception, it is unclear how speech-related brain regions are connected during natural bimodal audiovisual or unimodal speech perception with counterpart irrelevant noise. To investigate the topological changes of speech-related brain networks at all possible thresholds, we used a persistent homological framework through hierarchical clustering, such as single linkage distance, to analyze the connected component of the functional network during speech perception using functional magnetic resonance imaging. For speech perception, bimodal (audio-visual speech cue) or unimodal speech cues with counterpart irrelevant noise (auditory white-noise or visual gum-chewing) were delivered to 15 subjects. In terms of positive relationship, similar connected components were observed in bimodal and unimodal speech conditions during filtration. However, during speech perception by congruent audiovisual stimuli, the tighter couplings of left anterior temporal gyrus-anterior insula component and right premotor-visual components were observed than auditory or visual speech cue conditions, respectively. Interestingly, visual speech is perceived under white noise by tight negative coupling in the left inferior frontal region-right anterior cingulate, left anterior insula, and bilateral visual regions, including right middle temporal gyrus, right fusiform components. In conclusion, the speech brain network is tightly positively or negatively connected, and can reflect efficient or effortful processes during natural audiovisual integration or lip-reading, respectively, in speech perception. PMID:25495216

  1. Brain Networks Engaged in Audiovisual Integration During Speech Perception Revealed by Persistent Homology-Based Network Filtration

    PubMed Central

    Kim, Heejung; Hahm, Jarang; Lee, Hyekyoung; Kang, Eunjoo

    2015-01-01

    Abstract The human brain naturally integrates audiovisual information to improve speech perception. However, in noisy environments, understanding speech is difficult and may require much effort. Although the brain network is supposed to be engaged in speech perception, it is unclear how speech-related brain regions are connected during natural bimodal audiovisual or unimodal speech perception with counterpart irrelevant noise. To investigate the topological changes of speech-related brain networks at all possible thresholds, we used a persistent homological framework through hierarchical clustering, such as single linkage distance, to analyze the connected component of the functional network during speech perception using functional magnetic resonance imaging. For speech perception, bimodal (audio-visual speech cue) or unimodal speech cues with counterpart irrelevant noise (auditory white-noise or visual gum-chewing) were delivered to 15 subjects. In terms of positive relationship, similar connected components were observed in bimodal and unimodal speech conditions during filtration. However, during speech perception by congruent audiovisual stimuli, the tighter couplings of left anterior temporal gyrus-anterior insula component and right premotor-visual components were observed than auditory or visual speech cue conditions, respectively. Interestingly, visual speech is perceived under white noise by tight negative coupling in the left inferior frontal region–right anterior cingulate, left anterior insula, and bilateral visual regions, including right middle temporal gyrus, right fusiform components. In conclusion, the speech brain network is tightly positively or negatively connected, and can reflect efficient or effortful processes during natural audiovisual integration or lip-reading, respectively, in speech perception. PMID:25495216

  2. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput.

    PubMed

    Novich, Scott D; Eagleman, David M

    2015-10-01

    Touch receptors in the skin can relay various forms of abstract information, such as words (Braille), haptic feedback (cell phones, game controllers, feedback for prosthetic control), and basic visual information such as edges and shape (sensory substitution devices). The skin can support such applications with ease: They are all low bandwidth and do not require a fine temporal acuity. But what of high-throughput applications? We use sound-to-touch conversion as a motivating example, though others abound (e.g., vision, stock market data). In the past, vibrotactile hearing aids have demonstrated improvement in speech perceptions in the deaf. However, a sound-to-touch sensory substitution device that works with high efficacy and without the aid of lipreading has yet to be developed. Is this because skin simply does not have the capacity to effectively relay high-throughput streams such as sound? Or is this because the spatial and temporal properties of skin have not been leveraged to full advantage? Here, we begin to address these questions with two experiments. First, we seek to determine the best method of relaying information through the skin using an identification task on the lower back. We find that vibrotactile patterns encoding information in both space and time yield the best overall information transfer estimate. Patterns encoded in space and time or "intensity" (the coupled coding of vibration frequency and force) both far exceed performance of only spatially encoded patterns. Next, we determine the vibrotactile two-tacton resolution on the lower back-the distance necessary for resolving two vibrotactile patterns. We find that our vibratory motors conservatively require at least 6 cm of separation to resolve two independent tactile patterns (>80 % correct), regardless of stimulus type (e.g., spatiotemporal "sweeps" versus single vibratory pulses). Six centimeter is a greater distance than the inter-motor distances used in Experiment 1 (2.5 cm), which

  3. Home-based Early Intervention on Auditory and Speech Development in Mandarin-speaking Deaf Infants and Toddlers with Chronological Aged 7–24 Months

    PubMed Central

    Yang, Ying; Liu, Yue-Hui; Fu, Ming-Fu; Li, Chun-Lin; Wang, Li-Yan; Wang, Qi; Sun, Xi-Bin

    2015-01-01

    Background: Early auditory and speech development in home-based early intervention of infants and toddlers with hearing loss younger than 2 years are still spare in China. This study aimed to observe the development of auditory and speech in deaf infants and toddlers who were fitted with hearing aids and/or received cochlear implantation between the chronological ages of 7–24 months, and analyze the effect of chronological age and recovery time on auditory and speech development in the course of home-based early intervention. Methods: This longitudinal study included 55 hearing impaired children with severe and profound binaural deafness, who were divided into Group A (7–12 months), Group B (13–18 months) and Group C (19–24 months) based on the chronological age. Categories auditory performance (CAP) and speech intelligibility rating scale (SIR) were used to evaluate auditory and speech development at baseline and 3, 6, 9, 12, 18, and 24 months of habilitation. Descriptive statistics were used to describe demographic features and were analyzed by repeated measures analysis of variance. Results: With 24 months of hearing intervention, 78% of the patients were able to understand common phrases and conversation without lip-reading, 96% of the patients were intelligible to a listener. In three groups, children showed the rapid growth of trend features in each period of habilitation. CAP and SIR scores have developed rapidly within 24 months after fitted auxiliary device in Group A, which performed much better auditory and speech abilities than Group B (P < 0.05) and Group C (P < 0.05). Group B achieved better results than Group C, whereas no significant differences were observed between Group B and Group C (P > 0.05). Conclusions: The data suggested the early hearing intervention and home-based habilitation benefit auditory and speech development. Chronological age and recovery time may be major factors for aural verbal outcomes in hearing impaired children

  4. The multi-channel cochlear implant: past, present and future perspectives.

    PubMed

    Clark, Graeme

    2009-01-01

    Initial research demonstrated that only low frequencies could be mimicked with rate of electrical stimulation, and thus multi-channel rather than single-channel stimulation was required for the place coding of the mid-high speech frequencies. Place coding of mid-high frequencies was best achieved with electrodes inside the cochlea. Furthermore, correct biomechanical properties of a multiple electrode bundle were required for it to pass around the cochlear spiral to the speech frequency region. Biological studies showed too that intra-cochlear electrodes could be used with minimal trauma, safe electrical stimulus parameters, and methods to prevent inner ear infection and meningitis. The crucial discoveries for coding speech with electrical stimulation have been based on the discovery of: 1) the fact the brain processes frequency information along spatial and temporal channels, and 2) that the first patient experienced vowels when stimulating different electrodes that corresponded to the place of excitation for single formant vowels in people with normal hearing. The inaugural and subsequent speech processing strategies extracted frequencies of special importance for speech intelligibility, and transmitted the information along place coding channels. The voicing frequency and/or amplitude, was coded as temporal information across these spatial channels. As a result a great majority of severely-to-profoundly deaf people with previous hearing can not only communicate when electrical stimulation is combined with lipreading, but with electrical stimulation alone. In addition, the benefits of binaural hearing with bilateral cochlear implants or an implant in one ear and hearing aid in the other ear have been realized. Related psychophysical research has discovered the basic perceptual skills that process the complex patterns of brain excitation that underlie speech recognition both in the one ear as well as bilateral implants.In addition the development of the perceptual