Gaze on lips?

Gaze on lips? - Identification of optimal gaze patterns and support obtained from facial expressions in enhancing speechreading in children with and without impaired hearing is a research project carried out jointly in the Research Unit of Logopedics and in the CMVS. It addresses speechreading skills and intervention, focusing on the role of eye gaze patterns and utilization of facial expressions

Funders

Project information

Project duration

-

Project funder

Eudaimonia Institute, University of Oulu

Project coordinator

University of Oulu

Contact information

Project leader

  • Professor
    Kerttu Huttunen

Project description

During communication, speechreading (visual speech recognition) and facial expressions are inherently and unconsciously utilized to support auditory speech recognition. Speechreading is necessary especially in adverse listening conditions and for compensating hearing difficulties. For the best speechreading performance, there is, so far, only scarce international research evidence on where one should look at and for how long. However, some previous studies have indicated that speechreading skills can be trained but there is a need to explore the issue more closely.  

In Gaze on lips? we shall construct a speechreading test and an application for practising speechreading. Using them, we research how children use their gaze when they speechread silently produced words and sentences and how speechreading skills can be improved. Our results will help to make training of speechreading more effective e.g., in speech and language therapy. A speechreading application will be constructed, and it will be available as a freeware after the project. In addition, the effectiveness of computer vision algorithms on speechreading will be investigated. The aim is to develop further computer vision methods constructed for visual speech recognition. 

Research group

The principal investigator of the Gaze on lips? is professor Kerttu Huttunen and the project group includes in part- or full-time positions:

In addition, other researchers are Assistant Professor Xiaobai Li (Univ Oulu), Adjunct Professor Kaisa Tiippana (Helsinki Univ), Professor Veikko Surakka (Tampere Univ), Associate Professor Ignace Hooge (Utrecht University, Netherlands) and PhD Henri Olkoniemi (Univ Oulu). Company partner is  Outloud Oy.