Vision-based 3D perception for mixed reality applications

Project description

Strategic research project of the University of Oulu
Focus institute: Infotech Oulu
Faculty: Faculty of Information Technology and Electrical Engineering (ITEE)


Mixed reality (MR) refers to merging of real and virtual worlds using computer graphics. MR is implemented by overlaying virtual objects or scenes to the user’s field of view with optical or video see-through display systems, including wearable headsets and smartphones. A major challenge in MR is the alignment of the virtual objects with the visual information perceived from the real world. Accurate alignment is necessary to create realistic visualizations and immersive user experience, and it can only be achieved by exploiting the 3D structure of the environment. Another challenge is automatic analysis of the 3D scene that cannot be fully resolved by using 2D image recognition techniques. In this project, we focus on vision-based 3D perception that provides the basis for accurate alignment as well as for recognition. The aim is to develop novel methods for 3D sensing, modeling, and rendering, and build demonstrators to validate the performance of these methods in MR applications.