Artificial intelligence reads people like an open book

The latest trends in the research on artificial intelligence at the University of Oulu are related to human well-being and activity. Artificial intelligence identifies emotions from both human face and voice, and turns the operating environment into a ‘smart space’ Sanna Järvelä, professor in the field of learning and educational technology, envisions using artificial intelligence for stimulating learning. 

Susanna Pirttikangas and Sanna Järvelä are standing in front of a screen mounted on the wall of Tellus Innovation Arena and making faces. On the video screen, descriptions of emotions and percentage figures are hovering next to their faces: anger 9.4%, happiness 30.7%, sadness 13.6 %... The reading next to a small heart figure gives the heart rates of both women.

The application developed by the company Valossa, which began as a University of Oulu spin-off, illustrates the latest trends in the research on artificial intelligence. “Identifying micro expressions, emotions, human well-being and the general status using, for example, machine vision is one of the spearhead fields in artificial intelligence at the University of Oulu”, says Ms. Pirttikangas, who acts as adjunct professor in data science.
“We have already had a number of artificial intelligence spin-offs, whose applications are used for such purposes as indoor positioning, data security and quality assurance.”

But also humans are one of the subjects in the research of artificial intelligence, and that is why one of the persons standing next to the Valossa screen is a researcher of learning, Professor of Education Järvelä. She is currently involved in a joint project with machine vision researchers. The research, funded by the Academy of Finland, is aimed at developing practical applications that would facilitate learning and indicate problems related to that.
“We are halfway through the project, collecting masses of data on, for example, interaction within a group. Once we find out what is relevant about this data, we can monitor problem areas within the group and identify learning triggers', Järvelä explains.

“And machine vision is just one way of doing this. Data can be collected, for example, on the heart rate and skin conductance, and this information can be combined with video data. The physiological channel provides human sciences with new opportunities for understanding humans.”
Speech can also serve as source material: The University of Oulu has modelled how human voice behaves in different emotional states. In the identification of emotions effected by means of so-called physiological signals, the accuracy achieved already matches that of an average listener or viewer.

Smart space supports human activity

Even Tellus Innovation Arena displays several forms of artificial intelligence. Another screen next to the Valossa application indicates such data as air temperature, CO2 concentration and humidity in various parts of the Arena. It is part of an ongoing research project, where data collected by sensors is analysed to determine, for example, utilisation rates of different spaces.
“One of our goals is the optimal use of premises”, Susanna Pirttikangas points out. “The next step in the analysis process is the well-being and functional capacity of people: can it be improved with, say, lighting, or is a meeting starting to go wrong, is the room running out of oxygen, or is the placement of furniture optimal.”

In the examples presented by both Järvelä and Pirttikangas, the common denominator is the ‘smart space’. There is no single definition for such a thing, but as a general feature Pirttikangas highlights the efforts to improve and support human activity by means of artificial intelligence.
And a smart space is not necessarily tied to a specific location. The Smart Mirror application of the machine vision research group of the University of Oulu expands its use anywhere with mobile phone coverage. The application analyses heart rate and respiration information from a facial image and transmits it to a cloud service and, further, to a physician.
Smart space is also linked with a research project about to begin, aimed at improving safety and smooth running of work in a factory environment. “We will be able to tell more about that in 2018,” Pirttikangas says.

All this requires various types of expertise. Pirttikangas illustrates the matter with the smart space of Tellus Innovation Arena.
“Even this requires artificial intelligence and signal processing methods for analysing the data, servers for collecting it, as well as communications links; data is transferred via the 5G network, in the development of which the University of Oulu rates among the global leaders.”

Furthermore, the expansion of artificial intelligence to cover human activities requires an increasingly interdisciplinary approach. “The AI researchers at the University of Oulu collaborate with all faculties. We are not developing methods for their own sake, but there is always an expert within the application area and its needs involved,” Pirttikangas points out.
Järvelä confirms this. “Earlier, a typical scenario was that when some kind of a contraption was completed, human scientists were called in to figure out how it could be used,” she says, caricaturing the past approach. “But now the knowledge of human emotions, way of thinking and interaction is taken into account already at the stage when the developers are just beginning to think about artificial intelligence solutions.”

Long-term competence is visible in the Digital Barometer placement

A very early example can be found of the multidisciplinary AI research at the University of Oulu: The Polar Electron heart rate monitor, which has been developed in collaboration between AI researchers, electronics designers and physicians since the 1970s. The current well-being applications, such as the Smart Mirror, require an increasing contribution from medicine and human sciences.
The early onset of the development of this particular research sector is probably one of the factors that has contributed to the University of Oulu rating among the top four AI organisations in Finland (Digital Barometer 2017).

At best, the business potential of AI can be combined with societal benefits. This is the goal with Sanna Järvelä's learning research as well.
“When we have finished analysing the problems and triggers, we can consider could AI also be used for stimulating learning. AI could, for example, indicate whether the difficulties are related to emotional or cognitive aspects, and provide various options for objectives and strategies,” she envisions.
“We are now in the basic research stage. The operational needs and applications will become clearer as the study progresses.”

Text: Jarno Mällinen

CAPTION: Susanna Pirttikangas and Sanna Järvelä examine smart space that supports human activities. At the end of the line of development, they envision an “Internet of Things”, where everyday objects and equipment linked to the internet provide even more data for AI to process.

Last updated: 16.1.2018