Interaction in VR – what are the prospects and key challenges?

I am a conversation analyst with a background in social psychology. Ever since the beginning of my academic career, I have been fascinated by the interaction between people. How we talk and accomplish actions in conversations is a fundamental component of human behavior, and it shapes our understanding of the world.

As I was doing my master’s, I acquired an interest in technology-mediated interaction. This is when I was first acquainted with avatar-based interaction in social virtual environments, as I wrote my master’s thesis as a part of a research group specialized in inter-organizational collaboration. In my doctoral research, I studied how people displayed social presence during teamwork in the virtual world of Second Life.

But it was not until towards the end of my doctoral studies that I got introduced to the Conversation Analysis (CA) methodology. I found my way into a couple of CA data sessions, and noticed how this methodology gave me novel ideas as to how video-based data could be utilized. After my doctoral defense, I was hired to work as part of the University of Oulu’s iTask research group. Working in this team gave me a chance to explore my interest in avatar-based interaction even further in the context of social VR, this time by learning from my colleagues how to properly use CA.

The power of CA lies in its ability to capture detail in human conduct from video recordings. Just a few-second clip can envelop such intriguing phenomena. Video recordings take the analyst back to the original situation and, with the help of written transcripts, the details in how the situation proceeds turn by turn start to emerge.

In my years of studying VR interaction, I have come to understand the great possibilities that lie in this technology. As said by developers, VR could function as a space in which people could meet and perform tasks together. Also, VR provides a training ground for otherwise difficult or dangerous professions. Firefighters saving people from burning buildings or doctors performing brain surgery are just some examples; also in other professions, such as architecture or industrial manufacture, VR can be utilized for training and designing projects. Research also shows that VR can effectively facilitate the treatment of phobias.

For VR to reach its potential as a social and interactive platform, a lot more research is needed. Here are the current five key issues that I think need to be addressed along the way as VR keeps growing:

1. VR might evoke too powerful experiences

VR is a platform that allows experimentation and can help people practice and develop skills. Although VR is merely a simulation, neuropsychological research has shown that it still evokes physical responses in people. Encountering a dinosaur or monster in VR or being hit by an object evokes physical responses, indicating that the users experience these situations as if they were really happening to them.

In one of our ongoing studies, we have analyzed a violent situation in social VR, as we observed a person playfully throwing darts at another person. While both were laughing the situation off, the behavior of the player subjected to the throwing also revealed signs of not being entirely comfortable.

Thus, experimentation in VR is not entirely harmless. Researchers in the field of Human-Computer Interaction have also noted the need to attend to harassment and toxic behavior in social VR and find ways to mitigate them. I hope that our observations also contribute to online harassment prevention.

2. Some people experience motion sickness in VR

VR technology might cause people to feel motion sickness. This is something that I have also observed when collecting data, and it affects how long I can let my participants continue interacting. In addition to this, the headset is quite heavy and uncomfortable. These issues make VR less accessible and must be resolved for the platform to become more user-friendly.

3. Joint actions are accomplished while navigating in two bodies

Through motion capture technology, one’s physical body is used to produce embodied actions in VR. This results in operating with two bodies: the physical and the virtual (avatar). We call this dual embodiment. We have observed that in certain situations, the users’ real body movements do not translate to the avatars the ways in which they intend to, which requires further clarification for the co-participants to understand what is going on.

Also, when the user produces a deictic expression (for example, pointing a finger), they must often lift the avatar’s entire arm. The expression does not always get translated to the recipient of the message, and difficulties might arise in terms of achieving mutual understanding.

4. The impacts of high transmission delay

In many forms of online interaction, latency affects the sequential organization of actions during conversation. Due to the lag, people are more likely to speak at the same time and take turns to repair the conversation. In this way, latency might hinder the progressivity of interaction, causing disturbances to its smoothness. In many current VR environments, the delay is even longer than, for example, in video-mediated interaction, which might lead to entire turns in conversation to switch places for the participants. We are about to study if and how it affects mutual understanding.

5. Learning to use technology while conducting tasks

Interdisciplinary dialogue and dialogue with VR designers are essential parts of the equation for helping to develop VR technology. It is also essential to show how people use these technologies. CA is a particularly fruitful approach for this. For example, we have observed that when people collaboratively complete tasks in VR, they also engage in self-talk and express the so-called change of state tokens (such as ‘okay’ ‘aha’ ‘oh’), which means that they are constantly figuring out something new about the technology and the specific VR environment. Therefore, while trying to complete tasks and engage in interaction, people are simultaneously attempting to figure out how the environment works – even when they have used VR technology before. To no surprise, the tasks in VR are often interrupted when people are simultaneously adjusting to the use of this technology.

The future of VR

For now, the future role of VR remains somewhat unknown. VR might stay as an environment designated for gaming and entertainment, or it might take on new directions beyond what we can now imagine. In terms of technical development, I, for one, would like to see development in avatars and motion capture technology. Being able to engage in embodied actions in intuitive ways would likely enhance the progression of interaction as well as the experience of it.

In the meanwhile, we need more human-centric research and dialogue with VR developers to make the technology as user-friendly as possible. Also, because new technologies and applications are emerging fast, we often wake up only to the consequences of them. Thus, it would be optimal if research and development would go hand-in-hand so that we are better prepared and knowledgeable of the forms VR might take on in the future.

The blog post is about Laura Kohonen-Aho's research. The blog post was written by Laura Kohonen-Aho and Elina Nuutinen based on an interview conducted by Nuutinen.