Extending Realities: Pioneering Visual, Acoustic and Sensory Technologies in Transdisciplinary Research

This creative networking project will organize a series of transdisciplinary and multi-stakeholder workshops around the emerging use of digital and sensory research methods and practices in the social sciences, natural sciences and humanities. These include, for example, immersive 360º audio and video recording (XR, VR, AR); ecoacoustic practices; smart soundscape sensing; and big data analysis, and other digital humanities approaches to working with new forms of data and novel understandings of our environment.

Some of these technologies are "immersive", which is to say that they integrate virtual content with the physical environment in a way that allows the user to engage naturally with a new blended reality. For example, in immersive video, which is often used with virtual reality (VR) technology, video content aims to give viewers the sense that they are inside the video. In an immersive experience, the user often accepts virtual elements of their environment as part of the whole, potentially becoming less conscious that those elements are not part of physical reality. Over the past several years, immersive technology has been taken up by several academic disciplines, both as a means of capturing in situ data and of comprehensively representing various field sites (e.g. natural landscapes, human interaction spaces, etc.) to end users. For example, in spatial sciences such as architecture and GIS science, immersive VR is used to investigate how humans can perceive and experience a built or natural environment in realistic ways. Anthropologists are using XR to better understand how empathy and understanding function as connective emotions, bringing humans closer together to other humans, to animals, and to the environment.

Other innovative, emerging technological practices, such as those of soundscape ecology, employ recording devices, audio tools, and contextual indices to study the soundscape of a given environment via field recording technologies. These typically employ single or adjacent arrays of microphones, often out in the open air but occasionally embedded in soil or under water, which facilitate collecting sound data in novel ways. Such technologies allow for passively, remotely and continuously recording acoustic environments, while advances in machine learning provide new computational methods for digesting the big data generated. Audio recordings are being used increasingly as rapid, cost-effective monitoring tools that provide novel insights into the dynamics of ecosystems and e.g. monitoring of biodiversity, which are important in the face of global land-use change. For example, ecologists are using these techniques to survey areas containing significant biological activities such as mating, migration and foraging, at ecologically meaningful scales. This type of monitoring can be used to assess i.e. how particular species are affected by anthropogenic disturbances over large spans of time.

Yet due to the relative novelty of these fields and practices, and to the speed of innovation (many of these new technologies are regularly being developed apace by different scholars and/or industry practitioners, often not in sync with one another), there are currently few accepted standards or best practices regarding how these technologies are deployed or applied. Scholars across several Nordic institutions, including the University of Oulu, are leading the development of the methods, frameworks and applications of several of these technologies across different disciplines, which help scientists grapple with contemporary socio-cultural and environmental issues. This project seeks to bring together these researchers from across countries, institutions and fields in order to solidify a research agenda for these technologies; to create a series of best practices and lessons learned for scholars interested in bringing these new digital technologies into their work; and to cultivate a network of researchers that pushes forward paths for future scholarship.