Extending Realities: Pioneering Visual, Acoustic and Sensory Technologies in Transdisciplinary Research
Extending Realities

Project information
Project duration
-
Funded by
Other International
Project funder
Project coordinator
University of Oulu
Unit and faculty
History, Culture and Communication Studies
Faculty of Humanities
Contact information
Project leader
- Associate Professor (on leave)Roger Norum
Contact person
Other persons
- Jonathan Carruthers-Jones
Project description
This creative networking project will organize a series of transdisciplinary and multi-stakeholder workshops around the emerging use of digital and sensory research methods and practices in the social sciences, natural sciences and humanities. These include, for example, immersive 360º audio and video recording (XR, VR, AR); ecoacoustic practices; smart soundscape sensing; and big data analysis, and other digital humanities approaches to working with new forms of data and novel understandings of our environment.
Some of these technologies are "immersive", which is to say that they integrate virtual content with the physical environment in a way that allows the user to engage naturally with a new blended reality. For example, in immersive video, which is often used with virtual reality (VR) technology, video content aims to give viewers the sense that they are inside the video. In an immersive experience, the user often accepts virtual elements of their environment as part of the whole, potentially becoming less conscious that those elements are not part of physical reality. Over the past several years, immersive technology has been taken up by several academic disciplines, both as a means of capturing in situ data and of comprehensively representing various field sites (e.g. natural landscapes, human interaction spaces, etc.) to end users. For example, in spatial sciences such as architecture and GIS science, immersive VR is used to investigate how humans can perceive and experience a built or natural environment in realistic ways. Anthropologists are using XR to better understand how empathy and understanding function as connective emotions, bringing humans closer together to other humans, to animals, and to the environment.
Other innovative, emerging technological practices, such as those of soundscape ecology, employ recording devices, audio tools, and contextual indices to study the soundscape of a given environment via field recording technologies. These typically employ single or adjacent arrays of microphones, often out in the open air but occasionally embedded in soil or under water, which facilitate collecting sound data in novel ways. Such technologies allow for passively, remotely and continuously recording acoustic environments, while advances in machine learning provide new computational methods for digesting the big data generated. Audio recordings are being used increasingly as rapid, cost-effective monitoring tools that provide novel insights into the dynamics of ecosystems and e.g. monitoring of biodiversity, which are important in the face of global land-use change. For example, ecologists are using these techniques to survey areas containing significant biological activities such as mating, migration and foraging, at ecologically meaningful scales. This type of monitoring can be used to assess i.e. how particular species are affected by anthropogenic disturbances over large spans of time.
Yet due to the relative novelty of these fields and practices, and to the speed of innovation (many of these new technologies are regularly being developed apace by different scholars and/or industry practitioners, often not in sync with one another), there are currently few accepted standards or best practices regarding how these technologies are deployed or applied. Scholars across several Nordic institutions, including the University of Oulu, are leading the development of the methods, frameworks and applications of several of these technologies across different disciplines, which help scientists grapple with contemporary socio-cultural and environmental issues. This project seeks to bring together these researchers from across countries, institutions and fields in order to solidify a research agenda for these technologies; to create a series of best practices and lessons learned for scholars interested in bringing these new digital technologies into their work; and to cultivate a network of researchers that pushes forward paths for future scholarship.
Project actions
Workshop #1
Ecoacoustics and Long-term Ecological Monitoring Workshop
Tromsø (Norway) and Kilpisjärvi (Finland)
13.03-16.03.2023
The second workshop of Extending Realities explores transdisciplinary approaches to sonic interventions in research – specifically methods, practices and protocols for long-term ecoacoustic monitoring, in particular across the European Arctic. The workshop brings together twelve leading international scholars from Finnish, Norwegian, Swedish, French and British institutions, and the European-wide Long-Term Ecological Research, or LTSER network, working in sound and environment. The workshop is held at Kilpisjärvi Biological Research Station, which has for many years been an active Long Term Ecological Resaerch (LTER) site and is currently seeking to deepen its role in the LTER network. The network will discuss and develop protocols for the deployment of long-term ecoacoustic monitoring in the European North. The workshop will consider hardware deployment, sampling protocols, analysis and data workflows as well as transdisciplinary research considerations. During winter 2022/23, network members deployed eight Wildlife Acoustics sound recorders, which enable the monitoring of habitat health and conducting species inventory (e.g. presence/absence surveys to detecting endangered species). This was done to test the effect of extreme temperatures and will generate sample data to inform discussions at the workshop. Newly collected acoustic data from the Kilpisjärvi area will be analysed to consider ecoacoustic protocols and practices. This will serve as a proof of concept for wider implementation of these protocols for other Arctic and high-mountain sites within the pan-European LTSER network. Based on the workshop outcomes, this network of recorders can be reprogrammed and redeployed to form the basis of a long-term ecoacoustic monitoring platform.
The research infrastructure comprising eight Wildlife Acoustics SM-Mini sound recorders, deployed across various gradients of landscape in Kilpisjärvi, is funded from a grant from the Biodiverse Anthropocenes programme.

Workshop #2
Exploring Immersion: Using XR/VR/AR Technologies in SSH Resaerch
University of Oulu, Finland
12-14 June, 2023
This three-day symposium seeks to bring together researchers from across disciplines and non-academic stakeholders to discuss emerging innovations in the use of immersive audio-video technologies (XR/VR/AR) as a research tool. Fueled by the new mobilities engendered by the pandemic, research on immersive technologies has accelerated apace across academic fields. Scholars from anthropologists to zoologists are working with immersive tech as an experimental space for capturing and analysing data, and of representing field sites (e.g. natural landscapes, human spaces of interaction, etc.) to end users. While much of this research is groundbreaking, it typically happens in isolated laboratories of practice. This symposium seeks to break this siloization by convening a diverse group of thinkers and doers from across institutions and fields in order to explore synergies and potential for cross-pollination of ideas, methods and practices. Key outputs will be a) a white paper outlining a research roadmap for immersive technologies across disciplines and of relevance to non-academic spheres; b) a horizon-scanning and scoping document towards a targeted, EU-level consortium funding proposal. The participants will together comprise the founding of a new network of scholars working on immersive technologies.
