Extending Realities: Pioneering Visual, Acoustic and Sensory Technologies in Transdisciplinary Research
Extending Realities
Funders
Project information
Project duration
-
Funded by
Other International
Project funder
Project coordinator
University of Oulu
Unit and faculty
History, Culture and Communication Studies
Faculty of Humanities
Contact information
Project leader
- Associate Professor (on leave)
Contact person
Other persons
Project description
This creative networking project will organize a series of transdisciplinary and multi-stakeholder workshops around the emerging use of digital and sensory research methods and practices in the social sciences, natural sciences and humanities. These include, for example, immersive 360º audio and video recording (XR, VR, AR); ecoacoustic practices; smart soundscape sensing; and big data analysis, and other digital humanities approaches to working with new forms of data and novel understandings of our environment.
Some of these technologies are "immersive", which is to say that they integrate virtual content with the physical environment in a way that allows the user to engage naturally with a new blended reality. For example, in immersive video, which is often used with virtual reality (VR) technology, video content aims to give viewers the sense that they are inside the video. In an immersive experience, the user often accepts virtual elements of their environment as part of the whole, potentially becoming less conscious that those elements are not part of physical reality. Over the past several years, immersive technology has been taken up by several academic disciplines, both as a means of capturing in situ data and of comprehensively representing various field sites (e.g. natural landscapes, human interaction spaces, etc.) to end users. For example, in spatial sciences such as architecture and GIS science, immersive VR is used to investigate how humans can perceive and experience a built or natural environment in realistic ways. Anthropologists are using XR to better understand how empathy and understanding function as connective emotions, bringing humans closer together to other humans, to animals, and to the environment.
Other innovative, emerging technological practices, such as those of soundscape ecology, employ recording devices, audio tools, and contextual indices to study the soundscape of a given environment via field recording technologies. These typically employ single or adjacent arrays of microphones, often out in the open air but occasionally embedded in soil or under water, which facilitate collecting sound data in novel ways. Such technologies allow for passively, remotely and continuously recording acoustic environments, while advances in machine learning provide new computational methods for digesting the big data generated. Audio recordings are being used increasingly as rapid, cost-effective monitoring tools that provide novel insights into the dynamics of ecosystems and e.g. monitoring of biodiversity, which are important in the face of global land-use change. For example, ecologists are using these techniques to survey areas containing significant biological activities such as mating, migration and foraging, at ecologically meaningful scales. This type of monitoring can be used to assess i.e. how particular species are affected by anthropogenic disturbances over large spans of time.
Yet due to the relative novelty of these fields and practices, and to the speed of innovation (many of these new technologies are regularly being developed apace by different scholars and/or industry practitioners, often not in sync with one another), there are currently few accepted standards or best practices regarding how these technologies are deployed or applied. Scholars across several Nordic institutions, including the University of Oulu, are leading the development of the methods, frameworks and applications of several of these technologies across different disciplines, which help scientists grapple with contemporary socio-cultural and environmental issues. This project seeks to bring together these researchers from across countries, institutions and fields in order to solidify a research agenda for these technologies; to create a series of best practices and lessons learned for scholars interested in bringing these new digital technologies into their work; and to cultivate a network of researchers that pushes forward paths for future scholarship.
Project actions
Core Workshop Oulu
Extending Reality? Immersive Technologies in Transdisciplinary Science
University of Oulu, Finland
12-14 June, 2023
Lo124 (Linnanmaa Campus)
This three-day workshop seeks to bring together researchers from across scientific disciplines and nonacademic stakeholders to discuss emerging scholarly innovations in the use of immersive audio-video technologies (XR/VR/AR) as a toolkit for addressing pressing societal and environmental challenges.
Over the past several years, research on immersive visual and acoustic technologies has accelerated apace across academic disciplines. Scholars from anthropologists to zoologists now work with immersive tech as an experimental space for capturing and analysing data, and for representing and communicating various geospatial fields of research (e.g. natural landscapes and habitats, spaces of human interaction) to others. While much of this research breaks new ground, it also often happens in isolated laboratories of practice. This workshop seeks to break this siloization by convening a diverse group of thinkers and doers from across institutions and fields, and from outside of academia, in order to explore synergies and potential for cross-pollination of ideas, methods and practices. Key outputs comprise a) a white paper outlining a research roadmap for immersive technologies across disciplines and of relevance to non-academics; b) a horizon-scanning, scoping document aimed at a targeted, EUlevel consortium funding proposal. The participants will comprise a newly founded network of scholars working on immersive technologies
Workshop #2, Stockholm
Whose Reality?: Sensation, Representation, and Poetics of ‘extended’ environments via Artistic Research
Kungliga Konsthögskolan
Stockholm, Sweden
1-2 June
In this workshop, we aim to start a conversation among experts in environmental monitoring, (digital) environmental humanities, and artistic research whose work intersects with, draws from, or critiques emerging VR/AR/XR technologies and their application in environmental science and artistic research in particular. This dialogue could also extend to engage the interplay of these research discourses with popular contexts, such as gaming, tourism, and journalism. The workshop will take place over two days, with the intention to discuss the potential use of existing environmental monitoring data and archives in artistic research, as well as the implementation of VR/AR/XR technologies. We hope to also provide demonstrations and will have the use of an ambisonic audio listening room for at least one of these days. A possible outcome for this workshop will be to assemble a collective vision for an upcoming research-based exhibition on this theme. Nonetheless, it is a speculative beginning, open to input and reshaping from the presenters and all participants.
Workshop #1, Tromsø (Norway) and Kilpisjärvi (Finland)
Ecoacoustics and Long-term Ecological Monitoring Workshop
Tromsø (Norway) and Kilpisjärvi (Finland)
13.03-16.03.2023
The second workshop of Extending Realities explores transdisciplinary approaches to sonic interventions in research – specifically methods, practices and protocols for long-term ecoacoustic monitoring, in particular across the European Arctic. The workshop brings together twelve leading international scholars from Finnish, Norwegian, Swedish, French and British institutions, and the European-wide Long-Term Ecological Research, or LTSER network, working in sound and environment. The workshop is held at Kilpisjärvi Biological Research Station, which has for many years been an active Long Term Ecological Resaerch (LTER) site and is currently seeking to deepen its role in the LTER network. The network will discuss and develop protocols for the deployment of long-term ecoacoustic monitoring in the European North. The workshop will consider hardware deployment, sampling protocols, analysis and data workflows as well as transdisciplinary research considerations. During winter 2022/23, network members deployed eight Wildlife Acoustics sound recorders, which enable the monitoring of habitat health and conducting species inventory (e.g. presence/absence surveys to detecting endangered species). This was done to test the effect of extreme temperatures and will generate sample data to inform discussions at the workshop. Newly collected acoustic data from the Kilpisjärvi area will be analysed to consider ecoacoustic protocols and practices. This will serve as a proof of concept for wider implementation of these protocols for other Arctic and high-mountain sites within the pan-European LTSER network. Based on the workshop outcomes, this network of recorders can be reprogrammed and redeployed to form the basis of a long-term ecoacoustic monitoring platform.
The research infrastructure comprising eight Wildlife Acoustics SM-Mini sound recorders, deployed across various gradients of landscape in Kilpisjärvi, is funded from a grant from the Biodiverse Anthropocenes programme.
