Infotech Oulu Annual Report 2012 - MediaTeam Oulu

Professor Timo Ojala and Professor Vassilis Kostakos,
Department of Computer Science and Engineering, University of Oulu

timo.ojala(at)ee.oulu.fi, vassilis.kostakos(at)ee.oulu.fi
http://www.mediateam.oulu.fi


Background and Mission

MediaTeam Oulu (MediaTeam), founded in 1997, is a research group of about 35 people at the Department of Computer Science and Engineering at the University of Oulu. MediaTeam conducts leading edge research in selected areas, with the objective of making a visible and lasting impact on society. This involves technology-led research on selected topics, unique deployment of a pervasive computing infrastructure at downtown Oulu, and the development of “proof of concept” prototypes that are subjected to empirical evaluation in the hands of real users in authentic settings. The visible results of our research include, for example, the panOULU WLAN network and the UBI-hotspots deployed around Oulu, the Oulu3Dlive online virtual model of downtown Oulu, and the Kuukkeli-TV search engine for TV programs.


 

Scientific Progress

MediaTeam’s main research areas in 2012 were urban computing, multimedia computing, and Internet of Things (IoT).

Urban computing

Urban computing is an emerging interdisciplinary research field which considers public spaces as potential sites for the development of ubiquitous computing. Urban computing is driven by two important and related trends, urbanization and rapid deployment of computing infrastructure in urban spaces. However, while urban spaces offer the greatest opportunities and strongest demands for ubiquitous computing, there is no fundamental theory, knowledge base, principled methods nor tools for designing and building ubiquitous systems as integral elements of the urban landscape.

MediaTeam coordinates the UBI (UrBan Interactions) research program that aims at introducing a visible and lasting change in society by building a functional prototype of an open ubiquitous city in Oulu. From the user community’s point of view, such a city appears as a smart urban space providing rich interaction between physical, virtual and social spaces. From the R&D community’s point of view, the city appears as an open community testbed stimulating research, innovation and development of new services and applications. The testbed enables urban computing research in an authentic urban setting with real users and with sufficient scale and time span. Such studies are important because real world systems are culturally situated, and cannot be reliably evaluated with lab studies that are detached from the real world context. By deploying a system for a sufficiently long time “in the wild”, we can establish the technical and cultural readiness, and the critical mass of real users needed for determining whether the system can be deemed ’successful’ or not.

The UBI program is executed by a portfolio of projects funded by various sources. 2012 saw the conclusion of the UbiCity project funded by the ERDF and the City of Oulu. In 2012 we continued with the UBI Anthropos, the UBI Metrics and the Department of Architecture’s Adaptive Urban Lighting projects funded by the Academy of Finland, the Urban Flows and Networks project funded by Tekes under the Finland Distinguished Professor Programme and the City of Oulu, and the NIMO project funded by the INTERREG IV A North programme, the City of Oulu and industry. New projects that started in 2012 were the UBI Mingle project funded by the Academy of Finland, and the MAINIO project sponsored by the ERDF and the City of Oulu.

The UBI program brings together a multidisciplinary research consortium. In 2012, it comprised of MediaTeam Oulu (Professor Timo Ojala and Professor Vassilis Kostakos), Interactive Spaces (Professor Jukka Riekki), the Department of Architecture (Dr. Aulikki Herneoja) and the Department of Economics (Professor Rauli Svento) from the University of Oulu, the Department of Industrial Design from the University of Lapland (project manager Ismo Alakärppä), the Madeira Interactive Technologies Institute from the University of Madeira in Portugal (FiDiPro Fellow Vassilis Kostakos), and the Department of Computer Science, Electrical and Space Engineering from the Luleå University of Technology in Sweden (Dr. Karl Andersson).

To realize the open ubiquitous city, the UBI program engages an iterative cycle of technology-led and application-led research, together with the deployment of an open pervasive computing infrastructure at downtown Oulu. This is the unique dimension of our research strategy. The technology-led research produces new technology and knowledge, which together with the new computing infrastructure creates novel opportunities for application-led research. It is driven by domain-specific problems, which the UBI program addresses by developing two types of “proof of concept” prototypes, small-scale short-term demos and longitudinal large-scale pilots. Their empirical evaluation with real users in an authentic urban setting provides valuable feedback and creates new requirements on the technology-led research and the computing infrastructure.

In 2012, the infrastructure related activities included the maintenance of the panOULU WLAN/BT/WSN networks, the UBI-hotspots and the UBI-projectors at downtown Oulu, the design and assembly of portable panOULU BT boxes equipped with touch screen displays, and the procurement of a large modular multi-touch wall dubbed the UBI-wall (Figure 1), and two multi-touch UBI-tables. To cover the operational expenses of the infrastructure, we sold capacity for commercial use, mainly that of the UBI-hotspots, generating 72 500 EUR of revenue in 2012.

Figure 1. Toni Alatalo introducing the UBI-wall and the Oulu3Dlive virtual model to Professor Goshiro Yamamoto from the Nara Institute of Science and Technology.

Further, we launched the first 9-block version of Oulu3Dlive (http://world.oulu3dlive.net, Figure 2), an open virtual 3D model of downtown Oulu implemented atop the open source realXtend platform for the 3D Internet. Together with key stakeholders, licensing, operating and maintenance principles facilitating open access and community development were designed for the model. The first version of the model has been utilized in various ways, including pervasive gaming, mobile augmented reality, urban planning and the development of the realXtend platform.

Figure 2. View into the Oulu3Dlive virtual model.

These heterogeneous computing resources constitute a large distributed system which is virtualized by the UBI-middleware. In 2012, our middleware related activities focused on enhancing the open APIs providing access to the computing resources. The APIs allow the research community, the general public and the SMEs to develop their own applications and services atop the testbed.

We conducted a number of trials atop the testbed in 2012. In the Red Cross Blood Service field trial, the panOULU BT access points around Oulu were configured to conduct unsolicited Bluetooth proximity marketing for the purpose of recruiting blood donors. The BT access points attempted to push a multimedia invitation into passing by mobile phones that had their BT on and visible. During the 31-day trial, about 34 000 devices were discovered and about 400 000 transmission attempts were made. 0.12% of the attempts were successful and eventually 15 recipients went to donate blood. These numbers show what kind of performance can be expected from ‘carpet bombing’ type of unsolicited BT proximity marketing, without any support mechanisms such as visual cues. Current Bluetooth conventions on the authorization and push message delivery are the main technical obstacles towards achieving higher message delivery   rates. We also conducted a measurement campaign where the end-user QoS of the panOULU WLAN was found to be comparable to that of the commercial 3.5G (HSPA) mobile data networks.

Our anthropologic research of the ubiquitous Oulu continued with the collection of rich qualitative data on elderly and young adult citizens’ experiences and conceptions of ubiquitous technology in Oulu (Figure 3). The spatial dimension was taken into account both in the collection and analysis of the material to scrutinize people’s accounts of the urban space. Semi-structured theme interviews were employed together with more experimental methods borrowed from human-centered design. The analysis of the data supports our hypothesis: different people experience the ubiquitous Oulu in differing ways and have distinct technological needs and skills. However, these differences are not solely explained by age, but we have to take into account also people’s backgrounds and demography to better understand citizens’ technology-related experiences. Generally speaking, the results indicate that the social aspects connected to technology are highly valued and that ubiquitous computing technologies could also be developed in a more experimental and co-creative direction.

 

Figure 3. A citizen drawing her mental map of downtown Oulu.

An emerging research direction is community imaging that studies communities of users through the lens of technology. Increasingly, the use of technology is resulting in usage data being recorded, most often as a byproduct. Our community imaging research focuses mostly on analyzing such usage data, and deriving models and understandings of how groups of people behave, both in terms of interacting with each other but also in terms of individual behavior. This work involves three main axes: understanding, tools, and applications. The first focus in our work has been an analysis of the panOULU WLAN traces to model the movement of population across the City of Oulu. Using heuristics, we have been able to identify tourists visiting our city and using the panOULU network, and we have been able to contrast tourists’ mobility patterns to those of local residents. This analysis has helped us understand better how various parts of the city, or indeed our university campus, are used differently by visitors and locals, and which parts are more likely to act as “attractors” for visitors. An orthogonal set of activities involves developing instrumentation and analysis tools for capturing and treating community-level data. For instance, we have developed web-based tools that let us conduct an historic analysis of the WLAN mobility traces, and provides visualization and animation tools such as heat maps, firefly effects and catchment area analysis to investigate city-level mobility further. In addition, we are actively developing a pioneering Android platform instrumentation tool that lets us investigate in extreme detail what happens on smartphones, how they are used, and in what circumstances. Using this tool, we have been able to identify certain peculiarities in how smartphones are used. For instance, we have identified a phenomenon that we call “micro-usage”: about half of our use of smartphones every day takes place in short bursts of interactions that last less than 15 seconds.  Finally, our work seeks to develop interesting applications that communities of users can interact with. For instance, in our work we have used appstores to deploy our software on thousands of smartphones and collect data in real-time about their usage. Furthermore, we are the first group to systematically study the potential of public displays for crowdsourcing. Our work has demonstrated that the expertise and skills of a community may be reachable by developing interactive applications on public displays, and we have found that communities are willing to donate time to interact with public displays in order to contribute to a worthy cause or charity.

Figure 4. A heat map illustrating panOULU WLAN usage.

With invaluable support from the City of Oulu, industry and public funding bodies, the UBI program has established at downtown Oulu a globally unique open community testbed for studying ubiquitous computing in an authentic urban setting. We have engaged in various activities to make our work available to the R&D community at large, and to stimulate the innovation of new services. In 2012, we launched the UBI RIR (Researcher in Residence) program that invites researchers for residency in Oulu, to work in our city laboratory together with our researchers. This way we hope to stimulate international research collaboration on ubiquitous computing and on urban informatics in a very concrete manner. (http://www.ubioulu.fi/en)

Multimedia computing

Our research on multimedia computing includes topics such as content-based video technologies, digital watermarking and speech processing. These research themes have emerged to be highly essential for enabling large-scale and interconnected multimedia systems and services, and have attracted industrial collaboration on several joint research projects.

The pan-European industrial Adaptive Content Delivery Clusters (ACDC) project of the ITEA 2 framework program concluded in 2012. The goal of the ACDC project was to research, develop and demonstrate an adaptive content delivery cluster, intelligent user-aware applications and new business models. The project aimed at novel user-aware multimedia and entertainment TV applications (IPTV, web/internet TV and mobile TV), on demand video and entertainment, personal video recording and targeted advertising services on a variety of networks and terminals (set-top-boxes, PC and mobile) with adaptation, personalization and anticipation features based on semantic knowledge. As a technological enabler for user-aware multimedia and entertainment applications, the project conducted research and development through demonstrations of a distributed computing infrastructure for large scale digital content and semantic knowledge processing, storage and delivery. MediaTeam contributed by developing content-based technology in three collaboration scenarios: a visual content-based recommendation service for a Finnish web game site, a distributed video content analysis service and an interactive HbbTV content-based recommendation service.

The Next Media research program of the ICT SHOK continued in 2012. It aims to meet people’s insatiable need for engaging and activating media experiences by means of new business models, concepts and technology. Production and consumption of media is under radical transformation. Digitalization of production, distribution and consumption of media, as well as the growing penetration of broadband access and mobile internet allow increasingly rich media content to be distributed to a variety of terminals. This endeavor takes advantage of the current transition towards co-creation, interactivity and independence of time and place. The diversity of terminals and devices in consumers’ everyday life is increasing, and the significance of mobility is emphasized. The company led program consists of four work packages whose research themes cover all aspects of media production, from media content access to working processes and business models, as well as user experiences. All major Finnish research organizations and universities are participating in the program. In 2012, MediaTeam participated in the Affective Facets of Multimedia Content research topic. MediaTeam created novel methodology to analyze sentiment in video broadcasts, focusing on the genres of news, drama and movies.

MediaTeam continued research on digital watermarking in 2012. Digital watermarking is a method of embedding a secret sequence of bits in the host media such that it is hard to perceive or remove. This message can be then extracted from the host signal when the necessary algorithms are available. The essential technical challenges in watermarking include invisibility, robustness and capacity. A wide range of watermarking algorithms has been proposed, especially for watermarking of digital images. Generally, however, the resilience of this information in a physical printout, like binary images, holograms or color images has been less studied. This hardcopy watermarking, print-scan resilient and print-cam resilient watermarking, differs greatly from traditional watermarking. The detection, extraction and interpretation of the watermark are realized after a conversion of image information back to digital format using a scanner or mobile phone camera. The extraction algorithms have to handle both geometrical distortion, and pixel value distortions. Very specialized techniques compared to traditional watermarking are required. In 2012, we developed a content based multibit watermarking method which is robust to printing and scanning. The location of the watermark is tied to a coordinate system defined by robust feature points in the images. A watermark message sequence is mapped to a directional angle of periodic patterns, which are scattered and embedded into triangles in permuted locations. With a correct key, the content can be identified from the hardcopy using a scanner for digitalization. We have also been successful in developing robust methods that make possible the extraction of an embedded message from a print-out of a digital image with a camera phone. The distortions of the air interface have to be taken into account in the algorithmic level, including also, for example, optimization of the strength and visibility of the watermark. In 2012, we completed the TULI project “Digitaalisen vesileimauksen hyödyntäminen painotuotteissa” supported by the IPR office specialists and business specialists at the University of Oulu. In this project, we considered the applicability and relevance of mobile phone readable watermarks for industry. A typical usage scenario would enable user interaction with a mobile phone directly from, for example, a product label or a magazine. The process of how to utilize watermarking is illustrated in Figure 5. Also, we took advantage of our past knowledge on watermarking full-color images, binary images as well as holographic images, and considered possible techniques for using a few color images as a carrier for the watermark message. Few color images are of interest because of their wide usage in brand marketing.

Figure 5. The mobile phone based watermarking process.

In speech processing research, a novel method for the construction of low dimensional visualizations of emotional speech was developed based on prosodic speech features. Class labels assigned to data samples are utilized in nonlinear manifold learning in order to generate manifold structures that differentiate different emotions. A supervised classifier based method is used to emphasize the relevant emotional structures. The developed method was used to present the MediaTeam Speech Corpus emotional speech data in a way that resembles closely to the current knowledge of a dimensional model of emotion. The method was shown to be capable of robust emotional content discrimination, achieving the performance level of the human reference. The created visualization was further shown to be capable of mapping the perceived emotional intensity of the emotional speech data samples, even though intensity data was not used in the training of the model.

MediaTeam continued the development of the Kuukkeli-TV, a novel content-based search engine and service platform for analyzing, indexing and retrieving television programs, in 2012. The Kuukkeli-TV service allows users to search and browse TV programs broadcast on Finnish DVB networks, based on Finnish closed captions. The service matches query terms against metadata that has been extracted from the DVB stream, and returns a list of matching programs. The goal is to obtain knowledge of the impact and usage of novel content-based technology for accessing rich multimedia data. The users of the openly available service are encouraged to give feedback about the service in a short questionnaire and give votes on the quality of the hits. The service is unique so that it allows users to view matching content within the television program through multimedia quotes and allows them to search for a particular program’s web stream from the internet. In addition to the multimedia search capabilities, our service platform analyses the language content and uses data mining techniques to enable natural language summarization of content topics. This has allowed us to create and demonstrate new ways of accessing digital television broadcast information. The Mediaseinä service produces near real-time semantic concept summaries from Finnish television broadcasts and permits content-based skimming of the most recent programs in upper program categories, e.g. traveling, food, news and contemporary, science and research, and documentaries. The service is the first of its kind that utilizes multimedia content data to allow skimming of the most recent television broadcasting helping users identify programs that correspond better to their actual viewing interests. In addition to the Mediaseinä service, we have recently introduced the Uutispilvi service that displays monthly and weekly summaries of novel topics from news broadcasts, and lets users access television news related to a selected topic (Figure X). The service is based on novelty topic detection, where data mining and machine learning techniques are used to measure program topics with high novelty value. (http://www.kuukkelitv.fi)

Figure 6. Snapshot of Uutispilvi.

Internet of Things (IoT)

The Internet of Things (IoT) paradigm refers to uniquely identifiable objects (things) and their virtual representations in an Internet-like structure. The future IoT is likely to contain tens of billions of nodes providing universal control of electricity and water utilities, medical ICT and industry automation. Sensor networks on a scale of a million nodes are possible already today, and M2M (machine-to-machine) automatic metering systems of tens of millions of nodes have already been deployed. MediaTeam’s key research projects in this area are the Massive Scale Machine-to-Machine Service (MAMMOTH) project, running until the end of 2013, and the Future Internet program, which was completed in 2012.

The Future Internet program was a part of the ICT SHOK (ICT cluster of the Finnish Strategic Centres for Science, Technology and Innovation) coordinated by TIVIT Oy and funded by Tekes and industry. The Future Internet program aimed at creating a flexible and effective technology base for Internet applications and services in the future. It focused on the health of the routing system, improving the quality of end-to-end connectivity, investigating new ways of information storage and delivery, and Internet security. (http://www.futureinternet.fi)

MediaTeam’s work in the project focused primarily on applying distributed networking technologies in machine-to-machine (M2M) networking. In M2M scenarios, where large numbers of sensors and actuators are deployed over a large geographical area and managed in a distributed manner, peer-to-peer (P2P) systems provide an inherently suitable communication platform. The P2P for M2M prototype, developed in collaboration with Ericsson, enabled decentralized communication between the sensor and actuator nodes in the network. The prototype was based on distributed hash tables (DHT). The aim was to provide a failure-tolerant and cost-efficient alternative to the more established server-based M2M systems. The energy consumption of the distributed M2M prototype was the main object of study, because power efficiency is of the utmost importance in M2M devices that are often equipped with only scarce energy sources.

The MAMMOTH project, funded by the Tekes’s Ubicom technology program and industry, focuses on three complementary aspects in M2M communications: scalability of architectures and protocols, security and congestion aspects of embedded web services on sensor nodes, and platform application interfaces. In 2012, the focus was on developing fully decentralized service architecture for M2M communications. The architecture comprises of a Dynamic Task and Service Composition (DTSC), Execution Environments (EE) for different sensor and actuator devices, and a Distributed Resource Database (DRD). Together, the components provide a scalable, cost efficient and failure tolerant alternative to the current M2M network architectures. During 2012, the basic components of the system were developed. In 2013, the focus will be on integrating the components to enable large-scale evaluation scenarios. Implementing the evaluation scenarios we will gain a practical understanding of the requirements for M2M application interfaces.

Figure 7. UBI-AMI demo at BuildSys 2012 in Toronto.

International collaboration and events in 2012

MediaTeam is highly multidisciplinary and international in its activities, collaborating with domestic and international research partners from different disciplines, and contributing to the international academic community. Collaboration mainly takes place in the form of research visits and organizing international events.

Dr. Tiina Suopajärvi made a 12-month visit to Lancaster University hosted by Professor Lucy Suchman, Dr. Jiehan Zhou made a 4-month visit to Carleton University hosted by Professor Chung-Horng Lung, and Marko Jurmu started his 12-month visit to Keio University hosted by Professor Hideyuki Tokuda in April 2012. We hosted Dr. Nikolas Tselios from Hellenic Open University for 1.5 months in June-July 2012. Steffen Rölike and Laura Schuman from Heinrich-Heine-University Düsseldorf were the first visiting researchers supported by the UBI RIR program in May 2012.

We organized the 3rd International UBI Summer School 2012, held in Oulu on May 28 – June 2. It comprised of three parallel workshops instructed by leading international experts (Figure 8): “Information Visualization for UbiComp Data” by Professor Aaron Quigley from the University of St. Andrews, Scotland; “Supporting Community through Interactive Public Displays” by Dr. Keith Cheverst from Lancaster University, UK; and “Urban Sensoria: Human-Centered Computing in Practice” by Dr. Alejandro (Alex) Jaimes from Yahoo! Research. 51 students from 10 countries enrolled in the summer school via an open international call. All students attended a number of joint events, including a madness session where students presented their background and ongoing research, an opening plenary where each workshop was introduced and the closing plenary where each workshop presented their results. Each workshop had its own curriculum and activities, which included theoretical presentations by the instructor and practical projects conducted in groups of 3-5 students. (http://www.ubioulu.fi/en/UBI-summer-school-2012)

Figure 8. The instructors of the 3rd International UBI Summer School 2012 posing with Timo Ojala. From the left: Alex Jaimes, Keith Cheverst and Aaron Quigley.

We organized the 1st International Workshop on Ubiquitous Mobile Instrumentation (UbiMI 2012) at the UbiComp 2012 conference in Pittsburgh in Sep 2012. The workshop brought together researchers who use mobile phones as instruments in ubiquitous computing research. The program of the full-day workshop comprised of the presentations of eight papers and a panel discussion. (http://ubimi.blogspot.fi)


 

Personnel

professors, doctors

7

doctoral students

14

others

13

total

34

person years

26


 

External Funding

Source

EUR

Academy of Finland

357 000

Ministry of Education and Culture

35 000

Tekes

471 000

other domestic public

117 000

domestic private

82 000

internationtal

107 000

total

1 169 000


 

Doctoral Theses

Davidyuk O (2012) Automated and interactive composition of ubiquitous applications. Acta Universitatis Ouluensis C 420.

Kukka H (2012) Case studies in human information behaviour in smart urban spaces. Dissertation, Acta Universitatis Ouluensis C 426.


 

Selected Publications

Ferreira D, Kostakos V & Dey A K (2012) Lessons learned from large-scale user studies: Using Android market as a source of data. International Journal of Mobile Human Computer Interaction 4(3): 28-43.

Harjula E, Kassinen O & Ylianttila M (2012) Energy consumption model for mobile devices in 3G and WLAN networks. Proc. 9th IEEE International Conference on Consumer Communications and Networking (CCNC 2012), Las Vegas, NV, USA, 532-537.

Hosio S, Kostakos V, Kukka H, Jurmu M, Riekki J & Ojala T (2012) From school food to skate parks in a few clicks: Using public displays to bootstrap civic engagement of the young. Proc. 10th International Conference on Pervasive Computing (Pervasive 2012), Newcastle, UK, 425-442.

Keskinarkaus A, Pramila A & Seppänen T (2012) Image watermarking with feature point based synchronization robust to print–scan attack. Journal of Visual Communication and Image Representation 23(3): 507–515.

Lindén T, Heikkinen T, Kostakos V, Ferreira D & Ojala T (2012) Towards multi-application public interactive displays. Proc. 2012 International Symposium on Pervasive Displays (PerDis 2012), Porto, Portugal, article no. 9.

Memarovic N, Langheinrich M, Alt F, Elhart I, Hosio S & Rubegni E (2012) Using public displays to stimulate passive engagement, active engagement, and discovery in public spaces. Proc. Media Architecture Biennale 2012 (MAB 2012), Aarhus, Denmark, 55-64.

Ojala T, Kokkoniemi J, Luukkonen J, Hakanen T, Salmi O & Pokkinen P (2012) Reflecting QoS of low-cost multi-provider municipal WiFi on commercial 3.5G mobile data and ISM spectrum occupancy. Proc. 10th ACM International Symposium on Mobility Management and Wireless Access (MobiWac 2012), Paphos, Cyprus, 37-44.

Ojala T, Kostakos V, Kukka H, Heikkinen T, Lindén T, Jurmu M, Hosio S, Kruger F & Zanni D (2012) Multipurpose interactive public displays in the wild: Three years later. Computer 45(5): 42-49.

Ojala T, Kruger F, Kostakos V & Valkama V (2012) Two field trials on the efficiency of unsolicited Bluetooth proximity marketing. Proc. 11th International Conference on Mobile and Ubiquitous Multimedia (MUM 2012), Ulm, Germany, article no. 38.

Pramila A, Keskinarkaus A & Seppänen T (2012) Toward an interactive poster using digital watermarking and a mobile phone camera. Signal, Image and Video Processing 6(2): 211-222.

Suopajärvi T, Ylipulli J & Kinnunen T (2012) “Realities behind ICT dreams”: Designing a ubiquitous city in a living lab environment. International Journal of Gender, Science and Technology 4(2): 231-252.

Zhou J, Athukorala K, Gilman E, Riekki J & Ylianttila M (2012) Cloud architecture for dynamic service composition. International Journal of Grid and High Performance Computing 4(2): 17-31.

Zhou J, Sun J, Athukorala K, Wijekoon D & Ylianttila M (2012) Pervasive social computing: Augmenting five facets of human intelligence. Journal of Ambient Intelligence and Humanized Computing 3(2): 153-166.

 

Last updated: 15.4.2014