Information freshness optimization in energy harvesting IoT networks

Thesis event information

Date and time of the thesis defence

Place of the thesis defence

L5, Linnanmaa

Topic of the dissertation

Information freshness optimization in energy harvesting IoT networks

Doctoral candidate

Master of Science Mohammad Hatami

Faculty and unit

University of Oulu Graduate School, Faculty of Information Technology and Electrical Engineering, CWC - Radio Technologies

Subject of study

Communications Engineering


Doctor Emilio Calvanese Strinati, CEA-Leti, Grenoble


Associate Professor Marian Codreanu, University of Oulu

Visit thesis event

Add event to calendar

Information Freshness Optimization in Energy Harvesting IoT Networks

Information freshness is crucial for time-critical IoT applications, e.g., environment monitoring and control systems. We consider an Internet of things (IoT) network with multiple energy harvesting sensors, users, and an edge node. The users are interested in time-sensitive information about physical quantities, each measured by a sensor. Users make on-demand requests to a cache-enabled edge node where the cache contains the most recently received measurements from each sensor. The edge node serves the users' requests by deciding whether to command the corresponding sensors to send fresh status updates or use the aged data in the cache. We aim to find the best actions of the edge node at each time slot for each sensor, i.e., optimal policies, that minimizes the average age of information (AoI) of the served measurements, i.e., average on-demand AoI.

In the present work, we first study status updating for decoupled sensors, i.e., the sensors have independent communication channels to the edge node. We model this problem as a Markov decision process (MDP) and developed two classes of reinforcement learning (RL) based algorithms: a model-based relative value iteration algorithm (RVIA) relying on dynamic programming, and a model-free Q-learning method. Then, we study status updating in IoT networks under a transmission constraint where the edge node can command only a limited number of sensors at each time slot, i.e., leading to per-slot transmission constraint. We model the problem as an MDP for which an iterative algorithm is proposed to obtain an optimal policy. Note, however, that the computational complexity of finding an optimal policy increases exponentially in the number of sensors. Thus, we develop an asymptotically optimal low-complexity algorithm -- termed relax-then-truncate -- and prove that it is optimal as the number of sensors goes to infinity. Finally, we study status updating under inexact knowledge about the battery levels of the sensors; namely, the edge node is informed about the sensors’ battery levels only via the status update packets, leading to uncertainty about the battery levels for the decision-making. Accounting for the partial battery knowledge, we model the problem as a partially observable MDP (POMDP) for which we develop a novel dynamic programming algorithm that obtains an optimal policy.
Last updated: 23.1.2024