Multimodal data fusion for unobtrusive human physiological sensing

The increasing demand for continuous health monitoring, particularly in home environments, is driven by the demographic shift toward aging populations and the growing prevalence of individuals living alone. Traditional monitoring systems, often reliant on wearable devices or user-initiated actions, are limited by compliance, comfort, and accessibility. Unobtrusive technologies offer a solution by enabling remote sensing of physiological signals without requiring active user participation, providing continuous and passive monitoring while preserving comfort and autonomy.

This dissertation investigates unobtrusive physiological sensing for reliable human monitoring, focusing on multiple non-contact technologies, including RGB-D cameras, thermal imaging, and millimeter-wave radar. It proposes novel methods for fusing multimodal data to improve accuracy and robustness. To support this research, a multimodal dataset, OMuSense-23, was established, capturing respiratory and cardiac activities across different poses. This dataset serves both as a benchmark and as a foundation to evaluate challenges such as synchronization complexity, scalability, and ecological validity.

Findings demonstrate that combining multiple sensing modalities enhances monitoring performance, while they introduce challenges in data alignment, processing, and generalization to real-world conditions. The dissertation provides insights into designing and leveraging multimodal sensing frameworks, highlighting strategies for integrating handcrafted and learned features, self-supervised representation learning, and robust fusion techniques. This work contributes to scalable, interpretable, and reliable systems for unobtrusive physiological monitoring in home and assistive living environments.