Physiological signals measurement and spoofing detection from face video

Human faces contain rich biometric and physiological clues. Thus, identity recognition and physiological state monitoring from face videos are feasible. On one hand, subtle color changes in the facial skin can reveal important information about the heart pulse of individuals, which works as the base for remote photoplethysmography (rPPG) signal measurement. Benefitting from computer vision technology, physiological signals can be reconstructed from face videos under laboratory-controlled conditions. On the other hand, face anti-spoofing (FAS) is vital for biometric security as face recognition systems are vulnerable to various presentation attacks.

In the first part of this thesis, three end-to-end spatio-temporal methods are presented for reliable rPPG signals recovery. To exploit efficient contextual clues from both spatial and temporal perspectives, several handcrafted and automatically searched spatio-temporal networks are proposed. Moreover, negative Pearson-based temporal loss and cross-entropy-based frequency constraints as well as rPPG-related auxiliary supervision (e.g., skin segmentation) are proposed for accurate rPPG signal recovery.

In the second part of this thesis, seven deep learning based FAS methods are presented to resolve the issue of intrinsic spoof representation, which is crucial to real-world deployment under unseen scenarios and attack types. On one side, novel convolutional operators as well as the networks are designed for generalized, lightweight, and multi-modal FAS. On the other side, several material-based pixel-wise supervision signals (e.g., depth and reflection) are proposed with an advanced pyramid supervision strategy.

Finally, with the evidence that the spoofings like a face mask cannot reflect live heart pulses, a novel facial rPPG-based method using a vision transformer is proposed to extract discriminative periodic liveness clues for challenging 3D mask attack detection.