Face Biometrics: Image quality and physiology analysis for behavior understanding and improved security
- Autores: Javier Hernández Ortega
- Directores de la Tesis: Julián Fiérrez Aguilar (dir. tes.), Aythami Morales Moreno (codir. tes.)
- Lectura: En la Universidad Autónoma de Madrid ( España ) en 2022
- Idioma: inglés
- Número de páginas: 200
- Títulos paralelos:
- Biometría Facial: Calidad de imagen y análisis fisiológico para la comprensión del comportamiento y la Mejora de la Seguridad
- Tribunal Calificador de la Tesis: Javier Ortega García (presid.), José Miguel Buenaposada Biencinto (secret.), Christoph Busch (voc.)
- Programa de doctorado: Programa de Doctorado en Ingeniería Informática y de Telecomunicación por la Universidad Autónoma de Madrid
- Materias:
- Enlaces
- Tesis en acceso abierto en: Biblos-e Archivo
- Resumen
Since the beginning of the century there have been a large number of technological advances that have drastically changed people’s lives. Among these advances we should highlight the current ubiquity of devices like personal computers, smartphones, tablets, and wearables, mainly due to their prices that are becoming more and more affordable for the majority of the population. These devices introduced a vast number of services and new possibilities into people’s daily lives, services that previously were only possible for big entities. Thanks to this technological evolution, the public availability of personal information is immense. The explosion of social networks has made accessible a large amount of information about the general behavior, facial features, location, and personal tastes of almost every person with an online profile. The increase in the computing power of the mentioned devices also allowed biometricbased systems to have a growing presence throughout the world. Nowadays these systems are present in multiple areas such as security access, e-health systems, services (e.g. online banking) and, of course, video-surveillance.
Of all the existing biometric traits, facial recognition is the one that has experienced the highest growth in the last years thanks to its unique properties, mainly the possibility to acquire facial information remotely in a non-intrusive way using cheap devices, e.g., video cameras integrated in cell phones and computers. Due to this high level of deployment, there are certain factors surrounding facial biometrics that must be studied in depth in order to use it safely.
In this Thesis we focus on answering questions about some issues associated with facial recognition.
First, we cover the definition of face quality and how to estimate it. We also discuss the topic of detecting face presentation attacks to recognition systems using advanced artifacts such as hyper-realistic 3Dmasks and DeepFakes. Finally, we analize the use of facial video sequences for extracting physiological and behavioral information like heart rate.
For each one of the previous topics, this Dissertation proposes reviews of the state of the art, evaluations of methods already existing in the literature, and it introduces original approaches, often applying the potential of deep learning methods.
This Dissertation consists of five different parts. The first part focuses on the problem statement and the main contributions of the Dissertation, as well as a broad summary of the state of the art. The experimental chapters are divided into three parts, Part II, Part III and Part IV. Finally, Part V concludes the Thesis.
Part I introduces the basic concepts of biometrics, focusing on the modality of facial biometrics with video sequences. After that, it follows with an introduction to the main topics of study in this Thesis: presentation attack detection in facial recognition systems, facial image quality estimation, and heart rate estimation from video sequences. This part lists the challenges and opportunities in each one of these research fields. Later, a comprehensive review of the state of the art of the three aforementioned topics is made, as well as an introduction to the deep learning techniques that we have used in the experimental part of the Thesis. Finally, Part I concludes with a description of the most relevant databases for face recognition, presentation attack detection, and heart rate analysis, doing and special emphasis on those used during the Thesis.
The first experimental part of this Dissertation (Part II) is focused on face quality estimation and the relationship of this measure with face recognition accuracy. The research carried out in this part aims at implementing quality estimation methods able to obtain numerical scores related to the impact that image quality has on face recognition accuracy. To this end, we first describe FaceQnet, an innovative approach to face quality estimation based on convolutional neural networks, and we analyze its performance on several state-of-the-art face recognition databases. Then, motivated by the scarcity of databases with adequate conditions to train models like FaceQnet, we present FaceQgen, a deep learning model based on GANs, designed for quality estimation, whose main advantage consists in not needing numerical quality labels for training. In this part of the Dissertation the capability of this type of approach to face quality estimation is demonstrated.
The second experimental part (Part III of this Dissertation) is focused on analyzing some of the main existing state-of-the-art aproaches for heart rate estimation based on facial video sequences. We implement the most accurate algorithms and methods, and we improve them introducing concepts like quality estimation of raw rPPG signals. Finally, heart rate estimation methods are applied to the evaluation of the health status of drivers and to the assessment of students during online evaluation.
Part IV is the last experimental part of this Dissertation. It analyzes the vulnerability of face recognition systems to presentation attacks, mainly those using hyper-realistic 3D masks and DeepFake techniques based on deep learning. In this part, the heart rate estimation models developed in Part III are applied to detect the differences between the estimated heart rate from genuine facial videos and from presentation attacks. Additionally, continuous authentication techniques are implemented for detecting attacks with low latency and high accuracy. One of the main contributions of this part of the Thesis is the presentation and evaluation of DeepFakesON-Phys, a deep convolutional neural network designed for the detection of second-generation DeepFakes. Part IV also describes the Heart Rate Database, captured specifically for the experiments of this Thesis.
Finally, in Part V the conclusions drawn from the experimental part of the Thesis are presented and discussed, as well as the main lines of future work that have been identified.
The research conducted in this Dissertation has led to novel contributions including:
-The design, implementation and evaluation of FaceQnet, a convolutional neural network trained for quality estimation related to face recognition accuracy. FaceQnet has been compared with other quality assessment methods (both for facial and general image quality) achieving competitive results.
- The design, implementation and evaluation of FaceQgen, a face quality estimation system based on image restoration using a GAN. This semi-supervised system has been conceived as a way to alleviate the difficulty of obtaining correctly labeled datasets to train fully-supervised models such as FaceQnet.
- A comprehensive experimental analysis of rPPG-based heart rate estimation methods, drawing several practical conclusions that have been used to improve them by introducing a novel estimation of the quality of raw rPPG signals.
- Exploring the application of heart rate estimation methods to fields like driver monitoring and online student assessment (using the edBBplat platform).
- Exploring the potential of heart rate estimation methods for face presentation attack detection when facing hyper-realistic artifacts such as 3D masks.
- Implementing continuous authentication schemes using algorithms like Quickest Change Detection (QCD) for low-latency and high-accuracy detection of face presentation attacks.
- The acquisition of Heart Rate, a new database containing videos of presentation attacks captured at different frame rates, with various spectral bands (RGB and NIR), resolutions, and artifacts.
- The development of DeepFakesON-Phys, a system based on siamese convolutional networks designed for the detection of DeepFakes in video sequences. Its performance has been evaluated on second generation databases such as Celeb-DF v2 and DFDC, obtaining a superior performance to other state-of-the-art systems.
