Deep reinforcement learning for adaptive monitarizacion and patrolling of water resources with unmanned surface vehicles

• Autores: Samuel Yanes-Luis
• Directores de la Tesis: Daniel Gutiérrez Reina (dir. tes.), Sergio Luis Toral Marín (dir. tes.)
• Lectura: En la Universidad de Sevilla ( España ) en 2024
• Idioma: inglés
• Número de páginas: 262
• Enlaces
  • Tesis en acceso abierto en: Idus
• Resumen

  • The monitoring of hydrological resources represents a fundamental task for both the conservation of biological ecosystems and for human interest. Large water bodies such as lakes, rivers, and reservoirs deserve special attention as strategic resources for human development. However, monitoring these resources involves significant human efforts and high costs. The use of autonomous surface vehicles equipped with water quality measurement equipment and pollution variables increases efficiency and improves the precision and validity of biological models for such resources. Deploying these vehicles requires special attention to monitoring decision-making policies: where to take samples, how to take samples, and how often, to obtain the most accurate and up-to-date model. The intelligent vehicles must be equipped with adaptive decision-making policies capable of overcoming changing conditions, obstacles, and coordination among a variable number of agents. This thesis summarizes advances over the past 10 years in the development of Artificial Intelligence algorithms for autonomous decision-making in mobile robotics to apply to the monitoring of hydrological resources. The two fundamental problems of monitoring in rivers and lakes have been outlined: on one hand, informative path planning, consisting of obtaining the best possible model, and on the other hand, informative patrolling, consisting of a persistent monitoring of the environment. Both problems have been addressed from the single-agent and multiagent perspective using Deep Reinforcement Learning to deal with high computational complexity and dimensionality. These techniques allow the use of Neural Networks to estimate a policy capable of acting by maximizing a defined reward function for such tasks. Specifically, the Double Deep Q-Learning algorithm, used for decision problems with discrete actions has been modified to work in different problems and typical situations of persistent monitoring such as fully observable non-homogeneous patrolling, as well as the partially unobservable case, where both patrolling and the model are constructed simultaneously. The same algorithm has been used to solve informative patrolling to improve information acquisition and obtain more accurate models. In this sense, this thesis includes, beyond the policy optimization processes, two new model proposals for water quality variables. The first one is based on Local Gaussian Processes when no information about the function to be measured is available. On the other hand, in a deep architecture of Variational Auto-encoder, useful when there is a simulator of discharge phenomena or previous data of water quality variables available. These contributions are structured around six scientific articles published or under review for publication. The obtained results demonstrate that Deep Reinforcement Learning represents a flexible and accurate methodology for optimizing policies with multiple agents and that a single DRL algorithm can solve multiple problems of different nature simply learning through interaction.