Reliable localization methods for intelligent vehicles based on environment perception
- Autores: Francisco Miguel Moreno Olivo
- Directores de la Tesis: Fernando García Fenández (dir. tes.)
- Lectura: En la Universidad Carlos III de Madrid ( España ) en 2022
- Idioma: español
- Tribunal Calificador de la Tesis: Jorge Villagrá Serrano (presid.), Joshué Pérez Rastelli (secret.), Enrique David Martí Muñoz (voc.)
- Programa de doctorado: Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de Madrid
- Materias:
- Enlaces
- Tesis en acceso abierto en: e-Archivo
- Resumen
- español
En un pasado no muy lejano, los vehículos autónomos y los Sistemas Inteligentes del Transporte (ITS) se veían como un futuro para el transporte con gran potencial. Hoy, gracias a todos los avances tecnológicos de los últimos años, la viabilidad de estos sistemas ha dejado de ser una incógnita. Algunas de estas tecnologías de conducción autónoma ya están compartiendo nuestras carreteras, e incluso los vehículos comerciales cada vez incluyen más Sistemas Avanzados de Asistencia a la Conducción (ADAS) con el paso de los años. Como resultado, el transporte es cada vez más eficiente y las carreteras son considerablemente más seguras. Uno de los pilares fundamentales de un sistema autónomo es la autolocalización. Una estimación precisa y fiable de la posición del vehículo en el mundo es esencial para la navegación. En el contexto de los vehículos circulando en exteriores, el Sistema Global de Navegación por Satélite (GNSS) es el sistema de localización predominante. Sin embargo, estos sistemas están lejos de ser perfectos, y su rendimiento se degrada en entornos donde la visibilidad de los satélites es limitada. Además, los cambios en el entorno pueden provocar cambios en la estimación, lo que los hace poco fiables en ciertas situaciones. Por ello, el objetivo de esta tesis es utilizar la percepción del entorno para mejorar los sistemas de localización en vehículos inteligentes, con una especial atención a la fiabilidad de estos sistemas. Para ello, esta tesis presenta varias aportaciones: En primer lugar, se presenta un estudio sobre cómo aprovechar la información semántica 3D en la odometría LiDAR, generando una base de conocimiento sobre la contribución de cada tipo de elemento del entorno a la salida de la odometría. Los resultados experimentales se han obtenido utilizando una base de datos pública y se han validado en una plataforma de conducción del mundo real. En segundo lugar, se propone un método para estimar el error de localización utilizando detecciones de puntos de referencia, que posteriormente es explotado por un algoritmo de optimización de posicionamiento de puntos de referencia. Este método, que ha sido validado en un entorno de simulación, es capaz de determinar un conjunto de puntos de referencia para el cual el error de localización nunca supere un límite previamente fijado. Por último, se propone un algoritmo de localización cooperativa basado en un Filtro Genético de Partículas para utilizar las detecciones de vehículos con el fin de mejorar la estimación proporcionada por los sistemas GNSS. El método propuesto ha sido validado mediante múltiples experimentos en diferentes entornos de simulación.
- English
In the near past, we would see autonomous vehicles and Intelligent Transport Systems (ITS) as a potential future of transportation. Today, thanks to all the technological advances in recent years, the feasibility of such systems is no longer a question. Some of these autonomous driving technologies are already sharing our roads, and even commercial vehicles are including more Advanced Driver-Assistance Systems (ADAS) over the years. As a result, transportation is becoming more efficient and the roads are considerably safer.
One of the fundamental pillars of an autonomous system is self-localization. An accurate and reliable estimation of the vehicle’s pose in the world is essential to navigation. Prior to any move, maneuver, or even route planning, a vehicle must first know its position. If the vehicle's pose is not accurate or is very noisy, proper control of the vehicle is unfeasible.
Nowadays, GNSS systems are the unquestionable go-to solutions to solve the localization problem in vehicles. A good, accurate GNSS system is, however, expensive. High-end systems include multiple antennas and support different satellite constellations (GPS, GALILEO, GLONASS, BEIDOU, or QZSS). Moreover, most of the high-end GNSS systems can receive differential corrections to reduce the estimation error and also support Real-Time Kinematics (RTK) positioning to reach centimeter accuracy.
Nevertheless, GNSS solutions are not flawless since they require good satellite coverage in order to obtain an accurate estimate. Plus, they cannot work indoors, so applications that include underground parking lots or tunnels would require a different solution because a GNSS will not be able to work. Another challenging scenario for GNSS systems is navigating inside cities; due to urban canyons. If the vehicle is traversing an area with tall buildings or narrow streets, the sky visibility is reduced. Moreover, the remaining visible satellites will likely be concentrated. Thus, the system will suffer from poor geometric dilution of precision (GDOP).
Finally, although the error from the GNSS position can be estimated, it cannot be controlled. Satellites are constantly moving, and vehicles might enter an area with poor coverage, so the localization error cannot be predicted. That makes autonomous vehicles unable to anticipate a situation with a poor localization estimate.
These types of localization systems can be improved by exploiting the data received by other sensors, such as cameras and LiDAR sensors, which acquire information from the environment surrounding the vehicle.
Accordingly, the main goal of this thesis is to exploit the perception of the environment to enhance localization systems in intelligent vehicles, with special attention to their reliability.
On the one hand, the systems that allow autonomous driving are complex enough, so using the information already processed by perception systems helps reduce the overall system’s complexity. On the other hand, this thesis also aims to provide alternatives to GNSS systems, with the purpose of diversifying the viable solutions to the global localization problem in intelligent vehicles. Finally, another objective of this work is to lay some basics for cooperative localization algorithms based on environment perception.
More specifically, the objectives of this thesis are defined as follows:
- Exploit the advances in perception technologies applied to autonomous vehicles in the latest years; to improve their localization systems.
- Study the benefits of integrating advances in 3D scene understanding with LiDAR-based odometry algorithms.
- Provide novel methods and systems for localization validation. Design such systems to guarantee an upper-bounded localization error.
- Enhance localization algorithms by the means of a multimodal cooperative localization system for autonomous vehicles, combining information from vehicles and infrastructure.
To this end, the content presented in this thesis is divided into three different parts: First, a study on exploiting 3D semantic information in LiDAR odometry is presented, providing interesting insights regarding the contribution to the odometry output of each type of element in the scene. The experimental results have been obtained using a public dataset and validated on a real-world platform. Second, a method to estimate the localization error using landmark detections is proposed, which is later on exploited by a landmark placement optimization algorithm. This method, which has been validated in a simulation environment, is able to determine a set of landmarks so the localization error never exceeds a predefined limit. Finally, a cooperative localization algorithm based on a Genetic Particle Filter is proposed to utilize vehicle detections in order to enhance the estimation provided by GNSS systems. Multiple experiments are carried out in different simulation environments to validate the proposed method.
Before the main contributions of this thesis, a review of the state of the art regarding autonomous vehicles and the technologies involved in this thesis is presented. The literature reviewed starts with a background of autonomous driving platforms and demonstrations. Then, a brief review of perception algorithms for object and infrastructure detection is presented. Afterward, a deep analysis of the different localization methods is performed, including data filtering and fusion techniques. Next, vehicle cooperation mechanisms and cooperative localization algorithms are reviewed. Finally, the most common evaluation systems are reviewed, including a deep analysis of the several metrics that can be used to measure the performance of localization systems.
The first part of this thesis (Chapter 3) presents a study of the performance of LiDAR-based odometry algorithms after filtering the input point clouds using 3D semantic information. The work presented in this chapter exploits the semantic knowledge from a public dataset, the SemanticKITTI. This information is used to filter the LiDAR point clouds using different filtering configurations, like removing dynamic objects, removing the ground, filtering out the far points, or keeping only the points from structures and pole-like objects.
Afterwards, the filtered point clouds are fed to a state-of-the-art LiDAR odometry method (LOAM), and the output odometry is analyzed using two different localization performance metrics: the Absolute Pose Error (APE) and the Relative Pose Error (RPE).
The results obtained using the ground-truth data from the SemanticKITTI dataset provided very meaningful insights regarding the contribution of each type of element in the environment to the final odometry output. For example, the type of environment and the elements present in it are one of the key factors in the final performance of each filtering configuration. Additionally, the data regarding the downsizing of each filtering configuration and the computation time are presented, showing that the processing time can be reduced with some configurations without a big impact on performance.
Finally, the proposed comparison experiments that were carried out using the ground truth data from the SemanticKITTI dataset are replicated using a real-world driving research platform (ATLAS) in a suburban area of Madrid. In this last experiment, the 3D semantic ground truth is not available, so a state-of-the-art 3D semantic segmentation method is used instead to extract the semantic information in the point clouds. The results show some resemblance with the previous results from the experiments using the semanticKITTI dataset, thus validating all the conclusions extracted from the dataset results.
The second part of this thesis (Chapter 4) is focused on localization validation and reference systems based on landmark detection from LiDAR sensors, and it is divided in two parts. First, a method to estimate the localization error based on landmark measurements is presented. The second part details a landmark placement optimization algorithm that exploits the previous error estimation method, which can be used to generate a reference trajectory with a guaranteed bounded error.
First, a simple landmark detection algorithm based on LiDAR point clouds is proposed; in order to use it as a testing method in the rest of the chapter.
Then, a detailed calibration method to estimate the landmark measurement model is presented. This model can be used afterwards to determine an estimation of the landmark detection error based on different parameters. The proposed calibration method is performed semi-manually, based on data visualization techniques. A simulation environment is used to obtain multiple landmark measurements under different conditions.
Secondly, a method to build accuracy heat maps based on the calibrated landmark measurement model is presented. The obtained heat maps represent the standard deviation of the localization error at each point in the testing area, where the localization of the vehicle is estimated by solving the Non-linear Least Squares (NLLS) problem defined by the landmark measurements relative to the vehicle. The proposed accuracy heat maps can be used to estimate the expected localization error in any area with landmarks.
Finally, a landmark placement optimization method is presented. This optimization method exploits the previously presented accuracy heat maps as a function to determine the maximum localization error in the area. With this information, the algorithm is able to provide the position of new landmarks, such as the maximum error in the testing area is bounded, according to a user-defined parameter. The proposed method is based on a modified genetic algorithm, and has been designed to operate offline in a simulation environment that replicates the original testing area from the real world.
The contributions of this chapter have been validated in multiple simulation environments, using the simulation ground truth to determine the localization error of the proposed approach. The experimental results have validated the proposed methods successfully, thus showing the feasibility of having a localization reference system based on landmarks, with the reliability that provides the guaranteed upper error bound of the localization error. This method has great utility in controlled environments, such as proving grounds, where the environment can be modified in order to add new landmarks in the testing area.
Finally, the third part of this thesis (Chapter 5) is focused on cooperative localization for connected autonomous vehicles. This chapter revolves around the idea of exploiting the advances in communication and ITS technologies to improve the localization systems of vehicles. Localization systems usually rely on GNSS systems, which are known to behave badly in urban areas due to the reduced satellite visibility and the structured environment causing multipathing problems. Instead of proposing the use of specific sensors and technologies for an accurate localization system where GNSS systems fail, the work presented in this chapter advocates for the reuse of the perception systems already operating in autonomous vehicles.
More specifically, the cooperative localization system proposed in this chapter takes as inputs the initial a priori localization estimation from the GNSS system, a local odometry source to estimate the velocity of the vehicle and the list of detections from the perception system, containing the positions of the nearby vehicles relative to each vehicle in the cooperative system. Then a genetic particle filter is used as the main estimation algorithm, fusing all the information gathered from all the vehicles in the cooperative system and producing a joint estimation of all vehicle poses.
While other state-of-the-art methods have proved that the particle filter is a powerful estimator in the presented cooperative localization problem, they only estimate the pose of the ego vehicle, thus not considering the whole cooperating system. The problem of estimating all vehicle poses concurrently is, however, much more complex, and the simple particle filter is not able to converge. For this reason, we propose a particle filter where the resampling method is carried out by a genetic algorithm, thus improving the estimation method and allowing the filter to estimate all poses jointly.
In order to properly tune all the parameters of the particle filter and the genetic algorithm, multiple experiments have been carried out in an emulator of autonomous agents, providing a deep understanding of the contribution of each of the parameters to the behavior of the filter. Then, different experiments are performed to validate the proposed cooperative localization approach, first comparing the proposed genetic resampling method with another resampling method typically used in the literature, and then comparing the performance of the proposed cooperative approach with a very popular localization estimation algorithm used everywhere: the Unscented Kalman Filter (UKF).
The results show that the proposed cooperative localization method has a great potential, obtaining a better performance than the UKF, and it is definitely more suitable than the more typical particle filter to solve the cooperative localization problem that was presented.
To conclude, all these contributions properly cover the objectives that were initially marked for this thesis, which could be considered successfully covered. Consequently, the works presented in this thesis were focused on enhancing localization systems through perception data in intelligent vehicles, by covering the disadvantages of GNSS systems and concentrating on the reliability of the systems.
- español
