Computer-aided assessment of tuberculosis with radiological imaging: From rule-based methods to deep learning
- Autores: Pedro Macías Gordaliza
- Directores de la Tesis: Arrate Muñoz Barrutia (dir. tes.), Juan José Vaquero López (codir. tes.)
- Lectura: En la Universidad Carlos III de Madrid ( España ) en 2022
- Idioma: inglés
- Tribunal Calificador de la Tesis: María Jesús Ledesma Carbayo (presid.), David Expósito Singh (secret.), C. I. Sánchez Gutiérrez (voc.)
- Programa de doctorado: Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan Carlos
- Materias:
- Enlaces
- Tesis en acceso abierto en: e-Archivo
- Resumen
Tuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis (Mtb.) that produces pulmonary damage due to its airborne nature. This fact facilitates the disease fast-spreading, which, according to the World Health Organization (WHO), in 2021 caused 1.2 million deaths and 9.9 million new cases. Actually, the numbers are even more dramatic taking into account that, latent TB (the Mtb. is under control by the body’s immune system and do not cause any symptoms) is present in about a quarter of the world’s population. Within this infected population, Mtb., becomes active in 10% of the cases and mainly damages the lungs owing to its airborne nature.
The distribution of humans suffering TB is far from homogeneous around the globe, being low and middle-income countries much more hit by the pandemic; mainly due to the well-known relationship between undernutrition and a depressed immunological system. Meanwhile, rich countries understood TB as a disease of the past, turning it into a neglected disease associated with poverty, marginalization and social exclusion of individuals suffering from it.
This fact does not benefit the fight against the pandemic at all. However, a game-changer has become crucial in the last 25 years. Resistance to TB drugs has increased markedly during this time, especially in those westernized countries that considered the disease eradicated.
TB has become a global concern. Not in vain, the United Nations (UN) has included the WHO "End TB Strategy” among its SGDs (Sustainable Development Goals) for the period 2015-2035.
This strategy is deployed through several multisectoral projects with different objectives, some of which aimed at improving social risk factors, while others, such as the work carried out in this thesis framework (ERA4TB project), focus on developing new and more effective treatments for TB (e.g., new drugs, regimes, vaccines). In particular, this thesis presents several Artificial Intelligence (AI) methods to improve the automatic analysis of medical imaging from subjects infected with TB.
The role of medical imaging is essential for TB assessment. TB has been usually considered a binary disease (latent/active) due to the limited specificity of the traditional diagnostic tests. Such a simple model causes difficulties in the longitudinal assessment of pulmonary affectation needed for the development of novel drugs and to control the spread of the disease.
Fortunately, X-Ray Computed Tomography (CT) images enable capturing specific manifestations of TB that are undetectable using regular diagnostic tests, which suffer from limited specificity. In conventional workflows, expert radiologists inspect the CT images to obtain a more continuous representation of the TB burden.
However, this procedure is unfeasible to process the thousands of volume images belonging to the different TB animal models and humans required for a suitable (pre-)clinical trial.
To achieve suitable results, automatization of different image analysis processes is a must to quantify TB. It is also advisable to measure the uncertainty associated with this process and model causal relationships between the specific mechanisms that characterize each animal model and its level of damage. Thus, in this thesis, we introduce the set of aforementioned novel methods based on the state of the art AI and Computer Vision (CV).
Initially, we present an algorithm to assess Pathological Lung Segmentation (PLS) employing an unsupervised rule-based model which was traditionally considered a needed step before biomarker extraction.
The use of such traditional methodology at the thesis beginning serves a double purpose: 1) to promote a solution to the problems mentioned above in datasets composed of TB-infected lungs for different animal models and 2) to gain further insight into the segmentation problem. In this way, we avoid the usual black box effect of the more optimized but less informative Machine Learning (ML)/DL most employed models, to subsequently enrich them, by injecting in different ways, as far as possible, the knowledge acquired during this first approach.
This content included in the second thesis chapter details the implementation of an automatic pipeline able to segment lungs infected with Mtb. placing considerable importance on the robust and consistent identification of fuzzy boundaries. The method obeys to the following procedure: 1) Preliminary lung and airway tree Segmentation, which is formed by 1.a) automatic adaptive thresholding, 1.b) rib cage extraction and 1.c) connectivity and topological analysis. 2) Airway tree extraction, divided in 2.a) trachea detection and initialization, 2.b) wavefront propagation, 2.c) bifurcation detection and 2.d) leakage detection. 3) Morphological closing and fuzzy boundaries evaluation with the following sub-processes, 3.a) morphological 3D hole filling and 3.b) fuzzy lung border segmentation.
This procedure allows robust segmentation in a Mtb. infection model (Dice Similarity Coefficient, DSC, 94% ±4%, Hausdorff Distance, HD, 8.64 mm ±7.36 mm) of damaged lungs with lesions attached to the parenchyma and affected by respiratory movement artefacts for macaque models of TB with a mild damage.
Included in the same chapter, a Gaussian Mixture Model ruled by an Expectation-Maximization (EM) algorithm is employed to automatically quantify the burden of Mtb. using biomarkers extracted from the segmented CT images.
Although with limitations, the model is able to estimate the longitudinal evolution of TB burden and could be used to assess the response to treatment of infected subjects when there is a relationship between the damaged lung tissue and TB burden, which is a customary scenario in several animal models.
The GMM model divides the tissue belonging to the lungs into three disease-associated volumes mimicking human experts. This division depends on the grey level intensity of the voxels, measured employing Hounsfield Units (HU), which establish three regions in the lungs histogram corresponding with the following kind of tissues: Helthy tissue, soft tissue and hard tissue.
This approach achieves a strong correlation (R2 ≈ 0.8) between our automatic method and manual extraction.
Beyond meaningful quantitative differences between equal treatments, it can be observed how subjects under the same drug cocktail at the end of the study present a similar response to treatment to the baseline.
The results exhibit how the automatic biomarker extraction can provide good results by statistical modelling of the decision-making process carried out by an expert.
However, even when this approach works as ideally expected, it may be insufficient to our goal of achieving a characterization of the continuous spectrum of the disease that requires the discrimination between the different types of lesions.
Aforementioned, expert radiologists have claimed that TB lesions appear in high-resolution CT images at all disease stages, which radiological manifestations could be used as imaging biomarkers to provide information about the biological course of the disease. Therefore, there is a need to automate the extraction of such TB imaging biomarkers. Consequently, Chapter 3 introduces a model to automate the identification of TB lesions and the characterization of disease progression.
Thus, the chapter, presents a novel method to detect TB lesions on thorax CT scans and extract informative features from them. In particular, the method can be summarized as follows: 1) Lung segmentation and airway tree extraction; 2) Selection of relevant volumes employing the Statistical Region merging method matched with the expert annotations of lesions; 3) Extraction of texture features from each volume at 8, 16, 32, 64, 128 and 256 levels of quantization; 22 features are extracted from the Grey Level Co-Occurrence Matrix (GLCM) and 4 are global descriptor of the volume (Mean, Median, Maximum and Minimum); 4) Optimization of the Random Forest (RF) hyperparameters (number of trees, minimum number of samples for split and the maximum number of features to evaluate per node); The optimal RF classifier is computed per quantization level and number of features employed. The optimization employ a grid search process with 100-fold cross validation where the training data (80% of the total) in each fold is filtered employing Tomek Links to handle class imbalance; 5) Two-fold evaluation: a) The weighted F1 − score is employed as a measure of the classification quality of the most frequent TB lesion types; b) The importance of each feature is evaluated using as merit figure the Gini importance.
The model can provide an adequate classification for a complex multi-label problem, distinguishing between five different TB lesions types: granulomas, conglomerations, trees in bud, consolidations and ground-glass opacities.
This work is the basis for the studies presented in the next chapters on the characterization of the biological changes induced by TB infection. The Chapter 3 proves that ML can characterize segmented lesions, even employing a relatively medium/low capacity model.
Such innovation initiated through the preliminary work presented in this chapter escalates the models’ skill for characterization and quantification of TB to a new paradigm of infinite possibilities within clinical practice.
Thus, the next chapter, 4, proposes a DL-based method capable of identifying lesions from a whole volume without relying on previously segmented lesions.
We hypothesize that since a few handcrafted texture features can capture meaningful information to classify delimited lesions through a Random Forest, much more powerful DL models should perform a suitable classification without the need to delimit lesions and extract handcrafted features.
DL models are end-to-end. Therefore, the features do not have to be defined. Oppositely these are learnt, and their effectiveness is well demonstrated in medical imaging fields. Specifically, we investigate this hypothesis by building a model to mimic the radiologist generated reports by inferring the presence of TB manifestations on CT scans.
Our model exploits the well-known advantages of three-dimensional Convolutional Neural Networks (3D-CNNs). In particular, we adapt the V-Net encoder to distinguish among five different radiological manifestations of TB at each lung lobe.
Specifically, since usually TB manifestations do not appear in the infected lungs isolated (i.e., nodules, conglomeration or cavities appear together in the lung parenchyma), we propose a multitask model designed to identify single instances. A joint force strategy is established to overtake the issues (e.g., exploding/vanishing gradients, lack of sensibility) that generally appear when training complicated deep 3D models with limited size datasets and large medical imaging volumes. Thus, the method employs Self-Normalizing Neural Networks (SNNs) for regularization and propose a strategy to employ the uncertainty associated with each task (homoscedastic uncertainty) to weight them.
Our results are promising with a Root Mean Square Error of 1.14 in the number of nodules and F1- scores above 0.85 for the most prevalent TB lesions (i.e., conglomerations, cavitations, consolidations, trees in bud) when considering the whole lung.
This approach allows an improved extraction of relevant features on large medical volumes through multi-task learning guided by the uncertainty in the model predictions. Therefore, demonstrating that DL models with designs adapted to the context of this thesis allow the extraction of essential information for the characterization of TB. This fact represents a significant advance in the field, even bearing in mind the limitations mentioned for explicability and generalization terms addressed in chapter 5.
In Chapter 5, we present a DL model capable of extracting disentangled information from images of different animal models, as well as information of the mechanisms that generate the CT volumes.
This is essential for understanding disease progression and developing new treatments. The longitudinal characterization of animal models in (pre-)clinical experiments is crucial. For this, we need to extract comparable biomarkers in similar phase of the pathology. We also need to proof the existence of similar pathophysiological mechanisms modulating common causal factors, that give rise to the variability of trials.
In this context, medical imaging techniques enable the extraction of imaging biomarkers from in vivo studies. For example, the number of Mtb.colonies present in a subject correlates with the damaged lung volume in an image of a human, primate, or mouse.
The images contain meaningful information to interpret the mentioned physiological process.
Thus, developing Artificial Intelligence (AI) systems that can not only automate the extraction of particular markers for each animal model (e.g., the damaged lung volume) but are also capable of inferring the common agents of such particular indicators (e.g., bacterial burden) is essential.
To this aim, the last chapter presents a method that goes beyond the more common design premises for DL models that has lessened its inference capabilities. In particular, the DL models capabilities at extracting the statistical dependence between input-output pairs, i.e.,(xi , yi) ∈ X , Y, from assumed independent and identically distributed (i.i.d.) observational data. Since, under such assumption the models tend to learn correlated representations that only hold for specific environments or domains.
Therefore, limiting the extraction of shared biomarkers among inter-species.
However, by developing a method at the intersection between deep generative models, disentanglement and causal representation learning, we introduce a model based on a single trained architecture able to: a) from an input slice, infer its position in a volume, the animal model to which it belongs, the damage present and even more, generate its lung delimitation mask (similar overlap measures to the nnU-Net), b) generate realistic lung images by setting the above variables and c) generate counterfactual images, as healthy versions of a damaged input slice.
Concretely, we optimize our model employing just ten CT volumes per animal model. Namely, ten CTs from humans presenting symptoms of mild and severe tuberculosis infection, ten mice CTs from a model of severe TB infection and ten macaques CTs from a mild and severe TB infection model.
The model is evaluated employing CTs from different datasets (i.e., different disease models, different CT scanners. Therefore, different domains).
To assess the model generalization capacities for the segmentation tasks, we employ DSC and HD.
The segmentation task for the human model is to carry on subjects infected by COVID-19 obtaining an average DSC of 0.968 (standard deviation, sd, ±0.04) and HD = 9.48mm (sd = ±9.49mm).
Likewise, the results for the test mice dataset are DSC = 0.859 (sd = ±0.11) and HD = 1.519mm (sd = ±0.89mm) and, for the macaque dataset are DSC = 0.955 (sd = ±0.06) and HD = 2.95mm (sd = ±3.54mm). Besides, the model is compared with the overlapping results obtained for the dataset and rule based method employed in chapter two. Such comparison shows an improvement of ∼ 5% for the DSC and 2mm for HD in favour of the model presented in Chapter 5.
The qualitative results clearly show the model ability to substitute damaged tissue with healthy tissue in the case of counterfactual images keeping a realistic aspect, similar to the images synthesized by conditioning on the disentangled factors. As a quantitative results, we compare the Hounsfield Units (HU) of real CT slices. We compute the voxel-wise Root Mean Square Error (RMSE) for the reconstructed images per test dataset obtaining an average RMSE = 18.73±2.16. Besides, to illustrate similarities and differences in the HU scale for the counterfactual images, we compare the plot profiles as the damged/reconstructed images.
Such suitable results positively point towards this new AI framework for obtaining the shared factors between animal models that characterize the pathophysiological processes.
To sum up, the first part of Chapter 6 presents the join conclusions from the thesis collection of valuable tools to automate the quantification of pathological lungs and moreover extend the methodology to provide more explainable results which are vital for drug development purposes.
In a second part, the Chapter 6 introduces the main line of future actions to facilitate diagnosis, disease longitudinal monitoring, and understanding disease etiology from the developed works. Namely, extend the validation through triangulated approaches and extend the quantification to include information from other medical imaging modalities, clinical relevant factors, demographics, genomics, etc.
