Regularization for sparsity in statistical analysis and machine learning

• Autores: Diego Vidaurre Henche
• Directores de la Tesis: Concha Bielza Lozoya (dir. tes.), Pedro Larrañaga Múgica (dir. tes.)
• Lectura: En la Universidad Politécnica de Madrid ( España ) en 2012
• Idioma: inglés
• Tribunal Calificador de la Tesis: Serafín Moral Callejón (presid.), Ruben Armañanzas Arnedillo (secret.), Robert Castelo Valdueza (voc.), Antonio Salmerón Cerdán (voc.), Vicente Gómez Cerdà (voc.), Iñaki Inza Cano (voc.), Juan Antonio Fernández del Pozo de Salamanca (voc.)
• Materias:
  • Matemáticas
    • Ciencia de los ordenadores
      • Inteligencia artificial
    • Estadística
      • Calculo en estadística
• Enlaces
  • Tesis en acceso abierto en: Archivo Digital UPM
• Resumen

  • Pragmatism is the leading motivation of regularization. We can understand regularization as a modification of the maximum-likelihood estimator so that a reasonable answer could be given in an unstable or ill-posed situation.

    In this dissertation, i focus on the applications of regularization for obtaining sparse or parsimonious representations, where only a subset of the inputs is used. A particular form of regularization, L1-regularization, plays a key role for reaching sparsity. Most of the contributions presented revolve around L1-regularization, although other forms of regularization are explored (also pursuing sparsity in some sense). In addition to present a compact review of L1-regularization and its applications in statistical and machine learning, i devise methodology for regression, supervised classification and structure induction of graphical models. Within the regression paradigm, i focus on kernel smoothing learning, proposing techniques for kernel design that are suitable for high dimensional settings and sparse regression functions. I also present an application of regularized regression techniques for modeling the response of biological neurons. Supervised classification advances deal, on the one hand, with the application of regularization for obtaining a naive Bayes classifier and, on the other hand, with a novel algorithm for brain-computer interface design that uses group regularization in an efficient manner.