Resumen de Risk assessment in complex data settings: algorithmic fairness and causal inference

Marzieh Karimihaghighi

• español

  Intentamos abordar algunos desafíos en las herramientas estructuradas de evaluación de riesgos en dos áreas de aplicación de riesgo de reincidencia y riesgo de deserción universitaria. Usando métodos de ML, sugerimos un escenario de costo-beneficio para ahorrar tiempo, gastos y personal de manera eficiente en una evaluación basada en datos del riesgo de reincidencia violenta. Esto conduce a menos evaluaciones a cambio de una pequeña cantidad de cambios no detectados. Mitigamos impacto dispar del modelo en tasa de evaluación en algunos grupos demográficos. Obtenemos modelos de predicción de riesgo de ML más precisos en comparación con los modelos anteriores y mejoramos la equidad algorítmica de los modelos en algunos grupos sensibles en términos de disparidad de errores y calibración. Usando métodos estadísticos de inferencia causal, mostramos que una reducción en carga de trabajo universitaria reduce riesgo de deserción y liberación condicional puede reducir riesgos de reincidencia general y violenta.

• English

  Risk assessment tools (RATs) are widely used in several decision making Processes such as health system, ecosystem protection, information security, auditing, project management, education retention, and criminal justice.

  Although structured RATs have been appropriate alternatives to traditional prediction methods especially in terms of their accuracy [Grove and Meehl, 1996, Grove et al., 2000, Kirton and Kravitz, 2011], there are still some challenges with regards to the performance of these tools in terms of predictive accuracy [Yang et al., 2010, Myers and Nikoletti, 2003, Turner et al., 2019, Quinlivan et al., 2017], algorithmic bias [Singh et al., 2011, Angwin et al., 2016, Larson et al., 2016, Hamilton, 2019, Baker and Hawn, 2021, Coleman, 2019, Anderson et al., 2019, Yu et al., 2021], and their effectiveness [Wand, 2011, Ryan et al., 2010].

  We try to address these challenges of RATs in two application areas of recidivism risk in criminal justice and dropout risk in higher education domain.

  Using Machine Learning (ML) methods, we suggest a cost-benefit scenario to efficiently save time, expenses and staff in a data-driven assessment of violent recidivism risk [Karimi-Haghighi and Castillo,2021a]. This leads to fewer evaluations in exchange for some small number of undetected changes. Importantly, we mitigate the model’s disparate impact in the rate of evaluation across some demographics.

  We obtain more accurate ML risk prediction models compared to the previous models and improve algorithmic fairness of the models across some sensitive groups in terms of error disparity and calibration [Karimi-Haghighi and Castillo, 2021b, Karimi-Haghighi et al., 2021].

  We determine the effect of a treatment on the outcome risk using statistical causal inference methods [Karimi-Haghighi et al., 2022]. We show that a reduction in university workload reduces dropout risk and conditional release can reduce general and violent recidivism risks.