Dr. Mélina Verger

This thesis aims to evaluate and reduce algorithmic unfairness of machine learning models widely used in education. Such predictive models, based on increasing amounts of educational data and learning traces, are intended to improve the human learning experience. They can be used, for example, to predict dropout or to personalize the learning experience by adapting educational content to meet the needs of each learner.

However, it has been shown repeatedly that these models can produce biased and discriminatory predictions, most often by producing consistently worse predictions for Black people compared to White people, and for women compared to men. It has therefore become crucial to evaluate the fairness of predictive models' results, according to the different groups present in the data.

State-of-the-art research has focused on comparing the predictive performances of models between groups. For example, for a binary classifier and for male/female groups, the rate of correct predictions is calculated for each group, and the difference between these rates would indicate unfairness. Although this approach is predominant in the literature, it only captures unfairness in terms of predictive performance, while unfairness can manifest in other ways and in more nuanced forms than simple score differences, which needs further exploration.

The main objective of this thesis is thus to deepen the understanding and evaluation of algorithmic unfairness, and to then identify its potential presence in under-studied contexts. These contexts include both sensitive attributes and learner populations that have been little or not considered at all.

To this end, we designed a new algorithmic fairness metric, in short MADD, which is based on the distributions of the results of supervised learning models. This distribution-based approach additionally allows for graphical analyses to better understand the unfairness quantified by MADD. We have demonstrated, both theoretically and experimentally, the validity of this metric and discovered that potential unfairness observed in the data are not always reflected in the model outcomes, as was the case with gender bias in our experiments.

Moreover, we developed a technique to mitigate unfairness using MADD, along with new methods to evaluate fairness with multiple sensitive attributes simultaneously. Indeed, the literature typically considers each attribute separately, while Crenshaw's (1989, 1991) theory of intersectionality argues that combined influences produce unique and different forms of discrimination for certain groups. Our experimental results show that some combinations of attributes increase, reduce, or maintain the level of unfairness initially observed.

Finally, we conducted fairness analyses for new sensitive attributes, whether demographic or related to the learning context, and with new learner populations from African countries, the Philippines, Haiti, and France, thanks to data collected from a MOOC (massive open online course) and the Canvas LMS platform. These experiments revealed unfairness that had not been previously discovered, thus shedding light on potential real unfairness in these contexts.

To facilitate replication of our work and the application of our methods in other contexts, we created an open-source Python library, named maddlib. The data (except for those from the Philippines) and our documented source code are also available online.

The COVID-19 pandemic highlighted the crucial role of online learning and the importance of predicting students’ academic performance to better identify difficulties and adapt pedagogical support. This study set out to tackle that challenge by exploring how different sources of data—academic, behavioral, demographic, and personality-related—can be combined to predict learner success in online courses. While many existing approaches rely on a single stream of information, our aim was to capture a richer and more nuanced picture of student performance.

To do so, we designed a 90-minute online course on a Moodle-like plateform in collaboration with experts, allowing us to gather multiple types of data in a controlled environment. We analyzed these data with decision trees (CART), chosen for their interpretability, and applied recursive feature elimination to pinpoint the most influential predictors. Both the dataset and the code were made publicly available, and a Codalab challenge was created for open experimentation.

The results showed that academic and behavioral variables were far more predictive of success, while demographic variables contribute little. Three stood out in particular: time spent on learning activities, reading and memory skills, and deductive reasoning ability. These findings suggest that online learning outcomes could be improved by adapting teaching strategies to learners’ cognitive profiles. Thus, the study points toward the development of adaptive learning systems and new approaches to personalized educational support.