A Phenomenological Study on Exploring the Experiences of Data Scientists in Managing Bias in Machine Learning Models

Abstract

In the field of artificial intelligence (AI), the integration of machine learning models into decision-making processes has raised significant concerns about algorithmic bias. Despite the development of technical solutions for mitigating bias, little is known about how data scientists experience and navigate these challenges in practice. This study aims to explore the subjective experiences of data scientists in managing bias within machine learning models, addressing the gap in understanding the human and ethical dimensions of AI fairness. We adopt an interpretative phenomenological approach to investigate how data scientists perceive and respond to issues of bias in AI systems. Through in-depth interviews with 15 data scientists, we find that the challenges they face are not only technical but also deeply influenced by ethical dilemmas, organizational pressures, and psychological stress. These professionals encounter significant obstacles when trying to balance model performance with fairness, often under intense corporate and time constraints. The study highlights the importance of considering the personal and moral aspects of bias mitigation, offering insights into the lived experiences of data scientists that are often overlooked in existing research. The findings contribute to a more holistic understanding of AI fairness, suggesting that future research should explore interdisciplinary approaches and incorporate the human perspective into AI development and implementation.