Data-driven algorithmic decision-making systems, including many AI technologies, are ubiquitous in today’s society. Early engagement with these systems and the promotion of critical statistical literacy are therefore essential. This involves not only fostering a basic understanding of how such systems work, but also addressing their broader social impact. As part of a design research project, we developed a learning activity that enables upper secondary students and pre-service teachers to explore issues of fairness in the context of automated credit granting. In this paper, we present the design of the activity, outline the intended learning trajectory, and report on initial implementations with pre-service teachers. We also discuss preliminary findings from a qualitative analysis of students’ fairness-related arguments.