About | HeinOnline Law Journal Library | HeinOnline Law Journal Library | HeinOnline

123 W. Va. L. Rev. 735 (2020-2021)
Bias Preservation in Machine Learning: The Legality of Fairness Metrics under EU Non-Discrimination Law

handle is hein.journals/wvb123 and id is 765 raw text is: BIAS PRESERVATION IN MACHINE LEARNING:
THE LEGALITY OF FAIRNESS METRICS UNDER EU
NON-DISCRIMINATION LAW
Sandra Wachter, * Brent Mittelstadt, ** and Chris Russell***
Abstract
Western societies are marked by diverse and extensive biases and
inequality that are unavoidably embedded in the data used to train machine
learning. Algorithms trained on biased data will, without intervention, produce
biased outcomes and increase the inequality experienced by historically
disadvantaged groups. Recognizing this problem, much work has emerged in
recent years to test for bias in machine learning and AI systems using various
fairness and bias metrics. Often these fairness metrics address technical bias,
but not the underlying cause of inequality: social bias. In this Article we make
three contributions. First, we assess the compatibility offairness metrics used in
machine learning against the aims and purpose of EU non-discrimination law.
We show that the fundamental aim of the law is not only to prevent ongoing
discrimination, but also to change society, policies, and practices to level the
playing field  and achieve substantive rather than merely formal equality. Based
on this, we then propose a novel classification scheme for fairness metrics in
machine learning based on how they handle pre-existing bias and thus align with
the aims of non-discrimination law. Specifically, we distinguish between bias
preserving  and bias transforming fairness metrics. Our classification system
is intended to bridge the gap between non-discrimination law and decisions
around how to measure fairness in machine learning and AI in practice. Finally,
we show that the legal need for justification in cases of indirect discrimination
*    Oxford Internet Institute, University of Oxford, 1 St. Giles, Oxford, OXI 3JS, UK. Email:
sandra.wachter@oii.ox.ac.uk
**   Oxford Internet Institute, University of Oxford, 1 St. Giles, Oxford, OXI 3JS, UK.
*** Amazon Web Services, Inc. A great thank you is owed to the Harvard Law School, its
faculty and students, the participants of the Harvard Law Faculty's Workshop, and the members
of the Berkman Klein Center for Internet & Society for the inspiring discussions during Wachter's
research visit in Spring 2020. The authors are also indebted to Dr Silvia Milano, Dr Johann Laux,
and Prof Philipp Hacker for their detailed and immensely valuable feedback that greatly improved
the quality of the paper. This paper would not exist without Jade Thompson, thank you for opening
eyes, hearts, and minds, for caring and making others care. The paper also benefitted significantly
from the insightful comments and diligent work of the editorial team at West Virginia Law Review.
This work of the Governance of Emerging Technologies research programme at the Oxford
Internet Institute has been supported by the British Academy Postdoctoral Fellowship grant nr
PF2\180114 and grant nr PF\170151 from the Omidyar Group, and Miami Foundation.

735

What Is HeinOnline?

HeinOnline is a subscription-based resource containing thousands of academic and legal journals from inception; complete coverage of government documents such as U.S. Statutes at Large, U.S. Code, Federal Register, Code of Federal Regulations, U.S. Reports, and much more. Documents are image-based, fully searchable PDFs with the authority of print combined with the accessibility of a user-friendly and powerful database. For more information, request a quote or trial for your organization below.



Short-term subscription options include 24 hours, 48 hours, or 1 week to HeinOnline.

Contact us for annual subscription options:

Already a HeinOnline Subscriber?

profiles profiles most