About | HeinOnline Law Journal Library | HeinOnline Law Journal Library | HeinOnline
LOG IN
LOG IN

89 Wash. L. Rev. 1 (2014)
The Scored Society: Due Process for Automated Predictions

handle is hein.journals/washlr89 and id is 8 raw text is: THE SCORED SOCIETY: DUE PROCESS FOR AUTOMATED PREDICTIONS Danielle Keats Citron & Frank Pasquale Abstract: Big Data is increasingly mined to rank and rate individuals. Predictive algorithms assess whether we are good credit risks, desirable employees, reliable tenants, valuable customers-or deadbeats, shirkers, menaces, and wastes of time. Crucial opportunities are on the line, including the ability to obtain loans, work, housing, and insurance. Though automated scoring is pervasive and consequential, it is also opaque and lacking oversight. In one area where regulation does prevail-credit the law focuses on credit history, not the derivation of scores from data. Procedural regularity is essential for those stigmatized by artificially intelligent scoring systems. The American due process tradition should inform basic safeguards. Regulators should be able to test scoring systems to ensure their fairness and accuracy. Individuals should be granted meaningful opportunities to challenge adverse decisions based on scores miscategorizing them. Without such protections in place, systems could launder biased and arbitrary data into powerfully stigmatizing scores. INTRODUCTION TO THE SCORED SOCIETY ...... .......... 2 I. CASE STUDY OF FINANCIAL RISK SCORING ........... 8 A. A (Very) Brief History of Credit Scoring Systems ................ 8 B. The Problems of Credit Scoring ........... ........... 10 1. Opacity ........................................ 10 2. Arbitrary Assessments .................. 11 3. Disparate Impact .................. ............. 13 C. The Failure of the Current Regulatory Model................... 16 II. PROCEDURAL SAFEGUARDS FOR AUTOMATED SCORING SYSTEMS .............................. 18 A. Regulatory Oversight over Scoring Systems ........... 20 1. Transparency to Facilitate Testing ..... .......... 24 2. Risk Assessment Reports and Recommendations.......... 25 B. Protections for Individuals .................. 27 Lois K. Macht Research Professor & Professor of Law, University of Maryland Francis King Carey School of Law; Affiliate Scholar, Stanford Center on Internet and Society; Affiliate Fellow, Yale Information Society Project. Professor of Law, University of Maryland Francis King Carey School of Law; Affiliate Fellow, Yale Information Society Project. Cameron Connah and Susan G. McCarty have given us enormous research support. We are grateful to Ryan Calo, Lauren Watts, James Wendell, and the Washington Law Review for including us in the Artificial Intelligence Symposium. We would like to thank Roland Behm, Martha Poon, James Grimmelmann, and Neil Richards for generously offering comments on a draft. 1

What Is HeinOnline?

HeinOnline is a subscription-based resource containing thousands of academic and legal journals from inception; complete coverage of government documents such as U.S. Statutes at Large, U.S. Code, Federal Register, Code of Federal Regulations, U.S. Reports, and much more. Documents are image-based, fully searchable PDFs with the authority of print combined with the accessibility of a user-friendly and powerful database. For more information, request a quote or trial for your organization below.



Short-term subscription options include 24 hours, 48 hours, or 1 week to HeinOnline.

Purchase Access

Contact us for annual subscription options:

Contact Us

Already a HeinOnline Subscriber?

Log In

profiles profiles most