About | HeinOnline Law Journal Library | HeinOnline Law Journal Library | HeinOnline
LOG IN
LOG IN

37 Berkeley Tech. L.J. 71 (2022)
Predicting Consumer Contracts

handle is hein.journals/berktech37 and id is 84 raw text is: PREDICTING CONSUMER CONTRACTS Noam Koltt ABSTRACT This Article empirically examines whether a computational language model can read and understand consumer contracts. In recent years, language models have heralded a paradigm shift in artificial intelligence, characterized by unprecedented machine capabilities and new societal risks. These models, which are trained on immense quantities of data to predict the next word in a sequence, can perform a wide range of complex tasks. In the legal domain, language models can interpret statutes, draft transactional documents, and, as this Article will explore, inform consumers of their contractual rights and obligations. To showcase the opportunities and challenges of using language models to read consumer contracts, this Article studies the performance of GPT-3, the world's first commercial language model. The case study evaluates the model's ability to understand consumer contracts by testing its performance on a novel dataset comprised of questions relating to online terms of service. Although the results are not definitive, they offer several important insights. First, the model appears to be able to exploit subtle informational cues when answering questions about consumer contracts. Second, the model performs poorly in answering certain questions about contractual provisions that favor the rights and interests of consumers, suggesting that the model may contain an anti-consumer bias. Third, the model is brittle in unexpected ways. Performance in the case study was highly sensitive to the wording of questions, but surprisingly indifferent to variations in contractual language. These preliminary findings suggest that while language models have the potential to empower consumers, they also have the potential to provide misleading advice and entrench harmful biases. Leveraging the benefits of language models in performing legal tasks, such as reading consumer contracts, and confronting the associated challenges requires a combination of thoughtful engineering and governance. Before language models are deployed in the legal domain, policymakers should explore technical and institutional safeguards to ensure that language models are used responsibly and align with broader social values. DOI: https://doi.org/10.15779/Z382B8VC90 © 2022 Noam Kolt. t Doctoral Candidate and Vanier Scholar, University of Toronto Faculty of Law; Graduate Affiliate, Schwartz Reisman Institute for Technology and Society. For helpful comments and suggestions, I thank Anthony Niblett, David Hoffman, Gillian Hadfield, Albert Yoon, Margaret Mitchell, Rohan Alexander, John Giorgi, and discussants at the ETH Zurich Center for Law & Economics, Monash-Warwick-Zurich Text as Data Workshop, University of Toronto Centre for Ethics, and We Robot at the University of Miami School of Law. I am also grateful to the editors of the Berkeley Technology Law Journalfor their excellent contributions. OpenAI generously provided access to GPT-3 through the API Academic Access Program. Any errors or oversights are mine alone.

What Is HeinOnline?

HeinOnline is a subscription-based resource containing thousands of academic and legal journals from inception; complete coverage of government documents such as U.S. Statutes at Large, U.S. Code, Federal Register, Code of Federal Regulations, U.S. Reports, and much more. Documents are image-based, fully searchable PDFs with the authority of print combined with the accessibility of a user-friendly and powerful database. For more information, request a quote or trial for your organization below.



Short-term subscription options include 24 hours, 48 hours, or 1 week to HeinOnline.

Purchase Access

Contact us for annual subscription options:

Contact Us

Already a HeinOnline Subscriber?

Log In

profiles profiles most