About | HeinOnline Law Journal Library | HeinOnline Law Journal Library | HeinOnline

26 Loy. Consumer L. Rev. 1 (2013-2014)
A Randomized Experiment Assessing the Accuracy of Microsoft's Bing It on Challenge

handle is hein.journals/lyclr26 and id is 9 raw text is: A RANDOMIZED EXPERIMENT
ASSESSING THE AccuRAcY OF
MICROSOFT'S BING IT ON
CHALLENGE
Ian Ayres*
Emad Atiq
Sheng Li**
Michelle Lu**
Tom Maher
& Christine Tsang***
Abstract: In advertisements associated with its Bing It
On campaign, Microsoft claimed that people preferred Bing
web search results nearly 2:1 over Google in blind comparison
tests. We tested Microsoft's claims by way of a randomized
experiment involving U.S.-based Amazon's. Mechanical Turk
(MTurk) subjects   and   conducted  on   Microsoft's  own
www.bingiton.com  website. We found that (i) a statistically-
significant majority of participants preferred Google search
results to Bing search results (53% to 41%); and (ii) participants
were significantly less likely to prefer Bing results when
randomly assigned to use popular search terms or self-selected
search terms instead of the search terms Microsoft recommends
test-takers employ on its website. Our findings suggest that some
of the claims implicit in Microsoft's advertisements warrant legal
scrutiny. The Bing It On Ad Campaign may be viewed as
(falsely) implying that: (i) Microsoft's claims about consumer
preferences for search engines were based on a generalizable
study; (ii) the preferences of five million individuals who have
taken the Bing It On Challenge online are either consistent with
or the basis for Microsoft's claim that consumers prefer Bing
* William K. Townsend Professor, Yale Law School. ian.ayres@yale.edu.
Yale Law School, J.D. 2014.
Yale Law School, J.D. 2013.

1

What Is HeinOnline?

HeinOnline is a subscription-based resource containing nearly 3,000 academic and legal journals from inception; complete coverage of government documents such as U.S. Statutes at Large, U.S. Code, Federal Register, Code of Federal Regulations, U.S. Reports, and much more. Documents are image-based, fully searchable PDFs with the authority of print combined with the accessibility of a user-friendly and powerful database. For more information, request a quote or trial for your organization below.



Short-term subscription options include 24 hours, 48 hours, or 1 week to HeinOnline with pricing starting as low as $29.95

Contact us for annual subscription options:

Already a HeinOnline Subscriber?

profiles profiles most