About | HeinOnline Law Journal Library | HeinOnline Law Journal Library | HeinOnline
LOG IN
LOG IN

29 Psychol. Pub. Pol'y & L. 1 (2023)

handle is hein.journals/psypbclw29 and id is 1 raw text is: AMERICAN x - PSYCHOLOGICAL Psychology, Public Policy, and Law 2023, Vol. 29, No. 1, 1-18 https://doi.org/lO.1037/law0000383 © 2023 American Psychological Association ISSN: 1076-8971 How to Keep Unreproducible Neuroimaging Evidence Out of Court: A Case Study in fMRI and Psychopathy Jarkko Jalaval, Stephanie Griffiths2' 3, and Rasmus Rosenberg Larsen4 Department of Interdisciplinary Studies, Okanagan College 2 Department of Psychology, Okanagan College 3 Werklund School of Education, University of Calgary 4 Department of Philosophy and Forensic Science Program, University of Toronto Mississauga The amount of neuroimaging evidence introduced in courts continues to increase. Meanwhile, neuroimag- ing research is in the midst of a reproducibility crisis, as many published findings appear to be false positives. The problem is mostly due to small sample sizes, lack of direct replications, and questionable research prac- tices. There are concerns that a significant proportion of neuroimaging evidence introduced in court may therefore be unreliable. Guidelines governing the admissibility of scientific evidence Frye and Daubert are not designed to weed out such data. We propose supplementing Frye and Daubert with minimal repro- ducibility criteria that allow judges to make informed admissibility decisions about neuroimaging research. To demonstrate how this could work, we subjected functional magnetic resonance imaging (tMRI) findings on psychopathy evidence that has been admitted in court to a minimal reproducibility test. A systematic PRISMA search found 64 relevant studies but no sufficiently powered, directly replicated evidence of a psy- chopathy-related neurobiological profile. This illustrates two things: (a) the probability of false positives in this data set is likely to be unacceptably high and (b) the reproducibility of similar neuroimaging evidence can be evaluated in a straightforward way. Our findings suggest an urgent need to modify admissibility guidelines to exclude low-quality neuroimaging data. Keywords: reproducibility crisis, law, neuroimaging, psychopathy, admissibility Supplemental materials: https://doi.org/10.1037/law0000383.supp The amount of neuroimaging evidence introduced in court has increased over the past few decades (Alimardani & Chin, 2019; Catley & Claydon, 2015; Chandler, 2015; Denno, 2015; Greely & Farahany, 2019). In the meanwhile, neurosciences are facing a repro- ducibility crisis as many findings previously considered reliable have come under suspicion of being false positives (e.g., Button et al., 2013; Cremers et aL., 2017; Poldrack et al., 2017; Szucs & Ioannidis, 2017; Turner et al., 2018). Although the reproducibility crisis has brought about important changes in standards for conduct- ing, disseminating, and interpreting scientific research (e.g., Nelson Jarkko Jalava https://orcid.org/0000-0001-9111-3155 This work has not received any financial support. No ideas or data in it have been previously disseminated. For PRISMA preregistration and EndNote library, see Jalava (2022) https:I/osf.io/q86dw/ Jarkko Jalava served as lead for conceptualization, writing-original dmft, writing-review and editing, and served in a supporting role for formal analysis. Stephanie Griffiths served as lead for data curation, formal analysis and served in a supporting role for conceptualization, writing-review and editing. Rasmus Rosenberg Larsen served in a supporting role for conceptualization, methodol- ogy, writing-original draft, writing-review and editing. Jarkko Jalava and Stephanie Griffiths contributed to methodology equally. Correspondence concerning this article should be addressed to Jarkko Jalava, Department of Interdisciplinary Studies, Okanagan College, 583 Duncan Avenue West, Penticton, BC V2A 8E1, Canada. Email: jjalava@ okanagan.bc.ca et al., 2018; Scheel et al., 2021), it is not clear whether this has had any appreciable effect on how courts approach neuroscience evi- dence. It is possible, therefore, that courts have unwittingly admitted false positive scientific evidence and, without guidelines for distin- guishing it from reliable effects, may continue to do so (see Beerdsen, 2021; Chin, 2014; Kellmeyer, 2017; Morse, 2018). In this paper, we explain how U.S. courts can adopt lessons from the reproducibility crisis to make informed admissibility decisions about neuroimaging evidence. These decisions, we argue, should include minimal criteria for reproducible science. The criteria we propose here are intended to correct the most common and signifi- cant shortcomings in neuroimaging research. These criteria are also the easiest for courts to evaluate and do not require expertise in neuroimaging or statistics. As such, these criteria could be further developed, evaluated, and updated in regular consensus meetings of subject area experts (e.g., Butler et al., 2020; Lakens, 2022; Tugwell et al., 2007). In the meanwhile, our criteria should be useful for chal- lenging the poorest-quality neuroimaging evidence currently being introduced in court. We illustrate this process using evidence that has occasionally been introduced, and admitted, in court: functional neuroimaging research on psychopathy. The Reproducibility Crisis and Neuroimaging Modern neuroimaging technology allows researchers to measure both brain structure and function. Functional magnetic resonance imaging (fMRI) the focus of this paper measures changes in

What Is HeinOnline?

HeinOnline is a subscription-based resource containing thousands of academic and legal journals from inception; complete coverage of government documents such as U.S. Statutes at Large, U.S. Code, Federal Register, Code of Federal Regulations, U.S. Reports, and much more. Documents are image-based, fully searchable PDFs with the authority of print combined with the accessibility of a user-friendly and powerful database. For more information, request a quote or trial for your organization below.



Short-term subscription options include 24 hours, 48 hours, or 1 week to HeinOnline.

Purchase Access

Contact us for annual subscription options:

Contact Us

Already a HeinOnline Subscriber?

Log In

profiles profiles most