Balazs Aczel, Barnabas Szaszi, Gustav Nilsonne, Olmo R. Van Den Akker, Casper J. Albers, Marcel Alm Van Assen, Jojanneke A. Bastiaansen, Daniel Benjamin, Udo Boehm, Rotem Botvinik-Nezer, Laura F. Bringmann, Niko A. Busch, Emmanuel Caruyer, Andrea M. Cataldo, Nelson Cowan, Andrew Delios, Noah Nn Van Dongen, Chris Donkin, Johnny B. Van Doorn, Anna Dreber, Gilles Dutilh, Gary F. Egan, Morton Ann Gernsbacher, Rink Hoekstra, Sabine Hoffmann, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Kai J. Jonas, Alexander T. Kindel, Michael Kirchler, Yoram K. Kunkels, D. Stephen Lindsay, Jean-Francois Mangin, Dora Matzke, Marcus R. Munafò, Ben R. Newell, Brian A. Nosek, Russell A. Poldrack, Don Van Ravenzwaaij, Jörg Rieskamp, Matthew J. Salganik, Alexandra Sarafoglou, Tom Schonberg, Martin Schweinsberg, David Shanks, Raphael Silberzahn, Daniel J. Simons, Barbara A. Spellman, Samuel St-Jean, Jeffrey J. Starns, Eric Luis Uhlmann, Jelte Wicherts, Eric-Jan Wagenmakers
Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research. © Aczel et al.
ELTE Eotvos Lorand University, , Budapest, Hungary; ELTE Eotvos Lorand University, , Budapest, Hungary; Karolinska Institutet and Stockholm University, Stockholm, Sweden; Tilburg University, Tilburg, Netherlands; University of Groningen, Groningen, Netherlands; Utrecht University, Utrecht, Netherlands; University Medical Center Groningen, University of Groningen, Groningen, Netherlands; University of California Los Angeles, Los Angeles, United States; University of Amsterdam, Amsterdam, Netherlands; Dartmouth College, Hanover, United States; University of Groningen, Groningen, Netherlands; University of Münster, Münster, Germany; University of Rennes, CNRS, Inria and Inserm, Rennes, France; McLean Hospital, Belmont, and Harvard Medical School, Boston, United States; University of Missouri, Columbia, United States; National University of Singapore, Singapore; University of Amsterdam, Amsterdam, Netherlands; University of New South Wales, Sydney, Australia; University of Amsterdam, Amsterdam, Netherlands; Stockholm School of Economics, Stockholm, Sweden; University Hospital Basel, Basel, Switzerland; Monash University, Melbourne, Australia; University of Wisconsin-Madison, Madison, United States; University of Groningen, Groningen, Netherlands; Ludwig-Maximilians-University, Munich, Germany; University of Innsbruck, Innsbruck, Austria; University of Innsbruck, Innsbruck, Austria; Stockholm School of Economics, Stockholm, Sweden; Maastricht University, Maastricht, Netherlands; Princeton University, Princeton, United States; University of Innsbruck, Innsbruck, Austria; University Medical Center Groningen, University of Groningen, Groningen, Netherlands; University of Victoria, Victoria, Canada; Université Paris-Saclay, Paris, and Neurospin CEA, Paris, France; Amsterdam University, Amsterdam, Netherlands; University of Bristol, Bristol, United Kingdom; University of New South Wales, Sydney, Australia; Center for Open Science and the University of Virginia, Charlottesville, United States; Stanford University, Stanford, United States; University of Groningen, Groningen, Netherlands; University of Basel, Basel, Switzerland; Princeton University, Princeton, United States; Amsterdam University, Amsterdam, Netherlands; Tel Aviv University, Tel Aviv, Israel; ESMT Berlin, Berlin, Germany; University College London, London, United Kingdom; University of Sussex, Brighton, United Kingdom; University of Illinois at Urbana-Champaign, Urbana-Champaign, United States; University of Virginia, Charlottesville, United States; University of Alberta, Edmonton, Canada; University of Massachusetts Amherst, Amherst, United States; INSEAD, Singapore, Singapore; Tilburg University, Tilburg, Netherlands; Amsterdam University, Amsterdam, Netherlands; Friesland Mental Health Care Services, Leeuwarden, Netherlands; National Bureau of Economic Research, Cambridge, United States
