Reproducibility and robustness of economics and political science research

27 min read Original article ↗

Data availability

The data are available on Zenodo (https://zenodo.org/records/17792605 (ref. 41)) and OSF (https://osf.io/8wsqx/). See OSF for our preanalysis plan.

Code availability

Change history

References

  1. Vazire, S. Quality uncertainty erodes trust in science. Collabra Psychol. 3, 1 (2017).

    Article  Google Scholar 

  2. Donoho, D. L., Maleki, A., Rahman, I. U., Shahram, M. & Stodden, V. Reproducible research in computational harmonic analysis. Comput. Sci. Eng. 11, 8–18 (2008).

    Article  Google Scholar 

  3. King, G. Replication, replication. Polit. Sci. Polit. 28, 444–452 (1995).

    Article  Google Scholar 

  4. Goodman, S. N., Fanelli, D. & Ioannidis, J. P. What does research reproducibility mean? Sci. Transl. Med. 8, 341ps12 (2016).

    Article  PubMed  Google Scholar 

  5. Marcoci, A. et al. Predicting the replicability of social and behavioural science claims in COVID-19 preprints. Nat. Hum. Behav. 9, 287–304 (2025).

    Article  PubMed  Google Scholar 

  6. Milkowski, M., Hensel, W. M. & Hohol, M. Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. J. Comput. Neurosci. 45, 163–172 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Moonesinghe, R., Khoury, M. J. & Janssens, A. C. J. W. Most published research findings are false—but a little replication goes a long way. PLoS Med. 4, e28 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  8. National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in Science (National Academies Press, 2019).

  9. Peterson, D. & Panofsky, A. Self-correction in science: the diagnostic and integrative motives for replication. Soc. Stud. Sci. 51, 583–605 (2021).

    Article  PubMed  Google Scholar 

  10. Pérignon, C., Gadouche, K., Hurlin, C., Silberman, R. & Debonnel, E. Certify reproducibility with confidential data. Science 365, 127–128 (2019).

    Article  ADS  PubMed  Google Scholar 

  11. Brandon, A. & List, J. A. Markets for replication. Proc. Natl Acad. Sci. USA 112, 15267–15268 (2015).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  12. Freese, J. & Peterson, D. Replication in social science. Annu. Rev. Sociol. 43, 147–165 (2017).

    Article  Google Scholar 

  13. Gertler, P., Galiani, S. & Romero, M. How to make replication the norm. Nature 554, 417–9 (2018).

    Article  ADS  CAS  PubMed  Google Scholar 

  14. Maniadis, Z. & Tufano, F. The research reproducibility crisis and economics of science. Econ. J. 127, F200–F208 (2017).

    Article  Google Scholar 

  15. Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (2017).

    Article  PubMed  Google Scholar 

  16. Nosek, B. A. et al. Replicability, robustness, and reproducibility in psychological science. Annu. Rev. Psychol. 73, 719–748 (2022).

    Article  PubMed  Google Scholar 

  17. Askarov, Z., Doucouliagos, A., Doucouliagos, H. & Stanley, T. The significance of data-sharing policy. J. Eur. Econ. Assoc. 21, 1191–1226 (2023).

    Article  Google Scholar 

  18. Brodeur, A., Cook, N. & Neisser, C. P-hacking, data type and data-sharing policy. Econ. J. 134, 985–1018 (2024).

    Article  Google Scholar 

  19. Chang, A. C. & Li, P. Is economics research replicable? Sixty published papers from thirteen journals say ’often not’. Crit. Finance Rev. 11, 185–206 (2022).

    Article  Google Scholar 

  20. Christensen, G. & Miguel, E. Transparency, reproducibility, and the credibility of economics research. J. Econ. Lit. 56, 920–80 (2018).

    Article  Google Scholar 

  21. Dafoe, A. Science deserves better: the imperative to share complete replication files. Polit. Sci. Polit. 47, 60–66 (2014).

    Article  Google Scholar 

  22. McCullough, B., McGeary, K. A. & Harrison, T. D. Do economics journal archives promote replicable research?. Can. J. Econ. 41, 1406–1420 (2008).

    Article  Google Scholar 

  23. Pérignon, C. et al. Computational reproducibility in finance: evidence from 1,000 tests. Rev. Financ. Stud. 37, 3558–3593 (2024).

    Article  Google Scholar 

  24. Camerer, C. F. et al. Evaluating replicability of laboratory experiments in economics. Science 351, 1433–1436 (2016).

    Article  ADS  CAS  PubMed  Google Scholar 

  25. Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2, 637–644 (2018).

    Article  PubMed  Google Scholar 

  26. Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).

    Article  Google Scholar 

  27. Dreber, A. & Johannesson, M. A framework for evaluating reproducibility and replicability in economics. Econ. Inq. 63, 338–356 (2025).

    Article  Google Scholar 

  28. Brodeur, A., Dreber, A., Hoces de la Guardia, F. & Miguel, E. Replication games: how to make reproducibility research more systematic. Nature 621, 684–686 (2023).

    Article  ADS  CAS  PubMed  Google Scholar 

  29. Simonsohn, U., Simmons, J. P. & Nelson, L. D. Specification curve analysis. Nat. Hum. Behav. 4, 1208–1214 (2020).

    Article  PubMed  Google Scholar 

  30. Brodeur, A., Lé, M., Sangnier, M. & Zylberberg, Y. Star Wars: the empirics strike back. Am. Econ. J. 8, 1–32 (2016).

    Google Scholar 

  31. Brodeur, A., Cook, N. & Heyes, A. Methods matter: p-hacking and publication bias in causal analysis in economics. Am. Econ. Rev. 110, 3634–3660 (2020).

    Article  Google Scholar 

  32. Botvinik-Nezer, R. et al. Variability in the analysis of a single neuroimaging dataset by many teams. Nature 582, 84–88 (2020).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  33. Breznau, N. et al. Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proc. Natl Acad. Sci. USA 119, e2203150119 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Huntington-Klein, N. et al. The influence of hidden researcher decisions in applied microeconomics. Econ. Inq. 59, 944–960 (2021).

    Article  Google Scholar 

  35. Menkveld, A. J. et al. Nonstandard errors. J. Finance 79, 2339–2390 (2024).

    Article  Google Scholar 

  36. Silberzahn, R. et al. Many analysts, one data set: making transparent how variations in analytic choices affect results. Adv. Methods Pract. Psychol. Sci. 1, 337–356 (2018).

    Article  Google Scholar 

  37. Fišar, M. et al. Reproducibility in Management Science. Manag. Sci. 70, 1343–2022 (2024).

    Article  Google Scholar 

  38. Ankel-Peters, J., Fiala, N. & Neubauer, F. Do economists replicate?. J. Econ. Behav. Org. 212, 219–232 (2023).

    Article  Google Scholar 

  39. Vilhuber, L., Turrito, J. & Welch, K. Report by the AEA Data Editor. AEA Pap. Proc. 110, 764–765 (2020).

    Article  Google Scholar 

  40. Wood, B. D., Müller, R. & Brown, A. N. Push button replication: is impact evaluation evidence for international development verifiable?. PLoS ONE 13, e0209416 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Brodeur, A. Replication package for “Computational reproducibility and robustness of empirical economics and political science research between 2022 and 2023” [Data set]. Zenodo https://doi.org/10.5281/zenodo.17792605 (2025).

Download references

Acknowledgements

We acknowledge support from Coefficient Giving and the Social Sciences and Humanities Research Council. Any views expressed herein are the authors’ personal opinions and not those of Ontario Public Service. The work by J.D.G. was not undertaken under the auspices of the Ontario Public Service as part of his employment responsibilities. The views expressed in this paper are those of the authors. No responsibility for them should be attributed to the Bank of Canada. The findings, interpretations, and conclusions expressed in this work are entirely those of the authors and do not necessarily reflect the views of the World Bank or its Board of Directors. The Center for Crisis Early Warning (Kompetenzzentrum Krisenfrüherkennung) is funded by the German Federal Ministry of Defense and the German Federal Foreign Office. The views and opinions expressed in this article are those of the author(s) and do not necessarily reflect the official policy or position of any agency of the German government. The views expressed in this paper are those of the authors and do not necessarily reflect the position of the Banco de España or the Eurosystem. All remaining errors are the authors’ responsibility.

Author information

Authors and Affiliations

  1. University of Ottawa, Ottawa, Ontario, Canada

    Abel Brodeur, Derek Mikola, Jonathan Crechet, Guidon Fenig, Jonathan Holmes, Jenny John, Myra Mohnen, Louis-Philippe Morin, Nicholas Rivers, Vincent Thivierge, Wenjie Tian, Hung Truong, Myra Yazbeck & Aline Zayat

  2. Wilfrid Laurier University, Waterloo, Ontario, Canada

    Nikolai Cook

  3. Institute for Replication, University of Ottawa, Ottawa, Ontario, Canada

    Lenka Fiala

  4. Institute for Replication, Tilburg University, Tilburg, The Netherlands

    Lenka Fiala

  5. University of Oxford, Oxford, UK

    Thomas Brailey, Daniel Dao, Tsvetomira Dumbalska, Matthew Hepplewhite, Edmund Kelly, Tobias Roemer & Mirela Zaneva

  6. University of Guelph, Guelph, Ontario, Canada

    Ryan Briggs & Tongzhe Li

  7. The University of Melbourne, Carlton, Victoria, Australia

    Alexandra de Gendre, Adit Maitra & Nicolas Salamanca

  8. Paris Dauphine University, PSL University, LEDA, CNRS, IRD, Paris, France

    Yannick Dupraz

  9. World Bank, Washington, DC, USA

    Jacopo Gabani & Diego Ubfal

  10. Centre for Health Economics, University of York, York, UK

    Jacopo Gabani

  11. Deakin University, Melbourne, Victoria, Australia

    Romain Gauriot & Munirul Nabin

  12. Universitat Autonoma de Barcelona, Bellaterra, Spain

    Joanne Haddad & Eva Pérez Martel

  13. European University Institute, Florence, Italy

    Goncalo Lima

  14. University of Bologna, Bologna, Italy

    Goncalo Lima

  15. RWI-Leibniz Institute for Economic Research, Essen, Germany

    Jörg Ankel-Peters, Arijit Ghosh, Felix Heuer, Florian Neubauer & Julian Rose

  16. Stockholm School of Economics, Stockholm, Sweden

    Anna Dreber & Magnus Johannesson

  17. American University of Armenia, Santa Monica, CA, USA

    Douglas Campbell

  18. School of Foreign Service, Georgetown University Qatar, Doha, Qatar

    Lamis Kattan

  19. University Business Centre, Durham University, Durham, UK

    Diego Marino Fages

  20. Independent Researcher, Bavaria, Germany

    Fabian Mierisch, Barry Hashimoto & Cameron Sells

  21. Dongbei University of Finance and Economics, Dalian, China

    Pu Sun

  22. Brock University, St Catharines, Ontario, Canada

    Taylor Wright

  23. UQAM, Montreal, Quebec, Canada

    Marie Connolly

  24. Berkeley Initiative for Transparency in the Social Sciences, Berkeley, CA, USA

    Fernando Hoces de la Guardia

  25. University of California Berkeley, Berkeley, CA, USA

    Edward Miguel, Eric Guntermann, Gabriel Lenz & Idil Tanrisever

  26. Cornell University, Ithaca, NY, USA

    Lars Vilhuber & Andaleeb Rahman

  27. Texas Tech University, Lubbock, Texas, USA

    Alejandro Abarca

  28. University of Calgary, Calgary, Alberta, Canada

    Mahesh Acharya, Leonard Goff, Tazia Khushboo, Xuewen Lu, Lucija Muehlenbachs, Ali Niazi, Stefan Staubli & Ipek Tastan

  29. Oregon State University, Bloomfield, NM, USA

    Sossou Simplice Adjisse

  30. George Washington University, Washington, DC, USA

    Ahwaz Akhtar

  31. University of Oslo, Oslo, Norway

    Eduardo Alberto Ramirez Lizardi, Synøve Nygaard Andersen, Solveig Topstad Borgen, Astrid de Linde, Geir Godager, Alexi Gugushvili, Amund Hanson Kordt & Jonas Minet Kinge

  32. University of Queensland, Brisbane, Queensland, Australia

    Sabina Albrecht & Todd Morris

  33. Lancaster University, Lancaster, UK

    Zubaria Andlib & Joseph Richardson

  34. Federal Urdu University of Arts Science and Technology, Islamabad, Pakistan

    Zubaria Andlib

  35. University of Warwick, Coventry, UK

    Falak Arrora, Manuel Bagues, Sascha O. Becker, Elaheh Fatemipour, Nurlan Lalayev, Yuchen Lin, Sébastien Montpetit & Yaolang Zhong

  36. Anderson School of Management UCLA, Los Angeles, CA, USA

    Thomas Ash

  37. Luxembourg Institute of Socio-Economic Research (LISER), Esch-sur-Alzette, Luxembourg

    Etienne Bacher

  38. University of Innsbruck, Innsbruck, Austria

    Sebastian Bachler, Andrea Erhart & Armando Holzknecht

  39. Université Laval, Quebec City, Quebec, Canada

    Félix Bacon, Myriam Brown, Alexandre Fortier-Chouinard, Audrée Langlois & Rosalie Montambeault

  40. University of Strathclyde, Glasgow, UK

    Timea Balogh & Lorenzo Crippa

  41. University of California San Diego, La Jolla, CA, USA

    Alisher Batmanov & Idaliya Grigoryeva

  42. University of Bonn, Bonn, Germany

    Mara Barschkett

  43. IZA Berlin, Berlin, Germany

    Mara Barschkett

  44. Risk Software Technologies, Istanbul, Turkey

    Barış Kaan Basdil

  45. Institute of Economic Studies, Faculty of Social Sciences, Charles University, Prague, Czech Republic

    Jaromír Baxa, Tomáš Havránek & Zuzana Irsova

  46. Institute of Information Theory and Automation, Czech Academy of Sciences, Prague, Czech Republic

    Jaromír Baxa

  47. Monash University, Clayton, Victoria, Australia

    Sascha O. Becker

  48. University of Southampton, Southampton, UK

    Monica Beeder

  49. Carleton University, Ottawa, Ontario, Canada

    Louis-Philippe Beland, Sunny Karim, Ardyn Nordstrom, Matt Soosalu, Matthew D. Webb & Tahreen Zahra

  50. University of Montreal, Montreal, Quebec, Canada

    Abdel-Hamid Bello, Thomas Bergeron, Raphaël Jananji, Andreea Musulan, Alexandre Pavlov & Féraud Tchuisseu Seuyong

  51. Columbia University, New York, NY, USA

    Daniel Benenson Markovits & Jori Korpershoek

  52. University of Toronto, Toronto, Ontario, Canada

    Grant Benjamin, Moussa P. Blimpo, Hugo Cordeau, Ben Couillard, Michael Donnelly, Jamie Duncan, Lewis Krashinsky, Anthony McCanny, William Roelofs, Noah Spencer & Jacob Winter

  53. Institute of Intercultural and International Studies, University of Bremen, Bremen, Germany

    Marco Binetti

  54. Karlstad Business School, Karlstad University, Karlstad, Sweden

    Carl Bonander

  55. University of California Davis, Davis, CA, USA

    Joseph Bonneau, Haley Daarstad, Alexa Federice, Yu-Hsiang Dexter Hsu, Alice Malmberg, Linh Phan, Andrew Roskos-Ewoldsen, Daniel Scates, Anya Stewart & Wei-Chien Weng

  56. Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany

    Endre Borbáth

  57. Centre for Research on Equality in Education, University of Oslo, Oslo, Norway

    Nicolai Borgen

  58. Department of Special Needs Education, University of Oslo, Oslo, Norway

    Nicolai Borgen

  59. University of Minnesota, St Paul, MN, USA

    Jonathan Borowsky & Ryan McWay

  60. University of Florence, Florence, Italy

    Elisa Brini

  61. Finnish Centre of Excellence in Tax Systems Research, Tampere University, Tampere, Finland

    Martin Brun

  62. Centre for Environmental Sciences, Hasselt University, Hasselt, Belgium

    Stephan Bruns & Tuan Nguyen

  63. INCHER Kassel, Kassel, Germany

    Stephan Bruns

  64. METRICS Stanford, Stanford University, Stanford, CA, USA

    Stephan Bruns

  65. Nazarbayev University, Astana, Kazakhstan

    Nino Buliskeria & Ali Elminejad

  66. UCL School of Management, University College London, London, UK

    Andrea Calef

  67. Laterite Ltd, Kigali, Rwanda

    Alistair Cameron, Sharmi Sen & Van-Anh Tran

  68. Stockholm Institute of Transition Economics, Stockholm, Sweden

    Pamela Campa

  69. University of California Irvine, Irvine, CA, USA

    Santiago Campos-Rodríguez

  70. “Magna Graecia” University of Catanzaro, Catanzaro, Italy

    Giulio Giacomo Cantone

  71. Oslo Business School, Oslo Metropolitan University, Oslo, Norway

    Fenella Carpena

  72. NYU Abu Dhabi, Abu Dhabi, United Arab Emirates

    Perry Jess Carter

  73. University of Wisconsin-Madison, Madison, WI, USA

    Paul Castañeda Dower

  74. Masaryk University, Brno, Czech Republic

    Ondrej Castek, Mojtaba Firouzjaeiangalougah & Jakub Procházka

  75. Salisbury University, Salisbury, MD, USA

    Jill Caviglia-Harris & Laura Villalobos

  76. Institute of Medicine, University of Gothenburg, Gothenburg, Sweden

    Gabriella Chauca Strand

  77. Zhejiang University, Hangzhou, China

    Shi Chen

  78. University of East Anglia, Norwich, UK

    Sya In Chzhen & Ritchie Woodard

  79. Auburn University, Auburn, AL, USA

    Jong Chung

  80. University of Technology Sydney, Ultimo, New South Wales, Australia

    Jason Collins

  81. Northwestern University, Evanston, IL, USA

    Alexander Coppock

  82. Beijing Normal University, Beijing, China

    Jing Cui

  83. Netherlands Interdisciplinary Demographic Institute, Den Haag, Netherlands

    Christian Czymara

  84. Queen’s University, Kingston, Ontario, Canada

    Danh Chi Dao & Jin Young Yoon

  85. TU Berlin, Berlin, Germany

    Marco David Schmandt

  86. Nottingham Interdisciplinary Centre for Economic and Political Research, University of Nottingham, Nottingham, UK

    Lucas De Melo

  87. University of Melbourne, Carlton, Victoria, Australia

    Lachlan Deer

  88. Banco de España, Madrid, Spain

    Micole De Vera

  89. Social Research Institute, University College London, London, UK

    Velichka Dimitrova

  90. University College Dublin, Dublin, Ireland

    Jan Fabian Dollbaum

  91. University of Fribourg, Fribourg, Switzerland

    Jan Matti Dollbaum

  92. LMU Munich, Munich, Germany

    Jan Matti Dollbaum, Tilman Fries & Kimberly White

  93. Queen Mary University of London, London, UK

    Luu Duc Toan Huynh

  94. University of York, York, UK

    Kiet Tuan Duong, Christoph Dworschak & Xiaoying Gao

  95. Bank of Canada, Ottawa, Ontario, Canada

    Thibaut Duprey & Barbara Sabada

  96. German Institute for Development Evaluation, Bonn, Germany

    Christoph Dworschak

  97. BI Norwegian Business School, Oslo, Norway

    Sigmund Ellingsrud

  98. The American University in Cairo, Cairo, Egypt

    Yasmine Eissa

  99. ETH Zurich, Zurich, Switzerland

    Giulian Etingin-Frati

  100. Victoria University of Wellington, Wellington, New Zealand

    Jan Feld

  101. Oslo Economics, Oslo, Norway

    Erlend Fleisje

  102. Kiel University, Kiel, Germany

    Julia Francesca Engel

  103. Concordia University, Montreal, Quebec, Canada

    Nadjim Fréchet

  104. VisualAIM, Calgary, Alberta, Canada

    Reid Fortier

  105. University of Edinburgh, Edinburgh, UK

    Michael James Frith

  106. University of Toronto, Montreal, Quebec, Canada

    Thomas Galipeau

  107. UAI Business School, Santiago, Chile

    Sebastian Gallegos

  108. Independent Researcher, Ottawa, Ontario, Canada

    Areez Gangji

  109. Oslo Metropolitan University, Oslo, Norway

    Cloé Garnache & Anders Kjelsrud

  110. ELTE Centre for Economic and Regional Studies, Budapest, Hungary

    Attila Gáspár

  111. Central European University, Vienna, Austria

    Attila Gáspár

  112. NHH Norwegian School of Economics, Bergen, Norway

    Evelina Gavrilova & Erik Ø. Sørensen

  113. University of Galway, Galway, Ireland

    Garreth Gibney

  114. Canadian Research Data Centre Network, McMaster University, Hamilton, Ontario, Canada

    Grant Gibson

  115. State University of New York Geneseo, Geneseo, NY, USA

    Da Gong

  116. Southern Methodist University, Dallas, TX, USA

    Javier González

  117. Behavioural Insights Unit, Ontario Public Service, Toronto, Ontario, Canada

    Jeremy D. Gretton

  118. University of Chile, Santiago, Chile

    Cristina Griffa

  119. The Norwegian Institute of Public Health, Oslo, Norway

    Maja Grøtting

  120. University of Birmingham, Birmingham, UK

    Jiaqi Guo & Anthony Heyes

  121. Vienna University of Economics and Business, Vienna, Austria

    Hooman Habibnia, Jakob Moeller, Julian Quandt, Anna Shcherbiak & Dejan Tatić

  122. Peace Research Institute Oslo, Oslo, Norway

    Sonja Häffner

  123. University of Alabama, Tuscaloosa, AL, USA

    Jonathan D. Hall

  124. Linnaeus University, Växjö, Sweden

    Olle Hammar

  125. Institute for Futures Studies, Stockholm, Sweden

    Olle Hammar

  126. Stanford University, Stanford, CA, USA

    Jonathan S. Hartley

  127. University of Konstanz, Konstanz, Germany

    Carina I. Hausladen

  128. Faculty of International Relations, Prague University of Economics and Business, Prague, Czech Republic

    Tomáš Havránek

  129. University of California San Diego, San Diego, CA, USA

    Harry He, Joel Levin & Tereza Petrovičová

  130. Programa Estado de la Nacion, CREST-Ecole Polytechnique, Palaiseau, France

    Mario Herrera-Rodriguez

  131. Toronto Metropolitan University, Toronto, Ontario, Canada

    Anson T. Y. Ho

  132. California Institute of Technology, Pasadena, CA, USA

    Shiang-Hung Hu

  133. Department of Political Science, National Chengchi University, Taipei, Taiwan

    Yu-Shiuan Huang

  134. Federal Institute for Population Research BiB, Berlin, Germany

    Mathias Huebener

  135. Aalto University, Espoo, Finland

    Christoph Huber

  136. Department of Economics, Indiana University, Bloomington, IN, USA

    Kim P. Huynh

  137. Laboratoire d’Économie d’Orléans, Université d’Orléans, Orléans, France

    Kim P. Huynh

  138. Anglo-American University Prague, Prague, Czech Republic

    Zuzana Irsova

  139. The University of Queensland, St Lucia, Queensland, Australia

    Ozan Isler

  140. Karlstad University, Karlstad, Sweden

    Niklas Jakobsson

  141. FBK-IRVAPP, Trento, Italy

    Niklas Jakobsson

  142. Global Water Security Center, Tuscaloosa, AL, USA

    Tharaka A. Jayalath

  143. University of Western Australia, Perth, Western Australia, Australia

    Michael Jetter

  144. Heriot-Watt University, Edinburgh, UK

    Rachel Joy Forshaw

  145. Howard University, Washington, DC, USA

    Felipe Juan

  146. University of Kassel and INCHER, Kassel, Germany

    Valon Kadriu

  147. University College London, London, UK

    Duy Khanh Hoang Dang

  148. Chinese University of Hong Kong, Hong Kong, China

    Jin Kim

  149. Centre for Health Governance, University of Gothenburg, Gothenburg, Sweden

    Gustav Kjellsson

  150. HEPER School of Public Health and Community Medicine, University of Gothenburg, Gothenburg, Sweden

    Gustav Kjellsson

  151. Ragnar Frisch Centre for Economic Research, Oslo, Norway

    Andreas Kotsadam & Ole Rogeberg

  152. World Inequality Lab, Paris School of Economics, Paris, France

    Suranjana Kundu

  153. University of Notre Dame, Notre Dame, IN, USA

    Alexander Kustov

  154. UC Center Sacramento, University of California Davis, Sacramento, CA, USA

    Jill Laufer

  155. University of Western Ontario, London, Ontario, Canada

    Blake Lee-Whiting

  156. Dresden University of Technology, Dresden, Germany

    Andreas Leibing

  157. University of Bath, Bath, UK

    Peng Li

  158. University of Maryland, College Park, MD, USA

    Ariel Listo & Aqila Putri

  159. Australian National University, Canberra, Australian Capital Territory, Australia

    Dan Liu & Thomas Tao Yang

  160. New Economic School, Moscow, Russia

    Elvina Lukmanova

  161. Government of Canada, Ottawa, Ontario, Canada

    Alex Luscombe

  162. University of Pittsburgh, Pittsburgh, PA, USA

    Lester R. Lusher

  163. University of Nevada Reno, Reno, NV, USA

    Ke Lyu

  164. McGill University, Montreal, Quebec, Canada

    Hai Ma, Ali Ousman & Chris Yurris

  165. Knauss School of Business, University of San Diego, San Diego, CA, USA

    Nicolas Mäder

  166. Norwegian University of Life Sciences, Ås, Norway

    Clifton Makate

  167. University of Cagliari, Cagliari, Italy

    Marco Mandas

  168. Freie Universität Berlin, Berlin, Germany

    Jan Marcus & Lili Márk

  169. University of Potsdam, Potsdam, Germany

    Shushanik Margaryan

  170. University of Nottingham, Nottingham, UK

    Andres Martignano, Georg Sator, Kieran Stockley & Benjamin Tatlow

  171. Finance Canada, Ottawa, Ontario, Canada

    Abigail Marsh

  172. London School of Economics and Political Science, London, UK

    Isabella Masetto, Lennard Metson & Viet Nguyen-Tien

  173. Health Organisation, Policy and Economics Research Group, The University of Manchester, Manchester, UK

    Emma McManus

  174. Krea University, Sri City, India

    Sumit Mishra

  175. School of Politics and International Relations, University of Nottingham, Nottingham, UK

    Scott Moser

  176. University of Texas Rio Grande Valley, Edinburg, TX, USA

    Fabio Yoshio Suguri Motoki

  177. Resources for the Future, Washington, DC, USA

    Lucija Muehlenbachs

  178. IVADO, Montreal, Quebec, Canada

    Andreea Musulan

  179. Mila, Montreal, Quebec, Canada

    Andreea Musulan

  180. University of Padova, Padova, Italy

    Marco Musumeci

  181. Vanderbilt University, Nashville, TN, USA

    Karim Nchare

  182. University of Sussex, Brighton, UK

    Quan M. P. Nguyen

  183. Wake Forest University, Winston-Salem, NC, USA

    Giorgi Nikolaishvili

  184. IWH Halle, Halle, Germany

    Patrick Nüß

  185. Princeton University, Princeton, USA

    Angela Odermatt

  186. University of Pennsylvania Wharton, Chicago, IL, USA

    Matt Olson

  187. Department of Health Management and Health Economics, University of Oslo, Oslo, Norway

    Henning Øien

  188. Humboldt University zu Berlin, Berlin, Germany

    Tim Ölkers

  189. Universitat de Girona, Girona, Spain

    Miquel Oliver i Vert

  190. University of Mannheim, Mannheim, Germany

    Emre Oral

  191. University of the Bundeswehr Munich, Neubiberg, Germany

    Christian Oswald

  192. Department of Economics, Southern Methodist University, Dallas, TX, USA

    Ömer Özak

  193. IZA, Bonn, Germany

    Ömer Özak

  194. GLO, Essen, Germany

    Ömer Özak

  195. Institute of Psychology, Osnabrück University, Osnabrück, Germany

    Shubham Pandey

  196. Asian Development Bank, Manila, Philippines

    Martino Pelli

  197. University of Houston, Houston, TX, USA

    Romeo Penheiro

  198. William and Mary Department of Government, Williamsburg, VA, USA

    RyuGyung Park

  199. Towson University, Towson, MD, USA

    Alexa Prettyman

  200. McMaster University, Hamilton, Ontario, Canada

    Kangyu Qiu & Stephenson Strobel

  201. National Economics University, Hanoi, Vietnam

    Loan Quynh Thi Nguyen

  202. Emory University, Atlanta, GA, USA

    Carson H. Rea

  203. Norwegian School of Economics, Bergen, Norway

    Adam Reiremo

  204. Université de Montréal, Montreal, Quebec, Canada

    Laëtitia Renée

  205. Ministry of Research and Higher Education Luxembourg, Luxembourg, Luxembourg

    Bruno Rodrigues

  206. Berlin School of Economics, Humboldt University of Berlin, Berlin, Germany

    Paul Rosmer

  207. University of Manitoba, Winnipeg, Manitoba, Canada

    Soodeh Saberian

  208. Institute for Advanced Studies Vienna, Vienna, Austria

    Georg Sator

  209. Justus Liebig University Giessen, Giessen, Germany

    Elmar Schlüter

  210. University of Chicago, Chicago, IL, USA

    Ritika Sethi

  211. GATE, Rotman School, University of Toronto, Toronto, Ontario, Canada

    Moyosore Sogaolu

  212. Tufts University, Medford, MA, USA

    Manali Sovani

  213. The Netherlands Court of Audit, The Hague, The Netherlands

    Renske Stans

  214. Institute for Employment Research IAB, Nuremberg, Germany

    Felix Stips

  215. Carleton College, Northfield, MN, USA

    Ethan Struby

  216. Boston College, Chestnut Hill, MA, USA

    Ethan Struby

  217. Minnesota Supercomputing Institute, Minneapolis, MN, USA

    Ethan Struby

  218. Utrecht University, Utrecht, Netherlands

    John P. Tang

  219. New York University, New York, NY, USA

    Rémi Thériault

  220. Bucharest University of Economic Studies, Bucharest, Romania

    Filip-Mihai Toma

  221. Ifo Institute, Ludwig Maximilian University of Munich, Munich, Germany

    Maddalena Totarelli

  222. INSAIT Sofia University, Sofia, Bulgaria

    Nikita Tsoy

  223. Amazon, Seattle, WA, USA

    Kerem Tuzcuoglu

  224. University of Gothenburg, Gothenburg, Sweden

    Julian Walterskirchen

  225. Department of Economics and Taiwan Social Resilience Research Center, National Taiwan University, Taipei, Taiwan

    Joseph Tao-yi Wang

  226. The University of Manchester, Manchester, UK

    Vasudha Wattal

  227. College of Staten Island-CUNY, Staten Island, NY, USA

    Bryan S. Weber

  228. University of the West of England, Bristol, UK

    Reinhard Weisser

  229. Stockholm Business School, Stockholm University, Stockholm, Sweden

    Christian Westheide

  230. Leibniz Institute for Financial Research SAFE, Frankfurt, Germany

    Christian Westheide

  231. KOF Institute, ETH Zurich, Zürich, Switzerland

    Timo Wochner

  232. Colorado State University, Fort Collins, CO, USA

    Matt Woerman

  233. Yale University, New Haven, CT, USA

    Jared Wong

  234. SGH Warsaw School of Economics, Warsaw, Poland

    Marcin Wroński

  235. Harvard University, Cambridge, MA, USA

    Gustav Chung Yang

  236. National University of Singapore, Singapore, Singapore

    Luther Yap

  237. Hitotsubashi University, Tokyo, Japan

    Kareman Yassin

  238. Community for Rigor, University of Pennsylvania, Philadelphia, PA, USA

    Hao Ye

  239. Sanford School of Public Policy, Duke University, Durham, NC, USA

    Jonathan Zhang

  240. Swiss Finance Institute, University of Lausanne, Lausanne, Switzerland

    Ziwei Zhao

Authors

  1. Abel Brodeur
  2. Derek Mikola
  3. Nikolai Cook
  4. Lenka Fiala
  5. Thomas Brailey
  6. Ryan Briggs
  7. Alexandra de Gendre
  8. Yannick Dupraz
  9. Jacopo Gabani
  10. Romain Gauriot
  11. Joanne Haddad
  12. Goncalo Lima
  13. Jörg Ankel-Peters
  14. Anna Dreber
  15. Douglas Campbell
  16. Lamis Kattan
  17. Diego Marino Fages
  18. Fabian Mierisch
  19. Pu Sun
  20. Taylor Wright
  21. Marie Connolly
  22. Fernando Hoces de la Guardia
  23. Magnus Johannesson
  24. Edward Miguel
  25. Lars Vilhuber
  26. Alejandro Abarca
  27. Mahesh Acharya
  28. Sossou Simplice Adjisse
  29. Ahwaz Akhtar
  30. Eduardo Alberto Ramirez Lizardi
  31. Sabina Albrecht
  32. Synøve Nygaard Andersen
  33. Zubaria Andlib
  34. Falak Arrora
  35. Thomas Ash
  36. Etienne Bacher
  37. Sebastian Bachler
  38. Félix Bacon
  39. Manuel Bagues
  40. Timea Balogh
  41. Alisher Batmanov
  42. Mara Barschkett
  43. Barış Kaan Basdil
  44. Jaromír Baxa
  45. Sascha O. Becker
  46. Monica Beeder
  47. Louis-Philippe Beland
  48. Abdel-Hamid Bello
  49. Daniel Benenson Markovits
  50. Grant Benjamin
  51. Thomas Bergeron
  52. Moussa P. Blimpo
  53. Marco Binetti
  54. Carl Bonander
  55. Joseph Bonneau
  56. Endre Borbáth
  57. Nicolai Borgen
  58. Solveig Topstad Borgen
  59. Jonathan Borowsky
  60. Elisa Brini
  61. Myriam Brown
  62. Martin Brun
  63. Stephan Bruns
  64. Nino Buliskeria
  65. Andrea Calef
  66. Alistair Cameron
  67. Pamela Campa
  68. Santiago Campos-Rodríguez
  69. Giulio Giacomo Cantone
  70. Fenella Carpena
  71. Perry Jess Carter
  72. Paul Castañeda Dower
  73. Ondrej Castek
  74. Jill Caviglia-Harris
  75. Gabriella Chauca Strand
  76. Shi Chen
  77. Sya In Chzhen
  78. Jong Chung
  79. Jason Collins
  80. Alexander Coppock
  81. Hugo Cordeau
  82. Ben Couillard
  83. Jonathan Crechet
  84. Lorenzo Crippa
  85. Jing Cui
  86. Christian Czymara
  87. Haley Daarstad
  88. Danh Chi Dao
  89. Daniel Dao
  90. Marco David Schmandt
  91. Astrid de Linde
  92. Lucas De Melo
  93. Lachlan Deer
  94. Micole De Vera
  95. Velichka Dimitrova
  96. Jan Fabian Dollbaum
  97. Jan Matti Dollbaum
  98. Michael Donnelly
  99. Luu Duc Toan Huynh
  100. Tsvetomira Dumbalska
  101. Jamie Duncan
  102. Kiet Tuan Duong
  103. Thibaut Duprey
  104. Christoph Dworschak
  105. Sigmund Ellingsrud
  106. Ali Elminejad
  107. Yasmine Eissa
  108. Andrea Erhart
  109. Giulian Etingin-Frati
  110. Elaheh Fatemipour
  111. Alexa Federice
  112. Jan Feld
  113. Guidon Fenig
  114. Mojtaba Firouzjaeiangalougah
  115. Erlend Fleisje
  116. Alexandre Fortier-Chouinard
  117. Julia Francesca Engel
  118. Nadjim Fréchet
  119. Reid Fortier
  120. Tilman Fries
  121. Michael James Frith
  122. Thomas Galipeau
  123. Sebastian Gallegos
  124. Areez Gangji
  125. Xiaoying Gao
  126. Cloé Garnache
  127. Attila Gáspár
  128. Evelina Gavrilova
  129. Arijit Ghosh
  130. Garreth Gibney
  131. Grant Gibson
  132. Geir Godager
  133. Leonard Goff
  134. Da Gong
  135. Javier González
  136. Jeremy D. Gretton
  137. Cristina Griffa
  138. Idaliya Grigoryeva
  139. Maja Grøtting
  140. Eric Guntermann
  141. Jiaqi Guo
  142. Alexi Gugushvili
  143. Hooman Habibnia
  144. Sonja Häffner
  145. Jonathan D. Hall
  146. Olle Hammar
  147. Amund Hanson Kordt
  148. Barry Hashimoto
  149. Jonathan S. Hartley
  150. Carina I. Hausladen
  151. Tomáš Havránek
  152. Harry He
  153. Matthew Hepplewhite
  154. Mario Herrera-Rodriguez
  155. Felix Heuer
  156. Anthony Heyes
  157. Anson T. Y. Ho
  158. Jonathan Holmes
  159. Armando Holzknecht
  160. Yu-Hsiang Dexter Hsu
  161. Shiang-Hung Hu
  162. Yu-Shiuan Huang
  163. Mathias Huebener
  164. Christoph Huber
  165. Kim P. Huynh
  166. Zuzana Irsova
  167. Ozan Isler
  168. Niklas Jakobsson
  169. Raphaël Jananji
  170. Tharaka A. Jayalath
  171. Michael Jetter
  172. Jenny John
  173. Rachel Joy Forshaw
  174. Felipe Juan
  175. Valon Kadriu
  176. Sunny Karim
  177. Edmund Kelly
  178. Duy Khanh Hoang Dang
  179. Tazia Khushboo
  180. Jin Kim
  181. Gustav Kjellsson
  182. Anders Kjelsrud
  183. Andreas Kotsadam
  184. Jori Korpershoek
  185. Lewis Krashinsky
  186. Suranjana Kundu
  187. Alexander Kustov
  188. Nurlan Lalayev
  189. Audrée Langlois
  190. Jill Laufer
  191. Blake Lee-Whiting
  192. Andreas Leibing
  193. Gabriel Lenz
  194. Joel Levin
  195. Peng Li
  196. Tongzhe Li
  197. Yuchen Lin
  198. Ariel Listo
  199. Dan Liu
  200. Xuewen Lu
  201. Elvina Lukmanova
  202. Alex Luscombe
  203. Lester R. Lusher
  204. Ke Lyu
  205. Hai Ma
  206. Nicolas Mäder
  207. Clifton Makate
  208. Alice Malmberg
  209. Adit Maitra
  210. Marco Mandas
  211. Jan Marcus
  212. Shushanik Margaryan
  213. Lili Márk
  214. Andres Martignano
  215. Abigail Marsh
  216. Isabella Masetto
  217. Anthony McCanny
  218. Emma McManus
  219. Ryan McWay
  220. Lennard Metson
  221. Jonas Minet Kinge
  222. Sumit Mishra
  223. Myra Mohnen
  224. Jakob Moeller
  225. Rosalie Montambeault
  226. Sébastien Montpetit
  227. Louis-Philippe Morin
  228. Todd Morris
  229. Scott Moser
  230. Fabio Yoshio Suguri Motoki
  231. Lucija Muehlenbachs
  232. Andreea Musulan
  233. Marco Musumeci
  234. Munirul Nabin
  235. Karim Nchare
  236. Florian Neubauer
  237. Quan M. P. Nguyen
  238. Tuan Nguyen
  239. Viet Nguyen-Tien
  240. Ali Niazi
  241. Giorgi Nikolaishvili
  242. Ardyn Nordstrom
  243. Patrick Nüß
  244. Angela Odermatt
  245. Matt Olson
  246. Henning Øien
  247. Tim Ölkers
  248. Miquel Oliver i Vert
  249. Emre Oral
  250. Christian Oswald
  251. Ali Ousman
  252. Ömer Özak
  253. Shubham Pandey
  254. Alexandre Pavlov
  255. Martino Pelli
  256. Romeo Penheiro
  257. RyuGyung Park
  258. Eva Pérez Martel
  259. Tereza Petrovičová
  260. Linh Phan
  261. Alexa Prettyman
  262. Jakub Procházka
  263. Aqila Putri
  264. Julian Quandt
  265. Kangyu Qiu
  266. Loan Quynh Thi Nguyen
  267. Andaleeb Rahman
  268. Carson H. Rea
  269. Adam Reiremo
  270. Laëtitia Renée
  271. Joseph Richardson
  272. Nicholas Rivers
  273. Bruno Rodrigues
  274. William Roelofs
  275. Tobias Roemer
  276. Ole Rogeberg
  277. Julian Rose
  278. Andrew Roskos-Ewoldsen
  279. Paul Rosmer
  280. Barbara Sabada
  281. Soodeh Saberian
  282. Nicolas Salamanca
  283. Georg Sator
  284. Daniel Scates
  285. Elmar Schlüter
  286. Cameron Sells
  287. Sharmi Sen
  288. Ritika Sethi
  289. Anna Shcherbiak
  290. Moyosore Sogaolu
  291. Matt Soosalu
  292. Erik Ø. Sørensen
  293. Manali Sovani
  294. Noah Spencer
  295. Stefan Staubli
  296. Renske Stans
  297. Anya Stewart
  298. Felix Stips
  299. Kieran Stockley
  300. Stephenson Strobel
  301. Ethan Struby
  302. John P. Tang
  303. Idil Tanrisever
  304. Thomas Tao Yang
  305. Ipek Tastan
  306. Dejan Tatić
  307. Benjamin Tatlow
  308. Féraud Tchuisseu Seuyong
  309. Rémi Thériault
  310. Vincent Thivierge
  311. Wenjie Tian
  312. Filip-Mihai Toma
  313. Maddalena Totarelli
  314. Van-Anh Tran
  315. Hung Truong
  316. Nikita Tsoy
  317. Kerem Tuzcuoglu
  318. Diego Ubfal
  319. Laura Villalobos
  320. Julian Walterskirchen
  321. Joseph Tao-yi Wang
  322. Vasudha Wattal
  323. Matthew D. Webb
  324. Bryan S. Weber
  325. Reinhard Weisser
  326. Wei-Chien Weng
  327. Christian Westheide
  328. Kimberly White
  329. Jacob Winter
  330. Timo Wochner
  331. Matt Woerman
  332. Jared Wong
  333. Ritchie Woodard
  334. Marcin Wroński
  335. Myra Yazbeck
  336. Gustav Chung Yang
  337. Luther Yap
  338. Kareman Yassin
  339. Hao Ye
  340. Jin Young Yoon
  341. Chris Yurris
  342. Tahreen Zahra
  343. Mirela Zaneva
  344. Aline Zayat
  345. Jonathan Zhang
  346. Ziwei Zhao
  347. Yaolang Zhong

Contributions

Preparation of tables, figures and manuscript: A. Brodeur, N. Cook, D. Mikola and L. Fiala. Conception or design of the work: J. Ankel-Peters, A. Brodeur, M. Connolly, N. Cook, A. Dreber, F. Hoces de la Guardia, M. Johannesson, E. Miguel, D. Mikola and L. Vilhuber. Analysis or interpretation of the reproducibility data: T. Brailey, R. Briggs, A. Brodeur, N. Cook, A. de Gendre, Y. Dupraz, J. Gabani, R. Gauriot, G. Lima and D. Mikola. Analysis or interpretation of data and generating data and conception of a reproduction: D. Campbell, N. Cook, J. Haddad, L. Kattan, D. M. Fages, F. Mierisch, P. Sun, T. Wright, A. Abarca, M. Acharya, S. S. Adjisse, A. Akhtar, E. A. Ramirez Lizardi, S. Albrecht, S. Nygaard Andersen, Z. Andlib, F. Arrora, T. Ash, E. Bacher, S. Bachler, F. Bacon, M. Bagues, T. Balogh, A. Batmanov, M. Barschkett, B. K. Basdil, J. Baxa, S. Becker, M. Beeder, L.-P. Beland, A.-H. Bello, D. B. Markovits, G. Benjamin, T. Bergeron, M. P. Blimpo, M. Binetti, C. Bonander, J. Bonneau, E. Borbáth, N. Borgen, S. T. Borgen, J. Borowsky, T. Brailey, R. Briggs, E. Brini, M. Brown, M. Brun, S. Bruns, N. Buliskeria, A. Calef, A. Cameron, P. Campa, S. Campos-Rodríguez, G. G. Cantone, F. Carpena, P. Carter, P. Castañeda Dower, O. Castek, J. Caviglia-Harris, G. C. Strand, S. Chen, S. I. Chzhen, J. Chung, J. Collins, A. Coppock, H. Cordeau, B. Couillard, J. Crechet, L. Crippa, J. Cui, C. Czymara, H. Daarstad, D. C. Dao, D. Dao, M. D. Schmandt, A. de Linde, L. De Melo, L. Deer, A. de Gendre, M. De Vera, V. Dimitrova, J. F. Dollbaum, J. M. Dollbaum, M. Donnelly, L. D. Toan Huynh, T. Dumbalska, J. Duncan, K. T. Duong, Y. Dupraz, T. Duprey, C. Dworschak, S. Ellingsrud, A. Elminejad, Y. Eissa, A. Erhart, G. Etingin-Frati, E. Fatemipour, A. Federice, J. Feld, G. Fenig, L. Fiala, M. Firouzjaeiangalougah, E. Fleisje, A. Fortier-Chouinard, J. F. Engel, N. Fréchet, R. Fortier, T. Fries, M. J. Frith, J. Gabani, T. Galipeau, S. Gallegos, A. Gangji, X. Gao, C. Garnache, A. Gáspár, R. Gauriot, E. Gavrilova, A. Ghosh, G. Gibney, G. Gibson, G. Godager, L. Goff, D. Gong, J. González, J. D. Gretton, C. Griffa, I. Grigoryeva, M. Grøtting, E. Guntermann, J. Guo, A. Gugushvili, H. Habibnia, S. Häffner, J. D. Hall, O. Hammar, A. H. Kordt, B. Hashimoto, J. S. Hartley, C. I. Hausladen, T. Havránek, H. He, M. Hepplewhite, M. Herrera-Rodriguez, F. Heuer, A. Heyes, A. T. Y. Ho, J. Holmes, A. Holzknecht, Y.-H. D. Hsu, S.-H. Hu, Y.-S. Huang, M. Huebener, C. Huber, K. P. Huynh, Z. Irsova, O. Isler, N. Jakobsson, R. Jananji, T. A. Jayalath, M. Jetter, J. John, R. J. Forshaw, F. Juan, V. Kadriu, S. Karim, E. Kelly, D. K. H. Dang, T. Khushboo, J. Kim, G. Kjellsson, A. Kjelsrud, J. Korpershoek, A. Kotsadam, L. Krashinsky, S. Kundu, A. Kustov, N. Lalayev, A. Langlois, J. Laufer, B. Lee-Whiting, A. Leibing, G. Lenz, J. Levin, P. Li, T. Li, Y. Lin, G. Lima, A. Listo, D. Liu, X. Lu, E. Lukmanova, A. Luscombe, L. R. Lusher, K. Lyu, H. Ma, N. Mäder, C. Makate, A. Malmberg, A. Maitra, M. Mandas, J. Marcus, S. Margaryan, L. Márk, D. M. Fages, A. Martignano, A. Marsh, I. Masetto, A. McCanny, E. McManus, R. McWay, L. Metson, F. Mierisch, J. M. Kinge, S. Mishra, M. Mohnen, J. Möller, R. Montambeault, S. Montpetit, L.-P. Morin, T. Morris, S. Moser, F. Y. S. Motoki, L. Muehlenbachs, A. Musulan, M. Musumeci, M. Nabin, K. Nchare, F. Neubauer, Q. M. P. Nguyen, T. Nguyen, V. Nguyen-Tien, A. Niazi, G. Nikolaishvili, A. Nordstrom, P. Nüß, A. Odermatt, M. Olson, H. Øien, T. Ölkers, M. Oliver i Vert, E. Oral, C. Oswald, A. Ousman, Ö. Özak, S. Pandey, A. Pavlov, M. Pelli, R. Penheiro, R. Park, E. Pérez Martel, J. Ankel-Peters, T. Petrovičová, L. Phan, A. Prettyman, J. Procházka, A. Putri, J. Quandt, K. Qiu, L. Q. T. Nguyen, A. Rahman, C. H. Rea, A. Reiremo, L. Renée, J. Richardson, N. Rivers, B. Rodrigues, W. Roelofs, T. Roemer, O. Rogeberg, J. Rose, A. Roskos-Ewoldsen, P. Rosmer, B. Sabada, S. Saberian, N. Salamanca, G. Sator, D. Scates, E. Schlüter, C. Sells, S. Sen, R. Sethi, A. Shcherbiak, M. Sogaolu, M. Soosalu, E. Ø. Sørensen, M. Sovani, N. Spencer, S. Staubli, R. Stans, A. Stewart, F. Stips, K. Stockley, S. Strobel, E. Struby, J. Tang, I. Tanrisever, T. T. Yang, I. Tastan, D. Tatić, B. Tatlow, F. T. Seuyong, R. Thériault, V. Thivierge, W. Tian, F.-M. Toma, M. Totarelli, V.-A. Tran, H. Truong, N. Tsoy, K. Tuzcuoglu, D. Ubfal, L. Villalobos, J. Walterskirchen, J. T. Wang, V. Wattal, M. D. Webb, B. Weber, R. Weisser, W.-C. Weng, C. Westheide, K. White, J. Winter, T. Wochner, M. Woerman, J. Wong, R. Woodard, M. Wroński, G. C. Yang, M. Yazbeck, L. Yap, K. Yassin, H. Ye, J. Y. Yoon, C. Yurris, T. Zahra, M. Zaneva, A. Zayat, J. Zhang, Z. Zhao and Y. Zhong. Computational reproducibility: A. Brodeur, J. Haddad and P. Sun. Local organizer replication games: M. Connolly, R. Gauriot, L. Goff, C. Huber, A. Kotsadam and D. M. Fages.

Corresponding author

Correspondence to Abel Brodeur.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature thanks Colin Camerer who co-reviewed with Anastasia Buyalskaya; T. D. Stanley; and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer review reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Ten-point computationally reproducibility score.

Each team assigned a reproducibility score on a scale of one to ten to the paper reproduced. See Supplementary Materials for a description of each score. Level 10 (L10) means that all necessary materials are available and produce consistent results with those presented in the paper, while level 5 (L5) means that analytic data sets and analysis code are available and they produce the same results as presented in the paper.

Extended Data Fig. 2 Reasons for selecting paper? (Select all which apply).

Data collected via survey of our reproducers after completing their reports. This figure illustrates the responses to the question: ‘For what reasons did you select your specific paper to reproduce and/or replicate from the list of papers provided?

Extended Data Fig. 3 Percentage of papers with a replication folder.

The total sample is 1150 papers with 120 papers per year from 2019 to 2023 and 110 papers per year from 2018 to 2014. Each journal has 10 papers per year except American Economic Review: Insights which only formally became a journal in 2019 (and are omitted in earlier years). The journals sampled over correspond to those used in the manuscript’s main analysis, three from political science and nine from economics. The political science journals include: American Journal of Political Science, American Political Science Review, and Journal of Politics. The economics journals include: American Economic Review, Quarterly Journal of Economics, Review of Economic Studies, Journal of Political Economy, American Economic Journal: Macroeconomics, American Economic Journal: Applied Economics, American Economic Journal: Economic Policy, American Economic Review: Insights, Economic Journal.

Extended Data Fig. 4 Percentage of papers with a replication folder by discipline.

Panel (a) is for papers published in economics journals where Panel (b) is for papers published in political science. The total sample is the same as Extended Data Fig. 3 is 1150 papers, where 850 papers are in the economics sample and 300 papers are in the political science sample.

Extended Data Fig. 5 Percentage replication folders’ with contents conditional on they should have a replication folder.

Each subfigure represents the proportion of the replication folders which affirmatively (‘Yes’) contained the variable (displayed as the title). The ‘Not Yes’ in the legend corresponds to those replication folders which did not affirm (‘No’) or had only ‘Some’ of the required contents. Each sample is over those observations where categories are applicable (i.e. not all replication packages require the same contents).

Extended Data Fig. 6 Percentage replication folders’ with contents conditional on they should have a replication folder.

Each subfigure represents the proportion of the replication folders which affirmatively (‘Yes’) contained the variable (displayed as the title). The ‘Not Yes’ in the legend corresponds to those replication folders which did not affirm (‘No’) or had only ‘Some’ of the required contents. Each sample is over those observations where categories are applicable (i.e. not all replication packages require the same contents).

Extended Data Fig. 7 Reasons unable to conduct robustness checks.

This Figure illustrates the share of teams who were unable to perform robustness checks (top-left), replications (top-right), key variable recodes (bottom-right) or extensions (bottom-left) for various reasons represented by the different coloured bars.

Extended Data Fig. 8 Distributions of t-statistics for original studies and re-analyses.

The top panels display a histogram of test statistics for t ∈ [0, 5], with bins of width 0.1. The top left panel includes all original studies in our data set. The top right panel includes all re-analysis estimates in our data set. Vertical reference lines are displayed at conventional two-tailed significance levels. We superimpose an Epanechnikov kernel (which includes renormalization at 0). The bottom figures display histograms of test statistics for p-values ∈ [0.0025, 0.1500], with bins of width 0.0025, among original studies and those from re-analyses, respectively.

Extended Data Fig. 9 Distributions of t-statistics and P-values by field.

We restrict the sample to articles published in the indicated field journals. Top panels display histograms of test statistics for t ∈ [0, 5], with bins of width 0.1 respectively. Vertical reference lines are displayed at conventional two-tailed significance levels. We superimpose an Epanechnikov kernel density curve (which includes renormalization at 0). Bottom panels display histograms of test statistics for p-values ∈ [0.0025, 0.1500], with bins of width 0.0025.

Extended Data Fig. 10 Relative reproduced effect size.

48% of relative effect sizes are exactly equal to or greater than 1. This figure illustrates the ratio of re-analysis estimates and original estimates. The standardized effect sizes are normalized so that 1 equals the original effect size. A positive value indicates that the re-analysis estimate is in the same direction as in the original study. A negative value indicates that the re-analysis estimate is not in the same direction as in the original study. Outliers (3%) are excluded for visibility.

Supplementary information

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brodeur, A., Mikola, D., Cook, N. et al. Reproducibility and robustness of economics and political science research. Nature 652, 151–156 (2026). https://doi.org/10.1038/s41586-026-10251-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41586-026-10251-x