A benchmark of expert-level academic questions to assess AI capabilities

The HLE dataset is open-source and available at https://huggingface.co/datasets/cais/hle. Important updates to the project and dataset will be announced at https://lastexam.ai.

The inference script for benchmarking AI systems on HLE is available at GitHub (https://github.com/centerforaisafety/hle).

1. Hendrycks, D. et al. Measuring massive multitask language understanding. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=d7KBjmI3GmQ (ICLR, 2021).

2. Gemini Team Google. Gemini 1.5: unlocking multimodal understanding across millions of tokens of context. Preprint at https://arxiv.org/abs/2403.05530 (2024).

3. OpenAI et al. GPT-4 technical report. Preprint at https://arxiv.org/abs/2303.08774 (2024).

4. The Claude 3 Model Family: Opus, Sonnet, Haiku (Anthropic, 2024).

5. OpenAI o1 System Card (OpenAI, 2024).

6. Guo, D. et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 645, 633–638 (2025).

   Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

7. Wei, J. et al. Measuring short-form factuality in large language models. Preprint at https://arxiv.org/abs/2411.04368 (2024).

8. Hendrycks, D. et al. PixMix: Dreamlike pictures comprehensively improve safety measures. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16783–16792 (IEEE/CVF, 2022).

9. Rein, D. et al. GPQA: A graduate-level Google-proof Q&A benchmark. In Proc. First Conference on Language Modeling (COLM) https://openreview.net/forum?id=Ti67584b98 (COLM, 2024).

10. Chollet, F., Knoop, M., Kamradt, G. & Landers, B. ARC prize 2024: technical report. Preprint at https://arxiv.org/abs/2412.04604 (2024).

11. Hendrycks, D. et al. A definition of AGI. Preprint at https://arxiv.org/abs/2510.18212 (2025).

12. Li, N. et al. The WMDP benchmark: measuring and reducing malicious use with unlearning. In Proc. 41st International Conference on Machine Learning (ICML), 28713–28738 (PMLR, 2024).

13. Laurent, J. M. et al. LAB-bench: measuring capabilities of language models for biology research. Preprint at https://arxiv.org/abs/2407.10362 (2024).

14. Srivastava, A. et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res. https://openreview.net/forum?id=uyTL5Bvosj (2023).

15. Zhong, W. et al. Agieval: a human-centric benchmark for evaluating foundation models. Preprint at https://arxiv.org/abs/2304.06364 (2023).

16. Hendrycks, D. et al. Measuring mathematical problem solving with the MATH dataset. In Proc. 35th Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track https://openreview.net/forum?id=7Bywt2mQsCe (NeurIPS, 2021).

17. Lu, P. et al. MathVista: Evaluating mathematical reasoning of foundation models in visual contexts. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=KUNzEQMWU7 (ICLR, 2024).

18. Cobbe, K. et al. Training verifiers to solve math word problems. Preprint at https://arxiv.org/abs/2110.14168 (2021).

19. Glazer, E. et al. FrontierMath: a benchmark for evaluating advanced mathematical reasoning in AI. Preprint at https://arxiv.org/abs/2411.04872 (2024).

20. He, C. et al. OlympiadBench: A challenging benchmark for promoting AGI with Olympiad-level bilingual multimodal scientific problems. In Proc. 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 3828–3850 (ACL, 2024).

21. Gao, B. et al. Omni-MATH: A universal Olympiad level mathematic benchmark for large language models. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=yaqPf0KAlN (ICLR, 2025).

22. Chan, J. S. et al. MLE-bench: Evaluating machine learning agents on machine learning engineering. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=6s5uXNWGIh (ICLR, 2025).

23. Zhang, A. K. et al. Cybench: A framework for evaluating cybersecurity capabilities and risks of language models. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=tc90LV0yRL (ICLR, 2025).

24. Jimenez, C. E. et al. SWE-bench: Can language models resolve real-world Github issues? In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=VTF8yNQM66 (ICLR, 2024).

25. Chen, M. et al. Evaluating large language models trained on code. Preprint at https://arxiv.org/abs/2107.03374 (2021).

26. Hendrycks, D. et al. Measuring coding challenge competence with APPS. In Proc. 35th Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track https://openreview.net/forum?id=sD93GOzH3i5 (NeurIPS, 2021).

27. Bhatt, M. et al. Purple Llama CyberSecEval: a secure coding benchmark for language models. Preprint at https://arxiv.org/abs/2312.04724 (2023).

28. Austin, J. et al. Program synthesis with large language models. Preprint at https://arxiv.org/abs/2108.07732 (2021).

29. Bai, Y. et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. Preprint at https://arxiv.org/abs/2204.05862 (2022).

30. Perez, E. et al. Discovering language model behaviors with model-written evaluations. In Findings of the Association for Computational Linguistics: ACL 2023, 13387–13434 (ACL, 2023).

31. Rajpurkar, P. et al. SQuAD: 100,000+ questions for machine comprehension of text. In Proc. 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2383–2392 (EMNLP, 2016).

32. Rajpurkar, P. et al. Know what you don’t know: Unanswerable questions for SQuAD. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (ACL), 784–789 (ACL, 2018).

33. Bajaj, P. et al. MS MACRO: a human generated machine reading comprehension dataset. Preprint at https://arxiv.org/abs/1611.09268 (2018).

34. Hendrycks, D. et al. What would Jiminy Cricket do? Towards agents that behave morally. In Proc. 35th Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track https://openreview.net/forum?id=G1muTb5zuO7 (NeurIPS, 2021).

35. Phan, L., Mazeika, M., Zou, A. & Hendrycks, D. Textquests: how good are LLMs at text-based video games? Preprint at https://arxiv.org/abs/2507.23701 (2025).

36. Wang, A. et al. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=rJ4km2R5t7 (ICLR, 2019).

37. Wang, A. et al. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In Proc. Advances in Neural Information Processing Systems (NeurIPS), Vol. 32, 3261–3275 (NeurIPS, 2019).

38. Yang, Z. et al. HotpotQA: A dataset for diverse, explainable multihop question answering. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2369–2380 (EMNLP, 2018).

39. Dua, D. et al. DROP: a reading comprehension benchmark requiring discrete reasoning over paragraphs. Preprint at https://arxiv.org/abs/1903.00161 (2019).

40. Ott, S., Barbosa-Silva, A., Blagec, K., Brauner, J. & Samwald, M. Mapping global dynamics of benchmark creation and saturation in artificial intelligence. Nat. Commun. 13, 6793 (2022).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

41. Owen, D. How predictable is language model benchmark performance? Preprint at https://arxiv.org/abs/2401.04757 (2024).

42. Kiela, D. et al. Dynabench: Rethinking benchmarking in NLP. In Proc. 2021 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 4110–4124 (NAACL, 2021).

43. McIntosh, T. R. et al. Inadequacies of large language model benchmarks in the era of generative artificial intelligence. IEEE Trans. Artif. Intell. https://doi.org/10.1109/TAI.2025.3569516 (2025).

44. Wang, Y. et al. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. In Proc. Advances in Neural Information Processing Systems (NeurIPS), article no. 3018 (NeurIPS, 2024).

45. Taghanaki, S. A., Khani, A. & Khasahmadi, A. MMLU-Pro+: evaluating higher-order reasoning and shortcut learning in LLMS. Preprint at https://arxiv.org/abs/2409.02257 (2024).

46. Yao, S. et al. τ-bench: A benchmark for tool-agent-user interaction in real-world domains. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=roNSXZpUDN (ICLR, 2025).

47. Andriushchenko, M. et al. AgentHarm: A benchmark for measuring harmfulness of LLM agents. In Proc. International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=AC5n7xHuR1 (ICLR, 2025).

48. Kumar, P. et al. Refusal-trained LLMS are easily jailbroken as browser agents. Preprint at https://arxiv.org/abs/2410.13886 (2024).

49. Yan, F. et al. Berkeley Function Calling Leaderboard https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html (2024).

50. Srinivasan, V. K. et al. NexusRaven: a commercially-permissive language model for function calling. In NeurIPS 2023 Foundation Models for Decision Making Workshop (NeurIPS, 2023).

51. Hosseini, A., Sordoni, A., Toyama, D., Courville, A. & Agarwal, R. Not all LLM reasoners are created equal. Preprint at https://arxiv.org/abs/2410.01748 (2024).

52. Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

53. Nie, Y. et al. Adversarial NLI: A new benchmark for natural language understanding. In Proc. 58th Annual Meeting of the Association for Computational Linguistics (ACL), 4885–4901 (ACL, 2020).

54. Götting, J. et al. Virology capabilities test (VCT): a multimodal virology Q&A benchmark. Preprint at https://arxiv.org/abs/2504.16137 (2025).

55. Phuong, M. et al. Evaluating frontier models for dangerous capabilities. Preprint at https://arxiv.org/abs/2403.13793 (2024).

56. Anthropic’s Responsible Scaling Policy Updates https://www.anthropic.com/rsp-updates (Anthropic, 2024).

57. Mazeika, M. et al. Remote labor index: measuring AI automation of remote work. Preprint at https://arxiv.org/abs/2510.26787 (2025).

58. Patwardhan, T. et al. GDPval: evaluating ai model performance on real-world economically valuable tasks. Preprint at https://arxiv.org/abs/2510.04374 (2025).

59. Kwiatkowski, T. et al. Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguist. 7, 452–466 (2019).

    Google Scholar 

60. Antol, S. et al. VQA: Visual question answering. In Proc. IEEE International Conference on Computer Vision (ICCV), 2425–2433 (IEEE, 2015).

61. Reddy, S., Chen, D. & Manning, C. D. CoQA: a conversational question answering challenge. Trans. Assoc. Comput. Linguist. 7, 249–266 (2019).

    Article  Google Scholar 

62. Bowman, S. R., Angeli, G., Potts, C. & Manning, C. D. A large annotated corpus for learning natural language inference. In Proc. 2015 Conference on Empirical Methods in Natural Language Processing (eds Màrquez, L. et al.), 632–642 (ACL, 2015).

63. Skarlinski, M., Laurent, J., Bou, A. & White, A. About 30% of Humanity’s Last Exam chemistry/biology answers are likely wrong. FutureHouse https://www.futurehouse.org/research-announcements/hle-exam (2025).

64. Arora, R. K. et al. HealthBench: evaluating large language models towards improved human health. Preprint at https://arxiv.org/abs/2505.08775 (2025).

Download references

Authors and Affiliations

1. Center for AI Safety, San Francisco CA, USA

   Long Phan, Alice Gatti, Nathaniel Li, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Dan Hendrycks & Arunim Agarwal

2. Scale AI, San Francisco CA, USA

   Ziwen Han, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Aakaash Nattanmai, Gordon McKellips, Anish Cheraku, Asim Suhail, Ethan Luo, Marvin Deng, Jason Luo, Ashley Zhang, Kavin Jindel, Jay Paek, Kasper Halevy, Allen Baranov, Michael Liu, Advaith Avadhanam, David Zhang, Vincent Cheng, Brad Ma, Evan Fu, Liam Do, Joshua Lass, Hubert Yang, Surya Sunkari, Vishruth Bharath, Violet Ai, James Leung, Rishit Agrawal, Alan Zhou, Kevin Chen, Tejas Kalpathi, Ziqi Xu, Gavin Wang, Tyler Xiao, Erik Maung, Sam Lee, Ryan Yang, Roy Yue, Ben Zhao, Julia Yoon, Xiangwan Sun, Aryan Singh, Clark Peng, Tyler Osbey, Taozhi Wang, Daryl Echeazu, Timothy Wu, Spandan Patel, Vidhi Kulkarni, Vijaykaarti Sundarapandiyan, Andrew Le, Zafir Nasim, Srikar Yalam, Ritesh Kasamsetty, Soham Samal, David Sun, Nihar Shah, Abhijeet Saha, Alex Zhang, Leon Nguyen, Laasya Nagumalli, Kaixin Wang, Aidan Wu, Anwith Telluri, Summer Yue & Alexandr Wang

3. Independent Researcher https://lastexam.ai

   Dmitry Dodonov, Daron Anderson, Mikhail Doroshenko, Alun Cennyth Stokes, Michael Yu, Varun Gangal, Alvaro Sanchez, Fabian Giska, Gashaw M. Goshu, Zachary Giboney, Mark Nandor, Edwin Taylor, John B. Wydallis, Kengo Zenitani, Natanael Wildner Fraga, Gözdenur Demir, Dakotah Martinez, Ben Pageler, Sean R. Green, Alexei Kopylov, Benjamin Myklebust, Qiaochu Yuan, Anmol Sahu, James Koppel, Ali Dehghan, Sergey Ivanov, Vladislav Poritski, Michael Foster, Linh Ho, Aleksey Kuchkin, Alexandra Rodriguez-Romero, Keith Schneider, Piotr Padlewski, Stanislaw Barzowski, Hailey Schoelkopf, Martin Stehberger, David M. Cunningham, Vladimir Goryachev, Sangwon Lee, Sandy Zhao, Adithya Shenoy, Shreyas Verma, Himanshu Narayan, Aline Menezes, William Alley, Freddie Martin, Fereshteh Kazemi, Biró Bálint, Behzad Ansarinejad, Guillaume Douville, Hew Wolff, Murat Eron, Sherwin Abdoli, Jacob Drori, Jonathan Eicher, Jason Gross, Rohan Pandey, Ilya Gusev, Adam Jones, Kostiantyn Dobarskyi, Roman Leventov, Michelle X. Yuan, Katarzyna Olszewska, Timothy Manik, Hector Haffenden, Bita Golshani, Nick Winter, Dustin Wehr, Shaun Phillips, Fredrik Ekström, Angela Hammon, George Medley, Forough Mohammadzadeh, Eric Hallman, Mike Battaglia, Dave Hulbert,  Handoko, Anton Peristyy, Frank Reidegeld, Wanyoung Kim, Mariana Costa, Chiara Ceconello, Chao Zhuang, Jainam Shah, Lixin Zhang, Abram Jackson, Jesus Colino, Vladimir Vinnikov, Fatimah Adesanya, Julien Degorre, Gbenga Daniel Obikoya, Oleg Shumar, Thomas C. H. Lux, Ben Rank, Matthew Brooks, Stefano Cavalleri, Jules Robins, Brad Raynor, Blake Sims, Rebeka Plecnik, Omer Faruk Bodur, D. P. Shinde, Tania C. B. Santos, Nicholas Farina, Hodjat Mariji, Rasoul Pouriamanesh, Ross Finocchio, Danyelle Ferreira, Siriphan Arthornthurasuk, Isaac C. McAlister, Angel Ramirez-Trinidad, Samuel Perry, Raúl Adrián Huerta Rodrguez, Maja Somrak, Eric Vergo, Antoine Jallon, I. M. J. McInnis, Luk Gloor, Samuel Albanie, Warren S. Vaz, Yiyang Fan, Marcus Abramovitch, Juan Gonzalez, Daniel Bugas, Nasser Heydari, Ferenc Jeanplong, Antonella Pinto, Joan of Arc Xavier, Kanu Priya Agarwal, Gang Zhang, Dmitry Malishev, Stephen Mensah, Wiktor Morak, Javier Gimenez, Khalida Meer, Xavier Alapont, David Outevsky, Abdelkader Dendane, Priti Shukla, Tony Fruhauff, Glen Sherman, Dylan Ler, Innocent Enyekwe, Jiang Muzhen, Aleksandr Maksapetyan, Vivien Rossbach, Mohsen Bahaloohoreh, Jasdeep Sidhu, John Lai, Justine Leon Uro, Greg Bateman, Mohamed Sayed, Ashley Aaron, Murat Tiryakioglu, Sheeshram Siddh, Keith Krenek, Jun Jin, Scott Creighton, Ragavendran P V, Michael Richmond, Sergey Bogdanik, Nitin Chandok, Genta Indra Winata, Suchandra Datta, Bonan Pu, Adam Wecker, Felix Juefei-Xu, My Chiffon Nguyen, Dingsu Wang, Alexandre Oliveira Arrais, Grzegorz Luczyna, Mike Peterson, Ivan Fosin, Timothy Kang, Laila Bashmal, Ha Thi Hoang, Marc Sperzel, Julia Chernyavsky & Guilherme Maximiano

4. Texas A&M University, College Station, TX, USA

   Tung Nguyen

5. Brown University, Providence, RI, USA

   Jaeho Lee, Zheng-Xin Yong & Aaron Kirtland

6. McGill University, Montreal Quebec, Canada

   Mobeen Mahmood, Arkil Patel, Haile Kassahun, Xing Han Lù & Peter E. Chen

7. Institute of Mathematics of NAS of Ukraine, Kyiv, Ukraine

   Oleksandr Pokutnyi

8. Kiev School of Economics, Kyiv, Ukraine

   Oleksandr Pokutnyi

9. Carnegie Mellon University, Pittsburgh, PA, USA

   Oleg Iskra, Hyunwoo Park, Hangrui Cao, Tong Yang, Steffi Chern, Eric Zheng, Jeremiah Milbauer, Andy Zou, Colin Tang, Anupam Nayak, Yunze Xiao, Erica Weng, Ben Racz, Martin Q. Ma, Zhuo Cheng & Johnathan Morris

10. RWTH Aachen University, Aachen, Germany

    Jessica P. Wang

11. University of Cambridge, Cambridge, UK

    John-Clark Levin, Francesco Fournier-Facio, Lina Brüssel, Stefan Ivanov, Julian Wykowski, Mart Oller, Jonathan Roberts, Jacob Loader, Tim Santens, Zaki Hossain, Rami Aly, Daattavya Aggarwal, Antonio Franca, Julien Portier, Lawrence Hollom, Victor Souza, Justin Tan & Arshad Anil Fasiludeen

12. Kyiv Polytechnic Institute, Kyiv, Ukraine

    Mstyslav Kazakov

13. Queen’s University, Kingston Ontario, Canada

    Fiona Feng & Brandon Christof

14. Stanford University, Stanford, CA, USA

    Steven Y. Feng, Chelsea Zou, Noah Burns, Ciprian Manolescu, Declan Grabb, Max Lamparth, Anka Reuel, Allison Tee, G. Bruno De Luca, Niklas Muennighoff, Kushal Thaman, David K. Zhang, William Held, Kang Yong Loh, Nathan Cho, Brian Amaro, Vivek Vajipey, Gabriel Poesia Reis e Silva, Chris Harjadi, Jorge Sanz-Ros, Qizheng Zhang, Yuhui Zhang, Vivek Sanker, Orr Zohar, Sina Mollaei, Rui Li, Anjiang Wei, Xiaohan Wang, Kushin Mukherjee, Haoxuan Chen, Yinuo Ren, Stefano Ermon, Sherwin Lai & Genghan Zhang

15. University of Washington, Seattle, WA, USA

    Haoran Zhao, Jiaqi Wang, Andrew R. Tawfeek, Stefan Steinerberger, Krzysztof Burdzy, Stefan Todoran & Rongwu Xu

16. University of California San Diego, San Diego, CA, USA

    Zihan Wang, Daniel Munro & Junda Chen

17. University of Porto, Porto, Portugal

    Serguei Popov & Duarte V. Gonçalves

18. ELTE, Budapest, Hungary

    Robert Gerbicz

19. Nimbus AI, Honolulu, HI, USA

    Geoff Galgon

20. ETH Zürich, Zurich, Switzerland

    Johannes Schmitt, Lennart Finke, Tim Gehrunger, Antonio Terpin, Johannes Lengler, Miguel Orbegozo Rodriguez, Simon Weber, Emilien Duc, Lukas Lewark, Jakub Łucki, Niels Mündler, Vilém Zouhar, Kumar Shridhar, Alessandro Stolfo, Alice Bizeul & Shehzaad Dhuliawala

21. Durham University, Durham, UK

    Will Yeadon, Chris G. Willcocks & Joshua Newbould

22. Georgia Southern University, Statesboro, GA, USA

    Yongki Lee

23. University of Minnesota, Minneapolis, MN, USA

    Scott Sauers, Krishnamurthy Iyer & Gaoxiang Luo

24. Queen Mary University of London, London, UK

    Marc Roth & Søren Riis

25. Microsoft, Redmond, WA, USA

    Saiteja Utpala, Sivakanth Gopi, Yifan Xiong & Costin Cozianu

26. Auckland University of Technology, Auckland, New Zealand

    Mohinder Maheshbhai Naiya

27. Alberta Health Services, Edmonton Alberta, Canada

    Chidozie Agu

28. University of Illinois Chicago, Chicago, IL, USA

    Antrell Cheatom, Weizhi Zhang & Zishun Yu

29. Hereford College of Arts, Hereford, UK

    Sarah-Jane Crowson

30. Princeton University, Princeton, NJ, USA

    Zerui Cheng, Longke Tang, Kaiqu Liang, Zixuan Wang, Jacob Votava, Denis Peskoff, S. Ashwin Hebbar, Yinwei Dai, Mike He, Jianzhu Yao, Ming Yin, Yihao Liang, Luther Yap, Boyi Wei, Atharv Singh Patlan, Yuval Kansal & Rui Pan

31. University of Canterbury, Christchurch, New Zealand

    Jennifer Zampese

32. Metropolitan State University of Denver, Denver, CO, USA

    Ryan G. Hoerr

33. Massachusetts Institute of Technology, Cambridge, MA, USA

    Jiaqi Cai, Edward Vendrow, Joshua Vendrow, Jayson Lynch, Derek Lim, Richard Stanley, Alvin Jin, Yuexuan Zu, Zachary Berger, Parker Whitfill, Anji Zhang, Adam Zweiger, Jeffery Li, Zachary Brown, Robin Zhang, Linjie Dai, Evan Kim, Hans Gundlach, Evan Chen, Kalon J. Overholt, Changhao Li, Xinyao Han & Cedegao E. Zhang

34. Accenture Labs, Washington, DC, USA

    Ben McCarty

35. Tufts University, Medford, MA, USA

    Alexis C. Garretson

36. The Jackson Laboratory, Bar Harbor, ME, USA

    Alexis C. Garretson

37. INRIA, Paris, France

    Damien Sileo, Gabriel Loiseau & Armel Randy Zebaze

38. University of California, Berkeley, Berkeley, CA, USA

    Qiuyu Ren, Kevin Zhou, Carlo Bosio, Joseph M. Cavanagh, Tobias Kreiman, Alan Goldfarb, Archan Sen, David Aldous, Ning Tang, Michael K. Cohen, Orr Paradise, Michael Wang, Kunyang Sun, Vinh-Kha Le, Jiayi Pan, Milind Jagota, Wenjie Ma, Micah Carroll, Xiuyu Li, Long Tony Lian, Will Cai, Ruiji Sun, Yasin Sonmez, Xuandong Zhao, Muyan Jiang, Adam Bouyamourn, Haocheng Xi, Dawn Song, Zhun Wang, Zhe Ye, Jiaxin Ge & Tianneng Shi

39. University of British Columbia, Vancouver British Columbia, Canada

    Usman Qazi, Shannon Coleman, Ali Khajegili Mirabadi, Wentao Wu & Antonio A. W. L. Wong

40. Ross University School of Medicine, Bridgetown, Barbados

    Usman Qazi

41. École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Lianghui Li, Pierre Marion, Aleksandar Mikov, Philippe Schwaller, Shaipranesh Senthilkumar, Andres M. Bran, Julien Laurendeau, Filippo Bigi, Luca Arnaboldi & Eeshaan Jain

42. Concordia University, Montreal Quebec, Canada

    Jungbae Nam

43. Institute of Science and Technology Austria, Klosterneuburg, Austria

    Pavel Arkhipov

44. National University of Singapore, Singapore, Singapore

    Jack Wei Lun Shi & Kwok Hao Lee

45. California Institute of Technology, Pasadena, CA, USA

    Aras Bacho, Shi-Zhuo Looi, Michael Chen & Hongsen Qin

46. University of Oxford, Oxford, UK

    Sumeet Motwani, Henry Tang, Lisa Schut, Kaivalya Rawal, Younesse Kaddar, Kalyan Ramakrishnan, Elliott Thornley, Demosthenes Patramanis, Christian Schroeder de Witt, Liangti Dai, Ronald Clark, Avi Semler, Justin Xu, Pieter Francois, Runjia Li, Alexander Pondaven, Hunar Batra & Jacek Karwowski

47. University of São Paulo, São Paulo, Brazil

    Emily de Oliveira Santos, Felipe Meneguitti Dias & Benedito Alves de Oliveira Junior

48. Humboldt-Universität zu Berlin, Berlin, Germany

    Johannes Veith, Xiaoxiang Zhou & Christoph Demian

49. Charité – Universitätsmedizin, Berlin, Germany

    Johannes Veith

50. Columbia University, New York, NY, USA

    Doru Cojoc, Jaehyeok Jin, Immo Klose, Andrew Redenti, Ben Segev, Xilin Jiang & Wei Hao

51. University of Southern California, Los Angeles, CA, USA

    Joshua Robinson, Sam Ali, Long Le & Yizhuo Liang

52. C. N. Yang Institute for Theoretical Physics, Stony Brook, NY, USA

    Yuqi Li

53. University of Luxembourg, Luxembourg City, Luxembourg

    Vladyslav Kuchkin

54. Universidade Federal de Juiz de Fora, Juiz de Fora, Brazil

    Andrey Pupasov Maksimov

55. Rockwell Automation, Milwaukee, WI, USA

    Denis Efremov

56. Contramont Research, San Francisco, CA, USA

    Andrew Gritsevskiy

57. École Normale Supérieure, Paris, France

    Julien Guillod & Jérémy Andréoletti

58. Sorbonne Université, Paris, France

    Julien Guillod & Guillaume Malod

59. University of Toronto, Toronto Ontario, Canada

    Saeed Soori, Pavel Zhelnov, Mohammadreza Mofayezi, Mustafa Mehkary, Joseph McGowan, David Anugraha, Zewen Shen, Leo Smucker, Deepayan Banik & Zhanda Zhu

60. University of Tübingen, Tübingen, Germany

    Ori Press, Ameya Prabhu, Anna-Katharina Dick, Michael Kirchhof, Lukas S. Huber, Jules Kreuer, Robert Geirhos & Kristof Meding

61. Sapienza University of Rome, Rome, Italy

    Paolo Rissone, Donato Crisostomi, Andrea Caciolai & Emanuele Rodolà

62. University of North Texas, Denton, TX, USA

    Moon Twayana & Ignat Soroko

63. Institut Polytechnique de Paris, Palaiseau, France

    Aymeric Dieuleveut

64. National University Philippines, Manila, The Philippines

    Joseph Marvin Imperial

65. University of Bath, Bath, UK

    Joseph Marvin Imperial

66. Maastricht University, Maastricht, The Netherlands

    Jinzhou Yang

67. Washington University, St Louis, MO, USA

    Nick Crispino, Daofeng Li, Jiawei Shen, Wenjin Zhang, Chenguang Wang, Kyle Montgomery, Hannah Szlyk, Ting Wang & Yana Malysheva

68. University of California, Los Angeles, Los Angeles, CA, USA

    Arun Rao, Hubeyb Gurdogan, Colin Ni & Stephen Ebert

69. Université Paris-Saclay, Gif-sur-Yvette, France

    Dimitri Zvonkine & Rafael Sayous

70. CNRS, Paris, France

    Dimitri Zvonkine & Jean-Christophe Mourrat

71. Martin-Luther-University Halle-Wittenberg, Halle (Saale), Germany

    Mikhail Kalinin

72. Leibniz University Hannover, Hannover, Germany

    Marco Lukas & Ahmad Sakor

73. Diverging Mathematics, Boston, MA, USA

    Nate Stambaugh

74. Indian Institute of Technology Bombay, Mumbai, India

    Subrata Mishra

75. Institute for Molecular Manufacturing, Palo Alto, CA, USA

    Tad Hogg

76. University of Michigan, Ann Arbor, MI, USA

    Brian P. Coppola, Ziqiao Ma, Soham Sachin Purohit, Patrick Tser Jern Kon, Jiachen Liu, Shuyu Wu, Jeff J. Ma, Xi Lin, Jae-Won Chung, Sihan Xu & Zhiyi Sun

77. Google DeepMind, London, UK

    Julian Salazar, Johan Ferret & Eric Chu

78. Vrije Universiteit Brussel, Brussels, Belgium

    Andres Algaba, Kelsey Van den Houte, Lynn Van Der Sypt, Brecht Verbeken & Vincent Ginis

79. UZ Brussel, Brussels, Belgium

    Kelsey Van den Houte & Lynn Van Der Sypt

80. PeopleTec, Huntsville, AL, USA

    David Noever

81. University of Chicago, Chicago, IL, USA

    Bikun Li, Allen Zang, Linwei Xin, Zihao Wang, Rayner Hernandez Perez, Xi Jiang, Alex Hoover, Yibo Jiang, Claire Sparrow, Hanchen Li, Francesco Pinto, Theo Knights, Ziyi Zhang & Tianchi Zhang

82. Technion – Israel Institute of Technology, Haifa, Israel

    Evgenii Zheltonozhskii & Davide Manini

83. University of Miami, Coral Gables, FL, USA

    Richard Stanley

84. Technische Universität Berlin, Berlin, Germany

    John Maar & Robert Lauff

85. University of Manchester, Manchester, UK

    Cesare Giulio Ardito & Igor Chernyavsky

86. University of Illinois Urbana-Champaign, Urbana-Champaign, IL, USA

    Yuzheng Hu, Muhammad Fayez Aziz, Peter Bradshaw, Ruicheng Xian, Ting Sun, Chenkai Sun, Yaowen Chang, Wei Hu, Yueying Liu & Phuong M. Cao

87. University of Calgary, Calgary Alberta, Canada

    Ariel Ghislain Kemogne Kamdoum & Hamid Mostaghimi

88. Universidad Iberoamericana, Mexico City, Mexico

    Tobias Garcia Vilchis

89. TU Wien, Vienna, Austria

    Martin Lackner, Julian Noah Leser & Ignacio D. Lopez-Miguel

90. University of Wisconsin-Madison, Madison, WI, USA

    Gongbo Sun, Minghao Yan, Song Bian, Mu Cai, Mengze Tang, Anthony Gitter, Jiaxuan Wu, Siddharth Suresh & Ziqi Liu

91. Yale University, New Haven, CT, USA

    Daniil S. Antonenko & Hanmeng Xu

92. University of Edinburgh, Edinburgh, UK

    Bingchen Zhao, Richard Wheeler, Kaniuar Bacho, Edoardo M. Ponti, Brian Rabern & Ondrej Bohdal

93. École Normale Supérieure Paris-Saclay, Gif-sur-Yvette, France

    Pierrot Arsene & Robin Riblet

94. University of Western Australia, Perth Western Australia, Australia

    David Perrella & Sean Li

95. Snorkel AI, Redwood City, CA, USA

    Nurdin Kaparov

96. New York University, New York, NY, USA

    Ilia Sucholutsky, Niv Cohen, William Merrill, Bingsen Chen, Isaac Park & Qiutong Men

97. Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE

    Arina Kharlamova, Daniil Orel, Raghav Singhal, Xingyu Qu, Junyi Guan, Kaustubh Ponkshe & Alham Fikri Aji

98. University of Waterloo, Waterloo Ontario, Canada

    Shalev Ben-David, Ronak Pradeep & Alex Meiburg

99. University of Maryland, College Park, MD, USA

    Shankar Sivarajan, Kelin Zhu, Anh N. Nhu & Hieu Tran

100. Manhattan School of Music, New York, NY, USA

     Dan Bar Hava

101. Universiteit Leiden, Leiden, The Netherlands

     David Holmes & Jiale Chen

102. Synbionix, Casselberry, FL, USA

     Frank Sommerhage

103. The Open University, Milton Keynes, UK

     Richard Moat & Anna Plassart

104. Corteva Agriscience, Indianapolis, IN, USA

     Zakayo Kazibwe

105. Sanford Burnham Prebys, La Jolla, CA, USA

     Don Clarke

106. Yonsei University, Seoul, South Korea

     Dae Hyun Kim

107. Harvard University, Cambridge, MA, USA

     Sara Fish, Vincent Ginis, Jamie Tucker-Foltz, Harrison K. Wang, Qijia Chen, Oam Patel, Dmitry Kazakov, Jingxuan Fan, Sarah Martinson, Erik Y. Wang, Kaylie Hausknecht, Michael P. Brenner, Tong Jiang, Sarah Hoback, Dianzhuo Wang, Satyapriya Krishna, Zechen Zhang, Rishab Kumar Jain & Core Francisco Park

108. Cornell University, Ithaca, NY, USA

     Veit Elser, Wen-Ding Li, Zhibai Jia, Yingheng Wang, Claas Beger, Hongzheng Chen & Maosen Tang

109. University of Leeds, Leeds, UK

     Victor Efren Guadarrama Vilchis

110. Arizona State University, Tempe, AZ, USA

     Ujjwala Anantheswaran, Jennifer Sandlin, Kevin Joseph Scaria & Himanshu Gupta

111. Swinburne University of Technology, Melbourne Victoria, Australia

     Jeremy Nguyen

112. KU Leuven, Leuven, Belgium

     Nicolas Daans

113. St Petersburg College, St Petersburg, FL, USA

     Haline Heidinger

114. La Molina National Agrarian University, Lima, Peru

     Haline Heidinger

115. Brandenburg University of Technology, Cottbus, Germany

     Maksim Radionov

116. INSAIT, Sofia, Bulgaria

     Václav Rozhoň & Seri Khoury

117. Ruhr University Bochum, Bochum, Germany

     Christian Stump & Alexander Ivanov

118. National Information Processing Institute, Warsaw, Poland

     Rafał Poświata & Michał Perełkiewicz

119. Charles University, Prague, Czech Republic

     Josef Tkadlec

120. Cranfield University, Cranfield, UK

     Ryan Stendall, Petr Spelda & Vit Stritecky

121. University of Copenhagen, Copenhagen, Denmark

     Jack Stade & Sara Vera Marjanović

122. TRR Designs, Fayetteville, AR, USA

     T. Ryan Rogers

123. The University of Sydney, Sydney New South Wales, Australia

     Tom Goertzen, Dominic Williamson & Tony CY Pang

124. Indian Institute of Technology Delhi, New Delhi, India

     Abhishek Shukla & Laxman Prasad Goswami

125. Universidad de Buenos Aires, Buenos Aires, Argentina

     Alan Givré & Laila Yacar

126. University of Technology Sydney, Sydney New South Wales, Australia

     John Arnold Ambay

127. Indiana State University, Terre Haute, IN, USA

     Mark H. Inlow

128. Australian National University, Canberra Australian Capital Territory, Australia

     Hao He, Ling Zhang & Pascal Lauer

129. KTH Royal Institute of Technology, Stockholm, Sweden

     Ivar Ängquist, Nils Gustafsson & Emil Verkama

130. University of Amsterdam, Amsterdam, The Netherlands

     Yanxu Chen, David Stap & Leon Lang

131. Ben-Gurion University, Beersheba, Israel

     Avishy Carmi

132. Donald and Barbara Zucker School of Medicine, Hempstead, NY, USA

     Ethan D. L. Brown

133. Cohere, Toronto Ontario, Canada

     Max Bartolo

134. Siili Solutions, Helsinki, Finland

     JP Heimonen

135. University of Pennsylvania, Philadelphia, PA, USA

     Kaustubh Sridhar, George Balabanian, Zhehang Du, Li S. Yifei & Alex Slen

136. Aalto University, Espoo, Finland

     Ido Akov

137. Toyota Technological Institute at Chicago, Chicago, IL, USA

     Yury Makarychev

138. Northeastern University, Boston, MA, USA

     Joanna Tam & Jason O. Matos

139. Case Western Reserve University, Cleveland, OH, USA

     Hieu Hoang

140. University of Windsor, Windsor, Ontario, Canada

     Michael Krause

141. St. Jude Children’s Research Hospital, Memphis, TN, USA

     Jesyin Lai

142. Rochester Institute of Technology, Rochester, NY, USA

     Jiangnan Xu

143. Emory University, Atlanta, GA, USA

     Ilias Magoulas, Kaustubh Dhole & Isha Gupta

144. Anthropic, San Francisco, CA, USA

     Jan Hendrik Kirchner & Jack Lindsey

145. CERN, Geneva, Switzerland

     Maksym Ovchynnikov

146. University of California Santa Barbara, Santa Barbara, CA, USA

     Yuzhou Nie, Taom Sakal, Daniel Espinosa Gonzalez, Xianjun Yang & Xinlu Zhang

147. Warsaw University of Technology, Warsaw, Poland

     Anna Sztyber-Betley

148. Hewlett Packard Enterprise, San Francisco, CA, USA

     Paolo Faraboschi

149. North Carolina State University, Raleigh, NC, USA

     Jonathan Crozier & Anil Radhakrishnan

150. University of Houston, Houston, TX, USA

     Shiv Halasyamani

151. All India Institute of Medical Sciences, New Delhi, India

     Prashant Joshi

152. Tel Aviv University, Tel Aviv, Israel

     Eli Meril

153. Georgia Institute of Technology, Atlanta, GA, USA

     Jacob Platnick, William Held, Yuzhou Wang, Rynaa Grover, Gunjan Chhablani & Jiaqi Deng

154. Northwestern University, Evanston, IL, USA

     Volodymyr Nevirkovets, Yosi Kratish, Ali ElSheikh & Christopher R. Scotese

155. University of Arizona, Tucson, AZ, USA

     Luke Basler

156. Universidade de Lisboa, Lisbon, Portugal

     Marco Piccardo

157. Indian Institute of Technology Kharagpur, Kharagpur, India

     Virendra Singh

158. Posts and Telecommunications Institute of Technology, Hanoi, Vietnam

     Tran Quoc Khánh

159. Duke University, Durham, NC, USA

     Paul Rosu, Muthu Chidambaram, Sanxing Chen & Songyang Zhang

160. Mila - Québec AI Institute, Montreal Quebec, Canada

     Arkil Patel & Mohammed Mahfoud

161. University College London, London, UK

     Andrea Achilleos, Jean Kaddour, Maria del Rio-Chanona & Ed Chalstrey

162. University of Zurich, Zurich, Switzerland

     Thomas Preu, Nikola Zubić, Martyna Plomecka, Bruno Hebling Vieira & Davide Scaramuzza

163. UK AI Safety Institute, London, UK

     Tomek Korbak

164. University of Padua, Padua, Italy

     Ida Bosio

165. Boston University, Boston, MA, USA

     Ziye Chen, Hao Qi, Mao Mao, Xinyu Zhang, Gabe Maayan & Yuanli Wang

166. Royal Veterinary College, London, UK

     Eve J. Y. Lo

167. Instituto Superior Técnico, Lisbon, Portugal

     Maria Inês S. Nunes

168. SDAIA, Riyadh, Saudi Arabia

     M. Saiful Bari

169. The Ohio State University, Columbus, OH, USA

     Yewen Sun & Christopher W. Bartlett

170. University of Montreal, Montreal Quebec, Canada

     Stephane Durand

171. Cairo University Specialized Pediatric Hospital, Cairo, Egypt

     Hossam Elgnainy

172. Universidad de Valencia, Valencia, Spain

     Daniel Tordera & Pablo Hernández-Cámara

173. Monash University, Melbourne Victoria, Australia

     Lynna Kvistad

174. Van Andel Institute, Grand Rapids, MI, USA

     Hsiaoyun Milliron

175. Larkin Community Hospital, South Miami, FL, USA

     Andrew Favre

176. The University of Texas at Dallas, Richardson, TX, USA

     Shailesh Shah

177. Canadian University Dubai, Dubai, UAE

     Firuz Kamalov

178. The Hebrew University of Jerusalem, Jerusalem, Israel

     Shaul Barkan, Noam Kolt & Assaf Brown

179. Università di Milano-Bicocca, Milan, Italy

     Alessandro Tomasiello

180. University of Massachusetts Lowell, Lowell, MA, USA

     Emma Rodman

181. Virginia Tech, Blacksburg, VA, USA

     Carl J. Fossum

182. University of Geneva, Geneva, Switzerland

     Honglu Fan

183. Google Research, Mountain View, CA, USA

     Moritz Firsching

184. Cal Poly San Luis Obispo, San Luis Obispo, CA, USA

     Carter Harris

185. Alexandru Ioan Cuza University, Iasi, Romania

     Stefan Ciobâcă

186. University of Mannheim, Mannheim, Germany

     Shashank Agnihotri & David Avagian

187. Stockholm University, Stockholm, Sweden

     Alexander Piperski

188. College of Eastern Idaho, Idaho Falls, ID, USA

     Joshua Duersch

189. Intrinsic Innovation, Mountain View, CA, USA

     Vage Taamazyan

190. Ivy Natal, San Francisco, CA, USA

     Andrew Ho

191. King Saud University, Riyadh, Saudi Arabia

     Mohanad Mohamed

192. SAMPE Switzerland, Zurich, Switzerland

     Claudio Di Fratta

193. CERo Therapeutics Holdings, South San Francisco, CA, USA

     Edson Oliveira

194. University of Tennessee, Knoxville, TN, USA

     Joseph W. Jackson

195. Gray Swan AI, Pittsburgh, PA, USA

     Andy Zou

196. EleutherAI, Washington, DC, USA

     Dashiell Stander & Jianxin Wang

197. Johns Hopkins University, Baltimore, MD, USA

     Ali Dasouqi, Orion Weller, Xue Wang, Maxwell Shepherd, Chuanyang Jin, Yifan Yin & Guangyao Zheng

198. University of Montpellier, Montpellier, France

     Alexander Shen

199. Fraunhofer IMTE, Lübeck, Germany

     Egor Kretov

200. HomeEquity Bank, Toronto Ontario, Canada

     Mikalai Uzhou

201. Materials Platform for Data Science, Tallinn, Estonia

     Alina Borisovna Zhidkovskaya

202. University of Pisa, Pisa, Italy

     Fortuna Samuele

203. Georgia State University, Atlanta, GA, USA

     Faraz Farhidi

204. Polytechnic University of the Philippines, Manila, The Philippines

     Madellene Peñaflor

205. University of Oregon, Eugene, OR, USA

     Alena Friedrich

206. Drexel University, Philadelphia, PA, USA

     Daniel Pyda

207. University of Mumbai, Mumbai, India

     Omkar Dhamane

208. Gakushuin University, Tokyo, Japan

     Kenchi Okutsu

209. University of Guelph, Guelph Ontario, Canada

     Mohammad Maghsoudimehrabani

210. Intuit, Mountain View, CA, USA

     Alon Amit

211. CTTC / CERCA, Castelldefels, Spain

     Roberto Pereira

212. Dyno Therapeutics, Watertown, MA, USA

     Stephen Malina

213. The Hospital for Sick Children, Toronto Ontario, Canada

     Mustafa Mehkary & Loukmane Karim

214. Temple University, Philadelphia, PA, USA

     Cary Friday

215. Saint Mary’s University, Halifax Nova Scotia, Canada

     Mukhwinder Singh

216. Cisco, San Jose, CA, USA

     Hassan Shapourian

217. Indian Institute of Technology (BHU), Varanasi, India

     Harsh Kumar

218. AIM Intelligence, Seoul, South Korea

     Haon Park

219. Seoul National University, Seoul, South Korea

     Haon Park

220. The University of Texas at Arlington, Arlington, TX, USA

     Diana T. Pham

221. The Hartree Centre, Daresbury, UK

     Joshua Robinson

222. University of Vienna, Vienna, Austria

     Paolo Giordano & Philipp Petersen

223. POLITEHNICA Bucharest National University of Science and Technology, Bucharest, Romania

     Adrian Cosma

224. Abacus.AI, San Francisco, CA, USA

     Colin White

225. University of Galway, Galway, Ireland

     Ethan Delaney & Imad Ali Shah

226. Eastern Institute of Technology (EIT), Napier, New Zealand

     Syed M. Shahid

227. ENS Lyon, Lyon, France

     Jean-Christophe Mourrat

228. Czech Technical University in Prague, Prague, Czech Republic

     Lavr Vetoshkin

229. University of Hamburg, Hamburg, Germany

     Koen Sponselee

230. CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

     Renas Bacho

231. Universidad de Morón, Morón, Argentina

     Florencia de la Rosa

232. Université Paris Cité, Paris, France

     Guillaume Malod

233. Politecnico di Milano, Milan, Italy

     Guglielmo Albani

234. The New School, New York, NY, USA

     Yuchen Anna Zhou

235. Max Planck Institute for Software Systems, Saarbrücken, Germany

     Yiğit Yaln

236. OpenAI, San Francisco, CA, USA

     Rai Michael Pokorny & Tejal Patwardhan

237. Universidad de Granada, Granada, Spain

     M. C. Boscá

238. Modulo Research, Cambridge, UK

     Gabriel Recchia

239. Heidelberg University, Heidelberg, Germany

     Mara Popescu & Jing Fan

240. La Trobe University, Melbourne Victoria, Australia

     Nikita Shulga

241. University of Yaoundé I, Yaoundé, Cameroon

     Ngefor Mildred Tanwie & Gautier Abou Loume

242. University of Innsbruck, Innsbruck, Austria

     Alesia Yakimchyk

243. Nabu Technologies, San Francisco, CA, USA

     Huanxu Quinn Liu

244. Chalmers University of Technology, Gothenburg, Sweden

     Olle Häggström

245. Unidade Local de Saúde de Lisboa Ocidental, Lisbon, Portugal

     Leonor Brito-Santana

246. Children’s Hospital of Orange County, Orange, CA, USA

     Peyman Kassani

247. The Future Paralegals of America, New York, NY, USA

     Eshawn Jessica Scipio

248. Eastlake High School, Sammamish, WA, USA

     Alon Ragoler

249. Center for Scientific Research and Higher Education at Ensenada (CICESE), Ensenada, Mexico

     Yan Carlos Leyva Labrador

250. University of Bradford, Bradford, UK

     Zahra Adoul

251. Beni Suef University, Beni Suef, Egypt

     Mohamed Zekry

252. Bogazici University, Istanbul, Turkey

     Ali Karakoc

253. Mansoura University, Mansoura, Egypt

     Samir Shamseldeen

254. University of Bristol, Bristol, UK

     Anna Liakhovitskaia

255. University of Oklahoma, Norman, OK, USA

     Nate Resman & Elizabeth Kelley

256. Jala University, Honolulu, HI, USA

     Juan Carlos Gonzalez

257. University of Arkansas, Fayetteville, AR, USA

     Earth Anderson

258. Florida Atlantic University, Boca Raton, FL, USA

     Rodrigo De Oliveira Pena

259. Bournemouth University, Bournemouth, UK

     Ismail Alarab

260. University of Warwick, Coventry, UK

     Joshua Cole

261. University of Alabama Huntsville, Huntsville, AL, USA

     Bryan Johnson

262. University of Hertfordshire, Hatfield, UK

     Mohammad Safdari

263. OncoPrecision, New York, NY, USA

     Alejandro José Moyano

264. Central College, Pella, IA, USA

     Alexey Pronin

265. Nottingham Trent University, Nottingham, UK

     Daphiny Pottmaier

266. Max Planck Institute for Intelligent Systems, Stuttgart, Germany

     Omid Taheri & Fanfei Li

267. University of Virginia, Charlottesville, VA, USA

     Stanley Stepanic

268. Dartmouth College, Hanover, NH, USA

     Luke Askew

269. Cairo University, Giza, Egypt

     Ali M. R. Minissi & Zienab EL-Wasif

270. INESC Microsistemas e Nanotecnologias, Lisbon, Portugal

     Ricardo Lorena & Diogo M. Caetano

271. James Madison University, Harrisonburg, VA, USA

     Josh Ducey

272. Instituto Gonçalo Moniz, Salvador, Brazil

     Matheus Piza

273. Rice University, Houston, TX, USA

     Juehang Qin

274. HUN-REN, Budapest, Hungary

     Benjámin Borbás

275. Rutgers University, New Brunswick, NJ, USA

     Tej Shah

276. AE Studio, Marina Del Rey, CA, USA

     Marc Carauleanu

277. Saarland University, Saarbrücken, Germany

     Pascal Lauer

278. HUTECH, Ho Chi Minh City, Vietnam

     Tran Duc Huy

279. Pennsylvania College of Technology, Williamsport, PA, USA

     Hossein Shahrtash

280. Intelligent Geometries, Front Royal, VA, USA

     Brian Weber

281. École Polytechnique, Palaiseau, France

     Pierre Clavier & Sergei Bogdanov

282. CONICET, Buenos Aires, Argentina

     Sandra Mendoza

283. Universidad Tecnológica Nacional, Buenos Aires, Argentina

     Sandra Mendoza

284. John Crane UK, Slough, UK

     Murat Islam

285. Alan Turing Institute, London, UK

     Vasilios Mavroudis & Pieter Francois

286. Pondicherry Engineering College, Puducherry, India

     Pawan Kumar

287. Leibniz Institute for Science and Mathematics Education, Kiel, Germany

     Thorben Jansen

288. Royal Holloway, University of London, Egham, UK

     Archimedes Apronti

289. Tanta University, Tanta, Egypt

     Abdallah Galal

290. University of Malaya, Kuala Lumpur, Malaysia

     Ng Ze-An

291. Hemwati Nandan Bahuguna Garhwal University, Srinagar, India

     Ankit Singh

292. University Mohammed I, Oujda, Morocco

     Mohammed Berkani

293. LGM, Paris, France

     Nicolas Remy

294. Northern Illinois University, DeKalb, IL, USA

     Taylor D. Hartman & Jichao Fang

295. Bethune-Cookman University, Daytona Beach, FL, USA

     Tim Tarver

296. University of California, Irvine, Irvine, CA, USA

     Farzad Habibi, Sina Rismanchian & Christopher Toukmaji

297. Central Mindanao University, Maramag, The Philippines

     Roselynn Grace Montecillo

298. University of the Fraser Valley, Abbotsford British Columbia, Canada

     Russell Campbell

299. Patched Codes, San Francisco, CA, USA

     Asankhaya Sharma

300. Missouri University of Science and Technology, Rolla, MO, USA

     Shreen Gul

301. Quotient AI, Boston, MA, USA

     Freddie Vargus

302. CSMSS Chh. Shahu College of Engineering, Aurangabad, India

     Deepakkumar Patil

303. Genomia Diagnostics Research, New Delhi, India

     Rajat Maheshwari

304. Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, UK

     Ashley Cartwright

305. Forschungszentrum Jülich, Jülich, Germany

     Sören Möller

306. Standard Intelligence, San Francisco, CA, USA

     Kunvar Thaman

307. RMIT University, Melbourne Victoria, Australia

     Muhammad Rehan Siddiqi

308. German Research Center for Artificial Intelligence, Kaiserslautern, Germany

     Prajvi Saxena

309. Fondazione Bruno Kessler, Trento, Italy

     Mátyás Vincze & Lorenzo Vaquero

310. University of Trento, Trento, Italy

     Mátyás Vincze

311. Chulalongkorn University, Bangkok, Thailand

     Siranut Usawasutsakorn

312. Aligarh Muslim University, Aligarh, India

     Sk Md Salauddin

313. Happy Technologies LLC, Arlington, VA, USA

     Eric Singer

314. Menoufia University, Shebin El Kom, Egypt

     Ahmed Menshawy

315. Instituto Politécnico Nacional, Mexico City, Mexico

     Darling Duclosel

316. University of Bologna, Bologna, Italy

     Dario Bezzi

317. Manipal University Jaipur, Jaipur, India

     Yashaswini Jain

318. The University of Texas at Austin, Austin, TX, USA

     Hao-Yu Sun

319. Murdoch University, Perth Western Australia, Australia

     Samuele Sala

320. University of Delaware, Newark, DE, USA

     Manuel Schottdorf

321. Williams College, Williamstown, MA, USA

     Gerol Petruzella

322. Perimeter Institute for Theoretical Physics, Waterloo Ontario, Canada

     Alex Meiburg

323. University of Maribor, Maribor, Slovenia

     Tilen Medved

324. Brigham and Women’s Hospital, Boston, MA, USA

     Jason Poulos

325. The University of Tokyo, Tokyo, Japan

     Mingfang Zhang

326. Imperial College London, London, UK

     Ali Anil Demircali, Abdurrahim Yilmaz & Barbara Dworakowska

327. University of California Santa Cruz, Santa Cruz, CA, USA

     Yuyin Zhou, Juncheng Wu & Haoqin Tu

328. Vellore Institute of Technology, Vellore, India

     Aarush Sinha

329. CHRU de Nancy, Nancy, France

     Mickaël Noyé

330. Delft University of Technology, Delft, The Netherlands

     Ioannis Pantidis

331. Scripps Research, La Jolla, CA, USA

     Tianbo Qi & Shiqi Wang

332. Aleph Alpha, Heidelberg, Germany

     Letitia Parcalabescu & Philipp D. Siedler

333. George Mason University, Fairfax, VA, USA

     Thai-Hoa Nguyen

334. Atilim University, Ankara, Turkey

     Jongee Park

335. Leonardo Labs, Rome, Italy

     Dario Abbondanza

336. Complexity Science Hub, Vienna, Austria

     Maria del Rio-Chanona, Dániel Kondor, Jakob Zsambok, Dan Hoyer, Jenny Reddish, Jakob Hauser & Peter Turchin

337. Universidad Nacional de Educación a Distancia, Madrid, Spain

     Francisco-Javier Rodrigo-Ginés

338. Saxion University, Enschede, The Netherlands

     Thom Kamphuis

339. Korea Advanced Institute of Science and Technology, Daejeon, South Korea

     Hyunjun Kim, Sejong Kim & Woongyeong Yeo

340. Adobe Research, San Jose, CA, USA

     Franck Dernoncourt

341. National Aerospace University ‘Kharkiv Aviation Institute’, Kharkiv, Ukraine

     Glib Briia

342. Google, Mountain View, CA, USA

     Hieu Nguyen, Ivan Dewerpe & Subhashini Venugopalan

343. Hexworks, Barcelona, Spain

     David Quod Soler Bartomeu

344. Westmead Hospital, Sydney New South Wales, Australia

     Tony CY Pang

345. University of Bern, Bern, Switzerland

     Lukas S. Huber & Joshua Jaeger

346. Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau, Kaiserslautern, Germany

     Romano De Maddalena

347. SUMM AI, Munich, Germany

     Ankit Agrawal

348. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

     Shoubin Yu, Han Lin & Vaidehi Patil

349. Konkuk University, Seoul, South Korea

     Hyun Kyu Park

350. University of Groningen, Groningen, The Netherlands

     Gabriele Sarti

351. Jagiellonian University, Kraków, Poland

     Marcin Briański

352. Minerva University, San Francisco, CA, USA

     Truong An Nguyen

353. Aalborg University, Aalborg, Denmark

     Mike Zhang

354. IBM Research, Givatayim, Israel

     Yotam Perlitz

355. Universitat Politecnica de Valencia, Valencia, Spain

     Jose Hernandez-Orallo

356. RBC Borealis, Toronto Ontario, Canada

     Amin Shabani

357. Mayo Clinic, Rochester, MN, USA

     Shikhar Dhingra

358. University of Lausanne, Lausanne, Switzerland

     Martino Maggetti

359. Dalhousie University, Halifax Nova Scotia, Canada

     Jonathon Kean

360. Universitat de Lleida, Lleida, Spain

     Jorge Chamorro-Padial

361. Amazon, Seattle, WA, USA

     Shreyas Subramanian

362. Dell Technologies, Round Rock, TX, USA

     Yushun Chen & Samaksh Gulati

363. University of Seoul, Seoul, South Korea

     Junwoo Ha

364. University of Auckland, Auckland, New Zealand

     Gaël Gendron

365. Morgridge Institute for Research, Madison, WI, USA

     Anthony Gitter

366. Korea University of Technology and Education, Cheonan, South Korea

     Namkyu Park

367. University of California Davis, Davis, CA, USA

     Kunal Pai & Zijian Song

368. Baylor College of Medicine, Houston, TX, USA

     Ahmed Elkhanany

369. Indraprastha Institute of Information Technology Delhi, New Delhi, India

     Ritwik Mishra

370. Two Minute Papers, Pécs, Hungary

     Károly Zsolnai-Fehér

371. ADIA Lab, Abu Dhabi, UAE

     Shadab Khan

372. New Jersey Institute of Technology, Newark, NJ, USA

     Jun Yuan

373. Novo Nordisk, Bagsværd, Denmark

     Zhe Wang

374. Purdue University, West Lafayette, IN, USA

     Aditya Malusare & Veerupaksh Singla

375. Gakugei Shuppan-sha, Kyoto, Japan

     Kazuki Matsumoto

376. Universiteit Utrecht, Utrecht, The Netherlands

     Gerben Sewuster

377. T-Systems Iberia, Madrid, Spain

     Jorge Pretel Villanueva

378. University of Klagenfurt, Klagenfurt, Austria

     Ivan Rannev

379. Max Planck Institute for Security and Privacy, Bochum, Germany

     Wenchao Dong

380. InxiteOut, Bangalore, India

     Kaushik Bar

381. Goethe Universität Frankfurt, Frankfurt am Main, Germany

     Caroline Geirhos

382. Universidad del Valle, Cali, Colombia

     Julien Wist

383. Bilkent University, Ankara, Turkey

     Kutay Tire & Atak Talay Yücel

384. Trinity School, New York, NY, USA

     Joshua Mak

385. Universitat Pompeu Fabra, Barcelona, Spain

     Antoine Moulin

386. Brighton Law School, Brighton, UK

     Ketan Jha

387. University of Melbourne, Melbourne, Australia

     Vassilis Kostakos

388. Ankara University, Ankara, Turkey

     Eunmi Yu

389. Dr. Siyami Ersek Thoracic, Cardiovascular and Vascular Surgery Training and Research Hospital, Istanbul, Turkey

     Arif Engin Demircali

390. AIT Austrian Institute of Technology, Vienna, Austria

     Roman Pflugfelder

391. Technical University of Munich, Munich, Germany

     Roman Pflugfelder

392. Providence College, Providence, RI, USA

     James Bailey

393. University of Jyväskylä, Jyväskylä, Finland

     Ville Heilala

394. Weizmann Institute of Science, Rehovot, Israel

     Sybille Rosset

395. Indiana University, Bloomington, IN, USA

     Sreekar Chigurupati & Sai Prajwal Reddy

396. Nanyang Technological University, Singapore, Singapore

     Hu Shiyu

397. University of Sheffield, Sheffield, UK

     Ben Wu

Consortia

Contributions

All authors have contributed to the dataset creation process. The Center for AI Safety and Scale AI consortia jointly designed the dataset premise and pipeline; operated the data collection platform (https://lastexam.ai); and provided funding, inference infrastructure for LLMs and review/auditing resources. The authors in the HLE Contributors Consortium contributed to the dataset in various ways, including submitting at least one accepted question to one of the dataset versions, contributing to dataset refinement or assisting with evaluations. In the Center for AI Safety and Scale AI, Long Phan, Alice Gatti, Ziwen Han and Nathaniel Li led the project, and Summer Yue, Alexandr Wang and Dan Hendrycks provided senior supervision.

Corresponding authors

Correspondence to Long Phan or Dan Hendrycks.

Competing interests

The authors declare no competing interests.

Peer review information

Nature thanks the anonymous reviewers for their contribution to the peer review of this work. Peer reviewer reports are available.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions