Initial support for Nvidia Blackwell GPUs (sm_100).
The key contributions included in this PR are:
- Support for 5th generation Tensor Core.
- Modeling and support of Tensor Memory.
- Native support for microscaling formats mxfp4 and mxfp8.
- Improvements to the software pipeliner to take advantage of Tensor Cores and Tensor memory
This was developed in close collaboration between Nvidia and OpenAI.
From Nvidia:
dePaul Miller (@depaulmillz)
Samantha Hirsch (@Sam3077)
Yujia Zhai (@yzhaiustc)
Shang Zhang (@shangz-ai)
Pradeep Ramani (@IonThruster)
Matthew Brookhart (@mbrookhart)
Masahiro Masuda (@masahi)
Chris Sullivan (@csullivan)
Clive Unger (@CliveUnger)
Jason Knight (@binarybana)
From OpenAI:
Pawel Szczerbuk (@pawelszczerbuk)
Peter Bell (@peterbell10)
Phil Tillet (@ptillet)
Jeff Niu (@jeffniu-openai)
Thomas Raoux (@ThomasRaoux)