How to Optimize Scaled Dot Product Attention?
yash-sri.xyz
1 thread
In this worklog, I explore how to optimize SDPA across cache, quantization and device specific optimizations.
In this worklog, I explore how to optimize SDPA across cache, quantization and device specific optimizations.