Settings

Theme

WebGPU in Chrome Canary now supports float16 for faster local LLM inference

twitter.com

18 points by NiekvdMaas 2 years ago · 5 comments

Reader

no_time 2 years ago

I hope they gate webGPU behind a per domain allow/deny notification like they do with location and camera access.

This whole thing is just begging to be exploited by cryptominers.

  • charcircuit 2 years ago

    It may be more profitable to use the compute for LLMs or other GPU dependent AI than crypto mining. The downside is that there is no way to verify the client isn't returning random or bad data.

  • lostmsu 2 years ago

    Cryptomining is mostly gone.

NiekvdMaasOP 2 years ago

To test, run Chrome Canary with:

  --enable-dawn-features=allow_unsafe_apis,disable_robustness
Then, open: https://webllm.mlc.ai/#chat-demo

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection