Settings

Theme

Vibecoding an inference chip

ankitg.me

58 points by ankitvgupta · 19 comments

Reader

8 threads
cpldcpu

I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

It seems implementing transformers is the new "hello world" of AI assisted coding.

datadrivenangel

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

codekansas

This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

chalkycrimp

Good chance I'm not the only one thinking this: share the code

lxe

Repetitive "make it faster" prompting is essentially what all these "RSI" claims are

spwa4

I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

(unfortunately for a decent model it will also be price competitive with a big nvidia card)

saejox

i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.

Archit3ch

Now do silicon. ;)

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection