Settings

Theme

Explaining Llm.c in Layman Terms

nitter.poast.org

38 points by publius_0xf3 2 years ago · 11 comments

Reader

saagarjha 2 years ago

I'm curious who the target audience of this tweet is. Are there a lot of people who know all about tensors, deep learning, and PyTorch but have no idea what C is?

  • dagw 2 years ago

    Are there a lot of people who know all about tensors, deep learning, and PyTorch but have no idea what C is?

    They probably know what C is in the very general sense, but there are lots and lots of people people who know all the deep learning stuff but have never written a line of C or really understands the when/why you might want to use C and the benefits and limitations of the language. You have to remember that a lot of people currently working in this space don't come from a computer science and programming background and don't have that general background knowledge of computer science.

  • jampekka 2 years ago

    I'd guess there are a lot of such people. Probably most who do deep learning don't know much C. Or even much general purpose programming.

  • constantcrying 2 years ago

    I think so. Many people have some knowledge of high level LLM stuff, e.g. by using tensorflow, pytorch or similar. All of these libraries intentionally hide a lot of the details behind abstractions and having an example where every single detail is made totally explicit seems like something that could be very valuable if you want to understand the topic deeply.

  • ykonstant 2 years ago

    I think many practitioners with statistics or pure mathematics background have very little experience with systems level programming and program primarily in very high level languages.

teleforce 2 years ago

Very interesting notes and as commented by one of the sibling comments it is really a rumination on abstraction rather on than implementation off llm.c itself.

It will be very interesting to implement this small llm in D language and use the new tiny autograd library in D [1]. Since now D compiler does support C, this should results in less line of codes excluding the small autograd library in D [2]. Given that the D code will be able to inline C code, and even CTFE it, if done correctly the implementation will be faster as well.

[1] A tiny autograd library:

https://code.dlang.org/packages/tiny-autodiff

[2] Adding ANSI C11 C compiler to D so it can import and compile C files directly:

https://news.ycombinator.com/item?id=27102584

ChrisArchitect 2 years ago

Actual link: https://twitter.com/karpathy/status/1778153659106533806

It's not a thread, no nitter needed.

ChrisArchitect 2 years ago

Related:

Llm.c – LLM training in simple, pure C/CUDA

https://news.ycombinator.com/item?id=39973467

fefe23 2 years ago

This does not explain llm.c. It is more like a rumination on abstraction.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection