I’ve recently become fascinated by DeepSeek V4.1 Flash and its Causal-Encoder-Decoder architecture. Since the original paper has so many abstractions and calculations, it’s really tough to see that we are still dealing with matrices and multiplications at the end of the day. This is linear algebra, but algebra is comprehensible since humans created it. It is not some amorphous Rorschach test.
So I want to take you through a full analogy of the CED architecture. I will not explain encoders, decoders, or transformers. They are in previous articles of mine.
Let’s start from a huge novel of 100,000 pages. Call it a large Harry Potter novel. It’s full of literary context — meanings, themes, characters.
Now, let’s say for every single page, we’ve computed an embedding (literally a vector, a sequence of numbers). Now, this embedding is embued with rich contextual information.
As you’re reading the pages, you keep thinking, and process them into a hidden state (let’s say I have the feeling of DNFing this book)
Let’s say this hidden state is something only you know in your head - a mental bookmark.
This hidden state (bookmark) has 7,168 dimensions, which is to say they are very rich with information.
Now let’s say that you’re kind of tired of keeping these hidden states (thoughts) in your memory. After all, each hidden state is 7,168 dimensions (sentences)!
Now, in ML, dimensionality reduction is a very popular technique, and this is just to say that maybe each hidden thought doesn’t need all 7,168 dimensions.
In fact, at some point you need to turn your hidden thoughts into words again to write.
Let’s unwrap the analogy. The mental bookmarks are the hidden state, and you could also think of them as the learned representation of the hidden states. I think of them as really similar, but the LLM’s calculations are different.
One intuitive way is to turn your bookmarks into bullets, compressed learned representations of the hidden state.
Even though the words have changed, the actual understanding has not (for the most part). The reader (the LLM) has still internalized and read the book.
There is still a faithful mapping from book to page to mental bookmark!
The pages have undergone transformation (matrix multiplication really), but you still have the plot, themes, and characters. The dialogue (mostly) is the same.
However, it’s important to note, that we aren’t reading the book anymore. The analogy breaks here. In a real novel, each word, each sound, each phoneme or morpheme might be important, and once you’ve decided a bookmark is all you need, you’ve probably thrown away some information. But how much information, is the question.
But, the LLM nerds will say, we still have the facts! We still understand the story!
In fact, there is a way to compress the model’s understanding of the bookmarks with two mechanisms, Engram and CSA (not explained here).
The learned representation of the hidden state’s size has still not changed (still 7,168 dimensions). In Engram, maybe you keep the character-related hidden thoughts (mental bookmarks) together—you keep them in a particular part of your brain. In CSA, maybe you color-code the hidden thoughts, or use different handwriting so its easier to reference later. In the merge step, you retain all this “character-related” information.
Now for the exciting part. Let’s say that instead of one hidden state / bookmark with 7,168 dimensions (sentences), you could actually reduce it to 512 bullets / dimensions (compressions of learned representation of the hidden state) when you’re trying to understand the book. In fact, you might need to re-read the same chapter millions of times. It’s much easier to process 512 bullets instead of 7,168.
I’m using the word “idea” very generously — let’s just say it’s information.
Now you can see that this architecture is starting to work. You’ve really divided and conquered the information. You’re doing some advanced speed reading, color coding, dividing information, writing smaller, and looking at bullets (transformed representations), instead of bookmarks (hidden states or learned representations of those hidden states), or pages (input embeddings).
Those 512 ideas (or bullets), or compressed representations, are used to form keys and values (let’s say subjects and verbs). Those keys and values live in what’s called a global KV cache (global idea pool) in DeepSeek V4.1 Flash. Maybe you think of it as a word pool.
Now, previous decoders really were bottlenecked because they had to calculate and store KVs across multiple layers (read subjects and verbs for each layer).
In prefill, the model has to process the prompt in full and compute these learned representations. In the reading analogy, let’s just say that they have to turn pages into subjects and verbs that still contain all of the information. This is really hard and expensive for a human and it’s still really hard for a computer program.
Now, DeepSeek Flash’s amazing innovation is that instead of multiple pools, multiple collections of subjects and verbs (KV projections that pass through layers), you can have one pool, a smaller pool so to speak, and pull from that global KV cache.
The key insight is that the width of the stored representations can be much smaller, and those representations can be reused across layers.
If we are dealing with ideas (attention representations in the KV cache), we want to store them in a more compressed form.
To this point, I’ve been speaking of compression, but I am more interested in emotional compression, in whether the model still understands why Hermione in Harry Potter is missing classes or whether it’s internalized what sacrifice means.
What if I decide that encoding misses the point? That every word in the exact position is holy, sacred, immutable, and incapable of being summarized to an idea? That even a synonym of close to near is a non-negotiable? That colour to color is a boundary of oceans, and that u is a universe of meaning? If I won’t even budge on British spelling, can I believe in what LLMs do?
Here’s where I’ll leave you. There is no clean way to know that you can gain the same emotional experience of reading just by skimming the CliffNotes, and I’m skeptical that in the long run, we’d lose nothing. Maybe it’s not that much—a letter here, a word there. Maybe it’s just half an idea. Maybe ‘time machine’ is enough to understand that it’s an hourglass. I still want to see the sand fall.