How a conversation becomes a memory

· Squidler ·

7 min read Original article ↗

When you talk to an AI model, everything relevant has to go in with what you say. The model is stateless: it has no memory of its own, so each time, the context comes entirely from you. And there is a limit to how much fits, so you have to be selective about what goes in.

cost ∝ length²the best models todaylargest context windowa long-running agentkeeps going →history (context length)cost (compute + vram)

Processing a turn takes more compute and more VRAM (the memory on the GPU) as the history grows, so the time and energy climb with it. Even the largest windows today fill up over a long relationship. The curve is illustrative; it is why an agent keeps a compact set of memories instead of holding everything at once.

There is more than one way to deal with that, and which one fits depends on the task. It is often a combination.

SIMPLEit just dropsthe context windowSUMMARYsqueezed to a gist≈ summary of everythingMEMORYthe bits that matteredrecalled into the conversation

The context window is finite, so the older conversation has to go somewhere. The simplest agents just let it drop. Compaction squeezes all of it into a running summary, keeping everything but blurring the detail. Memory does neither: it pulls the few things that mattered out of the old turns and slots them back into the conversation, whole, when they are relevant.

For a long-running agent, it means remembering the details that mattered, no matter when they happened. Memories are exactly that: compact, relevant information that appears when we experience something familiar. But to be able to recall a memory, it has to be created first.

Of everything you experience in a day, only some of it becomes a memory. Two parts of your brain decide that. The hippocampus creates the memory, the what, the when, the who. The amygdala decides how strong it is.

For an agent, that experience is the conversations you have with it and the work it does.

this post

Hippocampus

Encode

conversation → separate episodes

this post

Amygdala

Weigh

salience at capture

Neocortex

Consolidate

episodes → facts, nightly

Recall

Resurface

the right memory, per turn

The two lit stages are this post. The dim ones run later, on what capture leaves them. For an overview of the whole system, see the first post.

Conversation becomes memory

The memories you keep of things that happened are episodic memories: what happened, when, and who said it. Someone tells you there's a meeting Wednesday, and that's what you keep. An agent makes the same kind of memory out of a conversation.

It runs in the background, not mid-reply. Once a conversation goes quiet, the hippocampus reads a window of recent messages in one pass. It's a single model call. Out come episodes: one per distinct thing that happened, tagged with what kind of moment it was, each linked back to the conversation it came from.

Some memories outrank others

Not every episode counts the same. The agent has your meeting down for Wednesday. Tell it no, it moved to Thursday, and you're correcting a fact it already had, not adding a new one. Capture marks that as its own kind of thing.

That kind is called an act, and who did what decides how much it carries. A correction from you sets the record straight, the highest authority there is. The agent taking back something it said matters too, but that's the agent fixing itself, not you. Yours wins.

Who said it changes what it's worth.

Later on, that's how conflicts get settled. So the tag had better be right.

barely countscounts the most

correction

you set a truth

decision

a choice made

commitment

a promise made

retraction

the agent walks it back

observation

information surfaced

routine

an ordinary lookup

How much each kind of act counts to start with. The further right the dot, the more it counts and the harder it is to lose. A correction from you sits at the far end; a routine lookup near zero.

What the memory weighs

How much an episode is worth comes from two things. The act sets a floor: a correction stays heavy even said flatly, a routine lookup starts near nothing. The other is intensity, how forcefully you said it, judged against your own usual tone. A normally terse person who suddenly types in capitals is a spike; the same words from someone who always writes that way are just their normal. The weight is whichever of the two is higher. That's the amygdala's job, and the number it lands on is called salience.

what did they write?

NO WE NEVER SAID FRIDAY, THIS IS WRONG!!

what kind of act is it?

whose message is it?

lowercase fragments, no exclamation marks

Floor from the act

Intensity vs this person's normal

the higher of the two wins

final salience

peak

intensity wins

Change any of the three and watch the salience move. (Intensity is used only when it's higher than the floor.)

Weight becomes lifespan

A routine lookup fades quickly. A peak moment holds far longer. Salience seeds that lifespan too.

That curve isn't ours. It's a model of human forgetting whose shape traces to 220 million real recall events, logged from people studying vocabulary in a language app. It turns on a single number: how long a memory lasts. We set that number from salience the moment the memory forms.

0%25%50%75%100%time since the memory formed →a peak momenta routine lookupone lifespan

memory lifespan

short-lived

how much is left

retrievable

Higher salience makes a memory last longer. It fades on the same curve human forgetting follows: most of it holds through the memory's lifespan, then a long tail that never quite hits zero. Drag the sliders to reshape it and move through time. Reading a memory never changes it, and time only lowers how easily it comes back, never what's stored.

Every memory fades, but none is ever deleted. Fading only decides what comes back first. When you think about something, or experience it again, a new memory forms, with its own fresh fade. Same with the agent: when something comes up again in a later conversation, that's a fresh experience, and another new memory. The pile of them is what makes a thing stick.

Tell an agent something once, and weeks later it can seem gone. It wasn't erased. It went in weighed low, so it faded, and a passing question might not be enough to pull it back up past everything newer. But give it enough of the context and it surfaces again. It's the same for us, sometimes you just need a bigger reminder before it comes back.

What actually gets captured

So what actually ends up in memory? Here's an exchange and exactly what capture makes of it: who did what, when, and how much it counts.

the conversation

userNo, the launch is Friday, not next week. I told you. Also grabbed coffee with Sara this morning.

agentGot it, Friday. How's Sara?

capture

the episodes

The user corrected the agent: the launch is Friday, not next week.

who the userwhat the launch datewhen Fridaycorrectionsalience peak

The user grabbed coffee with Sara this morning.

who the user & Sarawhat coffeewhen this morningobservationsalience low

One exchange, and everything capture makes of it: two events, each with its what, when, who, kind, and weight. No summaries, nothing invented, just the happenings, labeled.

Experience turned into memory, each one weighed for how much it matters. The same two jobs your hippocampus and amygdala do every day, without you noticing.

What capture hands off

Capture hands off a pile of fragments. Each episode is a single thing that happened, but when you experience the same thing enough times, it becomes a fact. That's what the neocortex does while you sleep. But that's the topic for another post.

Hippocampus

Encode

done

Amygdala

Weigh

done

still ahead

Neocortex

Consolidate

episodes become facts

still ahead

Recall

Resurface

the right memory, in time

The map from the top of the post, flipped: capture is done and hands off to the two lit stages. Facts next, then recall.