Press enter or click to view image in full size
An AI agent can have a million pieces of information and still have a terrible memory.
That sounds contradictory, but it is becoming one of the biggest problems in agentic AI.
We have gotten very good at giving AI systems more context. Bigger context windows. Longer conversation histories. Vector databases. RAG pipelines. Conversation summaries. Yet, after all of that, an agent can still forget what matters, remember what doesn’t, repeat mistakes, or confidently act on information that is no longer true.
So perhaps the question isn’t:
“How much memory can an AI agent have?”
The better question is:
“What kind of memory does an AI agent actually love?”
I think the answer is surprisingly human.
An agent loves memory that is relevant, structured, persistent, retrievable, and trustworthy.
Not a transcript.
Not a giant pile of embeddings.
Not 10,000 chunks of yesterday’s conversations.
Useful memory.
An AI agent doesn’t want to remember everything
Think about your own memory.
You don’t remember every conversation you’ve ever had word for word.
You remember things like:
“John prefers email.”
“We decided to launch in October.”
“That approach failed last time.”
“The customer hated the old dashboard.”
“Don’t make that mistake again.”
“Here is how we usually solve this problem.”
Those memories are useful because they have meaning.
They have context.
Some have a timestamp. Some have a source. Some are temporary. Some are incredibly important. And some should eventually be forgotten.
AI agents are starting to need exactly the same thing.
The most useful memory isn’t necessarily the memory that is easiest to store.
It is the memory that is easiest to use.
The four memories an agent actually needs
I think agent memory can be understood through four simple categories.
1. Working memory
This is what the agent needs right now.
The current task.
The current conversation.
The current objective.
The files it is looking at.
The decisions that matter for the task at hand.
Working memory is short-lived.
You don’t want to permanently store every thought an agent has while debugging a bug.
You want the agent to have enough context to finish the job.
2. Semantic memory
This is the agent’s knowledge about the world.
Facts.
Preferences.
People.
Products.
Projects.
Goals.
Relationships.
Constraints.
For example:
The API is built with FastAPI.The customer prefers AWS.The user prefers Python.The production database is PostgreSQL.The project deadline is October 15.
These are not necessarily events.
They are things the agent knows.
And this is where memory starts becoming dramatically more valuable than simple conversation history.
3. Episodic memory
This is the memory of what happened.
And I think this is one of the most underrated forms of agent memory.
For example:
The deployment failed on August 12.The failure was caused by the Lambda package size.We switched to a container image.The second deployment succeeded.
That is not just information.
That is experience.
And experience is incredibly valuable to an agent.
Imagine an autonomous coding agent working on a project for six months.
Would you rather give it a database containing thousands of previous conversations…
or give it a memory that says:
“We tried this architecture before. It failed because of X. We changed to Y, and Y worked.”
That’s a completely different kind of intelligence.
The agent isn’t just retrieving information.
It is learning from experience.
4. Procedural memory
This is the “how.”
How do we deploy?
How do we onboard a customer?
How do we run the test suite?
How do we troubleshoot this type of error?
How do we format a release?
How do we respond to a particular type of request?
Procedural memory is especially interesting because it starts to blur the line between memory and skills.
A good agent doesn’t just remember that something happened.
It remembers how to do something successfully.
This is where traditional RAG starts to break down
Traditional RAG is incredibly useful.
The basic idea is straightforward:
Find relevant information → retrieve it → put it into the prompt → let the LLM answer.
For many use cases, that works beautifully.
But agents introduce a different problem.
An agent is not simply answering questions.
An agent is:
planning → acting → observing → learning → acting again.
That changes everything.
Imagine an autonomous software agent working on your codebase.
On Monday it tries solution A.
It fails.
On Tuesday it tries solution B.
It succeeds.
Three weeks later it encounters the same problem again.
A traditional RAG system might retrieve documents about the technology.
That’s useful.
But what the agent really wants is:
“We already encountered this exact class of problem. Solution A failed for this reason. Solution B worked.”
That’s not ordinary retrieval.
That’s experience retrieval.
And experience retrieval is one of the foundations of truly useful agentic memory.
The problem with “just put everything in a vector database”
This is where I think the industry has taken a small detour.
For a while, the solution to almost every AI memory problem seemed to be:
Embed it. Put it in a vector database. Retrieve the top-k chunks.
And yes, that works.
But memory is not the same thing as documents.
A memory might be:
a fact
a decision
a goal
a preference
an event
a lesson
an error
a relationship
a procedure
These things behave differently.
They have different lifetimes.
Different importance.
Different retrieval patterns.
Different update rules.
And sometimes they contradict each other.
Imagine the agent has these two memories:
The customer uses AWS.
and six months later:
The customer migrated to GCP.
Which one should the agent trust?
Similarity search doesn’t magically solve that.
The system needs to understand that the second memory supersedes the first.
That is a memory problem, not merely a search problem.
The memory an agent loves is structured
Here’s a simple example.
Instead of storing:
“Conversation from March 4: user said they preferred Python for backend development…”
store:
TYPE: PreferenceFACT:
User prefers Python for backend development.SOURCE:
ConversationCONFIDENCE:
HighTIMESTAMP:
2026-03-04
Now imagine another memory:
TYPE: DecisionDECISION:
Production backend will use FastAPI.REASON:
Team expertise and existing Python infrastructure.TIMESTAMP:
2026-03-10
And another:
TYPE: ErrorPROBLEM:
Lambda deployment exceeded package size limit.FAILED APPROACH:
Bundling dependencies directly into the deployment package.SUCCESSFUL APPROACH:
Container image deployment.
Suddenly the agent has something much more interesting than a pile of chunks.
It has a memory system.
Good memory should know when not to speak
This might sound strange, but I think this is one of the most important properties of agent memory.
A good memory system should know when not to retrieve something.
Imagine you ask an agent:
“What should I do next?”
And your memory system injects 37 vaguely related memories into the context.
Technically, the retrieval system worked.
Practically, it failed.
The agent now has too much noise.
Good memory isn’t about maximizing retrieval.
It is about maximizing useful retrieval.
The best memory may be the memory that stays completely invisible until the right moment.
Memory should have a life
Human memory changes.
AI memory should too.
A fact can become outdated.
A goal can be completed.
A preference can change.
A decision can be reversed.
A lesson can become irrelevant.
An old memory shouldn’t necessarily be deleted.
Sometimes it should become inactive.
For example:
OLD DECISION
Use PostgreSQL.STATUS
Superseded.REPLACED BY
Use DynamoDB for this service.WHY
Scaling requirements changed.
Now the agent has history.
It knows not only what is true now, but also what used to be true.
That distinction becomes extremely important in long-running autonomous systems.
Memory needs provenance
Another thing humans naturally care about is:
“Where did you get that?”
Agents need this too.
Consider these two statements:
The customer wants enterprise SSO.
and
The customer mentioned they might want enterprise SSO someday.
Those are not the same memory.
One is a requirement.
The other is a possibility.
A trustworthy memory system therefore needs more than a fact.
It needs things like:
source
confidence
timestamp
context
provenance
Without these, an agent can gradually accumulate assumptions and eventually mistake those assumptions for facts.
That’s how memory can become dangerous.
Memory should help an agent become better over time
This is the part that excites me most.
Imagine an agent that remembers:
We tried this.
It didn’t work.
Here’s why.
We tried this instead.
That worked.
Now imagine the same agent six months later.
It doesn’t have to rediscover everything from scratch.
It has accumulated experience.
That begins to look less like RAG and more like learning.
Not model training.
Not fine-tuning.
Something different.
Operational learning through memory.
The model itself doesn’t necessarily need to change.
The agent’s experience changes.
And that experience changes what it does next.
So what does an AI agent actually love?
I would summarize it this way:
An AI agent loves memory that reduces uncertainty.
Memory that tells it:
“What matters?”
“What changed?”
“What did we decide?”
“What happened before?”
“What worked?”
“What failed?”
“What should I do differently this time?”
“Can I trust this information?”
That is vastly more useful than simply asking:
“What text is semantically similar to this query?”
From vector search to agentic memory
This leads to a broader shift that I think we are going to see accelerate.
The first generation of AI infrastructure was largely about finding information.
The next generation is increasingly about maintaining knowledge and experience.
That’s a fundamentally different problem.
Vector databases are excellent at similarity search.
RAG is excellent at grounding an LLM with external information.
But autonomous agents need something more.
They need a memory layer that can understand:
what to remember
how to remember it
when to retrieve it
when to update it
when to ignore it
when to forget it
and how memories relate to one another.
In other words:
Memory needs intelligence of its own.
The future agent won’t have one memory
I don’t think the future of agentic memory is going to be one enormous “memory database.”
It is more likely to become a collection of specialized memory systems working together.
Working memory for the current task.
Semantic memory for durable knowledge.
Episodic memory for experience.
Procedural memory for skills.
And an intelligent agentic layer deciding:
Which memory should I use right now?
That last part may turn out to be the most important part of all.
Because the smartest memory system isn’t the one that remembers the most.
It’s the one that knows what the agent needs to remember right now.
And that may be the real definition of agentic memory
We have spent a lot of time asking how to make models smarter.
Then we started asking how to give them tools.
Then how to give them context.
Now we’re asking how to give them memory.
But perhaps the deeper question is:
How do we give an AI agent experience?
Experience has structure.
Experience has consequences.
Experience changes behavior.
Experience contains failures, successes, preferences, decisions, and lessons.
And that is exactly what a good memory system should preserve.
So the next time someone tells you:
“Our AI agent has a million memories.”
I’d ask a different question:
“Great. Which ones does the agent actually love?”
Because in the end, the goal isn’t to build an AI that remembers everything.
The goal is to build an AI that remembers what matters.
And that may be the difference between an AI assistant that is impressive…
and an AI agent that genuinely gets better over time.