CIS 4394 Agentic AI · Fall 2026 · Dr. Xinyu Fu
CIS 4394 Agentic AI · Fall 2026 · Dr. Xinyu Fu
An agent that "remembers" is an agent whose harness wrote something down and read it back. Those are two separate design decisions — a write policy and a read policy — and neither one happens by default.
Both are "what the model sees." Only one of them still exists tomorrow.
| Context (the window) | Memory (a store) | |
|---|---|---|
| What it is | The tokens assembled for this model call: system prompt, tools, retrieved text, conversation so far. | Data your system deliberately saved outside the model — a row in a database, a file, a vector index. |
| Lifespan | One call. The next call gets whatever your harness re-assembles. | Until you delete it. Survives the run, the session, the deploy. |
| Where it lives | In the request. It is gone the moment the response returns. | In your infrastructure — and therefore in your backup, your access-control model, and your retention schedule. |
| Who controls it | Your prompt-assembly code, every turn (Week 3's context engineering). | Your write policy (what gets saved) and read policy (what gets loaded back). |
| What it costs | Tokens — every turn, forever. Bigger context = higher bill and, past a point, worse recall inside the window. | Storage plus a retrieval step. Cheap to keep, cheap to update — you pay tokens only for what you load. |
| Failure mode | Amnesia between runs; or, if you stuff everything in, context rot. | Stale, contradictory, or over-broad memories that quietly poison later runs. |
The standard taxonomy borrows its names from cognitive psychology: working, episodic, semantic, and procedural memory. IBM's explainer uses the same four for agent systems (IBM Think — What is AI agent memory? ↗). Click a box to see what belongs in it and what it costs you.
Amber = volatile. The three boxes at the bottom are long-term stores: they persist between runs, and something has to load them back in.
You are building a travel-booking agent for a mid-size firm. Four things happen during a run. For each one, decide which store it belongs in — or whether it belongs in a store at all.
Storing everything is the same mistake as putting everything in the context window — you just move the mess. A workable filter asks four questions of each candidate memory:
Who writes it matters too. Explicit writes ("remember that…") are auditable and rare. Automatic writes — where the agent decides at the end of a run what to save — scale better and are exactly where wrong facts get canonized. Keep a source and timestamp on every memory so a bad one can be traced and revoked.
The naive design loads the whole memory store into every prompt. That works for a demo with nine memories and collapses at nine thousand — you have re-created the context problem with a bigger bill. Real read policies are selective:
Then close the loop: log which memories were injected on the turn that produced a bad answer. Without that, a poisoned memory is invisible — the trace shows a confident model and no reason for it.
The engineering question is "what should the agent remember?" The business question is "what are we now holding, about whom, and for how long?" They have to be answered together, because the second one has consequences the first one cannot see.
A user talking to an assistant does not obviously expect a durable profile to be built from it. If your agent writes semantic facts about people, someone has to be told — and the agent should be able to say what it knows about you when asked.
"Forget me" has to be executable. That means every memory carries an owner ID and a timestamp, and deletion reaches the vector index and the backups too — not just the pretty table in the admin UI.
Memory is an input channel, so it is an attack surface. Text the agent read in one session and saved becomes an instruction it reads back later — prompt injection with a delay fuse (Week 2's J004). Treat stored memories as untrusted data, not as system prompt.