AI Agent Memory and Planning: Short-Term, Long-Term and State

A user is halfway through filing an expense report with an assistant. Twenty messages ago they said that all amounts should be in euros and charged to cost centre CC-4410. Now the assistant totals the report in dollars and bills it to the default cost centre. The model did not become less capable in the middle of the conversation. That instruction simply fell out of the text the model was shown. Memory in an agent is not a feature of the model; it is a decision about what text to put in front of it on each call.

Three kinds of memory, three different jobs

A model call has no state of its own. Everything it "remembers" is whatever your program includes in that call's input. That input has a size limit, and every token in it costs time and money, so agents split memory into layers with different lifetimes.

The last row is the one people skip. State is what your program knows for certain, such as "the report was submitted" or "step 2 failed". It should live in ordinary data structures, not inside the conversation text, so that trimming the conversation can never erase it.

A memory object you can inspect

The class below keeps a small verbatim window, folds older turns into a deliberately crude summary and holds long-term facts in a dictionary. Real systems use a model to write the summary and a vector or keyword index to recall facts, but the structure is the same, and the crude version makes the loss easy to see.

class AgentMemory:
    """Three layers: a small raw window, a lossy summary of older turns, and durable facts."""

    def __init__(self, window=3, summary_chars=70):
        self.window = window
        self.summary_chars = summary_chars
        self.recent = []      # short-term: verbatim recent turns
        self.summary = ""     # medium-term: compressed older turns (lossy!)
        self.facts = {}       # long-term: explicit key -> value, survives across sessions

    def add_turn(self, speaker, text):
        self.recent.append(f"{speaker}: {text}")
        while len(self.recent) > self.window:
            oldest = self.recent.pop(0)
            # Crude stand-in for an LLM-written summary: keep only the tail.
            self.summary = (self.summary + " / " + oldest)[-self.summary_chars:]

    def remember(self, key, value):
        self.facts[key] = value

    def build_context(self, task):
        """What would actually be sent to the model for this task."""
        words = task.lower().split()
        relevant = {k: v for k, v in self.facts.items() if k in words}
        return {"facts": relevant, "summary": self.summary, "recent": list(self.recent)}


mem = AgentMemory()
mem.remember("currency", "EUR")
mem.remember("costcentre", "CC-4410")

turns = [
    ("user", "I need to file my Lisbon trip, hotel was 3 nights"),
    ("agent", "Got it. Please send the hotel receipt."),
    ("user", "Receipt attached, total 412.50"),
    ("agent", "Received. Any meals to add?"),
    ("user", "Yes, two dinners, 38 and 41"),
    ("agent", "Added. Anything else?"),
    ("user", "A taxi, 23.10, then that's all"),
    ("agent", "Added the taxi. Ready to submit?"),
]
for speaker, text in turns:
    mem.add_turn(speaker, text)

ctx = mem.build_context("which currency and costcentre should the report use")
for layer, content in ctx.items():
    print(f"{layer:8}", content)

facts    {'currency': 'EUR', 'costcentre': 'CC-4410'}
summary  agent: Received. Any meals to add? / user: Yes, two dinners, 38 and 41
recent   ['agent: Added. Anything else?', "user: A taxi, 23.10, then that's all", 'agent: Added the taxi. Ready to submit?']

Look at the three lines of output. The durable facts were recalled because the task mentions the words "currency" and "costcentre", so those two instructions survive no matter how long the chat gets. The recent window holds only the last three turns verbatim. And the summary is the interesting part: it is only a truncated tail, so it still mentions the dinners but the Lisbon trip and the hotel total of 412.50, both stated at the start, are already gone. If the report needed that figure later, nothing in the context could supply it.

That is the central trade-off. Trimming and summarising keep cost and latency under control, and both are lossy. Anything the agent must never lose should be promoted out of the conversation into state or long-term facts at the moment it appears, for example by storing each receipt total as a structured record the moment it is read, rather than hoping a summary keeps it.

Plans: useful when steps depend on each other

A plan is an explicit list of intended steps, produced up front and then worked through. It helps when a task has many dependent steps, because writing the whole route first makes missing steps visible and lets you show progress to a user. It costs an extra model call and can go stale the moment the world disagrees. The sample below runs a plan with a bounded number of replans.

print("--- plan and replan ---")

def run_plan(steps, execute, replan, max_replans=1):
    plan = [{"step": s, "status": "todo"} for s in steps]
    replans = 0
    i = 0
    while i < len(plan):
        item = plan[i]
        ok, note = execute(item["step"])
        item["status"] = "done" if ok else "failed"
        print(f"{item['status']:6} {item['step']}  ({note})")
        if not ok:
            if replans == max_replans:
                print("replan budget used up, escalating to a human")
                break
            replans += 1
            extra = replan(item["step"], note)
            plan[i + 1:i + 1] = [{"step": s, "status": "todo"} for s in extra]
        i += 1
    return plan


def execute(step):
    # Fake world: the live exchange-rate service is down today.
    if step == "convert receipts to EUR":
        return False, "rate service timed out"
    return True, "ok"

def replan(failed_step, reason):
    if "rate service" in reason:
        return ["look up rate in cached monthly table", "convert receipts using cached rate"]
    return []

final = run_plan(
    ["collect receipts", "convert receipts to EUR", "attach receipts to report", "submit report"],
    execute, replan)
print([p["step"] for p in final if p["status"] == "todo"] == [])

--- plan and replan ---
done   collect receipts  (ok)
failed convert receipts to EUR  (rate service timed out)
done   look up rate in cached monthly table  (ok)
done   convert receipts using cached rate  (ok)
done   attach receipts to report  (ok)
done   submit report  (ok)
True

Step two failed because the pretend rate service timed out, so the runner asked for new steps, spliced them in directly after the failure, and carried on. The failed step stays in the plan marked failed, which keeps an honest record. The final check prints True because no step is left in the todo state. Had a second step failed, the runner would stop and hand over to a human instead of replanning forever.

Common mistakes

• Keeping task state only in the prompt. After trimming, the agent asks the user to confirm something it already did, or does it twice. Track step status and results in program variables.
• Summarising away the crucial detail. Symptom: the agent forgets an amount, a date or a name that was stated early. Pull exact values into structured state when they first appear.
• Recalling everything every time. Loading the whole long-term store into each call inflates cost and distracts the model. Retrieve only what is relevant to the current task, as discussed in Retrieval-Augmented Generation.
• Never expiring memories. Old preferences override new instructions. Symptom: the agent keeps using last quarter's cost centre. Date entries and let newer facts replace older ones.
• Replanning without a limit. An unbounded replan loop is just an infinite loop with extra steps. Give it a budget and an escalation path.

How this shows up in real projects and interviews

Production assistants usually combine a windowed conversation, a handful of persistent user settings and a small piece of task state kept in a database row. Fancy planners are rarer than demos suggest, because a short loop with a good checklist in the instructions is easier to debug. In interviews, "how does your agent remember things across sessions" is really asking whether you know that memory is application code: what is stored, how it is retrieved, how it is corrected and how privacy is respected. A good answer names what is stored, why, for how long and who can delete it.