- Agents
- Observability
We audited our coding agent's memory. Drift was the real risk.
Across two Macs, 15 notes, and 66 local sessions, Claude Code auto-memory was a small share of estimated tool-originated context. Knowledge drift mattered more.
A coding agent can learn something useful, save it to local memory, and carry that lesson into future work. That is useful—until the same repository has different memories on two laptops, a checked-in document says something else, or an old note keeps steering new sessions.
We audited that problem in our own repository: two machine-local Claude Code memory stores, every note in the smaller store, and local transcripts from 66 sessions.
The result changed our priority:
In this sample, memory was not a meaningful cost center. It was an unmanaged source-of-truth problem.
Scope matters: this is one repository and one operator, not a benchmark. Store and note counts are observed. Token composition and residency are estimated. The workflow is our recommendation.
Define the memory layer first
Agent memory is not one thing. Live conversation history, repository instructions, machine-local notes, compaction summaries, prompt caches, and external retrieval have different lifetimes and cost behavior.
Here, memory means Claude Code auto-memory: notes Claude writes in a local project directory for reuse across conversations. Claude Code documents that auto-memory is machine-local, shared across worktrees in the same repository, and stored as plain Markdown. The first 200 lines or 25 KB of MEMORY.md, whichever comes first, load at conversation start; topic files load on demand when Claude reads them.
That boundary matters. The startup index is bounded, but any recalled topic file can still enter live context later. Neither behavior tells you whether a note is correct, current, or duplicated elsewhere.
If you need the broader map, our guide separates seven layers of agent memory by persistence, scope, and how they enter context.
What we actually measured
We used three narrow checks:
- Store comparison. We counted and inspected memory notes for the same repository on two Macs.
- Truth reconciliation. We read all 15 notes on the newer machine and compared each claim with checked-in instructions, skills, runbooks, and session notes.
- Context composition. We scanned 66 local session transcripts and estimated supported tool-originated material entering context: file reads, shell output, screenshots, documentation, memory operations, harness attachments, and other tool channels.
The transcript analysis does not represent every token sent to a model. It covers the supported tool-originated inputs visible in those local transcripts. Text estimates use character-based token approximation; screenshots use a dimension-based estimate rather than encoded file size. Modeled residency is not provider billing.
Finding 1: machine-local knowledge diverged
One machine held 197 notes for the repository. The newer machine held 15. Same person, same codebase, different local knowledge.
Based on note timestamps, both stores added roughly two notes per active day during the observed period. Store age is therefore our strongest explanation for the 197-versus-15 gap—not the laptop model. That is an inference from this sample, not a vendor claim about memory growth.
The contents diverged too. We found machine-specific facts recorded on the other machine, different on-disk note formats, and lessons available to only one local agent environment. Claude Code’s current documentation makes this scope explicit: auto-memory is not shared across machines or cloud environments.
The practical asymmetry is simple:
- A local note helps one agent environment.
- A reviewed repository instruction can travel with the codebase.
- A team knowledge system can reach more people, but only when retrieval and permissions actually surface the right material.
The right destination depends on the fact. A laptop-specific path issue belongs locally. A release invariant or architecture rule usually belongs in versioned repository documentation.
Finding 2: 10 of 15 notes overlapped the repository
We manually classified all 15 notes on the newer machine:
- 40% had a strong repository twin. The same guidance already existed in checked-in documentation.
- 27% partially overlapped. The repository described the mechanism; memory held an operational lesson or correction.
- 33% were memory-only. The note captured useful knowledge that had not reached shared documentation.
These are human judgments over 15 notes—not semantic-similarity scores or a population estimate.
The duplicates were not harmless. One note corrected an optimistic repository description of a formatting hook: the checked-in note said the hook reduced diff churn, while local memory recorded a real case where it expanded a small edit into a much larger formatting diff. Another memory note pointed to the wrong canonical skill for the policy it described.
Both failures came from the same pattern: the agent wrote a durable session note and a local memory near the same time, then no process reconciled them.
The best note in the set used a better pattern. It kept a short reminder locally and linked to the repository document that owned the full explanation. Memory acted as an index into canon, not a competing copy of canon.
Finding 3: memory was 0.19% of the measured slice
Across the 66 inspected sessions, memory operations represented 0.19% of estimated tool-originated tokens entering context.
Selected categories in the same measurement were:
- Harness attachments and bookkeeping: 28.7%
- Shell output: 22.7%
- Code reads: 22.1%
- Browser screenshots: 19.9%
- Repository Markdown reads: 5.0%
- Memory operations across measured channels: 0.19%
- Other supported tool channels: about 1.4%
This is a composition estimate for one workload. It is not an API invoice, a model-context snapshot, or evidence that every memory system is cheap.
Two corrections mattered during measurement.
Estimate screenshots from dimensions, not bytes
Transcripts may store screenshots as encoded blobs. Counting those bytes as text made images look like roughly three-quarters of all context traffic. Applying a dimension-based image estimate moved screenshots to 19.9% in our sample. The first approach overstated them by roughly 20–30×.
Separate entry from residency
A tool result can enter once and remain in retained conversation history across later requests. We approximated that persistence as:
Estimated residency = estimated tokens introduced × later requests carrying them, bounded by a detected compaction or session end.
That is an analysis model, not observed provider telemetry or a billing formula. Cache hits, client retention, compaction, and provider token accounting can all change the real cost. Anthropic’s own context-engineering guidance makes the broader point: context is finite, and the goal is the smallest set of high-signal information that supports the task.
What mattered more than the percentage
The audit found three ways a small local store can still cause expensive work:
- A stale note can trigger a wrong action. The direct token cost may be tiny; the resulting debugging or release delay is not.
- A useful note can stay trapped on one machine. Another agent or teammate relearns the same lesson.
- A duplicated note can outlive its source. The repository changes, but the private copy keeps sounding authoritative.
Research on repository context files points in the same direction. In one 2026 evaluation, generated or developer-provided context files increased inference cost by more than 20% on average and did not generally improve task success. That study concerns repository-level context files, not Claude auto-memory, so it does not prove our local-memory result. It does support a narrower principle: more context is not automatically better. See Gloaguen et al., “Evaluating AGENTS.md”.
How to audit agent memory
You do not need a new platform to start. Claude Code memory is plain Markdown, and /memory opens the relevant files.
1. Fix the scope
Choose one repository, machine, agent client, and time range. Do not merge unlike workloads into one average.
2. Inventory each store
Record file count, source bytes and lines, last modification, and which files enter automatically versus on demand. Treat each machine and agent client as a separate scope unless its documentation says otherwise.
3. Classify every durable claim
Use four buckets:
- Local-only: true only for this machine or environment.
- Canonical pointer: a short route to reviewed documentation.
- Duplicate: the same claim exists in memory and canon.
- Conflict or stale: the two stores disagree, or reality has moved on.
4. Promote what should travel
Move durable, machine-independent lessons into the repository or another reviewed shared source. Keep the local note short and point it at the owner document.
5. Correct before deleting
Delete a note when it is false or no longer useful—not because memory sounds expensive. If the lesson still matters, repair the canonical source first so deletion does not erase the only correct account.
6. Measure one change
Compare the same repository, agent, workload, and time basis before and after a change. Separate estimated tokens introduced from modeled residency, and mark any compaction boundaries. A smaller file is not automatically a better outcome if the agent must repeatedly rediscover the missing guidance.
What Ottto measures—and what it leaves to you
Ottto’s shipped Optimize views keep evidence types explicit:
- Snapshot estimate: static context footprint by recognized category, machine, and repository.
- Transcript estimate: tokens entering supported context categories.
- Modeled residency: estimated persistence bounded by detected compaction or session end.
- Observed where available: first-prompt, peak-turn, model-window, and compaction posture.
- Source-accounted economics: prompt-cache results only when the source provides the required token classes, rates, and pricing period.
Raw prompts, responses, code, and tool-result content are not inputs to these aggregate product read models. Missing coverage stays unavailable rather than becoming zero. Ottto does not delete memory, choose compaction points, assign an “AI-native repository” score, or compare your repository with external codebases.
That boundary is deliberate. Ottto can show where supported context accumulates and where machine-level footprints diverge; it cannot decide which note is true. Promotion, revision, and deletion remain reviewable human decisions.
Promote durable knowledge. Do not purge blindly.
Our sample did not support the dramatic story that local memory was secretly consuming the context budget. It supported a quieter, more useful conclusion: agent knowledge had fragmented across machines and drifted from reviewed documentation.
So our maintenance rule is straightforward:
Keep machine facts local. Promote durable lessons to the repository. Replace duplicate prose with pointers. Remeasure before claiming savings.
Read the diary because it may hold the only copy of something your entire team should know.