Seven layers of agent memory
Live context, repository instructions, local notes, runtime state, compaction summaries, prompt caches, and retrieval all get called “memory.” They differ in persistence, scope, cost, and who can review them.
Do not ask only whether an agent has memory. Ask what is present, when it enters a request, where it persists, and who can review it. A laptop note is not a checked-in instruction. A prompt cache is not a knowledge store. A compaction summary is not the original conversation.
Seven distinct layers
Some layers persist for months; others disappear with a process. Some enter every request; others appear only after a lookup. Naming the layer first prevents false portability, privacy, and cost assumptions.
Live context
The messages, instructions, tool results, and retrieved material included in the current model request. The context window is the capacity; live context is what occupies it.
Watch: retained history may be sent again on later requests, raising input volume and crowding out relevant signal.
Repository and project context
AGENTS.md, CLAUDE.md, skills, runbooks, and architecture docs are reviewed, versioned context. They travel with the project and reach people or agent clients configured to read them.
Watch: broad or stale instructions can become recurring input. Keep standing guidance concise and route detail on demand.
Agent-managed local memory
Generated or curated notes a client can reuse across chats on one host. Storage, defaults, scope, and loading behavior are product-specific.
Watch: useful facts can stay on one machine, duplicate repository docs, or drift after the project changes.
Tool and runtime state
Open terminals, browser state, task queues, checkpoints, and other state kept by a client or tool. It supports continuity, but becomes model context only when surfaced to the model.
Watch: state may vanish when a process, sandbox, or machine changes. Record durable decisions elsewhere.
Compaction summaries
When a client or provider compacts a conversation, selected history continues in a smaller representation. It preserves continuity, not a verbatim record.
Watch: a summary can omit detail and alter the reusable prompt prefix. Mark the boundary before comparing context growth.
Prompt caches
A provider may reuse a matching input prefix. That can change processing latency and the billed rate, but cached tokens still occupy context and the cache is not durable knowledge.
Watch: high reuse can coexist with an oversized prompt. Token classes and cache lifetimes differ by provider.
External retrieval
Knowledge held in databases, search indexes, issue trackers, or document systems. Only the material returned by an authorized query or tool call enters live context.
Watch: “connected” does not mean “retrieved in this request.” Keep access, indexing, retrieval, and citation coverage separate.
Map scope before making claims
These are practical categories, not one vendor contract. Exact lifetime, scope, and loading behavior depend on the client, configuration, model, and version.
| Layer | Lifetime | Scope | Enters context | Use |
|---|---|---|---|---|
| Live context | Request or session | Conversation | Sent on this request | Work in progress |
| Repository context | Versioned | Repository or path | Startup, path match, or read | Shared rules and docs |
| Local memory | Across chats | Host, client, repo, or chat | Client-specific load or recall | Host-local lessons |
| Runtime state | Process or task | Client, tool, or sandbox | When surfaced | Operational continuity |
| Compaction summary | Until replaced or ended | Conversation segment | After compaction | Lossy continuity |
| Prompt cache | Provider-defined | Matching prefix | On a cache match | Input reuse, not storage |
| External retrieval | Store-defined | Source and permissions | When retrieved | On-demand knowledge |
Why context size affects cost and latency
Most agent clients build each model request from retained context plus new material. As a conversation grows, later requests may resend standing instructions, prior messages, tool results, and retrieved files. Exact retention and compaction behavior is client- and provider-specific.
A mental model, not a billing formula. Actual cost is the sum of provider-reported token classes priced per request.
Startup creates a floor
The first request can already contain system, tool, repository, and memory context. Treat a measured first prompt as a total, not automatic proof of which source caused it.
Long sessions compound
Material introduced early can ride through many later requests. Entry size alone misses this residency effect; the request-by-request curve reveals it.
Caching changes the rate
A cache match may reduce cost and input processing latency. Some providers report cache classes separately; others report cached tokens inside input. Use the source convention.
Compaction changes the curve
Compaction usually reduces retained history by carrying a smaller summary forward. The summary is lossy, and the compaction boundary must stay visible in comparisons.
Window fill is a capacity signal, not a bill. Its denominator must match the measured turn’s model and window; cost requires that source’s token classes, cache evidence, rates, and pricing period.
What Ottto observes — and what it does not
Ottto keeps observed, estimated, and unavailable evidence separate. It can connect local aggregate evidence with provider usage records, but only inside the source, machine, repository, session, model, and time scopes the data supports.
- Static footprint: supported local snapshots estimate recognized context categories, scoped to one machine and repository.
- Context composition: transcript-derived estimates group detected inputs into supported categories.
- Modeled residency: an estimate of how long detected inputs persist across later requests, bounded by session end or compaction.
- Context posture: supported first-prompt, peak-turn, model-window, and compaction evidence, with unresolved coverage kept explicit.
- Prompt-cache economics: provider usage fields interpreted under that source’s accounting convention; net value appears only when the required evidence exists.
- Raw prompts, responses, transcripts, tool results, and code content are not product read models for these aggregate views.
- Ottto does not delete memory, clear sessions, choose compaction points, or apply context changes for you.
- Observed memory tokens do not reveal source bytes or line count, so they cannot prove distance from a vendor file-size cap.
- Ottto does not assign an “AI-native repository” score or compare your repository with external codebases.
- Missing telemetry stays unavailable; it is not converted into zero usage, zero cost, or confirmed savings.
A six-step context review
Keep the comparison narrow enough to act on. Mixing unrelated tools, repositories, models, or workloads can flatten a real difference into an unhelpful average.
- Fix the scope.Choose one source, machine, repository, model or window basis, workload, and time range. Record missing coverage before comparing.
- Separate start from growth.Compare the first measured request with later requests and peaks. A high first prompt is a lead to investigate, not proof of one cause.
- Inspect entry and residency.“Estimated tokens entered” shows what appeared; modeled residency shows what persisted. Do not causally join today’s static snapshot to an unmatched historical session.
- Mark compaction boundaries.Compare segments before and after each detected boundary. A post-compaction drop is a new baseline, not automatically a telemetry error.
- Verify cache arithmetic.Identify the provider’s token convention and confirm reads, writes, rates, and pricing period. Missing write evidence makes net value unknown, not zero.
- Promote, change, remeasure.Move shared lessons into reviewed repository docs. Then review one change and compare the same scope and workload after it.
Provider contracts stay separate
Claude Code documents machine-local auto memory shared across worktrees in one repository. It loads the first 200 lines or 25 KB of MEMORY.md at conversation start, whichever comes first; topic files load on demand. That is a source-file boundary, not a percentage Ottto can reconstruct from token estimates.
Local Codex memories use a separate store under the Codex home directory, are off by default, and have per-chat controls for use and future generation. ChatGPT Work memory is a separate account or workspace surface. Claude’s storage layout, startup boundary, and controls do not apply to Codex.
External retrieval systems define another boundary: the store may be durable and shared, but only authorized material returned by retrieval enters the request. None of these contracts establishes behavior for another client.
Further reading
- Your coding agent keeps a diary. We read ours.An evidence-led audit of local memory and repository overlap.
- Claude Code memoryOfficial storage, scope, loading, and controls.
- Codex memoriesOfficial local-memory behavior and per-chat controls.
- Manage Claude Code costsSession length, context, compaction, and usage guidance.
- Effective context engineering for AI agentsWhy the smallest high-signal context is usually the most useful.
Ottto connects context composition, static footprint, measured first and peak context, compaction signals, and prompt-cache economics — with scope and evidence gaps attached.