72% fewer reader tokens,
once a session runs long.

The headline

Four answers

The study answers four questions, each labelled with the kind of evidence behind it.

By session length

The aggregate is real, but it rests on the long sessions. Change in reader input tokens, tiered context against full history, by the number of exchanges in the session:

Where the tokens go

Mean composition of the tiered context at the last step of a session, in tokens:

From six exchanges up, the tiered context is roughly flat at three thousand tokens however long the session grows. The digest is capped by its token budget at around nine hundred, the raw tail is always the latest three exchanges, and the background summary is bounded. Full history has no such ceiling: it grows with every exchange and is re-sent on every step. That is the whole mechanism, and why the saving concentrates where it does.

Two components are zero by construction. AAAK, Kemory's storage and export dialect, is never part of the default context — it is not a compression scheme for prompts, and no AAAK byte ratio is used as a token count anywhere in this study. Recalled memories are zero because a replay has no query to recall against; see the caveats for which way that cuts.

What it is worth

Every dollar figure here is a published-rate scenario: replayed token counts multiplied by list prices from a versioned rate card dated 22 July 2026. None of it is observed spend.

The compactor route

The identical compactor workload — 6.2M input and 1.7M output tokens — priced under each route:

This is a pricing comparison on a fixed workload. It claims nothing about whether the digests each model would write are equally good.

The net position

Two things move this in opposite directions. The tokenizer proxy undercounts Anthropic tokens, so the frontier costs are a floor. And prompt caching is not modelled; real full-history spend with prefix caching is lower, so the net saving is a no-caching upper bound. Quote it as one.

What we compared

For every real session, both policies are replayed at every decision step — once per exchange — and the reader input each would send to a stateless API is summed across the session.

Summing across steps is the point. A stateless API is sent the whole context again on every turn, so full history costs grow roughly with the square of session length, and that is where compaction earns its keep.

How the replay works

The replay mirrors the service's own functions rather than approximating them. Exchanges are reconstructed with the same turn-pairing the backend uses; the tiered context is assembled as the session-context service formats it; and the rolling digest is folded in batches of three exchanges, exactly as the service refreshes it.

The digest is produced by Kemory's deterministic extractive path — the code the service runs when no AI backend is configured. That makes the replay reproducible and free of external calls. An LLM-written digest would differ in wording but is bounded by the same token budget, so the reader-side accounting is representative.

The compactor workload is the token count of the prompt the digest generator would be fed, plus the digest it produces, per batch. That workload is what section A prices under each route.

Token counts use tiktoken's o200k_base encoding as a documented proxy: the open-weight models Kemory routes to have no tiktoken mapping. No byte count is used as a token count anywhere.

Evidence classes

Three kinds of evidence are used, and they are never merged without a label.

Privacy: the measurement reads conversation text into memory only to count it. Nothing it writes contains conversation text, personal data or credentials — only counts and salted pseudonyms — and it never writes to the database.

Caveats

Five things a careful reader should know before quoting any number on this page.

1. One account dominates the sample

This is the most important one. 1,227 of the 1,314 sessions belong to a single account, and so does every session longer than two exchanges. The other 23 accounts contribute 87 short sessions and about two thousand tokens between them. Since the saving lives in the long sessions, the headline describes one heavy user's working pattern, not a population of users.

The method does not depend on whose sessions they are, and the mechanism behind the result — a bounded context against an unbounded one — is general. But a different mix of session lengths gives a different aggregate, and a user whose sessions are mostly short would see tiered context cost them tokens. A multi-account re-run is the obvious next measurement.

2. Replay, not billing

Kemory does not currently persist what the compactor actually reads and writes: the provider's usage figures are discarded after each call, and no session digests were stored in the measured database. So the compactor numbers are a replayed workload, and every dollar figure is that workload at list price. The measurement package includes an analysis of the instrumentation that would turn the replay into an observation.

3. Bounds that cut both ways

4. Tokens, not answers

This measures what is sent to the model. It does not measure whether the model answers as well from a digest as from the full transcript. A saving that degraded answers would not be a saving, and this study cannot rule that out. The routing comparison is likewise a pricing comparison only.

5. A live snapshot

The figures are a snapshot of all persisted chats. The code is deterministic over a fixed database — two back-to-back runs produce byte-identical outputs — but a background worker refreshes namespace summaries, so re-running against a live database moves the tiered and compactor totals by around 0.2%. Full-history counts do not move. For an exactly citable sample, pin a date window and run against a quiesced snapshot.

What we have not measured

Reproduce it

The measurement package — the replay code, its unit tests, the rate card, the data dictionary, the instrumentation-gap analysis and the aggregate outputs behind every table on this page — is published in the Kemory Community Edition repository.

It reads the same chat tables Community Edition creates, so you can run it against your own local install and see what tiered context does with your sessions — which is a better answer to caveat 1 than anything we could publish.

cd benchmarks/context_compaction_empirical
pip install -r requirements.txt
export KEMORY_ANALYSIS_SALT="<any stable secret>"

# 1) unit tests — rate arithmetic, zero denominators, batching, the AAAK audit
python -m unittest discover -s tests -p 'test_*.py'

# 2) the measurement — writes out/*.csv and out/headline.json
python -m src.run_analysis --limit 0 --out out --tokenizer proxy

# 3) charts, then one self-contained HTML report
python charts/make_charts.py --out out
python build_report.py --out out --report kemory_compaction_report.html

The loader is strictly read-only and streams from Postgres through docker exec, so it needs no exposed port and no password; the package README has the exact invocation for a Community Edition stack. Content is held in memory only long enough to count it.

We publish the aggregate outputs. The per-session and per-step files stay private: they hold nothing but counts and salted pseudonyms, but together they would still show how one account uses the product. Run the package yourself and you get your own.