Repository navigation
curating-context: the persisted bytes/token ratio is calibrated on the policy file alone, so the write guard over-reports reference docs by ~13% #172
Description
Found while running curating-context on CannObserv/replicator (issue #35, PR #40). The write guard reported docs/contracts/replicator-boundaries.md 385 tokens over a 10,000 budget. The exact count is 9,152 — 848 under. A maintainer opened a tracking issue and deferred real work onto a file that was never over budget.
Cause
measure-context.sh derives the ratio it persists from the policy file only:
RATIO_X100=$(( P_BYTES * 100 / P_TOKENS ))
...
printf '%d.%02d\n' ... >"$ROOT/.skills/context-token-ratio"ctx_bytes_per_token_x100() then hands that single number to context-budget-guard.sh and context-delta.sh, which apply it to every file on the surface.
Markdown does not tokenize at one rate. On this repo:
| File | bytes/token |
|---|---|
AGENTS.md (policy) |
2.52 |
docs/ARCHITECTURE.md |
2.73 |
docs/contracts/*.md |
~2.83 |
The contract docs are dense in tables, inline code, and markdown links; the policy file is mostly prose. Calibrating on prose and applying to tables inflates the estimate by the ratio gap.
Measured error
| File | guard estimate (2.49) | exact | error |
|---|---|---|---|
docs/contracts/replicator-boundaries.md |
10,386 | 9,152 | +13.5% |
docs/contracts/content-replicate-issuer-contract.md |
11,708 | 10,284 | +13.8% |
docs/contracts/content-fetch-issuer-reference.md |
9,997 | 9,234 | +8.3% |
docs/ARCHITECTURE.md |
12,150 | 11,075 | +9.7% |
AGENTS.md |
6,427 | 6,349 | +1.2% |
The policy file is accurate — it is what the ratio was fitted to. Everything else is over-reported, worst on exactly the doc class most likely to sit near budget.
This is not a new observation in the repo's own ledger: the 2026-08-11 row already carries "issuer-reference was never over budget (guard estimate ran 9.4% high vs exact)". It recurred because the cause was in the script, not the run.
Why it matters more than the number
SKILL.md already names the failure mode — "a budget warning that is routinely ignored stops being a signal". A guard that cries wolf on the doc class nearest budget trains maintainers to ignore it, and the skill has no way to distinguish a false OVER from a real one without an --exact run, which is the thing the guard exists to avoid needing.
It also biases the rationalization table's own advice: "Gate on tokens" is right, but the continuous surfaces are not gating on tokens — they are gating on prose-calibrated bytes.
Suggested fix
Persist a ratio fitted over the whole measured surface, not the policy file:
RATIO_X100=$(( (P_BYTES + DOCS_BYTES) * 100 / (P_TOKENS + DOCS_TOKENS) ))Both totals are already computed for totals.tokens_live. On this repo that moves 2.51 → ~2.75 and drops the worst error from +13.5% to roughly +3%.
Better still, and cheap: persist per-file ratios for the docs alongside the aggregate, and have ctx_bytes_per_token_x100() prefer a file's own last-observed ratio when one exists. A doc's byte/token mix is stable between runs; that is exactly the signal being thrown away.
Either way .skills/context-token-ratio's single scalar becomes a format change, so it warrants the validation gate rather than a drop-in.
Smaller, separate
prove-no-loss.sh has no way to record "unaccounted and warranted". A run with legitimate class-C tightenings exits 3, and the honest ledger value becomes no_loss: "failed" — indistinguishable from a run that genuinely dropped a line. A --warrant <file> accepting justified lines, or a third verdict, would let the ledger stay truthful without conflating the two.
Activity
Merge pull request #175 from gregoryfoster/fix/172-surface-ratio
gregoryfoster commentedon Aug 15, 2026
Fixed in #175, taking your primary suggestion. Two things in the report turned out to be already-fixed, and one detail changed the shape of the fix.
Your headline symptom is a stale-pin artifact
replicatoris pinned at3fc7b71e(v1.2, 2026-08-05), which predates the per-file calibration —ctx_est_pair/ctx_est_tokens_forlanded infd839ea(#145) after it.replicatorhas no.skills/context-token-countsat all (404), so every file there is priced by the repo ratio.Your "better still" suggestion — persist per-file ratios and prefer a file's own last-observed one — is therefore already implemented and has been since #145.
context-budget-guard.shcallsctx_est_pairand logsest=fileorest=repoper decision. On a current pin,docs/contracts/replicator-boundaries.mdwould have been priced from its own last exact measurement, not fromAGENTS.md's ratio.That repo's pin is stuck because its refresh hook is half-installed — symlink present,
.claude/settings.jsonregistration absent (#167, repaired in #170, per-repo issue at CannObserv/replicator#39). Registering the hook clears the false positive as a side effect.Your primary suggestion was live, and is now fixed
The repo ratio is still the fallback for any file with no row, so fitting it to the policy file alone was a real defect. Validated against
usa-wa's committed calibration — 24 files of exact ground truth, since replicator's pin has none:Which matches your +13.5% → ~+3% estimate closely. The bias is the part that moves: 19 of 24 files over-reported, which is exactly the "cries wolf on the class nearest budget" mechanism you named.
One detail that changed the fix
Your suggested one-liner would have been wrong as a straight substitution.
RATIO_X100has three consumers with two meanings — it is also whatpolicy.bytes_per_tokenreports, and what the section/subsection figures divide by so the parts sum to the policy total. That site carries a comment recording having fixed that contradiction once already (a ~60% mismatch), so a surface-wide figure there would have reintroduced it.So there are two variables now:
RATIO_X100stays policy-only for reporting and sections,SURFACE_RATIO_X100is what gets persisted. Both pinned by tests.Also, only exactly-counted rows are aggregated — folding an estimated doc in would re-derive the divisor it was computed with, the same self-confirming measurement the
exact_flaggate already refuses.Your "smaller, separate" item is already implemented too
prove-no-loss.shexit 0 is "every line accounted for, or warranted",loss_warranted: <M>is in its output, the count reaches the row viarecord-telemetry.sh --no-loss-warrants M, and.skills/context-loss-okholds the warrants. Same cause as above: v1.2.Net
One live defect fixed; the other three observations were all the same stale pin. Worth stating for the cohort — a finding from a repo pinned below current should be re-checked against
mainbefore it is filed, or the same fixed bug gets re-reported once per stale repo. That is an argument for #167's repairs being sequenced ahead of further cohort work.Merge pull request #178 from gregoryfoster/fix/cr-round1
#159 fix: one band per population, and figures that are true today
#161 docs: restore two clauses the 13-token figure had bought
#218 docs: CR round 1 — one pointer, placed where it covers both gaps