claude-dig

English | 简体中文

Never lose context to compaction. Your Claude Code forgets what it just did — now it can dig it back.

When a long session triggers auto-compact (or /compact, or /clear), Claude Code replaces the conversation with a lossy summary. The model forgets rejected approaches, past errors, and its own decisions — and starts confidently guessing about things it did an hour ago.

The full history is still on disk in ~/.claude/projects/. claude-dig makes it archived, restored, and searchable by the model itself:

1. Archive — the PreCompact hook stores a byte-exact copy of the transcript before compaction hides it.
2. Restore — the SessionStart hook re-injects a working-state snapshot (recent user messages, todos, touched files) right after compaction.
3. Dig — the dig / dig_read / dig_state MCP tools let the model search the archive itself, so "didn't we reject FTS5?" becomes a lookup instead of a guess.

Inspired by OpenAI Codex's token-budget history & notes tools (openai/codex PRs #27488 / #39827) — Claude Code deserves the same. Built on the extension points the Claude Code team itself points to (anthropics/claude-code#70555: "…though not as a single built-in working-state object" — this is that, done properly).

Install

Requires Rust (or grab a release binary once published):

cargo install --git https://github.com/Dragonliu2018/claude-code-dig

Then register the plugin with Claude Code in your project:

cd your-project
claude plugin install /path/to/claude-dig   # or add as a marketplace plugin

Or wire it manually — add to .claude/settings.json:

{
  "hooks": {
    "PreCompact": [{ "hooks": [{ "type": "command", "command": "claude-dig precompact", "timeout": 15 }] }],
    "SessionStart": [{ "matcher": "compact|clear", "hooks": [{ "type": "command", "command": "claude-dig session-start", "timeout": 10 }] }]
  },
  "enableAllProjectMcpServers": true
}

and copy this repo's .mcp.json into the project. The skills/dig/SKILL.md teaches the model when to dig.

Zero configuration. Zero runtime services. Everything stays in <project>/.claude/dig/.

The tools the model gets

Tool	What it does
dig(query, session?, limit?)	Full-text search across all archived pre-compaction history of the project. Works in any language, including Chinese and Japanese.
dig_read(match_id)	Read the full archived message behind a match, including its tool calls.
dig_state(session?)	The working-state snapshot taken at each compaction: user messages, todos, touched files.

There are human-facing commands too: claude-dig dig "<query>" and claude-dig state — dig your own history from the terminal. claude-dig report shows your dogfooding telemetry (model dig counts, hit rate, inject→dig conversion, median latencies — all computed locally from events.jsonl).

Dogfooding results (real session)

From the first four days of daily use on a real project — a long Q&A session about Apache Doris internals, compacted twice:

$ claude-dig report

  Compactions  2 snapshots · median 4 ms
  Injections   2 restored · 2 followed by a model dig (100%)
  dig calls    2 model · 0 human · 100% with hits · median 1 ms
  dig_read     1 ok · 0 failed

  Top queries
      1×  MemTable flush segment Doris Stream Load
      1×  write_buffer_size 默认值 MemTable 阈值

What actually happened: the session's original explanation of Doris Stream Load internals (given eight days earlier) had been compacted away. After compaction, the model spontaneously searched for it — dig("MemTable flush segment Doris Stream Load") — then called dig_read on the best match and recovered the full original explanation, answering from it instead of from the lossy summary. Median hook cost: 4 ms; median search: 1 ms.

How it works

claude-dig is a single Rust binary that plugs into two documented Claude Code extension points and adds one MCP server. No daemon, no cloud, no API keys.

                        Claude Code session
                               │
                auto-compact or /compact or /clear
                               │
              ┌────────────────▼────────────────┐
              │   PreCompact hook (≤15s)        │
              │                                 │
              │  1. ARCHIVE  byte-exact copy of │
              │     the transcript .jsonl       │
              │  2. EXTRACT  deterministic      │
              │     working-state snapshot      │
              │     (user messages, todos,      │
              │      touched files)             │
              │  3. INDEX    rebuild the        │
              │     search index                │
              └────────────────┬────────────────┘
                               │ compaction happens
              ┌────────────────▼────────────────┐
              │  SessionStart hook              │
              │  (matcher: compact|clear)       │
              │                                 │
              │  RESTORE: inject the latest     │
              │  state snapshot + a reminder    │
              │  to dig before guessing         │
              └────────────────┬────────────────┘
                               │
                post-compaction session continues
                               │
        model is unsure about something earlier ──┐
                               │                 │
              ┌────────────────▼────────────────┐ │
              │      dig MCP server (stdio)     │◄┘
              │                                 │
              │  dig(query)      → snippets     │
              │  dig_read(id)    → full message │
              │  dig_state()     → snapshots    │
              │                                 │
              │  reads the byte-exact archives  │
              │  — not the lossy summary        │
              └─────────────────────────────────┘

Why byte-exact archiving matters

Compaction replaces your conversation with a summary — and the summary is where the loss happens. Claude Code's own transcript file (~/.claude/projects/<cwd>/<session>.jsonl) still holds everything, but it gets rotated/cleaned, and the model can't see it. claude-dig copies that file before compaction touches it, so the archive is the ground truth, not a summary of a summary.

Why deterministic extraction instead of an LLM summary

The PreCompact hook runs in seconds, inside your session, with no API key. Instead of asking a model to summarize (slow, costly, lossy, can fail mid-compaction), the state snapshot is distilled mechanically:

• Recent user messages, verbatim — the user's own words are the ground truth of intent
• The latest TodoWrite list — the ground truth of task state
• Recently touched files — the ground truth of focus

The predecessor project Uncompact (and later compact-plus) bet on LLM-generated "context bombs" pushed at the model after compaction. A summary can't know what the next question will need — a search tool can. Injection answers one guessed question; dig answers every question, including the ones nobody anticipated.

Why a two-stage read (dig → dig_read)

Search results return compact snippets with ids. Only when a snippet looks relevant does the model call dig_read to pull the full message — so one search never floods the freshly-compacted context window. (Codex's own history-tools review flagged exactly this as a P0 bug when a first version returned full texts — we designed around it from the start.)

Failure behavior

• A hook never blocks compaction and never fails your session: all errors go to .claude/dig/log, exit code stays 0.
• Unparseable transcript lines stay searchable as raw text — the .jsonl format is treated as unstable, and dig_read reads the byte-exact archive, not the index.
• The index is derived state, fully rebuilt on every compaction — corrupt or delete it, nothing is lost.

How it compares

	claude-dig	compact-plus	planning-with-files	claude-mem
Answers	"what did we just do/decide?"	"what was the working state?"	"what is the plan?"	"what has this project learned?"
Model can search pre-compaction history	✅ dig / dig_read / dig_state	❌ inject-only	❌ (plan files only)	❌ (memory, not history)
Zero LLM calls (no API key, works offline)	✅	❌ runs claude -p / codex exec in the hook	✅	❌
Byte-exact archive of the full transcript	✅	✅ (but unused by search)	—	—
Runtime	single 555 KB Rust binary	bash + LLM backends	Python + 6 hooks	Node + services
Failure mode	degrades to fewer results	hook can time out (80s × 2 backends)	—	—

Each tool occupies a different niche: planning-with-files keeps the future on disk, claude-mem keeps long-term memory, claude-dig keeps the past conversation — they compose.

Design notes

• Search is instant and local. tf-idf with a recency boost and distinct-term coverage, CJK-aware tokenization (Chinese/Japanese queries work without a segmentation dictionary), no vector store, no database. A project archive is tens of megabytes at most; an in-memory scan is faster than a DB round-trip.
• Storage layout (add .claude/dig/ to .gitignore — transcripts can contain secrets):

.claude/dig/
├── archive/<session-id>/
│   ├── pre-compact-1.jsonl   # byte-exact transcript copies
│   ├── state-1.md            # working-state snapshots
│   └── index.jsonl           # rebuildable search index
└── log

Development

cargo test                # 119 tests: unit + end-to-end + doctests
cargo clippy --all-targets --all-features   # zero warnings
cargo fmt --check

Not doing (yet)

• Vector / semantic search (tf-idf is plenty at this scale)
• Cross-project search (privacy boundary: one project, one archive)
• Other agents (Codex, Gemini) — the hook layer is Claude Code-specific
• Replacing or modifying Claude Code's own compaction — claude-dig only observes and supplements

License

MIT