Agent telemetry: request/response audit log + token usage estimates

[sshaaf]

Summary

rgctl today has performance telemetry for discover, but no request/response audit trail and no token usage estimates for agent queries. Agents are the primary consumers of -f json; without telemetry it is hard to answer: what did we call, what came back, and how many LLM tokens did that save (or cost)?

This issue tracks a unified telemetry layer that traces every rgctl invocation (CLI, HTTP, MCP) and emits token usage estimates (output size + optional savings vs naive alternatives).

Goals

1. Trace every rgctl invocation — CLI subprocess, HTTP API, MCP server.
2. Record request + response metadata — command, args, timing, success, sizes, result counts.
3. Token usage estimates — at least:
   • Output tokens (what the agent would inject from stdout/HTTP body)
   • Optional savings estimate vs naive alternatives (read whole files, dump full graph)
4. Agent-safe — telemetry on stderr or a side channel; never break -f json stdout parsing unless opt-in.
5. Configurable — off by default locally; on in CI/agent sessions.

---

Architecture: Single Funnel Point

All three interfaces (CLI, HTTP, MCP) converge at a single dispatch function:

crates/rgctl-service/src/execute.rs:10-106 — execute(session, command) -> Result<Value>

Every command (Query, Search, Impact, Metrics, Cpg, Check, Status) passes through this function and returns a serde_json::Value. This is the natural instrumentation point for token economics.

Current Response Envelope Pattern

All responses already follow { schema_version, command_fields... }:

Command	Response struct	File	Schema version
gql	GqlJsonResponse	gql_json.rs	v1
blast-radius	BlastRadiusResponse	blast_json.rs	v2
semantic query	SemanticQueryJsonResponse	semantic_json.rs	v3
semantic index	SemanticIndexJsonResponse	semantic_json.rs	v2
metrics	MetricsJsonResponse	metrics_json.rs	versioned
check	CheckJsonResponse	check_json.rs	versioned
inspect	InspectCfgResponse / InspectPdgResponse	inspect_json.rs	versioned
slice	SliceCfgResponse / SlicePdgResponse	slice_json.rs	versioned

No response currently includes timing, byte counts, or token estimates.

---

What "tokens" means in this codebase

There are three distinct token concepts:

Concept	Where it exists	Purpose
LLM tokens	Not tracked	Estimated cost of feeding query output into an LLM context window (~4 chars/token)
Graph tokens	TokenBloom in structural_sketch.rs	256-bit bloom filter of camelCase/snake_case identifiers per node; used for semantic fusion scoring
Semantic index tokens	semantic_vocab.rs, semantic_search.rs	Vocabulary-based embeddings (FNV bag-of-tokens, 256d) for similarity search

The user-facing need is about LLM tokens — how much of an LLM context budget a query response consumes.

---

Evaluated Approaches

Option A: Response envelope with _meta field (Recommended — Phase 0)

Add an optional _meta object to every JSON response, computed in execute.rs after the command handler returns.

{
  "schema_version": 1,
  "rows": [...],
  "count": 42,
  "_meta": {
    "duration_ms": 12,
    "response_bytes": 8432,
    "response_tokens_approx": 2108,
    "result_count": 42
  }
}

Implementation (~50 lines in execute.rs):

pub fn execute(session: &mut Session, command: Command) -> Result<Value> {
    let start = Instant::now();
    let mut result = execute_inner(session, command)?;
    let json_str = serde_json::to_string(&result)?;
    let bytes = json_str.len() as u64;
    result["_meta"] = serde_json::json!({
        "duration_ms": start.elapsed().as_millis() as u64,
        "response_bytes": bytes,
        "response_tokens_approx": bytes / 4,
    });
    Ok(result)
}

Pros	Cons
Single instrumentation point (execute.rs:10)	Slightly inflates response (~100-200 bytes)
Backward compatible (new optional field, no schema bump)	Token estimate is approximate (bytes/4 heuristic)
Works across CLI, HTTP, and MCP identically	Does not capture input size
Cheap — just serde_json::to_string().len()	nodes_scanned requires plumbing from query engine

Option B: Separate --dry-run / cost endpoint

A rgctl query-cost command or --dry-run flag that estimates cost before execution.

rgctl -f json gql "MATCH (n:Function) RETURN n LIMIT 100" --dry-run
# → { "estimated_nodes": 100, "estimated_bytes": 12000, "estimated_tokens": 3000 }

Pros	Cons
No change to existing response schemas	Requires running query engine twice (or separate estimator)
Agent can check cost before committing	Estimates may diverge from actual results
Enables budget-aware querying	Adds complexity to CLI surface

Option C: Session-level cumulative tallies (MCP/HTTP)

Track cumulative token usage across a session. Each HTTP or MCP session maintains a running tally.

{
  "session_tokens": {
    "queries": 12,
    "total_input_tokens": 450,
    "total_output_tokens": 28000,
    "total_duration_ms": 340
  }
}

Pros	Cons
Gives agents a budget meter	Only works for HTTP/MCP (not CLI one-shot)
Natural fit for MCP sessions	Requires session state management
Can enforce configurable token ceiling	"Input tokens" for graph queries are ambiguous

Option D: Per-node cost annotations in the graph

Pre-compute estimated token cost for each node at index time. Store as a node property.

{ "estimated_tokens": 45 }

Pros	Cons
Zero runtime overhead per query	Increases graph storage size
Agents can filter by cost (WHERE n.estimated_tokens < 50)	Cost is model-dependent
Enables "cheapest function to include" queries	Requires re-indexing to change cost model

Side-by-side comparison

Dimension	A: _meta	B: --dry-run	C: Session tallies	D: Per-node cost
Effort	~50 lines	~200 lines	~150 lines	~100 lines
Runtime cost	Near-zero	Double query	Accumulator only	Index-time only
Accuracy	±20%	±20%	Exact (cumulative)	±20% per node
Scope	Per-response	Pre-query	Per-session	Per-node
New deps	None	None	None	None
Works for CLI	Yes	Yes	No	Yes
Works for MCP	Yes	Yes (2 calls)	Yes	Yes
Backward compat	Yes	Yes	Yes	Schema bump

---

Recommendation

Start with Option A (_meta envelope), then layer Option C for MCP sessions.

Rationale

1. Single instrumentation point. execute() at execute.rs:10 is the funnel for all commands across all interfaces. Wrapping it is ~50 lines.
2. Backward compatible. _meta is additive. Existing consumers unaffected. No schema version bumps.
3. Covers the 80% case. Most agent workflows need "how big was this response?" — byte count and token estimate answer that.
4. Token estimate is good enough. For code-heavy JSON, bytes / 4 is within 20% of actual tokenizer output across Claude, GPT-4, and Llama models.
5. MCP session tallies are a natural follow-on. Once _meta exists, the MCP server can accumulate totals trivially.

---

Token Estimation Strategy (Tiered)

Tier 0 — Always available (ship first)

Metric	How
response_bytes	serde_json::to_string(&result).len()
response_tokens_approx	response_bytes / 4
duration_ms	Instant::now() around execute()

Include _meta.method: "bytes_div_4" so consumers know it is heuristic.

Tier 1 — Model-family heuristics (no new deps)

gpt/claude:     bytes / 4
gemini:         bytes / 4.2
code-heavy JSON: bytes / 3.5

--usage-model gpt-4o selects divisor from config table.

Tier 2 — Optional tiktoken / tokenizers (accurate count)

New optional feature flag telemetry-tiktoken. Count tokens on serialized JSON response with cl100k_base.

Tier 3 — Context savings estimate (agent ROI)

Compare response tokens to a declared baseline:

Baseline	When to use	How to estimate
naive_file_read	GQL returned symbols in known files	Sum file sizes for touched files
full_graph_export	Macro all_functions vs LIMIT query	node_count × avg_node_json_bytes from manifest
raw_source_slice	slice / cpg flows	Lines in slice × avg line length

"_meta": {
  "savings_est": {
    "baseline": "naive_file_read",
    "baseline_tokens": 48000,
    "output_tokens": 2105,
    "saved_tokens": 45895,
    "compression_ratio": 0.956
  }
}

Tier 4 — Input token estimate

Estimate request tokens (query string, symbol names) for billing dashboards.

---

_meta fields by command

Command	Useful _meta fields
gql	result_count (rows), bindings_per_row
blast-radius	impact_zone_size, direct_callers_count, score
semantic query	hits_count, model_id, dimensions
metrics	sections_returned
check	violations_count, symbols_checked
cpg	nodes, edges, lines
inspect	nodes, edges

---

Sinks (optional, combinable)

Sink	Mechanism	Best for
A. _meta in response	Injected in execute.rs (default)	Single-shot agent introspection
B. JSONL audit log	.rgctl/telemetry.jsonl (append, rotate)	Session replay, analytics
C. stderr tracing	tracing + JSON subscriber	Live debugging
D. HTTP headers	X-Rgctl-Bytes-Out, X-Rgctl-Tokens-Est	HTTP clients without parsing body
E. MCP session tally	Accumulated in MCP server state	Agent budget tracking
F. OpenTelemetry	Optional opentelemetry exporter	Production / Grafana

Default -f json keeps one JSON doc on stdout. _meta is part of that doc (backward compatible). JSONL/stderr/OTel are opt-in side channels.

---

Configuration (rgctl.toml)

[telemetry]
enabled = true
sink = "meta"              # meta | jsonl | stderr | otel | all
jsonl_path = ".rgctl/telemetry.jsonl"
token_estimator = "heuristic"  # heuristic | tiktoken
usage_model = "cl100k_base"
savings_baseline = "off"       # off | naive_file_read | full_graph
redact_queries = false
redact_paths = false

Env overrides: RGCTL_TELEMETRY=1, RGCTL_TELEMETRY_SINK=jsonl.

---

Implementation Phases

Phase	Deliverable	Effort	Details
P0	_meta envelope in execute.rs with duration_ms, response_bytes, response_tokens_approx	Small (~50 lines)	Wrap execute(), serialize result, measure, inject _meta
P1	Per-command result_count / domain-specific counts in _meta	Small	Extract count / hits.len() / impact_zone_size per command
P2	JSONL audit sink + TelemetryConfig in rgctl.toml	Medium	Append one JSONL line per query when enabled
P3	MCP session tallies	Small	Accumulate _meta values across tool calls in MCP server
P4	HTTP middleware + headers	Small	tower layer on /api/* routes
P5	Optional tiktoken + savings estimates	Medium	Feature flag, baseline comparison
P6	rgctl telemetry report (session summary) + docs	Small	Aggregate JSONL into summary
P7	OpenTelemetry exporter	Medium	Optional feature flag

P0–P1 deliver the core "every response shows its cost" story.

---

Testing

• Unit: Token estimator on existing fixtures (fixture_gql_response(), fixture_response() in blast_json.rs)
• CLI: Subprocess asserts _meta present in JSON output, verify response_bytes > 0
• HTTP: POST to /api/query → response contains _meta, optional headers
• MCP: Tool call response includes _meta
• Golden: Schema stability — _meta fields are optional, never required by consumers

---

Existing Infrastructure to Reuse

Component	Location	How it helps
execute() funnel	crates/rgctl-service/src/execute.rs:10	Single instrumentation point for all commands
Response structs with schema_version	*_json.rs files in rgctl-service	Established envelope pattern
RgctlConfig + rgctl.toml	crates/rgctl-project-config/src/project.rs	Config framework ready for [telemetry] section
WatchConfig / HooksConfig	Same file, lines 38-96	Precedent for tool config sections
TokenBloom	crates/rgctl-graph/src/structural_sketch.rs	Reusable for Tier 3 savings (upper bound on body sizes)
Dashboard manifest.json	.rgctl/dashboard/manifest.json	Has total node/edge counts for baseline estimates
emit_json_value()	src/cli/context.rs	Single stdout emission point for CLI
tracing infrastructure	Already in tree	Ready for stderr sink

---

Acceptance Criteria (MVP = P0–P1)

• Every command response includes _meta.duration_ms, _meta.response_bytes, _meta.response_tokens_approx
• _meta is injected in execute.rs (single point, all interfaces covered)
• Default -f json stdout includes _meta (backward compatible — new optional field)
• Per-command result_count populated for gql, blast-radius, semantic query
• Token estimate method documented (bytes / 4 heuristic)
• Existing test fixtures pass unchanged
• New unit tests for _meta injection

---

Example Output

rgctl -r "$REPO" -f json gql "MATCH (n:Function) RETURN n LIMIT 5"

{
  "schema_version": 1,
  "rows": [...],
  "count": 5,
  "explain": false,
  "_meta": {
    "duration_ms": 8,
    "response_bytes": 2840,
    "response_tokens_approx": 710,
    "result_count": 5,
    "method": "bytes_div_4"
  }
}

---

References

• docs/json-api.md — stdout/stderr rules, schema_version
• AGENTS.md — agent workflow, low token cost commands
• Related: Implement incremental graph updates via git hooks and file watcher #69 (git hooks + file watcher) — telemetry useful for hook/session analytics
• .scratch/todo/FILE_WATCHER_VS_GIT_HOOKS.md — detailed analysis