feat: generic extension MVP (odek-extension/v1) (#128)

* feat: generic extension MVP (odek-extension/v1)

Add five generic runtime capabilities for external MCP-based products,
all additive and backward compatible (contract: docs/EXTENSIONS.md):

- MCP per-server limits: timeout_seconds (30s/3600s cap),
  max_response_bytes (10MiB/64MiB ceiling), max_result_chars
  (200k/1M), artifact_roots; approval keys hash the new fields.
- Artifact references: odek.tool-result/v1 envelope with
  odek.artifact-ref/v1 file:// refs, validated fail-closed
  (root containment after symlink resolution, sha256/size checks);
  model sees compact metadata only, never paths or content.
- Structured runtime events: odek.event/v1 via Config.EventHandler
  (non-blocking, panic-isolated) and `run --events-jsonl` (0600,
  no symlinks, flush per event, SHA-256 args hashes, redacted).
- External session refs: Session.ExternalRefs (validated, deduped,
  opaque — never dereferenced) + --external-ref on run/continue.
- Hard execution budgets: limits section (runtime/tool-calls/
  tokens/cost) enforced in the loop, typed budget.Error, CLI exit
  code 4, session persisted before return; project config may only
  lower limits, never raise/disable. Per-model pricing via
  limits.model_prices with fallback to the flat price pair;
  project-level prices rejected outright.

Includes mock extension MCP server (internal/mcpclient/testdata),
unit/race/fuzz coverage (6 new fuzz targets), and docs sweep
(EXTENSIONS/CLI/CONFIG/API/MCP/SESSIONS/SECURITY + AGENTS.md).

* style: satisfy staticcheck ST1005/QF1001 in external-ref validation

* feat(cli): scaffold limits section in odek init --global template

All values default to 0 (enforcement off, same as unset), so a fresh
config behaves identically while making execution budgets — including
model_prices — discoverable. The local template intentionally omits
limits: project configs may only lower global limits, so the section
is meaningless without a global budget to tighten.

* feat(serve): GET /api/limits for client-side session cost rendering

Returns the server's model, the resolved budget.Limits (including
model_prices), and effective_prices resolved for the server model via
Limits.ResolvePrices (0/0 when unconfigured = costs unavailable).
Sits behind the same middleware stack as /api/models: per-instance
serve token, loopback Host check, local origin. GET-only; 405 otherwise.

Author: jkyberneees

AGENTS.md

@@ -18,7 +18,7 @@ It provides context about the project's architecture, conventions, and how to up
 ```
 odek.go                       Public API (Config, New, Run, Close, ModelProfile, KnownProfiles, Tool interface)
 cmd/odek/
-  main.go                     CLI entry point, flag parsing, commands, sandbox setup, system prompt
+  main.go                     CLI entry point, flag parsing, commands, sandbox setup, system prompt, --events-jsonl/--external-ref/budget flag wiring
   dispatch.go                 CLI subcommand dispatch
   shell.go                    Built-in shell tool (local or docker exec; danger-gated; optional timeout_seconds)
   serve.go                    Web UI server (HTTP + WebSocket; @-resource completion)
@@ -30,6 +30,7 @@ cmd/odek/
   subagent_key.go             FD-based API key handoff (parent → sub-agent, never via env)
   browser_tool.go             Built-in browser tool (HTTP fetch + headless navigation)
   file_tool.go                Built-in file tools (read_file, write_file, search_files, patch, batch_read, glob, file_info)
+  external_ref.go             --external-ref flag parsing (run + continue) → session.ExternalRef
   perf_tools.go               Performance/parallelism tools (batch_patch, parallel_shell, http_batch, math_eval, diff, count_lines, multi_grep, json_query, tree, checksum, sort, head_tail, base64, tr, word_count)
   mcp.go                      MCP server implementation (stdio + SSE transport)
   mcp_approval.go             Per-tool MCP server approval UI and persistence
@@ -55,29 +56,32 @@ cmd/odek/
   *_test.go                   250+ unit + E2E tests covering all tools
 internal/
   llm/                        OpenAI-compatible HTTP client with reasoning_content support
-  loop/                       ReAct engine: observe → think → parallel-act → repeat. signal.go — SignalEvent observability (context_trimmed, tool_recovery, tool_running heartbeat).
+  loop/                       ReAct engine: observe → think → parallel-act → repeat. signal.go — SignalEvent observability (context_trimmed, tool_recovery, tool_running heartbeat). Execution-budget enforcement (budget.Checker) + odek.event/v1 emission.
   tool/                       Thread-safe tool registry, clarify.go, send_message.go
   danger/                     Command/URL classification + bypass-resistant tokenizer. TTYApprover with friction mode.
   auth/                       Interactive approval system
   memory/                     MemoryManager (facts, buffer, episodes, merge, scan). EpisodeProvenance — tainted episodes never auto-replayed.
-  session/                    Session store (CRUD, trim, cleanup, compact JSON). AuditStore + divergence heuristic.
+  session/                    Session store (CRUD, trim, cleanup, compact JSON). AuditStore + divergence heuristic. ExternalRef — opaque operator-supplied refs, never dereferenced.
+  artifact/                   odek.artifact-ref/v1 + odek.tool-result/v1: fail-closed ref validation (roots/symlinks/sha256/size) and model-safe rendering (metadata only, no paths/content).
+  events/                     odek.event/v1 runtime event stream: Event, non-blocking panic-isolated Emitter (args hashed, redact applied), JSONLSink (0600, no symlinks, flush per event).
+  budget/                     Hard execution budgets: Limits, typed Error, per-run Checker (runtime/tokens/cost/tool-calls).
   maintenance/                Storage-maintenance janitor (session/audit/plan retention, log rotation, media sweep, skill skip-list GC). Config: `maintenance` section (operator-only).
   skills/                     Skill system (types, loader, triggers, self-improve, curator, import, cache). SkillProvenance gate.
-  config/                     Config file loading, env vars, secrets.env, priority merge
+  config/                     Config file loading, env vars, secrets.env, priority merge, limits clamp (project may only lower budgets)
   telegram/                   Telegram bot: bot.go, poller.go, handler.go, commands.go, session.go, health.go, plan.go, media_path.go
   render/                     Terminal output and narrator support
   narrate/                    LLM-powered emoji-rich progress messages
   redact/                     Secret redaction (20+ patterns)
   mcp/                        MCP server handler (tools/list, tools/call, SSE streaming)
-  mcpclient/                  MCP client (connect to external MCP servers)
+  mcpclient/                  MCP client (connect to external MCP servers); per-server limits (timeout/response bytes/result chars/artifact roots) + odek-extension/v1 contract (contract.go), artifact-ref enforcement in CallTool
   sandbox/                    Docker sandbox lifecycle
   flock/                      Advisory file-locking helpers
   fsatomic/                   Atomic file-write helpers
   pathutil/                   Path helpers
   resource/                   @-resource resolver (files, sessions) with size/symlink hardening
   transport/                  Shared HTTP transport with connection pooling
   ws/                         RFC 6455 WebSocket framing
-docs/                         Documentation (CLI, API, CONFIG, MCP, MEMORY, TELEGRAM, SECURITY, etc.)
+docs/                         Documentation (CLI, API, CONFIG, MCP, EXTENSIONS, MEMORY, TELEGRAM, SECURITY, etc.)
 ```
 
 ## How It Works
@@ -200,6 +204,11 @@ Layered prompt-injection / approval-fatigue defenses. Full reference: [docs/SECU
 - **MCP inputSchema hardening** (`cmd/odek/mcp_approval.go`) — every string in an MCP tool's `inputSchema` is recursively guard-scanned for injection patterns; schemas larger than 256 KiB are rejected; the interactive approval prompt shows a SHA-256 hash and byte size of the schema so operators can detect changes.
 - **MCP tool batch classification** (`internal/loop/loop.go`) — MCP tools (`<server>__<tool>`) are classified as `unknown` by `classifyToolCall`, so the batch approval gate shows them and untrusted sub-agents force them to `deny`.
 - **MCP client robustness** (`internal/mcpclient/client.go`, `cmd/odek/mcp_approval.go`) — MCP calls use a default timeout when the caller supplies no deadline; server names and tool names are validated (tool names are rejected if they contain `__` to prevent collisions with odek's `<server>__<tool>` namespace); the interactive approval prompt sanitises tool descriptions to strip ANSI escape sequences and terminal control characters.
+- **MCP per-server limits + approval-key coverage** (`internal/mcpclient/client.go`, `cmd/odek/mcp_approval.go`) — extension servers are bounded per server: `timeout_seconds` (default 30s, cap 3600s), `max_response_bytes` (default 10 MiB, hard ceiling 64 MiB — oversized lines fail closed), `max_result_chars` (default 200000, cap 1000000 — structured truncation notice, never a silent cut), and `artifact_roots` (empty ⇒ all artifact refs rejected). Persisted MCP approval keys hash all four fields, so a project server editing them (e.g. widening `artifact_roots`) re-prompts instead of reusing an old approval; pre-extension approvals re-prompt once after upgrade.
+- **MCP artifact-ref validation** (`internal/artifact/`, `internal/mcpclient/client.go`) — `odek.artifact-ref/v1` refs from `odek.tool-result/v1` envelopes are validated fail-closed before anything reaches the model: exact schema match, `file://` only, absolute clean path, containment inside a configured root after `EvalSymlinks` on both sides, regular-file check, sha256/size verification when present. Artifact content is never auto-read into context; the model sees metadata lines only (id, media type, size, short hash, summary) — never absolute paths.
+- **Execution-budget clamp merge** (`internal/budget/`, `internal/config/loader.go::clampProjectLimits`, `internal/loop/loop.go`) — the `limits` section uses a clamp, not an overlay: global config may set any budget; project `./odek.json` may only *lower* one (raises clamped with a warning, zero-outs re-inherit the global value); project-set per-million prices (flat pair and `model_prices` alike) are rejected outright (a lower price would weaken cost enforcement); `model_prices` maps exact model IDs to per-model prices, resolved once per run with per-field fallback to the flat pair; CLI flags set limits explicitly. On exhaustion the loop emits `budget_exceeded`, persists the latest safe session state, and returns a typed `budget.Error` (CLI exit code 4). Currently enforced by `odek run` only; no `ODEK_*` env layer.
+- **Runtime event stream redaction** (`internal/events/`) — the `odek.event/v1` stream never carries raw tool arguments (SHA-256 `args_sha256` + sizes only), collapses errors into low-cardinality `error_class` strings, and runs `internal/redact` over the tool name and string `data` values before dispatch. The JSONL sink (`--events-jsonl`) is 0600, refuses symlink targets, requires an existing parent dir, and fsyncs per event; dispatch is non-blocking (drop-on-full) and panic-isolated.
+- **External session refs are opaque** (`internal/session/session.go`, `cmd/odek/external_ref.go`) — `Session.ExternalRefs` (kind/uri/created_by/read_only/created_at) are validated (charset/length), deduped, and persisted verbatim; odek has no code path that resolves or dereferences their URIs, so a ref can never become a fetch/exfiltration vector.
 - **Sub-agent trust defaults + delegate_tasks gate** (`cmd/odek/subagent.go`, `internal/loop/loop.go`) — a missing `trust_level` in `delegate_tasks` defaults to `untrusted`; `delegate_tasks` itself is classified as `system_write` so it requires explicit approval before spawning child processes.
 - **Memory add/replace pipe-to-shell filter** (`internal/memory/memory.go`) — `AddFact` and `ReplaceFact` now run `FactLooksUnsafe` in addition to the general guard scan, blocking agent-driven planting of download-and-execute facts.
 - **Skill learn-loop provenance propagation** (`internal/skills/learnloop.go`) — conversation-extracted suggestions and LLM-enhanced suggestions both retain the session's `SkillProvenance`, so tainted sessions cannot produce clean-looking auto-saved skills.

budget_test.go

@@ -0,0 +1,96 @@
+package odek
+
+import (
+	"context"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"sync"
+	"testing"
+
+	"github.com/BackendStack21/odek/internal/budget"
+	"github.com/BackendStack21/odek/internal/events"
+)
+
+// TestAgent_Budget_LimitsWiredAndEventOrder pins the public-API wiring:
+// Config.Limits reaches the loop engine, the run returns a typed
+// budget.Error, and the event stream carries budget_exceeded BEFORE
+// run_failed (schema odek.event/v1, docs/EXTENSIONS.md).
+func TestAgent_Budget_LimitsWiredAndEventOrder(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/chat/completions" {
+			// /models discovery etc. — no context window discovered.
+			w.WriteHeader(http.StatusNotFound)
+			return
+		}
+		// One tool call with usage that blows the input-token budget.
+		fmt.Fprint(w, `{
+			"choices":[{"message":{"content":"working","tool_calls":[{
+				"id":"call_1","type":"function",
+				"function":{"name":"noop","arguments":"{}"}
+			}]}}],
+			"usage":{"prompt_tokens":1000,"completion_tokens":10}
+		}`)
+	}))
+	defer server.Close()
+
+	var mu sync.Mutex
+	var evs []events.Event
+	agent, err := New(Config{
+		APIKey:        "sk-test",
+		BaseURL:       server.URL,
+		Model:         "test-model",
+		NoProjectFile: true,
+		MemoryDir:     t.TempDir(),
+		Limits:        budget.Limits{MaxInputTokens: 100},
+		EventHandler: func(ev events.Event) {
+			mu.Lock()
+			evs = append(evs, ev)
+			mu.Unlock()
+		},
+	})
+	if err != nil {
+		t.Fatalf("New: %v", err)
+	}
+	defer agent.Close()
+
+	_, runErr := agent.Run(context.Background(), "do work")
+	berr, ok := budget.As(runErr)
+	if !ok {
+		t.Fatalf("expected typed budget.Error from Run, got %v", runErr)
+	}
+	if berr.Limit != budget.LimitInputTokens {
+		t.Errorf("limit = %q, want %q", berr.Limit, budget.LimitInputTokens)
+	}
+
+	// Drain the async emitter before asserting on the stream.
+	agent.Close()
+
+	mu.Lock()
+	defer mu.Unlock()
+	budgetIdx, failedIdx := -1, -1
+	for i, ev := range evs {
+		switch ev.Type {
+		case events.TypeBudgetExceeded:
+			if budgetIdx < 0 {
+				budgetIdx = i
+				if ev.Data["limit_name"] != budget.LimitInputTokens {
+					t.Errorf("budget_exceeded limit_name = %v", ev.Data["limit_name"])
+				}
+			}
+		case events.TypeRunFailed:
+			if failedIdx < 0 {
+				failedIdx = i
+			}
+		}
+	}
+	if budgetIdx < 0 {
+		t.Fatal("budget_exceeded missing from event stream")
+	}
+	if failedIdx < 0 {
+		t.Fatal("run_failed missing from event stream")
+	}
+	if budgetIdx > failedIdx {
+		t.Errorf("budget_exceeded (idx %d) must precede run_failed (idx %d)", budgetIdx, failedIdx)
+	}
+}

cmd/odek/budget_test.go

@@ -0,0 +1,68 @@
+package main
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/BackendStack21/odek/internal/budget"
+)
+
+// TestParseRunFlags_BudgetFlags covers the WP6 execution-budget flags.
+func TestParseRunFlags_BudgetFlags(t *testing.T) {
+	f, err := parseRunFlags([]string{
+		"--max-runtime", "300",
+		"--max-tool-calls", "50",
+		"--max-input-tokens", "100000",
+		"--max-output-tokens", "20000",
+		"--max-cost-usd", "1.5",
+		"do the task",
+	})
+	if err != nil {
+		t.Fatalf("parseRunFlags: %v", err)
+	}
+	if f.MaxRuntime != 300 || f.MaxToolCalls != 50 || f.MaxInputTokens != 100000 ||
+		f.MaxOutputTokens != 20000 || f.MaxCostUSD != 1.5 {
+		t.Errorf("budget flags not parsed: %+v", f)
+	}
+	if f.Task != "do the task" {
+		t.Errorf("Task = %q, want %q", f.Task, "do the task")
+	}
+}
+
+// TestParseRunFlags_BudgetFlagsInvalid: non-positive/garbage values are a
+// clear startup error, not a silently ignored budget.
+func TestParseRunFlags_BudgetFlagsInvalid(t *testing.T) {
+	for _, args := range [][]string{
+		{"--max-runtime", "0"},
+		{"--max-runtime", "-5"},
+		{"--max-runtime", "abc"},
+		{"--max-tool-calls", "-1"},
+		{"--max-input-tokens", "0"},
+		{"--max-output-tokens", "x"},
+		{"--max-cost-usd", "-0.5"},
+		{"--max-runtime"}, // missing value
+	} {
+		if _, err := parseRunFlags(args); err == nil {
+			t.Errorf("parseRunFlags(%v) should fail", args)
+		}
+	}
+}
+
+// TestRunExit_BudgetExitCode pins the dispatch mapping: a typed budget.Error
+// exits 4 (odek-extension/v1), other errors keep exit 1, success exits 0.
+func TestRunExit_BudgetExitCode(t *testing.T) {
+	if code := runExit(nil); code != 0 {
+		t.Errorf("runExit(nil) = %d, want 0", code)
+	}
+	berr := &budget.Error{Limit: budget.LimitToolCalls, Observed: 6, Maximum: 5}
+	if code := runExit(berr); code != 4 {
+		t.Errorf("runExit(budget.Error) = %d, want 4", code)
+	}
+	// Wrapped budget errors still map to 4.
+	if code := runExit(fmt.Errorf("iteration 2: %w", berr)); code != 4 {
+		t.Errorf("runExit(wrapped budget.Error) = %d, want 4", code)
+	}
+	if code := runExit(fmt.Errorf("model exploded")); code != 1 {
+		t.Errorf("runExit(other) = %d, want 1", code)
+	}
+}

cmd/odek/dispatch.go

@@ -5,6 +5,8 @@ import (
 	"fmt"
 	"os"
 	"runtime"
+
+	"github.com/BackendStack21/odek/internal/budget"
 )
 
 // dispatch routes a top-level CLI invocation to its handler. It takes the
@@ -27,7 +29,7 @@ func dispatch(args []string) int {
 
 	switch cmd {
 	case "run":
-		return cliExit(run(rest))
+		return runExit(run(rest))
 	case "version":
 		printVersion()
 		return 0
@@ -76,6 +78,21 @@ func cliExit(err error) int {
 	return 1
 }
 
+// runExit is the `odek run` error→exit translator: like cliExit, but maps a
+// typed budget.Error to exit code 4 (execution budget exhausted —
+// odek-extension/v1, see docs/EXTENSIONS.md) so orchestrators can tell a
+// budget stop apart from a task/model/tool failure (1).
+func runExit(err error) int {
+	if err == nil {
+		return 0
+	}
+	fmt.Fprintf(os.Stderr, "odek: %v\n", err)
+	if _, ok := budget.As(err); ok {
+		return 4
+	}
+	return 1
+}
+
 // subagentExit honours the sub-agent JSON contract: stderr gets the
 // human-readable line, stdout gets a JSON envelope the parent can parse,
 // and the exit code is 3 (reserved for setup/contract errors so the

cmd/odek/events_jsonl_test.go

@@ -0,0 +1,80 @@
+package main
+
+import (
+	"encoding/json"
+	"os"
+	"strings"
+	"testing"
+
+	"github.com/BackendStack21/odek/internal/events"
+)
+
+func TestParseRunFlags_EventsJSONL(t *testing.T) {
+	f, err := parseRunFlags([]string{"--events-jsonl", "/tmp/odek-events.jsonl", "do the thing"})
+	if err != nil {
+		t.Fatalf("parseRunFlags error: %v", err)
+	}
+	if f.EventsJSONL != "/tmp/odek-events.jsonl" {
+		t.Errorf("EventsJSONL = %q, want /tmp/odek-events.jsonl", f.EventsJSONL)
+	}
+	if f.Task != "do the thing" {
+		t.Errorf("Task = %q, want %q", f.Task, "do the thing")
+	}
+
+	// Missing value is a hard error.
+	if _, err := parseRunFlags([]string{"--events-jsonl"}); err == nil {
+		t.Error("expected error for --events-jsonl without a value")
+	}
+
+	// Unset by default.
+	f, err = parseRunFlags([]string{"do the thing"})
+	if err != nil {
+		t.Fatalf("parseRunFlags error: %v", err)
+	}
+	if f.EventsJSONL != "" {
+		t.Errorf("EventsJSONL default = %q, want empty", f.EventsJSONL)
+	}
+}
+
+func TestEventsJSONL_SinkEndToEnd(t *testing.T) {
+	// The exact handler shape wired into odek.Config.EventHandler by the run
+	// command: emitter → handler → JSONL sink, drained on Close.
+	dir := t.TempDir()
+	path := dir + "/events.jsonl"
+
+	sink, err := events.OpenJSONLSink(path)
+	if err != nil {
+		t.Fatalf("OpenJSONLSink: %v", err)
+	}
+	em := events.NewEmitter(func(ev events.Event) {
+		if err := sink.Write(ev); err != nil {
+			t.Errorf("sink write: %v", err)
+		}
+	}, events.NewRunID())
+	em.Emit(events.Event{Type: events.TypeRunStarted, Data: map[string]any{"model": "test-model"}})
+	em.SetSessionID("sess-1")
+	em.Emit(events.Event{Type: events.TypeSessionSaved, Data: map[string]any{"message_count": 5}})
+	em.Emit(events.Event{Type: events.TypeRunCompleted, Data: map[string]any{"duration_ms": 3}})
+	em.Close()
+	if err := sink.Close(); err != nil {
+		t.Fatalf("sink close: %v", err)
+	}
+
+	data, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	lines := strings.Split(strings.TrimRight(string(data), "\n"), "\n")
+	if len(lines) != 3 {
+		t.Fatalf("got %d lines, want 3", len(lines))
+	}
+	for i, line := range lines {
+		var ev events.Event
+		if err := json.Unmarshal([]byte(line), &ev); err != nil {
+			t.Fatalf("line %d not parseable: %v", i+1, err)
+		}
+		if ev.Schema != events.Schema {
+			t.Errorf("line %d schema = %q", i+1, ev.Schema)
+		}
+	}
+}