[Epic] Control plane absorbs Watcher's operator surface #359

Child of CannObserv/watcher#368: Headless Watcher and cohort boundaries.

Design of record (in watcher): 2026-10-14-headless-watcher-and-cohort-boundaries-design.md

• Last check-in: 2026-11-01T21:41:20Z
• Last probe: 2026-11-01T21:41Z
• Last issue seen: Adopt the unified shipping-work skill: symlinks and configuration, not overrides (gregoryfoster/skills#537) #457
• Last verified SHA: ea53550
• Next: info.watch-events consumer: shadow → dispatch #369 kickoff (briefed 2026-11-01; lands dark, T unset). Open: Notification composer, preview and test-send #367's Notifier read-back ([Epic] Archiver becomes the notifying tenant notifier#116). Then F3 (F3: notification cutover instant T; stop dispatching watcher#376): parity join, same T both sides. F2 is Watcher's (go given; F2: drop the domain cadence tier, temporal profiles, post-actions and throttle floor watcher#375)

Issues

Phase 1 — build additive and dark, in the order under Order below:

• Domains: become the sole owner (min_interval, default cadence, deactivation) #360 Domains: become the sole owner (min_interval, default cadence, deactivation) · M · PR #360 Domain ownership: suspension resolves active=false; fetch-policy columns; Watcher import #434 (492ac55); D1–D4 taken as recommended. CI green by job on a935d68 and 492ac55. Deployed 2026-10-25T18:52Z; live now 284bdc06dfc3 (with fix: API key reveal box shows the new key (empty since #37) #435). OpenAPI 4.33.0, SDK 5.12.0, alembic 173c1af91def (min_interval_seconds float NOT NULL default 1.0 with a finite/non-NaN CHECK; default_watch_spec JSONB). Operator import 2026-10-26: 4|t|t|t [measured]. 0 info.registry outbox rows since the deploy; snapshot 5 live; Watcher applied_active true ×5; sweep 16 checked, 0 invalid. Coordinator errors in the brief: (1) is_active gates nothing (the description was stale; the index only displays it); (2) the dashboard has no is_active control; (3) missed the watch panel reading suspension as a permanent "resume pending"; (4) missed the domain-toggle-vs-announce race and its lock order, which most of CR 1–31 hardened; (5) missed that info_sources.domain_name is nullable with no FK, so PATCH's insert branch can flip suspension; (6) docs/SCHEMA.md had no Domain entry to update; (7) the import writes 1 row, not 3 + 1; (8) missed the two export paths. Standing: Watcher re-exports and the operator re-imports before F1 and F2 (host step 9).
• Produce content.fetch-policy (dark until F1) #366 Produce content.fetch-policy, dark until F1 · M · PR #366 Produce content.fetch-policy, dark until F1 #436 (b0f5782). CI green by job on 3d45d7a and b0f5782. Live b0f57823c0f1 = /health = main, deployed 2026-10-26T16:57Z; OpenAPI 4.34.0; alembic 44b797877fdb. domains.fetch_policy_generation is 4 distinct non-null values (1–4); sequence CACHE 1; 0 tombstones; 0 content.fetch-policy outbox rows; journal "producer is dark"; the unit line is commented out; sweep 16 checked, 0 invalid [measured]. D1–D3 taken; D4 dropped by the implementer, rightly: Watcher's deleted hosts already end on a revoked frame, and the F1 runbook replaces it with a dry-run read-back. Coordinator errors in the brief: (1) the delta maxlen rule would have trimmed below the floor (deltas carry the last full set's cap); (2) D4's premise; (3) missed that DEFAULT nextval() is drawn before an insert waits on a racing delete (every draw is now UPDATE … RETURNING once the row is held); (4) "loudly once per tick" with no bus: there is no tick; (5) "first full set before any delta" is unneeded with generations. For the F1 flag PR: the schema and docstring "Dark until" text (contract path, so CHANGELOG plus an SDK regen) and the unit-test flip.
• F1: switch on the content.fetch-policy producer (after watcher#374 is live) #438 F1: switch on the content.fetch-policy producer · S · PR #438 F1: archiver.service produces content.fetch-policy (merge after watcher#374 + re-import) #439 (eb2215a), merged 21:36:16Z, after F1: stop the content.fetch-policy producer watcher#374 part 1 went live (18:57:22Z) and the re-export (21:33:18Z): D8's order held. CI green by job on 689845e and eb2215a. Live eb2215a8f86c = /health = main since 21:42:24Z; OpenAPI 4.35.0; ARCHIVER_FETCH_POLICY_PRODUCER=1 in the unit; producer started 21:42:27Z; full sets of 4 every 300 s, maxlen 500; no warnings; drift ok; bus-health 0 findings [measured]. Gap ~2 h 47 m on Replicator's default. Coordinator errors in the body: (1) "version bump" is the service only (4.35.0; SDK stays 5.12.0); (2) docs/DOMAINS.md didn't fully follow the flag; (3) the dashboard copy follows the process's flag, so dev/test render false text after F1; (4) missed further "dark until F1" sites in src/. Read-backs owed: see Verification debt.
• Watch lifecycle: archive and restore #361 Watch lifecycle: archive and restore · S · PR #361 feat: watch lifecycle — archive and restore an InfoItem #442 (764a0f9), merged 2026-10-27T21:12:50Z; D1–D5 taken. Q1 answered by the operator: D5's refusal lives in import_watcher_item_cadence, whose Resolve cadence and announce a concrete interval #364 line "an archived item imports like any other" is reversed. CI green by job on cdd9de1 and 764a0f9. Live 764a0f935fb6 = /health = main since 21:24:48Z (deployed by the implementer with the operator's authorization [stated]); OpenAPI 4.37.0, SDK 5.14.0 (archive_info_item/restore_info_item); alembic 44b797877fdb → da4d2a2e19c7; archived_at timestamptz NULL; no warnings since deploy; drift ok; bus-health 0 findings [measured]. Inert as predicted: info.registry still count 3, max 01M4ZW9JY8W2EXP2QVYN79WEC3; archived_at NULL 5/5; applied_active true ×5 [measured]. 8001 rehearsal (implementer, issuecomment-6452745452) [stated]: paused 2:false 3:false 4:false, active 2:true 3:false 4:true, repeats silent. Notes left for Temporal profiles: model, API/UI, daily boundary job, post-actions #365 (service entry points, safe with the item locked, idempotent) and Check-now via info.watch-check #370 (pre-flight archived_at). Watcher told to archive only in Archiver until the jump, and to hold 0 archived before the re-export ([Epic] Headless Watcher and cohort boundaries watcher#368 issuecomment-6454837910). Coordinator errors in the brief: (1) "the importer" for D5 without naming it, and missed that import_watcher_item_cadence's docstring said the opposite (Q1); (2) named Domain archive as the template without noting it overwrites archived_at on every call, where D2 keeps the first; (3) missed that D3 and restore on a suspended host re-announce with active unchanged (CR 3, stet); (4) missed the domain page's item table as a badge site (CR 2); (5) didn't flag that SCHEMA, PAGES, ARCHITECTURE and BUS had 2–39 tokens of budget headroom (CR 1); (6) didn't flag bare "Archive" beside archive.org replication (CR 6, now "Archive watch"). Still pending: step 5, the item importer's dry run at the pre-F2 re-export (0 archived).
• Resolve cadence and announce a concrete interval #364 Resolve cadence and announce a concrete interval · S–M · PR #364 feat: announce the resolved cadence (profile → item → domain) #441 (d49e292), merged 2026-10-27T00:40:56Z; D1–D5 taken, plus the implementer's D5' (default_watch_spec fill-if-absent in the domain importer). CI green by job on 5b9a4b4 and d49e292. Live d49e292ac4b6 = /health = main since 02:39:55Z; OpenAPI 4.36.0, SDK 5.13.0 (set_watch_spec); alembic 44b797877fdb (no migration); no warnings since deploy; 13 registry snapshots, no resolver WARNING; drift ok; bus-health 0 findings [measured]. Inert as predicted: 0 info.registry outbox rows past the baseline 01M4ZW9JY8W2EXP2QVYN79WEC3; all 4 domain defaults NULL; applied_interval 1d ×3, 1h, 6h, unchanged, each matching its watch_spec [measured]. Host steps (operator, issuecomment-6443906407) [stated]: domain importer dry run 4 unchanged, no differs; item import found nothing to fill (Watcher's own-cadence item is the 1h one, which Archiver already holds); 8001 fan-out checked end to end. Coordinator errors in the brief: (1) the builder took a bool (host_suspended), not the Domain row; (2) nothing validated the announced watch_spec (validation is write-time only); (3) gave the lock order but not the mode: a cadence write needs FOR UPDATE, because an announce's FOR KEY SHARE doesn't wait on FOR NO KEY UPDATE; (4) missed that the SDK had no PUT …/watch-spec wrapper, which D3's importer needs; (5) missed the pre-F2 ordering hazard (an Archiver domain default outranks Watcher's own item cadence until the item import runs); (6) missed the "dark until F2" text (DOMAINS.md, the domain page, DomainOut/DomainPatch); (7) D5 missed that the domain importer also overwrites and clears default_watch_spec; (8) the editor example "Watcher's default 1d" would mirror Watcher's constant (Refactor brittle Watcher↔Archiver "mirror discipline" for shared content-acquisition code watcher#159). Notes for Temporal profiles: model, API/UI, daily boundary job, post-actions #365 relayed (issuecomment-6445429892). · Coordinator miss in verification (found at Temporal profiles: model, API/UI, daily boundary job, post-actions #365's pre-flight): docs/API.md:23 still says domain cadence is "stored but dark until … Resolve cadence and announce a concrete interval #364 (F2)"; routed to Temporal profiles: model, API/UI, daily boundary job, post-actions #365's living docs.
• Temporal profiles: model, API/UI, daily boundary job, post-actions #365 Temporal profiles: model, API/UI, daily boundary job, post-actions · M · PR #365 feat: temporal profiles - model, API/UI, daily boundary job, post-actions #443 (f6bb673), merged 2026-10-28T16:01:03Z; D0–D5 taken, plus the implementer's latitude calls (issuecomment-6455054077): PK = info_item_id; PUT create-or-replace clears fired_at and 422s an already-ended window; tighter rule validation (one seasonal rule, no duplicate days_before, no zero or >10y interval); is_active=false contributes and fires nothing. CI green by job on 1c74575 and f6bb673. Live f6bb673c5cfb = /health = main since 16:19:17Z (deployed by the implementer with the operator's authorization [stated]); OpenAPI 4.38.0, SDK 5.15.0 (get_/set_/delete_temporal_profile); alembic da4d2a2e19c7 → b7e3c95a1f20; temporal_profiles 0 rows; timer enabled, next 2026-10-29 00:05 UTC, Persistent=true, unit User=archiver with the unit-scoped ARCHIVER_ALLOW_PRODUCTION_DB; hand run Result=success, evaluated 0, announced 0, skipped 0; no warnings since deploy; drift ok; bus-health 0 findings [measured]. Inert as predicted: info.registry still count 3, max 01M4ZW9JY8W2EXP2QVYN79WEC3; applied_interval 1d ×3, 1h, 6h and applied_active true ×5, unchanged [measured]. D5 dry run on the 10-14 export (issuecomment-6475356848) [stated]: no refusal; 0 profiles, 0 floors, 0 archived; 0 to fill, 1 unchanged. docs/API.md:23's stale line fixed. Curation split DEPLOYMENT, SCHEMA, PAGES and BUS into MONITORING, SCHEMA_SOURCES, SOURCE_PAGES and BUS_CLIENT. Coordinator errors in the brief: (1) said inherit_label needed a "profile" case; its else-branch already handled a profile win, and only naming the tier was missing; (2) said _announce_many does one joined read; it runs five statements, and the profile is read in the hydrate step; (3) left re-arm as an open question without naming the hazard that re-saving a fired profile with past dates fires it again (the implementer's 422); (4) missed that resolved_interval (String(32)) and Watcher's timedelta overflow on a very long interval (CR 8); (5) again didn't flag the docs' context budget: five docs over 10k needed a curation pass (repeat of Watch lifecycle: archive and restore #361's error 5); (6) post-merge step 2 assumed the deploy enables the new timer; it installs only, and needs enable --now; (7) didn't weigh audit_stored_documents for rules (exempted, with a reason).
• Notification subscriptions: model, API and UI (global/domain/item scopes) #363 Notification subscriptions: model, API and UI (global/domain/item scopes) · M · PR #363 Notification subscriptions: model, API, UI and the Watcher import #446 (fa9ee65), merged 2026-10-28T22:55:53Z; D1–D6 taken, plus the implementer's remote_channel_id NOT NULL (a null channel refuses on import). CI green by job on 8b7c41d and fa9ee65 (a Dependabot uv run on fa9ee65 failed: Dependabot security updates for uv fail on every advisory (co-core 404) and mark main red #437, unrelated). Live fa9ee65960ee = /health = main since 2026-10-28T23:04:31Z (deployed by the implementer with the operator's authorization [stated]); OpenAPI 4.39.0, SDK 5.16.0; alembic b7e3c95a1f20 → c4d81f6a2e07; no warnings since deploy; drift ok; bus-health 0 findings [measured]. Import done (issuecomment-6483086496): 2 global subscriptions, both active on all three events, imported_watcher_id set, channels mapped to 01M5AX8CE8PBMQZMQ5TBBB9RAP (Slack) and 01M5AX8CEAEKPM7JJ3D0VY20AF (Mailgun), no Jinja, no overrides [measured]; a re-run landed on the same ids [stated]. No subscription write reached the outbox; the only rows since deploy are two source_revision_captured pairs (2026-10-29 16:23Z, 21:28Z) [measured]. 8001 checks passed (issuecomment-6483149901) [stated]. Coordinator errors in the brief: (1) D1 listed remote_channel_id without saying Watcher's is nullable or ruling on it; (2) gave no lock rule for the refs: a create racing a Domain or item delete hit the FK and 500'd (CR 1, now FOR KEY SHARE), the Resolve cadence and announce a concrete interval #364 class again; (3) D3's whole-document PATCH plus D5's hidden Jinja fields meant a dashboard toggle edit would erase an API-set template; the implementer caught it in review; (4) again didn't weigh audit_stored_documents (repeat of Temporal profiles: model, API/UI, daily boundary job, post-actions #365's error 7): the new sweep kind crashed /dashboard/registry-audit, which domain_watch_spec had done unseen since Domains: become the sole owner (min_interval, default cadence, deactivation) #360 (CR 7); (5) traced both matchers but didn't flag that Watcher matches on its item's own domain_name while Archiver uses the announcing source's host, the first place to look on a info.watch-events consumer: shadow → dispatch #369 parity miss; (6) missed that the SDK body model silently drops unknown keys (update(id, {"scope": …}) answered 200 unchanged); (7) missed SCREENS.md Dashboard: Domain detail — align with detail-screen conventions #82's group-count rule (CR 4).
• Notifier credential behind a unit-only enable flag #368 Notifier credential behind a unit-only enable flag · S · PR #368 feat: Notifier credential behind a unit-only enable flag #450 (5cfb5da), merged 2026-10-30T04:49:50Z; D1–D5 taken, plus two operator rulings from the implementer review (issuecomment-6511197118): the page is /dashboard/settings/notifier, and the gate refuses an environment key aimed at :9000. CI green by job on 4018cfa and 5cfb5da (test re-run green after Notification subscriptions: model, API and UI (global/domain/item scopes) #363's tombstone flake, fixed by test: pick the cascade tombstone by generation, not row order #455; Dependabot uv failure is Dependabot security updates for uv fail on every advisory (co-core 404) and mark main red #437). Live 5cfb5dacbe27 = /health since 2026-10-30T14:58:57Z, NRestarts 0; main is 2 commits ahead, test and submodule only (drift ok, "none that runs"); OpenAPI 4.39.0, alembic c4d81f6a2e07 unchanged; no warnings since deploy; bus-health 0 findings [measured]. Journal: Notifier configured at http://notifier:9000 (key: credential), then exactly one Notifier call, GET /api/v1/channels 200 at 15:49:41Z from the operator's page load [measured]; key 0 times in the journal [stated]. Outbox unchanged: persist 28, replicate 3, changes 25, registry 3 [measured]. 8001 listed dev-sink/dev-sink-failing with the real dev key [stated]. Coordinator errors in the brief: (1) D4 named /dashboard/settings, a 404 (Settings is only /dashboard/settings/api-keys); (2) D3 had the parity test mirror a :9000 refusal D2's gate didn't contain; (3) gave v0.3.2 as f3efc68, the annotated tag object, not the commit 8e7c020; (4) traced the generated API but missed the SDK's channels facade, which also exposes send_test and channel writes, so the brief named no single-importer adapter or guard on them; (5) D4 promised a channel type ChannelOut doesn't carry (shown from the masked URL's scheme); (6) flagged Dependabot as a risk, but it tracks github-actions only; (7) named the ordering hazard but not the fix: deploy.sh now refuses a missing no-fallback credential before CI (CR 4), the deploy-time host-prerequisite class again.
• Notification composer, preview and test-send #367 Notification composer, preview and test-send · M · unblocked (Notification subscriptions: model, API and UI (global/domain/item scopes) #363 live 2026-10-28). CannObserv/cannobserv#568 is cleared. From Notification subscriptions: model, API and UI (global/domain/item scopes) #363: render in SandboxedEnvironment; put the Jinja fields beside the preview and drop the form's carry-over; test-send to a production channel is an operator action; include_tags inert until InfoItem tags #362. From Notifier credential behind a unit-only enable flag #368: src/core/notifier/ is the only notifier_client importer and exposes list_channels alone, with a guard test keeping send_test, test_channel, dispatch and channel writes out of src/; test-send widens both on purpose. Channel names come from /dashboard/settings/notifier's call (one attempt, 5 s); ChannelOut has no type field. · PR #367 feat: notification composer, preview and test-send (epic #359) #456 (ea53550), merged 2026-10-30T21:56Z; D1–D7 taken, Q1 answered (a): ARCHIVER_PUBLIC_BASE_URL is the origin, and the API's dashboard_url was fixed to /dashboard/info-items/{id} (v4.40.0, [service] entry; SDK unchanged 5.16.0). CI green by job on 4b1e03f and ea53550 (6 Dependabot uv failures on the merge SHA are Dependabot security updates for uv fail on every advisory (co-core 404) and mark main red #437). Live ea5355088791 = /health = main since 2026-10-31T00:12:28Z, NRestarts 0; OpenAPI 4.40.0; alembic c4d81f6a2e07 unchanged; no WARNING since deploy; drift ok; bus-health 0 findings [measured]. Journal: text reader configured (bucket co-gcs-processor), links at https://co-registrar.exe.xyz, Notifier configured; no Notifier call since deploy [measured]. Outbox: 0 rows created since deploy, max ids unchanged; counts fell only by the hourly changes_outbox grows without bound (no pruner of published rows) #189 prune of rows older than ~25 days [measured]. Host step 1 done by the implementer with the operator's authorization [stated]: the base-URL line removed from /etc/archiver/.env (the unit now carries it); SA archiver-processor-text-reader key installed. 8001: previews and a test-send to dev-sink (dispatch 01M672JR9QV071539XKXKGPC4D) [stated]. Coordinator errors in the brief: (1) D2's link didn't reconcile with the existing dashboard_url ({base}/info-items/{id}), so the meaning of ARCHIVER_PUBLIC_BASE_URL had to go to the operator (Q1); production's dashboard_url had been a 404 all along, which the brief missed; (2) predicted no contract path; the fix needed a CHANGELOG entry; (3) D1 gave no Archiver source for last_changed_at or the interval, and missed the previously_notified_at input Watcher's error body needs for reminders; (4) recommended the sandbox without bounding its resource use (*/**) or rendering off the loop (CR 1); a per-render budget is still open for info.watch-events consumer: shadow → dispatch #369; (5) step 4's proof leaned on Watcher's process_commands, though the processor's keys are content-addressed and Archiver holds candidate fingerprints.
• info.watch-events consumer: shadow → dispatch #369 info.watch-events consumer: shadow, then dispatch · M · unblocked (Notification composer, preview and test-send #367 live 2026-10-31, Notifier credential behind a unit-only enable flag #368 live 2026-10-30). Unblocks F3: notification cutover instant T; stop dispatching watcher#376 (F3). Carries TRIAGE_DLQS pin: three queues once archiver.watch-events consumes (broker#93 granted +xdel on info.watch-events.dlq) #399. Create the group at 0. From Notifier credential behind a unit-only enable flag #368: only archiver.service holds the credential, and a units test pins that no other unit carries notifier config; a consumer in its own unit needs its own LoadCredential and that test changed. The lifespan builds the client last, one per process. From Notification composer, preview and test-send #367: dispatch only via Notifier.dispatch_composed (D3); load the diff with load_change_diff(app.state.text_reader, event) only when diff_requested(options); feed previously_notified_at for reminders. Open: a render budget against nested-loop CPU in saved templates. Default text is Watcher's word for word, [Watcher] title prefix included, for shadow parity; decide the prefix at F3. · Briefed 2026-11-01 (issuecomment-6564279582), D1–D8. Recommended: D1 one table notification_dispatches (shadow log + dispatch outbox), unique (event_id, subscription_id), composed at consume; D2 group at 0 under the revisions gates, check_rejected ignored, TRIAGE_DLQS pin: three queues once archiver.watch-events consumes (broker#93 granted +xdel on info.watch-events.dlq) #399's three DLQs; D3 ARCHIVER_NOTIFY_CUTOVER_AT unit-only, send only when set and occurred_at ≥ T; D4 drain retries transport errors on the same key, any DispatchOut terminal (Notifier's key returns failed records too), no redeliver; D5 reminders by timer from the recorded episode, active/unarchived/unsuspended items only; D6 parity by evidence join (dashboard log + Watcher's audit export; told F3: notification cutover instant T; stop dispatching watcher#376 issuecomment-6564281863); D7 render caps (range 1 000, 64 KiB); D8 docs (ARCHITECTURE 48 tokens).
• Check-now via info.watch-check #370 Check-now via info.watch-check · S · unblocked. Unblocks F4 (no watcher issue).
• Watch panel: last_full_fetch_at and health reason #371 Watch panel: last_full_fetch_at and health reason · S · unblocked; rewrites test_co_core_037_status_fields_parse_and_are_ignored.
• InfoItem tags #362 InfoItem tags · S · unblocked; the composer reads them.

Phase 3 — the jump

• Docs: drop the stale "remove the WatchedItem by hand"; break-glass runbook; link the charter #372 Docs: drop the stale "remove the WatchedItem by hand"; break-glass runbook; link the charter · XS · unblocked (CannObserv/cannobserv#562 closed 2026-10-16).

Phase 4 — hygiene

• Spec/registry hygiene: schema $id URLs, rep_spec schema_version, cross-check wording, info.registry cap #373 Spec/registry hygiene: schema $id URLs, rep_spec schema_version, cross-check wording, info.registry cap · S · item 2 bundles with RepSpec provider schemas: drop gdrive folder_id / ia collection (host config, as gcs); API refuses writer-less providers #153 (on Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357).

After F3

• Digest notifications: batch watch events into scheduled summaries (re-sited from watcher#63) #383 Digest notifications: batch watch events into scheduled summaries · M · builds on info.watch-events consumer: shadow → dispatch #369's subscriber.

Order

The design doc's cutover phases (Phase 1 build dark → Phase 2 flips F1 → F2 → F3 → F4 → Phase 3 jump → Phase 4 hygiene), plus each line's blocked-by edges.

Tiebreaks the coordinator added, numbered so the operator can challenge one:

1. Flip order: a child on an earlier flip's path goes first (F1 → F2 → F3 → F4), because Watcher flips one at a time in that order.
2. Dependency before size.
3. Context freshness: the child whose code just changed goes next.

Overrides: 2026-10-24: the operator took this epic into the #357 coordinator thread, ahead of that epic's queue, because Watcher's flips all wait here.

Boundary decisions

• Coordinated from the Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357 thread, decided 2026-10-24. This reverses the 2026-10-15 split ("its own Archiver thread"). The epics stay separate bodies.
• Watcher's children, the flips and the operator's go for each flip are recorded on [Epic] Headless Watcher and cohort boundaries watcher#368, not here. This epic checks them read-only.
• Phase 1 builds dark: nothing here changes what Watcher or Replicator does until a flip, except what a child names as live and inert (for example deactivation with every domain active).
• 2026-10-26: Watcher folded [Epic] Headless Watcher and cohort boundaries watcher#368 into [Epic] Burn-down: backlog outside #368 → bug fixes, sibling waits, operational stability (2026-10) watcher#394's coordinator thread (hand-off at 18:30Z). [Epic] Headless Watcher and cohort boundaries watcher#368 stays the parent record, so flips and gos are still read there. In watcher comments, archiver refs are always written CannObserv/archiver#N.

Exit criterion

Every child closed and verified live, and Watcher's F1–F4 flipped (read on CannObserv/watcher#368).

Verify recipe

• Gate: CI on the final SHA and on main's tip, by job: test, lint, changelog, client-drift (and uv audit (report-only), which never fails).
• Deploy: readlink /srv/archiver/live equals /health's build_id equals main. Runtime paths merged without a deploy raise drift lag after 8 h.
• By deliverable: migration → alembic_version via psql -d archiver (read-only); route → the served /openapi.json; stream → the outbox and the broker grant; consumer → its group on the broker and bus-health; dashboard → the operator re-checks on production.
• Production writes (imports from the Watcher export, flips) are the operator's, each time.
• Baseline: CI green by job on 319405a, 2026-10-24.

Waits

Verification debt

Log

• 2026-10-24: adopted onto coordinating-epics in Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357's thread (operator decision). Measured: every outside blocker is closed (CannObserv/cannobserv#568, CannObserv/cannobserv#562, ACLs, STREAMS.md inventory and probes for info.watch-events and info.watch-check broker#93, Provision an Archiver tenant; recreate or move Watcher's 3 production channels notifier#117); unblocked: Domains: become the sole owner (min_interval, default cadence, deactivation) #360, Watch lifecycle: archive and restore #361, InfoItem tags #362, Notification subscriptions: model, API and UI (global/domain/item scopes) #363, Notifier credential behind a unit-only enable flag #368, Check-now via info.watch-check #370, Watch panel: last_full_fetch_at and health reason #371, Docs: drop the stale "remove the WatchedItem by hand"; break-glass runbook; link the charter #372, Spec/registry hygiene: schema $id URLs, rep_spec schema_version, cross-check wording, info.registry cap #373. Watcher's F1–F4 all wait here. Digest notifications: batch watch events into scheduled summaries (re-sited from watcher#63) #383 brought in (named a child here by Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357's boundary line). Next: Domains: become the sole owner (min_interval, default cadence, deactivation) #360.
• 2026-10-24: Domains: become the sole owner (min_interval, default cadence, deactivation) #360 pre-flighted and briefed (D1–D4). Measured: 3 domains in production against Watcher's 4 exported; is_active today gates suggestions only; one builder (build_live_announcement) feeds delta and snapshot. Next: Domains: become the sole owner (min_interval, default cadence, deactivation) #360.
• 2026-10-24: notice posted on [Epic] Headless Watcher and cohort boundaries watcher#368 (issuecomment-6389464870): this epic has a coordinator, Domains: become the sole owner (min_interval, default cadence, deactivation) #360 is briefed, and Watcher's Waits rows stand. Bare refs qualified in place.
• 2026-10-26: Domains: become the sole owner (min_interval, default cadence, deactivation) #360 verified (PR #360 Domain ownership: suspension resolves active=false; fetch-policy columns; Watcher import #434; live 284bdc06dfc3 = main; import 4/4). Produce content.fetch-policy (dark until F1) #366 and Resolve cadence and announce a concrete interval #364 unblocked. fix: API key reveal box shows the new key (empty since #37) #435 (API key reveal box, outside both epics) is in the range and green by job. Eight brief errors recorded on the line. Next: Produce content.fetch-policy (dark until F1) #366.
• 2026-10-26: Produce content.fetch-policy (dark until F1) #366 pre-flighted and briefed (D1–D4). Measured: 0 outbox rows on content.fetch-policy; broker mirrors 500 / 10 / 300 s; the ratchet is strict >. FullSetFloor: source the content.fetch-policy mirror constants from Archiver broker#94 and Revoke Watcher's content.fetch-policy publish grant broker#95 added to Waits. Next: Produce content.fetch-policy (dark until F1) #366.
• 2026-10-26: Produce content.fetch-policy (dark until F1) #366 verified (PR #366 Produce content.fetch-policy, dark until F1 #436; live b0f57823c0f1 = main; dark). F1 now waits only on the operator's go on [Epic] Headless Watcher and cohort boundaries watcher#368 plus Watcher's #233 Nightly database backup to GCS, create-only, with restore #374 deploy, then archiver's runbook. FullSetFloor: source the content.fetch-policy mirror constants from Archiver broker#94 unblocked. New: Dependabot security updates for uv fail on every advisory (co-core 404) and mark main red #437 (Dependabot uv security updates fail on co-core 404, from 2026-10-26; filed on Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357's side). Next: Resolve cadence and announce a concrete interval #364.
• 2026-10-26: operator set F1 in motion. Filed F1: switch on the content.fetch-policy producer (after watcher#374 is live) #438 (archiver's half, body = brief). Go relayed on [Epic] Headless Watcher and cohort boundaries watcher#368 (issuecomment-6422814226) with the five-step order and the handshake (Watcher posts F1: stop the content.fetch-policy producer watcher#374's deploy time). The F1: stop the content.fetch-policy producer watcher#374 Waits row now points at us. Next: F1: switch on the content.fetch-policy producer (after watcher#374 is live) #438 kickoff; Resolve cadence and announce a concrete interval #364 in parallel.
• 2026-10-26: Resolve cadence and announce a concrete interval #364 pre-flighted and briefed (D1–D4). Watcher's chain read (announced → own item → domain → 1d, floor on top): archiver's announced interval already wins at Watcher's top tier. Next: F1: switch on the content.fetch-policy producer (after watcher#374 is live) #438 and Resolve cadence and announce a concrete interval #364 in parallel.
• 2026-10-26: F1 done. F1: switch on the content.fetch-policy producer (after watcher#374 is live) #438 verified (PR #438 F1: archiver.service produces content.fetch-policy (merge after watcher#374 + re-import) #439; live eb2215a8f86c; Archiver produces content.fetch-policy). Posted F1-done on [Epic] Headless Watcher and cohort boundaries watcher#368 (issuecomment-6428417416), with read-back asks on [Epic] content.fetch-policy producer moves to Archiver replicator#135 and Revoke Watcher's content.fetch-policy publish grant broker#95. FullSetFloor: source the content.fetch-policy mirror constants from Archiver broker#94 had landed. Watcher's coordination moved to [Epic] Burn-down: backlog outside #368 → bug fixes, sibling waits, operational stability (2026-10) watcher#394. Next: Resolve cadence and announce a concrete interval #364.
• 2026-10-27: Resolve cadence and announce a concrete interval #364 verified (PR #364 feat: announce the resolved cadence (profile → item → domain) #441; live d49e292ac4b6 = main; inert, 0 outbox rows; item import filled nothing). F1 read-backs answered by Replicator and broker; step 2's dry-run discharged by Resolve cadence and announce a concrete interval #364's host step 4. GH_TOKEN_WATCHER 401 blocks the F2: drop the domain cadence tier, temporal profiles, post-actions and throttle floor watcher#375 note. Adopt the cohort disk GC on co-registrar (gregoryfoster/skills maintaining-host-disk) #440 (host disk GC) out of epic, onto Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357. Next: Temporal profiles: model, API/UI, daily boundary job, post-actions #365.
• 2026-10-27: Temporal profiles: model, API/UI, daily boundary job, post-actions #365 pre-flighted and briefed (D0–D5); D0 recommends Watch lifecycle: archive and restore #361 first. GH_TOKEN_WATCHER rotated by the operator; Resolve cadence and announce a concrete interval #364's F2 notes posted on F2: drop the domain cadence tier, temporal profiles, post-actions and throttle floor watcher#375, debt row closed. Next: Watch lifecycle: archive and restore #361 pre-flight.
• 2026-10-27: Watch lifecycle: archive and restore #361 pre-flighted and briefed (D1–D5). No flip waits on it directly; F2 does through Temporal profiles: model, API/UI, daily boundary job, post-actions #365's archive post-action. Next: Watch lifecycle: archive and restore #361 kickoff.
• 2026-10-27: Watch lifecycle: archive and restore #361 verified (PR #361 feat: watch lifecycle — archive and restore an InfoItem #442; live 764a0f935fb6 = main; migration da4d2a2e19c7; inert, 0 outbox rows). Watcher told on [Epic] Headless Watcher and cohort boundaries watcher#368. Temporal profiles: model, API/UI, daily boundary job, post-actions #365 unblocked. Next: Temporal profiles: model, API/UI, daily boundary job, post-actions #365 kickoff.
• 2026-10-28: Temporal profiles: model, API/UI, daily boundary job, post-actions #365 verified (PR #365 feat: temporal profiles - model, API/UI, daily boundary job, post-actions #443; live f6bb673c5cfb = main; migration b7e3c95a1f20; inert, 0 outbox rows; boundary timer enabled). Archiver's half of F2 is done; the operator gave F2's go, recorded by Watcher's coordinator on F2: drop the domain cadence tier, temporal profiles, post-actions and throttle floor watcher#375. Next: Notification subscriptions: model, API and UI (global/domain/item scopes) #363 pre-flight (F3 path).
• 2026-10-28: Notification subscriptions: model, API and UI (global/domain/item scopes) #363 pre-flighted and briefed (D1–D6), head of F3's path. Next: Notification subscriptions: model, API and UI (global/domain/item scopes) #363 kickoff.
• 2026-10-29: Notification subscriptions: model, API and UI (global/domain/item scopes) #363 verified (PR #363 Notification subscriptions: model, API, UI and the Watcher import #446; live fa9ee65960ee = main; migration c4d81f6a2e07; 2 Watcher templates imported, no outbox effect). Temporal profiles: model, API/UI, daily boundary job, post-actions #365's first scheduled boundary run passed. New, out of epic, for Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357: Disk GC VERSION 3: bump skills-vendor to gregoryfoster/skills b1d6e303 before adopting #444, Adopt cannobserv v0.39.4: a TLS read error on /tools fetches becomes httpx.ReadError (preview-extraction 500 → 422 target_unreachable) (cannobserv#597) #445, dev_server.sh prints the dev database URL with its password #448, SocratiCode on loopback (gregoryfoster/skills#494): bump skills-vendor to b5be0b55 so the health hook catches a managed-mode fallback #449. Next: Notification composer, preview and test-send #367 and Notifier credential behind a unit-only enable flag #368 pre-flights.
• 2026-10-29: Notifier credential behind a unit-only enable flag #368 pre-flighted and briefed (D1–D5); Notifier reachability from co-registrar measured and posted on [Epic] Archiver becomes the notifying tenant notifier#116. Next: Notifier credential behind a unit-only enable flag #368 kickoff; Notification composer, preview and test-send #367 pre-flight.
• 2026-10-30: Notifier credential behind a unit-only enable flag #368 verified (PR #368 feat: Notifier credential behind a unit-only enable flag #450; live 5cfb5dacbe27; no migration, outbox unchanged; one Notifier call, a channel list, 200). Read-back asked on [Epic] Archiver becomes the notifying tenant notifier#116. info.watch-events consumer: shadow → dispatch #369 now waits only on Notification composer, preview and test-send #367. Outside the epic: test: pick the cascade tombstone by generation, not row order #455 (test flake from Notification subscriptions: model, API and UI (global/domain/item scopes) #363, merged), chore: bump astral-sh/setup-uv from 10.3.0 to 10.3.2 #451 (Dependabot), and for Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357 Load env files by parsing, not set -a; . — the pre-ship override and dev helpers source the repo .env #452 (env-file loading; dev_server.sh reads the .env that now holds the dev Notifier key), SocratiCode health hook: the vendored default now bounds swap — bump, update docs/SOCRATICODE.md, check the host's override #453, Adopt cannobserv v0.40.0: ContentReplicateCommandEmit refuses an off-rule or mismatched credentials_alias; becomes a skip row, no production change (cannobserv#495) #454. Next: Notification composer, preview and test-send #367 pre-flight.
• 2026-10-30: Notifier credential behind a unit-only enable flag #368's Notifier read-back passed ([Epic] Archiver becomes the notifying tenant notifier#116 issuecomment-6528964636): 0 dispatches, fresh last_used_at, one channel-list GET; Provision an Archiver tenant; recreate or move Watcher's 3 production channels notifier#117's debt discharged. Next: Notification composer, preview and test-send #367 pre-flight.
• 2026-10-30: Notification composer, preview and test-send #367 pre-flighted and briefed (D1–D7); the first send, and the epic's one new GCS grant. Next: Notification composer, preview and test-send #367 kickoff.
• 2026-11-01: Notification composer, preview and test-send #367 verified (PR #367 feat: notification composer, preview and test-send (epic #359) #456; live ea5355088791 = main; no migration, no outbox rows, no Notifier call since deploy). Read proof and Notifier read-back open. info.watch-events consumer: shadow → dispatch #369 unblocked. Outside the epic: Adopt the unified shipping-work skill: symlinks and configuration, not overrides (gregoryfoster/skills#537) #457 (shipping-work skill adoption) for Epic: follow-up coordination after #300: the trigger-watched queue (successor coordinator thread) #357. Next: info.watch-events consumer: shadow → dispatch #369 pre-flight.
• 2026-11-01: Notification composer, preview and test-send #367's processor-text read proven by the operator (ok 17735); the operator confirmed authorizing the /etc/archiver/.env base-URL removal and the production GCS read. info.watch-events consumer: shadow → dispatch #369 pre-flighted and briefed (D1–D8); Watcher told on F3: notification cutover instant T; stop dispatching watcher#376. Next: info.watch-events consumer: shadow → dispatch #369 kickoff.