control-plane-api: live capability-ceiling threading (#3376 task 3c)

[bbartman]

Task 3c of the #3376 capability-mask plan (implementation plan), stacked on #3405: live ceiling threading. The Authority extractor's capability mask now reaches every request-path authorization walk, so enforcement of masked tokens goes live — while no production path mints masked tokens yet.

What changed

Definitive vs. retriable denials become a typed distinction. AuthZResult's error side is now AuthZError { Retriable(tonic::Status), Definitive(Forbidden) }. A walk denial stays retriable — a fresh Snapshot may reveal a just-committed grant, so authorization_outcome holds it for refresh as before. A mask shortfall is a pure function of the bearer's verified claims which no Snapshot can change: authorization_outcome returns it immediately as the new ApiError::Forbidden, which renders the structured 403 body (error: "missing_capabilities", missing_capabilities: [...]) on REST and a GraphQL error carrying the same fields in extensions. An MCP client parses the missing names identically on both surfaces and drives the upgrade_token re-mint.

The mask-shortfall pre-check lives inside the shared policy functions — evaluate_names_authorization, the three authorize_user_* evaluate_authorization helpers, and storage-mappings' check_authorization — computed as required − mask.apply(required) before the walk, so no call site can forget it and the error discloses nothing about the user's actual grants (no existence or grant oracle in the definitive/retriable distinction).

Route-level REQUIRED declarations are used where the capability is statically known. /api/v1/catalog/status and /api/v1/metrics/{*prefix} take Authority<RequireViewer>, which declares CapabilityBundle::Viewer — the bits legacy Read requires — in the same bundle vocabulary the capability_mask claim speaks, rejecting mask shortfalls at extraction before the handler or any DB access runs. A parity test pins the bundle's bits to bits_for_legacy(Read) so the declaration and the handler's walk requirement cannot drift. Everywhere a const structurally can't reach — GraphQL's single route, the body-dependent /authorize/user/* endpoints — the identical runtime pre-check applies.

Every request-path walk runs under the bearer's mask (~17 sites task 2 pinned to ALL_CAPABILITIES):

• REST: /authorize/user/{collection,task,prefix} (including the estuary_support/ secondary Admin walks — the mask applies uniformly to every walk a request performs), /api/v1/catalog/status (both the gate and the connected filter), /api/v1/metrics.
• GraphQL hard gates: verify_authorization (22 call sites), storage-mappings' check_authorization, and the direct evaluate_names_authorization callers (alerts, alert_configs, live_spec_refs).
• GraphQL filters and metadata: may_access, authorized_prefixes / filtered_authorized_prefixes and their consumers (prefixes, invite_links, alert_configs, storage_mappings, service_accounts), attach_user_capabilities (live_spec_refs, live_specs, data_planes), the data-planes list filter, and storage-mappings' attached user_capability — filter and metadata are computed under the same mask, so masked listings narrow coherently rather than erroring.

Plumbing follows the agreed split: ctx-taking GraphQL helpers (may_access) read the mask from context internally; &Envelope/claims-taking helpers take an explicit mask: CapabilityMask parameter, threaded from a resolver's single *ctx.data::<models::authz::CapabilityMask>()? pull or a REST handler's Authority { envelope, mask, .. } binding. No new context objects.

Deliberately not threaded (mask-evaporation class): the two agent/src/discovers.rs walks and live_specs::partition_by_authorization (consumed only by the discovers/publications executors) stay explicitly unmasked with comments — they run after the requesting JWT and its mask are gone, authorizing by user identity. Whether and how the mask should propagate across the async discover boundary is a new dedicated task on the #3376 plan (spike + implementation).

Decisions made in review of this task

1. No blanket route consts. REQUIRED consts only where the required capability is a static fact of the route (status, metrics); runtime pre-checks with the identical error shape everywhere else. Considered and rejected: dropping the const mechanism entirely (loses free extraction-time fail-fast), and consts-only (structurally impossible for GraphQL and body-dependent routes).
2. AuthZError enum over alternatives (option A): pre-check helpers at every call site risk omission; smuggling the structured body through tonic::Status would send definitive denials through the snapshot-retry limbo and force clients to parse JSON out of a message string.
3. The estuary_support/ secondary check is masked like every other walk — the invariant is that a masked token's authority is a subset of the same bearer's unmasked authority, with no special-cased walks.
4. Legacy user_capability metadata under a mask keeps task 2's semantics: the mask gates node reachability; a reached node's legacy value passes through un-attenuated as informational compatibility metadata.
5. Discovers mask propagation split out as its own task: a spike first — what could a mask-limited discovery expose or wrongly withhold, and should the mask propagate across the async boundary at all — then implementation only if the spike says yes.

Tests (written first, TDD)

• envelope.rs: definitive denials bypass snapshot-retry (stale and fresh snapshots); retriable denials keep their existing provisional/terminal behavior.
• server/mod.rs: the chokepoint pre-check — mask shortfall is definitive and names exactly the missing bits, evaluated before the walk; covering-mask and unmasked walks are ordinary.
• public/mod.rs: oneshot-router tests of the two RequireViewer routes — structured 403 at extraction with no DB touched; covering and unmasked bearers pass extraction identically; plus the const↔bits_for_legacy(Read) parity pin.
• authorize_user_collection.rs (class representative for /authorize/user/*): masked shortfall, empty-mask identity-only denial, covering-mask parity with the unmasked outcome, and the masked estuary_support/ Admin walk.
• graphql/mod.rs (DB-backed, class representative for GraphQL hard gates): masked denial with the full extensions shape pinned, empty-mask denial, covering-mask/unmasked parity.
• prefixes.rs (class representative for GraphQL filters): a mask without Delegate confines the listing to direct grants; adding Delegate restores role-graph traversal; an empty mask lists nothing — never an error.
• storage_mappings.rs: unit coverage of check_authorization under masks, plus the list-coherence case — row filter and attached user_capability computed under one mask.
• Every pre-existing test passes unchanged (all fixtures mint unmasked tokens), which is the unmasked-parity assertion for the whole surface.