Skip to content

feat(mcp): pick Core schema domains from an intent with Jev - #20

Merged
Plopix merged 13 commits into
mainfrom
feat/jev-core-domain-selection
Sep 29, 2026
Merged

Plopix merged 13 commits into
mainfrom
feat/jev-core-domain-selection

Conversation

@Plopix

@Plopix Plopix commented Sep 29, 2026

Copy link
Copy Markdown
Member
Q A
Branch? main
Bug fix? no
New feature? yes
BC breaks? no
Fixed tickets —

fetch-core-graphql-schema now takes an optional intent ("create an order for an existing customer"). TypeSafe's Jev model on Workers AI (typesafe/jev) picks the Core schema domains that intent needs, and the tool returns their combined schema in one call. Before, the agent had to guess one exact domain per call among 84 of them.

How it works

  • coreDomainSelector sends Jev one request: the intent plus the domain index as state, and one yes/no question per domain ("is this domain needed?").
  • Domains with probability >= 0.5 are kept, most likely first, capped at 4. The splitter compacts their union (getCompactedDomainsSchema).
  • domain still works and wins over intent (exact, no Jev call).
  • Jev can't break the tool. If it times out (8s), errors, returns unusable answers, or picks nothing, the tool returns the domain index, as it does today.
  • New ai binding in wrangler.jsonc. The cached Awilix container holds env.AI: bindings don't change within an isolate (explained in CLAUDE.md).

Measured on the live schema (bin/eval-domain-selection.ts)

12 real intents against the furnitut Core schema (84 domains):

threshold intents with all needed domains avg domains returned
0.2 12/12 2.50
0.3 12/12 2.08
0.5 12/12 1.42
  • Per call: about 12.8k input tokens (roughly $0.0005) and about 390ms over REST.
  • Thin margin: for "create an order for an existing customer", customer scored 0.54.

Also in here

  • AI Gateway envelope. Third-party models answer as { state, result, gatewayMetadata }, not the bare { model, answers } the Jev model page shows. The client unwraps both; without that, every call would have fallen back. The eval goes through the production client, so it covers this.
  • Tool description domains. The description's "Common domains" list named 8 domains (out of 12) that don't exist, such as subscription, pricelist, pipeline and invite. It now uses real ones (subscriptionContract, priceList, orderPipelineStage, inviteToken, …).

Before tagging a release

  • Only REST has been exercised live. The deployed binding should return the same envelope (per Cloudflare's docs), and the client accepts both shapes either way. Worth checking the first real intent call after deploy.
  • Billing. AI Gateway Unified Billing must stay funded; otherwise every intent call falls back to the index, costing agents one extra round-trip.

Known follow-ups (not in this PR)

  • A comment in services-provider.ts still says buildContainer "ignores env".
  • The domain input's description still uses 'subscription' as an example.
  • An unknown domain together with an intent returns the "Unknown domain" list, not the intent's pick. This precedence isn't documented.
  • There's no size cap on the combined schema of up to 4 domains.
  • Moving the shared question text into the state could roughly halve the tokens per call.
  • Use Jev to pick skills references by task (separate plan).

The implementation plan is in use-crystallize/mcp-servers/crystallize/docs/superpowers/plans/2026-09-29-jev-core-schema-domain-selection.md. bun test passes (145/145), and bun type-check, bun lint and bun run build are clean.

@Plopix
Plopix merged commit 0863e7f into main Sep 29, 2026
1 check passed
@Plopix
Plopix deleted the feat/jev-core-domain-selection branch September 29, 2026 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant