Repository navigation
feat(mcp): pick Core schema domains from an intent with Jev - #20
Merged
Merged
Conversation
…tion and eval cases
…he threshold to 0.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fetch-core-graphql-schemanow takes an optionalintent("create an order for an existing customer"). TypeSafe's Jev model on Workers AI (typesafe/jev) picks the Core schema domains that intent needs, and the tool returns their combined schema in one call. Before, the agent had to guess one exactdomainper call among 84 of them.How it works
coreDomainSelectorsends Jev one request: the intent plus the domain index as state, and one yes/no question per domain ("is this domain needed?").>= 0.5are kept, most likely first, capped at 4. The splitter compacts their union (getCompactedDomainsSchema).domainstill works and wins overintent(exact, no Jev call).aibinding inwrangler.jsonc. The cached Awilix container holdsenv.AI: bindings don't change within an isolate (explained inCLAUDE.md).Measured on the live schema (
bin/eval-domain-selection.ts)12 real intents against the
furnitutCore schema (84 domains):customerscored 0.54.Also in here
{ state, result, gatewayMetadata }, not the bare{ model, answers }the Jev model page shows. The client unwraps both; without that, every call would have fallen back. The eval goes through the production client, so it covers this.subscription,pricelist,pipelineandinvite. It now uses real ones (subscriptionContract,priceList,orderPipelineStage,inviteToken, …).Before tagging a release
intentcall after deploy.intentcall falls back to the index, costing agents one extra round-trip.Known follow-ups (not in this PR)
services-provider.tsstill saysbuildContainer"ignores env".domaininput's description still uses'subscription'as an example.domaintogether with anintentreturns the "Unknown domain" list, not the intent's pick. This precedence isn't documented.skillsreferences by task (separate plan).The implementation plan is in
use-crystallize/mcp-servers/crystallize/docs/superpowers/plans/2026-09-29-jev-core-schema-domain-selection.md.bun testpasses (145/145), andbun type-check,bun lintandbun run buildare clean.