Check Builds against accepted learnings with a small trained model #202

Let teams train a small vision-language model, a model that reads images and text together, to check each new Build against their accepted learnings.

Why it matters: Accepted learnings guide coding agents, but nothing checks the next Build against them. Reviewers still catch the same visual problems by eye, like a form field without a visible label.

Who needs this: Reviewers and repository administrators. They need a check that reads a Build's screenshots, and they need to know how sure that check is.

How it works: A decision model doesn't write prose. It answers typed questions with probabilities. Replica turns each accepted learning into a yes-or-no question about a page ("Does the candidate page follow this team learning in every screenshot?"), and adds two more: which learning the page breaks most clearly, and how far the page is from the learnings.

The team trains the model further on its own review history with reinforcement learning (RL). RL is practice with a scorekeeper: the model answers, gets a score, and shifts toward answers that score higher. The expected answers, called labels, come from what reviewers already did. A thread that reports a problem marks its Build as breaking that learning on that page. When the same reviewer resolves the thread after a later Build, that Build is marked as following it.

The score pays the most for honest probabilities. On a scale of 0 to 100, a confident right answer scores 100, a 50/50 answer scores 75, and a confident wrong answer scores 0. That rewards calibrated answers. A calibrated model that says 80% is right about 80% of the time. Training doesn't guarantee calibration, so it's measured on pages the model never practiced on, and Hub needs it before it can decide when to show a flag.

Practice happens in an environment, a small server that hands out tasks and scores answers. It follows OpenEnv, an open standard for connecting RL environments to trainers, and runs in a container on hardware the team controls or on a managed service. Only trainers with the team's secret token can connect.

flowchart LR
  threads["Review threads"] --> learnings["Accepted learnings"]
  learnings --> questions["Questions"]
  threads --> labels["Labels from resolved threads"]
  screenshots["Build screenshots"] --> taskset["Taskset"]
  questions --> taskset
  labels --> taskset
  taskset --> env["Environment in your container"]
  trainer["Trainer"] -- "answers" --> env
  env -- "scores" --> trainer
  trainer --> model["Trained model"]
  model --> person["Person decides"]

What changes for users:

• Administrators choose a base model and where training runs on the Learnings page, and see the questions the model will answer. The list holds six vision-language models under 10 billion parameters from Liquid AI, Google, IBM, and Microsoft.
• Operators turn a training export and Build screenshots into a taskset, a folder of screenshots with questions and labels, then run the environment and a trainer.
• Training data leaves the learning store only through a training export, which needs its own consent grant.
• A person decides whether a trained model replaces the base model.

Not in the first version: Hub doesn't train, store, or run models, and doesn't show the model's flags on new Builds yet. Only yes-or-no questions get labels, and only pages with Build screenshots can be labeled.

Risk: A "follows" label may be noisy, because the reviewer may have checked the fix at only one width and theme.

Other places RL could help later: training the agent that builds prototypes, with this model as one of its graders; learning from reviewers who confirm or dismiss the model's flags; and replacing the paid comment-labeling step with a small trained model.

Design record: docs/adr/0002-decision-model-and-rl-environment.md