Skip to content

doc tests: schema-check only the PR's pages, including new ones - #1540

Open
kristin-kronstain-brown wants to merge 2 commits into
mainfrom
schema-check-new-pages-on-pr
Open

kristin-kronstain-brown wants to merge 2 commits into
mainfrom
schema-check-new-pages-on-pr

Conversation

@kristin-kronstain-brown

Copy link
Copy Markdown
Collaborator

Why

A new page gets its schema test from the nightly scaffold in solo-io/docs-tests, which runs only after the page merges. Until then, nothing checks the page's YAML, and the first PR to run the new test afterward fails for a mistake it did not make.

docs-tests now handles this in doc_test_schema_check.py: any page passed with --file that has no test yet is checked with the schema test the nightly scaffold would add. The test goes into a temporary copy of docs-tests only.

What changed

  • The discover job exposes the pages the PR reaches (already computed in "Get changed markdown files") as a pages output.
  • "Run schema checks" passes those pages as --file, the same way kgateway.dev does.

Behavior change

On a pull request, the schema check now runs only for the pages the PR reaches, not for every schema test in the repo.

  • Upside: A PR no longer fails because of another page's schema test.
  • Downside: A change that affects every page without touching it, such as a CRD version change, is only checked on the pages the PR reaches. The nightly run still checks every page.
  • When the PR reaches no pages, and on scheduled and manual runs, the full suite runs as before.

Testing

Run locally with the updated docs-tests script against a scratch copy of this repo, using a new standalone page whose oidc policy was missing the required redirectURI:

  • Without --file, the page had no test and nothing checked it.
  • With --file <page>, the check added scratch-new-page-schema and failed with 'redirectURI' is a required property. The docs-tests checkout was not changed.

The workflow change itself has not run in CI yet. This PR changes doc-tests.yaml and reaches no content pages, so its own run uses the full suite.

🤖 Generated with Claude Code

Pass the pages a PR reaches to the schema check as --file, as
kgateway.dev does. docs-tests then also checks any of them that has no
test yet, so a new page is checked on the PR that adds it instead of
after the nightly scaffold adds its test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Deploying agentproxy with  Cloudflare Pages  Cloudflare Pages

Latest commit: f155c28
Status: ✅  Deploy successful!
Preview URL: https://02a61e3e.agentproxy.pages.dev
Branch Preview URL: https://schema-check-new-pages-on-pr.agentproxy.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant