Merge dev into CI to activate the new test coverage - #20
Merged
Conversation
New workflow
Add sitemap/robots.txt and GA analytics regression coverage
The platform moved off opub-kc.civicdatalab.in (which served every realm under an /auth prefix) onto auth.civicdatalab.in, which serves at the root. Nothing tested that, and this suite's own config had silently broken as a result. Adds tests/api/smoke/test_api_007_keycloak_migration.py: - OIDC discovery is served at /realms/<realm> and the issuer is the migrated one - the legacy /auth realm path is gone (404) - KEYCLOAK_URL resolves to a realm whose issuer IS the migrated one -- a 200 alone is not enough, because the old server is still running and still answers 200 under /auth, so a stale value resolves happily and hides the misconfiguration - the frontend sign-in hands off to the migrated authorization endpoint for client_id=dataspace with a callback to this frontend - /health/ reports database, elasticsearch and redis healthy. telemetry is deliberately not asserted: dev reports not_configured while prod reports healthy, so asserting it would fail on dev for a non-bug - the analytics app also hands off to the migrated Keycloak .env.example: KEYCLOAK_URL was http://localhost:8080 with no note that conftest appends /realms/... itself, which is how the real .env ended up carrying an /auth suffix that produced 404s. Documents the contract and adds the KEYCLOAK_CLIENT_SECRET line, which keycloak_config reads but which was missing from the example. Verified against dev: 6 passed. The issuer assertion was confirmed to fail against the stale value before .env was corrected.
DataSpaceFrontend #442 added the Bhashini translation plugin; it had no coverage. The plugin is injected client-side by next/script (afterInteractive), so it never appears in server-rendered HTML -- curl sees nothing and only a real browser session can observe it. Hence a Selenium smoke test rather than an API one. Asserts only OUR integration points: the injected script (by src and by id), the container we render, and the holder the bundle mounts into. Deliberately asserts nothing about the widget's own rendered contents -- those come from translation-plugin.bhashini.co.in, a third party we do not control, and on dev the container is still an empty shell after the script loads. Asserting on it would be flaky by construction. Checks wait on presence rather than visibility: a <script> is never visible and the container can legitimately be empty. Verified against dev: 4 passed. Confirmed the container check genuinely fails when pointed at a selector that does not exist.
DataSpaceFrontend #444 added the privacy policy page; it had no coverage. Asserts the page loads, renders a heading, and contains meaningful body content -- enough to catch it 404ing or rendering an empty shell. Note the route is /privacy: the locale-prefixed /en/privacy 307-redirects, so the unprefixed path is the stable target. Verified against dev: 3 passed.
The workflow passed no Keycloak variables at all, so keycloak_config skipped and every authenticated API test skipped with it. The summary still read 0 failed. That is how a stale KEYCLOAK_URL survived the migration to auth.civicdatalab.in unnoticed. Host, realm and client id are public, so they are repo variables with literal defaults - correct out of the box, visible in review, and overridable without a code change. `dataspace` is a confidential client (ROPC returns 401 unauthorized_client without a secret), so KEYCLOAK_CLIENT_SECRET is a real secret and has to be provided. A preflight step now fails the job when any of the four is missing, rather than letting the run go green on skips, and checks OIDC discovery before any test runs. The issuer is compared against EXPECTED_ISSUER, pinned independently rather than derived from KEYCLOAK_URL. Deriving it is vacuous: the decommissioned server reports its own /auth issuer quite happily, so comparing an issuer to the host it came from passes for any running Keycloak. Verified - that version passed with KEYCLOAK_URL still set to opub-kc. Declared optional in workflow_call so the three existing callers keep parsing; they must pass KEYCLOAK_CLIENT_SECRET or api-smoke will fail the preflight.
Registration tests create real users, so they need a way to remove them. This wraps the `dataspace` service account (client_credentials) with the few user-lifecycle calls those tests need. available() is the important part: it reports whether a service-account token can actually be obtained, so a test can skip rather than create an account it has no way to delete. The service account has manage-users but little else - listing identity providers returns 403 - so this stays scoped to users.
These pages come from DataSpaceKeycloakTheme and are served by auth.civicdatalab.in, not the Next.js app. Keycloakify renders them client-side from kcContext, so the server HTML is just a JSON blob and every check needs a real browser. Two waits here are load-bearing, both found by watching tests flake: accept_terms() waits until the checkbox reports checked. React updates its state from the click event, and submitting before that lands posts the form without consent - the realm then rejects it and the test looks like a product bug rather than a race. body_text() waits for <body> to be non-empty. These pages render after navigation, so reading straight away returns "" and any assertion on the copy fails for reasons unrelated to the product. submit() waits for the URL to change but swallows the timeout: a rejected submission legitimately stays put, and the caller asserts which outcome happened. The redirect_uri is the app's real NextAuth callback. An unregistered one makes the realm answer 400 with error.ftl, which presents as missing elements rather than a bad URL.
Seven read-only checks on the Keycloak sign-in and registration pages.
Nothing here submits a form or creates an account.
Google is a configured identity provider on the realm, and the sign-in
page renders "Continue With Google" pointing at /broker/google/login.
None of that was covered, so it could disappear without any test
noticing.
The privacy links are the interesting part. The theme builds them with
getPrivacyHref() as {baseUrl}/privacy, and the page they point at is
covered separately by test_con_privacy_policy.py - but nothing checked
that the link actually resolves. The two could drift apart with both
suites green, so these tests follow the href and assert 200.
Three tests that exercise registration end to end, including whether the privacy consent checkbox is actually enforced. The consent test is the reason this exists. termsAccepted carries required=false in the DOM, so nothing in the markup enforces it - only the outcome is evidence. Verified against dev: submitting without consent does not go through and creates no account. The test asserts that via the admin API rather than trusting the page. The other two cover the happy path: an account really is created, and the user is routed to VERIFY_EMAIL rather than straight into the app. These create real users, so two rules apply. Nothing is created that cannot be deleted - the keycloak_admin fixture skips the test when no service-account token is available. And cleanup runs in fixture teardown, so it happens even when a test fails mid-flow. Emails use the .invalid TLD so a stray can never be a deliverable address. Verified: 3 consecutive runs green, zero strays left in the realm, and non-vacuous - ticking the consent box makes the enforcement test fail with "an account was created WITHOUT the privacy consent box ticked".
…new-features test: cover the Keycloak migration, Bhashini, privacy, registration and Google sign-in
CI is the ref every caller uses (run-smoke.yml@CI) and the ref the workflow checks out, so nothing on dev executes until it lands here. This brings across the Keycloak migration, Bhashini, privacy, auth-page and registration coverage. Conflict in .github/workflows/run-smoke.yml resolved in favour of CI. Both branches carried the Keycloak preflight - dev's copy existed only so a merge could not silently revert it - and CI's version is the complete one, having since gained the API_BASE_URL default and the api_base_url / deployed_sha / min_passed inputs that never existed on dev. The resolved file is byte-identical to CI's. Verified on the merged tree: api-smoke 20 passed / 1 skipped, consumer smoke 35 passed / 1 failed where the one failure (usecase_cards) passes on its own here and on CI - a local flake under load, as were six seo_analytics failures in an earlier run that pass cleanly on re-run.
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the new tests onto the branch that actually runs them.
CIis the ref every caller uses (run-smoke.yml@CI) and the ref the workflow itself checks out:So nothing merged to
devexecutes anywhere until it lands here. #17 has been merged for a while and has never run.What comes across
test_api_007)utils/keycloak_admin.pyConflict resolution
One conflict, in
.github/workflows/run-smoke.yml, resolved in favour ofCI.Both branches carried the Keycloak preflight — the copy on
devexisted only so this merge could not silently revert it.CI's version is the complete one, having since gained theAPI_BASE_URLdefault (#19) and theapi_base_url/deployed_sha/min_passedinputs that never existed ondev. The resolved file is byte-identical toCI's.Verification on the merged tree
api-smoke: 20 passed, 1 skipped — up from 16, the new Keycloak tests now running.consumer smoke: 35 passed, 1 failed. That one (test_TC_UC_02_usecase_cards) passes on its own both here and onCI, twice each — a local flake under load. An earlier run also showed sixseo_analyticsfailures which pass cleanly on re-run for the same reason.Note
tests/consumer/functional/is not in any job's path —consumer-smokerunstests/consumer/smokeonly. So the three registration tests will not execute in CI even after this merge. Given they create real accounts, that is arguably the right default, but it is worth a deliberate decision rather than an accident.