Implement registry version check script

[thomashoneyman]

Closes #426 by adding a .#version-check script to report registry versions newer than the versions in the latest package set. This reuses the package set updater candidate traversal instead of being brand-new.

I ran the script against package set 77.13.1 and found 19 pending versions. Not all of them can compile together, but 14 of them do, and that's submitted as purescript/registry#560.

Five versions were intentionally left out:

• barlow-lens@1.0.0 breaks morello@0.4.0, which uses the old Barlow API and declares <0.10.0.
• elmish@0.14.0 breaks elmish-hooks@0.11.0; meanwhile, elmish-html@0.12.0 requires Elmish 0.14, so neither Elmish candidate can move until the dependent packages are compatible.
• hyrule@2.4.0 removes modules still imported by bolson@0.3.9, deku@0.9.24, and ocarina@1.5.4.
• node-child-process@12.0.0 changes the exit API and breaks dotenv@4.0.3 and node-execa@5.0.0.

So this is the point of the script: surface upgrades that automatic daily updates could not accept so maintainers can identify and submit a compatible coordinated batch.

[f-f]

This is cool - two things come to mind:

• we should have a policy of dropping packages that are holding on updates for too long. E.g. picture the scenario where a package that has many dependants is blocked by a package that has not many users - the former should not be held up. Maybe we can have some sort of expiration (e.g. holding a package for 30 days triggers removal or something like that)
• we could wire this up to @pacchettibotti automatically opening issues in the repos that need updating to the newer version, to speed up the process of the package sets catching up to things

[thomashoneyman]

Yeah, I agree. I don't have a specific policy in mind but directionally I'm on board. To what degree should that be encoded into this script's report? We could augment this with more data than it currently gives.

[thomashoneyman]

Also, we could extend this with a planner that makes it more obvious whether the discovered versions will even work together and suggest a batch. Something like:

1. Give every candidate package two choices: its current version or reported candidate.
2. Check every package manifest against the exact proposed package-set versions.
3. Maximize the number of accepted candidates while satisfying all declared ranges.
4. Repeat until stable, since rejecting one candidate can invalidate another; like for example, rejecting Elmish 0.14 also requires rejecting elmish-html@0.12.0.
5. Run one read-only atomic compilation of the selected plan to prove it

For this batch, this would have discovered the 14 packages that work without me having to do a separate manual triage:

• morello excludes Barlow 1.0.
• elmish-hooks excludes Elmish 0.14, which consequently excludes the new elmish-html.
• bolson and deku exclude Hyrule 2.4.
• dotenv and node-execa exclude node-child-process 12.

...this doesn't help with what you're describing, but maybe we can re-think this script such that it's a more useful swiss-army-knife for helping get the package sets updated with the best options when it can't be done automatically due to breaking changes.

[f-f]

To what degree should that be encoded into this script's report?

I to get us a good glance we could have:

• who is holding up who
• and for how long; that is: how many days ago the incompatible version was published
• how many dependants each of the packages involved have

Also I think we could be running this in a weekly cronjob thing in the Registry repo, opening an issue/PR if it finds something

[thomashoneyman]

d283822 has some followups that address your points:

1. The version check now produces JSON and markdown reports, which themselves cover staleness, direct and transitive dependant counts, blocked version ranges, reverse blockers, compiler compatibility, etc.
2. It also includes a planner which uses our bounded ranges; it considers current and compatible newer releases, maximizes upgrade count and then freshness, and supports coordinated upgrades
3. You can optionally compile the planned payload with --verify so that you know the proposed plan will work
4. Adds a weekly workflow which writes to a rolling issue with that week's report (doesn't actually submit anything)

Tried it out with a live run against package set 78.1.0 / compiler 0.15.15 and it produced a workable plan — including settling on hyrule@2.3.9 as an ugrade since 2.4.0 is blocked.

[thomashoneyman]

Latest commit is a prototype, still hacking on it a little. But as I was working on the version check script, I realized we could be doing a lot more automatically — and still safely — to update the package sets. Our current automatic upgrade is basically just "try all new packages in the last 24hrs, and if any fail, go sequentially and only include ones which succeed." Very naive, and if a package doesn't work in its 24h window for whatever reason it is never retried — even if it would work later on (such as a dependency finally upgrading).

So the new prototype implements one shared planner that can be used by both the package set updater and the version check workflow. Any version of a package that's already in the set which is newer than what's in the set is eligible for any new plan, plus recently uploaded packages even if they've never been in the set. We restrict to a single compiler version but otherwise ignore ranges. We still try all packages first, but then we do bounded best-first latest/intermediate/current searches.

The package set updater, if it is capable of finding a nonempty addition/upgrade plan, can just submit it. If it finds removals, downgrades, etc. then we can have that notify trustees for action, but it will never be automatically done.

[greptile-apps]

Greptile Summary

This PR adds a PackageSetVersionChecker script that compile-verifies all pending registry upgrades against the latest package set, surfacing what can be submitted automatically and what requires trustee intervention. To support this, a new PackageSetPlanner module extracts a 5-phase greedy planner that is now shared between the updater and checker scripts; packageSetUpdate in API.purs is simultaneously tightened from sequential (partial-success) to atomic (all-or-nothing) compilation.

• New PackageSetVersionChecker script – plans upgrades via plannerCandidates with no addition seeds, runs the full 5-phase planner + removal analysis, and renders a Markdown report (GitHub issue) with a JSON payload fence for every verified automatic upgrade and every trustee-reviewable removal-required upgrade.
• New PackageSetPlanner module – 5 phases: probe all-latest, greedy drop, rescue, round-robin fallback, interaction-component probing; budget split between planning (60) and removal analysis (100 total); infrastructure errors surface via EXCEPT, never as fake incompatibilities.
• API.purs semantic change – upgradeSequential replaced by upgradeAtomic; a submitted package-set payload is now treated as exact and atomic: the whole batch compiles or nothing is published. commitMessage correspondingly uses changeSet instead of the old succeeded partial map.

Confidence Score: 5/5

Safe to merge. The atomic-only API change is the biggest behavioral shift, but it is intentional, correctly implemented, and covered by a new unit test. The planner logic is sophisticated but well-tested with both synthetic and real-world (Elmish) fixtures.

The new planner is a large addition, but every phase has targeted tests, the budget invariants are verified by the 'reserves the removal-analysis budget' test, and the real-world Elmish fixture confirms the algorithm on a case the author actually ran against package set 77. The API change from sequential to atomic is a deliberate tightening of the contract, and the updated test confirms the error path. No mismatches between types, codecs, or effect rows were found.

Files Needing Attention: No files require special attention. The most complex new file is app/src/App/PackageSetPlanner.purs; its test coverage in app/test/App/PackageSetPlanner.purs is thorough.

Important Files Changed

Filename	Overview
scripts/src/PackageSetVersionChecker.purs	New script: plans and reports compile-verified pending package set upgrades. Correctly passes Map.empty as additionSeeds to restrict candidates to existing-package upgrades, interprets all required effects, and guards the output path.
app/src/App/PackageSetPlanner.purs	New 5-phase greedy planner module. Budget split (maxPlanProbes=60, maxTotalProbes=100) is correct; infrastructure errors surface via EXCEPT; removal closure expansion is iterative and correctly terminates; validateCandidates rejects malformed inputs before probing.
app/src/App/API.purs	upgradeSequential replaced by upgradeAtomic — payload is now all-or-nothing. commitMessage correctly uses changeSet (equivalent to succeeded when atomic succeeds). Log message updated to use packageSetOperationCodec on the full operation rather than the inner data.
scripts/src/PackageSetUpdater.purs	Refactored from ad-hoc validatePackageSetCandidates / UpgradeSequential to the new 5-phase planner. Now waits for the job to complete (pollPackageSetJob) rather than returning after enqueueing. Effect row correctly extended with PACKAGE_SETS, STORAGE, RESOURCE_ENV.
app/test/App/PackageSetPlanner.purs	Comprehensive unit tests covering all 5 planner phases, budget enforcement, infrastructure-error isolation, and the Elmish real-world fixture. Removal analysis tests verify iterative closure expansion and correct updates map construction.
app/test/App/API.purs	New packageSetUpdateSpec test verifies that an atomically-failing update is rejected entirely (no writes, no mirrors) and the error message contains 'failed to compile atomically'.
.github/workflows/package-set-version-check.yml	New workflow (workflow_dispatch only). Correctly guards issue creation with hashFiles check (addressing the previous review comment). 120-minute timeout is reasonable given up to 100 full-set compile probes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[PackageSetVersionChecker main] --> B[readLatestPackageSet]
    B --> |Nothing| C[Log warn, return Nothing]
    B --> |Just packageSet| D[plannerCandidates with empty additionSeeds]
    D --> E[Planner.planUpgrades probeAtomic]

    subgraph "5-Phase Planner"
        E --> P1[Phase 1+2: probe all-latest, greedily drop offenders]
        P1 --> P2[Phase 3: rescue each dropped candidate individually]
        P2 --> P3[Phase 4: round-robin fallback to older versions]
        P3 --> P4[Phase 5: probe interaction components]
        P4 --> PR[Plan: verified + blocked + truncated]
    end

    PR --> F[Planner.analyzeRemovals]

    subgraph "Removal Analysis"
        F --> RA1[For each BlockedGroup: compute reverse-dep closure]
        RA1 --> RA2[probe packageSet with removals + updates]
        RA2 --> |Compiles| RA3[RemovalsVerified]
        RA2 --> |Fails, new blockers| RA4[expand closure, retry]
        RA2 --> |Fails, no progress| RA5[RemovalsUnverified]
        RA4 --> RA2
    end

    F --> G[renderReport as Markdown]
    G --> H{--output flag?}
    H --> |Yes| I[Write to file]
    H --> |No| J[Print to stdout only]
    I --> K[Workflow: gh issue create]

Loading

_{Reviews (5): Last reviewed commit: "Merge commit '84b4028' into HEAD" | Re-trigger Greptile}