Skip to content

feat(dirty-set): stay incremental across package boundary changes - #24

Merged
honnix merged 7 commits into
mainfrom
honnix/797931
Oct 1, 2026
Merged

honnix merged 7 commits into
mainfrom
honnix/797931

Conversation

@honnix

@honnix honnix commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Summary

Seeded runs no longer fall back to a full rehash (package_boundary_change) when a BUILD file is added or deleted.

Adding or removing a package at directory D only changes the hashes of targets in D, targets in the nearest enclosing package (its globs and subpackages() gain or lose D's files), and their reverse deps. Packages further up can't see past the enclosing package. Both cases now reuse the existing dirty-package machinery (wildcard re-list, probe pruning, rdeps propagation):

  • Added package: re-listed with //P:all; the nearest enclosing package is dirtied as if its BUILD file changed.
  • Removed package: new DirtySetResult.RemovedPackages. Its seed labels are dirtied (dropped from the merge, rdeps propagated), but it never gets a wildcard or a probe, and its labels are never carried explicitly. The package absorbing its files is dirtied.
  • The nearest enclosing package is taken at both the seed and the target revision; they differ only when several boundaries change in one diff.
  • Whether a deleted BUILD file's package survives (the other BUILD file name remains) is decided from the diff when that file appears there, otherwise by one batched, path-limited git ls-tree per run. If the lookup fails, the run falls back with a new code, package_lookup_error, so package_boundary_change is no longer emitted.
  • Removed the dead R* status branch: gitDiffNameStatus uses --no-renames.

.bzl, module and other fallback triggers are unchanged.

Test plan

  • Unit tests for added / nested / root / deleted / move / parent-also-deleted / surviving-sibling / lookup-failure cases, batched single lookup, and git ls-tree against a temp repo
  • Replayed real commits in a large monorepo that add, nest, move and delete packages (including a package deleted with its parent): seeded output identical to full under hash-differ, all runs incremental with no fallback

🤖 Generated with Claude Code

honnix and others added 5 commits October 1, 2026 11:51
Adding or removing a package only changes the hashes of its own targets,
the targets of its nearest enclosing package (whose globs gain or lose
the directory's files), and their reverse deps. Route both cases through
the existing dirty-package machinery instead of a full rehash: added
packages are re-listed by wildcard, removed packages have their labels
dropped, and the nearest enclosing package on both sides is dirtied.

Whether a deleted BUILD file's package survives is resolved from the
diff when possible, otherwise with a single batched git ls-tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rename siblingBuildFile to otherBuildFile and ambiguousDeletions to
deletionsNeedingLookup, and spell out at the call site why the other
BUILD file name decides whether the package is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heir own code

ComputeDirtySet now reads as its steps: fallback triggers, seed package
index, boundary changes, dirty packages, rdeps propagation (reusing
propagateFrom).

A failing package existence lookup reports package_lookup_error instead
of package_boundary_change, so the latter no longer means two different
things.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@honnix
honnix marked this pull request as ready for review October 1, 2026 12:49
honnix and others added 2 commits October 1, 2026 15:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…package deletions

Also drop BUILD-file boundaries from the README's fallback triggers, and
explain why source files are attributed using the seed's packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@honnix
honnix merged commit e7f98d8 into main Oct 1, 2026
3 of 4 checks passed
Comment thread pkg/dirty_set.go
Comment on lines +238 to +241
// findDirtyPackages returns every package whose targets must be rehashed:
// packages of changed files, the added and removed packages themselves, and
// the nearest enclosing package of each. The result includes removed
// packages, whose seed labels must be invalidated.

@mattnworb mattnworb Oct 1, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked Codex to help me review the PR because I find the hypotheticals and possible conditions we are getting into at this point make my head hurt, and it had one thing to flag here - but I think the risk is acceptable:

indexSeedPackages only knows packages that appear in target hashes or dependency edges. When a BUILD file is added or removed, the nearest enclosing package may already exist but have no labels in that seed graph—for example, its rules produced no matching targets at the seed revision.

In that case, findDirtyPackages can miss the enclosing package. Its unchanged BUILD file may produce a different target set because glob() or subpackages() sees the new package boundary, but the scoped query never re-lists it. A removed child package with no seed labels can even take the “no targets affected” path and copy the seed output

IIUC the risk would be that a package exists but has no targets/labels, meaning it only defines something like a filegroup based on a glob - which seems rare

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants