Skip to content

fix(store): keep early capacity rebuilds read-only - #838

Closed
ShalyX wants to merge 1 commit into
flop-labs:mainfrom
ShalyX:fix/namespace-count-rebuild-race
Closed

ShalyX wants to merge 1 commit into
flop-labs:mainfrom
ShalyX:fix/namespace-count-rebuild-race

Conversation

@ShalyX

@ShalyX ShalyX commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Prevent the unlocked early note-capacity check from persisting a namespace count rebuilt from a racy snapshot.
  • Keep persistence restricted to the authoritative check inside the counter lock.
  • Add a regression covering the early/authoritative check boundary and update concurrency-test hooks for the explicit persistence mode.

Closes #637.

Why this matters

When a namespace .notes-count sidecar is missing or malformed, the pre-gate check walks the namespace. Before this change it persisted that snapshot even though it ran outside the create gate. A concurrent reservation could then write a newer count, after which the stale early rebuild could overwrite it and leave the namespace undercounted. Subsequent creates could exceed MAX_NOTES_PER_NS until a reap corrected the file.

Verification

  • uv run pytest tests/unit/test_note_count.py -q — 33 passed
  • git diff --check
  • uv run ruff check .
  • uv run ruff format --check .
  • uv run ty check
  • uv run sz.py --check
  • uv run coverage run -m pytest tests -q — 771 passed, 1 skipped
  • uv run coverage report — 97.97% total coverage

@github-actions

Copy link
Copy Markdown

Open pull requests citing the same issues:

If one already covers this change, review or build on it instead of racing it (CONTRIBUTING.md "Overlapping work").

@Minh3132 Minh3132 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On exact head 699ff4599a1370a5ed0ebb10e70b5cbec548b38e, the new regression does not actually prove the stale-sidecar bug is fixed; it proves only that _create_gate passes False to the early check and True to the authoritative one.

test_the_early_capacity_check_cannot_persist_a_stale_namespace_rebuild monkeypatches _check_note_capacity, records the persist argument, then asserts calls == [False, True, False, True] and finally reads the resulting count. It never forces the #637 interleaving where an early rebuild snapshots N notes, a competing locked reservation advances the sidecar to N+1, and the early path resumes afterward. Because of that, an implementation could accidentally regress to _note_totals(..., persist=True) unconditionally inside _check_note_capacity while still accepting/forwarding the persist argument; this test would continue recording the same [False, True, ...] sequence and the sequential final count would still be 2, even though the original stale overwrite race had returned.

That matters because CONTRIBUTING asks regressions to pin externally observable changed behavior, and #637 already gives a deterministic interleaving recipe. Please make the test gate the early rebuild between its read and persistence, let a second create reserve/update the namespace count, then resume the early path and assert the newer sidecar is not clobbered (and ideally that the cap cannot be exceeded). That would fail for the actual old behavior and would also catch future code that merely preserves the boolean plumbing while ignoring its semantics.

@sv

sv commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Closing as a duplicate of #640 — both fix #637 by keeping the unlocked early capacity rebuild read-only, in src/store.py and the same test files.

Keeping #640 because it was opened first. Not a judgement on the work here: if this PR covers something #640 does not, say what on #640 and it can be carried across with credit.

@sv sv closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unlocked early capacity check can clobber a concurrent note-namespace reservation, exceeding MAX_NOTES_PER_NS

3 participants