spaces alpha

[flo-bit]

No description provided.

[github-actions]

Native SQLite benchmark

Metric	Result
Total time	134.75 s
Backfill time	134.59 s
Indexed records	61,258
Accepted records/s	455.16
Indexed records/s	454.60
Complete accounts	1,702
Retrying accounts	53

Runtime: Node v22.14.0, SQLite 3.47.2

Artifact source: calendar-records-only-sqlite-r100-h20-d3-2026-08-27T06-21-56-371Z.json

View workflow run

[tompscanlan]

I spent some time testing PR #95 at head 0130fa7, reading it against the reference server and driving the sync engine against a live PDS.

Notes:

1. A create followed by an update before the engine checkpointed the create caused incremental sync to fall back to a full CAR download. I separately reproduced create then delete. As a control, a plain update after an earlier checkpoint stayed incremental. In both reproductions, recovery restored the projection, but it downloaded the full repo. The PDS intentionally omits the value from a stale operation that a later operation supersedes, while the engine treats any CID-bearing operation without a value as an error. Suggested change: continue applying every operation to RepoCommit, but omit confirmed superseded operations from projection instead of forcing recovery.

2. maxAtomicMutations is also used as the full-recovery chunk size. At the default of ten, more than ten configured-collection operations since the checkpoint forces full recovery. Recovery then serially awaits ingestRecords in chunks of up to ten in-scope records. A recovery containing 3,000 such records therefore makes 300 serial ingest batches in one syncRepo call. The worker factory accepts values through 50. Would it be clearer to expose separate incremental-limit and recovery-chunk settings? The ScheduledIngestBudget in scheduled ingestion bounds #94 looks like the right shape, one named field per bound rather than one number doing several jobs.

3. Incremental sync reads operations from every collection in the writer repo while projecting only the collections configured in spaceTypes. I configured events but not RSVPs, wrote 25 RSVP records, and reconciled once. All 25 operations were returned, none were projected, the default ten-event cap did not fire, and the checkpoint advanced. As a control, 12 event writes hit the cap and triggered full recovery. The operation metadata is needed for whole-repo commit verification, but values from excluded collections are fetched and unused, and those operations do not count toward the cap.

   incrementalRepo also receives no reconciliation deadline, so the scan can run past that deadline. Suggested change: check the deadline between pages. If the deadline is intended as a hard wall, the remaining time would also need to reach the request as an abort signal or timeout.