Skip to content

Make retention delete fast and show progress - #108

Merged
adn8naiagent merged 1 commit into
stagingfrom
dev
Aug 30, 2026
Merged

adn8naiagent merged 1 commit into
stagingfrom
dev

Conversation

@adn8naiagent

Copy link
Copy Markdown
Owner

A first sweep over a large backlog appeared to hang: delete_session made about five sequential R2 round trips per session, so deleting 134 sessions meant ~670 calls held open behind one request with nothing on screen. Measured against the bucket, 100 objects one at a time took 24.5s; the same 100 in bulk take 0.9s.

  • storage.keys_under lists once from the shared root; delete_keys removes exactly those keys in batches of 1000, reporting progress per batch
  • sweep uses them instead of per-session deletes, and skips tombstones: they exist to stop auto_precompute re-downloading a removed session, and it only looks at the last 7 days, which retention never touches
  • the sweep now runs in the background via start_sweep, with GET /api/storage/retention/status for progress, so the request no longer blocks and a refresh mid-delete doesn't lose it
  • the panel polls that status and shows a progress bar over files deleted
  • the retention controls stay hidden until usage has loaded, so a threshold can't be set against a total that hasn't arrived

A first sweep over a large backlog appeared to hang: delete_session made
about five sequential R2 round trips per session, so deleting 134 sessions
meant ~670 calls held open behind one request with nothing on screen.
Measured against the bucket, 100 objects one at a time took 24.5s; the same
100 in bulk take 0.9s.

- storage.keys_under lists once from the shared root; delete_keys removes
  exactly those keys in batches of 1000, reporting progress per batch
- sweep uses them instead of per-session deletes, and skips tombstones:
  they exist to stop auto_precompute re-downloading a removed session, and
  it only looks at the last 7 days, which retention never touches
- the sweep now runs in the background via start_sweep, with
  GET /api/storage/retention/status for progress, so the request no longer
  blocks and a refresh mid-delete doesn't lose it
- the panel polls that status and shows a progress bar over files deleted
- the retention controls stay hidden until usage has loaded, so a threshold
  can't be set against a total that hasn't arrived

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@adn8naiagent
adn8naiagent merged commit 3604b1b into staging Aug 30, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant