Skip to content

[Large-room policy 5/5] feat(chat): keep cold rooms out of people features - #1535

Open
wesm wants to merge 5 commits into
large-room/cold-vectorsfrom
large-room/cold-people
Open

wesm wants to merge 5 commits into
large-room/cold-vectorsfrom
large-room/cold-people

Conversation

@wesm

@wesm wesm commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Cold rooms no longer shape people data: identity suggestions, enrichment, person sweeps, person facts and person vectors. Before this change, a large room's roster could still supply names, emails and provider IDs to those features.

Provenance

Participants are shared across the archive, so excluding a room requires knowing which conversation backed each fact.

  • Observations. Contact observations record their supporting conversations in participant_contact_observation_conversations.
  • Readers. Identity, people and person-vector readers accept only values backed by a full room, an independent contact or a manual decision.
  • Importers. Importers resolve participants per conversation. This touches Beeper, Slack, Teams, Discord, iMessage, Google Voice, WhatsApp, iMazing and SMS imports.
  • Tier changes. A room turning cold invalidates matches, facts, sweep inputs, enrichment and person vectors it fed. Promoting it back replays its roster. The one-time bootstrap also covers rooms that were already cold before upgrade.
  • Envelope changes. Replacing or tombstoning the only envelope behind a chat participant's email or name advances tracked people's revisions, so in-flight enrichment is rejected at dispatch and at result publication. Generated name match evidence stops counting at the same moment. Deletion staging matches the same eligible names and addresses that aggregates display. Restoring such an envelope invalidates the contexts built without it.

Merge fix

This layer also restores a step from main that the original change dropped. When two participants merge and only the absorbed one has a stable provider ID for a shared address, the survivor now inherits it. This applies to sources without room tiers. Without it, the provider ID was lost and a resolved identity conflict stayed open. #1361's CI was failing on the two merge tests that cover this.

A source merge also no longer aborts when clearing a cold provider ID makes it match a providerless full observation of the same address. The providerless row is kept and the other becomes history.

Upgrade cost

Upgrading runs data migrations and moves the schema version to 18:

  • widen the contact-observation identity index;
  • rebuild the person-fact status check constraint;
  • backfill native email handles and Apple WhatsApp names from stored payloads.

These run during schema initialization, and the payload backfills decompress raw messages, so a large archive's first open after upgrade takes noticeably longer.

Where to look

  • internal/store/participant_observations.go and internal/store/conversation_policy_{observations,people,rich_identity,sweep_input}.go.
  • The importer changes, which are mostly mechanical: a conversation ID passed into participant resolution.
  • messages.go and source_merge_transfer.go: participant merges, source merges and recipient-change invalidation.

Stack 5/5, cut from #1361. With this layer the stack matches the combined change in #1517, apart from the opt-in default, the idle-refresh skip and the help trim.

Review items deferred from this stack are tracked in #1555.

🤖 Generated with Claude Code

@wesm
wesm added this pull request to stack #1536 October 27, 2026 22:04
@wesm wesm changed the title large room/cold people [Large-room policy 5/5] feat(chat): keep cold rooms out of people features Oct 27, 2026
@roborev-ci

roborev-ci Bot commented Oct 27, 2026

Copy link
Copy Markdown

roborev: Combined Review (bc9bdf2)

Verdict: Changes require fixes for 5 findings.

High

  • internal/beeper/conversation_policy_test.go:161: The self-proof ordering regression test leaves automatic classification disabled. DefaultSettings admits the room as Full regardless of activity, so removing early self-proof publication would still pass the Full and rich-identity assertions.

    Fix: Use RecommendedSettings and establish that the room is Cold before ensureConversation. Retain the final Full and rich-identity assertions.

Medium

  • internal/store/source_merge_transfer.go:342: Clearing a Cold observation's provider_user_id can collide with an existing providerless Full observation under idx_participant_observations_identity, aborting the source merge before later deduplication runs.

    Fix: Resolve colliding observation keys before clearing provider IDs, preserving history and conversation-support edges. Add a regression covering matching scoped providerless Full and provider-bound Cold observations.

  • internal/store/conversation_policy_people.go:92: Upgrade bootstrap excludes people without participant contact observations in already-Cold Slack, Teams, or Discord rooms. Existing Cold-derived person facts and current briefs can remain available indefinitely.

    Fix: Bootstrap people associated with existing Cold rooms and archived fact evidence, including people without contact observations, before marking bootstrap complete.

  • internal/store/person_enrichment_work.go:1100: Enrichment requests can retain email or name values after their last live Full recipient envelope is replaced or tombstoned. These mutations advance neither policy nor person revisions, allowing dispatch, polling, and publication to authorize ineligible values.

    Fix: Revalidate the original request's identity values at publication boundaries, or atomically invalidate affected person revisions whenever envelope support changes.

  • internal/beeper/importer_cancellation_test.go:256: The resumed-import assertion accepts any observation, but display-name observations already exist before roster capture and cancellation. The test can pass even if resumption never records the missing phone observations.

    Fix: Assert each member's expected phone observation using observationsOfKinds or the expected phone value.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 14s | Total: 25m27s

@wesm
wesm force-pushed the large-room/cold-people branch from bc9bdf2 to 514da27 Compare October 28, 2026 15:23
@wesm
wesm force-pushed the large-room/cold-people branch from 514da27 to 118d786 Compare October 28, 2026 15:27
@roborev-ci

roborev-ci Bot commented Oct 28, 2026

Copy link
Copy Markdown

roborev: Combined Review (118d786)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/store/messages.go:2790: Replacing or tombstoning the last Full envelope supporting a participant name invalidates contextual vectors but leaves generated display_name match evidence intact. Scoring reloads that evidence without checking current support, allowing later judgments to use an unsupported name signal.

    Fix: Reconcile generated name evidence after envelope mutations, or filter it against live exact-name support when loading review and scoring snapshots. Preserve manual evidence and cover recipient-name replacement and tombstoning.

  • internal/personenrichment/worker.go:712: The enrichment request check runs outside the publication transaction. If recipient replacement removes the email's last Full support after this check, CommitEnrichmentClaims can still publish the result because recipient changes advance neither the person revision nor the policy revision checked by the transaction.

    Fix: Validate the original request's identity support within the commit transaction and serialize validation with support mutations, or make those mutations invalidate the affected person's durable revision.

  • internal/store/dedup.go:728; internal/store/imap_memberships.go:353: Restoring dedup-hidden or source-deleted messages can restore the only eligible email or name supporting a shared participant without invalidating affected contextual embeddings. Conversations rebuilt while support was absent can indefinitely retain vectors missing the restored value.

    Fix: Invalidate affected participant contexts transactionally when restoring message eligibility, including UndoDedup, IMAP membership restoration, and ClearMessageDeletedFromSourceContext. Add a withdrawal, rebuild, and restore regression test.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 18s | Total: 50m44s

@wesm
wesm force-pushed the large-room/cold-people branch from 118d786 to 45f3713 Compare October 28, 2026 18:25
@wesm
wesm force-pushed the large-room/cold-people branch 2 times, most recently from 22a0f28 to d093c94 Compare October 28, 2026 18:31
@roborev-ci

roborev-ci Bot commented Oct 28, 2026

Copy link
Copy Markdown

roborev: Combined Review (d093c94)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/personenrichment/worker.go:712: Enrichment request validation occurs outside the result transaction. Concurrent recipient replacement can remove an email’s sole Full envelope support without changing persons.revision or conversation_policy_revision, allowing CommitEnrichmentClaims to accept a stale response.

    Fix: Validate the original request’s eligible identities inside the result transaction, or atomically invalidate the person/attempt revision when envelope support changes. Add a regression covering replacement between the worker check and result commit.

  • internal/store/messages.go:1495: Moving an envelope from a Cold room to a Full room misses contextual invalidation because the pre-move helper selects only currently Full support. Other Full rooms sharing those participants retain vectors that omit the newly eligible name or address, and unchanged recipient replay does not invalidate them.

    Fix: Invalidate affected participant contexts against both the old and new conversation state within the move transaction. Cover Cold-to-Full moves with an already published contextual embedding in another room.

  • internal/store/messages.go:2797: Replacing recipients or tombstoning a message can withdraw the last Full envelope supporting a generated display_name explanation while leaving that explanation stored. Candidate reviews and newly claimed scoring packets retain the ineligible evidence, and the scoring fingerprint check accepts it because the stored evidence is unchanged.

    Fix: Reconcile generated name evidence after envelope support changes, or apply live eligibility filtering when loading review and scoring evidence. Add regression coverage for removal of the final Full envelope.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 11s | Total: 25m57s

@wesm
wesm force-pushed the large-room/cold-people branch from d093c94 to b54c6fb Compare October 28, 2026 20:52
@roborev-ci

roborev-ci Bot commented Oct 28, 2026

Copy link
Copy Markdown

roborev: Combined Review (b54c6fb)

Verdict: Changes require fixes for 6 findings.

High

  • internal/query/sqlite.go:1625: Deletion staging uses raw participant names while aggregates and message lists use eligible names. Selecting a visible sender-name group can therefore stage messages from another sender whose matching name is supported only by a Cold room, including messages outside the displayed group.

    Fix: Reuse the eligibility-aware filter builder for deletion resolution while preserving deletion-specific guards. Add a regression verifying that staged targets match the displayed aggregate.

Medium

  • internal/store/messages.go:1494: Moving a message supporting an existing fact or brief from a Full room into an already Cold room does not reconcile derived people data because neither room changes tier. Persisted attribute/employment projections and current briefs remain available indefinitely when people sweep is disabled, and semantic documents can retain those facts.

    Fix: After changing the message's conversation, reconcile affected people's fact projections and invalidate their briefs and semantic inputs in the same transaction. Add coverage with people sweep disabled.

  • internal/store/messages.go:1495: Cold-to-Full message moves invalidate contextual embeddings before updating conversation_id. The helper selects only currently Full support, so it misses the newly eligible envelope and leaves other Full rooms sharing those participants with stale embeddings. Replaying unchanged recipients also skips invalidation.

    Fix: Invalidate affected contextual messages after the move using the updated message row and newly eligible envelope support. Add a regression with an existing embedding in a third room.

  • internal/store/conversation_policy_transition.go:532: Bootstrap invalidates people but skips contextual messages when existing rooms retain their tiers. Existing Full-room vectors can retain participant text excluded by the new value-level rules while their embedding revisions remain current, including when embedding generation is disabled.

    Fix: During bootstrap, invalidate contextual messages for affected chat participants, including participants without person rows, before recording the bootstrap marker.

  • internal/store/api.go:596: Free-text FTS searches bypass participant-value eligibility checks because storedFTSEnvelopeSQL backfills sender and recipient fields from raw participant addresses. A Full message can match an address supported only by a Cold room even though address filters and displayed metadata exclude it.

    Fix: Apply current value eligibility to FTS metadata matching, such as restricting indexed matching to subject/body and matching participant metadata separately with eligibility predicates. Handle existing indexed addresses and later support withdrawal.

  • internal/store/conversation_policy_people_internal_test.go:250: TestConversationPolicyPeopleBriefInvalidation duplicates the Full-to-Cold transition and brief disappearance, historical status, and preserved-text coverage in TestPeopleWorkersColdPolicyWithdrawsPersistedCurrentBrief. Both reach the same store invalidation path, and the handcrafted brief adds no distinct failure detection.

    Fix: Delete TestConversationPolicyPeopleBriefInvalidation and retain the worker-to-store test.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 18s | Total: 34m42s

@roborev-ci

roborev-ci Bot commented Oct 28, 2026

Copy link
Copy Markdown

roborev: Combined Review (8ae2eb5)

Verdict: Changes require fixes for 4 findings.

Medium

  • internal/store/conversation_policy_observations.go:549: Beeper mention-only recipient changes with blank display names and no envelope addresses invalidate embeddings across every Full room containing the participant, causing history-wide writes and hiding current vectors without adding identity information.

    Fix: Invalidate shared contextual embeddings only when recipient changes affect eligible display names or addresses; exclude blank Beeper mention-only changes.

  • internal/store/messages.go:1494: Moving an existing envelope from a Cold room to a Full room checks support before updating its conversation, so invalidation excludes the envelope. Other Full-room embeddings can retain stale participant fallbacks while remaining marked current, and replaying unchanged recipients does not repair them.

    Fix: After moving the message, invalidate shared contextual embeddings for newly eligible envelope support within the same transaction. Add coverage for a Cold-to-Full move with an existing embedding in another room.

  • cmd/msgvault/cmd/cache_derived.go:340: Linking a chat participant to a person with a matching curated contact point changes address eligibility but advances only the identity revision. participants.parquet remains unchanged, leaving previously redacted addresses missing from cached analytics even when the cache reports itself current.

    Fix: Refresh participant projections when person bindings change, either by handling identity drift in cache refresh or advancing the participant projection revision in binding mutations.

  • internal/store/api.go:596: Free-text FTS searches bypass address eligibility checks. When a participant's only supporting room becomes Cold, storedFTSEnvelopeSQL still indexes the raw email on messages in another Full room, allowing searches for the Cold-only address to return those messages. Rebuilding FTS preserves the issue.

    Fix: Apply value eligibility to FTS address matching and invalidate or refresh affected index entries when supporting evidence disappears.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 14s | Total: 32m41s

@wesm
wesm force-pushed the large-room/cold-people branch from 8ae2eb5 to 07e134f Compare October 28, 2026 23:18
@roborev-ci

roborev-ci Bot commented Oct 28, 2026

Copy link
Copy Markdown

roborev: Combined Review (07e134f)

Verdict: Changes require fixes for 1 finding.

Medium

  • internal/store/messages.go:3618: Message deletion bypasses the serialization lock used by recipient writers. Concurrent deletion and recipient replacement can acquire message/person row locks in conflicting order, causing a PostgreSQL deadlock. The Beeper importer continues after deletion errors, potentially leaving the message live.

    Fix: Serialize envelope-invalidating writers with recipient replacements before acquiring message/person row locks, or enforce a consistent lock order. Add a PostgreSQL regression covering concurrent deletion and recipient replacement.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 9s | Total: 23m25s

@wesm
wesm force-pushed the large-room/cold-people branch from 07e134f to c364176 Compare October 29, 2026 00:40
@roborev-ci

roborev-ci Bot commented Oct 29, 2026

Copy link
Copy Markdown

roborev: Combined Review (c364176)

Verdict: Changes require fixes for 1 finding.

Medium

  • internal/store/messages.go:1494: When a message carrying a participant's only name/email envelope moves from a Cold room to a Full room, pre-update invalidation finds no support because the message is still Cold. Its newly eligible values can affect other Full rooms, but their contextual embeddings retain unchanged revisions and stale content. The downstream cleanup only handles moves out of Full rooms.

    Fix: After moving the message, invalidate contexts supported by its newly eligible envelope. Add a Cold-to-Full move regression covering a shared participant in another Full room.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 8s | Total: 25m32s

@wesm
wesm force-pushed the large-room/cold-people branch from c364176 to 3d1e35c Compare October 29, 2026 01:10
@roborev-ci

roborev-ci Bot commented Oct 29, 2026

Copy link
Copy Markdown

roborev: Combined Review (3d1e35c)

Verdict: Changes require fixes for 4 findings.

Medium

  • internal/store/conversation_policy_observations.go:559: Recipient-envelope changes invalidate vectors across Beeper rooms sharing participants without journaling the affected scopes. Ordinary ContextWorker.RunOnce calls can leave those vectors excluded from search until the periodic backstop, which defaults to 24 hours. The participant-context invalidation helper has the same gap.

    Fix: Enqueue durable rebuild work for every affected chat scope alongside invalidation, and test recovery through ordinary ContextWorker.RunOnce.

  • internal/store/conversation_policy_sweep_input.go:172: PostgreSQL sweep publication locks cited messages before locking the person, while concurrent recipient replacement can lock the person before waiting for those messages. Compatible identity locks do not prevent this deadlock, and sweeps do not acquire the recipient invalidation advisory lock.

    Fix: Acquire the recipient invalidation advisory lock before sweep evidence message locks, or enforce a consistent person/message lock order across both paths.

  • internal/store/dedup.go:750: On PostgreSQL, UndoDedup locks a restored message before invalidating shared contexts without taking the recipient invalidation lock. Concurrent recipient replacement can lock another message sharing a participant, causing a deadlock and aborting one operation. ClearMessageDeletedFromSourceContext has the same ordering gap.

    Fix: Acquire the recipient invalidation lock before updating message rows in restoration transactions, and apply that ordering consistently to other envelope writers.

  • internal/store/messages.go:1494: Moving an envelope from a Cold room into an existing Full room invalidates only before the move, when FullMessageWhere excludes it. Newly eligible recipient names or emails can therefore leave stale contextual stamps in other Full rooms. Unchanged recipients skip invalidation, and move cleanup handles only the opposite direction.

    Fix: Invalidate contexts supported by the newly eligible envelope after the move. Cover a Cold-to-Full move with unchanged recipients and a shared participant in another Full room.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 16s | Total: 33m51s

@wesm
wesm force-pushed the large-room/cold-people branch from 2bb802b to 27b2325 Compare October 29, 2026 16:34
@roborev-ci

roborev-ci Bot commented Oct 29, 2026

Copy link
Copy Markdown

roborev: Combined Review (27b2325)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/beeper/participants.go:85: For each unseen Beeper user with a phone or email in a Full room, resolveID creates a bare participant immediately before rich resolution merges it. Each unnecessary merge loads the identity graph and scans every conversation's messaging-route metadata while holding exclusive policy and identity locks, causing repeated archive-wide work during imports.

    Fix: Use adoptLegacyIdentifier before guarded rich resolution, creating a bare participant only when falling back after admission fails. Preserve the separate bare-ID path for self attribution.

  • internal/store/messages.go:1211: On PostgreSQL, Teams reimports can deadlock with a people sweep: MigrateSourceMessageID locks the destination message before requesting the recipient advisory lock, while sweep admission acquires those locks in the opposite order. MergeDuplicates has the same inversion after updating replies, forcing PostgreSQL to abort one operation.

    Fix: Acquire the recipient advisory lock at the start of both transactions, before modifying message rows. Add a concurrent migration/sweep regression test.

  • internal/store/conversation_policy_observations.go:442: Native names with surrounding whitespace lose independent support after a chat sighting because participant values retain whitespace while RecordContactObservationContext normalizes it. The resulting equality failure hides valid native names from queries, caches, and people features.

    Fix: Compare names with consistent whitespace normalization, including existing participant values, while preserving case-sensitive name authority. Cover a native name with surrounding whitespace followed by a Cold chat sighting.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 14s | Total: 26m7s

@wesm
wesm force-pushed the large-room/cold-people branch from 27b2325 to 8a09e57 Compare October 30, 2026 00:46
@roborev-ci

roborev-ci Bot commented Oct 30, 2026

Copy link
Copy Markdown

roborev: Combined Review (8a09e57)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/personenrichment/worker.go:715: When a Full recipient envelope supporting the request identity is replaced during a successful provider call, the stale-result branch calls MarkTerminal without the known result.Cost. This discards the reported charge and records zero spending for attempts without hard caps.

    Fix: Atomically record result.Cost and settle the reservation while rejecting stale claims. Add coverage for identity drift during a paid completion.

  • internal/store/conversation_policy_observations.go:443: SyncTech contact names surrounded by nonbreaking spaces are stored verbatim in participants but normalized with strings.TrimSpace in observations. SQL TRIM removes only ordinary spaces, so the comparison fails. After a tiered chat sighting, the independently supported native name disappears from cache, matching, and enrichment projections.

    Fix: Use consistent whitespace normalization or recognize matching original_value evidence. Add coverage for a native import with Unicode whitespace followed by a Cold chat sighting.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 13s | Total: 28m7s

@wesm
wesm force-pushed the large-room/cold-people branch from 8a09e57 to ef76954 Compare October 30, 2026 02:49
@roborev-ci

roborev-ci Bot commented Oct 30, 2026

Copy link
Copy Markdown

roborev: Combined Review (ef76954)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/store/imap_memberships.go:362: An IMAP delta that restores a lower-ID message before tombstoning a higher-ID message can deadlock on PostgreSQL. Restoration locks contextual message rows before acquiring the recipient-invalidation advisory lock, while concurrent Beeper persistence can acquire those locks in the opposite order. Source attribution locks do not serialize the different sources.

    Fix: Acquire lockRecipientContextInvalidationTxContext at the start of the IMAP attribution transaction, before message or contextual-row writes. Add a PostgreSQL regression test covering a mixed restore/tombstone delta concurrent with Beeper message persistence.

  • internal/beeper/observations_test.go:786: TestObservationDedupeKeyIncludesParticipantAndSource does not isolate either key component: both captures change AccountID, and the source case also changes conversation ID. The cases would pass if participantID and sourceID were removed from observationKey.

    Fix: Reuse the same account and capture context for the participant case, varying only participantID. Delete the source case because TestBeeperColdPolicyCapturesEveryRoom already covers distinct-room behavior.

  • internal/vector/sqlitevec/conversation_policy_test.go:458; internal/vector/pgvector/conversation_policy_test.go:461: TestPersonColdPolicyCleanupRetainsCurrentRevisionAfterPromotion duplicates existing current-version retention and revision-replacement coverage in both vector backends. The promoted room is unrelated to the person, and replacement completes before cleanup, so the test does not exercise a pending stale selection.

    Fix: Delete this test from both backends. Preserve TestPersonColdPolicyCleanupPreservesCurrentVersions and TestPersonBackend_PublishSearchReplaceReconcile in each backend.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 18s | Total: 36m2s

@roborev-ci

roborev-ci Bot commented Oct 30, 2026

Copy link
Copy Markdown

roborev: Combined Review (062c1d1)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/store/imap_memberships.go:362: An IMAP delta that restores a lower-ID message before tombstoning a higher-ID message can deadlock with concurrent Beeper persistence on PostgreSQL because restoration locks contextual rows before acquiring the recipient-invalidation advisory lock.

    Fix: Acquire lockRecipientContextInvalidationTxContext at the beginning of the IMAP attribution transaction, before membership, message, or contextual-row writes. Add a PostgreSQL regression test for the mixed restore/tombstone interleaving.

  • internal/store/conversation_policy_people.go:168: Transitioning a room to Cold fails while an associated person has an authorized enrichment HTTP call in progress. forceInvalidatePersonEnrichmentTx returns ErrPersonEnrichmentDispatchInProgress, rolling back manual overrides and automatic expiry transitions.

    Fix: Allow policy invalidation to complete the tier change while retaining enough attempt state to account for the response and reject its claims. Add coverage for demotion during an authorized enrichment call.

  • internal/store/person_enrichment_results.go:271: A paid enrichment response can lose its reported charge when conversation policy changes during the request without changing its identity inputs. Publication returns ErrConversationPolicyStale before cost reconciliation or attempt completion, leaving the known charge unrecorded for attempts without a hard cost cap.

    Fix: On stale-policy completion, discard the claims and terminalize the attempt through MarkTerminalWithCost to record the returned cost. Add coverage for a paid response after a policy-only revision change.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 11s | Total: 26m9s

@wesm
wesm force-pushed the large-room/cold-people branch from 062c1d1 to af397a8 Compare October 30, 2026 13:33
@roborev-ci

roborev-ci Bot commented Oct 30, 2026

Copy link
Copy Markdown

roborev: Combined Review (af397a8)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/store/person_enrichment_results.go:271; internal/personenrichment/worker.go:734: When policy changes during a paid synchronous enrichment call without changing the request hash, requestStillCurrent succeeds but the publication fence rejects the result before cost reconciliation. The attempt remains starting and recovery marks it uncertain, losing the reported charge when no hard cost cap exists.

    Fix: Discard stale claims while durably terminalizing the completed attempt with its reported cost, including ErrConversationPolicyStale from the claim sink. Cover a policy revision change during a paid call with an unchanged request hash.

  • internal/store/imap_memberships.go:350: IMAP mailbox deltas can acquire message-row locks before the recipient advisory lock, while a concurrent people sweep acquires those locks in the opposite order. This can deadlock PostgreSQL and abort sync or sweep work.

    Fix: Acquire the recipient invalidation lock at the start of the mailbox-delta transaction, before any message updates. Apply the same ordering to IMAP draft retirement.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 13s | Total: 34m53s

@roborev-ci

roborev-ci Bot commented Nov 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (4118a12)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/store/imap_memberships.go:350; internal/store/imap_memberships.go:362: PostgreSQL IMAP reconciliation can mutate or lock message rows before acquiring the recipient advisory lock. Concurrent people sweeps or recipient replacement acquire these locks in the opposite order, allowing deadlocks that abort an operation.

    Fix: Acquire lockRecipientContextInvalidationTxContext at the start of applyIMAPMailboxDeltas, before source-key retirement or any message mutations. Add PostgreSQL concurrency coverage for retirement and mixed restore/tombstone interleavings.

  • internal/store/person_enrichment_results.go:271: A policy-only revision change during a paid enrichment request can reject publication while requestStillCurrent still succeeds. The completion path then skips MarkTerminalWithCost, losing the reported charge and leaving the attempt unsettled.

    Fix: When publication rejects completed work as policy-stale, record result.Cost and terminalize the attempt through the accounting path that permits stale work, without publishing claims. Cover a charged response after a policy-only revision change.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 26s | Total: 26m30s

@wesm
wesm force-pushed the large-room/cold-people branch from 4118a12 to 33be50b Compare November 1, 2026 17:39
@roborev-ci

roborev-ci Bot commented Nov 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (33be50b)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/store/person_enrichment_results.go:271: A policy revision change during paid synchronous enrichment can leave the request hash unchanged while causing the publication fence to reject the result before cost reconciliation. The attempt remains unsettled and its reported charge is lost, undercounting spending without a hard cost cap.

    Fix: Settle policy-stale completed attempts with their reported cost and release their leases without publishing claims. Add coverage for a nonzero charge when the policy revision changes but the request remains identical.

  • internal/store/conversation_policy_observations.go:563: Removing a participant's last eligible envelope clears contextual embedding stamps across their Beeper rooms without scheduling rebuilds. Journal triggers exclude Beeper embed_gen-only changes, and a converged ContextWorker skips discovery and reconciliation, leaving vectors hidden until the backstop runs, which defaults to 24 hours and can be disabled.

    Fix: Journal the affected participants or contextual scopes in the same transaction as invalidation so normal worker runs rebuild them.

  • internal/store/messages.go:2802: Every new recipient envelope invalidates contextual history even when eligible participant values do not change. Repeated Beeper mentions with blank display names increment embedding_revision across historical messages, causing quadratic historical rewrites and unnecessary loss of embedding coverage.

    Fix: Compare eligible display and address values before and after envelope mutation. Invalidate contextual history only when those values change; blank mentions and additional support for already-supported values should preserve existing stamps.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 12s | Total: 28m13s

@wesm
wesm force-pushed the large-room/cold-people branch from 33be50b to 882b052 Compare November 1, 2026 20:32
@roborev-ci

roborev-ci Bot commented Nov 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (882b052)

Verdict: Changes require fixes for 3 findings.

Medium

  • internal/store/person_enrichment_results.go:271: A policy revision during a synchronous paid enrichment call can cause publication to reject the result before cost reconciliation, even when the request hash is unchanged. The attempt remains in 'starting', and recovery cannot recover the reported charge.

    Fix: Settle completed attempts and record their returned cost even when policy authority is stale, while withholding claim publication. Add coverage for a paid response following an unrelated policy revision.

  • internal/store/participant_observations.go:927: Superseding the last eligible name or email observation changes participant display values without invalidating contextual embeddings. Vectors containing the withdrawn value remain searchable and are not queued for regeneration.

    Fix: Call invalidateContextualParticipantMessagesTx for the affected participant within the supersession transaction. Cover withdrawal of the last supporting observation after embedding publication.

  • internal/store/imap_memberships.go:350: On PostgreSQL, removing a message's last Sent mailbox membership can retain a message-row lock before acquiring the recipient-context advisory lock. A concurrent person sweep acquires these locks in the opposite order, creating a deadlock that aborts either operation.

    Fix: Acquire the recipient-context advisory lock at the start of the IMAP delta transaction, before attribution refreshes or other message-row locks.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 10s | Total: 23m13s

wesm and others added 5 commits November 1, 2026 15:23
A cold room should not shape who msgvault thinks people are. Before this
change, a large room's roster could still supply names, emails and
provider IDs to identity suggestions, enrichment, person sweeps and
person vectors.

Participants are shared across the archive, so excluding a room means
knowing which conversation backed each fact. Contact observations now
record their supporting conversations, and identity, people and person
vector readers accept only values backed by a full room, an independent
contact or a manual decision. Importers resolve participants per
conversation for the same reason. Older archives backfill this provenance
from stored payloads: native email handles and Apple WhatsApp names.

A room turning cold invalidates the people work it fed. That covers
matches, person facts, sweep inputs, enrichment and person vectors.
Promoting a room back to full replays its roster. Merges and source moves
carry provenance with them, and recipient changes invalidate the
contextual embeddings that named those participants.

The daemon's idle-refresh check also treats an unbootstrapped people
projection as due. Upgrading runs the new data migrations above, and the
schema version moves to 17.

This layer is cut from #1361 so it can be reviewed on its own.

Follow-up changes:

- fix(people): keep provider IDs when merging participants
- test(people): opt people-layer tests into automatic classification
- fix(people): close cold-room gaps in merges, bootstrap, and enrichment
- fix(people): follow envelope support when it is withdrawn or restored
- fix(people): stage deletions from the names and addresses users see
- fix(people): give envelope writers the recipient lock order

Co-authored-by: salmonumbrella <182032677+salmonumbrella@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recipient replacement takes the recipient advisory lock before it locks
message and person rows. Sweep publication locked its cited messages
first, and undoing a dedup batch or clearing a source deletion updated
message rows before invalidating shared contexts, both without that
lock. On PostgreSQL either could deadlock with a concurrent replacement
and abort one of the writes. These paths now take the advisory lock
first. Sweeps already hold the shared identity lock, which replacement
also takes before the advisory lock, so the order stays consistent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Source-message migration and duplicate merging locked message rows
before the recipient advisory lock, while sweep admission takes them in
the other order. On PostgreSQL a Teams reimport or a dedup merge could
deadlock with a people sweep. Both now take the advisory lock first.

Name observations store trimmed values, but participant rows keep the
raw name. A native name with surrounding spaces therefore lost its
independent support once a cold chat sighting gave the participant chat
provenance, and the name vanished from queries, caches and people
features. The comparison now trims the participant value.

Beeper created a bare participant for every unseen user before rich
resolution, which then merged it into the phone or email participant.
Each merge loaded the identity graph and scanned room metadata under the
exclusive policy and identity locks. Resolution now only adopts legacy
identifiers first and lets rich resolution bind the identifier; the bare
fallback remains for when admission fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stent

Replacing the only envelope behind a generated name match hid the name,
but three things still disagreed with that.

- The review page failed. Source rows for the hidden evidence had nothing
  to attach to, so listing a page returned an error.
- Review tokens stayed the same, because fingerprints hashed stored
  evidence but not its eligibility. A token issued while the name was
  shown could still accept the match after the name was withdrawn.
  Fingerprints now also hash the IDs of hidden name evidence. They add
  this group only when it is present, so tokens for unaffected candidates
  do not change.
- A native name padded with tabs or no-break spaces lost its support after
  a Cold chat sighting, because SQL TRIM strips only ASCII spaces. The
  check now trims the raw observed value the same way. Callers bind the
  name once per occurrence, so it still appears only once in the
  predicate.

The enrichment worker also discards a completed result when the request
identity changed during the provider call. That path ended the attempt
with no actual cost, so a paid call recorded no spending on profiles
without a hard cap. The worker now passes the reported charge through
MarkTerminalWithCost, which reconciles it when it ends the attempt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Policy bootstrap initializes identity state, so link and rejection tests
must assert the operation delta rather than an absolute starting revision.
Contact-point edits also invalidate cached contact times. Seed the paging
fixture with fresh contact state after those edits so it still tests sparse
pages in both chronological orders.

Generated with Codex
Co-authored-by: Codex <noreply@openai.com>
@wesm
wesm force-pushed the large-room/cold-people branch from 882b052 to 3432bac Compare November 1, 2026 21:31
@roborev-ci

roborev-ci Bot commented Nov 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (3432bac)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/store/person_enrichment_results.go:271: When policy changes during a paid synchronous enrichment call without changing the request hash, requestStillCurrent succeeds but the publication gate returns ErrConversationPolicyStale before reconciling cost. The attempt remains starting, and recovery treats the completed call as uncertain, losing its known charge.

    Fix: Terminalize the completed attempt with MarkTerminalWithCost and record its reported cost while discarding its claims. Add coverage for a paid result whose request hash remains unchanged across a policy revision.

  • internal/store/imap_memberships.go:350: PostgreSQL IMAP reconciliation can acquire message-row locks before the recipient advisory lock. A concurrent sweep acquires those locks in the opposite order, creating a deadlock that aborts one operation; stale sweep citations can also trigger this cycle.

    Fix: Acquire lockRecipientContextInvalidationTxContext at the start of applyIMAPMailboxDeltas, before source-key retirement, attribution changes, or message updates. Add a PostgreSQL regression test for the interleaving.


Reviewers: 2x codex, codex (security), codex (test-value) | Synthesis: codex, 11s | Total: 22m31s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant