Skip to content

fix(opensearch): ledger _mget must not resolve its index by CLR-type inference - #121

Merged
bfarmer67 merged 1 commit into
mainfrom
devs/bfarmer/opensearch-ledger-mget-index-inference
Aug 26, 2026
Merged

bfarmer67 merged 1 commit into
mainfrom
devs/bfarmer/opensearch-ledger-mget-index-inference

Conversation

@bfarmer67

Copy link
Copy Markdown
Contributor

Summary

IntersectWithAppliedAsync issues a single _mget. An _mget carries an index in two places — the URL, and each entry in the request body. We set the URL one explicitly from OpenSearchMigrationOptions.LedgerIndex and left the body entries to default, which resolves via IndexName.From<OpenSearchMigrationRecord>() → ConnectionSettings.DefaultMappingFor<T>() / DefaultIndex().

Neither AddOpenSearchClient nor AddOpenSearchAwsClient configures either. So request serialization threw before a byte reached the wire:

UnexpectedOpenSearchClientException: Index name is null for the given type and
no default index is set. Map an index name using ConnectionSettings.DefaultMappingFor<TDocument>()
or set a default index using ConnectionSettings.DefaultIndex().

  at OpenSearch.Client.IndexNameResolver.Resolve(Type type)
  at OpenSearch.Client.MultiGetRequestFormatter.Serialize(...)

MigrationRunner.RunAsync calls IntersectWithAppliedAsync unconditionally whenever at least one migration is discovered. This broke every OpenSearch run on 3.0.0 and 3.1.0, including through our own Hyperbee.MigrationRunner.OpenSearch and the CLI (StartupExtensions calls AddOpenSearchClient(config)).

Two things worth flagging about scope:

  • It is not AWS-specific. Both shipped client factories build a bare ConnectionSettings. The .Aws package is where it happened to surface first, not where it lives.
  • It is exactly one call site. Every other ledger operation already passes .Index(...) explicitly. The generalized probe test below confirms that.

The fix

.GetMany<OpenSearchMigrationRecord>( ids, ( op, _ ) => op.Index( _options.LedgerIndex ) )

No API change, no consumer action, patch-level. Consumers who worked around this with their own DefaultMappingFor<OpenSearchMigrationRecord> or a forked client factory can delete both.

I deliberately did not fix this by configuring DefaultMappingFor inside AddOpenSearchClient. The ledger index is runtime configuration on OpenSearchMigrationOptions, which the client factory cannot see — AddOpenSearchClient runs independently of, and usually before, AddOpenSearchMigrations. That route creates a registration-order dependency and still fails for anyone registering their own IOpenSearchClient. Rationale recorded in ADR-0029.

Why the suite missed it

Four gaps had to line up:

  1. OpenSearchRecordStoreTests substitutes IOpenSearchClient. A substitute never serializes, so no inference failure is reachable. That file also only covers ctor lock-tuning — IntersectWithAppliedAsync had zero unit coverage.
  2. RecordStoreContractTests / ReconciliationTests substitute IMigrationRecordStore — they pin interface semantics, not wire behavior.
  3. No test builds the client the way the library builds it. OpenSearchTestContainer hand-rolls its own ConnectionSettings; AddOpenSearchClient was tested only for registration guards (AWS-endpoint rejection, mutual exclusion), never for whether the client it returns can run a ledger operation.
  4. The tier that could have caught it does not run — integration tests are #if INTEGRATIONS-gated, all [TestCategory("LocalOnly")], and the matrix is off (if: vars.RUN_HEAVY_INTEGRATION == 'true', disabled in 7ff3808, which landed immediately before 3.1.0). Release runs unit tests only. Even enabled, OpenSearchRecordStoreIntegrationTests never called the method.

The structural miss: nothing existed between "mock the client" and "spin up Docker." That is precisely where serialization and inference bugs live.

Tests

New tier — OpenSearchLedgerWireTests. Real client, real descriptors, real serializer, over an InMemoryConnection. Only the socket is faked. Runs in ~60ms with no container, so it runs in the CI tier that actually executes.

  • 4 targeted tests: no-throw on a stock client, URL/query/body shape, found-vs-not-found filtering, empty-candidate short-circuit.
  • 1 generalized probe: drives all six ledger operations and asserts none depends on type→index inference. This guards the next regression, not just this one.

Integration. Two tests for IntersectWithAppliedAsync — one against a real cluster with realtime (no-refresh) semantics, one driving the client from services.AddOpenSearchClient(...), which closes gap 3.

Verified red → green:

before after
OpenSearchLedgerWireTests (5) 4 fail 5 pass
new integration tests (2) 2 fail 2 pass
OpenSearchRecordStoreIntegrationTests (8, real container) — 8 pass
full unit suite 433 pass 433 pass

Follow-ups

Two related PRs follow, both building on ADR-0029:

  • MongoDB IntersectWithSquashedAsync mixes a typed Kind term with a literal "replaces" term — the filter can never match, so squash reconciliation silently returns empty and squashed migrations re-run. Live today. Plus the latent Couchbase equivalent.
  • The ConnectionSettings escape hatch on both client factories. Worth having on its own merits, but not the remedy for this bug.

🤖 Generated with Claude Code

…inference

IntersectWithAppliedAsync issues one _mget, which carries an index in the
URL and in each body entry. The URL one was set explicitly from
OpenSearchMigrationOptions.LedgerIndex; the body entries were left to
resolve via IndexName.From<OpenSearchMigrationRecord>(), which reads
ConnectionSettings.DefaultMappingFor<T>() / DefaultIndex().

Neither AddOpenSearchClient nor AddOpenSearchAwsClient configures either,
so request serialization threw

    Index name is null for the given type and no default index is set

before any byte reached the wire. MigrationRunner.RunAsync calls
IntersectWithAppliedAsync unconditionally whenever at least one migration
is discovered, so this broke every OpenSearch run on 3.0.0 and 3.1.0 --
including through the library's own runner and CLI. Not AWS-specific.

Set the ledger index per operation. No API change, no consumer action.

Adds the wire-shape test tier that was missing: real client, real
serializer, faked socket (InMemoryConnection). The existing provider unit
tests substitute IOpenSearchClient, which never serializes, and the
container-backed tests are compile-gated behind #if INTEGRATIONS plus
[TestCategory("LocalOnly")] and excluded from the CI matrix -- so nothing
in the suite could see this class of defect. A generalized probe asserts
that no ledger operation depends on type->index inference, guarding the
next regression rather than only this one.

Two integration tests cover IntersectWithAppliedAsync against a real
cluster; one drives the client built by services.AddOpenSearchClient(...),
which nothing previously exercised end-to-end.

Records the invariant as ADR-0029.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bfarmer67
bfarmer67 force-pushed the devs/bfarmer/opensearch-ledger-mget-index-inference branch from f0517eb to 65e473a Compare August 26, 2026 17:31
@bfarmer67
bfarmer67 merged commit a5283d3 into main Aug 26, 2026
10 checks passed
@bfarmer67
bfarmer67 deleted the devs/bfarmer/opensearch-ledger-mget-index-inference branch August 29, 2026 03:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant