Repository navigation
Conversation
## Context Catalog events reached ClickHouse but never the charged-in-advance topic, so contracts with rate cards billed in advance got no fee on organizations using the ClickHouse event store. The processor did not know which contract served an event nor its rate cards. ## Description The catalog consumer resolves the contract serving each event, from the database or the memory cache, ranking a live contract above a terminated one and an active one above a pending one, and only matching its exact external id. When one of the contract's rate cards bills the event's metric in advance on a metered product, and the API did not already post-process the event, the event goes to the charged-in-advance topic in the legacy enriched shape: the API job finds the contract through external_subscription_id, where ingestion stores external_contract_id. With the memory cache, contracts, contract rate cards, rate cards and products are loaded from a snapshot and their Debezium topics when the catalog topic is configured; the Debezium connector config captures them. Refresh flags for contracts are left for the API side that consumes them.
## Context Contract rate cards, rate cards and products reach the memory cache through separate Debezium topics. When a contract card was cached before its rate card or product, the event was judged as not billed in advance and its fee was silently dropped. ## Description A contract card whose rate card or product is not cached yet now makes the advance lookup fail as retryable, so the event is processed again once the cache catches up. This is only lag: the API refuses to delete a rate card or a product that a contract card uses. The failure is not reported to Sentry, and an event still failing after the retry window goes to the dead letter queue.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Catalog events reach ClickHouse but never the charged-in-advance topic, so contracts with rate cards billed in advance get no fee for organizations on the ClickHouse event store.
Description
The catalog consumer resolves the contract serving each event (database or memory cache): a live contract wins over a terminated one, an active one over a pending one, and only an exact
external_idmatches. When one of the contract's rate cards bills the event's metric in advance on a metered product, and the API did not already post-process the event, the event goes to the charged-in-advance topic in the legacy enriched shape, withexternal_subscription_idset to the contract's external id, which is what the API job reads.With the memory cache,
contracts,contract_rate_cards,rate_cardsandproductsare loaded from a snapshot and their Debezium topics when the catalog topic is configured;extra/debezium_config.jsoncaptures them, and deployed connectors need the same tables and columns.Not included: refresh flags for contracts, which ship with the API consumer. Advance fees are only correct once the API reads
catalog_events_enriched, since the advance aggregation reads the event store.