fix(subscriptions): bound chain evaluator tip by per-trigger decoder progress

[Rapha-btc]

Symptom

Chain print_event subscriptions silently skipped blocks: no outbox/delivery row was ever created (not a webhook failure). Example on our OSS deployment: a sub on *fakfun-market-registry* got the print in block 8864250 but nothing for 8864267 / 8864295, where the same contract emitted the same kind of print. A sweep over 3 days found 283 missed prints across 13 subscriptions.

Root cause

The chain evaluator's block-source tip tracks block ingestion, which runs ahead of decode. Each event type is decoded by an independent, differently paced decoder; print (arbitrary Clarity payloads) routinely trails ft_transfer by 1-15 s. When the evaluator processed a height before the print decoder had committed it, the print was not yet in decoded_events, the match was missed, and the forward-only cursor never revisited that height. Blocks with no ft_transfer were unaffected, which made the misses look random.

Observed on the missed heights (decoded_events timestamps): 8864267 ft_transfer 02:19:40.74, print 02:19:41.66; 8864295 ft_transfer 02:25:31.48, print 02:25:46.96.

Fix

Bound the evaluator tip by the minimum decoder_checkpoints height over only the decoders feeding the event types its active subscriptions read (derived from referencedEventTypes, decoder name decode.<event_type>.v1). A stalled decoder that no subscription reads (e.g. an idle pox4) is excluded and can never gate deliveries. A null floor (no referenced decoders / none checkpointed) falls back to the raw tip rather than stalling.

We first tried a global min over all decoders in getIndexTip; that pinned the whole tip on a stalled pox4 decoder within minutes, hence the per-trigger design.

• trigger-evaluator.ts: referencedDecoderNames, lowestDecoderHeight, decoderFloorHeight
• trigger-evaluator-loop.ts: tip = min(rawTip, floor)
• trigger-evaluator-decoder-floor.test.ts: pure unit test reproducing the race (lagging print lowers the floor below ft_transfer; a far-behind unreferenced decoder is ignored)

Verified live on our multi-service OSS deployment: after deploying, the evaluator cursor tracks the print decoder checkpoint exactly, and a cursor rewind replayed all 283 misses (idempotent via (subscription_id, dedup_key)), post-sweep misses = 0.

Follow-up (not blocking)

A cursor rewind re-emits for every active subscription regardless of created_at, so a replay can back-deliver pre-creation events to recently created subs (58 in our case; harmless for idempotent receivers). Bounding emits by subscription.created_at, or storing a per-sub start block, would make rewinds exact.

Context: follows up on #47 (safe observer defaults). Deployed and validated on the fak.fun / STX Juice node.

[vercel]

@Rapha-btc is attempting to deploy a commit to the root Team on Vercel.

A member of the Team first needs to authorize it.

[ryanwaits]

thanks for this @Rapha-btc - i didn't take the patch as is (a missing decoder checkpoint fell back to the raw tip, which is the original miss), but the chain evaluator now bounds its tip by the slowest referenced decoder checkpoint, so a lagging print decoder cannot skip those heights.

what you can try to do now if you haven't repaired:

1. pull latest main and rebuild/redeploy
2. replay each print sub over the missed range

secondlayer subscriptions replay <name> --from-block 8864250 --to-block <last-miss> -y

it caps at 100k blocks so you can chunk if the window is larger