ObsChain (Backend)

Observe Bitcoin. Understand the event.

ObsChain is an open-source Bitcoin network observation, anomaly-detection, incident-analysis, and on-chain intelligence engine built in Rust.

Frontend: https://github.com/j-kon/obschain-web

Capabilities

Network Ingestion: Modular support for Bitcoin Core (RPC / ZMQ) and mempool.space (REST / WebSocket).
Anomaly Detection: Configurable detectors for large transfers, unusual block intervals, fee spikes, and structural patterns.
Incident Intelligence: Structured case management tracking incidents with verifiable evidence provenance.
Strict Evidence Standard: Uncompromising separation between cryptographic on-chain facts and heuristic claims.
High Performance: Native async Rust built on Tokio, Axum, and SQLx.

Workspace Layout

obschain/
├── crates/
│   ├── obschain-core/          # Core domain models, events, incidents, observations
│   ├── obschain-detectors/     # Detection logic (large transfers, intervals, etc.)
│   ├── obschain-ingest/        # Multi-source ingestion clients (RPC, ZMQ, REST, WS)
│   ├── obschain-incidents/     # Provenance verification & incident tracking
│   ├── obschain-intelligence/  # Graph modeling, clustering, report generation
│   └── obschain-storage/       # PostgreSQL / In-Memory repository storage
├── src/                        # API server binary (Axum)
├── migrations/                 # PostgreSQL schema migrations
└── docs/                       # Architecture, roadmaps, security, and event models

Getting Started

Prerequisites

Rust 1.85+ (stable toolchain)
Cargo
Docker & Docker Compose (optional for PostgreSQL)

Architecture & Live Sovereign Bitcoin Ingestion

Bitcoin Core (Full Node)               mempool.space (Secondary Witness)
  ├── RPC Client                          ├── REST Sync Client
  └── ZMQ Subscriber (rawtx/rawblock/seq) └── WebSocket Stream
            │                                       │
            └───────────────────┬───────────────────┘
                                │
                    ObsChain Ingestion Engine
                    (Priority & Multi-Witness)
                                │
                     Normalized Observations
            (Blocks, Transactions, Mempool, Reorgs)
                                │
                    Historical UTXO Enrichment
            (Local Cache → Core RPC → Public REST)
                                │
                        Detector Pipeline
            (8 Anomaly Detectors + Deduplicator)
                                │
                    Incident Watch Engine
             (Watch Targets, DAG Descendants)
                                │
                    Durable Storage & Stream
             (PostgreSQL / Memory + WebSocket)

Ingestion Source Priority & Multi-Witness

ObsChain implements authoritative source priority:

Bitcoin Core Validated Chain Data (Authoritative Local Source): Consensus-validated blocks and transactions received via ZMQ rawblock, rawtx, and sequence, cross-checked with JSON-RPC.
Bitcoin Core Mempool Backlog: Local node mempool state and sequence events ('A' add, 'R' remove).
mempool.space (Supplementary Witness / Fallback): Public REST and WebSocket streams act as secondary witnesses when enabled, providing cross-observer telemetry without overwriting authoritative local node data.

How Sovereign Ingestion Works

Bitcoin Core RPC & ZMQ Architecture:
- RPC Client (BitcoinCoreRpcClient): Constrained, typed JSON-RPC client supporting cookie authentication (.cookie) or credentials. Validates network identity on startup and polls chain tips and capabilities (getblockchaininfo, getnetworkinfo, getchaintips, getindexinfo).
- Authoritative ZeroMQ Multipart Protocol (v31.1 Specification): All Bitcoin Core ZeroMQ messages follow a strict 3-frame multipart structure:
```
Frame 1: Topic string ("rawtx", "rawblock", "sequence")
Frame 2: Body payload
Frame 3: 4-byte little-endian ZMQ notification sequence number (u32)
```
- Sequence Event Processing (BitcoinSequenceEvent): Bitcoin Core publishes mempool and block events on the sequence topic with exact body length validation:
  - 'C': Block Connected — 32-byte reversed block hash + 1-byte 'C' tag (total 33 bytes).
  - 'D': Block Disconnected — 32-byte reversed block hash + 1-byte 'D' tag (total 33 bytes; reorganization).
  - 'A': Transaction Added — 32-byte reversed txid + 1-byte 'A' tag + 8-byte LE mempool sequence number (total 41 bytes).
  - 'R': Transaction Removed — 32-byte reversed txid + 1-byte 'R' tag + 8-byte LE mempool sequence number (total 41 bytes; RBF replacement or mempool eviction).
- ZMQ Notification Sequence vs Mempool Sequence:
  - ZMQ Notification Sequence Number (u32, 4-byte LE in Frame 3): Per-topic sequence counter incremented by Bitcoin Core on every published message, allowing subscribers to detect network packet loss or dropped notifications independently across rawtx, rawblock, and sequence.
  - Mempool Sequence Number (u64, 8-byte LE in Frame 2 of 'A'/'R'): Node-wide transaction order sequence inside Bitcoin Core's mempool, strictly separate from transport sequence counters.
Per-Topic Notification Loss Detection & Bounded Recovery:
- Sequence Gap Tracking (ZmqSequenceTracker): Independently tracks sequence progression for each topic with seamless u32 wraparound handling (u32::MAX -> 0). Detects single-increment progression (100 -> 101) as normal and multi-step jumps (100 -> 104) as a gap with missed notification estimation (missed = 3).
- Source Health State Progression:
```
CONNECTED ──(gap detected)──> DEGRADED ──(reconcile)──> RECONCILING ──(synced)──> CONNECTED
```
- Bounded Reconciliation:
  - For block/sequence gaps: Triggers tip continuity check and reconciles missed blocks via RPC up to OBSCHAIN_RECONCILE_MAX_BLOCKS (default 100).
  - For mempool gaps: Refreshes mempool status (getmempoolinfo) and backlog tracking via RPC.
  - Sequence gaps increment telemetry counters: zmq_sequence_gaps_total and zmq_notifications_missed_estimate.
- Duplicate rawtx Lifecycle & Deduplication: Bitcoin Core publishes rawtx twice for transactions: once when entering the mempool, and again when arriving inside a confirmed block. ObsChain's EventDeduplicator ensures the second notification never produces duplicate logical anomaly events, updates confirmation state, and preserves multi-observation witness provenance.
Bitcoin Core Version Compatibility Matrix:
- v31.1 (Reference / Primary Tested): Full feature parity and protocol compliance with official doc/zmq.md v31.1 specifications, including 3-frame multipart ZMQ, 41-byte A/R events, getindexinfo txindex inspection, and little-endian sequence tracking. Native macOS/Linux reference test environment.
- v31.0: Fully compatible with the same multipart wire format and RPC interfaces.
- v28.x / v27.x: Compatible fallback for legacy or containerized environments (with -zmqpubsequence and -txindex=1). Note: Docker Hub community images (e.g. ruimarinho/bitcoin-core) currently lag behind upstream v31.x releases and pin v28.
- Older Releases (< v27.0): Best effort / not guaranteed.
RPC & ZMQ Network Exposure & High-Water Mark (HWM) Security:
- No ZMQ Authentication: Bitcoin Core's ZeroMQ implementation performs NO authentication and NO encryption. All ZMQ socket bindings (tcp://127.0.0.1:28332, 28333, 28334) and RPC bindings (127.0.0.1:18443) MUST be bound strictly to 127.0.0.1 (localhost). Never expose ZMQ ports on 0.0.0.0 or public internet interfaces.
- High-Water Marks (HWM): Unbounded queues risk memory exhaustion under high network load. Deployments should configure explicit high-water marks in bitcoin.conf:
  - -zmqpubrawtxhwm=10000 (buffers bursty mempool transaction traffic)
  - -zmqpubrawblockhwm=1000 (buffers block bursts during synchronization)
  - -zmqpubsequencehwm=10000 (buffers high-throughput mempool sequence events)
Sovereign-Only Mode (OBSCHAIN_SOVEREIGN_ONLY=true):
- Completely disables outbound network calls to mempool.space REST and WebSocket endpoints.
- UTXO enrichment resolves strictly against local cache and Bitcoin Core RPC, ensuring zero watch target or query leakage to public third parties.

Active Detectors

ReorgDetector: Emits EventType::ReorgDetected upon detecting chain reorganizations or competing tips (ReorgObservation). Evaluates reorganization depth and path, recording disconnected and connected block hashes. Deterministically assigns severity: 1 block -> Low (stale/competing block), 2 blocks -> Medium, 3-5 blocks -> High, 6+ blocks -> Critical.
DormantCoinDetector: Emits EventType::DormantCoinsMoved when Bitcoin UTXOs dormant for 5+ years are spent (OBSCHAIN_DORMANT_MIN_AGE_DAYS, default: 1,825 days, OBSCHAIN_DORMANT_MIN_VALUE_SATS, default: 1 BTC). Derives age strictly from historical confirmation context (SpentOutputContext), calculating Coin Age Destroyed in satoshi-days without floating-point overflow. Classifies outputs into Dormant, VeryOld, Ancient, or EarlyBitcoin (<2011 cutoff).
ConsolidationDetector: Emits EventType::Consolidation when transactions merge many inputs into few outputs (OBSCHAIN_CONSOLIDATION_MIN_INPUTS, default: 20 inputs into <= 5 outputs). Avoids false-positives on batch payouts or CoinJoins.
FanOutDetector: Emits EventType::FanOut when transactions distribute funds across an unusually high number of outputs (OBSCHAIN_FANOUT_MIN_OUTPUTS, default: 50 outputs).
ExtremeFeeDetector: Emits EventType::ExtremeFee when a transaction incurs extreme absolute fees (OBSCHAIN_EXTREME_FEE_SATS, default: 0.1 BTC) or extreme fee rates (OBSCHAIN_EXTREME_FEE_RATE_SAT_VB, default: 200 sat/vB).
RbfDetector: Emits EventType::TransactionReplacement upon observing confirmed transaction replacements in the mempool. Reports fee delta and percentage increase neutrally without assuming malicious intent.
LargeTransactionDetector: Emits EventType::LargeTransfer when an on-chain transaction exceeds OBSCHAIN_LARGE_TX_THRESHOLD_SATS (default: 10,000,000,000 sats / 100 BTC). Categorizes severity into Medium, High, or Critical.
LongBlockIntervalDetector: Emits EventType::LongBlockInterval when elapsed time between sequential blocks exceeds OBSCHAIN_LONG_BLOCK_INTERVAL_SECONDS (default: 1800 seconds / 30 minutes). Validates non-monotonic timestamps.

Important Boundary: Anomaly detectors evaluate transaction structure and historical confirmation. They describe observed blockchain mechanics, not wallet identity, entity attribution, or human intent.

Getting Started

Prerequisites

Rust 1.85+ (stable toolchain)
Cargo

Configuration

Copy the example environment file:

cp .env.example .env

Key environment variables:

Variable	Default	Description
`OBSCHAIN_HOST`	`127.0.0.1`	API server listen host
`OBSCHAIN_PORT`	`8080`	API server listen port
`MEMPOOL_API_URL`	`https://mempool.space/api`	Mempool.space REST base endpoint
`MEMPOOL_WS_URL`	`wss://mempool.space/api/v1/ws`	Mempool.space WebSocket stream endpoint
`OBSCHAIN_DORMANT_MIN_AGE_DAYS`	`1825`	Dormant coin detector age threshold (5 years)
`OBSCHAIN_DORMANT_MIN_VALUE_SATS`	`100000000`	Dormant coin minimum value threshold (1 BTC)
`OBSCHAIN_CONSOLIDATION_MIN_INPUTS`	`20`	Consolidation minimum inputs threshold
`OBSCHAIN_CONSOLIDATION_MAX_OUTPUTS`	`5`	Consolidation maximum outputs threshold
`OBSCHAIN_FANOUT_MIN_OUTPUTS`	`50`	Fan-out minimum outputs threshold
`OBSCHAIN_EXTREME_FEE_SATS`	`10000000`	Extreme fee absolute threshold (0.1 BTC)
`OBSCHAIN_EXTREME_FEE_RATE_SAT_VB`	`200.0`	Extreme fee rate threshold (sat/vB)
`OBSCHAIN_UTXO_CACHE_LIMIT`	`50000`	In-memory historical UTXO cache capacity
`OBSCHAIN_UTXO_CACHE_TTL_SECONDS`	`3600`	Historical UTXO cache TTL (1 hour)
`OBSCHAIN_MAX_INPUT_ENRICHMENT`	`500`	Maximum input lookups per transaction
`OBSCHAIN_UTXO_LOOKUP_CONCURRENCY`	`16`	Bounded concurrency for historical HTTP lookups
`OBSCHAIN_DEDUP_CAPACITY`	`10000`	Deterministic event deduplication capacity
`OBSCHAIN_LARGE_TX_THRESHOLD_SATS`	`10000000000`	Large transfer detector threshold (100 BTC)
`OBSCHAIN_LONG_BLOCK_INTERVAL_SECONDS`	`1800`	Block interval alert threshold (30 minutes)
`OBSCHAIN_EVENT_STORE_LIMIT`	`10000`	Maximum recent events in circular memory store
`OBSCHAIN_ACTIVITY_STORE_LIMIT`	`10000`	Maximum recent incident activities in circular memory store
`OBSCHAIN_INCIDENT_FOLLOW_DEPTH`	`3`	Maximum hop depth for tracking UTXO descendants (bounded 1-5)
`OBSCHAIN_MOCK_FEED`	`false`	Run live ingestion (`false`) or mock demo data (`true`)
`OBSCHAIN_BITCOIN_CORE_ENABLED`	`false`	Enable sovereign Bitcoin Core ingestion
`BITCOIN_RPC_URL`	`http://127.0.0.1:8332`	Bitcoin Core JSON-RPC endpoint
`BITCOIN_RPC_USER` / `BITCOIN_RPC_PASSWORD`	empty	RPC credentials (or use cookie auth)
`BITCOIN_COOKIE_FILE`	empty	Path to `.cookie` file for zero-credential authentication
`BITCOIN_ZMQ_RAWTX`	`tcp://127.0.0.1:28332`	ZMQ endpoint for raw transaction stream
`BITCOIN_ZMQ_RAWBLOCK`	`tcp://127.0.0.1:28333`	ZMQ endpoint for raw block stream
`BITCOIN_ZMQ_SEQUENCE`	`tcp://127.0.0.1:28334`	ZMQ endpoint for sequence notifications (`C`, `D`, `A`, `R`)
`OBSCHAIN_BITCOIN_NETWORK`	`bitcoin`	Network validation (`bitcoin`, `testnet`, `signet`, `regtest`)
`OBSCHAIN_PRIMARY_SOURCE`	`auto`	Primary ingestion source (`auto`, `bitcoin_core`, `mempool_space`)
`OBSCHAIN_SOVEREIGN_ONLY`	`false`	Strict sovereign mode: disables all public mempool.space calls
`OBSCHAIN_RECONCILE_MAX_BLOCKS`	`100`	Maximum chain gap blocks to replay before alerting

Running ObsChain

# Sovereign Bitcoin Core mode (Mainnet or Regtest)
OBSCHAIN_BITCOIN_CORE_ENABLED=true OBSCHAIN_SOVEREIGN_ONLY=true cargo run --bin obschain

# Development mode (mempool.space default)
cargo run --bin obschain

The server binds to http://127.0.0.1:8080 by default and immediately initiates live Bitcoin observation.

API & WebSocket Endpoints

GET /health - Service health status
GET /api/v1/status - Live network telemetry, tip height, active sources, detector metrics, and watch engine metrics
GET /api/v1/events - Paginated list of real detected on-chain anomalies
GET /api/v1/events/:id - Detailed observation payload for a specific event
GET /api/v1/incidents - Active and historical security incident dossiers
GET /api/v1/incidents/:id - Complete incident dossier (summary, status, recovery, evidence, timeline, graph) by Case ID (OC-2026-0001) or UUID
GET /api/v1/incidents/:id/timeline - Chronological incident milestones with evidence and transaction references
GET /api/v1/incidents/:id/evidence - Evidence items with strict provenance classification
GET /api/v1/incidents/:id/graph - Forensic relationship graph (nodes & typed edges) for interactive UI visualization
GET /api/v1/incidents/:id/activity - Incident-specific activity feed (filterable by activity_type, correlation_strength, min_confidence, limit)
GET /api/v1/incidents/:id/watch-targets - Public metadata for active watch targets associated with an incident (internal parameters redacted)
GET /api/v1/incident-activity - Global cross-incident activity feed
GET /api/v1/ws - ObsChain Live Stream WebSocket: Broadcasts newly detected ChainEvents, IncidentActivity, and IncidentAlert in real-time

Connecting to the Live WebSocket Feed

Connect any WebSocket client to ws://localhost:8080/api/v1/ws.

The stream broadcasts tagged JSON envelopes:

chain_event: General network anomaly event (also backwards-compatible with flat ChainEvent fields).
incident_activity: Verified correlation with a monitored security incident.
incident_alert: High-priority alert triggered by significant incident-linked movement.

const ws = new WebSocket("ws://localhost:8080/api/v1/ws");

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  if (msg.type === "incident_activity" || msg.type === "incident_alert") {
    console.log("Incident alert:", msg.type, msg.data.case_id, msg.data.title);
  } else {
    // Chain event (supports both msg.data and legacy flat properties)
    const chainEvent = msg.data || msg;
    console.log("Observed event:", chainEvent.event_type, chainEvent.title);
  }
};

Storage Architecture & PostgreSQL Persistence

ObsChain supports two interchangeable storage backends behind unified repository traits:

memory (Default): In-memory bounded circular storage using thread-safe RwLock<VecDeque> and hash maps. Ideal for fast local development, unit tests, and CI without database infrastructure.
postgres (Durable): Production-grade persistent storage powered by SQLx, connection pooling, and 17 normalized relational tables. Preserves all chain events, incident intelligence, recovery snapshots, watch targets, on-chain activities, and alerts across restarts.

Switching Storage Backend

Set via environment variables:

# In-Memory mode (default)
export OBSCHAIN_STORAGE_BACKEND=memory

# PostgreSQL mode
export OBSCHAIN_STORAGE_BACKEND=postgres
export DATABASE_URL=postgres://postgres:postgrespassword@localhost:5433/obschain
export OBSCHAIN_DB_MAX_CONNECTIONS=10
export OBSCHAIN_DB_MIN_CONNECTIONS=1
export OBSCHAIN_DB_ACQUIRE_TIMEOUT_SECONDS=5

Safety Rule: If OBSCHAIN_STORAGE_BACKEND=postgres is set but the database is unreachable or migrations fail, ObsChain fails startup immediately with a clear error. It does NOT silently fall back to in-memory mode, preventing operators from mistakenly assuming durability.

Local PostgreSQL Setup with Docker Compose

Start the PostgreSQL service in the background:

docker compose up -d postgres

The service is pre-configured on port 5433 (avoiding local standard port 5432 conflicts) with database obschain.

Running Migrations

Database migrations in migrations/ are applied automatically by the daemon on startup via SQLx embedded migrations (sqlx::migrate!). You can also execute them manually using the SQLx CLI:

cargo install sqlx-cli --no-default-features --features rustls,postgres
sqlx migrate run

Persistence Guarantees & Provenance Hardening (Phase 6A.1)

Event = What happened; Observation = How/when ObsChain learned about it: A logical Bitcoin event exists once in chain_events, but can possess multiple discrete event_observations records (e.g. observed live via ZMQ, witnessed via mempool.space, and reconstructed across replay jobs).
Conservative Merge Rule: Upserting a canonical event updates only event-intrinsic fields (title, description, metadata). It NEVER overwrites live observation provenance (observation_mode, replay_job_id, first_observed_at).
Observation Uniqueness: Observations enforce deterministic UUID v5 IDs and partial unique indexes:
- Replay: unique per (event_id, replay_job_id).
- Live: unique per (event_id, observation_mode, provider, transport).
Baseline Statistical Safety: Statistical baselines query canonical chain_events directly, never counting duplicate observations. Replaying a 4,000 BTC transaction 10 times counts exactly once in historical statistics.
Historical Recovery Preservation: Incident recovery state is append-only (incident_recovery_snapshots). Historical recovery figures are never overwritten, maintaining full audit trails.
Rule 19 (Movement != Recovery): Observed on-chain activity movements never automatically mutate incident recovery balances. Recovery balances change only via explicit verified recovery updates.
Satoshi Precision: All satoshi values are validated against signed 64-bit bounds (BIGINT) with checked Rust u64 <-> SQL i64 conversions.
Credential Protection: Database passwords and connection URIs are strictly redacted from logs, status endpoints, and panic payloads.

Local Bitcoin Core & Regtest Setup

ObsChain can operate directly against a local full node (Mainnet or Regtest) without any external third-party dependencies.

1. Using the Regtest Helper Script

A convenience management script is provided in scripts/regtest-node.sh:

# Start local bitcoind on regtest (RPC 18443, ZMQ 28332-28334, txindex=1)
./scripts/regtest-node.sh start

# Mine 101 blocks to mature coinbase rewards
./scripts/regtest-node.sh mine 101

# Inspect node status
./scripts/regtest-node.sh status

# Stop daemon when finished
./scripts/regtest-node.sh stop

2. Using Docker Compose for Bitcoin Core

Alternatively, run an isolated Bitcoin Core regtest container via docker-compose.bitcoin.yml:

docker compose -f docker-compose.bitcoin.yml up -d

3. Sovereign-Only Privacy Operation

In sovereign-only mode (OBSCHAIN_SOVEREIGN_ONLY=true), ObsChain guarantees:

Zero Third-Party Calls: Disables all outgoing connections to public mempool.space REST and WebSocket endpoints.
Privacy Preservation: Watched addresses, transaction outpoints, and queries are never leaked to external public infrastructure.
Authoritative Consensus: Validates all incoming blocks and transactions directly against your local node's consensus rules.

4. Node Capability & Pruning Behavior

txindex: Recommended (txindex=1). If disabled, historical UTXO lookups for dormant coin enrichment fall back to configured archival sources or fail gracefully without panicking.
Pruned Nodes: Detected on startup via pruned: true in getblockchaininfo. Missing historical transactions in pruned blocks are handled gracefully as non-fatal lookups.
IBD (Initial Block Download): If the node is synchronizing (initialblockdownload: true), ObsChain status displays syncing with progress percentage and will not report live until catchup is complete.

Historical Replay & Bitcoin Research Engine (Phase 6A)

ObsChain can reprocess historical Bitcoin blocks through the identical normalization, detector, persistence, and intelligence architecture used for live observations.

CLI Historical Replay

Replay ranges of confirmed blocks directly via the obschain CLI:

# Replay Bitcoin blocks 900000 through 900050
cargo run --bin obschain -- replay --start 900000 --end 900050

# Replay with custom batching and checkpoints
cargo run --bin obschain -- replay --start 800000 --end 800500 --batch-size 20 --checkpoint-interval 50

Research REST API & Filtering

Historical events are queryable via enriched research filter parameters on GET /api/v1/events:

# Query large transactions generated during replay between blocks 900000 and 901000
curl "http://localhost:8080/api/v1/events?from_height=900000&to_height=901000&observation_mode=historical_replay&event_type=large_transaction&limit=50"

Replay Job REST API (Optional)

Replay jobs can also be initiated and monitored over HTTP when OBSCHAIN_REPLAY_API_ENABLED=true:

POST /api/v1/replay/jobs: Launch a new background replay job.
GET /api/v1/replay/jobs: List active and historical replay jobs.
GET /api/v1/replay/jobs/:id: Inspect real-time job progress, metrics, and checkpoints.
POST /api/v1/replay/jobs/:id/cancel: Gracefully cancel a running replay job.
POST /api/v1/replay/jobs/:id/pause: Pause replay at current block boundary.
POST /api/v1/replay/jobs/:id/resume: Resume a paused replay job.

Replay Configuration

OBSCHAIN_REPLAY_BATCH_SIZE=10              # Blocks fetched per batch (default: 10)
OBSCHAIN_REPLAY_CONCURRENCY=2              # Maximum concurrent block fetch tasks (default: 2)
OBSCHAIN_REPLAY_CHECKPOINT_INTERVAL=25     # Blocks between persistent checkpoints (default: 25)
OBSCHAIN_REPLAY_MAX_RANGE=100000           # Maximum allowed block range per job (default: 100000)
OBSCHAIN_REPLAY_TX_CACHE_LIMIT=100000      # Bounded previous-transaction cache limit (default: 100000)
OBSCHAIN_REPLAY_DB_CONCURRENCY=2           # Dedicated PostgreSQL replay connection pool (default: 2)
OBSCHAIN_REPLAY_API_ENABLED=false          # Controls HTTP POST /api/v1/replay/jobs (default: false)

See docs/HISTORICAL_REPLAY.md for full architecture, detector availability matrix, and time-semantics specifications.

Canonical Events & Observation Lifecycle (Phase 6A.1 & 6A.2)

ObsChain enforces a strict architectural boundary between logical on-chain events and observation provenance:

Event ≠ Observation: ChainEvent models an immutable, logical Bitcoin event (what occurred on the Bitcoin network). EventObservation models an occurrence or witness of that event (how, when, and in what lifecycle state ObsChain learned about it).
Source ≠ Observation Lifecycle: A single source (e.g. Bitcoin Core ZMQ) can observe the same canonical event across multiple lifecycle transitions (MEMPOOL_SEEN -> CONFIRMED -> REORGED_OUT -> MEMPOOL_SEEN).
One Canonical Event, Multiple Observations: Lifecycle transitions and secondary witnesses (e.g. mempool.space WebSocket) append distinct observation records without creating duplicate ChainEvent entries or overwriting initial discovery provenance.
Statistical Baseline Invariant: Statistical baselines and rarity percentiles evaluate COUNT(chain_events) — never COUNT(event_observations). Provenance and lifecycle observations describe observer telemetry, not additional Bitcoin events.

See docs/ARCHITECTURE.md and docs/EVENT_MODEL.md for complete details.

Historical Baselines, Rarity & Impact Intelligence (Phase 6B & 6B.1)

ObsChain provides sovereign, verifiable empirical statistical context for Bitcoin chain events without relying on arbitrary hype scores or black-box machine learning models:

Exact Empirical CDF vs Quantile Interpolation: Distinctly separates exact empirical ranks calculated against raw metric populations (PercentileMethod::ExactEmpiricalCdf, estimated: false) from fast discrete quantile interpolation (PercentileMethod::QuantileInterpolationEstimate, estimated: true).
Normalized Canonical Metric Storage (event_metric_values): Deduplicated canonical metric rows keyed on (event_id, metric, metric_definition_version). Replay witness duplicates never distort sample frequency.
Event-Type-Specific Impact Models: Bounded impact models for each event type whose component weights sum to exactly 100.0 by construction, eliminating the saturation flaws and structural clamping of universal models.
PostgreSQL Numeric Precision (NUMERIC(50, 4)): Upgraded schema safely storing all 39 decimal integer digits of u128::MAX and Coin Age Destroyed satoshi-days without floating-point conversion or loss of precision. Mean is stored as NUMERIC(50, 4).
Strict Canonical Population Invariant: Statistical populations count chain_events exclusively. Multiple observation occurrences across replays never distort frequency distributions.
Data Quality & Coverage Accounting: Transparently reports sample size, candidate count, missing UTXO count, and metric coverage percentage (HIGH, MODERATE, DEGRADED, INSUFFICIENT).
Sample-Size Safeguards: Automatically returns INSUFFICIENT_DATA and withholds percentiles if sample count is below threshold ($N < 100$), avoiding spurious tail claims.
Historical Leakage Distinction: Explicitly flags evaluations as RETROSPECTIVE (baseline covers blocks after the event) vs POINT_IN_TIME (baseline only covers blocks before the event).

Baseline CLI Commands

# Generate a baseline for Mainnet blocks 840,000 to 850,000
cargo run --bin obschain -- baseline --start 840000 --end 850000 --network mainnet --algorithm-version obschain-baseline-v2

# Evaluate the historical rarity and impact of a canonical event
cargo run --bin obschain -- rarity --event-id <EVENT_UUID>

Research REST API

GET /api/v1/research/baselines: List completed historical baseline runs.
GET /api/v1/research/baselines/:id: Inspect baseline run metadata and distribution parameters.
GET /api/v1/research/distributions: Query specific metric distributions across runs.
POST /api/v1/research/baselines: Administratively trigger baseline computation (disabled by default; requires OBSCHAIN_BASELINE_API_ENABLED=true).
GET /api/v1/events/:id/rarity: Query comprehensive rarity results with exact empirical or interpolated method, tail counts, frequency descriptions, and event-specific impact models.
GET /api/v1/events/:id: Enriched with fast, non-blocking baseline rarity summary.

See docs/HISTORICAL_BASELINES.md for full statistical architecture and quantile specifications.

Testing & Quality

Run full workspace checks:

cargo fmt --check
cargo check --workspace
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace
cargo audit

Authoritative Test Manifest

To generate an authoritative tabular report of test suites and counts across the workspace:

./scripts/test_manifest.sh

See docs/TEST_MANIFEST.md for the complete baseline inventory and historical test audit reconciliation.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

Name		Name	Last commit message	Last commit date
.cargo		.cargo
.github/workflows		.github/workflows
crates		crates
docs		docs
migrations		migrations
scripts		scripts
src		src
tests		tests
.env.example		.env.example
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
docker-compose.bitcoin.yml		docker-compose.bitcoin.yml
docker-compose.yml		docker-compose.yml
rust-toolchain.toml		rust-toolchain.toml

Latest commit

History

Folders and files

Repository files navigation

ObsChain (Backend)

Capabilities

Workspace Layout

Getting Started

Prerequisites

Architecture & Live Sovereign Bitcoin Ingestion

Ingestion Source Priority & Multi-Witness

How Sovereign Ingestion Works

Active Detectors

Getting Started

Prerequisites

Configuration

Running ObsChain

API & WebSocket Endpoints

Connecting to the Live WebSocket Feed

Storage Architecture & PostgreSQL Persistence

Switching Storage Backend

Local PostgreSQL Setup with Docker Compose

Running Migrations

Persistence Guarantees & Provenance Hardening (Phase 6A.1)

Local Bitcoin Core & Regtest Setup

1. Using the Regtest Helper Script

2. Using Docker Compose for Bitcoin Core

3. Sovereign-Only Privacy Operation

4. Node Capability & Pruning Behavior

Historical Replay & Bitcoin Research Engine (Phase 6A)

CLI Historical Replay

Research REST API & Filtering

Replay Job REST API (Optional)

Replay Configuration

Canonical Events & Observation Lifecycle (Phase 6A.1 & 6A.2)

Historical Baselines, Rarity & Impact Intelligence (Phase 6B & 6B.1)

Baseline CLI Commands

Research REST API

Testing & Quality

Authoritative Test Manifest

License

About

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages