A discrete-event simulator for an orbit-aware decentralised Delivery Service (DS) for the Messaging Layer Security protocol (RFC 9420), targeting LEO satellite constellations with intermittent connectivity.
MLS is treated as a black box throughout: the contribution is entirely at the DS layer, and no key material is handled anywhere in this code.
See docs/design.md for the design and docs/evaluation.md for the measured results and their limitations.
| Rust | 1.75+ (edition 2021) |
| Python | 3.9+ |
| Python packages | see requirements.txt |
git clone https://github.com/theodoratrz/catchup-simulator
cd catchup-simulator
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cargo build --releaseKeep the virtual environment activated while running experiments: the shell scripts call python3 for trace generation and plotting.
To reproduce the entire evaluation from a clean tree:
python3 scripts/demo.pyThis deletes data/results, data/traces and figures, then runs all eight experiment groups through the run_*.sh scripts, renders the figures, and finishes with the test suite. Raw output goes to demo-run.log; the terminal shows a readable summary. Takes roughly five minutes and produces 116 result files.
To run one experiment on its own:
bash scripts/run_e1.shEach run_*.sh generates its own traces, runs every configuration of that experiment, and calls the relevant plot scripts. run_e1.sh covers E1 and E1t; run_e5.sh covers E5 and E5t.
cargo run --release --bin sim -- <trace> <experiment> <output> <param> <seed> <mode> [latency_ms]
| Argument | Values |
|---|---|
trace |
a contact trace under data/traces/ |
experiment |
e1 e1t e2 e3 e4 e5 hub concurrent |
output |
path for the result JSON |
param |
experiment-specific: drop probability (e1), per-hop latency in ms (e2), traffic rate (e1t, e5, hub) |
seed |
RNG seed (all published runs use 42) |
mode |
short normal off fullresync centralised; for e4, naive or aware |
latency_ms |
optional 7th argument overriding the experiment's per-hop latency |
e3 and concurrent take no meaningful param. The e4 experiment sets its own delivery mode, so its 6th argument selects the selector rather than the mode.
Reconciliation modes:
| mode | RTT | mechanism |
|---|---|---|
short |
0 | blind-push the ancestor differential from the peer's last-known head |
normal |
1 | compare Merkle roots, then leaf hashes, then transfer the set difference |
off |
2 | identical code path to normal, charged two round trips; the no-pre-computation ablation |
fullresync |
0 | naive baseline: retransmit the entire retained state on every contact |
centralised |
0 | push the ancestor differential with no negotiation and no per-hop delay |
Example:
cargo run --release --bin sim -- \
data/traces/e1_missed16.json e1 out.json 0.0 42 normalCargo.toml
src/
lib.rs crate root; re-exports the modules below
main.rs placeholder binary; the real entry point is bin/sim.rs
bin/
sim.rs experiment runner
dag/ Event, hashing, tips, topological ordering, ancestor differential
merkle/ binary Merkle tree over the retained event set
quorum/ majority rule, vote and reject counting
node/ per-node protocol state, voting, commit ordering, snapshots
sync/ per-peer state: last_known_head, sync_complete, last_sync_epoch
snapshot/ Snapshot and PendingProposal types
stream/ dual-stream separation (global evidence / local private)
schedule/
mod.rs contact schedule and per-peer schedule entries
planner.rs critical_path_to_quorum - Dijkstra forward pass over the trace
mls_interface/ the DS -> MLS boundary; MlsMock is the only mocked component
simulator/
harness.rs the window loop, metrics, DS-contract checks
hub.rs HubSimulator - the centralised baseline (E5)
contact.rs ContactWindow model and trace loading
ablation.rs SimConfig, SimMode, ExperimentGoal
metrics.rs ExperimentMetrics, ContractCheck, agreement_holds()
docs/
design.md the design, as implemented
evaluation.md the measured results and their limitations
protocol-diagrams.md Mermaid sources for the dissertation figures
scripts/
demo.py end-to-end run (see Quick start)
run_*.sh one script per experiment group: traces -> runs -> plots
gen_*_trace.py synthetic contact trace generators
plot_*.py figure rendering
plot_common.py shared plotting helpers
select_sats.py satellite selection from TLE data; not on any run path, needs skyfield
tests/
integration.rs 10 integration tests
data/
traces/ generated contact traces
results/ experiment output (JSON)
starlink-100626.tle TLE snapshot; provenance only, not propagated at runtime
figures/ rendered PNGs
| Question | Independent variable | |
|---|---|---|
| E1 | Does checking before sending pay for itself? | missed windows in {1,2,4,8,16} |
| E1t | The same, with background message traffic | missed windows, 5 msgs/generator/slot |
| E2 | How short can a contact window get before a mode fails? | window duration, 1 s - 300 s at 1 s/hop |
| E3 | Does the contact topology decide the outcome? | topology x group size (n in {6,10,14}) |
| E4 | Does orbit-aware peer selection pay off under a terminal budget? | naive vs orbit-aware selector, k = 2 |
| E5 | What does removing the central coordinator cost? | hub-pass interval H in {1,2,4,8,16} |
| E5t | The same under identical offered load in both arms | H, 5 msgs/generator/period |
| concurrent | Do the guarantees survive concurrency and partition? | connected (n in {3,5,7}) vs partitioned (n = 5) |
Across all 116 runs the three safety properties (Agreement, Integrity, Membership Soundness) hold 116/116. Termination fails in exactly five, all starved of contact: the permanent partition, and four E2 configurations where the window is too short for that mode to complete an exchange.
E1 - bytes exchanged to catch up after 16 missed windows:
| strategy | bytes |
|---|---|
| full resync | 91,148 |
| blind push (0 RTT) | 17,648 |
| verified differential (1 RTT) | 4,520 |
E2 - normal requires a 2000 ms window and off 4000 ms at 1 s per-hop latency, each floor equal to that mode's negotiation cost. short completes at every duration tested, down to the 1000 ms minimum of the sweep.
E3 - time to last commit at n=14: bridge 38 orbits, ring 25, star 20, mesh 10. The star arm is a hub-shaped contact graph running the decentralised protocol, not a centralised architecture; that comparison is E5.
E4 - time to quorum: 6 orbits with a recency-ordered selector, 1 orbit with orbit-aware ranking.
E5 - convergence against hub availability gap H:
| H | centralised hub | decentralised |
|---|---|---|
| 1 | 1 orbit | 1 orbit |
| 2 | 2 | 1 |
| 4 | 4 | 1 |
| 8 | 8 | 1 |
| 16 | 16 | 1 |
On a single membership operation the decentralised arm costs a flat 6.4x the hub's bytes; under matched message load (E5t) that narrows to 1.60x at H=16.
concurrent - with every member proposing at once, all proposals commit across consecutive epochs in owner order. Behind a partition that never heals, the majority side advances to epoch 5 while the minority holds at epoch 4: one epoch behind, never forked.
cargo test --release24 unit tests and 10 integration tests. The Agreement invariant is checked after every window via agreement_holds() in src/simulator/metrics.rs; two mutation tests deliberately violate it to confirm the check has teeth.
Synthetic traces. Contact traces are generated analytically rather than propagated from live ephemeris. This allows topology to be isolated as an independent variable, which a real orbital trace does not permit. data/starlink-100626.tle and scripts/select_sats.py document where the orbital parameters came from; neither is executed by any run_*.sh.
Add-only. Membership changes exercised in the experiments are Adds. Remove is implemented but never measured, and the Membership Soundness check assumes Add-only histories. Reject and PcsKeyUpdateProposal are declared in EventType but never constructed.
Byte accounting. bytes_transferred counts transferred events only. Merkle root and leaf-hash exchanges are charged as time through neg_cost but not as bytes, so normal and off figures are event bytes rather than total wire bytes. Every arm is measured identically, so the ordering between them is sound.
Single seed. All published runs use seed 42 with drop_probability = 0.0, so the stochastic bridge-drop path is not exercised by the shipped results.
Snapshot pruning. MAX_SNAPSHOTS is 1000 and the largest run produces 13 snapshots, so DAG pruning never fires in any evaluated configuration. Bounded storage is a property of the mechanism, not something these results demonstrate.
MLS is mocked. MlsMock records applied actions and performs no cryptography. This is a research prototype for the delivery layer, not a secure messaging system, and must not be used as one.
The code in this repository is licensed under the MIT License.