Skip to content

Tighten lightfuzz serial Error Resolution - #3427

Open
liquidsec wants to merge 1 commit into
devfrom
lightfuzz-serial-error-resolution
Open

liquidsec wants to merge 1 commit into
devfrom
lightfuzz-serial-error-resolution

Conversation

@liquidsec

Copy link
Copy Markdown
Collaborator

The serial submodule's "Error Resolution" check works like this: put garbage in a parameter, get an error back, then send a real serialized object instead. If the error goes away, maybe the app deserialized it.

For that to mean anything, three things have to be true:

  1. the error was the app failing to parse what we sent
  2. the error is still there while we're testing
  3. the error went away because of what the payload is, not what it looks like

Only the third one was actually checked. This handles the other two.

Changes

Only treat a 500 as an error worth resolving. Any baseline status used to count. A 302 isn't an error, it's the app redirecting you. A 404 means the route didn't match. A 502 or 503 comes from a proxy or CDN having a bad moment, not from the app parsing anything. None of those can be "resolved" by deserializing something. This also replaces two one-off status checks with a single list.

Send all three requests together, twice. We used to re-send the control once to check the error was still there, then send the corrupted twin once, both after the fact. If a server flips between working and failing over a few seconds (rate limiting, a flaky CDN, an origin restarting), sending the same request again tells you nothing, because you're still sampling the same bad moment. Now the payload, the twin and the control all go out together, twice, and it has to come out the same way both times. Also, a request that never comes back used to count as "yep, still broken"; now it throws the candidate away.

Track the one-language rule per host instead of per parameter. A real sink speaks one format. A host that looks like it deserializes Java and PHP and Python isn't deserializing anything. We already threw those out, but only when they turned up on the same parameter, so a host that spread them across several parameters kept all of them. It's now tracked for the whole host, plus a cap on how many parameters on one host can report this at all.

Dropped an error string that was too generic. One of the strings in SERIALIZATION_ERRORS shows up on any login form that rejects a bad username, so it couldn't tell a broken deserializer from a wrong password.

A bug found along the way

The baseline status checks were continue on the payload loop, so they skipped the whole rest of the loop body, Differential Error Analysis included. Their comments only mentioned Error Resolution, but a real Java error string was unreportable whenever the baseline came back with the wrong status. Differential Error Analysis doesn't care what the baseline status is, it just needs a framework error string in the response that isn't in the baseline, so it now runs either way.

What this costs us

We'll miss a sink whose error shows up as a 502 through a proxy, and we'll miss one that behaves differently every time you hit it. Fair trade for a LOW confidence check. The OOB interactsh payloads are the ones that actually prove deserialization, they emit CONFIRMED, and nothing here touches them.

Tests

125 pass on test_module_lightfuzz.py. Six new tests, one per point above. I checked that each one fails if you revert the source and keep the tests, so they're testing the fix rather than just passing.

Two existing tests needed changing:

  • The multi-language test wasn't testing anything. Its fake server returned 200 for the control payload too, so the baseline was already 200, nothing ever looked suspicious, and the test asserted "no findings" against a scan that couldn't produce any. Its server now recognizes serialization magic headers, so the real payloads succeed and the controls and twins don't.
  • The differential-analysis false positive test was built around the error string I removed. It would still pass, but only because none of the remaining strings appear in it. I swapped it for one the submodule still looks for, present in both the baseline and the probe, so it goes on testing the "don't fire if the string is always there" guard. Its assertion is unchanged.

Error Resolution infers a deserializer from an error that a serialized
payload makes disappear. That inference needs the baseline error to be an
application-level failure to interpret the payload, and needs the error to
hold for as long as the test runs. Neither was checked.

- Gate the technique on an allowlist of baseline statuses. A redirect is not
  an error, 404 is routing, 401/403/429 are policy, and 5xx above 500 is
  upstream infrastructure.
- Replace the single control re-send and the separate scrambled-header twin
  probe with one trial that samples payload, twin and control together over
  several rounds. Server state that varies over seconds makes consecutive
  samples of one request correlated, so the arms have to be interleaved to be
  comparable. An unanswered sample is now inconclusive rather than
  confirmation.
- Track the single-language-family invariant per host rather than per
  parameter, and cap how many parameters per host may report the technique.
- Drop an error string too generic to tell a deserialization failure from a
  rejected login.

The baseline status guards previously sat on the payload loop, so they also
disabled Differential Error Analysis. That technique now runs whatever status
the baseline carries.
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.12403% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 90%. Comparing base (dcd7a7a) to head (3107f56).

Files with missing lines Patch % Lines
bbot/modules/lightfuzz/submodules/serial.py 89% 4 Missing ⚠️
.../test_step_2/module_tests/test_module_lightfuzz.py 99% 1 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff           @@
##             dev   #3427    +/-   ##
======================================
+ Coverage     90%     90%    +1%     
======================================
  Files        454     454            
  Lines      47081   47177    +96     
======================================
+ Hits       42316   42425   +109     
+ Misses      4765    4752    -13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

📊 Performance Benchmark Report

Comparing dev (baseline) vs lightfuzz-serial-error-resolution (current)

📈 Detailed Results (All Benchmarks)

📋 Complete results for all benchmarks - includes both significant and insignificant changes

🧪 Test Name 📏 Base 📏 Current 📈 Change 🎯 Status
Bloom Filter Dns Mutation Tracking Performance 3.78ms 3.89ms +3.1%
Bloom Filter Large Scale Dns Brute Force 15.59ms 15.19ms -2.6%
Large Closest Match Lookup 329.79ms 339.44ms +2.9%
Realistic Closest Match Workload 182.30ms 179.76ms -1.4%
Event Memory Medium Scan 1402 B/event 1402 B/event +0.0%
Event Memory Large Scan 1527 B/event 1527 B/event +0.0%
Event Validation Full Scan Startup Small Batch 395.80ms 395.30ms -0.1%
Event Validation Full Scan Startup Large Batch 508.06ms 519.02ms +2.2%
Make Event Autodetection Small 21.16ms 21.24ms +0.4%
Make Event Autodetection Large 212.00ms 210.24ms -0.8%
Make Event Explicit Types 8.62ms 8.71ms +1.0%
Excavate Single Thread Small 3.683s 3.679s -0.1%
Excavate Single Thread Large 9.305s 9.333s +0.3%
Excavate Parallel Tasks Small 3.849s 3.860s +0.3%
Excavate Parallel Tasks Large 5.944s 5.922s -0.4%
Intercept Throughput Small 860.85ms 941.83ms +9.4%
Intercept Throughput Medium 866.82ms 862.11ms -0.5%
Dns Throughput Quiet 2.509s 2.575s +2.6%
Dns Throughput Loaded 1.695s 1.668s -1.6%
Dns Throughput Inherited 2.244s 2.104s -6.3%
Is Ip Performance 2.30ms 2.34ms +1.8%
Make Ip Type Performance 220.91µs 216.66µs -1.9%
Mixed Ip Operations 2.41ms 2.43ms +0.6%
Memory Use Web Crawl 1.0 MB 1.0 MB +0.0%
Memory Use Subdomain Enum 28.3 MB 28.4 MB +0.1%
Memory Use Deep Chain 5.0 MB 5.0 MB +0.0%
Memory Use Parallel Chains 10.2 MB 10.7 MB +5.0%
Scan Throughput 100 2.629s 2.515s -4.3%
Scan Throughput 1000 17.675s 17.921s +1.4%
Typical Queue Shuffle 5.59µs 5.63µs +0.7%
Priority Queue Shuffle 27.51µs 27.64µs +0.5%

🎯 Performance Summary

No significant performance changes detected (all changes <10%)


🐍 Python Version 3.11.16

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant