Repository navigation
Conversation
AsyncTCP calls tcpip_api_call() in AsyncServer::begin(), which asserts with "Invalid mbox" if the TCP/IP stack (esp_netif_init) has not been initialized. WiFi.mode() is what triggers the low-level init in the Arduino core; previously the web server was started before any WiFi call, crashing the board in a reboot loop before it could join the network.
std::find() is used in MockWebSocketServer::removeClient() but <algorithm> was never included; the native test_mcp_server suite failed to build, and CI's firmware matrix is gated behind the native tests.
CORE_DEBUG_LEVEL=5 floods the UART at 115200 baud; on a busy BLE radio the verbose per-advertisement logging stalls BTC_TASK on core 0 while it waits for the TX FIFO to drain, starving the async_tcp task and tripping the watchdog. INFO is the new default (opt-in for targeted debugging), and AsyncTCP runs on core 1 so BLE can never starve it.
The auto-config budget (freeHeap - 64KB) leaves Bluetooth no heap headroom, causing bad_alloc in the scan callback and BTU_StartUp failures. Each ESP32 board now declares BOARD_BUS_HISTORY_HEAP_CAP (S3: 160KB, others: 128KB); 0 means uncapped (legacy behaviour) via the board_config.h fallback.
MCPServer::begin() now starts a WebSocketsServer on port 9000 (the port mDNS was already advertising), handleClient() pumps the event loop, and text frames are routed through processMessage(). Notifications are pushed to subscribed clients via sendFunc_. All ESP32-specific code is NATIVE_TEST-guarded so the native test suite stays hardware-free.
Passive BLE capture for reverse-engineering broadcast formats: - ble/scan / ble/scan/results / ble/scan/stop: non-blocking scans with per-device payload history (32 devices x 8 distinct payloads) - ble/connect / ble/disconnect: GATT client - ble/services/start|results|stop: two-phase GATT enumeration on a dedicated FreeRTOS task so a slow peer can never block the MCP task - payloadIsConnectable + hex helpers in BLEScannerCore.h (natively unit-tested); gated by BOARD_HAS_BLE and excluded from nRF52/native builds via build_src_filter
Features-table row plus a 'Bluetooth (BLE)' section under the MCP API Reference covering ble/scan, ble/scan/results, ble/connect and the two-phase ble/services/* flow, with request/response examples and notes for reverse-engineering broadcast payloads.
wantDuplicates=false made the BLE library drop repeat advertisements for a known MAC before the callback fired, so recordPayload() never saw them and per-device payload history was capped at one entry — the state-change signal the feature exists for was never captured. - scan with wantDuplicates=true (the library frees each advertised device after the callback, so the heap stays bounded) - move BLEAdvReport/MAX_DEVICES/MAX_PAYLOADS and recordPayload() into BLEScannerCore.h so the history logic is natively unit-testable - evicted payloads are removed from the seen-set, so a re-broadcast of an evicted payload is recorded again and the set stays bounded - native regression tests: distinct append, duplicate skip, 8-entry eviction, evicted-payload reappearance
…machine The 2.0.17 Arduino BLE library has no timeouts on its discovery/read semaphores, so a slow or uncooperative peer blocked the MCP task forever; the 10 s cap only bounded the waiter while the task itself could hang, and disconnect() could free the client underneath it. - drive esp_gattc directly on a dedicated task with per-operation deadlines: open (connect timeoutMs), discovery (10 s), reads (5 s), notify capture (3 s), close (2 s) - event-driven: Bluedroid callbacks only record state under the mutex; the task is the only issuer of controller calls, and never holds the mutex across them; events are scoped to our app id / interface / conn id - failed lifecycles route through Closing so a late OPEN_EVT cannot leak the link; the task exits only from Idle - async connect: ble/connect returns immediately, ble/connect/results polls the outcome (timeoutMs is now genuinely enforced) - notify subscription via CCCD writes; reads via get_db cache walk with 128/32/16-bit UUID and properties formatting - new BLEGattCore.h holds the pure state/deadline/format helpers with a native test suite (test_gatt_core)
…d timeouts - 'Passive' -> 'Active' (the scanner uses active scans, which request scan responses; the docs promised passive capture the code never did) - document ble/connect/results and the async connect semantics, and that timeoutMs is enforced - clarify payloadHistory/count: bounded to 8 retained entries, oldest dropped on eviction, count is the number currently retained - document the per-operation bounds: 10 s discovery cap, 5 s per-read, notifyCaptureMs capture window, safe teardown via services/stop
…ards The BLE stack (Bluedroid/NimBLE) plus the MCP WebSocket transport and OTA cannot fit the 1.25 MB default OTA app slot on 4 MB-flash boards (devkitc, c3, huzzah32, m5stack): the original BLE commit never even compiled there (FreeRTOS.h include path) and the branch overflowed by ~800 KB with BLE linked. - include <freertos/FreeRTOS.h> instead of <FreeRTOS.h> so classic ESP32 SDKs resolve it (S3 SDK adds the extra include dir, others do not) — the board matrix now builds everywhere - gate BLE off on the 4 MB boards (BOARD_HAS_BLE 0 + exclude BLEScanner.cpp from their builds, same pattern as nRF52) since the hardware supports it but the flash budget does not; S3 keeps BLE - custom partitions_4mb_ota.csv for devkitc/huzzah32/m5stack: 1.86 MB OTA slots (was 1.25 MB) + 128 KB spiffs + coredump, fitting the transport at 70% with headroom; c3 keeps the default layout (RISC-V libs are smaller and it already fits) - guard bleScanner.begin() with BOARD_HAS_BLE in main.cpp - README: board matrix and BLE section reflect the S3-only BLE build
…d on hardware Hardware testing against a Galaxy Watch Ultra exposed four real defects that unit tests could not catch: - esp_ble_gattc_register_callback() silently fails after BLEDevice::init already registered the Bluedroid handler (one-shot registration), so the scanner never saw REG_EVT/OPEN_EVT and every connect timed out. Hook via the library's official extension point (BLEDevice::setCustomGattcHandler) instead. - The GATT app registration survives across connections; re-registering on the second connect returned ESP_GATT_ERROR and the open was never issued. Track appRegistered_ and only register once; verify REG_EVT status before accepting the interface. - conn_id 0 is legitimate on this stack (first GATT connection), so gating discovery/close on connId_ != 0 deadlocked the state machine at Connected. Gate on state instead. - SEARCH_CMPL_EVT set loadPending_ but never left Discovering, so the task loop only ever exited via the 10 s discovery timeout with empty results. Transition to Connected so the load runs. Verified on hardware over MCP WebSocket: scan (6 devices), async connect + poll, 5-service GATT enumeration with readable values, disconnect, and reconnect all pass end-to-end.
ble/scan/results serialised the whole capture as one JSON response while the scanner stored up to 32 devices x 8 payloads unbounded. Under a busy BLE radio the response grew past available heap, std::bad_alloc escaped the MCP task through the WebSockets library, and std::terminate aborted the board - dropping every client with 'ESP32 connection closed' and empty follow-up scans (reproduced live: abort() on core 1, backtrace into the ble/scan/results handler). - BLEScannerCore: bound capture storage to 12 devices x 4 payloads, cap payloads at 64 bytes (truncated before dedupe so the history and the serialised response stay small even under flood). - MCPServer::dispatch: catch handler exceptions and reply with a JSON-RPC -32603 error instead of letting the exception escape and reboot the device. Verified on hardware: the flood test that crashed the old firmware now returns clean captures across repeated scan cycles with no reboots.
Raise the per-device history cap from 4 to 64 distinct payloads so a full command set (levels x channels + heater + patterns) can be captured in a single scan, and keep the unfiltered ble/scan/results response small by serialising only the newest 4 payloads per device (count reports the true distinct total). - ble/scan/results?mac=<addr> (case-insensitive) returns ONLY that device with its FULL payload history; unseen MACs return an empty devices array. - Storage: 12 devices x 64 payloads x 64-byte cap (~98 KB pathological worst case, a few KB realistic) — the existing -32603 OOM backstop covers the worst case without rebooting. Verified on hardware: unfiltered (7 devices, newest-4 view), mac fetch (full history), case-insensitive match, and unseen-MAC all behave as specified.
… scan
The phone rotates its BLE MAC every ~1s, so per-device payload history
never accumulates (each rotated MAC carries 1 payload). The new registry
collapses all broadcasters into the actual command sequence:
- ble/scan/payloads: MAC-independent distinct payload registry, first-seen
order, each entry with count + firstSeen/lastSeen (ms relative to scan
start). Recorded in onAdv so it survives device FIFO eviction. Verified
on hardware: 6 distinct commands captured across 8 rotated MACs in one
45s scan.
- ble/scan/poll: stateless live read of the registry without stopping the
scan (diff by firstSeen); pairs with continuous scans.
- ble/scan: durationMs 0 = continuous until ble/scan/stop; already-running
scans report {started:false, scanning:true, remainingMs}.
- ble/scan/results: AND-combined response filters serviceUuid /
manufacturerId / macPrefix (never affect capture); results carry
scanStartedMs/durationMs.
Storage: registry bounded to 64 payloads x 64 bytes (~9KB worst case).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds BLE advertising capture for reverse-engineering broadcast formats, exposed over the MCP interface, plus the WebSocket transport that makes the MCP server reachable on the port it advertises.
The GATT client is a hardware-verified state machine built directly on the raw
esp_gattcAPI, replacing the Arduino BLE library's untimeoutable blocking calls.Highlights
BLE advertising scanner — Captures raw payloads from nearby devices, with per-device payload history limited to 32 devices and 8 distinct payloads per device. This makes broadcaster state changes, such as button presses or level changes, visible as payload deltas. Active scans request scan responses to capture advertised names and service data.
GATT client — Implements
ble/connect,ble/disconnect, and two-phaseble/services/start,ble/services/results, andble/services/stopenumeration using a rawesp_gattcstate machine. Per-operation deadlines are enforced:timeoutMsAsync connect —
ble/connectreturns immediately, whileble/connect/resultspolls the outcome. The requestedtimeoutMsis genuinely enforced.MCP WebSocket transport —
MCPServer::begin()now starts a WebSocket server on port9000, which mDNS was already advertising. JSON-RPC frames are routed throughprocessMessage(), and notifications are pushed to subscribed clients.New MCP methods
ble/scandurationMs.ble/scan/resultsble/scan/stopble/connectmacandtimeoutMs.ble/connect/resultsble/disconnectble/services/startble/services/resultsble/services/stopFixes included
lwIP initialization crash — The board previously crashed in a reboot loop before joining Wi-Fi because
AsyncServer::begin()calledtcpip_api_call()before the TCP/IP stack was initialized.WiFi.mode()now runs first.WDT starvation under BLE load —
CORE_DEBUG_LEVEL=5flooded the UART at 115200 baud. Under heavy radio activity, verbose per-advertisement logging stalledBTC_TASKon core 0, starvingasync_tcpand triggering a watchdog abort. The default log level is nowINFO, and AsyncTCP runs on core 1.Heap budget for BLE —
BusHistorypreviously reservedfreeHeap - 64 KB, leaving insufficient space for BLE controller and scan allocations and causingbad_allocin the scan callback. The heap cap is now configurable throughBOARD_BUS_HISTORY_HEAP_CAP:0: UncappedPayload history never populated —
wantDuplicates=falsecaused the library to discard repeated advertisements before invoking the callback, limiting each device's history to one entry. Scans now request duplicates, while the library still frees each device after the callback. History logic was moved into natively tested core code with eviction and reappearance handling.GATT client defects found on hardware — The raw state machine now:
setCustomGattcHandlerextension point becauseesp_ble_gattc_register_callbackis one-shot.REG_EVTstatus.0as valid by gating on state instead of connection ID.DiscoveringonSEARCH_CMPL_EVT, allowing the cache walk to run.4 MB-flash boards — The original BLE commit did not compile on DevKitC, C3, HUZZAH32, or M5Stack because of a
FreeRTOS.hinclude-path issue. BLE, transport, and OTA also exceeded the 1.25 MB OTA application slot. The include path is fixed, BLE is disabled on those boards, and the three classic ESP32 boards now use larger OTA slots throughpartitions_4mb_ota.csv:JSON-RPC error shape —
ble/connectnow returns a proper-32602error envelope whenmacis missing.Pre-existing CI blocker —
test_mcp_serverfailed to build becausestd::findwas used without including<algorithm>in the shared mock header. The firmware matrix is gated behind this test.Testing
pio test -e native— 173/173 passingFull ESP32 matrix passes:
nRF52 builds are unaffected.
Verified live on hardware over the MCP WebSocket using an ESP32-S3-WROOM:
Documentation
The README now includes a Bluetooth (BLE) section under the MCP API Reference, covering:
ble/connect/resultsChecklist
Attribution
This implementation was developed using Agentic AI with human decision-making, OpenCode, and DeepSeek V4 Flash 0731 with Max reasoning.