Feature: Standalone MQTT Broker Gateway / Proxy

[Crim]

Feature: Standalone MQTT Broker Gateway / Proxy

Overview

This pull request introduces the Standalone MQTT Gateway & Proxy subsystem. When enabled, the ESP32 operates as an autonomous 24/7 MQTT client for the connected Meshtastic radio ( when the radio has proxy_to_client enabled and mqtt enabled), eliminating the need to keep a mobile phone or desktop app continuously running.

It also introduces crucial memory optimizations, dual-queue packet prioritization, lock-free telemetry, and thread priority tuning that should improve stability across all ESP32 and ESP32-S3 targets.

---

Key Features & Architectural Changes

1. Auto-Sync MQTT Gateway (Zero-Config)

• Automatic Settings Discovery: Snoops ModuleConfig.mqtt packets over BLE to inherit broker address, port, credentials, root topic (msh, ptp, etc.), and TLS settings directly from the radio.
• Immediate Config Query: Sends a want_config_id packet upon BLE connection to immediately retrieve radio settings without waiting for heartbeats.
• 60-Second Reconnection Grace Period: Keeps the MQTT broker connection alive across transient Bluetooth drops, buffering downlink messages and resuming immediately if reconnected within 60 seconds.
• Packet Filtering: Intercepts and consumes MqttClientProxyMessage uplink packets, preventing duplicate publishes from connected phone apps.

2. 3-Tier TLS Security Pipeline

• Tier 1 (Public Brokers): Pre-compiled Mozilla Root CA bundle embedded in flash (x509_crt_bundle.bin) for verified TLS connections to public brokers (mqtt.meshtastic.org:8883, AWS IoT, HiveMQ, etc.).
• Tier 2 (Custom / Enterprise CA): Supports pasting / uploading custom PEM root certificates for private brokers.
• Tier 3 (Insecure Bypass): Optional toggle to skip certificate / hostname validation for local self-signed setups.

3. Dual-Queue Prioritization & Static .bss Sizing

• Prioritization: Phone app commands / chat packets (tcp_to_ble_queue) are drained first with 100ms backpressure, while background MQTT downlink packets (mqtt_to_ble_queue) are drained when idle using non-blocking (0 tick) enqueues.
• Unified Queue Boundaries: Sized to BRIDGE_QUEUE_SIZE = 32 and MQTT_QUEUE_SIZE = 16 in static .bss (zero heap fragmentation).
• Runtime Headroom: Leaves ~106 KB free heap on ESP32-S3 and ~95 KB on Classic ESP32 (esp32dev), providing >5× safety margin against heap exhaustion during simultaneous TLS handshakes and WebUI requests.

4. Thread Priorities & Deadlock-Free Concurrency

• Task Scheduling Hierarchy:
  • LwIP Stack: Priority 18
  • AsyncTCP Worker: Priority 3
  • mqtt_client Task: Priority 2 (8KB stack)
  • bridgeBleTask: Priority 1 (Core 1, 8KB stack, 1ms yield)
  • bridgeNetTask: Priority 1 (Core 0, 4KB stack)
  • diag_telemetry: Priority 1 (Core 0, 3KB stack, 15s interval)
• Lock Safety: Recursive MqttLockGuard released prior to task lifecycle operations (esp_mqtt_client_start / destroy), preventing task-destruction deadlocks.

5. Queue Drop Telemetry & WebUI Polish

• Real-Time Drop Tracking: Added atomic drop counters across all queues (tcp_to_ble_dropped, ble_to_tcp_dropped, and mqtt_to_ble_dropped).
• Telemetry Output: Periodic single-line [Diag] log displays queue fill levels and drops inline:

  [Diag] Heap: 106 KB (Min: 88 KB, MaxBlock: 95 KB) | WiFi: -50 dBm | TCP: 1 | Queues: [T->B: 0/32 (d:0), B->T: 0/32 (d:0), M->B: 0/16 (d:0)] | Stacks: [BLE: 4040 B, Net: 976 B] | MQTT: Connected (Tx: 3, Rx: 8)                                        
                                                                                                                                                                                                                                                          
  

• WebUI Updates:
  • Main HTML payload streamed directly out of flash (PROGMEM) via AsyncProgmemResponse with zero heap allocation.
  • Live traffic status line displays: ▲ X sent / ▼ Y rcvd / ✖ Z dropped.

──────