diff --git a/MQTT_IMPLEMENTATION.md b/MQTT_IMPLEMENTATION.md index 3b5bdbba..0a3408ca 100644 --- a/MQTT_IMPLEMENTATION.md +++ b/MQTT_IMPLEMENTATION.md @@ -4,29 +4,55 @@ This document describes the MQTT bridge implementation that allows MeshCore repe ## Quick Start Guide -### Essential Commands to Get MQTT Repeater Running +### Essential Commands to Get MQTT Observer Running -**1. Connect to device console via repeater login or serial console (115200 baud)** +**1. Flash the observer firmware to your device** -**2. Configure WiFi Credentials** +Use one of the observer build targets (e.g., `heltec_v4_repeater_observer_mqtt`). After flashing, connect to the device console via serial (115200 baud) or repeater login. + +**2. Configure radio settings** + +If this is a fresh flash or full erase, configure your radio parameters first. These must match other nodes in your mesh: + +```bash +set freq 906.0 +set sf 12 +set cr 5 +set bw 250 +set tx 22 +``` + +**3. Configure device identity** + +```bash +set name MyObserver +set mqtt.iata SEA +``` + +If migrating from an existing node (e.g., a Raspberry Pi gateway), restore the private key to keep the same identity: +```bash +set prv.key +``` + +**4. Configure WiFi credentials** ```bash set wifi.ssid YourWiFiNetwork set wifi.pwd YourWiFiPassword ``` -If you wish to upload to the MeshCore Analyzer, also `set mqtt.iata XXX` to a valid IATA airport code. +**5. (Optional) Disable packet repeating** -**3. Reboot to Connect to WiFi** +If this observer is receive-only (e.g., using a PCB antenna in a location where repeating would be harmful), disable forwarding: +```bash +set repeat off +``` + +**6. Reboot to connect** ```bash reboot ``` -**4. Toggle bridge.source to rx** -```bash -set bridge.source rx -``` - -**5. Verify Configuration** +**7. Verify configuration** ```bash get wifi.ssid get bridge.enabled @@ -36,16 +62,7 @@ get mqtt.iata get mqtt1.preset get mqtt2.preset get mqtt3.preset -``` - -**6. Restart Bridge (if needed)** -```bash -# Option A: Toggle bridge off then on -set bridge.enabled off -set bridge.enabled on - -# Option B: Full device reboot -reboot +get mqtt.status ``` **That's it!** The device will now: @@ -59,8 +76,8 @@ reboot ## Overview The MQTT bridge implementation provides: -- Up to 3 concurrent MQTT connection slots with built-in presets -- Built-in presets for LetsMesh Analyzer (US/EU) and MeshMapper +- Up to 6 MQTT connection slots with built-in presets +- Built-in presets for LetsMesh Analyzer (US/EU), MeshMapper, MeshRank, Waev, Meshomatic, and CascadiaMesh - Custom broker support with username/password authentication - JWT (Ed25519 device signing) authentication for preset brokers - WSS (WebSocket Secure) and direct MQTT transport @@ -73,7 +90,7 @@ The MQTT bridge implementation provides: ### Slot-Based Preset System -The MQTT bridge uses a slot-based architecture with up to 3 concurrent connections. Each slot can be configured with a built-in preset or custom broker settings. +The MQTT bridge uses a slot-based architecture with up to 6 concurrent connections. Each slot can be configured with a built-in preset or custom broker settings. **Built-in Presets:** @@ -83,17 +100,20 @@ The MQTT bridge uses a slot-based architecture with up to 3 concurrent connectio | `analyzer-eu` | mqtt-eu-v1.letsmesh.net:443 | JWT (Ed25519) | WSS | | `meshmapper` | mqtt.meshmapper.cc:443 | JWT (Ed25519) | WSS | | `meshrank` | meshrank.net:8883 | None (token in topic) | MQTT over TLS | +| `waev` | mqtt.waev.app:443 | JWT (Ed25519) | WSS | +| `meshomatic` | us-east.meshomatic.net:443 | JWT (Ed25519) | WSS | +| `cascadiamesh` | cascadiamesh.ddns.net:1883 | None | MQTT (unencrypted) | | `custom` | User-configured | Username/Password | MQTT or WSS | | `none` | (disabled) | — | — | **Default Configuration:** - Slot 1: `analyzer-us` - Slot 2: `analyzer-eu` -- Slot 3: `none` +- Slots 3-6: `none` **Memory Limits:** -- With PSRAM: All 3 slots can be active simultaneously -- Without PSRAM: Maximum 2 active slots (each WSS/TLS connection requires ~40KB internal heap) +- With PSRAM: All slots can be active simultaneously +- Without PSRAM: Maximum 2 active TLS/WSS slots (each WSS/TLS connection requires ~40KB internal heap) - If more slots are configured than the device supports, excess slots show as `(inactive)` in `get mqtt.status` - Slot configurations are preserved in preferences — moving firmware to a PSRAM device activates all slots @@ -130,39 +150,41 @@ pio run -e Station_G2_repeater_observer_mqtt ### Build Flags - `WITH_MQTT_BRIDGE=1` - Enable MQTT bridge (required) +- `WITH_SNMP=1` - Enable SNMP agent (optional, see [MQTT_SNMP.md](MQTT_SNMP.md)) - `MQTT_DEBUG=1` - Enable debug logging (optional) - `MQTT_WIFI_TX_POWER` - WiFi TX power level (default: `WIFI_POWER_11dBm`) -- `MQTT_WIFI_POWER_SAVE_DEFAULT` - Default WiFi power save mode (0=min, 1=none, 2=max) +- ~~`MQTT_WIFI_POWER_SAVE_DEFAULT`~~ - Removed; all builds now default to `none` (no power save) ## Default Configuration -The MQTT bridge comes with the following defaults: -- **Origin**: Device name (set automatically) -- **IATA**: (must be configured) +The MQTT bridge comes with the following defaults for fresh installs: +- **Origin**: Device name (set automatically from `set name`) +- **IATA**: (blank — must be configured for Analyzer presets) - **Status Messages**: Enabled - **Packet Messages**: Enabled - **Raw Messages**: Disabled - **TX Messages**: Disabled (RX only by default) - **Status Interval**: 5 minutes (300000 ms) -- **Slot 1**: `analyzer-us` (mqtt-us-v1.letsmesh.net:443) -- **Slot 2**: `analyzer-eu` (mqtt-eu-v1.letsmesh.net:443) -- **Slot 3**: `none` (disabled) -- **WiFi SSID**: "ssid_here" (must be configured) -- **WiFi Password**: "password_here" (must be configured) -- **WiFi Power Save**: "min" (minimum power saving, balanced performance and power) -- **Timezone**: "America/Los_Angeles" (Pacific Time with DST support) -- **Timezone Offset**: -8 hours (fallback) +- **Slot 1**: `analyzer-us` +- **Slot 2**: `analyzer-eu` +- **Slots 3-6**: `none` (disabled) +- **WiFi SSID**: (blank — must be configured) +- **WiFi Password**: (blank — must be configured) +- **WiFi Power Save**: `none` (no power save) +- **Timezone**: (blank — uses UTC until configured) +- **Timezone Offset**: 0 (fallback, no offset) +- **Repeat (forwarding)**: On (set `repeat off` for receive-only observers) ## CLI Commands ### MQTT Slot Commands -Each slot (1-3) supports the following commands: +Each slot (1-6) supports the following commands: #### Get Commands - `get mqtt1.preset` - Get slot 1 preset name - `get mqtt2.preset` - Get slot 2 preset name -- `get mqtt3.preset` - Get slot 3 preset name +- `get mqttN.preset` - Get slot N preset name (N = 1-6) - `get mqttN.server` - Get custom server hostname for slot N - `get mqttN.port` - Get custom server port for slot N - `get mqttN.username` - Get custom username for slot N @@ -171,12 +193,15 @@ Each slot (1-3) supports the following commands: - `get mqttN.topic` - Get custom topic template for slot N #### Set Commands -- `set mqtt1.preset analyzer-us` - Set slot 1 to LetsMesh Analyzer US -- `set mqtt1.preset analyzer-eu` - Set slot 1 to LetsMesh Analyzer EU -- `set mqtt1.preset meshmapper` - Set slot 1 to MeshMapper -- `set mqtt1.preset meshrank` - Set slot 1 to MeshRank (requires token) -- `set mqtt1.preset custom` - Set slot 1 to custom broker (configure server/port/username/password) -- `set mqtt1.preset none` - Disable slot 1 +- `set mqttN.preset analyzer-us` - Set slot N to LetsMesh Analyzer US +- `set mqttN.preset analyzer-eu` - Set slot N to LetsMesh Analyzer EU +- `set mqttN.preset meshmapper` - Set slot N to MeshMapper +- `set mqttN.preset meshrank` - Set slot N to MeshRank (requires token) +- `set mqttN.preset waev` - Set slot N to Waev +- `set mqttN.preset meshomatic` - Set slot N to Meshomatic +- `set mqttN.preset cascadiamesh` - Set slot N to CascadiaMesh +- `set mqttN.preset custom` - Set slot N to custom broker (configure server/port/username/password) +- `set mqttN.preset none` - Disable slot N - `set mqttN.server ` - Set custom server hostname for slot N - `set mqttN.port ` - Set custom server port for slot N (1-65535) - `set mqttN.username ` - Set custom username for slot N @@ -229,7 +254,7 @@ When a slot's preset is `custom`, you can define a custom topic template using p If no custom topic is set, custom slots default to: `meshcore/{iata}/{device}/{type}` -**Note:** Topic templates only apply to `custom` preset slots. Built-in presets (analyzer-us, analyzer-eu, meshmapper, meshrank) always use their hardcoded topic format. +**Note:** Topic templates only apply to `custom` preset slots. Built-in presets (analyzer-us, analyzer-eu, meshmapper, meshrank, etc.) always use their hardcoded topic format. ### MQTT Shared Commands @@ -270,7 +295,7 @@ These settings apply across all MQTT slots: - `set wifi.pwd ` - Set WiFi password - `set wifi.powersave none|min|max` - Set WiFi power save mode - `none` - No power saving (best performance, highest power consumption) - - `min` - Minimum power saving (default, balanced performance and power) + - `min` - Minimum power saving (balanced performance and power) - `max` - Maximum power saving (lowest power consumption, may affect performance) ### Timezone Commands @@ -288,6 +313,26 @@ These settings apply across all MQTT slots: - **Common abbreviations**: `PDT`, `PST`, `MDT`, `MST`, `CDT`, `CST`, `EDT`, `EST`, `BST`, `GMT`, `CEST`, `CET` - **UTC offsets**: `UTC-8`, `UTC+5`, `+5`, `-8`, etc. +### Device & Radio Commands + +These are standard MeshCore commands, not MQTT-specific, but important for observer setup: + +#### Get Commands +- `get name` - Get device name +- `get repeat` - Get repeat (forwarding) status (on/off) +- `get freq` - Get radio frequency +- `get public.key` - Get device public key (for migration) + +#### Set Commands +- `set name ` - Set device name (also sets MQTT origin) +- `set repeat on|off` - Enable/disable packet forwarding (use `off` for receive-only observers) +- `set prv.key <64-hex-char-key>` - Restore private key (for migrating identity from another device) +- `set freq ` - Set radio frequency +- `set sf <5-12>` - Set LoRa spreading factor +- `set bw ` - Set LoRa bandwidth +- `set cr <5-8>` - Set LoRa coding rate +- `set tx ` - Set transmit power + ### Bridge Commands #### Get Commands @@ -298,6 +343,18 @@ These settings apply across all MQTT slots: - `set bridge.source rx|tx` - Set packet source (rx for received, tx for transmitted) - `set bridge.enabled on|off` - Enable/disable bridge +### SNMP Commands + +#### Get Commands +- `get snmp` - Get SNMP agent status (on/off) +- `get snmp.community` - Get SNMP community string + +#### Set Commands +- `set snmp on|off` - Enable/disable SNMP agent (restart required) +- `set snmp.community ` - Set SNMP community string (restart required, default: `public`) + +See [MQTT_SNMP.md](MQTT_SNMP.md) for full SNMP documentation. + ## Command Architecture The CLI commands are organized into two levels: @@ -309,7 +366,7 @@ The CLI commands are organized into two levels: ### Bridge-Specific Commands (`mqtt.*`, `mqttN.*`, `wifi.*`, `timezone.*`) **Implementation-specific settings** - These only apply to the MQTT bridge: -- `mqttN.*` - Per-slot MQTT broker configuration (N = 1, 2, or 3) +- `mqttN.*` - Per-slot MQTT broker configuration (N = 1-6) - `mqtt.*` - Shared MQTT settings (message types, origin, IATA, etc.) - `wifi.*` - WiFi connection settings for MQTT connectivity - `timezone.*` - Timezone configuration for accurate timestamps @@ -381,12 +438,13 @@ Minimal raw packet data for map integration. ## Key Features ### Slot-Based Preset System -- Up to 3 concurrent MQTT connections (with PSRAM), 2 without PSRAM -- Built-in presets for LetsMesh Analyzer (US/EU), MeshMapper, and MeshRank +- Up to 6 concurrent MQTT connections (with PSRAM), 2 without PSRAM +- Built-in presets for LetsMesh Analyzer (US/EU), MeshMapper, MeshRank, Waev, Meshomatic, and CascadiaMesh - Custom broker support with username/password auth and custom topic templates - JWT (Ed25519) authentication for preset brokers, token-in-topic for MeshRank - WSS (WebSocket Secure) and direct MQTT over TLS transport - Automatic reconnection with exponential backoff per slot +- Circuit breaker pattern with periodic probes for recovery from prolonged outages - JWT token buffers only allocated for JWT-auth slots (memory efficient) - Deferred construction: MQTTBridge is heap-allocated in `begin()` to avoid ESP32 static init crashes @@ -413,7 +471,7 @@ Minimal raw packet data for map integration. - Proper UTC system time handling ### Authentication -- **JWT Authentication**: Ed25519-signed tokens for secure MQTT authentication (used by all built-in presets) +- **JWT Authentication**: Ed25519-signed tokens for secure MQTT authentication (used by all built-in presets except meshrank and cascadiamesh) - **Username/Password**: Standard MQTT authentication for custom brokers - **Username Format** (JWT): `v1_{UPPERCASE_PUBLIC_KEY}` - **Automatic Token Renewal**: Tokens are renewed before expiration @@ -432,48 +490,72 @@ The migration happens automatically on first boot after firmware update. No manu ## First-Time Setup ### Prerequisites -- MeshCore device with MQTT bridge firmware flashed +- MeshCore device with observer MQTT firmware flashed - WiFi network credentials -- LoRa-capable device for configuration (repeater console) +- Serial console access (115200 baud) or repeater login via companion app -### Step 1: Configure WiFi +### Step 1: Configure Radio (after fresh flash/full erase) + +If this is a fresh flash, radio parameters must be set to match your mesh network: +``` +set freq 906.0 +set sf 12 +set cr 5 +set bw 250 +set tx 22 +``` + +### Step 2: Configure Device Identity +``` +set name MyObserver +set mqtt.iata SEA +``` + +If migrating from an existing device, restore the private key to keep the same identity: +``` +set prv.key +``` + +### Step 3: Configure WiFi ``` set wifi.ssid YourWiFiNetwork set wifi.pwd YourWiFiPassword reboot ``` -### Step 2: Configure Device Identity +### Step 4: Configure Timezone (optional) ``` -set mqtt.iata SEA -get mqtt.origin +set timezone America/New_York ``` -### Step 3: Verify Slot Configuration +Or use an offset as a fallback: +``` +set timezone.offset -5 +``` + +### Step 5: (Optional) Disable Repeating + +For receive-only observers (e.g., using a PCB antenna or in a location where repeating is not desired): +``` +set repeat off +``` + +### Step 6: Verify Slot Configuration ``` get mqtt1.preset # Should show: analyzer-us get mqtt2.preset # Should show: analyzer-eu get mqtt3.preset # Should show: none ``` -### Step 4: (Optional) Add MeshMapper +### Step 7: (Optional) Add Additional Presets ``` set mqtt3.preset meshmapper ``` -### Step 5: (Optional) Configure Custom Broker +### Step 8: Verify Connection ``` -set mqtt3.preset custom -set mqtt3.server your-broker.example.com -set mqtt3.port 1883 -set mqtt3.username your-username -set mqtt3.password your-password -``` - -### Step 6: Verify Connection -``` -set bridge.source rx get bridge.enabled +get bridge.source get mqtt.status get wifi.status ``` @@ -492,6 +574,8 @@ reboot ``` get bridge.enabled set bridge.enabled on +get bridge.source # Should be "rx" +set bridge.source rx get mqtt.status # Check per-slot connection status get mqtt1.preset # Verify slots are configured get mqtt.iata # IATA must be set for Analyzer presets @@ -505,6 +589,10 @@ set timezone EST # Abbreviation set timezone UTC-5 # UTC offset ``` +## SNMP Monitoring + +Observer nodes include an optional SNMP v2c agent that exposes radio stats, MQTT connectivity, memory usage, and network information to standard monitoring tools. See [MQTT_SNMP.md](MQTT_SNMP.md) for setup and OID reference. + ## Dependencies - **PsychicMqttClient**: MQTT client library (supports WSS and direct MQTT) @@ -514,3 +602,4 @@ set timezone UTC-5 # UTC offset - **WiFi**: ESP32 WiFi functionality - **Ed25519**: Cryptographic library for JWT token signing - **JWTHelper**: Custom JWT token generation for device authentication +- **SNMP_Agent**: Optional SNMPv2c agent (0neblock/SNMP_Agent, observer builds only) diff --git a/MQTT_SNMP.md b/MQTT_SNMP.md new file mode 100644 index 00000000..82740092 --- /dev/null +++ b/MQTT_SNMP.md @@ -0,0 +1,149 @@ +# SNMP Agent for MeshCore Observer Firmware + +This document describes the optional SNMP v2c agent that allows network monitoring tools to poll MeshCore observer nodes for health and performance metrics. + +## Overview + +The SNMP agent exposes radio statistics, MQTT connectivity status, memory usage, and network information via standard SNMP v2c GET/GETNEXT/GETBULK operations. It is designed for use with monitoring tools such as Nagios, Zabbix, LibreNMS, PRTG, or any SNMP-capable system. + +- **Protocol**: SNMPv2c (read-only, no SET support) +- **Port**: UDP 161 (standard SNMP) +- **Library**: [0neblock/SNMP_Agent](https://github.com/0neblock/Arduino_SNMP) +- **Disabled by default** -- must be explicitly enabled via CLI + +## Quick Start + +```bash +# Enable SNMP +set snmp on +reboot + +# Verify from a host on the same network +snmpwalk -v2c -c public 1.3.6.1.4.1.99999 +``` + +## CLI Commands + +| Command | Description | +|---|---| +| `get snmp` | Show SNMP agent status (`on` / `off`) | +| `set snmp on` | Enable SNMP agent (restart required) | +| `set snmp off` | Disable SNMP agent (restart required) | +| `get snmp.community` | Show SNMP community string | +| `set snmp.community ` | Set community string (restart required, default: `public`) | + +## OID Reference + +All OIDs are under the private enterprise subtree `1.3.6.1.4.1.99999` (`.iso.org.dod.internet.private.enterprises.99999`). + +### System (.1.x.0) + +| OID | Type | Description | +|---|---|---| +| `.1.1.0` | INTEGER | System uptime (seconds) | +| `.1.2.0` | STRING | Firmware version | +| `.1.3.0` | STRING | Node name | + +### Radio (.2.x.0) + +| OID | Type | Description | +|---|---|---| +| `.2.1.0` | INTEGER | Total packets received | +| `.2.2.0` | INTEGER | Total packets sent | +| `.2.3.0` | INTEGER | Receive errors (CRC failures, etc.) | +| `.2.4.0` | INTEGER | Noise floor (dBm) | +| `.2.5.0` | INTEGER | Last RSSI (dBm) | +| `.2.6.0` | INTEGER | Last SNR (dB x 4) | +| `.2.7.0` | INTEGER | Flood packets sent | +| `.2.8.0` | INTEGER | Direct packets sent | +| `.2.9.0` | INTEGER | Flood packets received | +| `.2.10.0` | INTEGER | Direct packets received | +| `.2.11.0` | INTEGER | Total air time (seconds) | + +### MQTT (.3.x.0) + +| OID | Type | Description | +|---|---|---| +| `.3.1.0` | INTEGER | Connected MQTT slot count | +| `.3.2.0` | INTEGER | Packet queue depth | +| `.3.3.0` | INTEGER | Skipped publishes (memory pressure) | + +### Memory (.4.x.0) + +| OID | Type | Description | +|---|---|---| +| `.4.1.0` | INTEGER | Free heap (bytes) | +| `.4.2.0` | INTEGER | Max allocatable block (bytes) | +| `.4.3.0` | INTEGER | Free internal RAM (bytes) | +| `.4.4.0` | INTEGER | Free PSRAM (bytes, 0 if no PSRAM) | + +### Network (.5.x.0) + +| OID | Type | Description | +|---|---|---| +| `.5.1.0` | INTEGER | WiFi RSSI (dBm) | + +## Example Output + +``` +$ snmpwalk -v2c -c public 192.168.50.95 1.3.6.1.4.1.99999 + +SNMPv2-SMI::enterprises.99999.1.1.0 = INTEGER: 18 # uptime 18s +SNMPv2-SMI::enterprises.99999.1.2.0 = STRING: "v1.14.1" # firmware +SNMPv2-SMI::enterprises.99999.1.3.0 = STRING: "Heltec Repeater" +SNMPv2-SMI::enterprises.99999.2.1.0 = INTEGER: 3 # 3 packets received +SNMPv2-SMI::enterprises.99999.2.4.0 = INTEGER: -96 # noise floor -96 dBm +SNMPv2-SMI::enterprises.99999.2.5.0 = INTEGER: -25 # last RSSI -25 dBm +SNMPv2-SMI::enterprises.99999.4.1.0 = INTEGER: 199612 # ~195 KB free heap +SNMPv2-SMI::enterprises.99999.4.2.0 = INTEGER: 188404 # ~184 KB max alloc +SNMPv2-SMI::enterprises.99999.5.1.0 = INTEGER: -22 # WiFi RSSI -22 dBm +``` + +## Architecture + +The SNMP agent runs inside the existing MQTT FreeRTOS task on Core 0 (the WiFi/network core). No additional task is created. + +- **Radio stats** are pushed from Core 1 (mesh task) every 2 seconds via `updateRadioStats()` +- **MQTT stats** are updated from Core 0 each MQTT task loop iteration +- **Memory and WiFi stats** are read directly in the SNMP loop on Core 0 + +The agent starts automatically once WiFi connects and `snmp_enabled = 1` in prefs. It adds approximately 2-3 KB RAM overhead plus per-OID handler storage. + +## Build Configuration + +SNMP is enabled at compile time with the `WITH_SNMP=1` build flag and included in observer firmware targets: + +- `heltec_v3_repeater_observer_mqtt` +- `heltec_v4_repeater_observer_mqtt` +- `Station_G2_repeater_observer_mqtt` + +To add SNMP to another observer variant, add the following to its `platformio.ini`: + +```ini +build_flags = + ... + -D WITH_SNMP=1 + +build_src_filter = + ... + + + +lib_deps = + ... + 0neblock/SNMP_Agent +``` + +## Files + +| File | Description | +|---|---| +| `src/helpers/SNMPAgent.h` | SNMP agent wrapper class declaration | +| `src/helpers/SNMPAgent.cpp` | OID registration and stats update implementation | +| `src/helpers/CommonCLI.h` | `NodePrefs` struct (`snmp_enabled`, `snmp_community` fields) | +| `src/helpers/CommonCLI.cpp` | CLI command parsing and prefs persistence | +| `src/helpers/bridges/MQTTBridge.cpp` | SNMP loop integration in `mqttTaskLoop()` | +| `examples/simple_repeater/MyMesh.cpp` | Radio stats push and agent initialization | + +## Private Enterprise Number + +The OIDs currently use a temporary unregistered enterprise number (`99999`). A proper Private Enterprise Number (PEN) can be registered with IANA at no cost at https://pen.iana.org/pen/PenApplication.page. diff --git a/examples/simple_repeater/MyMesh.cpp b/examples/simple_repeater/MyMesh.cpp index 48adfb04..aa05bdf0 100644 --- a/examples/simple_repeater/MyMesh.cpp +++ b/examples/simple_repeater/MyMesh.cpp @@ -919,6 +919,9 @@ MyMesh::MyMesh(mesh::MainBoard &board, mesh::Radio &radio, mesh::MillisecondCloc _prefs.flood_advert_interval = 12; // 12 hours _prefs.flood_max = 64; _prefs.interference_threshold = 0; // disabled +#ifdef WITH_MQTT_BRIDGE + _prefs.agc_reset_interval = 7; // 28 seconds (secs/4) — prevents AGC drift on long-running observers +#endif // bridge defaults _prefs.bridge_enabled = 1; // enabled @@ -929,6 +932,10 @@ MyMesh::MyMesh(mesh::MainBoard &board, mesh::Radio &radio, mesh::MillisecondCloc StrHelper::strncpy(_prefs.bridge_secret, "LVSITANOS", sizeof(_prefs.bridge_secret)); + // SNMP defaults + _prefs.snmp_enabled = 0; + StrHelper::strncpy(_prefs.snmp_community, "public", sizeof(_prefs.snmp_community)); + // GPS defaults _prefs.gps_enabled = 0; _prefs.gps_interval = 0; @@ -1010,6 +1017,13 @@ void MyMesh::begin(FILESYSTEM *fs) { #ifdef WITH_MQTT_BRIDGE // Set stats sources for automatic stats collection bridge->setStatsSources(this, _radio, _cli.getBoard(), _ms); +#ifdef WITH_SNMP + if (_prefs.snmp_enabled) { + _snmp_agent.setNodeName(_prefs.node_name); + _snmp_agent.setFirmwareVersion(getFirmwareVer()); + bridge->setSNMPAgent(&_snmp_agent); + } +#endif #endif bridge->begin(); @@ -1434,6 +1448,25 @@ void MyMesh::loop() { uint32_t now = millis(); uptime_millis += now - last_millis; last_millis = now; + +#ifdef WITH_SNMP + // Push radio stats to SNMP agent every 2 seconds + if (_snmp_agent.isRunning()) { + static unsigned long last_snmp_stats = 0; + if (now - last_snmp_stats >= 2000) { + last_snmp_stats = now; + _snmp_agent.updateRadioStats( + radio_driver.getPacketsRecv(), radio_driver.getPacketsSent(), + radio_driver.getPacketsRecvErrors(), + (int16_t)_radio->getNoiseFloor(), + (int16_t)radio_driver.getLastRSSI(), + (int16_t)(radio_driver.getLastSNR() * 4), + getNumSentFlood(), getNumSentDirect(), + getNumRecvFlood(), getNumRecvDirect(), + getTotalAirTime() / 1000, uptime_millis / 1000); + } + } +#endif } // To check if there is pending work diff --git a/examples/simple_repeater/MyMesh.h b/examples/simple_repeater/MyMesh.h index d8625bd9..d957f6d9 100644 --- a/examples/simple_repeater/MyMesh.h +++ b/examples/simple_repeater/MyMesh.h @@ -29,6 +29,10 @@ #define WITH_BRIDGE #endif +#ifdef WITH_SNMP +#include "helpers/SNMPAgent.h" +#endif + #include #include #include @@ -122,6 +126,9 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks { #elif defined(WITH_MQTT_BRIDGE) MQTTBridge* bridge; #endif +#ifdef WITH_SNMP + MeshSNMPAgent _snmp_agent; +#endif void putNeighbour(const mesh::Identity& id, uint32_t timestamp, float snr); void sendNodeDiscoverReq(); diff --git a/src/helpers/CommonCLI.cpp b/src/helpers/CommonCLI.cpp index c0ce70f1..4d4a0ff1 100644 --- a/src/helpers/CommonCLI.cpp +++ b/src/helpers/CommonCLI.cpp @@ -159,7 +159,9 @@ void CommonCLI::loadPrefsInt(FILESYSTEM* fs, const char* filename) { remaining -= to_read; } file.read((uint8_t *)&_prefs->rx_boosted_gain, sizeof(_prefs->rx_boosted_gain)); // 290 - // next: 291 + file.read((uint8_t *)&_prefs->snmp_enabled, sizeof(_prefs->snmp_enabled)); // 291 + file.read((uint8_t *)&_prefs->snmp_community, sizeof(_prefs->snmp_community)); // 292 + // next: 316 // sanitise bad pref values _prefs->rx_delay_base = constrain(_prefs->rx_delay_base, 0, 20.0f); @@ -189,6 +191,8 @@ void CommonCLI::loadPrefsInt(FILESYSTEM* fs, const char* filename) { _prefs->advert_loc_policy = constrain(_prefs->advert_loc_policy, 0, 2); _prefs->rx_boosted_gain = constrain(_prefs->rx_boosted_gain, 0, 1); // boolean + _prefs->snmp_enabled = constrain(_prefs->snmp_enabled, 0, 1); + _prefs->snmp_community[sizeof(_prefs->snmp_community) - 1] = '\0'; // ensure null terminated file.close(); } @@ -274,7 +278,9 @@ void CommonCLI::savePrefs(FILESYSTEM* fs) { remaining -= to_write; } file.write((uint8_t *)&_prefs->rx_boosted_gain, sizeof(_prefs->rx_boosted_gain)); // 290 - // next: 291 + file.write((uint8_t *)&_prefs->snmp_enabled, sizeof(_prefs->snmp_enabled)); // 291 + file.write((uint8_t *)&_prefs->snmp_community, sizeof(_prefs->snmp_community)); // 292 + // next: 316 file.close(); } @@ -304,11 +310,7 @@ static void setMQTTPrefsDefaults(MQTTPrefs* prefs) { strncpy(prefs->mqtt_slot_preset[i], "none", sizeof(prefs->mqtt_slot_preset[i]) - 1); prefs->mqtt_slot_preset[i][sizeof(prefs->mqtt_slot_preset[i]) - 1] = '\0'; } - #ifdef MQTT_WIFI_POWER_SAVE_DEFAULT - prefs->wifi_power_save = MQTT_WIFI_POWER_SAVE_DEFAULT; // 0=min, 1=none, 2=max - #else - prefs->wifi_power_save = 0; // Default to WIFI_PS_MIN_MODEM (0=min) - #endif + prefs->wifi_power_save = 1; // Default to none (0=min, 1=none, 2=max) // String fields are already zero-initialized by memset } @@ -714,6 +716,10 @@ void CommonCLI::handleCommand(uint32_t sender_timestamp, const char* command, ch } else { strcpy(reply, "> strict"); } + } else if (memcmp(config, "snmp.community", 14) == 0) { + sprintf(reply, "> %s", _prefs->snmp_community); + } else if (memcmp(config, "snmp", 4) == 0 && (config[4] == '\0' || config[4] == '\n' || config[4] == '\r')) { + strcpy(reply, _prefs->snmp_enabled ? "> on" : "> off"); } else if (memcmp(config, "tx", 2) == 0 && (config[2] == 0 || config[2] == ' ')) { sprintf(reply, "> %d", (int32_t) _prefs->tx_power_dbm); } else if (memcmp(config, "freq", 4) == 0) { @@ -1104,6 +1110,14 @@ void CommonCLI::handleCommand(uint32_t sender_timestamp, const char* command, ch savePrefs(); strcpy(reply, "OK"); } + } else if (memcmp(config, "snmp.community ", 15) == 0) { + StrHelper::strncpy(_prefs->snmp_community, &config[15], sizeof(_prefs->snmp_community)); + savePrefs(); + strcpy(reply, "OK - restart to apply"); + } else if (memcmp(config, "snmp ", 5) == 0) { + _prefs->snmp_enabled = memcmp(&config[5], "on", 2) == 0; + savePrefs(); + strcpy(reply, "OK - restart to apply"); } else if (memcmp(config, "tx ", 3) == 0) { _prefs->tx_power_dbm = atoi(&config[3]); savePrefs(); diff --git a/src/helpers/CommonCLI.h b/src/helpers/CommonCLI.h index 209b13dc..982b6eab 100644 --- a/src/helpers/CommonCLI.h +++ b/src/helpers/CommonCLI.h @@ -72,7 +72,7 @@ struct NodePrefs { // persisted to file // WiFi settings char wifi_ssid[32]; // WiFi SSID char wifi_password[64]; // WiFi password - uint8_t wifi_power_save; // WiFi power save mode: 0=min, 1=none, 2=max (default: 0=min) + uint8_t wifi_power_save; // WiFi power save mode: 0=min, 1=none, 2=max (default: 1=none) // Timezone settings char timezone_string[32]; // Timezone string (e.g., "America/Los_Angeles") @@ -96,6 +96,10 @@ struct NodePrefs { // persisted to file char mqtt_slot_topic[MAX_MQTT_SLOTS][96]; // Per-slot custom topic template (custom preset only) uint8_t loop_detect; + + // SNMP settings (optional, only used when WITH_SNMP is defined) + uint8_t snmp_enabled; // boolean: 0=off, 1=on + char snmp_community[24]; // community string (default "public") }; #ifdef WITH_MQTT_BRIDGE @@ -137,7 +141,7 @@ struct MQTTPrefs { // WiFi settings char wifi_ssid[32]; // WiFi SSID char wifi_password[64]; // WiFi password - uint8_t wifi_power_save; // WiFi power save mode: 0=min, 1=none, 2=max (default: 0=min) + uint8_t wifi_power_save; // WiFi power save mode: 0=min, 1=none, 2=max (default: 1=none) // Timezone settings char timezone_string[32]; // Timezone string (e.g., "America/Los_Angeles") diff --git a/src/helpers/SNMPAgent.cpp b/src/helpers/SNMPAgent.cpp new file mode 100644 index 00000000..a70716f5 --- /dev/null +++ b/src/helpers/SNMPAgent.cpp @@ -0,0 +1,123 @@ +#ifdef WITH_SNMP + +#include "SNMPAgent.h" +#include + +#define SNMP_PORT 161 + +MeshSNMPAgent::MeshSNMPAgent() + : _snmp("public"), + _running(false), + _uptime_secs(0), + _packets_recv(0), _packets_sent(0), _recv_errors(0), + _noise_floor(0), _last_rssi(0), _last_snr(0), + _sent_flood(0), _sent_direct(0), _recv_flood(0), _recv_direct(0), + _total_air_time_secs(0), + _mqtt_connected_slots(0), _mqtt_queue_depth(0), _mqtt_skipped_publishes(0), + _free_heap(0), _max_alloc(0), _internal_free(0), _psram_free(0), + _wifi_rssi(0) +{ + _firmware_version[0] = '\0'; + _node_name[0] = '\0'; +} + +void MeshSNMPAgent::begin(const char* community) { + if (_running) return; + + _snmp = SNMPAgent(community); + _snmp.setUDP(&_udp); + _snmp.begin(); + + // System group (.1.x.0) + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".1.1.0", &_uptime_secs); + _snmp.addReadOnlyStaticStringHandler(MESHCORE_OID_BASE ".1.2.0", _firmware_version, sizeof(_firmware_version)); + _snmp.addReadOnlyStaticStringHandler(MESHCORE_OID_BASE ".1.3.0", _node_name, sizeof(_node_name)); + + // Radio group (.2.x.0) + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.1.0", &_packets_recv); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.2.0", &_packets_sent); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.3.0", &_recv_errors); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.4.0", &_noise_floor); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.5.0", &_last_rssi); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.6.0", &_last_snr); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.7.0", &_sent_flood); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.8.0", &_sent_direct); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.9.0", &_recv_flood); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.10.0", &_recv_direct); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".2.11.0", &_total_air_time_secs); + + // MQTT group (.3.x.0) + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".3.1.0", &_mqtt_connected_slots); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".3.2.0", &_mqtt_queue_depth); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".3.3.0", &_mqtt_skipped_publishes); + + // Memory group (.4.x.0) + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".4.1.0", &_free_heap); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".4.2.0", &_max_alloc); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".4.3.0", &_internal_free); + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".4.4.0", &_psram_free); + + // Network group (.5.x.0) + _snmp.addIntegerHandler(MESHCORE_OID_BASE ".5.1.0", &_wifi_rssi); + + _snmp.sortHandlers(); + _running = true; +} + +void MeshSNMPAgent::loop() { + if (!_running) return; + + // Update memory and network stats locally (we're on Core 0 with WiFi) + _free_heap = (int)ESP.getFreeHeap(); + _max_alloc = (int)ESP.getMaxAllocHeap(); + _internal_free = (int)heap_caps_get_free_size(MALLOC_CAP_INTERNAL); +#ifdef BOARD_HAS_PSRAM + _psram_free = (int)heap_caps_get_free_size(MALLOC_CAP_SPIRAM); +#else + _psram_free = 0; +#endif + + if (WiFi.isConnected()) { + _wifi_rssi = (int)WiFi.RSSI(); + } + + _snmp.loop(); +} + +void MeshSNMPAgent::updateRadioStats( + uint32_t packets_recv, uint32_t packets_sent, uint32_t recv_errors, + int16_t noise_floor, int16_t last_rssi, int16_t last_snr, + uint32_t sent_flood, uint32_t sent_direct, + uint32_t recv_flood, uint32_t recv_direct, + uint32_t total_air_time_secs, uint32_t uptime_secs) { + _packets_recv = (int)packets_recv; + _packets_sent = (int)packets_sent; + _recv_errors = (int)recv_errors; + _noise_floor = (int)noise_floor; + _last_rssi = (int)last_rssi; + _last_snr = (int)last_snr; + _sent_flood = (int)sent_flood; + _sent_direct = (int)sent_direct; + _recv_flood = (int)recv_flood; + _recv_direct = (int)recv_direct; + _total_air_time_secs = (int)total_air_time_secs; + _uptime_secs = (int)uptime_secs; +} + +void MeshSNMPAgent::updateMQTTStats(int connected_slots, int queue_depth, int skipped_publishes) { + _mqtt_connected_slots = connected_slots; + _mqtt_queue_depth = queue_depth; + _mqtt_skipped_publishes = skipped_publishes; +} + +void MeshSNMPAgent::setNodeName(const char* name) { + strncpy(_node_name, name, sizeof(_node_name) - 1); + _node_name[sizeof(_node_name) - 1] = '\0'; +} + +void MeshSNMPAgent::setFirmwareVersion(const char* version) { + strncpy(_firmware_version, version, sizeof(_firmware_version) - 1); + _firmware_version[sizeof(_firmware_version) - 1] = '\0'; +} + +#endif // WITH_SNMP diff --git a/src/helpers/SNMPAgent.h b/src/helpers/SNMPAgent.h new file mode 100644 index 00000000..c1bfe0f5 --- /dev/null +++ b/src/helpers/SNMPAgent.h @@ -0,0 +1,79 @@ +#pragma once + +#ifdef WITH_SNMP + +#include +#include +#include + +// Temporary private enterprise OID base — replace with registered PEN when available. +// All MeshCore OIDs live under this subtree. +#define MESHCORE_OID_BASE ".1.3.6.1.4.1.99999" + +// OID layout: +// .1.x.0 = system (uptime, version, node name) +// .2.x.0 = radio (packets, RSSI, SNR, noise floor, air time) +// .3.x.0 = mqtt (connected slots, queue depth, skipped publishes) +// .4.x.0 = memory (free heap, max alloc, internal free, PSRAM free) +// .5.x.0 = network (WiFi RSSI) + +class MeshSNMPAgent { +public: + MeshSNMPAgent(); + void begin(const char* community); + void loop(); + + // Called from the mesh task (Core 1) to push fresh stats into SNMP-visible variables. + // Copies are atomic for 32-bit aligned ints on ESP32, so no mutex needed. + void updateRadioStats(uint32_t packets_recv, uint32_t packets_sent, uint32_t recv_errors, + int16_t noise_floor, int16_t last_rssi, int16_t last_snr, + uint32_t sent_flood, uint32_t sent_direct, + uint32_t recv_flood, uint32_t recv_direct, + uint32_t total_air_time_secs, uint32_t uptime_secs); + + void updateMQTTStats(int connected_slots, int queue_depth, int skipped_publishes); + + void setNodeName(const char* name); + void setFirmwareVersion(const char* version); + + bool isRunning() const { return _running; } + +private: + WiFiUDP _udp; + SNMPAgent _snmp; + bool _running; + + // System OIDs + int _uptime_secs; + char _firmware_version[32]; + char _node_name[32]; + + // Radio OIDs + int _packets_recv; + int _packets_sent; + int _recv_errors; + int _noise_floor; + int _last_rssi; + int _last_snr; + int _sent_flood; + int _sent_direct; + int _recv_flood; + int _recv_direct; + int _total_air_time_secs; + + // MQTT OIDs + int _mqtt_connected_slots; + int _mqtt_queue_depth; + int _mqtt_skipped_publishes; + + // Memory OIDs (updated in loop() since we're on Core 0 with WiFi) + int _free_heap; + int _max_alloc; + int _internal_free; + int _psram_free; + + // Network OIDs + int _wifi_rssi; +}; + +#endif // WITH_SNMP diff --git a/src/helpers/bridges/MQTTBridge.cpp b/src/helpers/bridges/MQTTBridge.cpp index f4e2e5fa..725b3214 100644 --- a/src/helpers/bridges/MQTTBridge.cpp +++ b/src/helpers/bridges/MQTTBridge.cpp @@ -4,6 +4,10 @@ #include #include +#ifdef WITH_SNMP +#include "../SNMPAgent.h" +#endif + #ifdef ESP_PLATFORM #include #include @@ -168,8 +172,12 @@ MQTTBridge::MQTTBridge(NodePrefs *prefs, mesh::PacketManager *mgr, mesh::RTCCloc _cached_has_connected_slots(false), _last_memory_check(0), _skipped_publishes(0), _last_fragmentation_recovery(0), _fragmentation_pressure_since(0), _last_critical_check_run(0), - _last_no_broker_log(0), _last_config_warning(0), + _last_no_broker_log(0), _queue_disconnected_since(0), _all_tripped_since(0), + _last_config_warning(0), _dispatcher(nullptr), _radio(nullptr), _board(nullptr), _ms(nullptr), +#ifdef WITH_SNMP + _snmp_agent(nullptr), +#endif _last_wifi_check(0), _last_wifi_status(WL_DISCONNECTED), _wifi_status_initialized(false), _wifi_disconnected_time(0), _last_wifi_reconnect_attempt(0), _wifi_reconnect_backoff_attempt(0) #ifdef ESP_PLATFORM @@ -673,6 +681,25 @@ void MQTTBridge::mqttTaskLoop() { // Process packet queue processPacketQueue(); +#ifdef WITH_SNMP + // SNMP agent loop — process incoming UDP requests + if (_snmp_agent) { + if (!_snmp_agent->isRunning() && WiFi.isConnected() && _prefs->snmp_enabled) { + _snmp_agent->begin(_prefs->snmp_community); + MQTT_DEBUG_PRINTLN("SNMP agent started on port 161 (community: %s)", _prefs->snmp_community); + } + if (_snmp_agent->isRunning()) { + // Update MQTT stats from this core + int connected = 0; + for (int i = 0; i < MAX_MQTT_SLOTS; i++) { + if (_slots[i].enabled && _slots[i].connected) connected++; + } + _snmp_agent->updateMQTTStats(connected, _queue_count, _skipped_publishes); + _snmp_agent->loop(); + } + } +#endif + // Periodic configuration check (throttled to avoid spam) checkConfigurationMismatch(); @@ -937,6 +964,9 @@ void MQTTBridge::maintainSlotConnections() { // Only allow one reconnect attempt per maintenance cycle to avoid // multiple simultaneous TLS handshakes blocking the network stack bool reconnect_attempted_this_cycle = false; + // Only allow one full teardown+setup per cycle to limit heap fragmentation + // when multiple slots fail simultaneously + bool teardown_attempted_this_cycle = false; for (int i = 0; i < MAX_MQTT_SLOTS; i++) { if (!_slots[i].enabled || !_slots[i].client) continue; @@ -946,11 +976,11 @@ void MQTTBridge::maintainSlotConnections() { continue; } - maintainSlotConnection(i, now_millis, current_time, time_synced, reconnect_attempted_this_cycle); + maintainSlotConnection(i, now_millis, current_time, time_synced, reconnect_attempted_this_cycle, teardown_attempted_this_cycle); } } -void MQTTBridge::maintainSlotConnection(int index, unsigned long now_millis, unsigned long current_time, bool time_synced, bool& reconnect_attempted) { +void MQTTBridge::maintainSlotConnection(int index, unsigned long now_millis, unsigned long current_time, bool time_synced, bool& reconnect_attempted, bool& teardown_attempted) { MQTTSlot& slot = _slots[index]; if (slot.connected) { @@ -1037,13 +1067,23 @@ void MQTTBridge::maintainSlotConnection(int index, unsigned long now_millis, uns } slot.client->connect(); } else { - // Token expired or near expiry — full teardown for fresh TLS + new token - bool saved_tripped = slot.circuit_breaker_tripped; - MQTT_DEBUG_PRINTLN("MQTT%d token expired/near expiry, full teardown+setup for probe", index + 1); - teardownSlot(index); - setupSlot(index); - _slots[index].circuit_breaker_tripped = saved_tripped; - _slots[index].last_reconnect_attempt = now_millis; + // Token expired — regenerate token but avoid teardown+setup + // which would allocate new TLS context on potentially fragmented heap + if (slot.client) { + if (createSlotAuthToken(index)) { + slot.client->setCredentials(_jwt_username, slot.auth_token); + MQTT_DEBUG_PRINTLN("MQTT%d circuit breaker probe (regenerated expired token)", index + 1); + } + slot.client->connect(); + } else { + // Client was destroyed — must do full setup + bool saved_tripped = slot.circuit_breaker_tripped; + MQTT_DEBUG_PRINTLN("MQTT%d circuit breaker probe (full setup, no client)", index + 1); + teardownSlot(index); + setupSlot(index); + _slots[index].circuit_breaker_tripped = saved_tripped; + _slots[index].last_reconnect_attempt = now_millis; + } } } else { slot.client->connect(); @@ -1094,6 +1134,12 @@ void MQTTBridge::maintainSlotConnection(int index, unsigned long now_millis, uns slot.client->reconnect(); } else { // Full teardown: token expired/near expiry, or lightweight reconnect failed (backoff 3+) + if (teardown_attempted) { + // Defer to next cycle to limit heap fragmentation from simultaneous teardowns + slot.last_reconnect_attempt = now_millis; + return; + } + teardown_attempted = true; uint8_t saved_backoff = slot.reconnect_backoff; uint8_t saved_failures = slot.max_backoff_failures; MQTT_DEBUG_PRINTLN("MQTT%d full teardown+setup for reconnect (backoff %d)", index + 1, saved_backoff); @@ -1110,6 +1156,11 @@ void MQTTBridge::maintainSlotConnection(int index, unsigned long now_millis, uns slot.client->reconnect(); } else { // Full teardown after lightweight reconnect failed (backoff 3+) + if (teardown_attempted) { + slot.last_reconnect_attempt = now_millis; + return; + } + teardown_attempted = true; uint8_t saved_backoff = slot.reconnect_backoff; uint8_t saved_failures = slot.max_backoff_failures; MQTT_DEBUG_PRINTLN("MQTT%d full teardown+setup for reconnect (backoff %d)", index + 1, saved_backoff); @@ -1769,6 +1820,7 @@ void MQTTBridge::processPacketQueue() { _queue_count = uxQueueMessagesWaiting(_packet_queue_handle); if (_queue_count == 0) { + _queue_disconnected_since = 0; return; } @@ -1782,10 +1834,21 @@ void MQTTBridge::processPacketQueue() { MQTT_DEBUG_PRINTLN("Queue has %d packets but no slots connected", _queue_count); _last_no_broker_log = now; } + // Flush stale packets after extended disconnect + if (_queue_disconnected_since == 0) { + _queue_disconnected_since = now; + } else if ((now - _queue_disconnected_since) >= QUEUE_STALE_MS) { + QueuedPacket discard; + while (xQueueReceive(_packet_queue_handle, &discard, 0) == pdTRUE) {} + _queue_count = 0; + MQTT_DEBUG_PRINTLN("Flushed stale packet queue after %lu ms disconnected", now - _queue_disconnected_since); + _queue_disconnected_since = now; + } } return; } + _queue_disconnected_since = 0; _last_no_broker_log = 0; // Adaptive drain: burst-process when queue has backlog, gentle otherwise @@ -2273,6 +2336,31 @@ void MQTTBridge::runCriticalMemoryCheckAndRecovery() { MQTT_DEBUG_PRINTLN("Fragmentation recovery: recreating MQTT clients (max_alloc=%d, pressure %lu min)", (int)max_alloc, (unsigned long)(required_window_ms / 60000)); recreateMqttClientsForFragmentationRecovery(); } + + // Last resort: if ALL enabled slots have circuit breakers tripped for >1 hour, + // heap is likely too fragmented for TLS to ever succeed — reboot. + bool all_tripped = true; + int enabled_count = 0; + for (int i = 0; i < MAX_MQTT_SLOTS; i++) { + if (_slots[i].enabled) { + enabled_count++; + if (!_slots[i].circuit_breaker_tripped) { + all_tripped = false; + break; + } + } + } + if (enabled_count > 0 && all_tripped) { + if (_all_tripped_since == 0) { + _all_tripped_since = now; + } else if ((now - _all_tripped_since) >= 3600000UL) { + MQTT_DEBUG_PRINTLN("All MQTT slots circuit-breaker tripped for >1 hour. Restarting ESP."); + delay(100); + ESP.restart(); + } + } else { + _all_tripped_since = 0; + } } #endif diff --git a/src/helpers/bridges/MQTTBridge.h b/src/helpers/bridges/MQTTBridge.h index 643a7aca..94696ab6 100644 --- a/src/helpers/bridges/MQTTBridge.h +++ b/src/helpers/bridges/MQTTBridge.h @@ -10,6 +10,10 @@ #include "helpers/JWTHelper.h" #include "helpers/MQTTPresets.h" +#ifdef WITH_SNMP +class MeshSNMPAgent; // Forward declaration +#endif + #ifdef ESP_PLATFORM #include #include @@ -186,6 +190,17 @@ private: // Cached connection status (updated in callbacks to avoid redundant checks) bool _cached_has_connected_slots; + // Queue staleness tracking + unsigned long _queue_disconnected_since; // 0 = has connected slots + static const unsigned long QUEUE_STALE_MS = 300000UL; // Flush queue after 5 min disconnected + + // Recovery: restart ESP after prolonged total failure + unsigned long _all_tripped_since; // 0 = not all tripped + +#ifdef WITH_SNMP + MeshSNMPAgent* _snmp_agent; +#endif + // Throttle logging unsigned long _last_no_broker_log; static const unsigned long NO_BROKER_LOG_INTERVAL = 30000; // Log every 30 seconds max @@ -216,7 +231,7 @@ private: void setupSlot(int index); // Create/destroy client for a slot based on its preset void teardownSlot(int index); // Disconnect and free slot resources void maintainSlotConnections(); // Maintain all slot connections (token renewal, reconnect) - void maintainSlotConnection(int index, unsigned long now_millis, unsigned long current_time, bool time_synced, bool& reconnect_attempted); + void maintainSlotConnection(int index, unsigned long now_millis, unsigned long current_time, bool time_synced, bool& reconnect_attempted, bool& teardown_attempted); bool createSlotAuthToken(int index); // Create/renew JWT token for a slot bool publishToSlot(int index, const char* topic, const char* payload, bool retained = false); bool publishToAllSlots(const char* topic, const char* payload, bool retained = false); @@ -304,6 +319,10 @@ public: void setStatsSources(mesh::Dispatcher* dispatcher, mesh::Radio* radio, mesh::MainBoard* board, mesh::MillisecondClock* ms); + +#ifdef WITH_SNMP + void setSNMPAgent(MeshSNMPAgent* agent) { _snmp_agent = agent; } +#endif }; #endif diff --git a/variants/heltec_v3/platformio.ini b/variants/heltec_v3/platformio.ini index 68791843..28c7e26a 100644 --- a/variants/heltec_v3/platformio.ini +++ b/variants/heltec_v3/platformio.ini @@ -126,7 +126,7 @@ build_flags = -D CONFIG_MBEDTLS_CERTIFICATE_BUNDLE=y -D ESP32_CPU_FREQ=160 -D MQTT_WIFI_TX_POWER=WIFI_POWER_19_5dBm - -D MQTT_WIFI_POWER_SAVE_DEFAULT=1 + -D WITH_SNMP=1 # -D WIFI_SSID='"ssid"' # -D WIFI_PWD='"password"' # -D MQTT_SERVER='"your-mqtt-broker.com"' @@ -137,6 +137,7 @@ build_src_filter = ${Heltec_lora32_v3.build_src_filter} + + + + + + +<../examples/simple_repeater> lib_deps = @@ -146,6 +147,7 @@ lib_deps = bblanchon/ArduinoJson arduino-libraries/NTPClient JChristensen/Timezone + 0neblock/SNMP_Agent paulstoffregen/Time@1.6.1 [env:Heltec_v3_room_server] @@ -186,7 +188,6 @@ build_flags = -D CONFIG_MBEDTLS_CERTIFICATE_BUNDLE=y -D ESP32_CPU_FREQ=160 -D MQTT_WIFI_TX_POWER=WIFI_POWER_19_5dBm - -D MQTT_WIFI_POWER_SAVE_DEFAULT=1 build_src_filter = ${Heltec_lora32_v3.build_src_filter} + + diff --git a/variants/heltec_v4/platformio.ini b/variants/heltec_v4/platformio.ini index f5bb4ccc..ec1e6199 100644 --- a/variants/heltec_v4/platformio.ini +++ b/variants/heltec_v4/platformio.ini @@ -144,7 +144,7 @@ build_flags = -D CONFIG_MBEDTLS_CERTIFICATE_BUNDLE=y -D ESP32_CPU_FREQ=160 -D MQTT_WIFI_TX_POWER=WIFI_POWER_19_5dBm - -D MQTT_WIFI_POWER_SAVE_DEFAULT=1 + -D WITH_SNMP=1 # -D WIFI_SSID='"ssid"' # -D WIFI_PWD='"password"' # -D MQTT_SERVER='"your-mqtt-broker.com"' @@ -155,6 +155,7 @@ build_src_filter = ${heltec_v4_oled.build_src_filter} + + + + + + +<../examples/simple_repeater> lib_deps = @@ -165,6 +166,7 @@ lib_deps = arduino-libraries/NTPClient JChristensen/Timezone paulstoffregen/Time@1.6.1 + 0neblock/SNMP_Agent [env:heltec_v4_room_server_observer_mqtt] extends = heltec_v4_oled @@ -185,7 +187,6 @@ build_flags = -D CONFIG_MBEDTLS_CERTIFICATE_BUNDLE=y -D ESP32_CPU_FREQ=160 -D MQTT_WIFI_TX_POWER=WIFI_POWER_19_5dBm - -D MQTT_WIFI_POWER_SAVE_DEFAULT=1 build_src_filter = ${heltec_v4_oled.build_src_filter} + + diff --git a/variants/station_g2/platformio.ini b/variants/station_g2/platformio.ini index 928201df..925eb464 100644 --- a/variants/station_g2/platformio.ini +++ b/variants/station_g2/platformio.ini @@ -297,6 +297,7 @@ build_flags = -D MESH_DEBUG=1 -D CONFIG_MBEDTLS_CERTIFICATE_BUNDLE=y -D ESP32_CPU_FREQ=160 + -D WITH_SNMP=1 # -D WIFI_SSID='"ssid"' # -D WIFI_PWD='"password"' # -D MQTT_SERVER='"your-mqtt-broker.com"' @@ -307,6 +308,7 @@ build_src_filter = ${Station_G2.build_src_filter} + + + + + + +<../examples/simple_repeater> lib_deps = @@ -317,3 +319,4 @@ lib_deps = arduino-libraries/NTPClient JChristensen/Timezone paulstoffregen/Time@1.6.1 + 0neblock/SNMP_Agent