agessaman
f339beb22e
Add ota check/update/branch, display, and diagnostic commands to the observer guide; condense MQTT_IMPLEMENTATION.md and ALERTS.md; fix the nonexistent 'set bridge off' instruction; mark the T-LoRa observer envs parked; update the SNMP build-target list; move local-testing notes to MQTT_INTERNALS.md; drop MEMORY_MONITORING.md (its script never existed).
3.4 KiB
Fault Alerts
An observer can post a one-line message on a LoRa group channel when its network link or
an MQTT broker stays down too long. Alerts go out over the mesh as a flood GRP_TXT
packet, not over MQTT, since MQTT is what has failed. Anyone subscribed to the channel in
a companion app sees them as normal channel messages, sent under the node's name.
Setup
Point alerts at a private channel, test it, then turn them on.
Hashtag channel (easiest for an operator group):
set alert.hashtag ops-alerts # key = sha256("#ops-alerts")[0..15]
alert test
set alert on
Private key (openssl rand -hex 16, or paste the companion app's "Share Channel" output):
set alert.psk <32_hex_chars>
alert test
set alert on
Subscribers add a channel with the same hashtag or key.
Alerts refuse to use the Public channel, #test, or #bot, because they would spam every
node in range and trigger auto-responders. The list is BANNED_ALERT_CHANNELS[] in
src/helpers/AlertReporter.cpp. It is checked both when you
set the channel and when an alert is sent.
What triggers an alert
| Fault | Fires after | Setting |
|---|---|---|
| Network link (Ethernet or WiFi) down | 30 min | alert.wifi |
| MQTT slot down, after connecting at least once since boot | 240 min | alert.mqtt |
A recovery message goes out once when the link returns. A fault can't fire again until
alert.interval has passed (minimum 60 min).
MyObserver: WiFi down 47m (reason 201)
MyObserver: WiFi recovered after 1h3m
MyObserver: MQTT slot 1 (analyzer-us) down 4h12m
MyObserver: MQTT slot 1 (analyzer-us) recovered after 4h45m
OTA updates also post on the alert channel, so a remote operator gets feedback after
the command's reply window closes: OTA update starting, or OTA aborted: <reason> (the
bridge resumes either way). Success has no message; the node comes back on the new
version. OTA messages are not rate-limited.
CLI
| Command | Purpose | Default |
|---|---|---|
get/set alert on|off |
Master switch for fault and OTA alerts | off |
get/set alert.psk <32 hex> |
Channel key. get is serial only. Clears alert.hashtag |
unset |
get/set alert.hashtag <name> |
Derive the key from #name (# optional) |
unset |
set alert.psk / set alert.hashtag (no value) |
Clear the channel | |
get/set alert.region <name> |
Send on this region's scope instead of the default scope; bare set clears |
unset |
get/set alert.wifi <0-1440> |
Minutes before a link alert; 0 disables |
30 |
get/set alert.mqtt <0-10080> |
Minutes before a slot alert; 0 disables |
240 |
get/set alert.interval <60-10080> |
Minutes between repeats of one fault | 60 |
alert test [message] |
Send [test] <message> now, even with alert off |
alert test warns when alert is still off: a successful test doesn't mean automatic
alerts will fire.
Scope
Alerts flood on the node's default scope (region default). alert.region overrides it
for alerts only. It must name an existing region (region put <name>); an unknown name
falls back to the default scope. With neither set, alerts go out unscoped. Listeners need
the same scope to hear them.
Notes
- Outage timers live in RAM. A reboot restarts them, so the fault must last the full threshold after boot.
- Link alerts need a working LoRa radio; there is no other delivery path.