Fixes from a Codex review of the previous commit, plus the shared settings the rebase onto dev showed were still missing.
Settings that misstated or changed what runs:
- sports never read [Sports_Command] api_timeout (documented in the example config); the new field did nothing. It now sets the ESPN timeout.
- The path-byte requirement help on test, multitest and path said "bytes per hop"; the gate checks the path's total size (at least 2, or exactly 3).
- feed's allow_private_urls showed false while it inherited true from [Feed_Manager]. Fields can now name an inherit_section whose value they show when unset.
- notify_on_prefix_bytes was an int field for a value the service reads as a list (2,3), so the form refused a valid setting.
- The greeter's form defaults differed from the fallbacks it uses when the keys are missing. Both now come from shared constants.
- The MQTT weather limits accepted values the service clamps (payload 256 to 1 MiB, staleness 5 s to 7 days, length 64 to 4000).
Shared settings: the wx card now carries the [Weather] temperature, wind and precipitation units and the three temperature_*_format templates (with previews through format_temperature_high_low), next to wx's own [Wx_Command] unit overrides, which default to "Same as [Weather]". Two fields with one key in different sections are told apart as key@Section in the save payload; a bare key still resolves when only one field has it. An enum's "" option, like a blank number, removes the key. Packet capture brokers gain username, password, topic prefix, JWT timings and include_decoded; blank inheriting values are left out of block saves, since the service reads them with getint/getboolean when present.
Previews:
- They use the bot's saved config: a blank test format falls back to [Keywords] test, multitest lays out paths per condense_paths, and MQTT applies passthrough_max_length.
- A quoted empty template ("") counts as blank, as the commands strip the quotes first.
- A str.format width or precision over 500 isn't rendered, so `{sender:100000000}` can't make the endpoint build a huge string.
- The page invalidates an in-flight preview as soon as the field changes, and shows the server's note about fallbacks instead of guessing it.
The endpoint inventory golden gains the preview route (401 for anonymous requests when a password is set).
The Plugins page now has a field for every setting the plugins read. I compared each plugin's settings_schema against config.ini.example and the keys the code reads; fourteen plugins had gaps (greeter, trace, stats, path, wx/gwx, rain, test, multitest, sports, worldcup, cmd, feed, the webhook, weather, packet capture, earthquake and prefix-collision services) and the World Cup service had no form at all. Webhook URLs, bot tokens and channel keys use the password field. A guard test needed four [Rain_Command] keys documented in config.ini.example.
Template fields get a { } button that opens a reference of the placeholders they take, with click-to-insert. Fields rendered by the piped engine (test's response_format, path's reply_prefix) also list the filters and conditionals. Every template field previews as you type: piped fields against a 2-byte, a 1-byte and a direct sample message, so a conditional's effect shows before saving, and the str.format fields (greeter, mesh info, multitest, MQTT weather) as their plugin renders them, including what it sends when the template has a mistake. The specs and renderers live in modules/template_reference.py; the preview endpoint only renders and never saves. The greeter's channel_greetings parser and multi-part split moved into module functions so the command and the preview share them.
Fixes the new fields would otherwise have exposed:
- Clearing a number field saved `key =`, which makes getint/getfloat raise and the plugin fail to load. A cleared number now removes the key.
- The weather service raised in __init__ when only some blitz_area_* corners were set. It now needs all four and logs a warning otherwise.
- A blank MQTT weather json_template saved an empty template, so every reply said no message had arrived. Blank now saves the default.
S7's units item. GlobalWxCommand and WeatherService each read [Weather] temperature_unit, wind_speed_unit and precipitation_unit, lowercased them and fell back to fahrenheit/mph/inch with the same three warnings; weather_common.load_open_meteo_units does that once, next to load_open_meteo_model. Both still accept kn. wx keeps its own NOAA unit reading (_unit_setting), which has different fallbacks and sources. Old-vs-new over 5,000 random configs matched on values and warnings.
Phase 8's mechanical part. The 19 asyncio.get_event_loop() calls that run directly in an async def (found by walking each file's AST; calls inside nested sync functions or lambdas don't count) become asyncio.get_running_loop(). Inside a running coroutine both return the same loop; get_running_loop says so and never creates one. The five calls in sync functions (maintenance.run_data_retention, radio_link._handle_serial_probe_error, scheduler._run_in_current_loop and run_scheduler, weather_service._run_daily_forecast) can run on threads without a loop, where the two differ, so they stay. No test patches get_event_loop.
Codex's final review found that moving code out of modules dropped imports that local plugins could import from them, for example `from modules.service_plugins.discord_bridge_service import EventType`, which would make such a plugin fail to load. Comparing every top-level name bound on dev against the branch (both trees imported, so names under TYPE_CHECKING are excluded) found 24 in 11 modules: EventType, copy and contextlib in both bridges; sha256, Iterable and hmac_mod in message_handler; RotatingFileHandler, colorlog, contextlib, contextvars and functools in core; hashlib and re in utils; contextlib in feed_manager, earthquake_service and weather_service; Any in cmd and help; time in path. Each is re-imported exactly as dev imported it.
The public-import guard only covered `from modules... import` statements, so it missed these. It now records every unconditional top-level import and assignment, regenerated from dev's tree (461 → 1,597 names), and fails if the discord bridge stops exposing EventType.
_collect_alert_entry parses one ATOM entry and appends it to the new-alert list (its early continues become returns; the append stays before the debug line, so a timestamp the log cannot format still leaves the alert queued, as before); _send_new_alerts sends them two seconds apart. Each loop keeps its own try/except in place. AST comparison shows the helpers and the rest of _check_weather_alerts identical to the original apart from continue → return.
_forecast_location_name reverse geocodes the bot's position once and caches it; _append_tomorrow_forecast appends tomorrow's sky and high/low. Both bodies and the rest of _get_weather_forecast_result are AST-identical to the original statements, called at the same points inside the same try, so its error handling and log lines are unchanged.
TelegramBridgeService returned from __init__ as soon as it found no api_token (or no HTTP library), before setting message_queues, send_times, http_session and _queue_processor_task. start() then returned early as intended, but stop() read _queue_processor_task and raised AttributeError, so every shutdown logged "Failed to stop service 'telegram_bridge'". DiscordBridgeService had the same gap on its no-HTTP-library path. Both now set that state before any early return.
New tests start and stop each self-disabled bridge and check that only the reason it disabled itself is logged as an error; all three fail on dev.
Codex review of 9b86cd63: ChannelBridgeBase ran the DM guard and the channel lookup before parsing "sender: message", but both bridges evaluated `':' in text` first. A non-string text (None, an int) on a DM or unmapped channel used to log "Error handling mesh channel message" with exc_info and now logged a DEBUG skip instead. The split runs first again. The bridge replay now includes None, int, bytes and list texts; it differs from the pre-bridge code on 9b86cd63 and matches it after this commit.
start, stop, on_transport_reconnected, _on_mesh_channel_message (payload copy, DM guard, sender split, profanity filter) and _process_message_queues (retry, backoff, age limit) move to ChannelBridgeBase in modules/service_plugins/channel_bridge_utils.py; the _utils name keeps the service loader from taking the base class for a service. Each bridge keeps what differs through small hooks: its channel lookup (Telegram ignores a leading '#', Discord doesn't), text preparation (Discord formats mentions before the profanity filter, Telegram builds its HTML after it), delivery, rate limit (25 per minute per webhook vs one per second per chat), HTTP session setup and posting, and its own wording for the old-message, retry and processor-error lines through format templates. The requests fallback stays: it still runs when there is no aiohttp session.
Imports the two modules no longer use are gone (copy, contextlib, meshcore's EventType); BaseServicePlugin, censor and contains_profanity stay as re-exports.
Checked by driving both bridges through 48 scripted scenarios (four configs each, async and requests paths, a fake clock, scripted 2xx/429/5xx/timeout/error responses and Discord rate-limit headers, start/reconnect/stop): every post, logger and meshcore call, queue, send time and rate-limit record is identical before and after, apart from bound-method reprs naming ChannelBridgeBase. Eight mutations across the base and both bridges each change the output.
#323 ported this branch's weather slices to dev and #333 built the wx/gwx convergence on them, so dev's wx, gwx, weather_common and location are a superset of the branch's: I took dev's side there, with the characterization goldens #333 regenerated. The rest:
- core.py: kept the template file and carried dev's knots option into modules/templates/default_config.ini.
- utils.py: kept the shared _geocode_city_steps and applied dev's zero-coordinate fix (is not None instead of truthiness) to its seven checks.
- weather_service.py: kept run_periodic/_cancel_tasks and took dev's per-point NWS coverage cache.
- solarforecast: restored self.geolocator, which #323 kept for local plugins and the merge would have dropped.
fix/gwx-missing-current returns the fetch error as a bare string when the current temperature is missing, and this branch moved that code into a method whose callers unpack (reply, current conditions). Git merged it without a conflict; the merged early return now returns the pair, so the reply is the fetch error again instead of an unpacking error.
A review pointed out that Open-Meteo documents WMO code 97 as a heavy thunderstorm, and neither gwx nor the weather service knew it: the reply showed an unknown condition with the generic emoji and gwx gave no thunderstorm warning. It now has a description in every language, the ⛈️ emoji and the thunderstorm warning.
A no-coverage reply (400/404) from the NWS alerts API set a single flag on wx and on the weather service that skipped every later alert request for the rest of the run, so one wx request for a city abroad silently removed alerts from every US reply after it. NWSNoCoverageCache remembers no-coverage per point rounded to two decimals, for 24 hours, at most 256 points (least recently used dropped first); a successful response clears its point, and the warning is logged once per point. The weather service polls one position, so for it the change means a retry after 24 hours instead of never.
This brings the weather slices of my long-running refactor branch to dev ahead of converging wx and gwx, so the convergence PRs build on one copy of the code instead of rewriting it twice.
- `modules/weather_common.py` (WeatherCommandMixin): code wx and gwx shared by copy (MQTT and WXSIM source lookup, multi-day sending, high/low formatting, the forecast-option parser, the Open-Meteo model loader).
- `modules/nws_alerts.py`: one NWS alert entry parser for wx and the weather service, keeping each caller's own Special Weather Statement rules.
- `modules/http_retry.py`: one retrying session builder for the weather HTTP clients.
- wx: get_noaa_weather, the hourly, tomorrow and multi-day formatters, _add_period_details and the extract_* readers broken up or made table-driven; NOAA fetches go through `_noaa_fetch`.
- gwx: one fetch path; `_open_meteo_current` and `_open_meteo_daily_tail` split out of get_open_meteo_weather.
- solarforecast: `_format_forecast` built from one timestamp parser and one day builder (418 → 155 lines).
- aurora, rain, wx, gwx: companion and bot location lookups through `modules.location` (which gains optional logging arguments).
The changes outside these files that the refactor branch made (the base command's enabled_attr hook, the service base's run_periodic and _cancel_tasks) stay out: aurora and rain keep their can_execute overrides and the weather service its own loops, exactly as on dev.
Tests: characterization goldens over real captured NOAA, NWS, Open-Meteo and forecast.solar responses (tests/characterization, fixtures in tests/fixtures), 50 tests and several hundred cases, all of which also pass against dev's unmodified weather code. Each refactor step was additionally checked on the branch with an old-vs-new differential run (tens of thousands of generated inputs, including log calls and exceptions) and reviewed by codex.
Brings in #309, #311, #314, #317, #319 and #320. Conflicts: command_manager's imports (kept both `inspect` and `math`) and wx's alert send, where I kept the S1 removal of the dead `weather_data[3]` statement and took #319's `skip_user_rate_limit=True`. The NOAA wx goldens are regenerated because #320 changes the wind text (`💨N0` → `⬆️N0`); nothing else in them moved.
Review found the same reply-limiter drop in two more places: the earthquake service's link post right after its alert, and the announcements command's 'sent' confirmation right after the announcement. Both now skip the reply limiter. multitest called wait_for_tx itself just before send_response, which waits again; with wait_for_tx claiming the slot that pre-wait made the reply wait a full extra interval, so it goes. A test that only documented RateLimiter's behavior, and guarded nothing, is replaced by an earthquake test that drives _check_earthquakes.
Eight stop() blocks did 'if task: task.cancel(); with suppress(CancelledError): await task'. BaseServicePlugin._cancel_tasks(*tasks) does the same, in order, skipping None, and like awaiting directly it re-raises an error a task had already finished with. Callers that reset their task attributes to None still do; the weather service still does not. DARC MoWaS (cancels without waiting) and packet capture (skips finished tasks) keep their own loops.
The earthquake poll and the weather service's alert, rain-nowcast and lightning loops were the same twelve lines: run the work, sleep the configured interval, stop on cancellation, and on any other error log it and wait 60 s. BaseServicePlugin.run_periodic now holds that loop; the four loop methods keep their names (tests look for them among spawned tasks) and their start logs, and pass the interval as a callable so it is still read every round. The error lines render the same text, now always through %s formatting. The World Cup loop (wakes early on fastcast pushes) and the bridges' queue processors have their own shapes and stay as they are.
A second review of the restart fix found more of the same class. The map uploader's stop() removes its file log handler and start() never added it back, so after one restart it logged nothing to file; start() now restores it. DARC MoWaS set _running before binding its port, so a port in use left it dead but healthy, and werkzeug's make_server calls sys.exit when the port is taken, which got past every except Exception and stopped the bot; start() now binds first and turns that exit into an OSError.
Boot-time starts go through _start_service_at_boot, which records a restart failure when start() raises, so that service also waits out the backoff, and no longer reports a disabled service as a configuration problem. The health-loop step is _restart_unhealthy_services. New tests cover both, the bridges' session cleanup and the map uploader restart; all 13 lifecycle tests fail on dev.
Review found that marking a not-running service inactive for good also retired services that only returned early for a transient reason: packet capture and the map uploader give up after waiting 30 s for the radio, and the bridges return when meshcore is momentarily unset, which is exactly when the health loop restarts them. Such a service now records a restart failure, so the next attempt waits out service_restart_backoff_seconds; a transient failure recovers, a missing configuration costs one attempt per backoff instead of one every 5 s. The restart decision is _service_restart_due, which the tests now exercise directly.
Also from the review: packet capture and the map uploader never cleared the should_exit flag stop() sets, so after a restart their background loops exited at once while the service reported healthy; start() now clears it. The Discord and Telegram bridges close the HTTP session they open when they return early because meshcore is not available.
The health loop restarted every enabled service whose is_healthy() was False. A service that returns from start() without running because it has nothing to do (Discord or Telegram with no channels, the map uploader with no key) is never healthy, and its restart raised nothing, so it counted as a success and cleared the backoff: the service was restarted every 5 seconds, logging a warning each time.
Services whose start() returns without running are now recorded as inactive, logged once, and skipped by the health loop. A restart that leaves a service running but unhealthy records a failure, so the next attempt waits out service_restart_backoff_seconds.
Restarts also double-subscribed: services subscribed to meshcore events in start() and never unsubscribed (the comments said meshcore cleans up; it does not). BaseServicePlugin gains _subscribe and _unsubscribe_all; the Discord, Telegram, map uploader, packet capture and prefix-collision services subscribe through it, unsubscribe in stop(), and drop the old instance's subscription before re-subscribing after a transport reconnect. Tests use meshcore's real EventDispatcher and fail on dev.
wx's get_weather_alerts_noaa and the service's _parse_alert_entry each carried a copy of the same Atom entry parser: title, summary, NWSheadline, the title-based event and time extraction, the CAP child scan and the severity and urgency inference. modules/nws_alerts.py now holds it once as parse_alert_fields. The two Special Weather Statement classifiers stay distinct, as WX_SPECIAL_RULES and SERVICE_SPECIAL_RULES (headline classifier, summary keyword table, and whether to run wx's extra localName scan), so each caller produces exactly what it did before. The service keeps its link extraction, id and None-on-error contract; wx keeps its title-only fallback and warning.
Also dropped: wx's expiry-time block, which parsed nothing and assigned nothing, and the redundant endswith(':tag') checks in the CAP scan, which the substring checks already covered. The localName pass catches Exception instead of everything. The characterization goldens are unchanged.
A codex review of the branch found seven places where earlier refactor commits changed behavior. Each is restored here with a test:
- base_service imported flood_scope relatively; the local-service loader also runs that file as local_services.base_service, where the import cannot resolve. Now absolute.
- Radio-health fields were initialized after commands and services were built, so a plugin constructor reading is_radio_offline raised. They are now set before any plugin.
- Removing the per-command can_execute overrides made 31 commands honor skip_channel_check, which their overrides had dropped. BaseCommand.honors_skip_channel_check (False on exactly those commands) keeps them enforcing their channel list.
- CommandManager.get_plugin_metadata, WxCommand.get_weather_for_zipcode and PluginLoader.validate_plugin were public compatibility entry points; they are back.
- feed_manager stopped exporting DEFAULT_MAX_FEED_RESPONSE_BYTES and DEFAULT_MAX_PARSED_FEED_ITEMS despite the commit saying it re-exported them; it does now.
- wx and gwx logged companion-location lookup failures at WARNING; the shared helper logged at debug. It now takes the level.
- The INFO 'MESSAGE ROUTING' / 'DIRECT MESSAGE' lines for pubkey-correlated DMs were removed with the dead path_info computation; they are logged again by a log-only helper.
GlobalWxCommand and WeatherService had identical _load_weather_model methods. The logic is now weather_common.load_open_meteo_model(config, logger); both methods delegate, so the existing tests that call them are unchanged.
MeshMessage.effective_outgoing_flood_scope, CommandManager's per-channel and override readers and its config_section branch, and BaseServicePlugin.get_mesh_flood_scope each scanned config.ini themselves. modules/flood_scope.py now owns those reads: channel_scope_entry (stripped flood_scope.<channel>, None with no entry), outgoing_override, and section_flood_scope (normalized, None when empty). Callers keep their own contracts on top: MeshMessage still reports raw values, CommandManager still normalizes, and None still means 'apply the override later' in resolve_channel_send_scope. MeshMessage.is_global_flood_scope delegates to is_global_marker, the identical predicate. The scope matrix golden file is unchanged.
wx, rain and the weather service each built the same requests session (two retries on connection errors and 5xx for GET, 0.3 s backoff, status codes not raised, 10x20 connection pool). It now lives in modules/http_retry.py; each class keeps its _create_retry_session as a one-line delegate, and each keeps its own session lifetime (rain still builds one per call).
An oversized webhook or channel body was handed to the firmware whole. The device dropped it without an ack, retries stalled the transport, and the radio looked dead. Split to the frame budget, keep links intact, and tag multi-part sends so they can be reassembled.
Reserve the wx location prefix from that same budget, and let NOAA alert polling be turned off without disabling the rest of the weather service.
waev.app rejects a token whose exp is more than an hour past its iat, so the
project-wide 24-hour default could never authenticate there — the operator saw
only a bare auth failure with no hint that the lifetime was the problem, while
the same public key worked from meshcoretomqtt.
waev.app hosts now resolve to a 3600s TTL with renewal at 3500s when no value
was configured for them, per broker or globally; a configured value still wins,
and the substitution is logged. Matching is the apex and its subdomains only, so
a lookalike host like waev.app.example.com does not qualify.
The published packet payload mixed clocks: "timestamp" was UTC ISO 8601
with a Z suffix, but the "time" and "date" fields beside it were rendered
from a naive datetime.now(), i.e. the host's local wall clock. A consumer
reading those two off a bot in a non-UTC zone sees a skew of exactly that
zone's UTC offset and flags the observer's clock as wrong.
Local time was never the intended reading. The original script took time
and date off the firmware log line, and mctomqtt sets the device clock
from calendar.timegm(), so upstream both fields are already UTC.
All three now render one aware UTC instant, so they cannot disagree with
each other or straddle a second boundary. _utc_iso_timestamp() takes an
optional instant to make that possible; its no-arg behavior is unchanged.
- Extracted `modules/alert_format.py` as the single NWS alert formatter, replacing four copies of the event-type abbreviation table and two of the time compactor. `!wx alerts` and the proactive `WeatherService` broadcasts now localize from one code path, so a Russian bot no longer answers `!wx alerts` in English while its proactive alerts are Russian.
- Stopped leaking translation key paths into mesh broadcasts. `Translator.translate` returns the dotted key when a lookup misses in both the locale and the English fallback, which is right for development but reached the air in production: an unclassifiable NWS title rendered as `⚪Hazardous services.weather_service.event_types.Unknown`, an unmapped WMO code as `services.weather_service.weather_descriptions.4`, and an oddly-cased `wind_speed_unit` as `services.weather_service.wind_speed_units.KMH`. `alert_format.translate_or()` carries an English default at each site, and `WeatherService` now normalizes and validates its three `[Weather]` unit settings the way `GlobalWxCommand` already did.
- Fixed alert expiry rendering in every locale. The formatter rendered a timestamp to a string and re-parsed its own output with `(\d+)(AM|PM)` against a hardcoded English month list, so translated months took the wrong branch and truncated mid-string. Times now carry parsed parts and render through a per-locale `common.alerts.time_12h` template — the space before AM/PM was correct (Russian writes "6 дня", not "6дня"); the downstream regex was the bug.
- Restored month abbreviation. `_compact_time` iterated over abbreviations and replaced them in the string instead of mapping full names, so English stopped shortening "June 28" and Russian replaced the "Jun" inside "June", leaving a stray Latin "e" (`июнe 28`). Reuses the existing `common.date_time.month_abbreviations` rather than the duplicate `services.weather_service.months` block.
- Made `!gwx` display units follow `[Weather]` config instead of the response language. Visibility switched on `base_language != 'en'`, so `language = ru` with the default `temperature_unit = fahrenheit` printed Fahrenheit beside kilometers, and `en-GB` was forced to miles. Pressure is a locale convention rather than a metric/imperial split, so each catalog names its own via `commands.gwx.pressure_unit` — previously every non-English locale inherited mmHg from the English catalog, whose `pressure_mmhg` string contained Russian text, giving German and French users Cyrillic pressure units.
- Let localized `H`/`L` labels reach a standard install. `config.ini.example` shipped the three `temperature_*_format` keys uncommented with literal `H:`/`L:`, and a config value always beats the new locale-aware default, so a Russian bot built from the documented example still rendered `H:47°C L:33°C`. The example now uses the `{high_label}`/`{low_label}` placeholders, which were documented in the docstring but not in the file.
- Routed high/low labels through the reply's translator. `_format_high_low` passed `bot.translator`, so with `auto_detect_language` on, an English-default bot answering a Russian sender localized the rest of the line but not `H:`/`L:`. Added `BaseCommand.response_translator` for this, replacing `wx_international`'s reach into the private `_response_translator` ContextVar.
- Fixed a byte-budget overrun in `!gwx`. The guard on the extra conditions block compared a character count against a byte-derived budget while the rest of the function used `_count_display_width`; Cyrillic is two bytes per character, so the block was appended after the budget was spent.
- Reverted nine `commands.gwx` English rewordings that were not localization work, including the configuration hint in `mqtt_weather_no_subscriber` — dropping it left a mis-configured operator with no pointer to the two keys they need.
- Fixed the Russian `visibility` string, which said "км" on the miles key — the same locale/config conflation as the code bug, in the data. Shortened the Russian event-type abbreviations, which were full words consuming a quarter of the 130-byte budget at two bytes per character.
- Added `commands.wx.hourly_not_available`, missing from every catalog so `!wx hourly` printed the raw key path. Predates this branch; found while auditing every translation key the weather modules reference.
- Moved alert strings to `common.alerts.*` and wind directions to `common.wind_directions.*`, since a command and a service both read them.
This commit addresses several issues related to MQTT connections. It prevents MQTT brokers from entering a reconnect storm by ensuring that the packet-capture watchdog only triggers a reconnect when necessary, avoiding duplicate CONNACKs and spurious disconnects. Additionally, it ensures that renewed MQTT auth tokens are properly enforced by implementing a clean reconnect process.
New configuration options are introduced: `mqttN_keepalive` for setting the PINGREQ interval per broker, and `mqttN_jwt_reconnect_on_renew` to control reconnect behavior after token renewal. Documentation has been updated to reflect these changes and to clarify the importance of unique client IDs for brokers in the same cluster.
- Add _translate() helper to WeatherService using self.bot.translator
- Replace all hardcoded English strings with services.weather_service.* keys
- Add services.weather_service section to en.json (fallback for all locales)
- Add ru.json with full Russian translation (~450 keys)
Enhance the handling of the global IATA configuration to ensure that blank values are treated correctly, preventing unintended namespace pollution. Introduce methods for claiming and releasing the neighbors discovery cycle, ensuring that overlapping requests are managed effectively. Update the NeighborsCommand to utilize these new methods, improving the accuracy of cooldown management for users. Additionally, add tests to validate the new behavior and ensure that cooldowns are respected during busy and disabled states.
Add a new state key for tracking the last attempt to discover neighbors, allowing the service to persist this information across restarts. This change ensures that airtime is managed effectively, preventing unnecessary retries after a failed cycle. The implementation includes methods for loading and saving the neighbors attempt state, enhancing the overall reliability of the packet capture service. Additionally, update tests to validate the new state management functionality.
Move the 15-minute gap between cycles from the DM command to the service, as
MIN_CYCLE_GAP_SECONDS. The command was one caller among several: the scheduler
retried a failed cycle on its own 300s backoff, keyed off last_neighbors_publish
which a failed cycle never stamps, so a discover round whose acknowledgement was
lost went back on the air every five minutes. run_neighbors_cycle now refuses a
cycle inside the gap whoever asks, and the scheduler's retry backoff waits out
whatever remains of it. Failures that never reached the radio still stamp
nothing, so re-checking a disconnected radio stays on the short backoff.
The command asks the service for the remaining wait instead of computing its own,
so the two cannot drift, and rewinds the sender's per-user cooldown to expire
with the shared one. The command manager records an execution before calling
execute(), so a refusal had been consuming the sender's full 15 minutes: told to
wait one more minute, they would retry and be refused for another fourteen.
Rewound rather than cleared — a refusal reply is airtime too.
Follow-up to 7cffe1d; all three confirmed against meshcore 2.3.8.
Charge the node-wide cooldown for the transmission, not the result.
last_neighbors_publish is stamped only by a cycle that completes, so a discover
request whose acknowledgement was lost spent the airtime and left the clock at
zero -- another sender could immediately start a second round. The service now
stamps last_neighbors_attempt before the request, and the command rations on
whichever stamp is later. A cycle that bails out before touching the radio still
records nothing, so retries stay possible.
Restore the contact path when req_regions_sync returns None. send_anon_req gives
up early when change_contact_path reports an error -- which is also what a lost
acknowledgement for an applied path change looks like -- and that return skips
the library's own reset_path. req_regions_sync collapses it to None, previously
treated as a plain no-response. On the common "neighbour did not answer" path
the extra reset is a redundant device command: no airtime, idempotent, and it
re-syncs the contact cache.
Stop reporting a rejected restore as a success. reset_path returns an ERROR
event for a device rejection or its own response timeout rather than raising, so
the helper logged "restored flood path" either way and hid a contact left pinned
to zero-hop. It now inspects the event and warns with the reason.
Four review findings on this branch, all confirmed against the code and the
installed meshcore 2.3.8:
Single-flight the discovery cycle. The command guarded only its own task, so
the scheduler's independent call could overlap a manual cycle and each round
would collect into the other's discover window. run_neighbors_cycle is now a
guard around the cycle body, refusing whichever trigger arrives second.
Make the 15-minute cooldown per node rather than per sender. The base class
rations per user, but the cost here is mesh airtime: users could take turns and
keep the radio discovering continuously. Measured from the last cycle that
produced a result, so the scheduler's cycles count and a cycle that bailed out
without transmitting does not start the clock.
Window the neighbours evidence label. The combined viewer applied `days` only
to mesh_connections while reading every lifetime row from neighbor_links, which
is deliberately never pruned — so a link last heard years ago kept claiming a
recent path-derived edge was a current direct neighbour.
Match that label on full public keys too. MeshGraph.add_edge does not promote a
1-byte edge that has no public key, but still fills in the keys discovery
supplied, so the 3-byte prefix comparison alone left confirmed neighbours
labelled singlebyte. Truncating our keys to 2 chars instead would relabel every
other node sharing that byte.
Restore a contact's flood path after an interrupted scope request. send_anon_req
pins a path-less contact to zero-hop and restores it after the send with no
try/finally, so our own budget cancelling the request left the contact pinned
and every later message to it sent direct-only.
The meshcore>=2.3.8 pin needed no change: PyPI publishes 2.3.8 now.
Port the observer firmware's neighbours feature into the bot's packet capture
service, by way of meshcore-packet-capture (upstream PRs #42/#43). On a long
interval the bot asks which repeaters it hears directly and records each
confirmed link with its measured SNR.
This is the strongest link evidence the bot collects: a first-party RF
measurement between two full 32-byte public keys. Path inference works from
1-3 byte prefixes with no keys, and complete_contact_tracking.hop_count
over-claims zero-hop (800 claimed vs 68 corroborated on the live database).
modules/neighbors_discovery.py keeps upstream's public names so its fixes and
tests stay portable. Two deliberate divergences:
- No command_lock plumbing. _SerializedCommands in modules/core.py already
serialises and paces every radio command, strictly more than upstream's
reentrant lock did.
- neighbors_collect_scopes defaults off. Upstream's zero-hop scope probe
relies on a neighbour not being a known contact; this bot tracks contacts,
and for a repeater with no stored path the library reaches zero-hop by
calling change_contact_path() then reset_path() -- mutating the device's
contact table per neighbour. Scope requests also hold the radio lock for
their whole round trip (~25s), stalling bot replies. The default cycle is
one command plus a passive listen window, during which the bot stays
responsive.
Evidence lands in neighbor_links and neighbor_observations (migration 22)
rather than mesh_connections, which cannot persist provenance. The viewer
exposes it as evidence=neighbors on /api/mesh/edges and a Neighbours Only
mode on the mesh page, with populated public keys and real SNR; confirmed
neighbours also relabel edges in the combined view and count as
provenance-trusted when framing the initial map.
neighbors_enabled is the single switch. Every enabled broker publishes once
it is on (mqttN_neighbors defaults true; set false to hold one back). The
topic derives from each broker's packets topic with the last segment swapped,
so a templated broker gets meshcore/{IATA}/{PUBLIC_KEY}/neighbors -- the
topic the firmware uses -- instead of an unrelated flat one. A derived
location-routed topic is skipped with a warning when no iata is set, rather
than publishing into meshcore/XYZ/... on a shared namespace. Snapshots are
non-retained: heard_secs_ago is relative to publish time, so a retained copy
would read as current days later.
Also adds a DM-gated `neighbors` command (the 12h interval floor makes
waiting for the scheduler impractical), which acks immediately and reports in
a second message once the window closes.
Requires meshcore >= 2.3.8 for send_node_discover_req / req_regions_sync.
- Introduced a locking mechanism to ensure journal mode is initialized once per configuration section, preventing redundant database operations.
- Added a validation function to check for required repeater tables, raising an error with actionable messages if any are missing.
- Updated the DBManager to handle journal modes more efficiently, ensuring that only persistent modes are applied across connections.
- Refactored the RepeaterManager to utilize the new validation function, improving error handling during initialization.
- Enhanced tests to cover new journal mode behaviors and validate repeater table existence, ensuring robust database management.
Four code paths that produced no result, or a wrong one, without ever
logging an error. Each looked correct in review and failed only against
real inputs.
- packet_capture: read RAW_DATA from the "payload" field. meshcore's
reader dispatches RAW_DATA as {"SNR", "RSSI", "payload"}, but the
handler looked for "data" and "raw_hex" and returned early when it
found neither, so every RAW_DATA event was dropped with no log line.
Existing tests passed because they fabricated {"data": ...}. The
sibling RX_LOG handler already preferred "payload".
- packet_capture: key both RF correlation caches on uppercase hex.
RX_LOG_DATA cached prefixes in the lowercase the wire supplies, while
RAW_DATA uppercased before looking up, so the two never correlated and
no RAW_DATA packet ever picked up cached SNR/RSSI. This was masked
until now by the drop above. The event's own SNR/RSSI are also folded
into the lowercase spelling the rest of the pipeline reads, via an
explicit None check so a legitimate 0 is not discarded.
- path_inference: run the bot path-validation bonus. Its scoring block
sat inside an `if (len(decoded_path_hex) % path_n) != 0` branch, and
for a uniform-width path that remainder is always 0, so the body was
unreachable. A candidate with a perfect historical path match scored
0.0 instead of 0.3. The web sibling was indented correctly.
- path_inference: compare path segments case-insensitively in the web
variant. decoded_nodes came from the uppercased path_context while
stored_nodes were lowercased, so a segment match was impossible for
any path containing a hex letter; a perfect match scored 0.15 rather
than 0.3. This raises some web decode confidence values, so the
module docstring's byte-for-byte parity note now records the
deliberate exception. No node resolves to a different repeater purely
from case.
Also resolves node hop position by index rather than by value, so a
1-byte prefix appearing twice in one path no longer gives every
occurrence the neighbours and final-hop status of the first. This is
threaded through the shared engine and used by the web decode path,
which returns a list. The bot `path` command deliberately does not pass
it: repeater_info is keyed by node_id, so resolving each occurrence
separately would let the later hop overwrite the earlier one's display.
Fixing that needs repeater_info re-keyed by hop index.
agessaman