Fixes from a Codex review of the previous commit, plus the shared settings the rebase onto dev showed were still missing.
Settings that misstated or changed what runs:
- sports never read [Sports_Command] api_timeout (documented in the example config); the new field did nothing. It now sets the ESPN timeout.
- The path-byte requirement help on test, multitest and path said "bytes per hop"; the gate checks the path's total size (at least 2, or exactly 3).
- feed's allow_private_urls showed false while it inherited true from [Feed_Manager]. Fields can now name an inherit_section whose value they show when unset.
- notify_on_prefix_bytes was an int field for a value the service reads as a list (2,3), so the form refused a valid setting.
- The greeter's form defaults differed from the fallbacks it uses when the keys are missing. Both now come from shared constants.
- The MQTT weather limits accepted values the service clamps (payload 256 to 1 MiB, staleness 5 s to 7 days, length 64 to 4000).
Shared settings: the wx card now carries the [Weather] temperature, wind and precipitation units and the three temperature_*_format templates (with previews through format_temperature_high_low), next to wx's own [Wx_Command] unit overrides, which default to "Same as [Weather]". Two fields with one key in different sections are told apart as key@Section in the save payload; a bare key still resolves when only one field has it. An enum's "" option, like a blank number, removes the key. Packet capture brokers gain username, password, topic prefix, JWT timings and include_decoded; blank inheriting values are left out of block saves, since the service reads them with getint/getboolean when present.
Previews:
- They use the bot's saved config: a blank test format falls back to [Keywords] test, multitest lays out paths per condense_paths, and MQTT applies passthrough_max_length.
- A quoted empty template ("") counts as blank, as the commands strip the quotes first.
- A str.format width or precision over 500 isn't rendered, so `{sender:100000000}` can't make the endpoint build a huge string.
- The page invalidates an in-flight preview as soon as the field changes, and shows the server's note about fallbacks instead of guessing it.
The endpoint inventory golden gains the preview route (401 for anonymous requests when a password is set).
The Plugins page now has a field for every setting the plugins read. I compared each plugin's settings_schema against config.ini.example and the keys the code reads; fourteen plugins had gaps (greeter, trace, stats, path, wx/gwx, rain, test, multitest, sports, worldcup, cmd, feed, the webhook, weather, packet capture, earthquake and prefix-collision services) and the World Cup service had no form at all. Webhook URLs, bot tokens and channel keys use the password field. A guard test needed four [Rain_Command] keys documented in config.ini.example.
Template fields get a { } button that opens a reference of the placeholders they take, with click-to-insert. Fields rendered by the piped engine (test's response_format, path's reply_prefix) also list the filters and conditionals. Every template field previews as you type: piped fields against a 2-byte, a 1-byte and a direct sample message, so a conditional's effect shows before saving, and the str.format fields (greeter, mesh info, multitest, MQTT weather) as their plugin renders them, including what it sends when the template has a mistake. The specs and renderers live in modules/template_reference.py; the preview endpoint only renders and never saves. The greeter's channel_greetings parser and multi-part split moved into module functions so the command and the preview share them.
Fixes the new fields would otherwise have exposed:
- Clearing a number field saved `key =`, which makes getint/getfloat raise and the plugin fail to load. A cleared number now removes the key.
- The weather service raised in __init__ when only some blitz_area_* corners were set. It now needs all four and logs a warning otherwise.
- A blank MQTT weather json_template saved an empty template, so every reply said no message had arrived. Blank now saves the default.
S4. BaseCommand and the 40 bundled overrides that took no argument now take `message: MeshMessage | None = None`, matching joke and rain, which already use the message. None of the 40 bodies read it, so their help text is unchanged. help and command_manager keep their TypeError retry, since local plugins may still implement get_help_text(self). For bundled commands the retry no longer runs: `help <command>` calls get_help_text(message) once instead of failing and calling it again. A new test pins the signature for every bundled command class. The offline smoke run against the real config produces the same help text for all 46 commands.
Brings in #309, #311, #314, #317, #319 and #320. Conflicts: command_manager's imports (kept both `inspect` and `math`) and wx's alert send, where I kept the S1 removal of the dead `weather_data[3]` statement and took #319's `skip_user_rate_limit=True`. The NOAA wx goldens are regenerated because #320 changes the wind text (`💨N0` → `⬆️N0`); nothing else in them moved.
Review found the same reply-limiter drop in two more places: the earthquake service's link post right after its alert, and the announcements command's 'sent' confirmation right after the announcement. Both now skip the reply limiter. multitest called wait_for_tx itself just before send_response, which waits again; with wait_for_tx claiming the slot that pre-wait made the reply wait a full extra interval, so it goes. A test that only documented RateLimiter's behavior, and guarded nothing, is replaced by an earthquake test that drives _check_earthquakes.
A codex review of the branch found seven places where earlier refactor commits changed behavior. Each is restored here with a test:
- base_service imported flood_scope relatively; the local-service loader also runs that file as local_services.base_service, where the import cannot resolve. Now absolute.
- Radio-health fields were initialized after commands and services were built, so a plugin constructor reading is_radio_offline raised. They are now set before any plugin.
- Removing the per-command can_execute overrides made 31 commands honor skip_channel_check, which their overrides had dropped. BaseCommand.honors_skip_channel_check (False on exactly those commands) keeps them enforcing their channel list.
- CommandManager.get_plugin_metadata, WxCommand.get_weather_for_zipcode and PluginLoader.validate_plugin were public compatibility entry points; they are back.
- feed_manager stopped exporting DEFAULT_MAX_FEED_RESPONSE_BYTES and DEFAULT_MAX_PARSED_FEED_ITEMS despite the commit saying it re-exported them; it does now.
- wx and gwx logged companion-location lookup failures at WARNING; the shared helper logged at debug. It now takes the level.
- The INFO 'MESSAGE ROUTING' / 'DIRECT MESSAGE' lines for pubkey-correlated DMs were removed with the dead path_info computation; they are logged again by a log-only helper.
32 commands overrode can_execute only to return False while their own <name>_enabled flag was off and otherwise defer to the base. BaseCommand now has an enabled_attr class attribute naming that flag, which can_execute checks first. Each command keeps reading its flag exactly as before (same section, default and legacy aliases), and code and tests that set help_enabled, webviewer_enabled and the like still work, since the flag is read at call time.
Only overrides that were exactly 'if not self.<flag>: return False; return super().can_execute(...)' were converted, checked by AST. Commands with extra conditions (wx's delegate, joke's dark-joke DM rule, admin-only and DM-only checks, dadjoke's reversed order) keep their overrides. Several of the removed overrides called super().can_execute(message) without passing skip_channel_check; nothing calls them with it set, so honoring it now changes nothing.
Every name removed here has zero references across modules, tests, docs, local plugins, scripts and generate_website.py, and none is part of the documented plugin surface (db_manager, command_manager send methods, channel_manager stay untouched). About 1,160 lines:
- multitest: four formatter helpers that only referenced each other (the flat and nested variants are the ones in use)
- path: _get_api_cache_data, _filter_recent_repeaters
- repeater command: _handle_contact_stats, _get_geocoding_status
- airplanes: _send_split_response; greeter: get_recent_greeted_users; plugin_loader: validate_plugin
- wx: get_weather_for_zipcode, abbreviate_alert_title, get_current_conditions
- message_handler: discover_message_path, and an ADVERT debug line behind a self.debug attribute that is never set
- utils: _get_node_location_and_key_from_db
- repeater_manager: test_meshcore_cli_commands, toggle_auto_add, restore_repeater, get_purging_stats, populate_missing_geolocation_data, get_daily_advertisement_stats, get_nodes_per_day_stats
- command_manager: get_available_commands_list and get_plugin_metadata (HelpCommand and PluginLoader have the live versions)
- base_command: _load_command_prefix, and a no-op .replace('_', '_')
- scheduler: the first of two _run_data_retention definitions, shadowed by the second since it was added
- core: the write-only last_clock_sync_time, and _setup_routing_capture, which only logged a line
Full suite, ruff and mypy pass, and the characterization golden files are unchanged.
Added a new method `split_trigger_and_args` to the `BaseCommand` class, which splits message content into a matched keyword and arguments. This enhancement allows for better handling of command triggers and arguments across various command classes, ensuring that multi-word triggers are prioritized and leading characters are stripped appropriately. Updated the `ChannelsCommand`, `DiceCommand`, `HackerCommand`, `MultitestCommand`, `RollCommand`, and `TraceCommand` classes to utilize this new method for cleaner and more consistent command execution logic.
Remaining findings from the same triage pass. None are security
relevant; each produces a wrong user-visible result.
- greeter: treat rollout_started_at as UTC. It is written by SQLite
CURRENT_TIMESTAMP, but datetime.timestamp() read the naive value as
local time, shifting the backfill cutoff by the host's offset (7h on
PT). Users who posted inside that window were never marked as already
greeted and could be sent a welcome they should not have received.
- greeter: write greeted_at in SQLite's own format. The rollout backfill
used isoformat() while every other path used CURRENT_TIMESTAMP. "T"
sorts above a space, so ORDER BY greeted_at interleaved the two
formats wrongly, corrupting duplicate cleanup and the web viewer's
recently-greeted list.
- greeter: let an empty `channels =` disable the command. BaseCommand
reads an empty value as disabled-on-channels, but the greeter
collapsed "key absent" and "key present but empty" into one fallback
and kept greeting via monitor_channels.
- greeter: stop comma-splitting greeting text. channel_greetings split
entries on ",", so "Public:Welcome to the mesh, {sender}!" was stored
as "Welcome to the mesh" with the placeholder silently dropped. A
fragment now starts a new entry only when the text before its first
colon looks like a channel name, which keeps commas, URLs and clock
times attached to the greeting they belong to.
- wxsim_parser: match condition abbreviations longest-first. Substring
matching in dict order let RAIN shadow CHNC. RAIN, so five conditions
lost their "chance" qualifier (rain, snow, drizzle, t-storm, and
FAIR-P.C.).
- wxsim_parser: re-anchor "now" on each parse. current_date and
current_year were fixed at construction and wx_command builds one
parser at startup, so after a few days forecast dates rolled back a
year and staleness checks read permanently true.
- thesportsdb_client: hold the rate-limit read/sleep/write under a lock.
Concurrent callers read the same last_request_time, slept the same
interval and fired together, bursting past the 2.1s throttle. This
exposes an unrelated request fan-out problem in fetch_league_scores;
filed in TODO.md rather than fixed here.
- alert_command: stop duplicating the first incident. The tail treated
any single-line buffer as "header only" and appended incidents[0],
but messages after the first carry no header, so a final chunk holding
one incident had incident 0 pasted onto it. The same confusion inside
the loop also let a second incident be appended past the 130-character
limit.
- feed: reject non-positive poll intervals. -1 is truthy and was stored,
and the poller's `now - last_check >= interval` then treats the feed
as permanently due and re-fetches the URL every cycle; 0 was ignored
while still reporting success. The web viewer, which is the primary
editor, had no validation at all, and a JSON null there raised a
TypeError that aborted the poll cycle for every feed rather than one.
feed_manager falls back to the default for rows written before this.
- transmission_tracker: age out confirmed transmissions that have
repeats. Cleanup removed only repeat_count == 0, so repeated records
accumulated for the lifetime of the process, which matters on a Pi
Zero. Repeat counts are already persisted to packet_stream, so the
longer 30-minute retention loses nothing.
- mesh_graph: weighted-merge avg_hop_position when promoting an edge.
Promotion overwrote the average with the single new observation, so an
edge averaging 2.0 over 4 observations became 9.0 instead of 3.4 and
skewed subsequent path scoring.
- multitest_command: show the path that ends exactly at the display LCP.
The suffix helpers are correct in isolation; the loss happens in the
cluster formatters, where _shrink_display_lcp refuses to shrink a
single-token LCP and the trunk then rendered as "96 ┐", meaning
"everything continues past here" and dropping the bare 96 route while
the header still counted it. Now uses the file's existing "├ common"
marker. Empty suffixes are also no longer handed to the nested
renderer, which mapped them onto the same [] as a route ending at the
inner LCP and drew a row for a route that did not exist.
- sports_mappings: stop shadowing seven unique team nicknames. In one
flat dict a repeated key silently drops the earlier team, so hawks
resolved to the NBA Hawks rather than the Seahawks alias it was added
as, blazers to Kamloops rather than Portland, and rockets to Kelowna
rather than Houston. First definition now wins for hawks, giants,
jets, rangers, kings, blazers and rockets; every shadowed team keeps
its unambiguous full-name alias. The 55 city and abbreviation
collisions (chicago, sf, la, ...) are genuinely ambiguous and stay
last-wins, now pinned by a test so a new collision fails loudly
instead of passing unnoticed.
- Added a mechanism to check the byte length of direct messages (DMs) before sending, ensuring they do not exceed the firmware's maximum limit.
- Implemented a method to split oversized messages into smaller chunks, preserving UTF-8 encoding and avoiding mid-codepoint splits.
- Enhanced logging to provide warnings when messages are auto-split, including details on the number of chunks created.
- Updated the DM sending logic to handle both single and split messages efficiently.
- Introduced a new static method for splitting text into UTF-8 chunks, improving message handling across the application.
- Added configuration options to specify minimum path byte length requirements for the path, test, and multitest commands.
- Introduced methods to enforce these requirements and handle failure responses.
- Updated the packet capture service to correctly calculate packet hashes using the raw wire byte length.
- Enhanced unit tests to verify the correct behavior of the new path byte length logic.
PR #152 introduced trailing whitespace in several command files, set
message.content_lower at runtime without declaring it on the MeshMessage
dataclass, and left unused/unsorted imports in test and service files.
All cause ruff/mypy CI failures on branches that rebase onto dev after
that merge.
Linting fixes (ruff --fix):
- modules/command_manager.py: restore PUBLIC_CHANNEL_KEY_HEX re-export
with noqa guard (core.py imports it from here; auto-fix silently dropped it)
- modules/commands/multitest_command.py: strip W291/W293 trailing whitespace
- modules/commands/path_command.py: strip W293 trailing whitespace
- modules/commands/prefix_command.py: strip W291 trailing whitespace
- modules/commands/roll_command.py: strip W293 trailing whitespace
- modules/commands/sports_command.py: strip W293 trailing whitespace
- modules/core.py: strip W293 blank-line whitespace
- modules/service_plugins/packet_capture_service.py: sort imports (I001)
- modules/version_info.py: remove unused typing.Any import (F401)
- tests/integration/test_flood_scope_reply.py: remove unused call import
- tests/unit/test_log_data_scope_fields.py: remove unused asyncio import
- tests/unit/test_public_channel_guard.py: sort imports, remove unused
patch and validate_config imports
Dataclass fix (mypy attr-defined):
- modules/models.py: declare content_lower field on MeshMessage so mypy
resolves the attribute set by base_command.cleanup_message_for_matching
Must be merged before or alongside PRs #155–#158 to clear CI on those
branches.
- Updated `config.ini.example` to introduce the `respond_to_mentions` setting, allowing configuration of how the bot responds to mentions in channel messages.
- Refactored `MessageHandler` to implement logic for handling mentions based on the new configuration, including stripping mentions when appropriate.
- Added `cleanup_message_for_matching` method in `BaseCommand` to streamline message processing and mention validation.
- Enhanced various command classes to utilize the new cleanup method for consistent mention handling.
- Introduced tests to validate the behavior of the new mention handling logic across different configurations.
- Introduced a new `CondensePathsMode` type to support multiple condensing styles: "off", "flat", and "nested".
- Updated `_condense_path_lines` to handle different condensing modes, improving path representation.
- Enhanced test coverage for path condensing scenarios, ensuring accurate output for various path configurations.
These changes improve the flexibility and usability of path handling in the MultitestCommand, allowing for better visualization of path structures.
- Changed the fallback value for condense_paths from False to True in the MultitestCommand class, ensuring that paths are condensed by default. This adjustment improves the command's usability and aligns with expected behavior.
- Modified path formatting in `multitest_command.py` to use ideographic space for nested branches, enhancing visual clarity.
- Updated the `schedule_command.py` to strip control characters from messages, ensuring safe and clean previews.
- Adjusted test cases in `test_multitest_command.py` to reflect changes in path formatting, improving consistency across tests.
These changes enhance the readability of path outputs and improve message handling in scheduled commands.
- Updated path formatting logic in `MultitestCommand` to include a corner marker (┐) for shared paths, enhancing visual clarity.
- Improved documentation in `config.ini.example` to reflect changes in path formatting behavior.
- Expanded test cases to validate the new corner marker functionality and ensure accurate representation of unique paths with shared prefixes.
- Updated the path formatting logic in `MultitestCommand` to support nested branches using shared prefixes and horizontal continuations.
- Introduced new utility functions for grouping and formatting suffix lines, improving the clarity of output paths.
- Modified the configuration example to clarify the behavior of the `condense_paths` option.
- Expanded test coverage to validate the new path formatting behavior, ensuring accurate representation of unique paths.
- Added a new configuration option `condense_paths` in `config.ini.example` to control path formatting.
- Enhanced `MultitestCommand` to condense output paths using shared prefixes and tree branches when `condense_paths` is enabled.
- Introduced utility functions for path processing, including `_condense_path_lines` and `_path_to_tokens`.
- Updated tests to validate the new path condensing functionality and configuration behavior.
BUG-001: web viewer login/session auth (in web viewer commit)
BUG-002: db_manager ALTER TABLE for missing channel_operations and
feed_message_queue columns on startup
BUG-015: scheduler thread blocked on future.result(); replaced all
blocking waits with add_done_callback (fire-and-forget)
BUG-016: reboot_radio sends meshcore.commands.reboot() before disconnect
BUG-017: radio disconnect uses asyncio.wait_for(timeout=10)
BUG-022: custom asyncio loop exception handler suppresses IndexError
from meshcore parser at DEBUG level
BUG-024: last_db_backup_run updated after each run; 2-min startup
window; last-run seeded from DB on restart
BUG-025: send_channel_message retries up to 2 times (2s delay) on
no_event_received via _is_no_event_received() helper
BUG-026: split_text_into_chunks() and get_max_message_length() added
to CommandManager; keyword dispatch uses send_response_chunked()
BUG-028: byte_data = b"" initialised before try block in
decode_meshcore_packet to prevent UnboundLocalError in except handler
TraceCommand: path nodes reversed and return path truncated; fixed
format_elapsed_display: UTC normalisation before elapsed computation (#75)
RepeaterManager: auto_manage_contacts guard before any purge logic (#50)
Command aliases: [Aliases] config section injects shorthands at startup
JSON logging: _JsonFormatter; json_logging = true in [Logging]
Structured JSON logging compatible with Loki, Elasticsearch, Splunk
Discord bridge, Telegram bridge, and all service plugins updated
MeshGraph edge promotion logic corrected
Shutdown: scheduler and meshcore disconnect joined cleanly; log spam fixed
All modules: ruff and mypy cleanup applied (type annotations, imports)
- Updated `meshcore` dependency version to `2.2.14` in both `pyproject.toml` and `requirements.txt`.
- Added multi-byte path support in the `PathCommand`, allowing for 1-, 2-, and 3-byte-per-hop paths.
- Enhanced `MessageHandler` to utilize `routing_info` for accurate path extraction and validation.
- Improved path extraction methods in `MultitestCommand` and `TestCommand` to prefer `routing_info` for node IDs.
- Refactored path handling logic across various commands to ensure consistent multi-byte path processing.
- Introduced a new utility function `decode_path_len_byte` to decode RF packet path length bytes, supporting both legacy and multi-byte paths.
- Updated various modules to utilize the new decoding logic, ensuring compatibility with configured prefix lengths.
- Modified database schemas to include `bytes_per_hop` and `out_bytes_per_hop` columns for better path management.
- Enhanced path parsing and validation across commands and services to accommodate variable prefix lengths.
- Improved logging and error handling for path-related operations, ensuring robustness during transitions.
- Modified the behavior of the `MultitestCommand` class to silently ignore additional session requests from users who already have an active session, allowing the first session to complete without interruption.
- Introduced a new MultitestSession dataclass to manage active multitest sessions per user, enhancing concurrency handling.
- Replaced global variables with session-specific attributes to track listening duration, collected paths, and target packet hashes.
- Updated message handling logic to check active sessions and prevent race conditions during multitest execution.
- Enhanced path collection and RF data scanning to be user-specific, improving accuracy and logging for each session.
- Implemented execution locks to ensure thread safety during multitest command execution.
- Updated the `generate_html` function to include detailed command usage information, including syntax, examples, and parameters for better user guidance.
- Added CSS styles for improved presentation of command usage and parameters in the generated website documentation.
- Enhanced command classes with structured documentation fields, allowing for consistent and informative command descriptions across the platform.
- Implemented a configuration option for enabling or disabling commands across multiple command classes.
- Each command now checks its enabled state before execution, improving control over command availability.
- Updated the configuration loading mechanism to retrieve the enabled state from the config file for commands like Advert, AQI, Catfact, and others.
agessaman