- Updated `update_mesh_graph_from_trace_data` to clarify the format of `path_hashes` as per-hop hash strings from the trace payload.
- Modified `MessageHandler` to differentiate between TRACE packets and regular transmissions, preventing incorrect extraction of repeater prefixes from RF path bytes.
- Introduced `parse_trace_payload_route_hashes` utility to extract TRACE route hash segments from payloads, ensuring accurate handling of path data.
- Enhanced `PacketCaptureService` to correctly populate packet information for TRACE packets, including SNR path and route hashes.
- Expanded test coverage for TRACE payload decoding and path extraction to validate functionality and correctness.
These changes improve the accuracy and reliability of TRACE data processing in the application.
- Introduced `extract_path_node_ids_from_message` to streamline extraction of node IDs from mesh messages, prioritizing `routing_info.path_nodes` and supporting multi-byte comma parsing.
- Updated `calculate_path_distances` to accept an optional message parameter, enhancing its functionality by allowing it to derive node IDs from messages.
- Added `node_ids_from_path_string` to handle parsing of path strings into node IDs, improving the handling of both legacy and multi-byte formats.
- Refactored `TestCommand` to utilize the new extraction function, ensuring consistent behavior across commands.
- Expanded test coverage for new utility functions to validate their correctness and robustness.
These changes improve the clarity and maintainability of path-related utilities, enhancing overall functionality in message processing.
- Added new configuration options in `config.ini.example` for customizing the display of daily high/low temperatures.
- Introduced `format_temperature_high_low` function in `utils.py` to format temperature strings based on user-defined templates.
- Updated `WxCommand`, `GlobalWxCommand`, and `WeatherService` classes to utilize the new formatting function, enhancing the presentation of temperature data in weather forecasts.
- Refactored existing high/low temperature handling to improve code clarity and maintainability.
These changes enhance the flexibility and readability of temperature displays in weather-related outputs.
- Introduced a new configuration option for selecting Open-Meteo models in `config.ini.example`, allowing users to specify a model or default to "best_match".
- Implemented `_load_weather_model` method in `GlobalWxCommand` and `WeatherService` classes to handle model selection logic, including validation and fallback mechanisms.
- Updated API request parameters to include the selected weather model when available, enhancing the flexibility of weather data retrieval.
These changes improve the customization of weather data requests and ensure robust handling of model selection in the application.
- Introduced a new function to configure Unix signal handlers for SIGTERM, SIGINT, and SIGHUP, allowing for graceful shutdown and in-process configuration reload.
- Updated the main function to utilize the new signal handling setup, improving the bot's responsiveness to system signals.
- Enhanced documentation in the service installation guide to clarify the use of the reload command for configuration changes without restarting the service.
These changes improve the bot's operational flexibility and user experience during configuration updates.
- Modified path formatting in `multitest_command.py` to use ideographic space for nested branches, enhancing visual clarity.
- Updated the `schedule_command.py` to strip control characters from messages, ensuring safe and clean previews.
- Adjusted test cases in `test_multitest_command.py` to reflect changes in path formatting, improving consistency across tests.
These changes enhance the readability of path outputs and improve message handling in scheduled commands.
- Added a regex pattern to validate SQLite column definitions, preventing SQL injection in the _add_column() function.
- Introduced a new validation function to ensure safe column definitions are used.
- Updated repeater_manager.py to use list() for iterating over contacts, ensuring compatibility with potential changes in the underlying data structure.
- Enhanced error handling in the MessageScheduler for better debugging during scheduler shutdown and message sending.
These changes improve the security and reliability of database operations and enhance the robustness of the repeater management system.
- Updated TODO.md with the latest status and coverage metrics.
- Added date and datetime adapters for SQLite in db_manager.py to improve date handling.
- Refactored repeater_manager.py to ensure atomic database operations within transactions, enhancing data integrity.
- Implemented CSRF protection and security headers in web_viewer/app.py to safeguard against cross-origin requests and improve response security.
- Enforced authentication for non-loopback interface binding in web_viewer/integration.py to prevent unauthorized access.
These changes enhance the overall security, reliability, and maintainability of the application.
- Updated MessageHandler to use OrderedDict for SNR and RSSI caches, implementing LRU eviction to maintain a maximum size.
- Added thread safety to rate limiting classes by introducing locks, ensuring consistent behavior in concurrent environments.
- Introduced periodic cleanup in TransmissionTracker to manage memory usage effectively.
- Added unit tests for LRU cache behavior and automatic cleanup functionality.
These changes improve performance and reliability in handling message data and rate limiting.
CommandManager.get_max_message_length now uses 158-byte DM budget and 160-byte channel body budget minus UTF-8 username and ": " prefix, with the same 130-byte floor as BaseCommand. Update TestGetMaxMessageLength expectations.
Refs: https://github.com/agessaman/meshcore-bot/pull/128
- Updated path formatting logic in `MultitestCommand` to include a corner marker (┐) for shared paths, enhancing visual clarity.
- Improved documentation in `config.ini.example` to reflect changes in path formatting behavior.
- Expanded test cases to validate the new corner marker functionality and ensure accurate representation of unique paths with shared prefixes.
- Updated the path formatting logic in `MultitestCommand` to support nested branches using shared prefixes and horizontal continuations.
- Introduced new utility functions for grouping and formatting suffix lines, improving the clarity of output paths.
- Modified the configuration example to clarify the behavior of the `condense_paths` option.
- Expanded test coverage to validate the new path formatting behavior, ensuring accurate representation of unique paths.
- Added a new configuration option `condense_paths` in `config.ini.example` to control path formatting.
- Enhanced `MultitestCommand` to condense output paths using shared prefixes and tree branches when `condense_paths` is enabled.
- Introduced utility functions for path processing, including `_condense_path_lines` and `_path_to_tokens`.
- Updated tests to validate the new path condensing functionality and configuration behavior.
- Introduced validation for SQLite journal modes in DBManager, defaulting to "WAL" for invalid inputs and logging warnings.
- Added a new utility function in security_utils for validating SQL identifiers to prevent SQL injection.
- Updated BotDataViewer to utilize the new journal mode validation and SQL identifier checks, ensuring safer database operations.
- Enhanced test coverage for restore functionality, including checks for backup directory configuration and path traversal prevention.
- Adjusted assertions in test cases to account for the title-casing of channel keys by the DiscordBridgeService, ensuring compatibility with the ConfigParser behavior.
- Updated test_single_webhook_backward_compatible and test_multiple_webhooks_parsed_and_queued to check for "Public" instead of "public" in channel_webhooks.
- Removed MQTT configuration from `config.ini.example` to streamline settings.
- Added new section for Repeater Prefix Collision Service in `config.ini.example`, including options for enabling notifications, alert channels, and prefix handling.
- Updated `service-plugins.md` to include documentation for the new Repeater Prefix Collision Service, enhancing user awareness of available plugins.
- Added new CSS styles for live message metadata badges and layout adjustments for better visual organization.
- Reorganized the HTML structure of the real-time message and packet stream sections for improved responsiveness and clarity.
- Updated JavaScript functions to support new channel key normalization and hashing for enhanced message handling.
- Modified test cases to reflect changes in message entry templates and ensure proper functionality of new features.
- Introduced a new `_utc_now` function in `maintenance.py` to standardize UTC time retrieval, replacing deprecated `utcnow()` calls.
- Updated all instances of `datetime.utcnow()` in `maintenance.py` to use the new `_utc_now` function for consistency.
- Modified `app.py` in the web viewer to use `datetime.now(timezone.utc)` instead of `datetime.utcnow()`, ensuring timezone awareness.
- Enhanced MQTT test suite with new v2 callback methods for improved connection handling and error reporting.
- Introduced `_seed_feed_subscription` function to insert minimal feed subscription data into the database, ensuring successful foreign key inserts for feed activity and errors during tests.
- Updated `TestProcessRssFeed` to utilize the new seeding function, enhancing test reliability and setup consistency.
- Introduced `{field|auto}` placeholder in message formats to fill remaining characters up to `max_message_length`, improving message customization.
- Implemented logic in `FeedManager` to handle multiple `{field|auto}` placeholders, logging a warning if more than one is present.
- Updated `BotDataViewer` to utilize the new auto field feature, ensuring compatibility with existing message formatting.
- Added unit tests to validate the behavior of the new auto field functionality, including handling of message length constraints and multiple placeholders.
- Added configuration options for URL shortening in `config.ini.example` and updated documentation in `FEEDS.md`.
- Enhanced `FeedManager` to support URL shortening based on new settings, allowing for both global and per-link shortening.
- Refactored message formatting logic to incorporate URL shortening features, ensuring compatibility with existing link handling.
- Introduced new utility functions for encoding path length bytes in `utils.py`, improving path management in message handling.
- Added unit tests to validate the new URL shortening functionality and ensure proper behavior under various conditions.
- Introduced `within_days` and `within_weeks` operators for filtering items based on their timestamps, allowing for more dynamic date range conditions.
- Updated documentation in `FEEDS.md` to include examples and usage of the new operators.
- Refactored filter evaluation logic in `FeedManager` and `BotDataViewer` to utilize a shared function for condition checking, improving code maintainability.
- Added unit tests to validate the new date-based filtering functionality.
- Updated the web viewer documentation to replace the cache data section with a new config panel overview, enhancing clarity on configuration options.
- Removed the legacy cache management route, redirecting to the new configuration panel for better user experience.
- Enhanced the configuration page to include structured settings with categorized topics and database tools, improving navigation and usability.
- Updated tests to reflect changes in cache management and ensure proper functionality of the new configuration routes.
- Introduced a new `maintenance` module to handle data retention, log rotation, and nightly email tasks.
- Updated the `scheduler` to utilize the `MaintenanceRunner` for executing maintenance tasks, improving code organization and clarity.
- Enhanced documentation to reflect changes in logging configuration and data retention processes.
- Adjusted tests to accommodate the refactored scheduler methods and ensure proper functionality.
- Updated configuration examples to remove command prefix from aliases in `config.ini.example` and documentation.
- Enhanced `CommandManager` to normalize command names and resolve aliases through both direct mapping and plugin keyword mappings.
- Introduced a method in `BaseCommand` to normalize aliases from configuration, ensuring consistency in keyword handling.
- Added tests to verify that aliases resolve correctly from both keyword mappings and runtime keywords without legacy prefixes.
- Added `_apply_sqlite_pragmas` method in `DBManager` to configure SQLite connection settings such as foreign keys, busy timeout, and journal mode.
- Updated `connection` methods in `DBManager` and `BotDataViewer` to utilize the new pragma settings.
- Introduced validation functions in `db_migrations.py` to ensure proper identifier formats and table existence checks.
- Created new migration functions for managing `packet_stream` and repeater-related tables, ensuring they are created and indexed correctly.
- Removed redundant table initialization code from `RepeaterManager` and `BotDataViewer`, relying on migrations for table setup.
- Enhanced tests to verify the creation of repeater tables and indexes during migrations.
- Updated the `resolve_path` function in `utils.py` to clarify behavior regarding absolute paths.
- Changed type hints in `discord_bridge_service.py` for better clarity and consistency.
- Removed unused imports and unnecessary comments in various test files to improve code cleanliness and readability.
Log ruff/mypy/ShellCheck fixes (e0eae09) and Python 3.9 matrix
removal (92c5910) in BUGS.md fixed section; update TODO.md
last-updated line to reflect current CI status.
ConfigParser lowercases all config keys, so bridge.Public is stored as
bridge.public in channel_webhooks. The test assertions expected title-case
"Public" but the actual stored key is "public". Runtime channel matching
was already case-insensitive; only the test expectations needed updating.
Each command's aliases are now configured as an `aliases` key in its own
config section (e.g. [Wx_Command] aliases = !weather, !w) rather than a
separate [Aliases] section. BaseCommand._load_aliases_from_config() reads
and injects them into keywords at startup. CommandManager.load_aliases()
and _apply_aliases() are removed. No behaviour change for commands without
aliases configured.
ConfigParser lowercases all config keys, so bridge.Public is stored as
bridge.public in channel_webhooks. The test assertions expected title-case
"Public" but the actual stored key is "public". Runtime channel matching
was already case-insensitive; only the test expectations needed updating.
Each command's aliases are now configured as an `aliases` key in its own
config section (e.g. [Wx_Command] aliases = !weather, !w) rather than a
separate [Aliases] section. BaseCommand._load_aliases_from_config() reads
and injects them into keywords at startup. CommandManager.load_aliases()
and _apply_aliases() are removed. No behaviour change for commands without
aliases configured.
Add live and offline packet-parsing tests using paho-mqtt:
- tests/test_mqtt_live.py: schema validation + live integration tests;
subscribes to meshcore/SEA/+/packets; validates JSON against schema
- tests/mqtt_test_config.ini: broker/topic/timeout config; primary LAN
broker (10.0.2.123:1883); letsmesh as documented alternative
- tests/fixtures/mqtt_packets.json: 8 real packets from SEA region for
offline fallback
Run live: pytest -m mqtt
Run offline: pytest -m "not mqtt"
Collect fixtures: python tests/test_mqtt_live.py --collect-fixtures
Auth (BUG-001):
- Optional password via web_viewer_password in [Web_Viewer]; /login and
/logout; Flask session guard on all routes and SocketIO handlers
Contact management and export:
- Star contacts of any type; purge-preview + purge inactive contacts
- GET /api/export/contacts and /api/export/paths: CSV/JSON with time-range
Config tab and maintenance:
- /config page: SMTP, log rotation, DB backup settings in bot_metadata
- Nightly email digest (uptime, contacts, DB size, log errors); SMTP
timeout=30s; pre-rotation log attachment hook
- GET /api/maintenance/status: Maintenance Status card
DB backup, restore, and purge:
- POST /api/maintenance/backup_now; GET /api/maintenance/list_backups;
POST /api/maintenance/restore (SQLite magic-byte validation)
- POST /api/maintenance/purge: remove rows older than threshold
- Scheduled backups: daily/weekly/manual with retention pruning
- Config save validates db_backup_dir exists; 400 on missing path
Live streaming and realtime monitoring:
- Live Activity panel: colour-coded SocketIO feed with pause/clear
- capture_channel_message() feeds packet_stream; message_data event
- /realtime page: three independent stream panels; [#channel] prefix
- /logs page: subscribe_logs/log_line; log-tail thread; level colouring
- History replay: last 50/50/200 items on connect
- Werkzeug 3.1 WebSocket fix: _apply_werkzeug_websocket_fix()
- BUG-029: db_path resolved via config_base = Path(config_path).parent;
stored as self._config_base; dead _get_db_path() removed
Scroll/filter controls and connected agents:
- Scroll-to-top/bottom on Live Activity and all realtime panels
- Type-filter checkboxes (Packets/Commands/Messages) with applyFilters()
- GET /api/connected_clients: agent count clickable; Bootstrap modal
Add geographic_scoring_enabled = true/false to [Path_Command] config.
When disabled, path scoring uses hop count only and ignores GPS
coordinates. Evaluated per-command invocation; no restart required.
Add ScheduleCommand (DM-only by default). Displays configured scheduled
message times, target channels, message previews, and the current
advertisement interval. Read-only; does not modify schedule state.
Add ChannelRateLimiter to rate_limiter.py. Configure per-channel
cooldowns via [Rate_Limits] channel.<name>_seconds. Integrated into
_check_rate_limits() and send_channel_message() in command_manager.py.
GET /api/stats/rate_limiters exposes live stats for all four limiter
types.
Add POST /webhook endpoint to the web viewer. Authenticated via
Authorization: Bearer <token> set in [Webhook] config section. Relays
JSON or text payload to a configured MeshCore channel or user DM.
Test modules:
- test_enums: enum values and flag combinations
- test_models: MeshMessage dataclass field and type validation
- test_transmission_tracker: full TransmissionTracker coverage
- test_message_handler: path parsing, RF correlation, message routing
- test_repeater_manager: role detection, ACL, device type classification
- test_core: config loading, radio settings, reload paths
Tracking files:
- BUGS.md: known bugs and fix history log
- TESTING.md: test strategy, coverage targets, and how-to guide
- TODO.md: feature and task backlog with completion status
- scripts/update_todos.py: scans source for # TODO/FIXME/HACK markers
and regenerates the Inline TODOs section in TODO.md
- Updated config.ini.example and discord-bridge.md to reflect the ability to fan out a single MeshCore channel to multiple Discord servers using a comma- or whitespace-separated list of webhook URLs.
- Modified DiscordBridgeService to handle multiple webhooks per channel, including validation and logging improvements for better monitoring of configured webhooks.
Add live and offline packet-parsing tests using paho-mqtt:
- tests/test_mqtt_live.py: schema validation + live integration tests;
subscribes to meshcore/SEA/+/packets; validates JSON against schema
- tests/mqtt_test_config.ini: broker/topic/timeout config; primary LAN
broker (10.0.2.123:1883); letsmesh as documented alternative
- tests/fixtures/mqtt_packets.json: 8 real packets from SEA region for
offline fallback
Run live: pytest -m mqtt
Run offline: pytest -m "not mqtt"
Collect fixtures: python tests/test_mqtt_live.py --collect-fixtures
Auth (BUG-001):
- Optional password via web_viewer_password in [Web_Viewer]; /login and
/logout; Flask session guard on all routes and SocketIO handlers
Contact management and export:
- Star contacts of any type; purge-preview + purge inactive contacts
- GET /api/export/contacts and /api/export/paths: CSV/JSON with time-range
Config tab and maintenance:
- /config page: SMTP, log rotation, DB backup settings in bot_metadata
- Nightly email digest (uptime, contacts, DB size, log errors); SMTP
timeout=30s; pre-rotation log attachment hook
- GET /api/maintenance/status: Maintenance Status card
DB backup, restore, and purge:
- POST /api/maintenance/backup_now; GET /api/maintenance/list_backups;
POST /api/maintenance/restore (SQLite magic-byte validation)
- POST /api/maintenance/purge: remove rows older than threshold
- Scheduled backups: daily/weekly/manual with retention pruning
- Config save validates db_backup_dir exists; 400 on missing path
Live streaming and realtime monitoring:
- Live Activity panel: colour-coded SocketIO feed with pause/clear
- capture_channel_message() feeds packet_stream; message_data event
- /realtime page: three independent stream panels; [#channel] prefix
- /logs page: subscribe_logs/log_line; log-tail thread; level colouring
- History replay: last 50/50/200 items on connect
- Werkzeug 3.1 WebSocket fix: _apply_werkzeug_websocket_fix()
- BUG-029: db_path resolved via config_base = Path(config_path).parent;
stored as self._config_base; dead _get_db_path() removed
Scroll/filter controls and connected agents:
- Scroll-to-top/bottom on Live Activity and all realtime panels
- Type-filter checkboxes (Packets/Commands/Messages) with applyFilters()
- GET /api/connected_clients: agent count clickable; Bootstrap modal
agessaman