- Introduced a responsive toolbar for filtering and searching contacts, improving usability on mobile devices.
- Added a dropdown for timespan selection, allowing users to filter contacts based on different timeframes.
- Updated styles to ensure consistent alignment and height for toolbar elements, enhancing the overall appearance and functionality.
These changes improve the user experience for managing contacts in the web viewer.
- Introduced a mobile-friendly layout for the contacts section, including a new mobile stack for displaying contact cards.
- Added sorting options for contacts on mobile devices, allowing users to sort by various criteria such as name, device type, and distance.
- Implemented a select-all checkbox for mobile view to improve user interaction with contact selection.
- Updated event listeners to handle changes in selection and sorting, ensuring a seamless experience across devices.
These changes improve usability and accessibility for mobile users in the web viewer.
- Added a function to strip ANSI color codes from log lines for better display in SocketIO web clients, improving log readability.
- Implemented dark mode styles for dropdown menus and other UI components to enhance user experience in dark theme.
- Updated the contacts template to include a new overflow menu for additional actions, improving accessibility and usability.
- Enhanced the login page with a more visually appealing layout and improved theme handling to prevent flash of unstyled content.
- Refined log level toggles in the logs template for better user interaction and visibility of log levels.
These changes improve the overall functionality and aesthetics of the web viewer.
- Added 'X-Requested-With' header to various API requests in channel_operations.js, cache.html, config.html, contacts.html, feeds.html, greeter.html, mesh.html, radio.html, and other templates to improve request handling and prevent potential issues with cross-origin requests.
- Ensured consistent header usage across all relevant fetch calls to enhance security and compatibility.
These changes improve the robustness of API interactions within the web viewer.
- Introduced an asyncio lock to serialize deduplication checks and updates to seen adverts, preventing race conditions during concurrent advert processing.
- Enhanced replay attack detection by ensuring deduplication state is consistent across await boundaries.
- Improved logging for duplicate advert uploads and invalid coordinate handling, providing clearer feedback on advert processing outcomes.
These changes enhance the reliability and integrity of advert uploads in the MapUploaderService.
- Updated TODO.md with the latest status and coverage metrics.
- Added date and datetime adapters for SQLite in db_manager.py to improve date handling.
- Refactored repeater_manager.py to ensure atomic database operations within transactions, enhancing data integrity.
- Implemented CSRF protection and security headers in web_viewer/app.py to safeguard against cross-origin requests and improve response security.
- Enforced authentication for non-loopback interface binding in web_viewer/integration.py to prevent unauthorized access.
These changes enhance the overall security, reliability, and maintainability of the application.
- Updated MessageHandler to use OrderedDict for SNR and RSSI caches, implementing LRU eviction to maintain a maximum size.
- Added thread safety to rate limiting classes by introducing locks, ensuring consistent behavior in concurrent environments.
- Introduced periodic cleanup in TransmissionTracker to manage memory usage effectively.
- Added unit tests for LRU cache behavior and automatic cleanup functionality.
These changes improve performance and reliability in handling message data and rate limiting.
CommandManager.get_max_message_length now uses 158-byte DM budget and 160-byte channel body budget minus UTF-8 username and ": " prefix, with the same 130-byte floor as BaseCommand. Update TestGetMaxMessageLength expectations.
Refs: https://github.com/agessaman/meshcore-bot/pull/128
Use UTF-8 byte length instead of character count or display width for message
truncation. This fixes an issue where emoji characters like 😀 (4 UTF-8 bytes,
1 character, 2 display units) caused messages to exceed the RF packet size.
Changes:
- Replace _count_display_width with byte-based length calculation
- Add _count_byte_length and _truncate_to_byte_length helper methods
- Update get_max_message_length to use 127 bytes (channel limit) not 150
- Add type hints to new methods
- Updated path formatting logic in `MultitestCommand` to include a corner marker (┐) for shared paths, enhancing visual clarity.
- Improved documentation in `config.ini.example` to reflect changes in path formatting behavior.
- Expanded test cases to validate the new corner marker functionality and ensure accurate representation of unique paths with shared prefixes.
- Updated the path formatting logic in `MultitestCommand` to support nested branches using shared prefixes and horizontal continuations.
- Introduced new utility functions for grouping and formatting suffix lines, improving the clarity of output paths.
- Modified the configuration example to clarify the behavior of the `condense_paths` option.
- Expanded test coverage to validate the new path formatting behavior, ensuring accurate representation of unique paths.
- Added a new configuration option `condense_paths` in `config.ini.example` to control path formatting.
- Enhanced `MultitestCommand` to condense output paths using shared prefixes and tree branches when `condense_paths` is enabled.
- Introduced utility functions for path processing, including `_condense_path_lines` and `_path_to_tokens`.
- Updated tests to validate the new path condensing functionality and configuration behavior.
- Introduced validation for SQLite journal modes in DBManager, defaulting to "WAL" for invalid inputs and logging warnings.
- Added a new utility function in security_utils for validating SQL identifiers to prevent SQL injection.
- Updated BotDataViewer to utilize the new journal mode validation and SQL identifier checks, ensuring safer database operations.
- Enhanced test coverage for restore functionality, including checks for backup directory configuration and path traversal prevention.
- Removed MQTT configuration from `config.ini.example` to streamline settings.
- Added new section for Repeater Prefix Collision Service in `config.ini.example`, including options for enabling notifications, alert channels, and prefix handling.
- Updated `service-plugins.md` to include documentation for the new Repeater Prefix Collision Service, enhancing user awareness of available plugins.
- Added today's high and low temperatures to the weather forecast output, improving clarity for users.
- Updated formatting to include labels for high and low temperatures, ensuring better readability of the forecast information.
- Introduced a lock to serialize access to the message processing queue, preventing concurrent executions that could lead to race conditions.
- Updated the process_message_queue method to utilize the new lock, ensuring safe access to the inner processing logic.
- Increased the timeout for awaiting feed message queue processing in the MessageScheduler to 600 seconds, accommodating longer processing times for queued items.
- Added new CSS styles for live message metadata badges and layout adjustments for better visual organization.
- Reorganized the HTML structure of the real-time message and packet stream sections for improved responsiveness and clarity.
- Updated JavaScript functions to support new channel key normalization and hashing for enhanced message handling.
- Modified test cases to reflect changes in message entry templates and ensure proper functionality of new features.
- Introduced a new `_utc_now` function in `maintenance.py` to standardize UTC time retrieval, replacing deprecated `utcnow()` calls.
- Updated all instances of `datetime.utcnow()` in `maintenance.py` to use the new `_utc_now` function for consistency.
- Modified `app.py` in the web viewer to use `datetime.now(timezone.utc)` instead of `datetime.utcnow()`, ensuring timezone awareness.
- Enhanced MQTT test suite with new v2 callback methods for improved connection handling and error reporting.
- Introduced `{field|auto}` placeholder in message formats to fill remaining characters up to `max_message_length`, improving message customization.
- Implemented logic in `FeedManager` to handle multiple `{field|auto}` placeholders, logging a warning if more than one is present.
- Updated `BotDataViewer` to utilize the new auto field feature, ensuring compatibility with existing message formatting.
- Added unit tests to validate the behavior of the new auto field functionality, including handling of message length constraints and multiple placeholders.
- Added configuration options for URL shortening in `config.ini.example` and updated documentation in `FEEDS.md`.
- Enhanced `FeedManager` to support URL shortening based on new settings, allowing for both global and per-link shortening.
- Refactored message formatting logic to incorporate URL shortening features, ensuring compatibility with existing link handling.
- Introduced new utility functions for encoding path length bytes in `utils.py`, improving path management in message handling.
- Added unit tests to validate the new URL shortening functionality and ensure proper behavior under various conditions.
- Updated `pyproject.toml` to include JavaScript files for the web viewer.
- Added a new script reference in `base.html` for channel operations.
- Improved the channel creation process in `feeds.html` with enhanced UI elements and error handling.
- Refactored channel index retrieval in `radio.html` to utilize a centralized method for better maintainability.
- Implemented asynchronous channel statistics loading to improve responsiveness during channel operations.
- Introduced `within_days` and `within_weeks` operators for filtering items based on their timestamps, allowing for more dynamic date range conditions.
- Updated documentation in `FEEDS.md` to include examples and usage of the new operators.
- Refactored filter evaluation logic in `FeedManager` and `BotDataViewer` to utilize a shared function for condition checking, improving code maintainability.
- Added unit tests to validate the new date-based filtering functionality.
- Added a new configuration option `use_bot_location_when_no_location` to allow the use of bot's configured coordinates if no companion location is available.
- Updated `wx_command.py` and `wx_international.py` to utilize the new configuration, enhancing the user experience by providing a fallback mechanism for location-based commands.
- Improved logging to reflect the usage of bot coordinates and handle cases where bot location is not set.
- Introduced support for multiday forecasts up to 16 days, allowing users to specify days in various formats (e.g., Nd, 7day).
- Updated usage documentation and parameter descriptions to reflect new options for both US and international weather commands.
- Adjusted internal logic to validate and parse new forecast formats, ensuring compatibility with the extended range.
- Added `channelpause` and `channelresume` commands to the admin commands list in configuration files, allowing admins to pause or resume bot responses on public channels via DM.
- Updated documentation to reflect the new command functionality and its implications for channel responses.
- Modified validation and message handling to incorporate the new channel response control feature.
- Updated the web viewer documentation to replace the cache data section with a new config panel overview, enhancing clarity on configuration options.
- Removed the legacy cache management route, redirecting to the new configuration panel for better user experience.
- Enhanced the configuration page to include structured settings with categorized topics and database tools, improving navigation and usability.
- Updated tests to reflect changes in cache management and ensure proper functionality of the new configuration routes.
- Updated the command execution flow to skip commands that are not allowed in the current channel, improving control over command usage.
- Ensured that the command handling mirrors the existing checks in the system, maintaining consistency in command execution rules.
- Enhanced the `add_contact` method to handle variations in parameter requirements across device bindings, ensuring robust functionality.
- Avoided passing unnecessary `contact_data` fields that could lead to issues with unsigned integer handling.
- Implemented a fallback mechanism for `add_contact` to accommodate signature mismatches, improving error handling.
- Updated the `add_contact` method call to only pass necessary fields (name and optional public_key), improving compatibility with device APIs.
- Removed unnecessary RSSI/SNR values from `contact_data` to prevent issues with unsigned integer handling in some devices.
- Introduced a new `maintenance` module to handle data retention, log rotation, and nightly email tasks.
- Updated the `scheduler` to utilize the `MaintenanceRunner` for executing maintenance tasks, improving code organization and clarity.
- Enhanced documentation to reflect changes in logging configuration and data retention processes.
- Adjusted tests to accommodate the refactored scheduler methods and ensure proper functionality.
- Updated configuration examples to remove command prefix from aliases in `config.ini.example` and documentation.
- Enhanced `CommandManager` to normalize command names and resolve aliases through both direct mapping and plugin keyword mappings.
- Introduced a method in `BaseCommand` to normalize aliases from configuration, ensuring consistency in keyword handling.
- Added tests to verify that aliases resolve correctly from both keyword mappings and runtime keywords without legacy prefixes.
- Removed chunking logic for keyword-dispatched help/command responses in `message_handler.py`, allowing long responses to be sent as single messages.
- Updated the response sending mechanism to directly use the full response text for both direct messages and channel messages, ensuring clarity and consistency in message delivery.
- Expanded package data in `pyproject.toml` to include additional static files for the web viewer.
- Removed testing dependencies from `requirements.txt`, noting that they are now included in pyproject extras.
- Updated the base HTML template to reference the manifest from the Flask static folder.
- Modified the build script to exclude `.cursor` files and updated documentation URLs to the correct repository.
- Changed the installation command in the build script to use `requirements.txt` instead of the previous method.
- Normalized root logger early to avoid duplicate output from dependencies.
- Cleared root logger handlers and set logging level explicitly.
- Configured third-party loggers (APScheduler and tzlocal) to prevent unformatted console output and manage their logging levels.
- Ensured that logger propagation is disabled to avoid duplicate messages.
- Added `_apply_sqlite_pragmas` method in `DBManager` to configure SQLite connection settings such as foreign keys, busy timeout, and journal mode.
- Updated `connection` methods in `DBManager` and `BotDataViewer` to utilize the new pragma settings.
- Introduced validation functions in `db_migrations.py` to ensure proper identifier formats and table existence checks.
- Created new migration functions for managing `packet_stream` and repeater-related tables, ensuring they are created and indexed correctly.
- Removed redundant table initialization code from `RepeaterManager` and `BotDataViewer`, relying on migrations for table setup.
- Enhanced tests to verify the creation of repeater tables and indexes during migrations.
- Updated `MeshCoreBot` to normalize channel names when setting rate limits.
- Refactored `PerUserRateLimiter` to use `OrderedDict` for efficient key management and added normalization for keys.
- Improved `ChannelRateLimiter` to normalize channel names during initialization and when checking limits, ensuring consistent behavior.
- Updated the `resolve_path` function in `utils.py` to clarify behavior regarding absolute paths.
- Changed type hints in `discord_bridge_service.py` for better clarity and consistency.
- Removed unused imports and unnecessary comments in various test files to improve code cleanliness and readability.
Each command's aliases are now configured as an `aliases` key in its own
config section (e.g. [Wx_Command] aliases = !weather, !w) rather than a
separate [Aliases] section. BaseCommand._load_aliases_from_config() reads
and injects them into keywords at startup. CommandManager.load_aliases()
and _apply_aliases() are removed. No behaviour change for commands without
aliases configured.
BUG-001: web viewer login/session auth (in web viewer commit)
BUG-002: db_manager ALTER TABLE for missing channel_operations and
feed_message_queue columns on startup
BUG-015: scheduler thread blocked on future.result(); replaced all
blocking waits with add_done_callback (fire-and-forget)
BUG-016: reboot_radio sends meshcore.commands.reboot() before disconnect
BUG-017: radio disconnect uses asyncio.wait_for(timeout=10)
BUG-022: custom asyncio loop exception handler suppresses IndexError
from meshcore parser at DEBUG level
BUG-024: last_db_backup_run updated after each run; 2-min startup
window; last-run seeded from DB on restart
BUG-025: send_channel_message retries up to 2 times (2s delay) on
no_event_received via _is_no_event_received() helper
BUG-026: split_text_into_chunks() and get_max_message_length() added
to CommandManager; keyword dispatch uses send_response_chunked()
BUG-028: byte_data = b"" initialised before try block in
decode_meshcore_packet to prevent UnboundLocalError in except handler
TraceCommand: path nodes reversed and return path truncated; fixed
format_elapsed_display: UTC normalisation before elapsed computation (#75)
RepeaterManager: auto_manage_contacts guard before any purge logic (#50)
Command aliases: [Aliases] config section injects shorthands at startup
JSON logging: _JsonFormatter; json_logging = true in [Logging]
Structured JSON logging compatible with Loki, Elasticsearch, Splunk
Discord bridge, Telegram bridge, and all service plugins updated
MeshGraph edge promotion logic corrected
Shutdown: scheduler and meshcore disconnect joined cleanly; log spam fixed
All modules: ruff and mypy cleanup applied (type annotations, imports)
Auth (BUG-001):
- Optional password via web_viewer_password in [Web_Viewer]; /login and
/logout; Flask session guard on all routes and SocketIO handlers
Contact management and export:
- Star contacts of any type; purge-preview + purge inactive contacts
- GET /api/export/contacts and /api/export/paths: CSV/JSON with time-range
Config tab and maintenance:
- /config page: SMTP, log rotation, DB backup settings in bot_metadata
- Nightly email digest (uptime, contacts, DB size, log errors); SMTP
timeout=30s; pre-rotation log attachment hook
- GET /api/maintenance/status: Maintenance Status card
DB backup, restore, and purge:
- POST /api/maintenance/backup_now; GET /api/maintenance/list_backups;
POST /api/maintenance/restore (SQLite magic-byte validation)
- POST /api/maintenance/purge: remove rows older than threshold
- Scheduled backups: daily/weekly/manual with retention pruning
- Config save validates db_backup_dir exists; 400 on missing path
Live streaming and realtime monitoring:
- Live Activity panel: colour-coded SocketIO feed with pause/clear
- capture_channel_message() feeds packet_stream; message_data event
- /realtime page: three independent stream panels; [#channel] prefix
- /logs page: subscribe_logs/log_line; log-tail thread; level colouring
- History replay: last 50/50/200 items on connect
- Werkzeug 3.1 WebSocket fix: _apply_werkzeug_websocket_fix()
- BUG-029: db_path resolved via config_base = Path(config_path).parent;
stored as self._config_base; dead _get_db_path() removed
Scroll/filter controls and connected agents:
- Scroll-to-top/bottom on Live Activity and all realtime panels
- Type-filter checkboxes (Packets/Commands/Messages) with applyFilters()
- GET /api/connected_clients: agent count clickable; Bootstrap modal
Add geographic_scoring_enabled = true/false to [Path_Command] config.
When disabled, path scoring uses hop count only and ignores GPS
coordinates. Evaluated per-command invocation; no restart required.
Add ScheduleCommand (DM-only by default). Displays configured scheduled
message times, target channels, message previews, and the current
advertisement interval. Read-only; does not modify schedule state.
Add ChannelRateLimiter to rate_limiter.py. Configure per-channel
cooldowns via [Rate_Limits] channel.<name>_seconds. Integrated into
_check_rate_limits() and send_channel_message() in command_manager.py.
GET /api/stats/rate_limiters exposes live stats for all four limiter
types.
Add POST /webhook endpoint to the web viewer. Authenticated via
Authorization: Bearer <token> set in [Webhook] config section. Relays
JSON or text payload to a configured MeshCore channel or user DM.
- Updated config.ini.example and discord-bridge.md to reflect the ability to fan out a single MeshCore channel to multiple Discord servers using a comma- or whitespace-separated list of webhook URLs.
- Modified DiscordBridgeService to handle multiple webhooks per channel, including validation and logging improvements for better monitoring of configured webhooks.
- Updated the _cleanup_web_viewer and _cleanup_mesh_graph methods to avoid logging errors during shutdown, as the logger's stream may be closed at that time.
- Modified the shutdown method in MeshGraph to prevent logging of flushing errors, enhancing stability during the atexit process.
- Adjusted test configurations to use Path objects for bot_root and local_root, improving path handling in tests.
agessaman