- Removed redundant command list in config example for clarity.
- Improved plugin validation by adding checks for required attributes and types, ensuring better error handling during plugin instantiation.
- Enhanced rate limiter classes to track total sends and throttled attempts, providing statistics for better monitoring.
- Updated command implementations to include metadata for better organization and clarity in command handling.
- Introduced a thread-safe InternetStatusCache class to manage internet connectivity status with caching.
- Refactored internet connectivity checks to utilize the new cache, reducing redundant checks.
- Added a unified method for handling message send results, improving error logging and response management.
- Enhanced rate limit checks to streamline command execution and prevent spam.
- Improved plugin loading error handling and validation, ensuring robust plugin management.
- Improved exception handling in MeshCoreBot to catch specific errors related to database and service initialization.
- Added a new method in MessageHandler for cleaning up stale RF data cache entries, enforcing maximum size limits and periodic cleanup.
- Updated message processing to handle potential AttributeError in multitest listener, ensuring robustness during message handling.
- Updated README to specify submitting pull requests against the dev branch.
- Added per-trigger lockout tracking in AnnouncementsCommand to prevent duplicate sends within a 60-second window.
- Implemented dynamic maximum message length calculation in BaseCommand for better message formatting.
- Enhanced response handling in PrefixCommand to support message splitting based on calculated length.
- Replaced the validate_safe_path function with a new resolve_path utility to simplify database path resolution in BotDataViewer, BotIntegration, and MapUploaderService.
- Updated the logic to ensure that both relative and absolute paths are handled correctly, enhancing the robustness of database connections.
- Improved code readability and maintainability by centralizing path resolution logic.
- Centralized placeholder handling in utils instead of individual function handlers
- Added db_path configuration options in config.ini.example for both the main bot and web viewer
- Updated BotDataViewer and BotIntegration classes to retrieve the database path from the new Web_Viewer section
- Set a timeout of 30 seconds for SQLite connections in MessageScheduler and BotIntegration to improve reliability.
- Added validation for the database path in BotIntegration, ensuring it is safe and relative to the bot root, enhancing robustness.
- Introduced a bot_root attribute in BotDataViewer for path validation.
- Removed redundant team aliases for Spokane Chiefs and Vancouver Giants.
- Added new teams and their aliases for the WHL, including Tri-City Americans, Wenatchee Wild, and several others, enhancing the sports command's coverage of hockey teams.
- Updated city-to-code mappings to include new entries for better identification of teams.
- Added a new method in MessageScheduler to fetch and format mesh statistics for scheduled messages, including total contacts, repeaters, companions, and recent activity.
- Updated the message formatting logic to replace placeholders with actual mesh data, improving the relevance of scheduled updates.
- Enhanced logging for better debugging of mesh info retrieval and message processing.
- Updated config.ini.example to include examples of new placeholders for mesh network information.
- Added a new configuration option in config.ini.example for selecting the weather provider (NOAA or Open-Meteo), defaulting to NOAA.
- Updated WxCommand to delegate to wx_international when the Open-Meteo provider is selected, improving global weather command support.
- Implemented fallback behavior to use wx_international as the wx command if the default wx command is missing, enhancing user experience.
- Improved logging to provide clearer information on command delegation and provider usage.
- Introduced TheSportsDBClient class to handle API requests with rate limiting, allowing for efficient data fetching for teams and events.
- Updated SportsCommand to utilize TheSportsDB API for leagues not supported by ESPN, including Canadian Football League (CFL) and Western Hockey League (WHL) teams.
- Implemented methods for fetching team and league events, including upcoming and past games, with robust error handling and logging.
- Enhanced team and league score retrieval to support both ESPN and TheSportsDB APIs, improving overall sports command functionality.
- Modified database connection calls in DBManager, FeedManager, and MessageScheduler to convert db_path to a string and set a timeout of 30 seconds, improving reliability and performance of database operations.
- Enhanced error logging to include detailed information about database path existence and permissions, aiding in debugging and ensuring smoother operation.
- Added validation for the database path using the validate_safe_path function to ensure paths are safe and relative to the bot root.
- Enhanced error handling to log warnings when invalid paths are detected, defaulting to 'bot_data.db' if necessary.
- This change improves the robustness of database connections across the application.
- Updated the validate_safe_path function to resolve the base directory to an absolute path and handle target paths more accurately.
- Enhanced logic to ensure that if the file_path is absolute, it is used directly; otherwise, it is joined with the base directory before resolution, improving handling of relative paths.
- Updated connection and reconnection logic in PacketCaptureService to use asyncio's run_in_executor, preventing blocking of the event loop during MQTT operations.
- Enhanced WeatherService to implement non-blocking connection and subscription handling, ensuring smoother operation and improved error logging for connection failures.
- Added configuration sections for joke and dadjoke commands in config.ini.example, allowing channel restrictions for command usage.
- Updated BaseCommand to derive configuration section names for commands, improving consistency in command management.
- Implemented channel access checks in can_execute methods for JokeCommand and DadJokeCommand to ensure commands are only executed in allowed channels.
- Improved error logging in FeedManager and MessageScheduler to include database path information for better debugging.
- Updated WxCommand to better handle period identification for tomorrow's weather, ensuring accurate retrieval of daytime and nighttime periods based on current conditions.
- Added a cleanup mechanism in MapUploaderService to manage memory usage by periodically removing old entries from seen_adverts, preventing unbounded growth and improving performance.
- Optimized packet capture service to utilize indexed lookups for faster data retrieval and ensure fallback mechanisms for backward compatibility.
- Clarified comments in config.ini.example and map_uploader_service.py to specify that adverts with any coordinate exactly 0.0 are considered invalid and will be skipped during uploads. This enhances understanding of the coordinate validation logic in the map uploader service.
- Introduced a new [MapUploader] section in config.ini.example to configure the map uploader service, including options for enabling the service, API endpoint, private key path, re-upload interval, and verbose logging.
- Updated requirements.txt to include cryptography and pynacl packages for enhanced security features in the map uploader service.
- Introduced a new service plugin architecture in core.py to manage multiple services, including the packet capture service.
- Updated the configuration file (config.ini.example) to include a new [Weather_Service] section for enabling weather forecasts and alerts, along with relevant settings.
- Enhanced the base service plugin class to support optional configuration section names and service descriptions for better metadata management.
- Improved the packet capture service with explicit configuration section and description for clarity.
- Introduced a new [PacketCapture] section in config.ini.example to enable packet capture service with options for output file, verbosity, and MQTT configuration.
- Updated core.py to initialize and manage the packet capture service, including starting and stopping the service during bot operation.
- Enhanced DiceCommand to support mixed dice notation and added functionality for decade dice rolls, improving user experience for tabletop gaming commands.
- Updated translations to reflect new dice command usage and descriptions, ensuring clarity for users.
- Introduced caching mechanisms for internet connectivity status in CommandManager to optimize performance and reduce redundant checks.
- Updated command execution logic to skip commands requiring internet access when connectivity is unavailable, improving user experience and error handling.
- Added synchronous and asynchronous utility functions for checking internet connectivity in utils.py.
- Marked relevant commands (e.g., AlertCommand, AqiCommand, DadJokeCommand) as requiring internet access to ensure proper execution conditions.
- Updated CommandManager to log rate limiting warnings only for meaningful wait times, avoiding misleading messages.
- Enhanced WxCommand to support a new "alerts" keyword for fetching weather alerts, with special handling for alert data.
- Improved alert fetching logic to differentiate duplicate special statements and prioritize alerts based on severity and urgency.
- Added methods for compactly formatting alerts and abbreviating city names for better display in responses.
- Updated MessageHandler to extract and store path information from packet_info and routing_info, improving data tracking.
- Added a new API endpoint in the web viewer for decoding path hex strings to repeater names.
- Enhanced the contacts template to display path information with tooltips, improving user experience.
- Implemented tooltip functionality for path data in the web viewer, allowing users to view detailed repeater information on hover.
- Added mappings for Professional Women's Hockey League (PWHL) teams with placeholders for team IDs.
- Updated city mappings to include PWHL teams for relevant cities.
- Enhanced game fetching logic to prioritize live games, recent past games, and upcoming games within specified timeframes.
- Introduced methods for fetching and formatting team schedules, improving user experience for schedule queries.
- Improved handling of live event data to ensure real-time score updates for ongoing games.
- Simplified comments in config.ini.example and core.py to clarify API endpoint configuration.
- Changed default value for prefix_free_days from 30 to 7 in prefix_command.py to align with recent adjustments in prefix handling logic.
- Enhanced logic in prefix_command.py to prioritize API cache for used prefixes and fallback to database only when necessary, improving efficiency and accuracy in prefix data retrieval.
This PR builds upon excellent security work to add comprehensive input validation, path traversal protection, and SQL injection prevention, with additional refinements for robustness and maintainability.
Security Improvements:
- Add DNS timeout (2.0s default) to validate_external_url() to prevent DoS
attacks from malicious URLs causing DNS resolution to hang
- Make path validation OS-aware: supports Windows, macOS (Darwin), and Linux
with platform-specific dangerous path detection
- Add validation for negative max_length values in sanitize_input()
Code Quality Improvements:
- Extract bot_root property in MeshCoreBot class to eliminate code duplication
(was calculated twice in __init__ and setup_logging)
- Use explicit bot_root directory instead of '.' for predictable path validation
in both database and log file path validation
- Make sanitize_input() max_length parameter Optional to allow disabling length
check for radio messages (firmware enforces 150-char limit at hardware level)
- Update message_handler.py to use max_length=None for radio messages while
preserving control character stripping for security
- Replace inline regex with centralized validate_pubkey_format() function in
base_command.py for consistency and maintainability
- Improve documentation: add comments about socket timeout behavior and
firmware-enforced message length limits
All improvements have been tested and verified:
- Syntax checks pass
- All functions work correctly
- No circular dependencies
- Bot initializes successfully with all attributes present
Files modified:
- modules/security_utils.py: DNS timeout, OS-aware paths, Optional max_length
- modules/core.py: bot_root property, explicit base directory usage
- modules/message_handler.py: max_length=None for radio messages
- modules/commands/base_command.py: centralized validation function
- Add DNS timeout to validate_external_url() to prevent DoS
- Remove duplicate import in base_command.py
- Make path validation OS-aware (Windows, macOS, Linux)
- Use centralized validate_pubkey_format() function
- Make sanitize_input max_length optional for radio messages
- Use explicit base directory instead of '.'
- Extract bot_root property to eliminate duplication
- Add validation for negative max_length
- Improve documentation and type hints
agessaman