- Added new configuration options for path selection presets and proximity methods in config.ini.example, allowing users to customize routing behavior.
- Implemented a new database table for storing observed paths, enabling better tracking of paths from advertisements and messages.
- Updated MessageHandler to store complete paths in the observed_paths table, improving path validation and selection accuracy.
- Enhanced the PathCommand class to utilize new proximity and recency settings, optimizing repeater selection based on user-defined criteria.
- Improved web viewer functionality to display multiple paths for contacts, enhancing user experience and interaction with path data.
- Added a new mesh graph feature for improved path validation, allowing for enhanced routing accuracy.
- Introduced configuration options for recency decay half-life and graph-based validation settings in config.ini.example.
- Updated the PathCommand class to utilize graph-based selection methods, combining graph and geographic scores for better repeater selection.
- Implemented new methods in MessageHandler to update the mesh graph with advertisement paths and trace packet data.
- Created a new database table for mesh connections to support graph-based path validation.
- Enhanced web viewer integration to display mesh graph updates in real-time, improving user interaction and monitoring capabilities.
- Updated `send_dm` and `send_channel_message` methods to include an optional `command_id` for tracking message repeats.
- Integrated transmission tracking to record and manage repeat messages, improving message handling and response accuracy.
- Enhanced `capture_command` method in `BotIntegration` to store repeat information for better analysis in the web viewer.
- Added favicon support and improved web viewer templates to display repeat information effectively.
- Implemented JavaScript updates in the web viewer to handle command updates and display repeat counts dynamically.
- Implemented comprehensive light mode styles for various components in the web viewer, including modals, segment cards, and tables.
- Enhanced visual consistency by applying theme-based background and text colors across different UI elements.
- Updated styles for header and message content to ensure readability and aesthetic appeal in light mode.
- Updated `config.ini.example` to include a new option for additional hashtag channels to decode in the packet stream.
- Modified `BotDataViewer` to retrieve and display additional decode-only channels from the configuration.
- Improved packet handling in `message_handler.py` to capture full packet data for web viewer integration.
- Enhanced the web viewer's JavaScript to support detailed packet analysis and display, including color-coded hex breakdowns and improved user interface elements.
- Added new styles and scripts to the web viewer templates for better visual representation of packet data and improved user experience.
- Added a rotating file handler for logging to manage log file size and backups.
- Implemented thread safety for connected clients using a lock to prevent race conditions.
- Introduced a method to clean up stale clients based on inactivity, improving resource management.
- Updated logging messages to reflect changes in logging setup and client connection status.
- Enhanced error logging to include stack traces for better debugging.
- Optimized the cleanup of old packet stream data by implementing smaller batch deletions to reduce lock contention.
- Adjusted database connection settings for improved concurrency and added handling for database busy errors.
- Removed periodic cleanup from BotIntegration, delegating this responsibility to the web viewer subprocess to avoid contention.
- Refactored bot's main execution flow to use asyncio.run() with improved signal handling for graceful shutdown on Unix systems.
- Added new dependencies in pyproject.toml: urllib3, paho-mqtt, cryptography, and pynacl for enhanced functionality.
- Updated Nix flake to include flake-parts for better modularization and added translation path for NixOS module compatibility.
- Improved argument parsing in web viewer to maintain clarity and consistency.
This commit adds comprehensive Nix/NixOS support and modernizes the
Python packaging structure to enable declarative system-wide deployments.
Added complete Nix flake infrastructure in nix/:
- packages.nix: Package definition with all dependencies
- nixos-module.nix: NixOS module for declarative service configuration
- nixos-test.nix: Comprehensive NixOS VM tests (basic + web viewer)
- shell.nix: Development shell with all build tools
The NixOS module (services.meshcore-bot) provides:
- Automatic user/group creation (meshcore-bot:meshcore-bot)
- Serial port access (dialout group membership)
- Systemd service with security hardening
- Automatic directory management via StateDirectory/LogsDirectory
- INI config generation from Nix attribute sets
- Sensible defaults for system paths
Example NixOS configuration:
services.meshcore-bot = {
enable = true;
settings = {
Connection.connection_type = "serial";
Connection.serial_port = "/dev/ttyUSB0";
Bot.bot_name = "MyBot";
};
};
Migrated from requirements.txt to pyproject.toml:
- Defined proper package metadata and dependencies
- Created entry points: meshcore-bot, meshcore-viewer
- Enables standard 'pip install -e .' workflow
- Maintains compatibility with existing setups
Changes to existing code are minimal and only where necessary:
1. **Added --config parameter** (meshcore_bot.py, app.py)
- Wrapped main logic in main() function for entry point
- Added argparse to accept --config parameter
- Required for NixOS to pass generated config from /nix/store
- Backwards compatible: defaults to 'config.ini' in current directory
- No changes to core bot logic
2. **Fixed web viewer subprocess spawning** (modules/web_viewer/integration.py)
- Changed from sys.executable to 'meshcore-viewer' entry point
- Ensures Nix wrapper script sets up correct PYTHONPATH
- Resolves Flask module import issues in Nix environment
- No functional change for traditional installations
Note: The dev branch's resolve_path() utility already handles both
relative and absolute paths correctly, making it compatible with
NixOS system paths (/var/lib, /var/log) without additional changes.
All NixOS VM tests passing:
- nixos-module-basic: Service startup, file creation, TCP connection
- nixos-module-webviewer: Flask available, port listening, HTTP responses
No breaking changes to existing workflows. The bot continues to work
exactly as before when run with traditional Python methods.
- Enhanced the handle_connect, handle_disconnect, handle_subscribe_commands, handle_subscribe_packets, and handle_ping methods to include better error handling and logging.
- Added checks for client_id to prevent issues during connection and disconnection events.
- Implemented database file existence and accessibility checks in the polling loop, with detailed logging for errors.
- Improved handling of database connection errors, including exponential backoff for persistent issues, ensuring more robust database interactions.
- Implemented periodic updates of system health status to the database every 30 seconds.
- Added a new method to aggregate and return comprehensive health status of all components, including core connection, database, services, and web viewer.
- Introduced an API endpoint at /api/system-health to retrieve system health data, providing a structured response for health status and component details.
- Enhanced error handling for health data storage and retrieval processes, ensuring robust logging and feedback in case of issues.
- Simplified database connection management by using context managers to ensure connections are properly closed.
- Enhanced error handling during MQTT client disconnection in packet capture service, logging specific exceptions.
- Updated message handling in MessageHandler to capture command data for web viewer integration, improving response tracking.
- Replaced the validate_safe_path function with a new resolve_path utility to simplify database path resolution in BotDataViewer, BotIntegration, and MapUploaderService.
- Updated the logic to ensure that both relative and absolute paths are handled correctly, enhancing the robustness of database connections.
- Improved code readability and maintainability by centralizing path resolution logic.
- Centralized placeholder handling in utils instead of individual function handlers
- Added db_path configuration options in config.ini.example for both the main bot and web viewer
- Updated BotDataViewer and BotIntegration classes to retrieve the database path from the new Web_Viewer section
- Set a timeout of 30 seconds for SQLite connections in MessageScheduler and BotIntegration to improve reliability.
- Added validation for the database path in BotIntegration, ensuring it is safe and relative to the bot root, enhancing robustness.
- Introduced a bot_root attribute in BotDataViewer for path validation.
- Added validation for the database path using the validate_safe_path function to ensure paths are safe and relative to the bot root.
- Enhanced error handling to log warnings when invalid paths are detected, defaulting to 'bot_data.db' if necessary.
- This change improves the robustness of database connections across the application.
- Updated MessageHandler to extract and store path information from packet_info and routing_info, improving data tracking.
- Added a new API endpoint in the web viewer for decoding path hex strings to repeater names.
- Enhanced the contacts template to display path information with tooltips, improving user experience.
- Implemented tooltip functionality for path data in the web viewer, allowing users to view detailed repeater information on hover.
- Added configuration options for companion contact purging, including thresholds for inactivity based on direct messages and advertisements.
- Enhanced the RepeaterManager to support automatic purging of companions when contact limits are exceeded.
- Updated PathCommand to utilize configurable recency/proximity weighting for improved path routing decisions.
- Introduced new commands for purging companions and updated existing commands to handle companion purging logic.
- Added API endpoint for manual geocoding of contacts and improved web viewer functionality for geocoding contacts.
- Added a new 'daily_stats' table for tracking daily advertisement statistics.
- Implemented methods to track daily advertisement counts and retrieve statistics over specified date ranges.
- Updated existing advertisement tracking logic to utilize the new daily statistics.
- Modified web viewer to display advertisement metrics using the new daily tracking data.
- Improved path command logic to prioritize database queries over API cache for path decoding.
- Updated routes in 'app.py' to reflect the new contacts template.
- Enhanced 'sports_command.py' adding support for additional Portland teams
- Improved game data fetching logic to check multiple dates for recent and upcoming games.
- Add Flask + Flask-SocketIO web viewer with dashboard (modules/web_viewer/app.py and related)
- Add web viewer templates: index, realtime, tracking (contacts), cache, purging, stats (modules/web_viewer/templates/)
- Add integration hooks and utility functions for web viewer (modules/web_viewer/integration.py, modules/utils.py)
- Add command to launch web viewer from bot CLI (modules/commands/webviewer_command.py)
- Update .gitignore: ignore db/log files, test scripts, and web viewer artifacts
- Add restart_viewer.sh helper script for standalone web viewer restart/troubleshooting
- Add guidance and documentation for modern viewer in WEB_VIEWER.md and docs/
- Various code structure and import improvements to core bot and command modules to support integration
- Add ACL support for sensitive commands
- Example config updates
Benefits:
- Decouples monitoring/UI from bot core process
- Enables real-time browser dashboard and unified contact/repeater tracking
- Easier integration, dev, and troubleshooting
