Commit Graph

16 Commits

Author SHA1 Message Date
Stacy Olivas 904303ff00 infra: DB migration versioning, aiosqlite AsyncDBManager, and APScheduler
Migration versioning:
- db_migrations.py: MigrationRunner with five numbered migrations;
  schema_version table tracks applied state; migrations are append-only;
  runner called on startup from db_manager.py

AsyncDBManager:
- AsyncDBManager in db_manager.py provides non-blocking DB access in
  async coroutines via aiosqlite; exposed as bot.async_db_manager
- aiosqlite>=0.19.0 added to dependencies

APScheduler:
- scheduler.py migrated from schedule lib to APScheduler
  BackgroundScheduler + CronTrigger; schedule dependency removed

Message write queue:
- Background drain thread eliminates per-packet sqlite3.connect();
  executemany batch insert every 0.5s; shutdown path flushes remaining rows
2026-03-17 18:07:18 -07:00
agessaman 217d2a4089 Refactor database connection handling across multiple modules
- Replaced direct SQLite connection calls with a context manager in various modules to ensure proper resource management and prevent file descriptor leaks.
- Introduced a new `connection` method in `DBManager` to standardize connection handling.
- Updated all relevant database interactions in modules such as `feed_manager`, `scheduler`, `commands`, and others to utilize the new connection method.
- Improved code readability and maintainability by consolidating connection logic.
2026-03-01 14:12:22 -08:00
agessaman e9a615e120 feat: Add database query execution methods for connection reuse
- Introduced `execute_query_on_connection` and `execute_update_on_connection` methods in DBManager to facilitate executing queries and updates on existing database connections.
- Enhanced the `_recalculate_distance_if_needed`, `_get_location_by_public_key`, and `_get_location_by_prefix` methods in MeshGraph to support optional database connection and location caching, improving performance during batch operations.
2026-01-31 09:25:17 -08:00
agessaman df8dc8cafb feat: Enhance path selection and observation tracking
- Added new configuration options for path selection presets and proximity methods in config.ini.example, allowing users to customize routing behavior.
- Implemented a new database table for storing observed paths, enabling better tracking of paths from advertisements and messages.
- Updated MessageHandler to store complete paths in the observed_paths table, improving path validation and selection accuracy.
- Enhanced the PathCommand class to utilize new proximity and recency settings, optimizing repeater selection based on user-defined criteria.
- Improved web viewer functionality to display multiple paths for contacts, enhancing user experience and interaction with path data.
2026-01-25 10:12:46 -08:00
agessaman bd2849655c feat: Enhance mesh graph functionality and path validation
- Added a new mesh graph feature for improved path validation, allowing for enhanced routing accuracy.
- Introduced configuration options for recency decay half-life and graph-based validation settings in config.ini.example.
- Updated the PathCommand class to utilize graph-based selection methods, combining graph and geographic scores for better repeater selection.
- Implemented new methods in MessageHandler to update the mesh graph with advertisement paths and trace packet data.
- Created a new database table for mesh connections to support graph-based path validation.
- Enhanced web viewer integration to display mesh graph updates in real-time, improving user interaction and monitoring capabilities.
2026-01-23 20:26:56 -08:00
agessaman 93c4004e2d feat: Add unique advert packet tracking and leaderboard functionality
- Introduced a new database table 'unique_advert_packets' for tracking unique advert packets by their hash.
- Enhanced the RepeaterManager to handle unique packet tracking during daily advertisement statistics.
- Updated StatsCommand to include a new subcommand for displaying the leaderboard of nodes with the most unique advert packets in the last 24 hours.
- Modified translations to support the new advert statistics feature, ensuring user-friendly command descriptions and error messages.
2026-01-18 14:16:04 -08:00
agessaman a32fe0dcfd docs: Add docstrings and type hints across modules for improved clarity and maintainability. 2026-01-01 20:12:49 -08:00
agessaman 7b2cdf411a Refactor database connection handling in GreeterCommand to use centralized method
- Introduced a new get_connection method in DBManager to standardize database connection creation with proper configuration.
- Updated all instances in GreeterCommand where sqlite3.connect was used to utilize the new get_connection method, improving code consistency and maintainability.
2026-01-01 16:54:05 -08:00
agessaman 73492c8ff1 Add system health monitoring and reporting functionality
- Implemented periodic updates of system health status to the database every 30 seconds.
- Added a new method to aggregate and return comprehensive health status of all components, including core connection, database, services, and web viewer.
- Introduced an API endpoint at /api/system-health to retrieve system health data, providing a structured response for health status and component details.
- Enhanced error handling for health data storage and retrieval processes, ensuring robust logging and feedback in case of issues.
2026-01-01 16:37:03 -08:00
agessaman 63ed0fd73d Update database connection handling to ensure string paths and set timeout
- Modified database connection calls in DBManager, FeedManager, and MessageScheduler to convert db_path to a string and set a timeout of 30 seconds, improving reliability and performance of database operations.
- Enhanced error logging to include detailed information about database path existence and permissions, aiding in debugging and ensuring smoother operation.
2025-12-28 12:12:15 -08:00
agessaman 4d997afbcc Remove 'repeater_interactions' from the DBManager class whitelist as it isn't used. 2025-12-09 22:08:40 -08:00
eddieoz 8a35dc8270 feat: Add security utilities for path validation, SQL injection prevention, and input sanitization. 2025-12-09 21:26:42 -08:00
agessaman c151fba7a5 Enhance configuration and database management for feed and channel operations. Add channel refresh interval setting to config.ini.example, implement feed manager functionality with new database tables for feed subscriptions and activity tracking, and improve greeter command logic with human greeting detection and Levenshtein distance matching for user greetings. Update web viewer to include feed management pages and integrate global template variables for better user experience. 2025-12-06 10:18:12 -08:00
agessaman 92f063b4bc Add Nominatim rate limiting and caching enhancements across commands. Introduce NominatimRateLimiter for API compliance, update geocoding methods to utilize rate-limited calls, and extend caching duration for geocoding results to 30 days. Refactor command implementations to improve efficiency and prevent duplicate API requests. 2025-11-26 14:34:45 -08:00
agessaman 7bb51f219b Web Viewer Integration
- Add Flask + Flask-SocketIO web viewer with dashboard (modules/web_viewer/app.py and related)
- Add web viewer templates: index, realtime, tracking (contacts), cache, purging, stats (modules/web_viewer/templates/)
- Add integration hooks and utility functions for web viewer (modules/web_viewer/integration.py, modules/utils.py)
- Add command to launch web viewer from bot CLI (modules/commands/webviewer_command.py)
- Update .gitignore: ignore db/log files, test scripts, and web viewer artifacts
- Add restart_viewer.sh helper script for standalone web viewer restart/troubleshooting
- Add guidance and documentation for modern viewer in WEB_VIEWER.md and docs/
- Various code structure and import improvements to core bot and command modules to support integration
- Add ACL support for sensitive commands
- Example config updates

Benefits:
- Decouples monitoring/UI from bot core process
- Enables real-time browser dashboard and unified contact/repeater tracking
- Easier integration, dev, and troubleshooting
2025-10-21 21:57:00 -07:00
agessaman 2206265e10 Update modules with latest development changes
- Enhanced command processing with exclamation mark handling
- Improved message handling and RF data correlation
- Updated repeater management functionality
- Enhanced command implementations across all modules
- Updated database manager and core functionality
- All changes maintain backward compatibility
2025-09-06 11:22:58 -07:00