27 Commits

Author SHA1 Message Date
agessaman
507c7ad31e Enhance database migration safety and improve repeater management
- Added a regex pattern to validate SQLite column definitions, preventing SQL injection in the _add_column() function.
- Introduced a new validation function to ensure safe column definitions are used.
- Updated repeater_manager.py to use list() for iterating over contacts, ensuring compatibility with potential changes in the underlying data structure.
- Enhanced error handling in the MessageScheduler for better debugging during scheduler shutdown and message sending.

These changes improve the security and reliability of database operations and enhance the robustness of the repeater management system.
2026-03-29 20:00:44 -07:00
agessaman
433cd5675a Implement locking mechanism for message processing in FeedManager
- Introduced a lock to serialize access to the message processing queue, preventing concurrent executions that could lead to race conditions.
- Updated the process_message_queue method to utilize the new lock, ensuring safe access to the inner processing logic.
- Increased the timeout for awaiting feed message queue processing in the MessageScheduler to 600 seconds, accommodating longer processing times for queued items.
2026-03-29 08:55:36 -07:00
agessaman
07a2db4ede Slim scheduler, add maintenance module
- Introduced a new `maintenance` module to handle data retention, log rotation, and nightly email tasks.
- Updated the `scheduler` to utilize the `MaintenanceRunner` for executing maintenance tasks, improving code organization and clarity.
- Enhanced documentation to reflect changes in logging configuration and data retention processes.
- Adjusted tests to accommodate the refactored scheduler methods and ensure proper functionality.
2026-03-19 19:40:33 -07:00
Stacy Olivas
ad77d7b00d fix: BUG-025/026/027/028/029 implementations and ruff/mypy refinements
BUG-025: send_channel_message retry logic on no_event_received
BUG-026: split_text_into_chunks and chunked dispatch in message_handler
BUG-027: test_weekly_on_wrong_day_does_not_run patch uses fake_now
BUG-028: byte_data = b"" initialised before try in decode_meshcore_packet
BUG-029: app.py db_path via self._config_base; realtime.html socket race
  fixed; base.html forceNew removed; ping_timeout 5 to 20s

Additional: ruff and mypy refinements across all modules; discord bridge,
telegram bridge, rate limiter, and service plugin updates
2026-03-17 18:07:19 -07:00
Stacy Olivas
904303ff00 infra: DB migration versioning, aiosqlite AsyncDBManager, and APScheduler
Migration versioning:
- db_migrations.py: MigrationRunner with five numbered migrations;
  schema_version table tracks applied state; migrations are append-only;
  runner called on startup from db_manager.py

AsyncDBManager:
- AsyncDBManager in db_manager.py provides non-blocking DB access in
  async coroutines via aiosqlite; exposed as bot.async_db_manager
- aiosqlite>=0.19.0 added to dependencies

APScheduler:
- scheduler.py migrated from schedule lib to APScheduler
  BackgroundScheduler + CronTrigger; schedule dependency removed

Message write queue:
- Background drain thread eliminates per-packet sqlite3.connect();
  executemany batch insert every 0.5s; shutdown path flushes remaining rows
2026-03-17 18:07:18 -07:00
agessaman
e9f17ec48e Enhance shutdown process with scheduler thread handling and improved meshcore disconnect logic
- Added a `join` method in `MessageScheduler` to wait for the scheduler thread to finish during shutdown.
- Updated the shutdown sequence in `core.py` to include a timeout for the `meshcore.disconnect()` method, with error handling for potential timeouts and exceptions.
- Improved logging to provide clearer information during the shutdown process.
2026-03-07 11:21:43 -08:00
agessaman
217d2a4089 Refactor database connection handling across multiple modules
- Replaced direct SQLite connection calls with a context manager in various modules to ensure proper resource management and prevent file descriptor leaks.
- Introduced a new `connection` method in `DBManager` to standardize connection handling.
- Updated all relevant database interactions in modules such as `feed_manager`, `scheduler`, `commands`, and others to utilize the new connection method.
- Improved code readability and maintainability by consolidating connection logic.
2026-03-01 14:12:22 -08:00
agessaman
fcfde7ea33 Add data retention configuration and cleanup functionality
- Introduced a new `[Data_Retention]` section in `config.ini.example` to manage retention periods for various database tables, including packet stream, daily stats, and observed paths.
- Updated `mkdocs.yml` and `configuration.md` to include documentation for the new data retention settings.
- Implemented data retention cleanup methods in `mesh_graph.py`, `repeater_manager.py`, and `scheduler.py` to enforce retention policies automatically.
- Enhanced the web viewer's data cleanup logic to utilize the new retention settings, ensuring efficient database management.
2026-02-24 22:01:49 -08:00
agessaman
87b58d7ff8 Merge main into dev (port changes since 0.8.2) 2026-02-22 21:29:54 -08:00
agessaman
3604125b14 Enhance FAQ and error handling in bot operations
- Added a new FAQ section addressing common issues when moving a database to a new install, including schema mismatches, stale operations, and timeout errors.
- Updated error logging in `feed_manager.py` and `scheduler.py` to use `logger.exception` for better traceback visibility during message queue and channel operation errors.
- Improved command help text generation in `command_manager.py` to ensure proper formatting and context filtering based on message length.
- Enhanced command availability checks in `cmd_command.py` and `help_command.py` to respect channel-specific overrides and improve user experience.
2026-02-22 18:55:35 -08:00
agessaman
bb1dd95f1c Refactor timezone handling across commands and utilities
- Introduced a new utility function `get_config_timezone` to centralize timezone retrieval and validation from the bot's configuration.
- Updated various commands and the scheduler to utilize the new function, ensuring consistent timezone handling and fallback mechanisms.
- Removed direct dependencies on `pytz` in favor of a more flexible approach that supports both `pytz` and `zoneinfo` for timezone management.
- Enhanced logging for invalid timezone configurations to improve troubleshooting.
2026-02-22 11:53:33 -08:00
agessaman
098ae7814b fix: feed manager and plugin loader imports; update message scheduler checks
- Removed unnecessary import of the `html` module in `feed_manager.py`.
- Moved the `import inspect` statement to the module level in `plugin_loader.py` for better organization.
- Initialized `last_channel_ops_check_time` and `last_message_queue_check_time` attributes in `scheduler.py` to streamline message scheduling logic.
- Cleaned up redundant checks for attribute existence in the `MessageScheduler` class.
2026-02-13 09:44:44 -08:00
agessaman
e6bec91c2b refactor: Consolidate escape sequence decoding across modules
- Removed the custom `_decode_escape_sequences` method from multiple classes and replaced it with a centralized `decode_escape_sequences` function in the utils module.
- Updated all relevant modules (CommandManager, MessageScheduler, GreeterCommand) to utilize the new function for decoding escape sequences in configuration strings, improving code maintainability and consistency.
- Enhanced the config example to clarify the usage of escape sequences in messages.
2026-02-02 16:18:14 -08:00
agessaman
db0db96f7e feat: Add reload configuration functionality and update admin commands
- Introduced a new method `reload_config` to allow dynamic reloading of the configuration without restarting the bot, ensuring seamless updates to settings.
- Added a helper method `_get_radio_settings` to retrieve current radio settings for comparison during reload.
- Updated `admin_commands` in `config.ini.example` to include the new `reload` command, enabling admin users to reload configurations on-the-fly.
- Enhanced the `setup_scheduled_messages` method in the scheduler to clear existing jobs before reloading, preventing duplicates.
2026-01-16 19:55:44 -08:00
agessaman
50724d1fed feat: Enhance asyncio event loop handling across modules
- Introduced a mechanism to utilize the main event loop for scheduling coroutines, preventing deadlocks when the main loop is running.
- Updated the JWT renewal interval to 12 hours, with tokens now valid for 24 hours, improving token management.
- Refactored various async function calls in the MessageScheduler, WeatherService, and PacketCaptureService to ensure consistent event loop usage.
- Improved error handling and logging for scheduled tasks, enhancing robustness and maintainability.
2026-01-02 09:25:31 -08:00
agessaman
afda22f0fe Refactor database path and placeholder handling across modules
- Replaced the validate_safe_path function with a new resolve_path utility to simplify database path resolution in BotDataViewer, BotIntegration, and MapUploaderService.
- Updated the logic to ensure that both relative and absolute paths are handled correctly, enhancing the robustness of database connections.
- Improved code readability and maintainability by centralizing path resolution logic.
- Centralized placeholder handling in utils instead of individual function handlers
2025-12-29 15:42:35 -08:00
agessaman
ccc38a3294 Update database connection handling in scheduler and integration modules
- Set a timeout of 30 seconds for SQLite connections in MessageScheduler and BotIntegration to improve reliability.
- Added validation for the database path in BotIntegration, ensuring it is safe and relative to the bot root, enhancing robustness.
- Introduced a bot_root attribute in BotDataViewer for path validation.
2025-12-29 09:33:57 -08:00
agessaman
7b2288f578 Enhance scheduled messaging with mesh information retrieval
- Added a new method in MessageScheduler to fetch and format mesh statistics for scheduled messages, including total contacts, repeaters, companions, and recent activity.
- Updated the message formatting logic to replace placeholders with actual mesh data, improving the relevance of scheduled updates.
- Enhanced logging for better debugging of mesh info retrieval and message processing.
- Updated config.ini.example to include examples of new placeholders for mesh network information.
2025-12-28 23:01:49 -08:00
agessaman
63ed0fd73d Update database connection handling to ensure string paths and set timeout
- Modified database connection calls in DBManager, FeedManager, and MessageScheduler to convert db_path to a string and set a timeout of 30 seconds, improving reliability and performance of database operations.
- Enhanced error logging to include detailed information about database path existence and permissions, aiding in debugging and ensuring smoother operation.
2025-12-28 12:12:15 -08:00
agessaman
35bc6260a4 Enhance command configuration and logging for joke commands
- Added configuration sections for joke and dadjoke commands in config.ini.example, allowing channel restrictions for command usage.
- Updated BaseCommand to derive configuration section names for commands, improving consistency in command management.
- Implemented channel access checks in can_execute methods for JokeCommand and DadJokeCommand to ensure commands are only executed in allowed channels.
- Improved error logging in FeedManager and MessageScheduler to include database path information for better debugging.
2025-12-27 08:21:51 -08:00
agessaman
7bacd5b485 Implement upgrade mode in install-service.sh for non-destructive updates and enhance channel fetching logic in MessageScheduler to eliminate periodic refresh. The upgrade mode allows users to update files and dependencies without losing existing configurations. Additionally, the channel fetching now occurs only once on launch to prevent data loss during updates. 2025-12-07 19:40:38 -08:00
agessaman
d446640a50 Enhance channel fetching logic in ChannelManager to include device connectivity checks and implement timeout protection during channel refresh in MessageScheduler. Introduce sender location prioritization in PathCommand and TestCommand for improved repeater selection accuracy. Update logging for better diagnostics on fetch operations and proximity calculations. 2025-12-07 14:02:14 -08:00
agessaman
c151fba7a5 Enhance configuration and database management for feed and channel operations. Add channel refresh interval setting to config.ini.example, implement feed manager functionality with new database tables for feed subscriptions and activity tracking, and improve greeter command logic with human greeting detection and Levenshtein distance matching for user greetings. Update web viewer to include feed management pages and integrate global template variables for better user experience. 2025-12-06 10:18:12 -08:00
agessaman
52c7e5fa31 added channel overrides for sports command; updated channels command; added stats command; consolidated t_phrase as a shortcut to test; added scheduled adverts 2025-10-05 16:09:39 -07:00
agessaman
9cf17e9145 Fix scheduled messages and add debugging 2025-09-07 10:49:12 -07:00
agessaman
8c8c50889d Fix scheduler async/await runtime warning
- Convert send_scheduled_message to synchronous wrapper for schedule library
- Add _send_scheduled_message_async for actual async implementation
- Handle event loop creation in scheduler thread
- Fixes RuntimeWarning: coroutine was never awaited
2025-09-06 12:01:47 -07:00
agessaman
9fa8b02be6 Initial commit: MeshCore Bot project 2025-09-04 15:33:51 -07:00