Commit Graph

40 Commits

Author SHA1 Message Date
agessaman e058da4968 fix: harden shutdown (single stop, viewer, MQTT logs, scheduler)
- Make MeshCoreBot.stop idempotent; remove duplicate stop from start().
- Avoid web viewer restart during shutdown; gate main-loop restarts.
- Fix packet capture MQTT callbacks to log each broker’s host.
- Only shutdown APScheduler when running; silence double-shutdown noise.
- Add regression tests in tests/test_shutdown_reliability.py.
2026-04-14 12:46:01 -07:00
agessaman 313dfccf75 fix: Align send suppression and security compatibility behavior
Make outbound send suppression consistent by honoring both radio-offline and zombie states across command-manager and scheduler paths. Preserve strict SSRF defaults while adding explicit private-feed URL opt-ins, persist allow_local_smtp from notifications config writes, reconcile zombie alert setting precedence, and replace deprecated UTC timestamp calls with timezone-aware UTC usage.
2026-04-14 12:18:02 -07:00
agessaman b8210b0fbe feat: enhance radio and admin configuration settings
Added new configuration options for radio health monitoring and alerting in config.ini.example, including radio_offline_threshold, radio_offline_alert_enabled, and radio_offline_alert_email. Introduced an [Admin] section with settings for a local admin API, including enabled, port, and token fields. Updated related files to reflect these changes, improving the organization and functionality of the configuration structure.
2026-04-14 11:44:17 -07:00
agessaman 293383f885 refactor: reorganize radio configuration settings
Moved radio-related configuration options from the [Bot] section to a new [Connection] section in config.ini.example for better organization. Updated the core logic to reference the new configuration paths, ensuring that radio health monitoring and alert settings are correctly handled. This change enhances clarity and maintainability of the configuration structure.
2026-04-14 11:14:28 -07:00
agessaman 8b6864471f fix: resolve scheduler duplicate and mypy fallback types
Remove duplicate zombie-alert implementation and add typed unidecode fallback to restore lint compliance on the integration branch.

Made-with: Cursor
2026-04-14 10:14:38 -07:00
Stacy Olivas 973d1fc947 fix: BUG-028 byte_data init; BUG-LOG-1/4 SocketIO and cleanup_database handler fixes 2026-04-14 10:02:46 -07:00
Stacy Olivas 54aeb28bf0 security: SSRF hardening, log injection sanitization, and allow_local_smtp
Add SSRF host validation to maintenance.py send_nightly_email and
scheduler.py send_zombie_alert_email using validate_external_url().
New allow_local_smtp config key permits private-IP SMTP for local
relay setups.

Add sanitize_name() to security_utils and apply it to all log calls
in message_handler, repeater_manager, path_command, solarforecast_command,
command_manager, and discord_bridge_service to prevent log injection.

Move nightly email logic from duplicate scheduler._send_nightly_email()
into the canonical maintenance.py implementation, removing the duplicate.
Update tests to call maintenance.send_nightly_email() directly.

Add validate_external_url allow_private parameter with support for
loopback, RFC1918, CGN, and link-local address ranges.
2026-04-14 10:02:36 -07:00
Stacy Olivas fd0875611d fix: restore MaintenanceRunner arch and correct zombie-detection tests 2026-04-14 10:01:51 -07:00
Stacy Olivas c543cac5bf fix: import validate_external_url in scheduler.py for SMTP SSRF guard 2026-04-14 10:01:51 -07:00
Stacy Olivas 51ab5d312c feat: radio-offline fail state — suppress sends, auto-restart, banner, and docs 2026-04-14 10:01:51 -07:00
Stacy Olivas 8b14c40958 feat: zombie radio banner on all pages + zombie alert config screen options
- base.html: persistent danger banner appears on every web page when
  is_radio_zombie is true; shows datetime zombie was detected; includes
  "Restart Bot Processing" button (POST /api/admin/zombie-recover) that
  clears _radio_zombie_detected and _radio_fail_count on the live bot
  object and removes the persisted DB flag; banner turns green on success

- config.html: new "Zombie Radio Alert" card with enable/disable toggle
  and alert-email field; "Save" writes to bot_metadata (immediate,
  survives restarts); "Save to config.ini" also persists values to
  config.ini and keeps the in-memory config in sync; card shows
  current config.ini values as baseline defaults

- app.py: inject_template_vars context processor now provides
  radio_zombie and radio_zombie_since to all templates; added
  GET/POST /api/config/zombie-alert endpoints (GET returns both
  bot_metadata and config.ini values; POST supports write_to_config
  flag); added POST /api/admin/zombie-recover endpoint; stored
  config_path on self for write-back use

- scheduler.py: send_zombie_alert_email now prefers bot_metadata
  (zombie.alert_enabled, zombie.alert_email) over config.ini so web
  UI changes take effect without a restart; uses isinstance(..., str)
  guard so mock/None values safely fall through to config.ini defaults
2026-04-14 10:01:51 -07:00
Stacy Olivas 22e1b2b53c fix: guard send_advert() with asyncio.wait_for(timeout=30) to prevent event loop lockup
When the radio firmware is unresponsive but not yet flagged as zombie,
send_advert() would hang indefinitely on the main event loop, blocking
all packet processing (no data in/out) for 60+ seconds until the
scheduler thread's future.result() timed out — but that only timed
out the waiting thread, not the coroutine itself.

Fix: wrap every send_advert() call that runs on the event loop with
asyncio.wait_for(timeout=30.0).  On timeout in the interval-advert
path, _radio_fail_count is incremented so repeated timeouts feed into
the existing zombie-detection threshold.
2026-04-14 10:01:51 -07:00
Stacy Olivas d0ae737066 feat: zombie radio detection — health probe, timeout guards, and alert system 2026-04-14 10:01:51 -07:00
agessaman 507c7ad31e Enhance database migration safety and improve repeater management
- Added a regex pattern to validate SQLite column definitions, preventing SQL injection in the _add_column() function.
- Introduced a new validation function to ensure safe column definitions are used.
- Updated repeater_manager.py to use list() for iterating over contacts, ensuring compatibility with potential changes in the underlying data structure.
- Enhanced error handling in the MessageScheduler for better debugging during scheduler shutdown and message sending.

These changes improve the security and reliability of database operations and enhance the robustness of the repeater management system.
2026-03-29 20:00:44 -07:00
agessaman 433cd5675a Implement locking mechanism for message processing in FeedManager
- Introduced a lock to serialize access to the message processing queue, preventing concurrent executions that could lead to race conditions.
- Updated the process_message_queue method to utilize the new lock, ensuring safe access to the inner processing logic.
- Increased the timeout for awaiting feed message queue processing in the MessageScheduler to 600 seconds, accommodating longer processing times for queued items.
2026-03-29 08:55:36 -07:00
agessaman 07a2db4ede Slim scheduler, add maintenance module
- Introduced a new `maintenance` module to handle data retention, log rotation, and nightly email tasks.
- Updated the `scheduler` to utilize the `MaintenanceRunner` for executing maintenance tasks, improving code organization and clarity.
- Enhanced documentation to reflect changes in logging configuration and data retention processes.
- Adjusted tests to accommodate the refactored scheduler methods and ensure proper functionality.
2026-03-19 19:40:33 -07:00
Stacy Olivas ad77d7b00d fix: BUG-025/026/027/028/029 implementations and ruff/mypy refinements
BUG-025: send_channel_message retry logic on no_event_received
BUG-026: split_text_into_chunks and chunked dispatch in message_handler
BUG-027: test_weekly_on_wrong_day_does_not_run patch uses fake_now
BUG-028: byte_data = b"" initialised before try in decode_meshcore_packet
BUG-029: app.py db_path via self._config_base; realtime.html socket race
  fixed; base.html forceNew removed; ping_timeout 5 to 20s

Additional: ruff and mypy refinements across all modules; discord bridge,
telegram bridge, rate limiter, and service plugin updates
2026-03-17 18:07:19 -07:00
Stacy Olivas 904303ff00 infra: DB migration versioning, aiosqlite AsyncDBManager, and APScheduler
Migration versioning:
- db_migrations.py: MigrationRunner with five numbered migrations;
  schema_version table tracks applied state; migrations are append-only;
  runner called on startup from db_manager.py

AsyncDBManager:
- AsyncDBManager in db_manager.py provides non-blocking DB access in
  async coroutines via aiosqlite; exposed as bot.async_db_manager
- aiosqlite>=0.19.0 added to dependencies

APScheduler:
- scheduler.py migrated from schedule lib to APScheduler
  BackgroundScheduler + CronTrigger; schedule dependency removed

Message write queue:
- Background drain thread eliminates per-packet sqlite3.connect();
  executemany batch insert every 0.5s; shutdown path flushes remaining rows
2026-03-17 18:07:18 -07:00
agessaman e9f17ec48e Enhance shutdown process with scheduler thread handling and improved meshcore disconnect logic
- Added a `join` method in `MessageScheduler` to wait for the scheduler thread to finish during shutdown.
- Updated the shutdown sequence in `core.py` to include a timeout for the `meshcore.disconnect()` method, with error handling for potential timeouts and exceptions.
- Improved logging to provide clearer information during the shutdown process.
2026-03-07 11:21:43 -08:00
agessaman 217d2a4089 Refactor database connection handling across multiple modules
- Replaced direct SQLite connection calls with a context manager in various modules to ensure proper resource management and prevent file descriptor leaks.
- Introduced a new `connection` method in `DBManager` to standardize connection handling.
- Updated all relevant database interactions in modules such as `feed_manager`, `scheduler`, `commands`, and others to utilize the new connection method.
- Improved code readability and maintainability by consolidating connection logic.
2026-03-01 14:12:22 -08:00
agessaman fcfde7ea33 Add data retention configuration and cleanup functionality
- Introduced a new `[Data_Retention]` section in `config.ini.example` to manage retention periods for various database tables, including packet stream, daily stats, and observed paths.
- Updated `mkdocs.yml` and `configuration.md` to include documentation for the new data retention settings.
- Implemented data retention cleanup methods in `mesh_graph.py`, `repeater_manager.py`, and `scheduler.py` to enforce retention policies automatically.
- Enhanced the web viewer's data cleanup logic to utilize the new retention settings, ensuring efficient database management.
2026-02-24 22:01:49 -08:00
agessaman 87b58d7ff8 Merge main into dev (port changes since 0.8.2) 2026-02-22 21:29:54 -08:00
agessaman 3604125b14 Enhance FAQ and error handling in bot operations
- Added a new FAQ section addressing common issues when moving a database to a new install, including schema mismatches, stale operations, and timeout errors.
- Updated error logging in `feed_manager.py` and `scheduler.py` to use `logger.exception` for better traceback visibility during message queue and channel operation errors.
- Improved command help text generation in `command_manager.py` to ensure proper formatting and context filtering based on message length.
- Enhanced command availability checks in `cmd_command.py` and `help_command.py` to respect channel-specific overrides and improve user experience.
2026-02-22 18:55:35 -08:00
agessaman bb1dd95f1c Refactor timezone handling across commands and utilities
- Introduced a new utility function `get_config_timezone` to centralize timezone retrieval and validation from the bot's configuration.
- Updated various commands and the scheduler to utilize the new function, ensuring consistent timezone handling and fallback mechanisms.
- Removed direct dependencies on `pytz` in favor of a more flexible approach that supports both `pytz` and `zoneinfo` for timezone management.
- Enhanced logging for invalid timezone configurations to improve troubleshooting.
2026-02-22 11:53:33 -08:00
agessaman 098ae7814b fix: feed manager and plugin loader imports; update message scheduler checks
- Removed unnecessary import of the `html` module in `feed_manager.py`.
- Moved the `import inspect` statement to the module level in `plugin_loader.py` for better organization.
- Initialized `last_channel_ops_check_time` and `last_message_queue_check_time` attributes in `scheduler.py` to streamline message scheduling logic.
- Cleaned up redundant checks for attribute existence in the `MessageScheduler` class.
2026-02-13 09:44:44 -08:00
agessaman e6bec91c2b refactor: Consolidate escape sequence decoding across modules
- Removed the custom `_decode_escape_sequences` method from multiple classes and replaced it with a centralized `decode_escape_sequences` function in the utils module.
- Updated all relevant modules (CommandManager, MessageScheduler, GreeterCommand) to utilize the new function for decoding escape sequences in configuration strings, improving code maintainability and consistency.
- Enhanced the config example to clarify the usage of escape sequences in messages.
2026-02-02 16:18:14 -08:00
agessaman db0db96f7e feat: Add reload configuration functionality and update admin commands
- Introduced a new method `reload_config` to allow dynamic reloading of the configuration without restarting the bot, ensuring seamless updates to settings.
- Added a helper method `_get_radio_settings` to retrieve current radio settings for comparison during reload.
- Updated `admin_commands` in `config.ini.example` to include the new `reload` command, enabling admin users to reload configurations on-the-fly.
- Enhanced the `setup_scheduled_messages` method in the scheduler to clear existing jobs before reloading, preventing duplicates.
2026-01-16 19:55:44 -08:00
agessaman 50724d1fed feat: Enhance asyncio event loop handling across modules
- Introduced a mechanism to utilize the main event loop for scheduling coroutines, preventing deadlocks when the main loop is running.
- Updated the JWT renewal interval to 12 hours, with tokens now valid for 24 hours, improving token management.
- Refactored various async function calls in the MessageScheduler, WeatherService, and PacketCaptureService to ensure consistent event loop usage.
- Improved error handling and logging for scheduled tasks, enhancing robustness and maintainability.
2026-01-02 09:25:31 -08:00
agessaman afda22f0fe Refactor database path and placeholder handling across modules
- Replaced the validate_safe_path function with a new resolve_path utility to simplify database path resolution in BotDataViewer, BotIntegration, and MapUploaderService.
- Updated the logic to ensure that both relative and absolute paths are handled correctly, enhancing the robustness of database connections.
- Improved code readability and maintainability by centralizing path resolution logic.
- Centralized placeholder handling in utils instead of individual function handlers
2025-12-29 15:42:35 -08:00
agessaman ccc38a3294 Update database connection handling in scheduler and integration modules
- Set a timeout of 30 seconds for SQLite connections in MessageScheduler and BotIntegration to improve reliability.
- Added validation for the database path in BotIntegration, ensuring it is safe and relative to the bot root, enhancing robustness.
- Introduced a bot_root attribute in BotDataViewer for path validation.
2025-12-29 09:33:57 -08:00
agessaman 7b2288f578 Enhance scheduled messaging with mesh information retrieval
- Added a new method in MessageScheduler to fetch and format mesh statistics for scheduled messages, including total contacts, repeaters, companions, and recent activity.
- Updated the message formatting logic to replace placeholders with actual mesh data, improving the relevance of scheduled updates.
- Enhanced logging for better debugging of mesh info retrieval and message processing.
- Updated config.ini.example to include examples of new placeholders for mesh network information.
2025-12-28 23:01:49 -08:00
agessaman 63ed0fd73d Update database connection handling to ensure string paths and set timeout
- Modified database connection calls in DBManager, FeedManager, and MessageScheduler to convert db_path to a string and set a timeout of 30 seconds, improving reliability and performance of database operations.
- Enhanced error logging to include detailed information about database path existence and permissions, aiding in debugging and ensuring smoother operation.
2025-12-28 12:12:15 -08:00
agessaman 35bc6260a4 Enhance command configuration and logging for joke commands
- Added configuration sections for joke and dadjoke commands in config.ini.example, allowing channel restrictions for command usage.
- Updated BaseCommand to derive configuration section names for commands, improving consistency in command management.
- Implemented channel access checks in can_execute methods for JokeCommand and DadJokeCommand to ensure commands are only executed in allowed channels.
- Improved error logging in FeedManager and MessageScheduler to include database path information for better debugging.
2025-12-27 08:21:51 -08:00
agessaman 7bacd5b485 Implement upgrade mode in install-service.sh for non-destructive updates and enhance channel fetching logic in MessageScheduler to eliminate periodic refresh. The upgrade mode allows users to update files and dependencies without losing existing configurations. Additionally, the channel fetching now occurs only once on launch to prevent data loss during updates. 2025-12-07 19:40:38 -08:00
agessaman d446640a50 Enhance channel fetching logic in ChannelManager to include device connectivity checks and implement timeout protection during channel refresh in MessageScheduler. Introduce sender location prioritization in PathCommand and TestCommand for improved repeater selection accuracy. Update logging for better diagnostics on fetch operations and proximity calculations. 2025-12-07 14:02:14 -08:00
agessaman c151fba7a5 Enhance configuration and database management for feed and channel operations. Add channel refresh interval setting to config.ini.example, implement feed manager functionality with new database tables for feed subscriptions and activity tracking, and improve greeter command logic with human greeting detection and Levenshtein distance matching for user greetings. Update web viewer to include feed management pages and integrate global template variables for better user experience. 2025-12-06 10:18:12 -08:00
agessaman 52c7e5fa31 added channel overrides for sports command; updated channels command; added stats command; consolidated t_phrase as a shortcut to test; added scheduled adverts 2025-10-05 16:09:39 -07:00
agessaman 9cf17e9145 Fix scheduled messages and add debugging 2025-09-07 10:49:12 -07:00
agessaman 8c8c50889d Fix scheduler async/await runtime warning
- Convert send_scheduled_message to synchronous wrapper for schedule library
- Add _send_scheduled_message_async for actual async implementation
- Handle event loop creation in scheduler thread
- Fixes RuntimeWarning: coroutine was never awaited
2025-09-06 12:01:47 -07:00
agessaman 9fa8b02be6 Initial commit: MeshCore Bot project 2025-09-04 15:33:51 -07:00