Commit Graph

38 Commits

Author SHA1 Message Date
Stacy Olivas f971e9716f fix: correct channel name case assertion in discord bridge tests
ConfigParser lowercases all config keys, so bridge.Public is stored as
bridge.public in channel_webhooks. The test assertions expected title-case
"Public" but the actual stored key is "public". Runtime channel matching
was already case-insensitive; only the test expectations needed updating.
2026-03-17 19:56:18 -07:00
Stacy Olivas 164dbaeeb7 refactor: move command aliases to per-command config section
Each command's aliases are now configured as an `aliases` key in its own
config section (e.g. [Wx_Command] aliases = !weather, !w) rather than a
separate [Aliases] section. BaseCommand._load_aliases_from_config() reads
and injects them into keywords at startup. CommandManager.load_aliases()
and _apply_aliases() are removed. No behaviour change for commands without
aliases configured.
2026-03-17 19:56:11 -07:00
Stacy Olivas 9f49e15d6b test: MQTT live test framework and packet fixtures
Add live and offline packet-parsing tests using paho-mqtt:
- tests/test_mqtt_live.py: schema validation + live integration tests;
  subscribes to meshcore/SEA/+/packets; validates JSON against schema
- tests/mqtt_test_config.ini: broker/topic/timeout config; primary LAN
  broker (10.0.2.123:1883); letsmesh as documented alternative
- tests/fixtures/mqtt_packets.json: 8 real packets from SEA region for
  offline fallback

Run live: pytest -m mqtt
Run offline: pytest -m "not mqtt"
Collect fixtures: python tests/test_mqtt_live.py --collect-fixtures
2026-03-17 17:46:56 -07:00
Stacy Olivas e392468d39 test: coverage expansion — commands, web viewer, and infrastructure
New test modules:
- test_announcements_command: parse, record_trigger, execute paths
- test_aurora_command: KP index parsing, alert levels, execute paths
- test_channel_manager: generate_hashtag_key, cache lookups, validation
- test_channels_command: remaining channel info display paths
- test_dadjoke_command: format, split, length, execute
- test_graph_trace_helper: geo-location helper and graph algorithm paths
- test_hacker_command: text transform logic
- test_help_command: format list, channel filter, general/specific help
- test_i18n: fallback loops, format failure, PermissionError, get_value
- test_joke_command: seasonal, format, split, dark, execute
- test_moon_command: phase calc, execute success/error
- test_multitest_command: multi-channel test sequences
- test_stats_command: adverts leaderboard, get_stats_summary, cleanup
- test_trace_command: path extract, parse, format inline/vertical
- test_web_viewer_integration: circuit breaker, JSON serializer,
  packet capture, channel message
- test_webviewer_command: 100% coverage

Extended existing: test_command_manager, test_feed_manager,
test_message_handler, test_rate_limiter, test_repeater_manager,
test_scheduler_logic, test_security_utils, test_transmission_tracker,
test_utils, test_web_viewer
2026-03-17 17:46:44 -07:00
Stacy Olivas 2c4daa1720 test: expanded test suite for v0.9.0 modules
Command tests:
- tests/commands/: test_base_command, test_cmd_command, test_dice_command,
  test_hello_command, test_help_command, test_magic8_command,
  test_ping_command, test_roll_command
- tests/test_bridge_bot_responses, test_channel_manager_logic,
  test_checkin_service, test_command_manager, test_command_prefix,
  test_config_merge, test_config_validation, test_db_manager,
  test_plugin_loader, test_profanity_filter, test_security_utils,
  test_service_plugin_loader, test_utils

Integration and unit:
- tests/integration/: test_path_graph_integration, test_path_resolution
- tests/regression/: test_keyword_escapes
- tests/unit/: test_mesh_graph, test_mesh_graph_edges,
  test_mesh_graph_multihop, test_mesh_graph_optimizations,
  test_mesh_graph_scoring, test_mesh_graph_validation,
  test_path_command_graph, test_path_command_graph_selection,
  test_path_command_multibyte

Helpers: tests/conftest.py, tests/helpers.py
2026-03-17 17:45:21 -07:00
Stacy Olivas a1a10ff024 feat: web viewer — auth, contact management, live streaming, config, maintenance, and backup
Auth (BUG-001):
- Optional password via web_viewer_password in [Web_Viewer]; /login and
  /logout; Flask session guard on all routes and SocketIO handlers

Contact management and export:
- Star contacts of any type; purge-preview + purge inactive contacts
- GET /api/export/contacts and /api/export/paths: CSV/JSON with time-range

Config tab and maintenance:
- /config page: SMTP, log rotation, DB backup settings in bot_metadata
- Nightly email digest (uptime, contacts, DB size, log errors); SMTP
  timeout=30s; pre-rotation log attachment hook
- GET /api/maintenance/status: Maintenance Status card

DB backup, restore, and purge:
- POST /api/maintenance/backup_now; GET /api/maintenance/list_backups;
  POST /api/maintenance/restore (SQLite magic-byte validation)
- POST /api/maintenance/purge: remove rows older than threshold
- Scheduled backups: daily/weekly/manual with retention pruning
- Config save validates db_backup_dir exists; 400 on missing path

Live streaming and realtime monitoring:
- Live Activity panel: colour-coded SocketIO feed with pause/clear
- capture_channel_message() feeds packet_stream; message_data event
- /realtime page: three independent stream panels; [#channel] prefix
- /logs page: subscribe_logs/log_line; log-tail thread; level colouring
- History replay: last 50/50/200 items on connect
- Werkzeug 3.1 WebSocket fix: _apply_werkzeug_websocket_fix()
- BUG-029: db_path resolved via config_base = Path(config_path).parent;
  stored as self._config_base; dead _get_db_path() removed

Scroll/filter controls and connected agents:
- Scroll-to-top/bottom on Live Activity and all realtime panels
- Type-filter checkboxes (Packets/Commands/Messages) with applyFilters()
- GET /api/connected_clients: agent count clickable; Bootstrap modal
2026-03-17 17:44:14 -07:00
Stacy Olivas 7bd1f9934b feat: !path geographic scoring toggle
Add geographic_scoring_enabled = true/false to [Path_Command] config.
When disabled, path scoring uses hop count only and ignores GPS
coordinates. Evaluated per-command invocation; no restart required.
2026-03-17 17:43:46 -07:00
Stacy Olivas 9e4610c52d feat: !schedule command listing scheduled messages and advert interval
Add ScheduleCommand (DM-only by default). Displays configured scheduled
message times, target channels, message previews, and the current
advertisement interval. Read-only; does not modify schedule state.
2026-03-17 17:43:37 -07:00
Stacy Olivas 11b44e9234 feat: per-channel rate limiting
Add ChannelRateLimiter to rate_limiter.py. Configure per-channel
cooldowns via [Rate_Limits] channel.<name>_seconds. Integrated into
_check_rate_limits() and send_channel_message() in command_manager.py.
GET /api/stats/rate_limiters exposes live stats for all four limiter
types.
2026-03-17 17:43:29 -07:00
Stacy Olivas 96bf1747c2 feat: inbound webhook relay with bearer token authentication
Add POST /webhook endpoint to the web viewer. Authenticated via
Authorization: Bearer <token> set in [Webhook] config section. Relays
JSON or text payload to a configured MeshCore channel or user DM.
2026-03-17 17:43:21 -07:00
Stacy Olivas d33169fa53 infra: DB migration versioning, aiosqlite AsyncDBManager, and APScheduler
Migration versioning:
- db_migrations.py: MigrationRunner with five numbered migrations;
  schema_version table tracks applied state; migrations are append-only;
  runner called on startup from db_manager.py

AsyncDBManager:
- AsyncDBManager in db_manager.py provides non-blocking DB access in
  async coroutines via aiosqlite; exposed as bot.async_db_manager
- aiosqlite>=0.19.0 added to dependencies

APScheduler:
- scheduler.py migrated from schedule lib to APScheduler
  BackgroundScheduler + CronTrigger; schedule dependency removed

Message write queue:
- Background drain thread eliminates per-packet sqlite3.connect();
  executemany batch insert every 0.5s; shutdown path flushes remaining rows
2026-03-17 17:43:14 -07:00
Stacy Olivas bdd71b2762 infra: initial test suite and project tracking files
Test modules:
- test_enums: enum values and flag combinations
- test_models: MeshMessage dataclass field and type validation
- test_transmission_tracker: full TransmissionTracker coverage
- test_message_handler: path parsing, RF correlation, message routing
- test_repeater_manager: role detection, ACL, device type classification
- test_core: config loading, radio settings, reload paths

Tracking files:
- BUGS.md: known bugs and fix history log
- TESTING.md: test strategy, coverage targets, and how-to guide
- TODO.md: feature and task backlog with completion status
- scripts/update_todos.py: scans source for # TODO/FIXME/HACK markers
  and regenerates the Inline TODOs section in TODO.md
2026-03-17 17:41:46 -07:00
agessaman b72eaab171 Enhance Discord bridge configuration to support multiple webhooks per channel
- Updated config.ini.example and discord-bridge.md to reflect the ability to fan out a single MeshCore channel to multiple Discord servers using a comma- or whitespace-separated list of webhook URLs.
- Modified DiscordBridgeService to handle multiple webhooks per channel, including validation and logging improvements for better monitoring of configured webhooks.
2026-03-16 21:00:57 -07:00
agessaman 2178a80dca Refactor cleanup methods in core.py and mesh_graph.py to suppress logging during shutdown
- Updated the _cleanup_web_viewer and _cleanup_mesh_graph methods to avoid logging errors during shutdown, as the logger's stream may be closed at that time.
- Modified the shutdown method in MeshGraph to prevent logging of flushing errors, enhancing stability during the atexit process.
- Adjusted test configurations to use Path objects for bot_root and local_root, improving path handling in tests.
2026-03-11 20:41:44 -07:00
agessaman 6ee4639ab7 Update test_checkin_service.py to skip tests when checkin_service plugin is not present
- Modified the test file to include a conditional import for the CheckInService, allowing tests to be skipped if the plugin is not installed. This change improves test reliability and clarity regarding plugin dependencies.
2026-03-11 20:29:03 -07:00
agessaman 6c8151389b Enhance MeshGraph edge promotion logic and update BotDataViewer API
- Updated the MeshGraph class to prevent promoting a 1-byte edge to a 3-byte edge when the existing 1-byte edge lacks public keys, ensuring accurate observation attribution.
- Added a new test case to verify the behavior of edge promotion under specific conditions.
- Modified the BotDataViewer API to return the prefix length dynamically based on the edges, improving data consistency and user experience in the web viewer.
- Enhanced the mesh.html template to support displaying prefix byte counts, providing clearer information on node connections.
2026-03-09 09:43:49 -07:00
agessaman 0c060a515b Implement chunked message sending for improved rate limit handling
- Added `send_channel_messages_chunked` method to `CommandManager` for sending multiple messages with appropriate rate-limit spacing.
- Introduced `send_response_chunked` method in `BaseCommand` to facilitate chunked responses in both channels and DMs.
- Updated `GreeterCommand` to utilize the new chunked response functionality, simplifying the greeting message sending process.
- Enhanced documentation in `local-plugins.md` with examples for using chunked message sending, improving clarity on handling long messages and rate limits.
2026-03-07 09:02:12 -08:00
agessaman f57f480c9f Enhance MeshGraph and BotDataViewer for multi-byte support and improved edge handling
- Updated MeshGraph to support multi-resolution storage and node identity, allowing for better handling of 1-byte, 2-byte, and 3-byte edges.
- Implemented logic for merging and promoting edges based on observation counts and specificity.
- Enhanced BotDataViewer API to dynamically adjust prefix length based on configuration, improving data representation.
- Updated mesh.html template to display prefix byte counts, enhancing user experience with clearer information on node connections.
2026-03-06 16:57:57 -08:00
agessaman 7d76ed443b Update dependencies and enhance path handling for multi-byte support
- Updated `meshcore` dependency version to `2.2.14` in both `pyproject.toml` and `requirements.txt`.
- Added multi-byte path support in the `PathCommand`, allowing for 1-, 2-, and 3-byte-per-hop paths.
- Enhanced `MessageHandler` to utilize `routing_info` for accurate path extraction and validation.
- Improved path extraction methods in `MultitestCommand` and `TestCommand` to prefer `routing_info` for node IDs.
- Refactored path handling logic across various commands to ensure consistent multi-byte path processing.
2026-03-05 13:37:38 -08:00
agessaman 4e5addd5df Add support for local plugins and services
- Updated `.gitignore` to include local configuration and plugin directories, allowing users to add custom commands and services without modifying core code.
- Enhanced `config.ini.example` with instructions for using local plugins and added sections for local service configurations.
- Refactored `PluginLoader` and `ServicePluginLoader` to support loading local commands and services from specified directories, improving extensibility.
- Updated `mkdocs.yml` to include documentation for local plugins and the check-in API.
- Added tests to verify the discovery and loading of local plugins, ensuring functionality and preventing name collisions with built-in plugins.
2026-03-04 18:33:45 -08:00
agessaman f789b72b92 Implement bot response bridging for Discord and Telegram
- Added configuration options to bridge the bot's own channel responses to Discord and Telegram, allowing command replies to be sent to respective channels.
- Updated `DiscordBridgeService` and `TelegramBridgeService` to register listeners for bot-sent messages, ensuring they are included in the bridging process.
- Enhanced `CommandManager` to invoke listeners with a synthetic event when a message is successfully sent, providing context about the message.
- Updated `config.ini.example` to reflect the new `bridge_bot_responses` option for both services.
- Added tests to verify the correct invocation of listeners upon successful message sending.
2026-03-02 16:39:30 -08:00
agessaman d2f63c1025 Implement channel-specific triggers for random lines
- Added support for channel-specific triggers in the CommandManager, allowing triggers to be restricted to designated channels via `channel.<key>` or `channels.<key>` configuration.
- Updated `config.ini.example` to include examples of channel restrictions for triggers.
- Enhanced tests to verify that triggers only match in specified channels, ensuring proper functionality of the new feature.
2026-03-02 16:04:12 -08:00
agessaman e348fd8d53 Enhance profanity filtering to include hate symbol detection and censorship
- Updated the profanity filter to check for hate symbols (e.g., swastika Unicode) in addition to word-based profanity, ensuring comprehensive message filtering.
- Modified the `censor` function to replace hate symbols with `***`, maintaining functionality even when the `better-profanity` package is unavailable.
- Updated documentation in `discord-bridge.md` and `telegram-bridge.md` to reflect the new hate symbol handling features.
- Added tests to verify detection and censorship of hate symbols, ensuring robustness of the profanity filter.
2026-03-01 21:56:13 -08:00
agessaman 3c56cbb69d Add profanity filter options for Discord and Telegram bridges
- Introduced `filter_profanity` configuration option in `config.ini.example` to handle profanity in bridged messages: options include `drop` (default), `censor`, or `off`.
- Updated `requirements.txt` to include `better-profanity` for profanity filtering functionality.
- Enhanced `DiscordBridgeService` and `TelegramBridgeService` to implement the profanity filter logic, allowing messages with profanity to be dropped or censored before bridging.
- Updated documentation in `discord-bridge.md` and `telegram-bridge.md` to reflect the new profanity handling feature.
2026-03-01 21:47:53 -08:00
agessaman 4982ba482d Enhance mesh graph handling for multi-resolution prefix support
- Updated `MeshGraph` to support multi-resolution storage of edges, allowing prefixes of 2, 4, or 6 hex chars without truncation.
- Implemented prefix matching logic to ensure distinct links are maintained and accurately retrieved based on prefix queries.
- Refactored methods in `MessageHandler` and `PathCommand` to accommodate variable prefix lengths during graph lookups.
- Enhanced tests to validate prefix match functionality and edge management in the mesh graph.
2026-03-01 20:42:06 -08:00
agessaman 217d2a4089 Refactor database connection handling across multiple modules
- Replaced direct SQLite connection calls with a context manager in various modules to ensure proper resource management and prevent file descriptor leaks.
- Introduced a new `connection` method in `DBManager` to standardize connection handling.
- Updated all relevant database interactions in modules such as `feed_manager`, `scheduler`, `commands`, and others to utilize the new connection method.
- Improved code readability and maintainability by consolidating connection logic.
2026-03-01 14:12:22 -08:00
agessaman 6b624c567a Enhance path handling and database schema for multi-byte prefix support
- Introduced a new utility function `decode_path_len_byte` to decode RF packet path length bytes, supporting both legacy and multi-byte paths.
- Updated various modules to utilize the new decoding logic, ensuring compatibility with configured prefix lengths.
- Modified database schemas to include `bytes_per_hop` and `out_bytes_per_hop` columns for better path management.
- Enhanced path parsing and validation across commands and services to accommodate variable prefix lengths.
- Improved logging and error handling for path-related operations, ensuring robustness during transitions.
2026-02-28 13:03:48 -08:00
Ian Rifkin 36a8a67543 Add transitional support for 2-byte prefixes while keeping legacy 1-byte compatibility
- Update prefix command to accept BOTH legacy 2-char prefixes and
  configured prefix_hex_chars (e.g. 4-char) during firmware transition
- Replace strict length validation with dual-length validation (2 or N)
- Ensure prefix lookups work with either input length via LIKE matching
- Update related SQL prefix extraction to use configured prefix length
- Add fallback handling in path parsing for legacy 2-char route data

Notes:
- This is an interim compatibility change to support mixed networks
  where RF path data is still 1-byte while bot config may be 2-byte.
- Needs additional testing across real multi-hop scenarios and mixed
  bot configurations.
- Translation updates are incomplete: only English strings were updated;
  other translation files still need review.
- Behavior and UX may need refinement after real-world testing.
2026-02-27 23:48:27 -05:00
Ian Rifkin 7671c3a8de Add RandomLine matcher for file-based trigger responses with default momjokes and fun facts 2026-02-25 00:06:09 -05:00
agessaman 5b7250cdd4 Enhance graph data handling and configuration options
- Updated `config.ini.example` to include new settings for edge loading and graph data capture, providing clearer guidance on usage.
- Modified `MeshGraph` class to respect the new `graph_capture_enabled` setting, controlling the collection of edge data from incoming packets.
- Adjusted message handling to ensure graph updates only occur when data capture is enabled, optimizing performance and resource usage.
- Added tests to validate the new configuration options, ensuring proper functionality in various scenarios.
2026-02-18 15:09:53 -08:00
agessaman 04bd004f36 Improve monitor channel handling by adding support for quoted values
- Introduced `strip_optional_quotes` function to handle monitor channel values with optional surrounding quotes.
- Updated `load_monitor_channels` method in `CommandManager` to utilize the new function, ensuring compatibility with quoted and unquoted channel configurations.
- Modified `generate_samples` function to apply the same logic for consistency.
- Added tests for `strip_optional_quotes` to validate behavior with various input cases.
2026-02-15 21:02:48 -08:00
agessaman 8cf7348321 Enhance command tests with additional assertions and new truncation test
- Added assertions to ensure call arguments are not None in multiple command tests.
- Introduced a new test for the CmdCommand class to verify that long command lists are truncated correctly with a '(N more)' suffix, improving command list readability.
2026-02-13 15:45:29 -08:00
agessaman cc91b6ee7a Refactor command configuration handling
- Standardized the configuration keys for various commands by replacing specific `*_enabled` keys with a unified `enabled` key across configuration files.
- Updated command classes to support fallback mechanisms for legacy configuration keys, ensuring backward compatibility.
- Enhanced the logic in the `BaseCommand` class to handle both standard and legacy keys for command enabling.
- Added tests to verify the correct behavior of the new configuration handling and legacy support for commands including Stats, Sports, Hacker, and Alert.
2026-02-12 20:51:52 -08:00
agessaman 2e4d7d0db4 Refactor configuration handling for joke commands
- Standardized the configuration keys for joke commands by replacing `joke_enabled` and `dadjoke_enabled` with a unified `enabled` key in the configuration files.
- Updated the validation logic to warn users when both legacy `[Jokes]` and new `[Joke_Command]`/`[DadJoke_Command]` sections are present, suggesting the removal of the legacy section.
- Enhanced the command classes to support fallback mechanisms for legacy configuration keys, ensuring backward compatibility.
- Added tests to verify the correct behavior of the new configuration handling and legacy support.
2026-02-12 20:31:18 -08:00
agessaman d699ea1cf1 Update configuration handling and validation for bot sections
- Enhanced .gitignore to allow test files in the tests/ directory and committed pytest.ini for test discovery.
- Added checks for missing sections in configuration files, specifically for Admin_ACL and Banned_Users, to prevent errors during bot startup.
- Updated generate_website.py and command_manager.py to handle cases where required sections are absent, returning empty lists instead of raising exceptions.
- Introduced optional dependencies for testing in pyproject.toml, ensuring a smoother development experience.
- Improved localization handling in core.py to default to English when the Localization section is missing, enhancing user experience.
2026-02-12 19:23:35 -08:00
agessaman c21743137c Move tests directory to local docs folder
- Move entire tests/ directory to docs/local/tests/
- Remove all test files from git tracking (20 files removed)
- Add tests/ to .gitignore to prevent future commits of test files
- Tests are development/experimental code not needed for public repository
- Repository is now cleaner and more focused on production code
2025-09-06 11:50:05 -07:00
agessaman 8803070d78 move demo script to tests directory 2025-09-06 11:48:14 -07:00
agessaman 04b3542411 Enhance configuration and command structure for MeshCore Bot. Added RF data correlation settings in config.ini, implemented a BotTxRateLimiter for transmission control, and refactored command handling to support plugins. Removed obsolete scripts and improved command metadata for better usability. 2025-09-04 18:48:59 -07:00