Commit Graph
25 Commits
Author SHA1 Message Date
agessaman 96c4a01788 fix(commands,clients): correct scheduling, parsing and formatting defects
Remaining findings from the same triage pass. None are security
relevant; each produces a wrong user-visible result.

- greeter: treat rollout_started_at as UTC. It is written by SQLite
  CURRENT_TIMESTAMP, but datetime.timestamp() read the naive value as
  local time, shifting the backfill cutoff by the host's offset (7h on
  PT). Users who posted inside that window were never marked as already
  greeted and could be sent a welcome they should not have received.

- greeter: write greeted_at in SQLite's own format. The rollout backfill
  used isoformat() while every other path used CURRENT_TIMESTAMP. "T"
  sorts above a space, so ORDER BY greeted_at interleaved the two
  formats wrongly, corrupting duplicate cleanup and the web viewer's
  recently-greeted list.

- greeter: let an empty `channels =` disable the command. BaseCommand
  reads an empty value as disabled-on-channels, but the greeter
  collapsed "key absent" and "key present but empty" into one fallback
  and kept greeting via monitor_channels.

- greeter: stop comma-splitting greeting text. channel_greetings split
  entries on ",", so "Public:Welcome to the mesh, {sender}!" was stored
  as "Welcome to the mesh" with the placeholder silently dropped. A
  fragment now starts a new entry only when the text before its first
  colon looks like a channel name, which keeps commas, URLs and clock
  times attached to the greeting they belong to.

- wxsim_parser: match condition abbreviations longest-first. Substring
  matching in dict order let RAIN shadow CHNC. RAIN, so five conditions
  lost their "chance" qualifier (rain, snow, drizzle, t-storm, and
  FAIR-P.C.).

- wxsim_parser: re-anchor "now" on each parse. current_date and
  current_year were fixed at construction and wx_command builds one
  parser at startup, so after a few days forecast dates rolled back a
  year and staleness checks read permanently true.

- thesportsdb_client: hold the rate-limit read/sleep/write under a lock.
  Concurrent callers read the same last_request_time, slept the same
  interval and fired together, bursting past the 2.1s throttle. This
  exposes an unrelated request fan-out problem in fetch_league_scores;
  filed in TODO.md rather than fixed here.

- alert_command: stop duplicating the first incident. The tail treated
  any single-line buffer as "header only" and appended incidents[0],
  but messages after the first carry no header, so a final chunk holding
  one incident had incident 0 pasted onto it. The same confusion inside
  the loop also let a second incident be appended past the 130-character
  limit.

- feed: reject non-positive poll intervals. -1 is truthy and was stored,
  and the poller's `now - last_check >= interval` then treats the feed
  as permanently due and re-fetches the URL every cycle; 0 was ignored
  while still reporting success. The web viewer, which is the primary
  editor, had no validation at all, and a JSON null there raised a
  TypeError that aborted the poll cycle for every feed rather than one.
  feed_manager falls back to the default for rows written before this.

- transmission_tracker: age out confirmed transmissions that have
  repeats. Cleanup removed only repeat_count == 0, so repeated records
  accumulated for the lifetime of the process, which matters on a Pi
  Zero. Repeat counts are already persisted to packet_stream, so the
  longer 30-minute retention loses nothing.

- mesh_graph: weighted-merge avg_hop_position when promoting an edge.
  Promotion overwrote the average with the single new observation, so an
  edge averaging 2.0 over 4 observations became 9.0 instead of 3.4 and
  skewed subsequent path scoring.

- multitest_command: show the path that ends exactly at the display LCP.
  The suffix helpers are correct in isolation; the loss happens in the
  cluster formatters, where _shrink_display_lcp refuses to shrink a
  single-token LCP and the trunk then rendered as "96 ┐", meaning
  "everything continues past here" and dropping the bare 96 route while
  the header still counted it. Now uses the file's existing "├ common"
  marker. Empty suffixes are also no longer handed to the nested
  renderer, which mapped them onto the same [] as a route ending at the
  inner LCP and drew a row for a route that did not exist.

- sports_mappings: stop shadowing seven unique team nicknames. In one
  flat dict a repeated key silently drops the earlier team, so hawks
  resolved to the NBA Hawks rather than the Seahawks alias it was added
  as, blazers to Kamloops rather than Portland, and rockets to Kelowna
  rather than Houston. First definition now wins for hawks, giants,
  jets, rangers, kings, blazers and rockets; every shadowed team keeps
  its unambiguous full-name alias. The 55 city and abbreviation
  collisions (chicago, sf, la, ...) are genuinely ambiguous and stay
  last-wins, now pinned by a test so a new collision fails loudly
  instead of passing unnoticed.
2026-07-28 20:08:57 -07:00
agessaman f804d05d7f feat(command-manager): implement DM length guard and auto-splitting for oversized messages
- Added a mechanism to check the byte length of direct messages (DMs) before sending, ensuring they do not exceed the firmware's maximum limit.
- Implemented a method to split oversized messages into smaller chunks, preserving UTF-8 encoding and avoiding mid-codepoint splits.
- Enhanced logging to provide warnings when messages are auto-split, including details on the number of chunks created.
- Updated the DM sending logic to handle both single and split messages efficiently.
- Introduced a new static method for splitting text into UTF-8 chunks, improving message handling across the application.
2026-07-16 12:49:38 -07:00
agessaman 3ccfd63613 feat(config): add config panel for modifying plugin settings, with config schema in each command and service plugin. 2026-07-12 11:57:28 -07:00
agessaman 63592bdd65 feat(commands, config): implement path byte length requirements for commands
- Added configuration options to specify minimum path byte length requirements for the path, test, and multitest commands.
- Introduced methods to enforce these requirements and handle failure responses.
- Updated the packet capture service to correctly calculate packet hashes using the raw wire byte length.
- Enhanced unit tests to verify the correct behavior of the new path byte length logic.
2026-04-25 09:05:17 -07:00
Stacy Olivasandagessaman 242e0a5df1 fix: ruff/mypy compliance for issues introduced by PR #152
PR #152 introduced trailing whitespace in several command files, set
message.content_lower at runtime without declaring it on the MeshMessage
dataclass, and left unused/unsorted imports in test and service files.
All cause ruff/mypy CI failures on branches that rebase onto dev after
that merge.

Linting fixes (ruff --fix):
- modules/command_manager.py: restore PUBLIC_CHANNEL_KEY_HEX re-export
  with noqa guard (core.py imports it from here; auto-fix silently dropped it)
- modules/commands/multitest_command.py: strip W291/W293 trailing whitespace
- modules/commands/path_command.py: strip W293 trailing whitespace
- modules/commands/prefix_command.py: strip W291 trailing whitespace
- modules/commands/roll_command.py: strip W293 trailing whitespace
- modules/commands/sports_command.py: strip W293 trailing whitespace
- modules/core.py: strip W293 blank-line whitespace
- modules/service_plugins/packet_capture_service.py: sort imports (I001)
- modules/version_info.py: remove unused typing.Any import (F401)
- tests/integration/test_flood_scope_reply.py: remove unused call import
- tests/unit/test_log_data_scope_fields.py: remove unused asyncio import
- tests/unit/test_public_channel_guard.py: sort imports, remove unused
  patch and validate_config imports

Dataclass fix (mypy attr-defined):
- modules/models.py: declare content_lower field on MeshMessage so mypy
  resolves the attribute set by base_command.cleanup_message_for_matching

Must be merged before or alongside PRs #155–#158 to clear CI on those
branches.
2026-04-14 10:10:41 -07:00
Adam GessamanandGitHub d9e32c86f2 Merge branch 'dev' into main 2026-04-11 19:00:06 -07:00
agessaman 8bea10cdc3 Enhance mention handling in message processing
- Updated `config.ini.example` to introduce the `respond_to_mentions` setting, allowing configuration of how the bot responds to mentions in channel messages.
- Refactored `MessageHandler` to implement logic for handling mentions based on the new configuration, including stripping mentions when appropriate.
- Added `cleanup_message_for_matching` method in `BaseCommand` to streamline message processing and mention validation.
- Enhanced various command classes to utilize the new cleanup method for consistent mention handling.
- Introduced tests to validate the behavior of the new mention handling logic across different configurations.
2026-04-11 18:45:34 -07:00
Chris Wiegand 277491f535 create common base function to match replies of bot name even when command class is inherited, patch all overriding implementations to use this 2026-04-11 15:34:00 -06:00
agessaman 3670f5db41 Enhance path condensing functionality in MultitestCommand
- Introduced a new `CondensePathsMode` type to support multiple condensing styles: "off", "flat", and "nested".
- Updated `_condense_path_lines` to handle different condensing modes, improving path representation.
- Enhanced test coverage for path condensing scenarios, ensuring accurate output for various path configurations.

These changes improve the flexibility and usability of path handling in the MultitestCommand, allowing for better visualization of path structures.
2026-04-05 10:47:13 -07:00
agessaman 12e677a9cf Update default value for condense_paths in MultitestCommand
- Changed the fallback value for condense_paths from False to True in the MultitestCommand class, ensuring that paths are condensed by default. This adjustment improves the command's usability and aligns with expected behavior.
2026-04-04 09:52:22 -07:00
agessaman 43f450e702 Update path formatting and message handling in commands
- Modified path formatting in `multitest_command.py` to use ideographic space for nested branches, enhancing visual clarity.
- Updated the `schedule_command.py` to strip control characters from messages, ensuring safe and clean previews.
- Adjusted test cases in `test_multitest_command.py` to reflect changes in path formatting, improving consistency across tests.

These changes enhance the readability of path outputs and improve message handling in scheduled commands.
2026-03-29 20:27:18 -07:00
agessaman 83b23385a7 Refine path formatting and testing in MultitestCommand
- Updated path formatting logic in `MultitestCommand` to include a corner marker (┐) for shared paths, enhancing visual clarity.
- Improved documentation in `config.ini.example` to reflect changes in path formatting behavior.
- Expanded test cases to validate the new corner marker functionality and ensure accurate representation of unique paths with shared prefixes.
2026-03-29 13:32:30 -07:00
agessaman 99e660afc3 Enhance path formatting in MultitestCommand with new grouping logic
- Updated the path formatting logic in `MultitestCommand` to support nested branches using shared prefixes and horizontal continuations.
- Introduced new utility functions for grouping and formatting suffix lines, improving the clarity of output paths.
- Modified the configuration example to clarify the behavior of the `condense_paths` option.
- Expanded test coverage to validate the new path formatting behavior, ensuring accurate representation of unique paths.
2026-03-29 11:02:30 -07:00
agessaman 9859b41d9e Implement path condensing feature in MultitestCommand and update configuration
- Added a new configuration option `condense_paths` in `config.ini.example` to control path formatting.
- Enhanced `MultitestCommand` to condense output paths using shared prefixes and tree branches when `condense_paths` is enabled.
- Introduced utility functions for path processing, including `_condense_path_lines` and `_path_to_tokens`.
- Updated tests to validate the new path condensing functionality and configuration behavior.
2026-03-29 10:14:38 -07:00
Stacy Olivas ad77d7b00d fix: BUG-025/026/027/028/029 implementations and ruff/mypy refinements
BUG-025: send_channel_message retry logic on no_event_received
BUG-026: split_text_into_chunks and chunked dispatch in message_handler
BUG-027: test_weekly_on_wrong_day_does_not_run patch uses fake_now
BUG-028: byte_data = b"" initialised before try in decode_meshcore_packet
BUG-029: app.py db_path via self._config_base; realtime.html socket race
  fixed; base.html forceNew removed; ping_timeout 5 to 20s

Additional: ruff and mypy refinements across all modules; discord bridge,
telegram bridge, rate limiter, and service plugin updates
2026-03-17 18:07:19 -07:00
Stacy Olivas ce884cee87 fix: auth, db migrations, retry, chunking, socket race, trace, timezone, repeater, and ruff/mypy cleanup
BUG-001: web viewer login/session auth (in web viewer commit)
BUG-002: db_manager ALTER TABLE for missing channel_operations and
  feed_message_queue columns on startup
BUG-015: scheduler thread blocked on future.result(); replaced all
  blocking waits with add_done_callback (fire-and-forget)
BUG-016: reboot_radio sends meshcore.commands.reboot() before disconnect
BUG-017: radio disconnect uses asyncio.wait_for(timeout=10)
BUG-022: custom asyncio loop exception handler suppresses IndexError
  from meshcore parser at DEBUG level
BUG-024: last_db_backup_run updated after each run; 2-min startup
  window; last-run seeded from DB on restart
BUG-025: send_channel_message retries up to 2 times (2s delay) on
  no_event_received via _is_no_event_received() helper
BUG-026: split_text_into_chunks() and get_max_message_length() added
  to CommandManager; keyword dispatch uses send_response_chunked()
BUG-028: byte_data = b"" initialised before try block in
  decode_meshcore_packet to prevent UnboundLocalError in except handler
TraceCommand: path nodes reversed and return path truncated; fixed
format_elapsed_display: UTC normalisation before elapsed computation (#75)
RepeaterManager: auto_manage_contacts guard before any purge logic (#50)
Command aliases: [Aliases] config section injects shorthands at startup
JSON logging: _JsonFormatter; json_logging = true in [Logging]
Structured JSON logging compatible with Loki, Elasticsearch, Splunk
Discord bridge, Telegram bridge, and all service plugins updated
MeshGraph edge promotion logic corrected
Shutdown: scheduler and meshcore disconnect joined cleanly; log spam fixed
All modules: ruff and mypy cleanup applied (type annotations, imports)
2026-03-17 18:07:18 -07:00
agessaman 7d76ed443b Update dependencies and enhance path handling for multi-byte support
- Updated `meshcore` dependency version to `2.2.14` in both `pyproject.toml` and `requirements.txt`.
- Added multi-byte path support in the `PathCommand`, allowing for 1-, 2-, and 3-byte-per-hop paths.
- Enhanced `MessageHandler` to utilize `routing_info` for accurate path extraction and validation.
- Improved path extraction methods in `MultitestCommand` and `TestCommand` to prefer `routing_info` for node IDs.
- Refactored path handling logic across various commands to ensure consistent multi-byte path processing.
2026-03-05 13:37:38 -08:00
agessaman 6b624c567a Enhance path handling and database schema for multi-byte prefix support
- Introduced a new utility function `decode_path_len_byte` to decode RF packet path length bytes, supporting both legacy and multi-byte paths.
- Updated various modules to utilize the new decoding logic, ensuring compatibility with configured prefix lengths.
- Modified database schemas to include `bytes_per_hop` and `out_bytes_per_hop` columns for better path management.
- Enhanced path parsing and validation across commands and services to accommodate variable prefix lengths.
- Improved logging and error handling for path-related operations, ensuring robustness during transitions.
2026-02-28 13:03:48 -08:00
agessaman 0e908cd793 Update multitest command to silently ignore subsequent session requests
- Modified the behavior of the `MultitestCommand` class to silently ignore additional session requests from users who already have an active session, allowing the first session to complete without interruption.
2026-02-16 19:24:00 -08:00
agessaman d97c0886e7 feat: Refactor multitest command to support user sessions and improve path tracking
- Introduced a new MultitestSession dataclass to manage active multitest sessions per user, enhancing concurrency handling.
- Replaced global variables with session-specific attributes to track listening duration, collected paths, and target packet hashes.
- Updated message handling logic to check active sessions and prevent race conditions during multitest execution.
- Enhanced path collection and RF data scanning to be user-specific, improving accuracy and logging for each session.
- Implemented execution locks to ensure thread safety during multitest command execution.
2026-01-25 20:47:01 -08:00
agessaman d2abc49048 feat: Enhance command documentation and usage information
- Updated the `generate_html` function to include detailed command usage information, including syntax, examples, and parameters for better user guidance.
- Added CSS styles for improved presentation of command usage and parameters in the generated website documentation.
- Enhanced command classes with structured documentation fields, allowing for consistent and informative command descriptions across the platform.
2026-01-15 20:04:58 -08:00
agessaman cc3fffeb54 feat: Add command enable/disable configuration for various commands
- Implemented a configuration option for enabling or disabling commands across multiple command classes.
- Each command now checks its enabled state before execution, improving control over command availability.
- Updated the configuration loading mechanism to retrieve the enabled state from the config file for commands like Advert, AQI, Catfact, and others.
2026-01-10 09:33:15 -08:00
agessaman 3687dde3fd Add response format configuration for multitest command and enhance greeter command rollout logic. Update .gitignore to exclude database write-ahead logs and shared memory files. Improve error handling and logging in greeter command for better tracking of rollout status and user greetings. 2025-12-02 19:46:32 -08:00
agessaman 6c660dc14c Enhance configuration and command files to support new distance metrics in message responses. Updated Keywords section to include {path_distance} and {firstlast_distance} placeholders. Refactored distance calculation methods into utils and improved path command logic for better accuracy in distance reporting. Added detailed logging for packet hash calculations and message handling in multitest command. 2025-11-20 21:41:10 -08:00
agessaman 0ec60e48c2 Add multitest functionality to message handler for path collection and implement packet hash calculation for message tracking. Updated translations to support multitest commands. 2025-11-18 20:11:11 -08:00