Commit Graph
10 Commits
Author SHA1 Message Date
agessaman 96c4a01788 fix(commands,clients): correct scheduling, parsing and formatting defects
Remaining findings from the same triage pass. None are security
relevant; each produces a wrong user-visible result.

- greeter: treat rollout_started_at as UTC. It is written by SQLite
  CURRENT_TIMESTAMP, but datetime.timestamp() read the naive value as
  local time, shifting the backfill cutoff by the host's offset (7h on
  PT). Users who posted inside that window were never marked as already
  greeted and could be sent a welcome they should not have received.

- greeter: write greeted_at in SQLite's own format. The rollout backfill
  used isoformat() while every other path used CURRENT_TIMESTAMP. "T"
  sorts above a space, so ORDER BY greeted_at interleaved the two
  formats wrongly, corrupting duplicate cleanup and the web viewer's
  recently-greeted list.

- greeter: let an empty `channels =` disable the command. BaseCommand
  reads an empty value as disabled-on-channels, but the greeter
  collapsed "key absent" and "key present but empty" into one fallback
  and kept greeting via monitor_channels.

- greeter: stop comma-splitting greeting text. channel_greetings split
  entries on ",", so "Public:Welcome to the mesh, {sender}!" was stored
  as "Welcome to the mesh" with the placeholder silently dropped. A
  fragment now starts a new entry only when the text before its first
  colon looks like a channel name, which keeps commas, URLs and clock
  times attached to the greeting they belong to.

- wxsim_parser: match condition abbreviations longest-first. Substring
  matching in dict order let RAIN shadow CHNC. RAIN, so five conditions
  lost their "chance" qualifier (rain, snow, drizzle, t-storm, and
  FAIR-P.C.).

- wxsim_parser: re-anchor "now" on each parse. current_date and
  current_year were fixed at construction and wx_command builds one
  parser at startup, so after a few days forecast dates rolled back a
  year and staleness checks read permanently true.

- thesportsdb_client: hold the rate-limit read/sleep/write under a lock.
  Concurrent callers read the same last_request_time, slept the same
  interval and fired together, bursting past the 2.1s throttle. This
  exposes an unrelated request fan-out problem in fetch_league_scores;
  filed in TODO.md rather than fixed here.

- alert_command: stop duplicating the first incident. The tail treated
  any single-line buffer as "header only" and appended incidents[0],
  but messages after the first carry no header, so a final chunk holding
  one incident had incident 0 pasted onto it. The same confusion inside
  the loop also let a second incident be appended past the 130-character
  limit.

- feed: reject non-positive poll intervals. -1 is truthy and was stored,
  and the poller's `now - last_check >= interval` then treats the feed
  as permanently due and re-fetches the URL every cycle; 0 was ignored
  while still reporting success. The web viewer, which is the primary
  editor, had no validation at all, and a JSON null there raised a
  TypeError that aborted the poll cycle for every feed rather than one.
  feed_manager falls back to the default for rows written before this.

- transmission_tracker: age out confirmed transmissions that have
  repeats. Cleanup removed only repeat_count == 0, so repeated records
  accumulated for the lifetime of the process, which matters on a Pi
  Zero. Repeat counts are already persisted to packet_stream, so the
  longer 30-minute retention loses nothing.

- mesh_graph: weighted-merge avg_hop_position when promoting an edge.
  Promotion overwrote the average with the single new observation, so an
  edge averaging 2.0 over 4 observations became 9.0 instead of 3.4 and
  skewed subsequent path scoring.

- multitest_command: show the path that ends exactly at the display LCP.
  The suffix helpers are correct in isolation; the loss happens in the
  cluster formatters, where _shrink_display_lcp refuses to shrink a
  single-token LCP and the trunk then rendered as "96 ┐", meaning
  "everything continues past here" and dropping the bare 96 route while
  the header still counted it. Now uses the file's existing "├ common"
  marker. Empty suffixes are also no longer handed to the nested
  renderer, which mapped them onto the same [] as a route ending at the
  inner LCP and drew a row for a route that did not exist.

- sports_mappings: stop shadowing seven unique team nicknames. In one
  flat dict a repeated key silently drops the earlier team, so hawks
  resolved to the NBA Hawks rather than the Seahawks alias it was added
  as, blazers to Kamloops rather than Portland, and rockets to Kelowna
  rather than Houston. First definition now wins for hawks, giants,
  jets, rangers, kings, blazers and rockets; every shadowed team keeps
  its unambiguous full-name alias. The 55 city and abbreviation
  collisions (chicago, sf, la, ...) are genuinely ambiguous and stay
  last-wins, now pinned by a test so a new collision fails loudly
  instead of passing unnoticed.
2026-07-28 20:08:57 -07:00
agessaman f804d05d7f feat(command-manager): implement DM length guard and auto-splitting for oversized messages
- Added a mechanism to check the byte length of direct messages (DMs) before sending, ensuring they do not exceed the firmware's maximum limit.
- Implemented a method to split oversized messages into smaller chunks, preserving UTF-8 encoding and avoiding mid-codepoint splits.
- Enhanced logging to provide warnings when messages are auto-split, including details on the number of chunks created.
- Updated the DM sending logic to handle both single and split messages efficiently.
- Introduced a new static method for splitting text into UTF-8 chunks, improving message handling across the application.
2026-07-16 12:49:38 -07:00
agessaman 313dfccf75 fix: Align send suppression and security compatibility behavior
Make outbound send suppression consistent by honoring both radio-offline and zombie states across command-manager and scheduler paths. Preserve strict SSRF defaults while adding explicit private-feed URL opt-ins, persist allow_local_smtp from notifications config writes, reconcile zombie alert setting precedence, and replace deprecated UTC timestamp calls with timezone-aware UTC usage.
2026-04-14 12:18:02 -07:00
Stacy Olivasandagessaman 54aeb28bf0 security: SSRF hardening, log injection sanitization, and allow_local_smtp
Add SSRF host validation to maintenance.py send_nightly_email and
scheduler.py send_zombie_alert_email using validate_external_url().
New allow_local_smtp config key permits private-IP SMTP for local
relay setups.

Add sanitize_name() to security_utils and apply it to all log calls
in message_handler, repeater_manager, path_command, solarforecast_command,
command_manager, and discord_bridge_service to prevent log injection.

Move nightly email logic from duplicate scheduler._send_nightly_email()
into the canonical maintenance.py implementation, removing the duplicate.
Update tests to call maintenance.send_nightly_email() directly.

Add validate_external_url allow_private parameter with support for
loopback, RFC1918, CGN, and link-local address ranges.
2026-04-14 10:02:36 -07:00
Stacy Olivas ad77d7b00d fix: BUG-025/026/027/028/029 implementations and ruff/mypy refinements
BUG-025: send_channel_message retry logic on no_event_received
BUG-026: split_text_into_chunks and chunked dispatch in message_handler
BUG-027: test_weekly_on_wrong_day_does_not_run patch uses fake_now
BUG-028: byte_data = b"" initialised before try in decode_meshcore_packet
BUG-029: app.py db_path via self._config_base; realtime.html socket race
  fixed; base.html forceNew removed; ping_timeout 5 to 20s

Additional: ruff and mypy refinements across all modules; discord bridge,
telegram bridge, rate limiter, and service plugin updates
2026-03-17 18:07:19 -07:00
Stacy Olivas ce884cee87 fix: auth, db migrations, retry, chunking, socket race, trace, timezone, repeater, and ruff/mypy cleanup
BUG-001: web viewer login/session auth (in web viewer commit)
BUG-002: db_manager ALTER TABLE for missing channel_operations and
  feed_message_queue columns on startup
BUG-015: scheduler thread blocked on future.result(); replaced all
  blocking waits with add_done_callback (fire-and-forget)
BUG-016: reboot_radio sends meshcore.commands.reboot() before disconnect
BUG-017: radio disconnect uses asyncio.wait_for(timeout=10)
BUG-022: custom asyncio loop exception handler suppresses IndexError
  from meshcore parser at DEBUG level
BUG-024: last_db_backup_run updated after each run; 2-min startup
  window; last-run seeded from DB on restart
BUG-025: send_channel_message retries up to 2 times (2s delay) on
  no_event_received via _is_no_event_received() helper
BUG-026: split_text_into_chunks() and get_max_message_length() added
  to CommandManager; keyword dispatch uses send_response_chunked()
BUG-028: byte_data = b"" initialised before try block in
  decode_meshcore_packet to prevent UnboundLocalError in except handler
TraceCommand: path nodes reversed and return path truncated; fixed
format_elapsed_display: UTC normalisation before elapsed computation (#75)
RepeaterManager: auto_manage_contacts guard before any purge logic (#50)
Command aliases: [Aliases] config section injects shorthands at startup
JSON logging: _JsonFormatter; json_logging = true in [Logging]
Structured JSON logging compatible with Loki, Elasticsearch, Splunk
Discord bridge, Telegram bridge, and all service plugins updated
MeshGraph edge promotion logic corrected
Shutdown: scheduler and meshcore disconnect joined cleanly; log spam fixed
All modules: ruff and mypy cleanup applied (type annotations, imports)
2026-03-17 18:07:18 -07:00
agessaman 217d2a4089 Refactor database connection handling across multiple modules
- Replaced direct SQLite connection calls with a context manager in various modules to ensure proper resource management and prevent file descriptor leaks.
- Introduced a new `connection` method in `DBManager` to standardize connection handling.
- Updated all relevant database interactions in modules such as `feed_manager`, `scheduler`, `commands`, and others to utilize the new connection method.
- Improved code readability and maintainability by consolidating connection logic.
2026-03-01 14:12:22 -08:00
agessaman bb1dd95f1c Refactor timezone handling across commands and utilities
- Introduced a new utility function `get_config_timezone` to centralize timezone retrieval and validation from the bot's configuration.
- Updated various commands and the scheduler to utilize the new function, ensuring consistent timezone handling and fallback mechanisms.
- Removed direct dependencies on `pytz` in favor of a more flexible approach that supports both `pytz` and `zoneinfo` for timezone management.
- Enhanced logging for invalid timezone configurations to improve troubleshooting.
2026-02-22 11:53:33 -08:00
agessaman 32929dedad Update CommandManager with internet connectivity checks for commands
- Introduced caching mechanisms for internet connectivity status in CommandManager to optimize performance and reduce redundant checks.
- Updated command execution logic to skip commands requiring internet access when connectivity is unavailable, improving user experience and error handling.
- Added synchronous and asynchronous utility functions for checking internet connectivity in utils.py.
- Marked relevant commands (e.g., AlertCommand, AqiCommand, DadJokeCommand) as requiring internet access to ensure proper execution conditions.
2025-12-17 12:38:54 -08:00
agessaman c151fba7a5 Enhance configuration and database management for feed and channel operations. Add channel refresh interval setting to config.ini.example, implement feed manager functionality with new database tables for feed subscriptions and activity tracking, and improve greeter command logic with human greeting detection and Levenshtein distance matching for user greetings. Update web viewer to include feed management pages and integrate global template variables for better user experience. 2025-12-06 10:18:12 -08:00