Commit Graph

650 Commits

Author SHA1 Message Date
Stacy Olivas d7cf0d5d2b fix: add per-test timeout to pytest.ini to prevent CI hangs
pytest.ini takes precedence over pyproject.toml [tool.pytest.ini_options]
so the timeout = 30 setting there was silently ignored. Add timeout = 30
and timeout_method = thread directly to pytest.ini so blocking tests are
killed after 30 s in CI.
2026-04-16 18:35:48 -07:00
Stacy Olivas 2a80f76432 sync: use explicit CGN network check in validate_external_url (from PR #140)
Replace is_global fallback with canonical _CGN_NETWORK constant check to
stay consistent with PR #140's approach across all post-140 branches.
2026-04-16 18:35:48 -07:00
Stacy Olivas db0e1c6539 fix: resolve pr/web-viewer-ux CI failures
- Add allow_private param to validate_external_url (alias for
  allow_localhost) to unblock web viewer SSRF guard using allow_private=
- Block non-globally-routable IPs (RFC 6598 100.64.0.0/10 CGN) on
  Python 3.10 which does not classify them as private or reserved
- Remove tests for greeter DB tables and admin_config template that
  depend on features not present on this branch
2026-04-16 18:35:14 -07:00
Stacy Olivas a15827be8f usability: API Explorer tab, actionable error messages (USE-05, USE-06)
- USE-05: Add /api-explorer page listing all ~65 API endpoints in 9
  categories (System, Contacts, Mesh, Channels, Feeds, Radio, Admin,
  Maintenance, Config, Greeter) with method badges, descriptions, and
  curl example modal. Filter bar and collapse per section. Nav item
  added to base.html.

- USE-06: Three targeted error-message improvements:
  1. 500 handler now returns user-friendly HTML page (error.html) for
     browser requests and sanitized JSON for API/JSON requests instead
     of a bare string.
  2. Feed processed-items query failures promoted from logger.debug to
     logger.warning so operators see them in normal log output.
  3. Global JS fetch interceptor in base.html redirects to /login?next=
     on any 401 response, handling session expiry mid-page.

- Fix pre-existing test bug: test_reload_endpoint_success mock return
  value did not match actual code message from reload_config.
2026-04-16 18:33:40 -07:00
Stacy Olivas 23f652fe88 feat: early web viewer start with initializing banner, live banner polling 2026-04-16 18:25:19 -07:00
Adam Gessaman de6279c7c5 Merge pull request #162 from agessaman/integration/kg7qin
Enhance zombie detection, alert system, and radio configuration

Add fortune command via [RandomLine]
Bump meshcore>=2.3.6
2026-04-16 15:41:40 -07:00
agessaman 6ef3d16656 chore: update meshcore dependency version in configuration files
Bumped the meshcore dependency version from 2.2.31 to 2.3.6 in both pyproject.toml and requirements.txt to ensure compatibility with the latest features and improvements. This change helps maintain the project's dependency integrity.
2026-04-16 14:17:30 -07:00
agessaman 572652e712 refactor: improve database connection handling in web viewer and tests
Updated the BotDataViewer class to utilize a context manager for database connections, enhancing resource management. Additionally, refactored test files to implement a centralized approach for managing SQLite connections, ensuring proper cleanup after tests. This change improves code maintainability and reliability across the application.
2026-04-16 11:18:59 -07:00
agessaman a4d5f5478b feat: add RandomLine command support for website generation
Introduced a new command structure for RandomLine entries in the website generation process, allowing for dynamic rendering of random lines based on configuration. Updated the configuration file to include a new category option for commands and enhanced documentation to reflect these changes. This addition improves the command reference organization and user experience on the website.
2026-04-15 09:40:39 -07:00
agessaman 90fdd0c77a feat: add cmd_reference_url option for Cmd_Command
Introduced a new configuration option `cmd_reference_url` in the Cmd_Command section, allowing users to override the default `cmd` output with a link to a full documentation page. Updated the CmdCommand class to utilize this new setting and modified related documentation and tests to ensure proper functionality.
2026-04-15 09:26:19 -07:00
agessaman e058da4968 fix: harden shutdown (single stop, viewer, MQTT logs, scheduler)
- Make MeshCoreBot.stop idempotent; remove duplicate stop from start().
- Avoid web viewer restart during shutdown; gate main-loop restarts.
- Fix packet capture MQTT callbacks to log each broker’s host.
- Only shutdown APScheduler when running; silence double-shutdown noise.
- Add regression tests in tests/test_shutdown_reliability.py.
2026-04-14 12:46:01 -07:00
agessaman 313dfccf75 fix: Align send suppression and security compatibility behavior
Make outbound send suppression consistent by honoring both radio-offline and zombie states across command-manager and scheduler paths. Preserve strict SSRF defaults while adding explicit private-feed URL opt-ins, persist allow_local_smtp from notifications config writes, reconcile zombie alert setting precedence, and replace deprecated UTC timestamp calls with timezone-aware UTC usage.
2026-04-14 12:18:02 -07:00
agessaman b8210b0fbe feat: enhance radio and admin configuration settings
Added new configuration options for radio health monitoring and alerting in config.ini.example, including radio_offline_threshold, radio_offline_alert_enabled, and radio_offline_alert_email. Introduced an [Admin] section with settings for a local admin API, including enabled, port, and token fields. Updated related files to reflect these changes, improving the organization and functionality of the configuration structure.
2026-04-14 11:44:17 -07:00
agessaman af6f693cff refactor: update max_results configuration for airplanes command
Changed the default value of max_results in config.ini.example from 0 (no limit) to 3, ensuring a more manageable output size for aircraft listings. Updated the AirplanesCommand class to reflect this new default and added logic to handle message size constraints when formatting the aircraft list. This refactor improves the user experience by preventing excessive data in responses while maintaining functionality.
2026-04-14 11:35:00 -07:00
agessaman 52227ae6ac refactor: replace fortune command with random lines fortunes command
Removed the fortune command and its associated files, transitioning to existing new random lines command that retrieves fortunes from a different file structure. Updated configuration settings in config.ini.example to reflect this change, including the new file path for fortunes. Adjusted the .gitignore to accommodate the new directory structure for random lines. This refactor enhances the organization of fortune-related content and improves maintainability.
2026-04-14 11:27:40 -07:00
agessaman 293383f885 refactor: reorganize radio configuration settings
Moved radio-related configuration options from the [Bot] section to a new [Connection] section in config.ini.example for better organization. Updated the core logic to reference the new configuration paths, ensuring that radio health monitoring and alert settings are correctly handled. This change enhances clarity and maintainability of the configuration structure.
2026-04-14 11:14:28 -07:00
agessaman 6e8204c1ec fix: enhance path validation for security
Updated the path validation logic in security_utils.py and web_viewer/app.py to include additional dangerous prefixes, specifically targeting private directories. This change aims to strengthen security by preventing access to sensitive system paths. Additionally, modified the test for posting feeds to mock the external URL validation, ensuring consistent behavior during tests.
2026-04-14 10:25:18 -07:00
agessaman f2936be473 docs: record kg7qin integration execution results
Document integrated and skipped PR slices plus current validation status so follow-up work can target remaining failing tests.

Made-with: Cursor
2026-04-14 10:15:15 -07:00
agessaman 8b6864471f fix: resolve scheduler duplicate and mypy fallback types
Remove duplicate zombie-alert implementation and add typed unidecode fallback to restore lint compliance on the integration branch.

Made-with: Cursor
2026-04-14 10:14:38 -07:00
agessaman 46d3fab125 fix: keep airplanes output single-message
Prevent multi-message aircraft list replies by capping formatted output to one frame, preserving anti-flood behavior on mesh channels.

Made-with: Cursor
2026-04-14 10:11:06 -07:00
Stacy Olivas 242e0a5df1 fix: ruff/mypy compliance for issues introduced by PR #152
PR #152 introduced trailing whitespace in several command files, set
message.content_lower at runtime without declaring it on the MeshMessage
dataclass, and left unused/unsorted imports in test and service files.
All cause ruff/mypy CI failures on branches that rebase onto dev after
that merge.

Linting fixes (ruff --fix):
- modules/command_manager.py: restore PUBLIC_CHANNEL_KEY_HEX re-export
  with noqa guard (core.py imports it from here; auto-fix silently dropped it)
- modules/commands/multitest_command.py: strip W291/W293 trailing whitespace
- modules/commands/path_command.py: strip W293 trailing whitespace
- modules/commands/prefix_command.py: strip W291 trailing whitespace
- modules/commands/roll_command.py: strip W293 trailing whitespace
- modules/commands/sports_command.py: strip W293 trailing whitespace
- modules/core.py: strip W293 blank-line whitespace
- modules/service_plugins/packet_capture_service.py: sort imports (I001)
- modules/version_info.py: remove unused typing.Any import (F401)
- tests/integration/test_flood_scope_reply.py: remove unused call import
- tests/unit/test_log_data_scope_fields.py: remove unused asyncio import
- tests/unit/test_public_channel_guard.py: sort imports, remove unused
  patch and validate_config imports

Dataclass fix (mypy attr-defined):
- modules/models.py: declare content_lower field on MeshMessage so mypy
  resolves the attribute set by base_command.cleanup_message_for_matching

Must be merged before or alongside PRs #155–#158 to clear CI on those
branches.
2026-04-14 10:10:41 -07:00
Stacy Olivas 7403c1e946 feat: airplanes command sends all aircraft without truncation
Remove the "(x more)" truncation from _format_aircraft_list — all
matching aircraft are now returned. The hard max_results=10 cap is
replaced with max_results=0 (no limit) as the default; operators can
still set max_results in config.ini to impose a ceiling, and users can
pass limit=N per-query. List results always route through
_send_split_response for chunked multi-message delivery.
2026-04-14 10:07:38 -07:00
Stacy Olivas 13c10fd1a5 feat: add fortune command reading BSD fortune file format
Adds a new !fortune command that picks a random fortune from a
configured BSD-format fortune file (entries separated by a line
containing only %). The parser handles: % as separator or terminator,
multi-line fortunes, trailing whitespace, and % embedded mid-line.

Path validation via validate_safe_path() blocks path traversal before
any file open. Command is auto-discovered by the plugin loader.

Config section [Fortune_Command] added to config.ini.example with
enabled and file keys. Default file: data/fortune/fortunes.txt.

22 tests cover: parsing edge cases, security path rejection,
can_execute enable/disable, execute happy path, empty file fallback,
and plugin metadata.

Fortune file: BSD V8-fortunes (fortunes-freebsd-classic)
Source: https://github.com/HubTou/fortunes-freebsd-classic
1,903 fortunes drawn from the classic BSD UNIX fortune database.

Content note -- data/fortune/fortunes.txt (1,903 fortunes):
No racist, lewd, or pornographic content found. Six lines flagged
during review as mildly coarse; retained but noted here for reference
in case a future maintainer wishes to remove them:

  L1763: Some days chicken salad -- some days chicken shit.
  L2289: We retard what we cannot repel, we palliate what we cannot
         cure. -Johnson
  L3009: In New York they signal the wrong way just to fuck you up.
         -David Yost
  L3161: Sure [Somoza]'s a son-of-a-bitch, but he's OUR
         son-of-a-bitch.  -F.D. Roosevelt
  L3305: When better machines are built, jks will break them, and td
         will bitch about it.
  L3589: It's the thought, if any, that counts! -Dick Grantges
         ("Dick" is a proper name, not a slur)
2026-04-14 10:07:35 -07:00
Stacy Olivas 773b80f6ae feat: bot admin HTTP server + reload_config.sh CLI
- core.py: add _BotAdminServer daemon thread (Flask, 127.0.0.1 only,
  bearer token auth); POST /api/admin/reload calls reload_config() and
  returns JSON {success, message}; GET /api/admin/health; started from
  start() when [Admin] enabled = true and token is set
- scripts/reload_config.sh: curl wrapper for the reload API; reads
  port/token from config.ini [Admin] section; exits 1 on rejection
- tests/test_core.py: TestBotAdminServer — 7 tests covering server
  creation, missing token guard, reload success/failure/auth, health
2026-04-14 10:07:04 -07:00
agessaman 887068faa2 fix: resolve merge-marker cleanup and concise config docs
Clean up residual cherry-pick conflict markers and keep SMTP guidance in config templates brief while preserving full behavior in code and tests.

Made-with: Cursor
2026-04-14 10:06:44 -07:00
Stacy Olivas 485edc6c91 fix: post-rebase compatibility fixes for #147 (stability hardening)
Two incompatibilities surfaced after rebasing onto upstream/dev in code
introduced by #147:

- tests/integration/test_flood_scope_reply.py: add bot.is_radio_zombie = False
  mock; the zombie-detection guard added to send_channel_message in #147
  causes the call to short-circuit without it, failing the scope assertion;
  also removes unused `call` import
- modules/web_viewer/app.py: replace ~50-line inline _get_version_info()
  with 5-line delegate to resolve_runtime_version(); removes now-unnecessary
  import subprocess; the resolve_runtime_version import is now actually used

Do not merge until #147 is merged.
2026-04-14 10:03:05 -07:00
Stacy Olivas a75ea16609 fix: post-rebase compatibility fixes for #138 (radio reliability)
Two small incompatibilities surfaced after rebasing dev-kg7qin-changes
onto upstream/dev in code introduced by #138:

- message_handler.py: replace two remaining Optional[str] annotations
  with str | None; the Optional import was removed during typing
  modernization but two annotations in this module were missed
- command_manager.py: restore # noqa: F401 guard on PUBLIC_CHANNEL_KEY_HEX
  re-export so ruff auto-fix does not silently remove it (imported by core.py)

Do not merge until #138 is merged.
2026-04-14 10:02:48 -07:00
Stacy Olivas 973d1fc947 fix: BUG-028 byte_data init; BUG-LOG-1/4 SocketIO and cleanup_database handler fixes 2026-04-14 10:02:46 -07:00
Stacy Olivas 7679eb7e2c fix: ruff and mypy compliance across utility scripts and tests
Fix ruff violations (import sort, unused imports, whitespace) in
backup_database.py, generate_website.py, meshcore_bot.py,
migrate_webviewer_db.py, packet_capture_service.py, version_info.py,
scripts/config_tui.py, scripts/update_todos.py, and validate_config.py.

Fix mypy stale type-ignore and remove unused imports in test files.
Add is_radio_zombie/is_radio_offline mock attributes to command_manager
test fixtures to prevent truthy Mock early returns.
2026-04-14 10:02:36 -07:00
Stacy Olivas ce7adc55f8 ci: add log injection regression check to CI pipeline
Add scripts/check_log_injection.py to scan for unsanitized variables in
log calls and fail CI if new violations are introduced. Baseline is
committed at zero violations after fixing all 26 pre-existing ones.

Update TESTING.md with instructions for running the check locally.
2026-04-14 10:02:36 -07:00
Stacy Olivas 54aeb28bf0 security: SSRF hardening, log injection sanitization, and allow_local_smtp
Add SSRF host validation to maintenance.py send_nightly_email and
scheduler.py send_zombie_alert_email using validate_external_url().
New allow_local_smtp config key permits private-IP SMTP for local
relay setups.

Add sanitize_name() to security_utils and apply it to all log calls
in message_handler, repeater_manager, path_command, solarforecast_command,
command_manager, and discord_bridge_service to prevent log injection.

Move nightly email logic from duplicate scheduler._send_nightly_email()
into the canonical maintenance.py implementation, removing the duplicate.
Update tests to call maintenance.send_nightly_email() directly.

Add validate_external_url allow_private parameter with support for
loopback, RFC1918, CGN, and link-local address ranges.
2026-04-14 10:02:36 -07:00
Stacy Olivas c7fa0ba3d2 fix: resolve ruff F401 violations in command_manager and upstream test files
- Add noqa: F401 to PUBLIC_CHANNEL_KEY_HEX import (re-exported for core.py)
- Remove unused `call` import from test_flood_scope_reply.py
- Remove unused `asyncio` import from test_log_data_scope_fields.py
- Remove unused `patch` and `validate_config` imports from
  test_public_channel_guard.py; fix import sort order
2026-04-14 10:01:51 -07:00
Stacy Olivas 68fb1ef080 fix: stub is_radio_zombie in flood scope test mock
The send_channel_message test was leaving bot.is_radio_zombie as an
unset MagicMock attribute, which evaluates truthy and triggers the
zombie-state early return before flood scope logic runs.

Add bot.is_radio_zombie = False to the mock setup so the function
reaches the set_flood_scope call as intended.
2026-04-14 10:01:51 -07:00
Stacy Olivas a35aecd0d1 fix: ruff compliance for backup_database.py 2026-04-14 10:01:51 -07:00
Stacy Olivas 640568bcc9 fix: add X-Requested-With header to zombie-recover and radio-offline-clear fetch calls
Both banner action buttons were posting without the required
X-Requested-With header, causing the CSRF guard to return 403.
2026-04-14 10:01:51 -07:00
Stacy Olivas ae71179940 fix: ruff compliance for upstream version_info.py and packet_capture_service.py 2026-04-14 10:01:51 -07:00
Stacy Olivas eaf5324f0d fix: restore MaintenanceRunner patch targets in TestMaybeRunDbBackup
ff7917e incorrectly changed all test patches from
`scheduler.maintenance.run_db_backup` to `scheduler._run_db_backup`,
and rewrote `_setup` to bypass the delegation architecture upstream uses.

- Restore `_setup` to use `scheduler.maintenance.get_maint` side_effect
- Restore all 6 tests to patch `scheduler.maintenance.run_db_backup`
- Add `patch.object(scheduler.maintenance, "_get_current_time")` to
  `test_weekly_on_wrong_day_does_not_run` to prevent Monday flakiness
  in UTC CI environments (maintenance captures `_get_current_time` as a
  bound method, so patching only `scheduler.get_current_time` is not
  sufficient)
2026-04-14 10:01:51 -07:00
Stacy Olivas b56fa83fe3 fix: remove cross-branch security test classes from radio-reliability PR 2026-04-14 10:01:51 -07:00
Stacy Olivas fd0875611d fix: restore MaintenanceRunner arch and correct zombie-detection tests 2026-04-14 10:01:51 -07:00
Stacy Olivas c543cac5bf fix: import validate_external_url in scheduler.py for SMTP SSRF guard 2026-04-14 10:01:51 -07:00
Stacy Olivas cac999bcde fix: remove trailing whitespace in core.py (W293) 2026-04-14 10:01:51 -07:00
Stacy Olivas f09b214a73 fix: prevent packetcapture restart storm during radio reconnect 2026-04-14 10:01:51 -07:00
Stacy Olivas 9ce69702c2 feat: radio debug logging mode with web UI toggle 2026-04-14 10:01:51 -07:00
Stacy Olivas 51ab5d312c feat: radio-offline fail state — suppress sends, auto-restart, banner, and docs 2026-04-14 10:01:51 -07:00
Stacy Olivas 8b14c40958 feat: zombie radio banner on all pages + zombie alert config screen options
- base.html: persistent danger banner appears on every web page when
  is_radio_zombie is true; shows datetime zombie was detected; includes
  "Restart Bot Processing" button (POST /api/admin/zombie-recover) that
  clears _radio_zombie_detected and _radio_fail_count on the live bot
  object and removes the persisted DB flag; banner turns green on success

- config.html: new "Zombie Radio Alert" card with enable/disable toggle
  and alert-email field; "Save" writes to bot_metadata (immediate,
  survives restarts); "Save to config.ini" also persists values to
  config.ini and keeps the in-memory config in sync; card shows
  current config.ini values as baseline defaults

- app.py: inject_template_vars context processor now provides
  radio_zombie and radio_zombie_since to all templates; added
  GET/POST /api/config/zombie-alert endpoints (GET returns both
  bot_metadata and config.ini values; POST supports write_to_config
  flag); added POST /api/admin/zombie-recover endpoint; stored
  config_path on self for write-back use

- scheduler.py: send_zombie_alert_email now prefers bot_metadata
  (zombie.alert_enabled, zombie.alert_email) over config.ini so web
  UI changes take effect without a restart; uses isinstance(..., str)
  guard so mock/None values safely fall through to config.ini defaults
2026-04-14 10:01:51 -07:00
Stacy Olivas 22e1b2b53c fix: guard send_advert() with asyncio.wait_for(timeout=30) to prevent event loop lockup
When the radio firmware is unresponsive but not yet flagged as zombie,
send_advert() would hang indefinitely on the main event loop, blocking
all packet processing (no data in/out) for 60+ seconds until the
scheduler thread's future.result() timed out — but that only timed
out the waiting thread, not the coroutine itself.

Fix: wrap every send_advert() call that runs on the event loop with
asyncio.wait_for(timeout=30.0).  On timeout in the interval-advert
path, _radio_fail_count is incremented so repeated timeouts feed into
the existing zombie-detection threshold.
2026-04-14 10:01:51 -07:00
Stacy Olivas d0ae737066 feat: zombie radio detection — health probe, timeout guards, and alert system 2026-04-14 10:01:51 -07:00
agessaman 13e3e040b2 Refactor project root path handling in web viewer
- Updated `app.py` to ensure the project root is correctly added to `sys.path` when the script is executed directly, allowing for proper module imports.
- Removed redundant code that previously handled project root path insertion, streamlining the import process.
2026-04-12 15:53:37 -07:00
Adam Gessaman 5a4cde4818 Merge pull request #152 from cdwiegand/main
resolves issue of overriding command classes not honoring replies in channels

I had already implemented about 60% of this when your PR came in, so I adopted some of your pattern into my changes.
2026-04-11 19:01:23 -07:00
Adam Gessaman d9e32c86f2 Merge branch 'dev' into main 2026-04-11 19:00:06 -07:00