Commit Graph
93 Commits
Author SHA1 Message Date
agessaman ba23899c31 fix(correlation): tie channel messages to their own RF packet
{packet_hash} came back empty on a channel, and so did {path} and
{connection_info}. message.routing_info is only attached when the RF row is
correlated, and a channel message can never correlate: MeshCore's CHAN event
carries neither raw_hex nor a pubkey prefix, so every prefix strategy in
find_recent_rf_data is skipped and the message lands on the most-recent-packet
fallback, which #80 rightly refuses to take a route from.

The fallback is in fact almost always the right packet—the firmware emits the
RF log row and the decoded CHAN event for one reception back to back—but
"almost always" is a guess, and #80 is what guessing costs. The CHAN payload
does restate three things the RF row records independently: payload type,
path length and SNR. Requiring all three to agree, on a row no older than the
correlation window, turns the fallback into a checked hypothesis and earns a
real match kind rather than a fallback tag.

SNR carries the weight: it is one reception's measured value, quantised to
0.25 dB, so an unrelated packet agreeing on all three is improbable rather
than merely unlikely. Across the 55 channel messages in my logs the
immediately preceding RF row agreed on all three every time, always within a
second, while a window-wide search on the same fields was ambiguous for half
of them—so this verifies the ordering the firmware already gives us instead
of searching for a match.

Because these messages now correlate, they also satisfy the '*' gate by the
primary route, resolve their scope properly when flood_scopes is regional,
and contribute their path to the mesh graph, which they never did before.
2026-08-25 09:28:19 -07:00
agessaman 9627230b55 fix(scope): let flood_scopes '*' authorize replies on channels again
MeshCore's CHAN payload carries neither raw_hex nor a pubkey prefix, so a
channel message has no correlation key and find_recent_rf_data always lands
on the most-recent-packet fallback. rf_data_is_correlated() is therefore
never true for one, and _is_confirmed_global_flood could never pass: across
three log files every one of 45 received channel messages was rejected.

I raised this when the gate went in and accepted the counter that the
handler already has the general RF correlation and decoded packet info for
this message's own packet. That holds for DMs and is false for channel
messages.

'*' asks one question—was this a scoped regional flood?—and there is a
second way to answer it that the handler already computes and then discards.
A scoped message travels as TRANSPORT_FLOOD GRP_TXT, so if no such packet is
anywhere in the RF window, the message cannot have been scoped. That is a
window-wide fact, so unlike a route type read off a fallback row it does not
depend on having picked the right cached packet.

Correlated rows stay authoritative: a correlated TRANSPORT_FLOOD is still
blocked even when the scope window is empty, so an empty window cannot
launder a message the radio positively identified as scoped. Replaying the
real packets from the log, the 43 plain-FLOOD messages now get a reply and
the 2 that had a TC_FLOOD GRP_TXT alongside them still fail closed.
2026-08-25 09:15:52 -07:00
agessaman bf70594b41 fix: close the '*'-only flood_scopes bypass and schedule write race
A flood_scopes of "*" alone leaves scope_keys empty while setting
flood_scope_allow_global, and the loader already logs that as an active
allowlist. The handler gated on scope_keys alone, so that configuration
skipped authorisation entirely and admitted absent, uncorrelated and
TRANSPORT_FLOOD traffic. The gate now fires when either is set, so "*"
means global-only rather than everything.

The scheduled-message duplicate check ran outside the write lock inside
update_ini_values, so two concurrent creates for the same schedule could
both pass and the second silently replace the first instead of getting
the 409. Check and write are now one critical section, and delete is too.
Repaired the delete path's error handling while moving it, so an OSError
during the write is still a 500 rather than escaping.
2026-08-22 01:06:08 -07:00
agessaman 088097734b fix: require confirmed global FLOOD before '*' authorises a reply (#240 review)
Accepting the reviewer's rejection of my narrower version. I had argued
that requiring positive proof would silence legitimate global replies
whenever correlation is unavailable, but that objection does not hold:
the handler already has the general RF correlation and decoded packet
info for this message's own packet, so the normal global case can be
proven rather than assumed. Only genuinely uncorrelated traffic is
affected, and for an allowlist that should fail closed.

_is_confirmed_global_flood() now requires RF data correlated to this
message showing RouteType.FLOOD. Absent, uncorrelated, or
TRANSPORT_FLOOD data means the scope is unknown, and '*' no longer
admits it.

The channel-message tests never set flood_scope_keys, so it was a Mock
and read as truthy, meaning they were unintentionally exercising the
allowlist and only passed because `not allow_global` on a Mock is False.
Set explicitly to the unconfigured default they meant to test.
2026-08-22 00:51:15 -07:00
agessaman 33ed32a6c1 fix: address Codex round 3 findings
Path distance was still leaking between concurrent requests three ways,
all of which Codex reproduced:

- The formatter fell back to shared instance state whenever the request's
  own value was None, so a request with no measurable distance rendered
  another request's. It now consults instance state only when there is no
  request to read from.
- One extraction branch still called _decode_path without the request, so
  that route stored its distance on shared state alone.
- _get_sender_location read the shared _current_message, so an
  interleaved command could measure from the wrong sender. It takes the
  request's message, falling back to the shared value only for direct
  calls that pass nothing.

Scope authorisation: '*' permits unscoped global traffic, not traffic of
unknown scope. When a scope-eligible packet was heard but could not be
tied to this message, its scope is unknown and '*' no longer admits it.
Deliberately narrower than the reviewer suggested: when no scope-eligible
packet was heard at all, '*' still applies, because nothing scoped being
heard is what a genuinely global message looks like. Requiring positive
proof of FLOOD in that case would silence legitimate global replies
whenever correlation is unavailable, which is too much availability to
trade for the residual risk.
2026-08-22 00:42:08 -07:00
agessaman 4cf37f9614 fix: address Codex round 2 findings
Two fixes from the previous round were incomplete:

- The scheduled-message chunk budget used the schedule's explicit scope,
  but send_channel_message resolves an unset scope from
  flood_scope.<channel> and then outgoing_flood_scope_override. A
  schedule with an implicit regional scope was therefore sized for a
  global send and every chunk could overshoot once the sender added the
  regional header. The budget now resolves the effective scope, and
  assumes regional if that resolution fails, since guessing regional only
  ever makes chunks smaller.

- Resetting _last_path_distance_km per request fixed sequential reuse but
  not concurrency: the decode awaits a database lookup, and the
  dispatcher runs handlers as independent tasks, so two path commands can
  interleave and render each other's distance. The distance now rides on
  the request's own message, with the instance attribute kept only as a
  fallback for direct calls.

Two new findings:

- rf_data_is_correlated() treated pubkey and partial-prefix matches as
  packet-unique, but a sender prefix identifies a sender, not one
  transmission. With several cached packets from the same sender, the
  first (usually oldest) was returned and allowed to supply a route.
  Those strategies now take the newest match and are authoritative only
  when the match is unambiguous; otherwise the entry is marked fallback,
  so it still provides SNR/RSSI but never a route.

- The flood_scopes allowlist accepted a scope resolved from an
  uncorrelated fallback packet. The HMAC proves the cached packet is in
  an allowed scope, not that this message is, so a recent allowed-scope
  packet could admit an unrelated message. Scope authorisation now
  requires packet-bound correlation and logs plainly when it does not
  have it.

That last one is a deliberate fail-closed change to an authorisation
path that predates tonight. Channel messages normally carry raw_hex and
correlate exactly, so the fallback is the exception rather than the rule,
but a deployment using flood_scopes will now stay quiet in cases where it
previously replied on an assumed scope.
2026-08-22 00:32:05 -07:00
agessaman fc63039a48 fix: address Codex review of tonight's work
Correctness:

- {path_distance} was always blank in production. The resolution code that
  builds repeater_info dropped latitude/longitude in every branch, so the
  calculator could never find a coordinate. My tests passed hand-built
  dicts straight to the calculator and never exercised the builder, which
  is why they stayed green. Coordinates are now carried through all four
  construction sites, and the new tests drive _lookup_repeater_names via
  its lookup_func hook so the real builder runs.

- The #80 route guard was defeated two ways in the channel handler. When
  the RF data was an uncorrelated fallback, control fell through to the
  raw-hex and routing_info fallbacks below, which took the route from the
  unrelated packet anyway; the guard had actually made that path
  reachable. message.routing_info was also assigned unconditionally, and
  the path command reads it. "Not attributable" is now a terminal branch
  and the routing_info hand-off checks provenance.

- Same fix was incomplete for DMs: routing_info was captured and turned
  into path_info before the provenance check ran, so the later check only
  declined to overwrite an already-wrong value. Guarded at the source.

- Rendering could transmit for real. Capture only intercepts
  send_response, but advert calls send_advert() directly and
  send_response_chunked never checked capture_sink. Chunked sends are now
  captured, and rendering is opt-in via BaseCommand.render_safe (default
  False) instead of a denylist that cannot be complete. This also closes
  the DM-only leak: schedule is not marked safe, so {cmd:schedule} can no
  longer broadcast configuration to a channel.

- Multi-part rendered output was rejoined into one oversized send.
  Scheduled messages are now split to the RF body budget and sent through
  send_channel_messages_chunked, on character boundaries so multi-byte
  text is not corrupted.

- _last_path_distance_km is instance state that was only set on success,
  so an invalid path request could show the previous request's distance.
  Reset at the start of every execute().

- Stale-contact retries were only counted on non-OK results, so timeouts
  and exceptions left a contact eligible forever and could recreate the
  storm. All failed attempts count now.

Web viewer:

- A failed config reload was reported as a successful save. The API and
  UI now distinguish "saved and active" from "saved, restart needed".

- Preview count was unbounded; clamped to 1-20.

- update_ini_values is a read-modify-replace with no locking, so two
  concurrent viewer saves could lose one. Serialised behind a lock.
2026-08-22 00:16:03 -07:00
agessaman 4f6287788b fix(path): never attribute an uncorrelated packet's route (#80)
find_recent_rf_data has four strategies, and the fourth returns the most
recent packet in the cache when the first three fail to correlate. That
fallback exists for SNR/RSSI timing issues, but callers were also taking
its route. So when correlation missed, a message was recorded with some
other transmission's path and hop count — the reporter's four-hop message
stored as a single direct hop via 79, which is the last hop of an
unrelated packet. Rare, because it only fires when correlation fails.

Results are now tagged with how they were matched (exact, pubkey, partial
or fallback) and rf_data_is_correlated() gates the route. The tag rides on
a shallow copy so the cache entry is never marked, and it fails closed: an
untagged dict is treated as uncorrelated.

Guarded in both handlers. The DM path was worse than the channel one — it
falls back to find_recent_rf_data() with no correlation key at all, which
can only ever return a fallback, and then overwrote message.path and
message.routing_info from it. routing_info is what the path command reads,
so a wrong route reached the user directly.

The route is now left unresolved rather than fabricated, which also stops
a bogus edge being written to the mesh graph from path_nodes that belong
to a different packet. SNR and RSSI still use the fallback as before;
mis-correlated signal figures are approximate rather than structurally
wrong, and changing them is a separate call.

Existing tests asserted the returned dict was the cache entry itself.
Identity was incidental, so they now compare contents and additionally
assert the provenance tag.
2026-08-21 22:45:05 -07:00
agessaman a594b72a85 fix(neighbors): update zero-hop neighbor discovery and data handling
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
2026-08-12 18:01:28 -07:00
agessaman f4587392db fix(config): update auto_manage_contacts default behavior and documentation
Changed the default setting for `auto_manage_contacts` from `false` to `device` across configuration files and updated related documentation. This ensures that the device handles auto-addition of contacts while the bot manages capacity. Adjusted comments and documentation to reflect this change for clarity and consistency.
2026-07-29 19:49:50 -07:00
agessaman 055fa3291b refactor(message_handler): change logging level for event handling
- Updated logging statements in the `MessageHandler` class to use `debug` level instead of `info` for `RAW_DATA` and `NEW_CONTACT` events, reducing log verbosity while maintaining essential information.
- This change enhances the clarity of logs by reserving `info` level for more significant events.
- Resolves issue #214
2026-07-08 09:12:18 -07:00
agessaman 75f9fae374 fix(new-contact): distinguish known vs new contacts in NEW_CONTACT handling
NEW_CONTACT adverts were always logged as newly discovered, even for
nodes heard many times before. Check the tracking DB (get_tracked_contact_row)
and the live device contact list (is_contact_on_device) to classify the
advert, log "known" vs "new" accordingly, and skip the
new_contact_discovered audit entry for contacts that are already known.

Also dedup the is_currently_tracked device-contact lookups onto the shared
is_contact_on_device() helper.

Behavior preserved: repeaters remain database-only, device-mode companion
adds are still left to the firmware, and no per-advert device round-trip is
introduced.
2026-06-21 11:27:07 -07:00
agessaman cb54ca6a45 fix(config): clarify flood scope configuration and logging behavior
- Updated `config.ini.example` and `configuration.md` to enhance clarity on the `outgoing_flood_scope_override` and `flood_scopes` settings, detailing their behavior and interactions.
- Improved logging in `CommandManager` to provide better insights into scope resolution and potential issues during message sending.
- Added new methods in `MessageHandler` for improved RF data correlation, ensuring eligibility checks for flood scope matching.
- Enhanced unit tests to cover new behaviors and ensure robust handling of flood scope configurations.
2026-05-17 14:38:47 -07:00
agessaman 74b7de6f72 feat(flood_scope): enhance flood scope handling across services
- Introduced optional regional TC_FLOOD scope configuration for various services, allowing for more granular control over message routing.
- Updated the CommandManager to resolve and apply flood scopes from messages, configuration sections, and explicit parameters, ensuring correct message delivery.
- Enhanced service plugins to utilize the new flood scope functionality, including weather, earthquake, and webhook services.
- Added unit tests to verify the correct resolution and application of flood scopes in different scenarios, ensuring robust functionality.
2026-05-16 12:08:24 -07:00
agessaman c91baf1fdf refactor(message_handler, repeater_manager): enhance contact advertisement tracking
- Updated the `track_contact_advertisement` method to return a structured result indicating success and duplicate packet status, improving clarity in handling contact advertisements.
- Adjusted the `MessageHandler` to utilize the new result structure, allowing for better decision-making based on tracking outcomes.
- Enhanced unit tests to cover new behaviors and ensure correct handling of duplicate packets and advertisement tracking results.
2026-05-15 20:03:04 -07:00
agessaman 5879838d8b feat(command_manager): Fix flood scope on keyword and RandomLine channel replies
Route process_message keyword/RandomLine sends through send_response so
reply_scope reaches set_flood_scope; add optional command_id to
send_response for transmission tracking.
2026-05-10 19:20:22 -07:00
agessaman 6d496934d9 refactor(advert_parsing): improve flag handling in message and map uploader services
- Replaced the use of AdvertFlags enum for flag parsing with bitwise operations to handle invalid values correctly.
- Updated the parsing logic in both MessageHandler and MapUploaderService to use boolean flags for better clarity and performance.
- Adjusted error logging to use warnings instead of errors for parsing issues, enhancing the logging strategy.
- Added unit tests to ensure correct parsing behavior for edge cases with invalid flag values.
2026-05-04 15:45:38 -07:00
agessaman 2183f8e246 refactor(message_handler): standardize string quotes and improve code readability
- Updated string quotes from single to double quotes for consistency across the MessageHandler class.
- Enhanced formatting of method parameters for better readability.
- Minor adjustments to logging statements for clarity.
2026-05-01 21:32:54 -07:00
agessaman 263c73c2e2 feat(packet_capture): enhance MQTT packet publishing controls and path length decoding
- Added configuration options `mqtt_skip_unparseable_packets` and `advert_require_valid_signature` to control MQTT publishing behavior based on packet validity.
- Updated `decode_path_len_byte` function to return `None` for reserved size codes, improving path length validation.
- Implemented logic in `PacketCaptureService` to skip publishing unparseable packets and ADVERT packets with invalid signatures.
- Introduced `verify_meshcore_advert_ed25519` function for signature verification of ADVERT packets, with corresponding unit tests to ensure functionality.
- Enhanced documentation to reflect new configuration options and their effects on packet processing.
2026-05-01 21:32:40 -07:00
agessaman 2e7c56207a fix(message_handler): improve public key retrieval logic
- Updated the logic for retrieving the sender's public key in the MessageHandler class to ensure it defaults to sender_id when pubkey_prefix is not available.
- Enhanced the condition to check for sender_id presence before accessing contacts, improving robustness in public key resolution.
2026-04-25 19:59:16 -07:00
agessaman 23cdce03e8 feat(db_migrations, message_handler, repeater_manager): enhance purging log functionality
- Added a new migration to include a 'details' column in the 'purging_log' table for improved logging of actions.
- Refactored the `log_purging_action` method in `RepeaterManager` to handle both new and legacy schemas, ensuring compatibility with the updated database structure.
- Updated `MessageHandler` to utilize the new logging method, replacing direct database updates with a unified logging approach.
- Added tests to verify the presence of the new column and the correct functionality of the logging mechanism across different scenarios.
2026-04-21 17:04:51 -07:00
agessaman 0e5daadd04 feat(config, message_handler, repeater_manager, scheduler): implement device mode for auto-managing contacts
- Enhanced configuration options for `auto_manage_contacts` to support 'device' mode, allowing firmware to handle companion auto-addition and favourite hygiene.
- Updated `MessageHandler` to track new companions based on the `auto_manage_contacts` setting, with distinct behaviors for 'false', 'device', and 'bot' modes.
- Introduced scheduled jobs in `MessageScheduler` for device mode to manage firmware preferences and favourite keys with specified delays.
- Modified `RepeaterManager` to skip companion auto-purge in device mode, ensuring firmware manages contact slots effectively.
- Added tests to validate new behaviors and configurations, ensuring robust handling of contact management across different modes.
2026-04-20 21:26:17 -07:00
agessaman f061df391e revert: back out #80 find_recent_rf_data changes due to regression 2026-04-18 11:14:34 -07:00
agessaman 879a86bd80 fix(message_handler): keep #80 correlation strictly observation-level
Simplifies find_recent_rf_data so the sample it returns is always a single
wire observation — snr, rssi, raw_hex, and routing_info all belong to the
same decoded copy. The #80 fix stays intact via the two earlier invariants:
unmatched correlation_key returns None (no fallback to an unrelated recent
packet), and the no-key DM path is constrained to a narrow
rf_fallback_window.

An earlier iteration tried to enrich routing_info with the longest observed
same-packet_hash forward, but that would have left message_stats rows with
hops sourced from the matched observation while path was a different
forward's — an internal skew visible in the web viewer and !path output.
The enrichment is documented in TODO.md as a v0.9.1+ option-C follow-up
that must also sync message.hops for consistency.

Regression tests in tests/test_message_handler.py::TestFindRecentRfData
cover: return-None on unmatched key, narrow-fallback-window rejection of
stale samples, and the no-cross-wiring assertion for same-packet_hash
forwards.

Made-with: Cursor
2026-04-17 14:37:21 -07:00
agessaman bf6bc14de7 release(v0.9.0): readiness — silent-subscribe tests, py310, RF correlation fix
- Rewrite test_subscribe_packets/messages_emits_status_ack to match the
  silent subscription UX from 1ee84f2.
- Reconcile Python version: requires-python>=3.10, ruff target py310, CI
  matrix adds 3.13, pyupgrade UP0xx ignored pending a separate typing-rewrite
  PR; fix two B905 zip(strict=...) lints.
- Issue #80 fix in find_recent_rf_data: return None when correlation_key is
  provided but unmatched; prefer the longest observed path among samples
  sharing a packet_hash; narrow the no-key fallback to a configurable
  rf_fallback_window (default 2s).
- Issue #161: lower shipped max_response_hops default 10 -> 7.
- Add CHANGELOG.md, restructure BUGS.md around a ## v0.9.0 Fixed Bugs
  table, prune crossed-out duplicate outstanding rows, and add a
  Deferred-from-v0.9.0 triage section to TODO.md.
- Untrack coverage.json and add it to .gitignore.

Made-with: Cursor
2026-04-17 13:53:13 -07:00
agessaman 4677a238ac feat(message_handler): enhance message handling with type hints and new data structure
Added a new PendingMessageEntry TypedDict to improve the structure of pending messages. Updated type hints throughout the MessageHandler class for better clarity and consistency, including the initialization method and message processing functions. This refactor enhances code readability and maintainability without altering existing functionality. Additionally, included the new message_handler module in the pyproject.toml for better module organization.
2026-04-16 20:43:19 -07:00
agessaman 329905dd08 feat(core, scheduler): implement timeout handling for advert sending
Added asyncio timeout handling for sending startup and interval-based adverts, improving reliability by recording failures on timeout. Updated relevant tests to ensure coverage for timeout scenarios and increment failure counts appropriately. Introduced a new Radio Reliability panel in the web viewer for monitoring radio status.
2026-04-16 19:29:28 -07:00
Stacy Olivas a75ea16609 fix: post-rebase compatibility fixes for #138 (radio reliability)
Two small incompatibilities surfaced after rebasing dev-kg7qin-changes
onto upstream/dev in code introduced by #138:

- message_handler.py: replace two remaining Optional[str] annotations
  with str | None; the Optional import was removed during typing
  modernization but two annotations in this module were missed
- command_manager.py: restore # noqa: F401 guard on PUBLIC_CHANNEL_KEY_HEX
  re-export so ruff auto-fix does not silently remove it (imported by core.py)

Do not merge until #138 is merged.
2026-04-14 10:02:48 -07:00
Stacy Olivas 54aeb28bf0 security: SSRF hardening, log injection sanitization, and allow_local_smtp
Add SSRF host validation to maintenance.py send_nightly_email and
scheduler.py send_zombie_alert_email using validate_external_url().
New allow_local_smtp config key permits private-IP SMTP for local
relay setups.

Add sanitize_name() to security_utils and apply it to all log calls
in message_handler, repeater_manager, path_command, solarforecast_command,
command_manager, and discord_bridge_service to prevent log injection.

Move nightly email logic from duplicate scheduler._send_nightly_email()
into the canonical maintenance.py implementation, removing the duplicate.
Update tests to call maintenance.send_nightly_email() directly.

Add validate_external_url allow_private parameter with support for
loopback, RFC1918, CGN, and link-local address ranges.
2026-04-14 10:02:36 -07:00
agessaman 8bea10cdc3 Enhance mention handling in message processing
- Updated `config.ini.example` to introduce the `respond_to_mentions` setting, allowing configuration of how the bot responds to mentions in channel messages.
- Refactored `MessageHandler` to implement logic for handling mentions based on the new configuration, including stripping mentions when appropriate.
- Added `cleanup_message_for_matching` method in `BaseCommand` to streamline message processing and mention validation.
- Enhanced various command classes to utilize the new cleanup method for consistent mention handling.
- Introduced tests to validate the behavior of the new mention handling logic across different configurations.
2026-04-11 18:45:34 -07:00
agessaman 2ac1722014 Refactor flood scope configuration and enhance message handling
- Updated `config.ini.example` to clarify the usage of `outgoing_flood_scope_override` and `flood_scopes`, providing examples for better understanding.
- Modified `configuration.md` to reflect changes in flood scope handling, emphasizing the distinction between `outgoing_flood_scope_override` and `flood_scopes`.
- Refactored `CommandManager` to utilize the new `outgoing_flood_scope_override` for sending messages, ensuring consistent scope handling.
- Enhanced `MessageHandler` to prioritize library-provided scope fields for improved accuracy in flood scope matching.
- Added tests to validate the handling of flood scope fields from library payloads, ensuring robustness in message processing.
2026-04-10 16:29:52 -07:00
Adam Gessaman 4f820ffbd4 Enhance flood scope handling and channel fetching logic
- Updated `config.ini.example` to clarify flood scope configuration, introducing the auto-hashtag format for region names and adding support for multi-scope replies.
- Refactored `ChannelManager` to improve handling of empty channels, adjusting timeout logic and increasing request delay to prevent overwhelming devices.
- Enhanced `CommandManager` to load flood scope keys for HMAC matching and normalize scope names for consistency.
- Implemented scope matching in `MessageHandler` to ensure replies respect configured flood scopes, improving message routing accuracy.
- Updated `MeshMessage` model to include a `reply_scope` attribute for tracking matched flood scopes.
2026-04-08 21:14:04 -07:00
agessaman ba52c3ba07 Fix path length calculation and hash mode handling in MessageHandler
- Improved error handling for `out_path_hash_mode` and `out_path_len` to ensure proper type conversion and validation.
- Added logic to derive `out_path_len` from `out_path` when it is missing, enhancing robustness against incomplete data.
- Updated tests to verify correct behavior when `out_path_hash_mode` is provided as a string and `out_path_len` is absent.
2026-04-05 11:07:52 -07:00
agessaman b3d5054074 Merge branch 'integration/massive-pr' into dev
Resolve conflicts by combining v0.9 integration work with dev-only behavior:
- Keep channel_responses_enabled, greeter pause checks, and max_response_hops gating
- Retain TRACE/repeat handling, MQTT weather, temperature format helpers, and feed tooling
- Unify package-data globs, ruff/mypy/pytest config, Rate_Limits/Webhook in config example
- Web viewer: config panels + X-Requested-With on channel API; drop redundant DBManager import

Made-with: Cursor
2026-04-03 10:51:50 -07:00
agessaman 81866487ab Fix UnboundLocalError in decode_meshcore_packet for invalid hex input
- Ensured `byte_data` and `hex_data` are always defined for error logging, preventing UnboundLocalError when handling invalid hex strings.
- Updated test case to confirm that invalid hex input returns None without raising exceptions, improving error handling in the message decoding process.

These changes enhance the robustness of the message handling system by addressing potential error scenarios.
2026-03-30 19:49:39 -07:00
agessaman f3e667e64f Enhance TRACE payload handling and path extraction in message processing
- Updated `update_mesh_graph_from_trace_data` to clarify the format of `path_hashes` as per-hop hash strings from the trace payload.
- Modified `MessageHandler` to differentiate between TRACE packets and regular transmissions, preventing incorrect extraction of repeater prefixes from RF path bytes.
- Introduced `parse_trace_payload_route_hashes` utility to extract TRACE route hash segments from payloads, ensuring accurate handling of path data.
- Enhanced `PacketCaptureService` to correctly populate packet information for TRACE packets, including SNR path and route hashes.
- Expanded test coverage for TRACE payload decoding and path extraction to validate functionality and correctness.

These changes improve the accuracy and reliability of TRACE data processing in the application.
2026-03-30 17:36:52 -07:00
Chris Barker 8966be89a6 Adds max_response_hops
Before this there was no way to prevent the bot to reply to random
channel noise from a temprorary strong link to another distant mesh.

This provides the ability for users in larger meshes to cap the bot
to reply to only messages sent likely from nearby repeaters. This works
well in conjunction with region scoping.

Default value is still 64, to not change behavior on existing installs,
but the example config.ini's all include a suggested start valu of 10.
2026-03-30 15:11:05 +01:00
agessaman ea0e25d746 Implement LRU caching for SNR and RSSI in MessageHandler and enhance rate limiting with thread safety
- Updated MessageHandler to use OrderedDict for SNR and RSSI caches, implementing LRU eviction to maintain a maximum size.
- Added thread safety to rate limiting classes by introducing locks, ensuring consistent behavior in concurrent environments.
- Introduced periodic cleanup in TransmissionTracker to manage memory usage effectively.
- Added unit tests for LRU cache behavior and automatic cleanup functionality.

These changes improve performance and reliability in handling message data and rate limiting.
2026-03-29 14:11:12 -07:00
agessaman 6a8ad5260a Implement URL shortening functionality in feed manager and related components
- Added configuration options for URL shortening in `config.ini.example` and updated documentation in `FEEDS.md`.
- Enhanced `FeedManager` to support URL shortening based on new settings, allowing for both global and per-link shortening.
- Refactored message formatting logic to incorporate URL shortening features, ensuring compatibility with existing link handling.
- Introduced new utility functions for encoding path length bytes in `utils.py`, improving path management in message handling.
- Added unit tests to validate the new URL shortening functionality and ensure proper behavior under various conditions.
2026-03-21 20:45:42 -07:00
agessaman 7a3bb56d70 Update admin commands to include channelpause functionality
- Added `channelpause` and `channelresume` commands to the admin commands list in configuration files, allowing admins to pause or resume bot responses on public channels via DM.
- Updated documentation to reflect the new command functionality and its implications for channel responses.
- Modified validation and message handling to incorporate the new channel response control feature.
2026-03-21 20:43:58 -07:00
agessaman 21718ba438 Implement URL shortening functionality in feed manager and related components
- Added configuration options for URL shortening in `config.ini.example` and updated documentation in `FEEDS.md`.
- Enhanced `FeedManager` to support URL shortening based on new settings, allowing for both global and per-link shortening.
- Refactored message formatting logic to incorporate URL shortening features, ensuring compatibility with existing link handling.
- Introduced new utility functions for encoding path length bytes in `utils.py`, improving path management in message handling.
- Added unit tests to validate the new URL shortening functionality and ensure proper behavior under various conditions.
2026-03-21 18:24:59 -07:00
agessaman cc2228177f Enhance command execution logic in CommandManager to enforce channel restrictions
- Updated the command execution flow to skip commands that are not allowed in the current channel, improving control over command usage.
- Ensured that the command handling mirrors the existing checks in the system, maintaining consistency in command execution rules.
2026-03-20 20:04:52 -07:00
agessaman 469d818e83 Refactor contact addition logic in MessageHandler for improved compatibility
- Enhanced the `add_contact` method to handle variations in parameter requirements across device bindings, ensuring robust functionality.
- Avoided passing unnecessary `contact_data` fields that could lead to issues with unsigned integer handling.
- Implemented a fallback mechanism for `add_contact` to accommodate signature mismatches, improving error handling.
2026-03-19 20:24:06 -07:00
agessaman e5d0804069 Refactor contact addition in MessageHandler to streamline parameters
- Updated the `add_contact` method call to only pass necessary fields (name and optional public_key), improving compatibility with device APIs.
- Removed unnecessary RSSI/SNR values from `contact_data` to prevent issues with unsigned integer handling in some devices.
2026-03-19 20:12:44 -07:00
agessaman 32fd416df8 Update admin commands to include channelpause functionality
- Added `channelpause` and `channelresume` commands to the admin commands list in configuration files, allowing admins to pause or resume bot responses on public channels via DM.
- Updated documentation to reflect the new command functionality and its implications for channel responses.
- Modified validation and message handling to incorporate the new channel response control feature.
2026-03-19 19:53:27 -07:00
agessaman 0c2adaa44e Refactor message handling to remove "BUG-026" chunking. This was not undesired behavior but a mesh-friendly design choice.
- Removed chunking logic for keyword-dispatched help/command responses in `message_handler.py`, allowing long responses to be sent as single messages.
- Updated the response sending mechanism to directly use the full response text for both direct messages and channel messages, ensuring clarity and consistency in message delivery.
2026-03-19 10:55:58 -07:00
Stacy Olivas ad77d7b00d fix: BUG-025/026/027/028/029 implementations and ruff/mypy refinements
BUG-025: send_channel_message retry logic on no_event_received
BUG-026: split_text_into_chunks and chunked dispatch in message_handler
BUG-027: test_weekly_on_wrong_day_does_not_run patch uses fake_now
BUG-028: byte_data = b"" initialised before try in decode_meshcore_packet
BUG-029: app.py db_path via self._config_base; realtime.html socket race
  fixed; base.html forceNew removed; ping_timeout 5 to 20s

Additional: ruff and mypy refinements across all modules; discord bridge,
telegram bridge, rate limiter, and service plugin updates
2026-03-17 18:07:19 -07:00
Stacy Olivas ce884cee87 fix: auth, db migrations, retry, chunking, socket race, trace, timezone, repeater, and ruff/mypy cleanup
BUG-001: web viewer login/session auth (in web viewer commit)
BUG-002: db_manager ALTER TABLE for missing channel_operations and
  feed_message_queue columns on startup
BUG-015: scheduler thread blocked on future.result(); replaced all
  blocking waits with add_done_callback (fire-and-forget)
BUG-016: reboot_radio sends meshcore.commands.reboot() before disconnect
BUG-017: radio disconnect uses asyncio.wait_for(timeout=10)
BUG-022: custom asyncio loop exception handler suppresses IndexError
  from meshcore parser at DEBUG level
BUG-024: last_db_backup_run updated after each run; 2-min startup
  window; last-run seeded from DB on restart
BUG-025: send_channel_message retries up to 2 times (2s delay) on
  no_event_received via _is_no_event_received() helper
BUG-026: split_text_into_chunks() and get_max_message_length() added
  to CommandManager; keyword dispatch uses send_response_chunked()
BUG-028: byte_data = b"" initialised before try block in
  decode_meshcore_packet to prevent UnboundLocalError in except handler
TraceCommand: path nodes reversed and return path truncated; fixed
format_elapsed_display: UTC normalisation before elapsed computation (#75)
RepeaterManager: auto_manage_contacts guard before any purge logic (#50)
Command aliases: [Aliases] config section injects shorthands at startup
JSON logging: _JsonFormatter; json_logging = true in [Logging]
Structured JSON logging compatible with Loki, Elasticsearch, Splunk
Discord bridge, Telegram bridge, and all service plugins updated
MeshGraph edge promotion logic corrected
Shutdown: scheduler and meshcore disconnect joined cleanly; log spam fixed
All modules: ruff and mypy cleanup applied (type annotations, imports)
2026-03-17 18:07:18 -07:00
agessaman 7d76ed443b Update dependencies and enhance path handling for multi-byte support
- Updated `meshcore` dependency version to `2.2.14` in both `pyproject.toml` and `requirements.txt`.
- Added multi-byte path support in the `PathCommand`, allowing for 1-, 2-, and 3-byte-per-hop paths.
- Enhanced `MessageHandler` to utilize `routing_info` for accurate path extraction and validation.
- Improved path extraction methods in `MultitestCommand` and `TestCommand` to prefer `routing_info` for node IDs.
- Refactored path handling logic across various commands to ensure consistent multi-byte path processing.
2026-03-05 13:37:38 -08:00
agessaman 4982ba482d Enhance mesh graph handling for multi-resolution prefix support
- Updated `MeshGraph` to support multi-resolution storage of edges, allowing prefixes of 2, 4, or 6 hex chars without truncation.
- Implemented prefix matching logic to ensure distinct links are maintained and accurately retrieved based on prefix queries.
- Refactored methods in `MessageHandler` and `PathCommand` to accommodate variable prefix lengths during graph lookups.
- Enhanced tests to validate prefix match functionality and edge management in the mesh graph.
2026-03-01 20:42:06 -08:00