Accepting the reviewer's rejection of my narrower version. I had argued
that requiring positive proof would silence legitimate global replies
whenever correlation is unavailable, but that objection does not hold:
the handler already has the general RF correlation and decoded packet
info for this message's own packet, so the normal global case can be
proven rather than assumed. Only genuinely uncorrelated traffic is
affected, and for an allowlist that should fail closed.
_is_confirmed_global_flood() now requires RF data correlated to this
message showing RouteType.FLOOD. Absent, uncorrelated, or
TRANSPORT_FLOOD data means the scope is unknown, and '*' no longer
admits it.
The channel-message tests never set flood_scope_keys, so it was a Mock
and read as truthy, meaning they were unintentionally exercising the
allowlist and only passed because `not allow_global` on a Mock is False.
Set explicitly to the unconfigured default they meant to test.
Two fixes from the previous round were incomplete:
- The scheduled-message chunk budget used the schedule's explicit scope,
but send_channel_message resolves an unset scope from
flood_scope.<channel> and then outgoing_flood_scope_override. A
schedule with an implicit regional scope was therefore sized for a
global send and every chunk could overshoot once the sender added the
regional header. The budget now resolves the effective scope, and
assumes regional if that resolution fails, since guessing regional only
ever makes chunks smaller.
- Resetting _last_path_distance_km per request fixed sequential reuse but
not concurrency: the decode awaits a database lookup, and the
dispatcher runs handlers as independent tasks, so two path commands can
interleave and render each other's distance. The distance now rides on
the request's own message, with the instance attribute kept only as a
fallback for direct calls.
Two new findings:
- rf_data_is_correlated() treated pubkey and partial-prefix matches as
packet-unique, but a sender prefix identifies a sender, not one
transmission. With several cached packets from the same sender, the
first (usually oldest) was returned and allowed to supply a route.
Those strategies now take the newest match and are authoritative only
when the match is unambiguous; otherwise the entry is marked fallback,
so it still provides SNR/RSSI but never a route.
- The flood_scopes allowlist accepted a scope resolved from an
uncorrelated fallback packet. The HMAC proves the cached packet is in
an allowed scope, not that this message is, so a recent allowed-scope
packet could admit an unrelated message. Scope authorisation now
requires packet-bound correlation and logs plainly when it does not
have it.
That last one is a deliberate fail-closed change to an authorisation
path that predates tonight. Channel messages normally carry raw_hex and
correlate exactly, so the fallback is the exception rather than the rule,
but a deployment using flood_scopes will now stay quiet in cases where it
previously replied on an assumed scope.
find_recent_rf_data has four strategies, and the fourth returns the most
recent packet in the cache when the first three fail to correlate. That
fallback exists for SNR/RSSI timing issues, but callers were also taking
its route. So when correlation missed, a message was recorded with some
other transmission's path and hop count — the reporter's four-hop message
stored as a single direct hop via 79, which is the last hop of an
unrelated packet. Rare, because it only fires when correlation fails.
Results are now tagged with how they were matched (exact, pubkey, partial
or fallback) and rf_data_is_correlated() gates the route. The tag rides on
a shallow copy so the cache entry is never marked, and it fails closed: an
untagged dict is treated as uncorrelated.
Guarded in both handlers. The DM path was worse than the channel one — it
falls back to find_recent_rf_data() with no correlation key at all, which
can only ever return a fallback, and then overwrote message.path and
message.routing_info from it. routing_info is what the path command reads,
so a wrong route reached the user directly.
The route is now left unresolved rather than fabricated, which also stops
a bogus edge being written to the mesh graph from path_nodes that belong
to a different packet. SNR and RSSI still use the fallback as before;
mis-correlated signal figures are approximate rather than structurally
wrong, and changing them is a separate call.
Existing tests asserted the returned dict was the cache entry itself.
Identity was incidental, so they now compare contents and additionally
assert the provenance tag.
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
NEW_CONTACT adverts were always logged as newly discovered, even for
nodes heard many times before. Check the tracking DB (get_tracked_contact_row)
and the live device contact list (is_contact_on_device) to classify the
advert, log "known" vs "new" accordingly, and skip the
new_contact_discovered audit entry for contacts that are already known.
Also dedup the is_currently_tracked device-contact lookups onto the shared
is_contact_on_device() helper.
Behavior preserved: repeaters remain database-only, device-mode companion
adds are still left to the firmware, and no per-advert device round-trip is
introduced.
Updated the CommandManager to prioritize sender_pubkey over sender_id when sending direct messages. This change ensures correct routing of messages, especially in cases where multiple nodes share similar display names. Added tests to verify the new behavior and ensure proper functionality.
- Updated the `track_contact_advertisement` method to return a structured result indicating success and duplicate packet status, improving clarity in handling contact advertisements.
- Adjusted the `MessageHandler` to utilize the new result structure, allowing for better decision-making based on tracking outcomes.
- Enhanced unit tests to cover new behaviors and ensure correct handling of duplicate packets and advertisement tracking results.
Route process_message keyword/RandomLine sends through send_response so
reply_scope reaches set_flood_scope; add optional command_id to
send_response for transmission tracking.
- Replaced the use of AdvertFlags enum for flag parsing with bitwise operations to handle invalid values correctly.
- Updated the parsing logic in both MessageHandler and MapUploaderService to use boolean flags for better clarity and performance.
- Adjusted error logging to use warnings instead of errors for parsing issues, enhancing the logging strategy.
- Added unit tests to ensure correct parsing behavior for edge cases with invalid flag values.
- Added a new migration to include a 'details' column in the 'purging_log' table for improved logging of actions.
- Refactored the `log_purging_action` method in `RepeaterManager` to handle both new and legacy schemas, ensuring compatibility with the updated database structure.
- Updated `MessageHandler` to utilize the new logging method, replacing direct database updates with a unified logging approach.
- Added tests to verify the presence of the new column and the correct functionality of the logging mechanism across different scenarios.
- Updated `send_dm` method to first attempt contact resolution by name, and if that fails, fallback to resolving by public key prefix.
- Improved logging to indicate the method of recipient resolution used.
- Added tests to verify the new recipient resolution logic, ensuring correct behavior when looking up contacts by public key prefix and handling failures appropriately.
- Enhanced configuration options for `auto_manage_contacts` to support 'device' mode, allowing firmware to handle companion auto-addition and favourite hygiene.
- Updated `MessageHandler` to track new companions based on the `auto_manage_contacts` setting, with distinct behaviors for 'false', 'device', and 'bot' modes.
- Introduced scheduled jobs in `MessageScheduler` for device mode to manage firmware preferences and favourite keys with specified delays.
- Modified `RepeaterManager` to skip companion auto-purge in device mode, ensuring firmware manages contact slots effectively.
- Added tests to validate new behaviors and configurations, ensuring robust handling of contact management across different modes.
Simplifies find_recent_rf_data so the sample it returns is always a single
wire observation — snr, rssi, raw_hex, and routing_info all belong to the
same decoded copy. The #80 fix stays intact via the two earlier invariants:
unmatched correlation_key returns None (no fallback to an unrelated recent
packet), and the no-key DM path is constrained to a narrow
rf_fallback_window.
An earlier iteration tried to enrich routing_info with the longest observed
same-packet_hash forward, but that would have left message_stats rows with
hops sourced from the matched observation while path was a different
forward's — an internal skew visible in the web viewer and !path output.
The enrichment is documented in TODO.md as a v0.9.1+ option-C follow-up
that must also sync message.hops for consistency.
Regression tests in tests/test_message_handler.py::TestFindRecentRfData
cover: return-None on unmatched key, narrow-fallback-window rejection of
stale samples, and the no-cross-wiring assertion for same-packet_hash
forwards.
Made-with: Cursor
- Rewrite test_subscribe_packets/messages_emits_status_ack to match the
silent subscription UX from 1ee84f2.
- Reconcile Python version: requires-python>=3.10, ruff target py310, CI
matrix adds 3.13, pyupgrade UP0xx ignored pending a separate typing-rewrite
PR; fix two B905 zip(strict=...) lints.
- Issue #80 fix in find_recent_rf_data: return None when correlation_key is
provided but unmatched; prefer the longest observed path among samples
sharing a packet_hash; narrow the no-key fallback to a configurable
rf_fallback_window (default 2s).
- Issue #161: lower shipped max_response_hops default 10 -> 7.
- Add CHANGELOG.md, restructure BUGS.md around a ## v0.9.0 Fixed Bugs
table, prune crossed-out duplicate outstanding rows, and add a
Deferred-from-v0.9.0 triage section to TODO.md.
- Untrack coverage.json and add it to .gitignore.
Made-with: Cursor
- Updated `config.ini.example` to introduce the `respond_to_mentions` setting, allowing configuration of how the bot responds to mentions in channel messages.
- Refactored `MessageHandler` to implement logic for handling mentions based on the new configuration, including stripping mentions when appropriate.
- Added `cleanup_message_for_matching` method in `BaseCommand` to streamline message processing and mention validation.
- Enhanced various command classes to utilize the new cleanup method for consistent mention handling.
- Introduced tests to validate the behavior of the new mention handling logic across different configurations.
- Updated `config.ini.example` to clarify the usage of `outgoing_flood_scope_override` and `flood_scopes`, providing examples for better understanding.
- Modified `configuration.md` to reflect changes in flood scope handling, emphasizing the distinction between `outgoing_flood_scope_override` and `flood_scopes`.
- Refactored `CommandManager` to utilize the new `outgoing_flood_scope_override` for sending messages, ensuring consistent scope handling.
- Enhanced `MessageHandler` to prioritize library-provided scope fields for improved accuracy in flood scope matching.
- Added tests to validate the handling of flood scope fields from library payloads, ensuring robustness in message processing.
- Ensured `byte_data` and `hex_data` are always defined for error logging, preventing UnboundLocalError when handling invalid hex strings.
- Updated test case to confirm that invalid hex input returns None without raising exceptions, improving error handling in the message decoding process.
These changes enhance the robustness of the message handling system by addressing potential error scenarios.
- Updated `update_mesh_graph_from_trace_data` to clarify the format of `path_hashes` as per-hop hash strings from the trace payload.
- Modified `MessageHandler` to differentiate between TRACE packets and regular transmissions, preventing incorrect extraction of repeater prefixes from RF path bytes.
- Introduced `parse_trace_payload_route_hashes` utility to extract TRACE route hash segments from payloads, ensuring accurate handling of path data.
- Enhanced `PacketCaptureService` to correctly populate packet information for TRACE packets, including SNR path and route hashes.
- Expanded test coverage for TRACE payload decoding and path extraction to validate functionality and correctness.
These changes improve the accuracy and reliability of TRACE data processing in the application.
- Updated MessageHandler to use OrderedDict for SNR and RSSI caches, implementing LRU eviction to maintain a maximum size.
- Added thread safety to rate limiting classes by introducing locks, ensuring consistent behavior in concurrent environments.
- Introduced periodic cleanup in TransmissionTracker to manage memory usage effectively.
- Added unit tests for LRU cache behavior and automatic cleanup functionality.
These changes improve performance and reliability in handling message data and rate limiting.
Test modules:
- test_enums: enum values and flag combinations
- test_models: MeshMessage dataclass field and type validation
- test_transmission_tracker: full TransmissionTracker coverage
- test_message_handler: path parsing, RF correlation, message routing
- test_repeater_manager: role detection, ACL, device type classification
- test_core: config loading, radio settings, reload paths
Tracking files:
- BUGS.md: known bugs and fix history log
- TESTING.md: test strategy, coverage targets, and how-to guide
- TODO.md: feature and task backlog with completion status
- scripts/update_todos.py: scans source for # TODO/FIXME/HACK markers
and regenerates the Inline TODOs section in TODO.md
agessaman