A codex review found that moving dashboard code out of app.py dropped `modules.web_viewer.app.DashboardStatsService`. The public-API golden only recorded names a module defines, so I added a second one, `public_imports.json`, recording the public names each module imports from another project module, generated from dev's tree as the baseline. It found 20 more names earlier refactor commits had stopped importing (the Nominatim and MQTT-weather helpers in wx, gwx, aqi and solarforecast; `PayloadVersion`, the region-warning verdicts and `decode_path_len_byte` in message_handler; two feed helpers and two feed limits in the web viewer). All are imported again as re-exports, so `modules.<module>.<name>` and patches of it resolve as they did on dev.
Seven places in message_handler and graph_trace_helper ran the same pair of queries, byte for byte: count the repeaters and room servers whose key starts with the prefix and were heard within the recency window, and if exactly one was, fetch its key (starred first, then most recent). contacts_repo.unique_recent_repeater_key now runs it and returns (count, key); each caller keeps its own logging, including the two that report the match count. Same queries, same order, same handling of execute_query's empty result on a database error.
#315 changed populate_missing_geolocation_data, which S1 had deleted as dead. It is public, and local plugins may call public manager and command methods, so it comes back in dev's form, along with 11 other public methods S1 removed: CommandManager.get_available_commands_list, GreeterCommand.get_recent_greeted_users, WxCommand.abbreviate_alert_title and get_current_conditions, MessageHandler.discover_message_path, and RepeaterManager.add_discovered_contact, get_daily_advertisement_stats, get_nodes_per_day_stats, get_purging_stats, restore_repeater, test_meshcore_cli_commands and toggle_auto_add. All are verbatim from dev; every self. attribute they use still exists. A scan of every public name in modules/ on dev against this branch now finds nothing missing.
A codex review of the branch found seven places where earlier refactor commits changed behavior. Each is restored here with a test:
- base_service imported flood_scope relatively; the local-service loader also runs that file as local_services.base_service, where the import cannot resolve. Now absolute.
- Radio-health fields were initialized after commands and services were built, so a plugin constructor reading is_radio_offline raised. They are now set before any plugin.
- Removing the per-command can_execute overrides made 31 commands honor skip_channel_check, which their overrides had dropped. BaseCommand.honors_skip_channel_check (False on exactly those commands) keeps them enforcing their channel list.
- CommandManager.get_plugin_metadata, WxCommand.get_weather_for_zipcode and PluginLoader.validate_plugin were public compatibility entry points; they are back.
- feed_manager stopped exporting DEFAULT_MAX_FEED_RESPONSE_BYTES and DEFAULT_MAX_PARSED_FEED_ITEMS despite the commit saying it re-exported them; it does now.
- wx and gwx logged companion-location lookup failures at WARNING; the shared helper logged at debug. It now takes the level.
- The INFO 'MESSAGE ROUTING' / 'DIRECT MESSAGE' lines for pubkey-correlated DMs were removed with the dead path_info computation; they are logged again by a log-only helper.
decode_meshcore_packet, the path processor, the path chunkers, format_path_string and the route/payload type-name tables are now plain functions in packet_decode.py. MessageHandler keeps every old method as a delegate that supplies the configured prefix width and its logger, so callers and tests that patch decode_meshcore_packet on a handler still work. The existing quirk is preserved: path_info uses the configured width while path/path_hex follow the packet's own bytes-per-hop.
Parity: decoding a sample of 4,009 real packets from the dev database (plus malformed inputs) at prefix widths 2 and 4, with the path and format helpers, produces byte-identical output before and after. A handler built with no bot attribute now decodes path_info at width 2 instead of falling into the error branch; only bare test objects can hit that.
The eight scope helpers (transport-code HMAC match, decode field extraction, reply-scope resolution, route-type and eligibility checks, the confirmed-global test and the scoped/global/unknown classifier) used no handler state beyond the logger. They are now plain functions in scope_gate.py; MessageHandler keeps every old private name as a staticmethod alias or a one-line delegate passing its logger, so the tests that call them on handler instances, including object.__new__ ones, are untouched. The RF-correlation markers and rf_data_is_correlated move to rf_match.py, which both modules import; message_handler re-exports them.
Bodies moved verbatim apart from dropping self. scope_gate is a new module, so mypy checks it.
Four if/elif ladders (SNR and RSSI, for DMs and channel messages) checked a list of payload keys, then a list of metadata keys, converting the first present value. _signal_value takes the key lists and converter, so each site keeps its own keys: DMs still accept signal_to_noise and signal_noise_ratio, channel messages do not. A present-but-None key still ends the search, metadata is still read only when no payload key is present, and the cache and RF fallbacks after each call are untouched. A test checks it against the original ladder for every combination of absent, None and valued keys.
Five loops scanned meshcore.contacts for the first entry matching a public-key prefix or adv_name. _find_contact(predicate) does the scan; each call site keeps its own predicate and guard. In handle_contact_message the display-name and full-key lookups used the same predicate, so they now share one scan; the display-name lookup still runs with an empty prefix (matching the first contact, as before) while the full-key lookup still requires a non-empty one.
Five copies split a hex path into node IDs of N characters and fell back to one-byte nodes when the path did not divide evenly. They now call split_path_hex; the one caller that wanted uppercase nodes upper-cases the result. The observed-path parser that drops an incomplete trailing node instead of falling back keeps its own loop.
Six places in message_handler and command_manager spelled out hasattr(bot, 'web_viewer_integration') and ... and ....bot_integration, then reached through the same chain again to call it. A _viewer_bridge() method on each class returns the bridge or None, and the call sites bind it once with :=.
handle_contact_message built path_info from the contact table, the payload's path_len and a correlated RF packet's routing_info, then unconditionally replaced it (and path_len) before constructing the MeshMessage; the RF lookup it did was repeated after the message was built. The only effect of the removed block was a pair of INFO log lines reporting a route the DM never carried. The helpers it called (find_recent_rf_data, decode_meshcore_packet) mutate nothing.
handle_new_contact's 'unknown contact type' fallback could not run: the branch above returns on both of its paths whenever the bot has a repeater_manager, and every statement in the fallback was guarded by that same hasattr. RepeaterManager.add_discovered_contact was called only from there.
Every name removed here has zero references across modules, tests, docs, local plugins, scripts and generate_website.py, and none is part of the documented plugin surface (db_manager, command_manager send methods, channel_manager stay untouched). About 1,160 lines:
- multitest: four formatter helpers that only referenced each other (the flat and nested variants are the ones in use)
- path: _get_api_cache_data, _filter_recent_repeaters
- repeater command: _handle_contact_stats, _get_geocoding_status
- airplanes: _send_split_response; greeter: get_recent_greeted_users; plugin_loader: validate_plugin
- wx: get_weather_for_zipcode, abbreviate_alert_title, get_current_conditions
- message_handler: discover_message_path, and an ADVERT debug line behind a self.debug attribute that is never set
- utils: _get_node_location_and_key_from_db
- repeater_manager: test_meshcore_cli_commands, toggle_auto_add, restore_repeater, get_purging_stats, populate_missing_geolocation_data, get_daily_advertisement_stats, get_nodes_per_day_stats
- command_manager: get_available_commands_list and get_plugin_metadata (HelpCommand and PluginLoader have the live versions)
- base_command: _load_command_prefix, and a no-op .replace('_', '_')
- scheduler: the first of two _run_data_retention definitions, shadowed by the second since it was added
- core: the write-only last_clock_sync_time, and _setup_routing_capture, which only logged a line
Full suite, ruff and mypy pass, and the characterization golden files are unchanged.
_channel_secrets() assigned dict_items and enumerate to the same
variable, and mypy inferred Any | None for the channel_idx .get() with
a mixed Any | int default. Declare items as Iterable[tuple[int, Any]]
and hold the raw index as Any. Runtime behavior is unchanged.
Also cover the list layout meshcore actually uses, where entries
without channel_idx fall back to their position.
**A `global` verdict now requires RF correlated to the message.** It was also
reachable through `_is_confirmed_global_flood`'s argument-from-absence route
("no scope-eligible packet in the window, therefore unscoped"), so a row that
literally said TC_FLOOD, or a scoped ADVERT that the GRP_TXT filter excluded,
came back as "no region code". That inference is fine for deciding whether a
`*` in flood_scopes authorizes a reply; it is not fine for accusing someone of
a misconfiguration. Channel messages still correlate through the payload match
(#255), so the ordinary case is unaffected.
**A `%` in the warning message no longer wedges config reload.** `_get` read
without `raw=True`, so configparser's interpolation raised, the error was
swallowed, and the bot transmitted the default wording instead. Worse, the
bot's own `_validate_config_snapshot` iterates `config.items(section)` and
would reject every hot reload until the file was hand-edited. The save endpoint
now rejects `%` outright and caps the message at 500 characters, and the read
is raw so a hand-edited value is still honored.
**One unreachable sender no longer eats the daily cap forever.** The cap counted
failed attempts but the per-sender cooldown did not, so a node the radio cannot
reach was retried every `min_unscoped_messages` messages indefinitely and no
real offender was ever warned. Both count attempts now. The regression test
fails with three sends against the old filter.
**The sender is a display name, not an identity.** MeshCore's CHANNEL_MSG_RECV
carries no public key, so `sender_pubkey` was always empty on this path and the
only identity is a prefix anyone with the channel key can forge. DM delivery
now requires a contact the radio already holds, which bounds the bot to nodes
it knows and stops failed sends spending cap slots. Two tests asserted the
opposite because the fixture supplied a pubkey the radio never sends; they now
run with what the call site actually passes, and the docs no longer claim
pubkey identity.
Also: a negative `max_warnings_per_day` fell back instead of clamping to the
"unlimited" sentinel; tallies group case-insensitively so a `#` or case change
does not split a channel; retention uses the same clock the rows are written
in; the status figure shows delivered with attempts beside it, rather than a
count of four next to "last warning: none yet"; and the channel bars are scaled
over classified traffic so their width equals the percentage printed beside
them (33.7% was drawn at 28.7%).
A channel message with no "Name: " prefix falls back to a stand-in name.
That is not a node: every such message shares the identity, so they would
accumulate a run together and then get a DM addressed to a contact that
does not exist. The hook now passes no sender for those, so they are still
counted but can never earn a warning. The literal is a named constant and a
test pins the hook's guard to it.
Also folds the channel body-budget formula into models.channel_body_limit
rather than letting the web viewer keep a fourth copy of
"max(130, 160 - len(name) - 2)"; BaseCommand and CommandManager now call it
too, and 158 becomes models.DM_BODY_LIMIT.
Closes#279.
A MeshCore client with no region configured sends every channel message as
a plain FLOOD, which every repeater on the mesh rebroadcasts. This adds two
things: free observation of how much of that the bot hears, and an opt-in
warning to the senders.
Observation classifies each channel message as scoped, global or unknown and
tallies it per channel per local day. It costs one upsert and no airtime, and
it is what lets an operator see the size of the problem before deciding to
spend airtime on it. The classification runs ahead of the flood_scopes
allowlist, because an unscoped message is exactly what that allowlist drops.
Warnings only ever fire on positive RF evidence of an unscoped FLOOD. Absence
of correlation is not proof that a sender omitted a region, so it classifies
as unknown and stays quiet. They are off by default, start in dry run, and are
fenced by min_unscoped_messages, a per-sender cooldown, a mesh-wide cooldown
and a daily cap on attempts. Dry run consumes the same budget it previews, so
the log is what going live would put on the air, not an upper bound. All three
limits read from the database, so a restart cannot release a burst.
Channel-delivered warnings go out at global scope on purpose: the recipient is
by definition outside any region the bot replies under.
New Settings -> Region Warnings page in the web viewer covers all of it. Its
dark-mode striped rows exposed a pre-existing base.html bug where Bootstrap's
light-theme text color survived on a dark row background (~1.3:1); fixed there
for every table in the app.
Verifying a channel message against the RF cache only ever checked the row
strategy 4 returned — the most recent packet heard. That assumes the RF log row
and the decoded CHAN event for one reception arrive back to back with nothing in
between. On a dense mesh they do not: a repeater's echo of the very same packet
is routinely logged in the gap.
The reporter's message was heard directly (SNR 13.25, 0 hops) and again via
repeater f0 185 ms later (SNR 12.0, 1 hop). Both rows carry packet hash
392926C85DCB87D0, but the check saw only the echo, disagreed on path length and
SNR, and left the route unresolved — so the bot withheld a path it had decoded
correctly and answered "No path information available in current message". The
reporter's own observation that MultiTest still reported paths is the tell:
multitest reads recent_rf_data directly and never consults the match tag, so the
radio data was there the whole time.
The cache is now searched for the row the payload matches rather than testing
just the newest one. #80's guarantee is unchanged — a route is still only ever
attributed on a positive match, never on recency — so this widens where the
check looks, not what it accepts. When more than one row matches they must
resolve to a single non-empty packet hash, which is only true of receptions of
one packet; two unrelated packets that happen to agree on all three fields stay
a fallback. A debug line now names the case where the newest row was not the
message's packet, so this failure mode is visible in logs rather than silent.
Side effect worth knowing: SNR and RSSI now come from the message's own
reception too. The reporter's message was logged at SNR 12.0 / RSSI -10, the
echo's figures, when its actual reception was 13.25 / -32.
Five tests cover it, including a reproduction built from the issue's log. Three
fail on the current code; the other two pin behaviour the fix must not break —
newest-wins among receptions of one packet, and scope_eligible_only still
filtering the search so the scope correlation cannot be handed an ineligible row.
{packet_hash} came back empty on a channel, and so did {path} and
{connection_info}. message.routing_info is only attached when the RF row is
correlated, and a channel message can never correlate: MeshCore's CHAN event
carries neither raw_hex nor a pubkey prefix, so every prefix strategy in
find_recent_rf_data is skipped and the message lands on the most-recent-packet
fallback, which #80 rightly refuses to take a route from.
The fallback is in fact almost always the right packet—the firmware emits the
RF log row and the decoded CHAN event for one reception back to back—but
"almost always" is a guess, and #80 is what guessing costs. The CHAN payload
does restate three things the RF row records independently: payload type,
path length and SNR. Requiring all three to agree, on a row no older than the
correlation window, turns the fallback into a checked hypothesis and earns a
real match kind rather than a fallback tag.
SNR carries the weight: it is one reception's measured value, quantised to
0.25 dB, so an unrelated packet agreeing on all three is improbable rather
than merely unlikely. Across the 55 channel messages in my logs the
immediately preceding RF row agreed on all three every time, always within a
second, while a window-wide search on the same fields was ambiguous for half
of them—so this verifies the ordering the firmware already gives us instead
of searching for a match.
Because these messages now correlate, they also satisfy the '*' gate by the
primary route, resolve their scope properly when flood_scopes is regional,
and contribute their path to the mesh graph, which they never did before.
MeshCore's CHAN payload carries neither raw_hex nor a pubkey prefix, so a
channel message has no correlation key and find_recent_rf_data always lands
on the most-recent-packet fallback. rf_data_is_correlated() is therefore
never true for one, and _is_confirmed_global_flood could never pass: across
three log files every one of 45 received channel messages was rejected.
I raised this when the gate went in and accepted the counter that the
handler already has the general RF correlation and decoded packet info for
this message's own packet. That holds for DMs and is false for channel
messages.
'*' asks one question—was this a scoped regional flood?—and there is a
second way to answer it that the handler already computes and then discards.
A scoped message travels as TRANSPORT_FLOOD GRP_TXT, so if no such packet is
anywhere in the RF window, the message cannot have been scoped. That is a
window-wide fact, so unlike a route type read off a fallback row it does not
depend on having picked the right cached packet.
Correlated rows stay authoritative: a correlated TRANSPORT_FLOOD is still
blocked even when the scope window is empty, so an empty window cannot
launder a message the radio positively identified as scoped. Replaying the
real packets from the log, the 43 plain-FLOOD messages now get a reply and
the 2 that had a TC_FLOOD GRP_TXT alongside them still fail closed.
A flood_scopes of "*" alone leaves scope_keys empty while setting
flood_scope_allow_global, and the loader already logs that as an active
allowlist. The handler gated on scope_keys alone, so that configuration
skipped authorisation entirely and admitted absent, uncorrelated and
TRANSPORT_FLOOD traffic. The gate now fires when either is set, so "*"
means global-only rather than everything.
The scheduled-message duplicate check ran outside the write lock inside
update_ini_values, so two concurrent creates for the same schedule could
both pass and the second silently replace the first instead of getting
the 409. Check and write are now one critical section, and delete is too.
Repaired the delete path's error handling while moving it, so an OSError
during the write is still a 500 rather than escaping.
Accepting the reviewer's rejection of my narrower version. I had argued
that requiring positive proof would silence legitimate global replies
whenever correlation is unavailable, but that objection does not hold:
the handler already has the general RF correlation and decoded packet
info for this message's own packet, so the normal global case can be
proven rather than assumed. Only genuinely uncorrelated traffic is
affected, and for an allowlist that should fail closed.
_is_confirmed_global_flood() now requires RF data correlated to this
message showing RouteType.FLOOD. Absent, uncorrelated, or
TRANSPORT_FLOOD data means the scope is unknown, and '*' no longer
admits it.
The channel-message tests never set flood_scope_keys, so it was a Mock
and read as truthy, meaning they were unintentionally exercising the
allowlist and only passed because `not allow_global` on a Mock is False.
Set explicitly to the unconfigured default they meant to test.
Path distance was still leaking between concurrent requests three ways,
all of which Codex reproduced:
- The formatter fell back to shared instance state whenever the request's
own value was None, so a request with no measurable distance rendered
another request's. It now consults instance state only when there is no
request to read from.
- One extraction branch still called _decode_path without the request, so
that route stored its distance on shared state alone.
- _get_sender_location read the shared _current_message, so an
interleaved command could measure from the wrong sender. It takes the
request's message, falling back to the shared value only for direct
calls that pass nothing.
Scope authorisation: '*' permits unscoped global traffic, not traffic of
unknown scope. When a scope-eligible packet was heard but could not be
tied to this message, its scope is unknown and '*' no longer admits it.
Deliberately narrower than the reviewer suggested: when no scope-eligible
packet was heard at all, '*' still applies, because nothing scoped being
heard is what a genuinely global message looks like. Requiring positive
proof of FLOOD in that case would silence legitimate global replies
whenever correlation is unavailable, which is too much availability to
trade for the residual risk.
Two fixes from the previous round were incomplete:
- The scheduled-message chunk budget used the schedule's explicit scope,
but send_channel_message resolves an unset scope from
flood_scope.<channel> and then outgoing_flood_scope_override. A
schedule with an implicit regional scope was therefore sized for a
global send and every chunk could overshoot once the sender added the
regional header. The budget now resolves the effective scope, and
assumes regional if that resolution fails, since guessing regional only
ever makes chunks smaller.
- Resetting _last_path_distance_km per request fixed sequential reuse but
not concurrency: the decode awaits a database lookup, and the
dispatcher runs handlers as independent tasks, so two path commands can
interleave and render each other's distance. The distance now rides on
the request's own message, with the instance attribute kept only as a
fallback for direct calls.
Two new findings:
- rf_data_is_correlated() treated pubkey and partial-prefix matches as
packet-unique, but a sender prefix identifies a sender, not one
transmission. With several cached packets from the same sender, the
first (usually oldest) was returned and allowed to supply a route.
Those strategies now take the newest match and are authoritative only
when the match is unambiguous; otherwise the entry is marked fallback,
so it still provides SNR/RSSI but never a route.
- The flood_scopes allowlist accepted a scope resolved from an
uncorrelated fallback packet. The HMAC proves the cached packet is in
an allowed scope, not that this message is, so a recent allowed-scope
packet could admit an unrelated message. Scope authorisation now
requires packet-bound correlation and logs plainly when it does not
have it.
That last one is a deliberate fail-closed change to an authorisation
path that predates tonight. Channel messages normally carry raw_hex and
correlate exactly, so the fallback is the exception rather than the rule,
but a deployment using flood_scopes will now stay quiet in cases where it
previously replied on an assumed scope.
Correctness:
- {path_distance} was always blank in production. The resolution code that
builds repeater_info dropped latitude/longitude in every branch, so the
calculator could never find a coordinate. My tests passed hand-built
dicts straight to the calculator and never exercised the builder, which
is why they stayed green. Coordinates are now carried through all four
construction sites, and the new tests drive _lookup_repeater_names via
its lookup_func hook so the real builder runs.
- The #80 route guard was defeated two ways in the channel handler. When
the RF data was an uncorrelated fallback, control fell through to the
raw-hex and routing_info fallbacks below, which took the route from the
unrelated packet anyway; the guard had actually made that path
reachable. message.routing_info was also assigned unconditionally, and
the path command reads it. "Not attributable" is now a terminal branch
and the routing_info hand-off checks provenance.
- Same fix was incomplete for DMs: routing_info was captured and turned
into path_info before the provenance check ran, so the later check only
declined to overwrite an already-wrong value. Guarded at the source.
- Rendering could transmit for real. Capture only intercepts
send_response, but advert calls send_advert() directly and
send_response_chunked never checked capture_sink. Chunked sends are now
captured, and rendering is opt-in via BaseCommand.render_safe (default
False) instead of a denylist that cannot be complete. This also closes
the DM-only leak: schedule is not marked safe, so {cmd:schedule} can no
longer broadcast configuration to a channel.
- Multi-part rendered output was rejoined into one oversized send.
Scheduled messages are now split to the RF body budget and sent through
send_channel_messages_chunked, on character boundaries so multi-byte
text is not corrupted.
- _last_path_distance_km is instance state that was only set on success,
so an invalid path request could show the previous request's distance.
Reset at the start of every execute().
- Stale-contact retries were only counted on non-OK results, so timeouts
and exceptions left a contact eligible forever and could recreate the
storm. All failed attempts count now.
Web viewer:
- A failed config reload was reported as a successful save. The API and
UI now distinguish "saved and active" from "saved, restart needed".
- Preview count was unbounded; clamped to 1-20.
- update_ini_values is a read-modify-replace with no locking, so two
concurrent viewer saves could lose one. Serialised behind a lock.
find_recent_rf_data has four strategies, and the fourth returns the most
recent packet in the cache when the first three fail to correlate. That
fallback exists for SNR/RSSI timing issues, but callers were also taking
its route. So when correlation missed, a message was recorded with some
other transmission's path and hop count — the reporter's four-hop message
stored as a single direct hop via 79, which is the last hop of an
unrelated packet. Rare, because it only fires when correlation fails.
Results are now tagged with how they were matched (exact, pubkey, partial
or fallback) and rf_data_is_correlated() gates the route. The tag rides on
a shallow copy so the cache entry is never marked, and it fails closed: an
untagged dict is treated as uncorrelated.
Guarded in both handlers. The DM path was worse than the channel one — it
falls back to find_recent_rf_data() with no correlation key at all, which
can only ever return a fallback, and then overwrote message.path and
message.routing_info from it. routing_info is what the path command reads,
so a wrong route reached the user directly.
The route is now left unresolved rather than fabricated, which also stops
a bogus edge being written to the mesh graph from path_nodes that belong
to a different packet. SNR and RSSI still use the fallback as before;
mis-correlated signal figures are approximate rather than structurally
wrong, and changing them is a separate call.
Existing tests asserted the returned dict was the cache entry itself.
Identity was incidental, so they now compare contents and additionally
assert the provenance tag.
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
Changed the default setting for `auto_manage_contacts` from `false` to `device` across configuration files and updated related documentation. This ensures that the device handles auto-addition of contacts while the bot manages capacity. Adjusted comments and documentation to reflect this change for clarity and consistency.
- Updated logging statements in the `MessageHandler` class to use `debug` level instead of `info` for `RAW_DATA` and `NEW_CONTACT` events, reducing log verbosity while maintaining essential information.
- This change enhances the clarity of logs by reserving `info` level for more significant events.
- Resolves issue #214
NEW_CONTACT adverts were always logged as newly discovered, even for
nodes heard many times before. Check the tracking DB (get_tracked_contact_row)
and the live device contact list (is_contact_on_device) to classify the
advert, log "known" vs "new" accordingly, and skip the
new_contact_discovered audit entry for contacts that are already known.
Also dedup the is_currently_tracked device-contact lookups onto the shared
is_contact_on_device() helper.
Behavior preserved: repeaters remain database-only, device-mode companion
adds are still left to the firmware, and no per-advert device round-trip is
introduced.
- Updated `config.ini.example` and `configuration.md` to enhance clarity on the `outgoing_flood_scope_override` and `flood_scopes` settings, detailing their behavior and interactions.
- Improved logging in `CommandManager` to provide better insights into scope resolution and potential issues during message sending.
- Added new methods in `MessageHandler` for improved RF data correlation, ensuring eligibility checks for flood scope matching.
- Enhanced unit tests to cover new behaviors and ensure robust handling of flood scope configurations.
- Introduced optional regional TC_FLOOD scope configuration for various services, allowing for more granular control over message routing.
- Updated the CommandManager to resolve and apply flood scopes from messages, configuration sections, and explicit parameters, ensuring correct message delivery.
- Enhanced service plugins to utilize the new flood scope functionality, including weather, earthquake, and webhook services.
- Added unit tests to verify the correct resolution and application of flood scopes in different scenarios, ensuring robust functionality.
- Updated the `track_contact_advertisement` method to return a structured result indicating success and duplicate packet status, improving clarity in handling contact advertisements.
- Adjusted the `MessageHandler` to utilize the new result structure, allowing for better decision-making based on tracking outcomes.
- Enhanced unit tests to cover new behaviors and ensure correct handling of duplicate packets and advertisement tracking results.
Route process_message keyword/RandomLine sends through send_response so
reply_scope reaches set_flood_scope; add optional command_id to
send_response for transmission tracking.
- Replaced the use of AdvertFlags enum for flag parsing with bitwise operations to handle invalid values correctly.
- Updated the parsing logic in both MessageHandler and MapUploaderService to use boolean flags for better clarity and performance.
- Adjusted error logging to use warnings instead of errors for parsing issues, enhancing the logging strategy.
- Added unit tests to ensure correct parsing behavior for edge cases with invalid flag values.
- Updated string quotes from single to double quotes for consistency across the MessageHandler class.
- Enhanced formatting of method parameters for better readability.
- Minor adjustments to logging statements for clarity.
- Added configuration options `mqtt_skip_unparseable_packets` and `advert_require_valid_signature` to control MQTT publishing behavior based on packet validity.
- Updated `decode_path_len_byte` function to return `None` for reserved size codes, improving path length validation.
- Implemented logic in `PacketCaptureService` to skip publishing unparseable packets and ADVERT packets with invalid signatures.
- Introduced `verify_meshcore_advert_ed25519` function for signature verification of ADVERT packets, with corresponding unit tests to ensure functionality.
- Enhanced documentation to reflect new configuration options and their effects on packet processing.
- Updated the logic for retrieving the sender's public key in the MessageHandler class to ensure it defaults to sender_id when pubkey_prefix is not available.
- Enhanced the condition to check for sender_id presence before accessing contacts, improving robustness in public key resolution.
- Added a new migration to include a 'details' column in the 'purging_log' table for improved logging of actions.
- Refactored the `log_purging_action` method in `RepeaterManager` to handle both new and legacy schemas, ensuring compatibility with the updated database structure.
- Updated `MessageHandler` to utilize the new logging method, replacing direct database updates with a unified logging approach.
- Added tests to verify the presence of the new column and the correct functionality of the logging mechanism across different scenarios.
- Enhanced configuration options for `auto_manage_contacts` to support 'device' mode, allowing firmware to handle companion auto-addition and favourite hygiene.
- Updated `MessageHandler` to track new companions based on the `auto_manage_contacts` setting, with distinct behaviors for 'false', 'device', and 'bot' modes.
- Introduced scheduled jobs in `MessageScheduler` for device mode to manage firmware preferences and favourite keys with specified delays.
- Modified `RepeaterManager` to skip companion auto-purge in device mode, ensuring firmware manages contact slots effectively.
- Added tests to validate new behaviors and configurations, ensuring robust handling of contact management across different modes.
Simplifies find_recent_rf_data so the sample it returns is always a single
wire observation — snr, rssi, raw_hex, and routing_info all belong to the
same decoded copy. The #80 fix stays intact via the two earlier invariants:
unmatched correlation_key returns None (no fallback to an unrelated recent
packet), and the no-key DM path is constrained to a narrow
rf_fallback_window.
An earlier iteration tried to enrich routing_info with the longest observed
same-packet_hash forward, but that would have left message_stats rows with
hops sourced from the matched observation while path was a different
forward's — an internal skew visible in the web viewer and !path output.
The enrichment is documented in TODO.md as a v0.9.1+ option-C follow-up
that must also sync message.hops for consistency.
Regression tests in tests/test_message_handler.py::TestFindRecentRfData
cover: return-None on unmatched key, narrow-fallback-window rejection of
stale samples, and the no-cross-wiring assertion for same-packet_hash
forwards.
Made-with: Cursor
- Rewrite test_subscribe_packets/messages_emits_status_ack to match the
silent subscription UX from 1ee84f2.
- Reconcile Python version: requires-python>=3.10, ruff target py310, CI
matrix adds 3.13, pyupgrade UP0xx ignored pending a separate typing-rewrite
PR; fix two B905 zip(strict=...) lints.
- Issue #80 fix in find_recent_rf_data: return None when correlation_key is
provided but unmatched; prefer the longest observed path among samples
sharing a packet_hash; narrow the no-key fallback to a configurable
rf_fallback_window (default 2s).
- Issue #161: lower shipped max_response_hops default 10 -> 7.
- Add CHANGELOG.md, restructure BUGS.md around a ## v0.9.0 Fixed Bugs
table, prune crossed-out duplicate outstanding rows, and add a
Deferred-from-v0.9.0 triage section to TODO.md.
- Untrack coverage.json and add it to .gitignore.
Made-with: Cursor
Added a new PendingMessageEntry TypedDict to improve the structure of pending messages. Updated type hints throughout the MessageHandler class for better clarity and consistency, including the initialization method and message processing functions. This refactor enhances code readability and maintainability without altering existing functionality. Additionally, included the new message_handler module in the pyproject.toml for better module organization.
Added asyncio timeout handling for sending startup and interval-based adverts, improving reliability by recording failures on timeout. Updated relevant tests to ensure coverage for timeout scenarios and increment failure counts appropriately. Introduced a new Radio Reliability panel in the web viewer for monitoring radio status.
Two small incompatibilities surfaced after rebasing dev-kg7qin-changes
onto upstream/dev in code introduced by #138:
- message_handler.py: replace two remaining Optional[str] annotations
with str | None; the Optional import was removed during typing
modernization but two annotations in this module were missed
- command_manager.py: restore # noqa: F401 guard on PUBLIC_CHANNEL_KEY_HEX
re-export so ruff auto-fix does not silently remove it (imported by core.py)
Do not merge until #138 is merged.
Add SSRF host validation to maintenance.py send_nightly_email and
scheduler.py send_zombie_alert_email using validate_external_url().
New allow_local_smtp config key permits private-IP SMTP for local
relay setups.
Add sanitize_name() to security_utils and apply it to all log calls
in message_handler, repeater_manager, path_command, solarforecast_command,
command_manager, and discord_bridge_service to prevent log injection.
Move nightly email logic from duplicate scheduler._send_nightly_email()
into the canonical maintenance.py implementation, removing the duplicate.
Update tests to call maintenance.send_nightly_email() directly.
Add validate_external_url allow_private parameter with support for
loopback, RFC1918, CGN, and link-local address ranges.
- Updated `config.ini.example` to introduce the `respond_to_mentions` setting, allowing configuration of how the bot responds to mentions in channel messages.
- Refactored `MessageHandler` to implement logic for handling mentions based on the new configuration, including stripping mentions when appropriate.
- Added `cleanup_message_for_matching` method in `BaseCommand` to streamline message processing and mention validation.
- Enhanced various command classes to utilize the new cleanup method for consistent mention handling.
- Introduced tests to validate the behavior of the new mention handling logic across different configurations.
- Updated `config.ini.example` to clarify the usage of `outgoing_flood_scope_override` and `flood_scopes`, providing examples for better understanding.
- Modified `configuration.md` to reflect changes in flood scope handling, emphasizing the distinction between `outgoing_flood_scope_override` and `flood_scopes`.
- Refactored `CommandManager` to utilize the new `outgoing_flood_scope_override` for sending messages, ensuring consistent scope handling.
- Enhanced `MessageHandler` to prioritize library-provided scope fields for improved accuracy in flood scope matching.
- Added tests to validate the handling of flood scope fields from library payloads, ensuring robustness in message processing.
agessaman