[Test_Command] distance_unit (auto/km/mi) controls {path_distance} and {firstlast_distance}. Auto follows the reply language (miles for en/en-US, kilometres otherwise, including en-GB). Elapsed times of 1s and above render as seconds to keep the ack short.
The published packet payload mixed clocks: "timestamp" was UTC ISO 8601
with a Z suffix, but the "time" and "date" fields beside it were rendered
from a naive datetime.now(), i.e. the host's local wall clock. A consumer
reading those two off a bot in a non-UTC zone sees a skew of exactly that
zone's UTC offset and flags the observer's clock as wrong.
Local time was never the intended reading. The original script took time
and date off the firmware log line, and mctomqtt sets the device clock
from calendar.timegm(), so upstream both fields are already UTC.
All three now render one aware UTC instant, so they cannot disagree with
each other or straddle a second boundary. _utc_iso_timestamp() takes an
optional instant to make that possible; its no-arg behavior is unchanged.
test_neighbor_evidence_edges.py pinned RECENT to 2026-08-01 and used it as the
default last_seen for the seeded links. The two days=30 tests compare that against
wall-clock now, so once 2026-08-31 passed, the "recent" link filtered out as stale
and both assertions saw an empty set. RECENT, EARLIER and ANCIENT now derive from
datetime.now(timezone.utc), which fixes their relationship to the window no matter
when CI runs. Nothing else in the file depended on the literal values—the other
assertions compare against the same constants, and the filter only ever reads
last_seen, never first_seen.
Swept the rest of tests/ for the same shape by running the suite under a clock
shifted forward 400 days, then 10 years. That turned up one more:
test_packet_capture_neighbors.py stored 1774482900.0 (2026-03-25) as the persisted
neighbors timestamp, and _load_neighbors_timestamp rejects anything older than
now-400d, so those two round-trip tests would have started failing on 2027-04-29.
That value and the far-future one (2**31, i.e. 2038) are both relative now.
The failures that remain under a shifted clock are all fixtures seeding through
SQLite's datetime('now') or time.strftime() while the code under test reads
Python's clock. Those two move together on a real runner, so they are artifacts of
the sweep rather than time bombs.
Add shlink support and rework response templates onto a state machine.
Merged via this branch rather than the PR head: the fork is org-owned, so
"allow edits by maintainers" does not grant push access and the review fixes
could not be pushed to #254 directly. Roger Fedor's four commits are included
with authorship intact, followed by two commits addressing the review.
Differentially tested the current parser against the pre-PR one over every template
shape shipped in config.ini.example and docs (31 shapes x 3 field sets x 4 message
states, 372 comparisons) plus 32 adversarial inputs. That found one regression I had
introduced: reading a quote immediately after ':' as the start of a quoted argument
voided `prefix_if_nonempty:"`, whose unterminated string rejected the whole
placeholder and emitted raw template text. An unterminated quote now falls back to
greedy parsing, so a literal quote prefix behaves as it always has. Both cases are
pinned by tests.
Also covered the shorten_url alias in feed formats, which had none: accepted as a
function, chains like shorten, falls back to the original on failure, is seen through
by shorten_feed_urls so a link is not shortened twice, and does not swallow an
unrelated name that merely starts with it.
Reset the warn-once flag in the async no-warning test. It asserted warning was never
called while a preceding test could already have tripped the flag, so it would have
passed vacuously.
Security: a shlink deployment with short_url_website unset POSTed the operator's
API key to v.gd. _normalize_base falls back to the public default and the shlink
branch guarded only on a missing key, never on a missing base. shlink now requires
an explicit base and refuses a v.gd/is.gd host outright rather than sending an
X-Api-Key there.
Correctness: _shorten_url_with_gd lost its response.ok guard when the backends were
split out, so a 502 whose body starts with http was returned as the short URL and
transmitted over RF. Restored, with a warning. The regression test that should have
caught this passed only because it left mock_resp.text as a MagicMock; it now uses
a realistic proxy maintenance body.
_shorten_url_with_shlink never checked the status either. Shlink reports failures as
RFC 7807 problem details, which parse as JSON and simply lack shortUrl, so a bad API
key was indistinguishable from an unshortenable URL at DEBUG. It now warns on a
non-OK status, a non-JSON body, and a missing shortUrl. Dropped the shortUrlSlug
fallback: that is a request field, not a response field, and returning it puts a
bare slug where a link belongs.
Timeouts and connection errors are back on their own DEBUG handler. They had fallen
through to the broad handler, whose level rose to ERROR in the same diff, so a
routine intermittent uplink logged "Unexpected error shortening URL" on every reply.
Parser: _GREEDY_ARG_FILTERS was tested before the quoted-argument branch, so
prefix_if_nonempty, the one filter already in shipped configs, could not use the
quoted syntax this PR adds. {path_distance|prefix_if_nonempty:"Dist {sender}: "}
rendered raw template text. A quote immediately after the ':' now selects the quoted
grammar; anything else stays greedy, so config.ini.example's
`prefix_if_nonempty: | Path Dist: ` keeps its pipe and its whitespace.
Blocking HTTP on the event loop: the path command rendered its reply prefix inline
from async code, so a slow shortener stalled radio RX, MQTT and every other handler
for the full 5s timeout. Added format_piped_template_async and switched the path
command to it. The test command still renders synchronously through the sync
check_keywords dispatcher; making that async is a separate change, so the filter now
warns once when it runs on the loop.
Naming: one operation should not have two names in an operator-facing DSL. shorten
and shorten_url are aliases in both response templates and feed formats, and
if_nonempty is canonical with if_notempty as an alias, so a filter chain copied
between a feed format and a response_format works either way.
Also: reduced _build_create_shlink_url to the one parameter it uses and dropped its
dead query/startswith lines, renamed the shlink POST callable from `get`, documented
the config argument on format_piped_template, stopped gating the render trace on an
unrelated parameter and logging field values (sender IDs, user phrases) with it, and
moved the changelog entry from Fixed to Added and Changed.
- Extracted `modules/alert_format.py` as the single NWS alert formatter, replacing four copies of the event-type abbreviation table and two of the time compactor. `!wx alerts` and the proactive `WeatherService` broadcasts now localize from one code path, so a Russian bot no longer answers `!wx alerts` in English while its proactive alerts are Russian.
- Stopped leaking translation key paths into mesh broadcasts. `Translator.translate` returns the dotted key when a lookup misses in both the locale and the English fallback, which is right for development but reached the air in production: an unclassifiable NWS title rendered as `⚪Hazardous services.weather_service.event_types.Unknown`, an unmapped WMO code as `services.weather_service.weather_descriptions.4`, and an oddly-cased `wind_speed_unit` as `services.weather_service.wind_speed_units.KMH`. `alert_format.translate_or()` carries an English default at each site, and `WeatherService` now normalizes and validates its three `[Weather]` unit settings the way `GlobalWxCommand` already did.
- Fixed alert expiry rendering in every locale. The formatter rendered a timestamp to a string and re-parsed its own output with `(\d+)(AM|PM)` against a hardcoded English month list, so translated months took the wrong branch and truncated mid-string. Times now carry parsed parts and render through a per-locale `common.alerts.time_12h` template — the space before AM/PM was correct (Russian writes "6 дня", not "6дня"); the downstream regex was the bug.
- Restored month abbreviation. `_compact_time` iterated over abbreviations and replaced them in the string instead of mapping full names, so English stopped shortening "June 28" and Russian replaced the "Jun" inside "June", leaving a stray Latin "e" (`июнe 28`). Reuses the existing `common.date_time.month_abbreviations` rather than the duplicate `services.weather_service.months` block.
- Made `!gwx` display units follow `[Weather]` config instead of the response language. Visibility switched on `base_language != 'en'`, so `language = ru` with the default `temperature_unit = fahrenheit` printed Fahrenheit beside kilometers, and `en-GB` was forced to miles. Pressure is a locale convention rather than a metric/imperial split, so each catalog names its own via `commands.gwx.pressure_unit` — previously every non-English locale inherited mmHg from the English catalog, whose `pressure_mmhg` string contained Russian text, giving German and French users Cyrillic pressure units.
- Let localized `H`/`L` labels reach a standard install. `config.ini.example` shipped the three `temperature_*_format` keys uncommented with literal `H:`/`L:`, and a config value always beats the new locale-aware default, so a Russian bot built from the documented example still rendered `H:47°C L:33°C`. The example now uses the `{high_label}`/`{low_label}` placeholders, which were documented in the docstring but not in the file.
- Routed high/low labels through the reply's translator. `_format_high_low` passed `bot.translator`, so with `auto_detect_language` on, an English-default bot answering a Russian sender localized the rest of the line but not `H:`/`L:`. Added `BaseCommand.response_translator` for this, replacing `wx_international`'s reach into the private `_response_translator` ContextVar.
- Fixed a byte-budget overrun in `!gwx`. The guard on the extra conditions block compared a character count against a byte-derived budget while the rest of the function used `_count_display_width`; Cyrillic is two bytes per character, so the block was appended after the budget was spent.
- Reverted nine `commands.gwx` English rewordings that were not localization work, including the configuration hint in `mqtt_weather_no_subscriber` — dropping it left a mis-configured operator with no pointer to the two keys they need.
- Fixed the Russian `visibility` string, which said "км" on the miles key — the same locale/config conflation as the code bug, in the data. Shortened the Russian event-type abbreviations, which were full words consuming a quarter of the 130-byte budget at two bytes per character.
- Added `commands.wx.hourly_not_available`, missing from every catalog so `!wx hourly` printed the raw key path. Predates this branch; found while auditing every translation key the weather modules reference.
- Moved alert strings to `common.alerts.*` and wind directions to `common.wind_directions.*`, since a command and a service both read them.
Render GWX pressure in mm Hg (rounded hPa->mmHg conversion) for
non-English locales instead of hPa, matching the metric handling used for
visibility. Also fix format_temperature_high_low so custom [Weather]
templates can use the documented {high_label}/{low_label} placeholders,
and add tests for the translator param.
This commit addresses several issues related to MQTT connections. It prevents MQTT brokers from entering a reconnect storm by ensuring that the packet-capture watchdog only triggers a reconnect when necessary, avoiding duplicate CONNACKs and spurious disconnects. Additionally, it ensures that renewed MQTT auth tokens are properly enforced by implementing a clean reconnect process.
New configuration options are introduced: `mqttN_keepalive` for setting the PINGREQ interval per broker, and `mqttN_jwt_reconnect_on_renew` to control reconnect behavior after token renewal. Documentation has been updated to reflect these changes and to clarify the importance of unique client IDs for brokers in the same cluster.
{firstlast_distance|prefix_if_nonempty: | F/L Dist: } prints "| F/L Dist: N/A"
on a direct message: the distance helpers return the literal "N/A" when there
is no path to measure, prefix_if_nonempty only asks whether the value is
non-empty, and "N/A" is. Suppressing that needed a gate, and pathbytes_min was
the only one available—so it was being used as a stand-in for "did this
message take any hops", which is not what it asks. It asks how the path is
encoded, so pathbytes_min:2 also discards a one-byte multi-hop path whose
distance is real and measurable.
hops_min:N asks about the route itself. hops_min:1 drops a clause on a direct
message and nothing else:
hops_min:1 pathbytes_min:2
direct cleared cleared
1-byte 2 hops kept cleared <- the difference
2-byte 2 hops kept kept
unknown cleared cleared
An unknown hop count clears the value: a gate that cannot confirm the route
should suppress rather than guess, matching pathbytes_min.
The hop-count logic moves to utils.message_hop_count so the filter and
BaseCommand.get_hops_display_values share one implementation instead of two.
It returns None for unknown rather than 0, since a gate must not read
"cannot tell" as "direct".
{path_distance|pathbytes_min:2|prefix_if_nonempty: | Path Dist: } printed
"| Path Dist: N/A" on a direct message. bytes_per_hop_from_routing_and_nodes
returned routing_info's bytes_per_hop before it ever looked at whether there
were any hops, so a 0-hop packet whose format happens to use 2-byte hops
reported 2, the gate passed, and prefix_if_nonempty found "N/A" waiting for
it and dutifully attached the label.
bytes_per_hop describes how a path is encoded. A direct packet has no path
for it to describe, so the format field is not evidence of a wide path and
must not stand in for one. Both docstrings already promised 1 for the direct
case—"Returns 1 when no nodes (direct / unknown)"—so this is the code
catching up to its stated contract rather than a change of policy.
Latent until now: channel messages never had routing_info attached, so the
field was absent and the fallback returned 1 by accident. Direct DMs have
been hitting it all along.
Adds the 16-char MeshCore packet identity hash to the three formatters that
render user-facing templates: [Keywords] responses, the test command's
response_format, and the path command's reply_prefix. It is read only from
routing_info, which is attached solely from an RF packet correlated to the
message, so a hash from an unrelated transmission is never shown. Missing,
empty, and the 0000000000000000 error sentinel all render empty.
get_packet_hash_placeholder returns "" rather than a False sentinel. Empty
gives byte-identical output on every path—prefix_if_nonempty already tests
falsiness and str.format renders "" as nothing—whereas False would print
the literal string "False" into a mesh reply from any consumer of
get_standard_placeholder_fields that formatted it directly.
TestCommand.format_response now builds on get_standard_placeholder_fields
instead of duplicating it. That also honors the extra argument its docstring
already promised but silently ignored, and picks up the path-string hop
fallback, so {hops} reads "0" rather than "?" for a message whose hops and
routing_info are both unset but whose path says "Direct (0 hops)".
Documents the placeholder in config.ini.example, the default config that
core.py writes on a fresh install, the README and docs/path-command-config.md.
The generated config was also missing {elapsed}; added that too.
Keep the disabled-command fallback in one place. A disabled command still
matches; can_execute is what holds it back, and the dispatcher continues past it
so another command's alias can claim the trigger. The per-command enabled guards
in matches_keyword were a second mechanism for a problem already solved once, and
one every command would have to repeat, so drop them along with PathCommand's
matches_keyword override entirely.
Restore control-character tolerance in the test command. Matching runs against
clean_content-normalized text again, so a garbled "test" still matches. A test is
exactly the message someone sends when their link is marginal. The cleaned text
only sticks when the test command claims the message, since matching runs early
in the command scan and collapsing whitespace on another command's free-form
argument is not its business.
Make enable_p_shortcut = false actually work. Appending "p" to the inherited
class list left it on PathCommand.keywords for every instance built afterwards,
so a reload with the shortcut off still answered "p".
Have split_trigger_and_args honour the configured command prefix instead of
stripping a leading "!" unconditionally, and match word by word rather than
slicing the lowered copy by keyword length, which cuts the args in the wrong
place when str.lower() changes length for a non-ASCII alias.
Mention gating now applies to the test command, which the bespoke matcher had
skipped: "test @[Bob]" in a channel no longer gets an ack.
Added a new method `split_trigger_and_args` to the `BaseCommand` class, which splits message content into a matched keyword and arguments. This enhancement allows for better handling of command triggers and arguments across various command classes, ensuring that multi-word triggers are prioritized and leading characters are stripped appropriately. Updated the `ChannelsCommand`, `DiceCommand`, `HackerCommand`, `MultitestCommand`, `RollCommand`, and `TraceCommand` classes to utilize this new method for cleaner and more consistent command execution logic.
Removed redundant logging for command execution and added detailed debug logging for cases where a command cannot execute due to soft rejections. This allows for better tracking of command flow and ensures that keyword matching continues for subsequent commands. Updated the test_command to include config aliases in keyword matching, enhancing its flexibility.
Path distance was still leaking between concurrent requests three ways,
all of which Codex reproduced:
- The formatter fell back to shared instance state whenever the request's
own value was None, so a request with no measurable distance rendered
another request's. It now consults instance state only when there is no
request to read from.
- One extraction branch still called _decode_path without the request, so
that route stored its distance on shared state alone.
- _get_sender_location read the shared _current_message, so an
interleaved command could measure from the wrong sender. It takes the
request's message, falling back to the shared value only for direct
calls that pass nothing.
Scope authorisation: '*' permits unscoped global traffic, not traffic of
unknown scope. When a scope-eligible packet was heard but could not be
tied to this message, its scope is unknown and '*' no longer admits it.
Deliberately narrower than the reviewer suggested: when no scope-eligible
packet was heard at all, '*' still applies, because nothing scoped being
heard is what a genuinely global message looks like. Requiring positive
proof of FLOOD in that case would silence legitimate global replies
whenever correlation is unavailable, which is too much availability to
trade for the residual risk.
Correctness:
- {path_distance} was always blank in production. The resolution code that
builds repeater_info dropped latitude/longitude in every branch, so the
calculator could never find a coordinate. My tests passed hand-built
dicts straight to the calculator and never exercised the builder, which
is why they stayed green. Coordinates are now carried through all four
construction sites, and the new tests drive _lookup_repeater_names via
its lookup_func hook so the real builder runs.
- The #80 route guard was defeated two ways in the channel handler. When
the RF data was an uncorrelated fallback, control fell through to the
raw-hex and routing_info fallbacks below, which took the route from the
unrelated packet anyway; the guard had actually made that path
reachable. message.routing_info was also assigned unconditionally, and
the path command reads it. "Not attributable" is now a terminal branch
and the routing_info hand-off checks provenance.
- Same fix was incomplete for DMs: routing_info was captured and turned
into path_info before the provenance check ran, so the later check only
declined to overwrite an already-wrong value. Guarded at the source.
- Rendering could transmit for real. Capture only intercepts
send_response, but advert calls send_advert() directly and
send_response_chunked never checked capture_sink. Chunked sends are now
captured, and rendering is opt-in via BaseCommand.render_safe (default
False) instead of a denylist that cannot be complete. This also closes
the DM-only leak: schedule is not marked safe, so {cmd:schedule} can no
longer broadcast configuration to a channel.
- Multi-part rendered output was rejoined into one oversized send.
Scheduled messages are now split to the RF body budget and sent through
send_channel_messages_chunked, on character boundaries so multi-byte
text is not corrupted.
- _last_path_distance_km is instance state that was only set on success,
so an invalid path request could show the previous request's distance.
Reset at the start of every execute().
- Stale-contact retries were only counted on non-OK results, so timeouts
and exceptions left a contact eligible forever and could recreate the
storm. All failed attempts count now.
Web viewer:
- A failed config reload was reported as a successful save. The API and
UI now distinguish "saved and active" from "saved, restart needed".
- Preview count was unbounded; clamped to 1-20.
- update_ini_values is a read-modify-replace with no locking, so two
concurrent viewer saves could lose one. Serialised behind a lock.
Adds a Schedule page that lists every [Scheduled_Messages] entry with its
next run time and supports add, edit and delete. Writes go to config.ini
and queue a config reload, so schedules change without restarting the
bot, which was the actual request.
It edits the config section the bot already reads rather than
introducing a database table. reload_config() already re-runs
setup_scheduled_messages() with rollback, so there is nothing to keep in
sync and the schedule command lists exactly what the page shows.
Validation runs through the same parsers the scheduler uses, so the UI
cannot accept a schedule the bot would later reject. The builder
composes cron from plain-language options and previews the next five
runs; entries the bot cannot run are listed as "Not scheduled" with the
reason instead of being hidden, since a typo that silences a message is
what an operator most needs to see. The 15-minute floor for {cmd:...}
messages is enforced at save time too.
Also relabels the radio Disconnect button to "Stop Bot" behind a
confirmation (#240). It was never a radio-only disconnect: the main loop
runs while self.connected is true, so disconnecting exits the process.
That surprised an operator running under tmux with nothing to restart
it. disconnect_radio()'s docstring now says so as well.
Two guards on command placeholders in scheduled messages, neither
configurable, because both exist to protect a shared medium:
A 15-minute floor. A schedule containing {cmd:...} that fires more often
is rejected at startup with an error rather than quietly running slower.
The interval is sampled and measured by the tightest gap between
firings, so "0,1 * * * *" is correctly treated as every 60 seconds and
not as hourly. Schedules without a command placeholder are unaffected.
The command's own cooldown_seconds now applies to a render. Scheduling
is not a way around the rate a command was configured to run at. The
execution is recorded before the command runs, matching execute_commands,
so a slow or failing render cannot be retried straight past the cooldown.
Also fills documentation gaps from the preceding commits:
- --install-extras was only in the script's own --help; documented in
service-installation.md (including alongside --update-venv) and
upgrade.md.
- weather-service.md documented weather_alarm's once-a-day scheduling
with no route to more than one forecast a day, which is exactly what
people go there looking for. It now points at {cmd:wx ...} and notes
that sunrise/sunset still belong to weather_alarm.
A scheduled message can now embed the reply of any bot command:
[Scheduled_Messages]
0 6,12,18 * * * = Public:{cmd:wx Seattle}
The schedule side was never the missing piece — [Scheduled_Messages]
keys have been 5-field cron with @presets for a while, so multiple
times, intervals and @hourly were all already expressible. What was
missing was any way to get a service's output into a scheduled message:
placeholders were limited to mesh info (contact counts). That gap is
why services were growing their own schedule parsers.
CommandManager.render_command_output runs a command for its text alone.
MeshMessage.capture_sink makes that safe: when set, send_response
collects the reply and transmits nothing, checked before the
_last_response bookkeeping so a background render cannot overwrite the
response captured for a real user's command.
Degrades quietly in every failure mode — unknown, disabled, admin-only,
non-renderable, timed out or silent commands expand to nothing, so raw
{cmd:...} text is never put on the air, and a message left empty is not
sent at all. Command output is not re-scanned, so a reply containing
{cmd:...} cannot recurse. Commands that transmit directly rather than
returning text (announcements) are refused outright, since rendering
them would broadcast for real.
Bounded by [Bot] scheduled_command_timeout_seconds (default 30).
Total path distance was already a solved problem here: the test command
computes it and exposes {path_distance} through the piped-template
engine, with pipe filters and docs. The path command just had no way to
reach it, since reply_prefix went through plain str.format.
Rather than add a second distance implementation, this reuses what
exists. BaseCommand.get_standard_placeholder_fields now returns the
shared placeholder set so subclasses can extend it, the path command
renders reply_prefix through format_piped_template with an added
{path_distance}, and the sum itself uses utils.calculate_distance over
the nodes the path command has already resolved.
Distance covers sender -> each hop -> bot and is deliberately blank when
the chain cannot be measured end to end (unresolved hop, prefix
collision, missing or 0,0 coordinates, unknown sender, no configured
bot position), so a partial sum is never reported as the real distance.
Because the prefix now supports pipe filters, an empty value takes its
label with it: {path_distance|prefix_if_nonempty:📏 }
Supersedes #198, which added a parallel haversine, a show_path_distance
flag, and a distance_traveled key across 10 locales.
Add configurable PacketCapture observer name
Adds an optional [PacketCapture] observer_name used as the MQTT `origin`
for packet and status reporting, letting the observer identity differ
from the connected MeshCore device name. Unset keeps the previous
behavior. The public key backing origin_id, authentication, and topic
resolution is unchanged.
CHANGELOG conflict resolved by keeping both Unreleased entries.
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
This test suite verifies the behavior of the PacketCaptureService regarding the observer_name functionality, ensuring it correctly overrides and falls back to the device name as expected.
Enhance the handling of the global IATA configuration to ensure that blank values are treated correctly, preventing unintended namespace pollution. Introduce methods for claiming and releasing the neighbors discovery cycle, ensuring that overlapping requests are managed effectively. Update the NeighborsCommand to utilize these new methods, improving the accuracy of cooldown management for users. Additionally, add tests to validate the new behavior and ensure that cooldowns are respected during busy and disabled states.
Add a new state key for tracking the last attempt to discover neighbors, allowing the service to persist this information across restarts. This change ensures that airtime is managed effectively, preventing unnecessary retries after a failed cycle. The implementation includes methods for loading and saving the neighbors attempt state, enhancing the overall reliability of the packet capture service. Additionally, update tests to validate the new state management functionality.
Move the 15-minute gap between cycles from the DM command to the service, as
MIN_CYCLE_GAP_SECONDS. The command was one caller among several: the scheduler
retried a failed cycle on its own 300s backoff, keyed off last_neighbors_publish
which a failed cycle never stamps, so a discover round whose acknowledgement was
lost went back on the air every five minutes. run_neighbors_cycle now refuses a
cycle inside the gap whoever asks, and the scheduler's retry backoff waits out
whatever remains of it. Failures that never reached the radio still stamp
nothing, so re-checking a disconnected radio stays on the short backoff.
The command asks the service for the remaining wait instead of computing its own,
so the two cannot drift, and rewinds the sender's per-user cooldown to expire
with the shared one. The command manager records an execution before calling
execute(), so a refusal had been consuming the sender's full 15 minutes: told to
wait one more minute, they would retry and be refused for another fourteen.
Rewound rather than cleared — a refusal reply is airtime too.
Follow-up to 7cffe1d; all three confirmed against meshcore 2.3.8.
Charge the node-wide cooldown for the transmission, not the result.
last_neighbors_publish is stamped only by a cycle that completes, so a discover
request whose acknowledgement was lost spent the airtime and left the clock at
zero -- another sender could immediately start a second round. The service now
stamps last_neighbors_attempt before the request, and the command rations on
whichever stamp is later. A cycle that bails out before touching the radio still
records nothing, so retries stay possible.
Restore the contact path when req_regions_sync returns None. send_anon_req gives
up early when change_contact_path reports an error -- which is also what a lost
acknowledgement for an applied path change looks like -- and that return skips
the library's own reset_path. req_regions_sync collapses it to None, previously
treated as a plain no-response. On the common "neighbour did not answer" path
the extra reset is a redundant device command: no airtime, idempotent, and it
re-syncs the contact cache.
Stop reporting a rejected restore as a success. reset_path returns an ERROR
event for a device rejection or its own response timeout rather than raising, so
the helper logged "restored flood path" either way and hid a contact left pinned
to zero-hop. It now inspects the event and warns with the reason.
Four review findings on this branch, all confirmed against the code and the
installed meshcore 2.3.8:
Single-flight the discovery cycle. The command guarded only its own task, so
the scheduler's independent call could overlap a manual cycle and each round
would collect into the other's discover window. run_neighbors_cycle is now a
guard around the cycle body, refusing whichever trigger arrives second.
Make the 15-minute cooldown per node rather than per sender. The base class
rations per user, but the cost here is mesh airtime: users could take turns and
keep the radio discovering continuously. Measured from the last cycle that
produced a result, so the scheduler's cycles count and a cycle that bailed out
without transmitting does not start the clock.
Window the neighbours evidence label. The combined viewer applied `days` only
to mesh_connections while reading every lifetime row from neighbor_links, which
is deliberately never pruned — so a link last heard years ago kept claiming a
recent path-derived edge was a current direct neighbour.
Match that label on full public keys too. MeshGraph.add_edge does not promote a
1-byte edge that has no public key, but still fills in the keys discovery
supplied, so the 3-byte prefix comparison alone left confirmed neighbours
labelled singlebyte. Truncating our keys to 2 chars instead would relabel every
other node sharing that byte.
Restore a contact's flood path after an interrupted scope request. send_anon_req
pins a path-less contact to zero-hop and restores it after the send with no
try/finally, so our own budget cancelling the request left the contact pinned
and every later message to it sent direct-only.
The meshcore>=2.3.8 pin needed no change: PyPI publishes 2.3.8 now.
Port the observer firmware's neighbours feature into the bot's packet capture
service, by way of meshcore-packet-capture (upstream PRs #42/#43). On a long
interval the bot asks which repeaters it hears directly and records each
confirmed link with its measured SNR.
This is the strongest link evidence the bot collects: a first-party RF
measurement between two full 32-byte public keys. Path inference works from
1-3 byte prefixes with no keys, and complete_contact_tracking.hop_count
over-claims zero-hop (800 claimed vs 68 corroborated on the live database).
modules/neighbors_discovery.py keeps upstream's public names so its fixes and
tests stay portable. Two deliberate divergences:
- No command_lock plumbing. _SerializedCommands in modules/core.py already
serialises and paces every radio command, strictly more than upstream's
reentrant lock did.
- neighbors_collect_scopes defaults off. Upstream's zero-hop scope probe
relies on a neighbour not being a known contact; this bot tracks contacts,
and for a repeater with no stored path the library reaches zero-hop by
calling change_contact_path() then reset_path() -- mutating the device's
contact table per neighbour. Scope requests also hold the radio lock for
their whole round trip (~25s), stalling bot replies. The default cycle is
one command plus a passive listen window, during which the bot stays
responsive.
Evidence lands in neighbor_links and neighbor_observations (migration 22)
rather than mesh_connections, which cannot persist provenance. The viewer
exposes it as evidence=neighbors on /api/mesh/edges and a Neighbours Only
mode on the mesh page, with populated public keys and real SNR; confirmed
neighbours also relabel edges in the combined view and count as
provenance-trusted when framing the initial map.
neighbors_enabled is the single switch. Every enabled broker publishes once
it is on (mqttN_neighbors defaults true; set false to hold one back). The
topic derives from each broker's packets topic with the last segment swapped,
so a templated broker gets meshcore/{IATA}/{PUBLIC_KEY}/neighbors -- the
topic the firmware uses -- instead of an unrelated flat one. A derived
location-routed topic is skipped with a warning when no iata is set, rather
than publishing into meshcore/XYZ/... on a shared namespace. Snapshots are
non-retained: heard_secs_ago is relative to publish time, so a retained copy
would read as current days later.
Also adds a DM-gated `neighbors` command (the 12h interval floor makes
waiting for the scheduler impractical), which acks immediately and reports in
a second message once the window closes.
Requires meshcore >= 2.3.8 for send_node_discover_req / req_regions_sync.
- Updated mesh graph path splitting and aggregation to run in SQLite, improving performance by avoiding Python materialization.
- Defaulted graph persistence to batched writes for new installations, reducing WAL churn and SD-card writes.
- Enhanced data retention to execute shortly after startup, independent of the nightly maintenance schedule.
- Added a table-specific index for `mesh_connections` to support window and retention queries, ensuring efficient data access.
Remaining findings from the same triage pass. None are security
relevant; each produces a wrong user-visible result.
- greeter: treat rollout_started_at as UTC. It is written by SQLite
CURRENT_TIMESTAMP, but datetime.timestamp() read the naive value as
local time, shifting the backfill cutoff by the host's offset (7h on
PT). Users who posted inside that window were never marked as already
greeted and could be sent a welcome they should not have received.
- greeter: write greeted_at in SQLite's own format. The rollout backfill
used isoformat() while every other path used CURRENT_TIMESTAMP. "T"
sorts above a space, so ORDER BY greeted_at interleaved the two
formats wrongly, corrupting duplicate cleanup and the web viewer's
recently-greeted list.
- greeter: let an empty `channels =` disable the command. BaseCommand
reads an empty value as disabled-on-channels, but the greeter
collapsed "key absent" and "key present but empty" into one fallback
and kept greeting via monitor_channels.
- greeter: stop comma-splitting greeting text. channel_greetings split
entries on ",", so "Public:Welcome to the mesh, {sender}!" was stored
as "Welcome to the mesh" with the placeholder silently dropped. A
fragment now starts a new entry only when the text before its first
colon looks like a channel name, which keeps commas, URLs and clock
times attached to the greeting they belong to.
- wxsim_parser: match condition abbreviations longest-first. Substring
matching in dict order let RAIN shadow CHNC. RAIN, so five conditions
lost their "chance" qualifier (rain, snow, drizzle, t-storm, and
FAIR-P.C.).
- wxsim_parser: re-anchor "now" on each parse. current_date and
current_year were fixed at construction and wx_command builds one
parser at startup, so after a few days forecast dates rolled back a
year and staleness checks read permanently true.
- thesportsdb_client: hold the rate-limit read/sleep/write under a lock.
Concurrent callers read the same last_request_time, slept the same
interval and fired together, bursting past the 2.1s throttle. This
exposes an unrelated request fan-out problem in fetch_league_scores;
filed in TODO.md rather than fixed here.
- alert_command: stop duplicating the first incident. The tail treated
any single-line buffer as "header only" and appended incidents[0],
but messages after the first carry no header, so a final chunk holding
one incident had incident 0 pasted onto it. The same confusion inside
the loop also let a second incident be appended past the 130-character
limit.
- feed: reject non-positive poll intervals. -1 is truthy and was stored,
and the poller's `now - last_check >= interval` then treats the feed
as permanently due and re-fetches the URL every cycle; 0 was ignored
while still reporting success. The web viewer, which is the primary
editor, had no validation at all, and a JSON null there raised a
TypeError that aborted the poll cycle for every feed rather than one.
feed_manager falls back to the default for rows written before this.
- transmission_tracker: age out confirmed transmissions that have
repeats. Cleanup removed only repeat_count == 0, so repeated records
accumulated for the lifetime of the process, which matters on a Pi
Zero. Repeat counts are already persisted to packet_stream, so the
longer 30-minute retention loses nothing.
- mesh_graph: weighted-merge avg_hop_position when promoting an edge.
Promotion overwrote the average with the single new observation, so an
edge averaging 2.0 over 4 observations became 9.0 instead of 3.4 and
skewed subsequent path scoring.
- multitest_command: show the path that ends exactly at the display LCP.
The suffix helpers are correct in isolation; the loss happens in the
cluster formatters, where _shrink_display_lcp refuses to shrink a
single-token LCP and the trunk then rendered as "96 ┐", meaning
"everything continues past here" and dropping the bare 96 route while
the header still counted it. Now uses the file's existing "├ common"
marker. Empty suffixes are also no longer handed to the nested
renderer, which mapped them onto the same [] as a route ending at the
inner LCP and drew a row for a route that did not exist.
- sports_mappings: stop shadowing seven unique team nicknames. In one
flat dict a repeated key silently drops the earlier team, so hawks
resolved to the NBA Hawks rather than the Seahawks alias it was added
as, blazers to Kamloops rather than Portland, and rockets to Kelowna
rather than Houston. First definition now wins for hawks, giants,
jets, rangers, kings, blazers and rockets; every shadowed team keeps
its unambiguous full-name alias. The 55 city and abbreviation
collisions (chicago, sf, la, ...) are genuinely ambiguous and stay
last-wins, now pinned by a test so a new collision fails loudly
instead of passing unnoticed.
- Added a tracked `LICENSE` file and updated `pyproject.toml` to include license metadata.
- Enhanced `CHANGELOG.md` with recent changes and clarifications.
- Updated `config.ini.example` and related documentation to reflect clamping behavior for numeric limits in `[Feed_Manager]`.
- Improved startup validation to suggest corrections for unknown and misspelled keys.
- Refactored geocoding and HTTP request handling to run off-thread, preventing event loop stalls.
- Added thread safety to cache management in geocoding functions to avoid race conditions.
- Added a type ignore comment for the assignment of optionxform in config_schema.py to suppress type checker warnings.
- Updated path inference logic to return None when bot latitude or longitude is not set, ensuring safer handling of missing values.
- Enhanced the decoding of path nodes to check for None before appending repeater details, preventing potential errors.
Add optional payload decoding to the packet capture service. GRP_TXT
channel messages are decrypted (sender/text), ADVERTs are parsed
(name/role/lat-lon), and a nested "decoded" object is attached to each
packet alongside the unchanged raw fields.
- Comprehensive channel key store: bot's configured radio channels plus
decode_hashtag_channels, [Channels_List], decode_channel_keys, and the
built-in default Public key.
- Publishing the decoded object to MQTT is off by default and
configurable per broker via mqttN_include_decoded.
- Configurable packet-log rotation (off/size/time) for historical dumps.
The decoder lives in a standalone, dependency-free module
(modules/meshcore_payload_decode.py) so it can be shared verbatim with
the meshcore-packet-capture project.
Closes#197Closes#35
- Updated configuration examples to include new `{hops}` and `{hops_label}` placeholders for path command replies, enhancing response detail.
- Modified `BaseCommand` to support hop count retrieval and formatting, ensuring accurate display of hop information in responses.
- Enhanced unit tests to verify correct formatting of hop-related placeholders in response messages, improving overall command functionality.
- Introduced per-channel flood scope configuration in `config.ini.example`, allowing for more granular control over message routing.
- Updated `CommandManager` to normalize channel names and resolve flood scopes based on channel-specific settings, improving message delivery accuracy.
- Enhanced `MeshMessage` to consider channel-specific flood scopes when determining the effective outgoing flood scope.
- Added unit tests to verify the new flood scope resolution logic and ensure correct behavior across various scenarios.
- Enhanced the `TestCommand` class to include the `rssi` value in the response format, allowing for more detailed output.
- Updated the example configuration to reflect the new placeholder usage for `rssi`.
- Added a unit test to verify that the `rssi` placeholder expands correctly in the response message.
- Modified `WxCommand` to use its own keywords along with any aliases from `BaseCommand` when delegating to `GlobalWxCommand` for the Open-Meteo weather provider.
- Added a new test fixture for `WxCommand` configured to delegate to `GlobalWxCommand`, ensuring that aliases are preserved and correctly matched in keyword checks.
- Enhanced unit tests to verify the delegation behavior and keyword matching for both standard and aliased commands.
- Added a new function `nws_http_means_no_coverage` to determine if NWS HTTP status codes indicate no coverage for a location.
- Updated `WxCommand` and `WeatherService` classes to utilize this function, improving error handling for NOAA alerts.
- Introduced a lazy loading mechanism for `_nws_alerts_available` to manage alert availability based on NWS coverage status.
- Enhanced logging to provide clearer warnings when NWS alerts are unavailable due to HTTP errors.
- Introduced a new class `_SerializedCommands` to serialize host-to-radio commands, ensuring only one command is in-flight at a time to prevent USB-CDC buffer overruns and parser desynchronization.
- Added configuration options for `command_min_interval_ms` and `channel_fetch_interval_ms` to control pacing between commands and channel scans, improving firmware stability during reconnect sequences.
- Updated `ChannelManager` to utilize the new fetch interval during channel scans, and modified the reconnect delay to 10 seconds for gentler recovery from connection issues.
- Enhanced documentation in `config.ini.example` to reflect new settings and their purposes.
- Added cache_ttl parameter to fetch_precip_series_nws to enable caching of results for improved performance.
- Implemented logic to reuse cached results based on location and cache expiration.
- Updated tests to ensure proper handling of cache_ttl without causing errors.
- Refactored WorldCupFastcastClient to streamline connection handling and improve readability.
CoderNemesis