PUT verified original_schedule outside the lock, so a concurrent delete
between the check and the write resurrected the entry as a new one, and
two concurrent renames of the same original left both results present.
The existence check now runs inside _save_scheduled_message_locked
against the same snapshot the duplicate check uses, so create, update and
delete are each a single critical section.
Added a concurrency test: two simultaneous creates of one schedule now
produce exactly one 200 and one 409 with a single entry on disk.
A flood_scopes of "*" alone leaves scope_keys empty while setting
flood_scope_allow_global, and the loader already logs that as an active
allowlist. The handler gated on scope_keys alone, so that configuration
skipped authorisation entirely and admitted absent, uncorrelated and
TRANSPORT_FLOOD traffic. The gate now fires when either is set, so "*"
means global-only rather than everything.
The scheduled-message duplicate check ran outside the write lock inside
update_ini_values, so two concurrent creates for the same schedule could
both pass and the second silently replace the first instead of getting
the 409. Check and write are now one critical section, and delete is too.
Repaired the delete path's error handling while moving it, so an OSError
during the write is still a 500 rather than escaping.
Accepting the reviewer's rejection of my narrower version. I had argued
that requiring positive proof would silence legitimate global replies
whenever correlation is unavailable, but that objection does not hold:
the handler already has the general RF correlation and decoded packet
info for this message's own packet, so the normal global case can be
proven rather than assumed. Only genuinely uncorrelated traffic is
affected, and for an allowlist that should fail closed.
_is_confirmed_global_flood() now requires RF data correlated to this
message showing RouteType.FLOOD. Absent, uncorrelated, or
TRANSPORT_FLOOD data means the scope is unknown, and '*' no longer
admits it.
The channel-message tests never set flood_scope_keys, so it was a Mock
and read as truthy, meaning they were unintentionally exercising the
allowlist and only passed because `not allow_global` on a Mock is False.
Set explicitly to the unconfigured default they meant to test.
Path distance was still leaking between concurrent requests three ways,
all of which Codex reproduced:
- The formatter fell back to shared instance state whenever the request's
own value was None, so a request with no measurable distance rendered
another request's. It now consults instance state only when there is no
request to read from.
- One extraction branch still called _decode_path without the request, so
that route stored its distance on shared state alone.
- _get_sender_location read the shared _current_message, so an
interleaved command could measure from the wrong sender. It takes the
request's message, falling back to the shared value only for direct
calls that pass nothing.
Scope authorisation: '*' permits unscoped global traffic, not traffic of
unknown scope. When a scope-eligible packet was heard but could not be
tied to this message, its scope is unknown and '*' no longer admits it.
Deliberately narrower than the reviewer suggested: when no scope-eligible
packet was heard at all, '*' still applies, because nothing scoped being
heard is what a genuinely global message looks like. Requiring positive
proof of FLOOD in that case would silence legitimate global replies
whenever correlation is unavailable, which is too much availability to
trade for the residual risk.
Two fixes from the previous round were incomplete:
- The scheduled-message chunk budget used the schedule's explicit scope,
but send_channel_message resolves an unset scope from
flood_scope.<channel> and then outgoing_flood_scope_override. A
schedule with an implicit regional scope was therefore sized for a
global send and every chunk could overshoot once the sender added the
regional header. The budget now resolves the effective scope, and
assumes regional if that resolution fails, since guessing regional only
ever makes chunks smaller.
- Resetting _last_path_distance_km per request fixed sequential reuse but
not concurrency: the decode awaits a database lookup, and the
dispatcher runs handlers as independent tasks, so two path commands can
interleave and render each other's distance. The distance now rides on
the request's own message, with the instance attribute kept only as a
fallback for direct calls.
Two new findings:
- rf_data_is_correlated() treated pubkey and partial-prefix matches as
packet-unique, but a sender prefix identifies a sender, not one
transmission. With several cached packets from the same sender, the
first (usually oldest) was returned and allowed to supply a route.
Those strategies now take the newest match and are authoritative only
when the match is unambiguous; otherwise the entry is marked fallback,
so it still provides SNR/RSSI but never a route.
- The flood_scopes allowlist accepted a scope resolved from an
uncorrelated fallback packet. The HMAC proves the cached packet is in
an allowed scope, not that this message is, so a recent allowed-scope
packet could admit an unrelated message. Scope authorisation now
requires packet-bound correlation and logs plainly when it does not
have it.
That last one is a deliberate fail-closed change to an authorisation
path that predates tonight. Channel messages normally carry raw_hex and
correlate exactly, so the fallback is the exception rather than the rule,
but a deployment using flood_scopes will now stay quiet in cases where it
previously replied on an assumed scope.
Correctness:
- {path_distance} was always blank in production. The resolution code that
builds repeater_info dropped latitude/longitude in every branch, so the
calculator could never find a coordinate. My tests passed hand-built
dicts straight to the calculator and never exercised the builder, which
is why they stayed green. Coordinates are now carried through all four
construction sites, and the new tests drive _lookup_repeater_names via
its lookup_func hook so the real builder runs.
- The #80 route guard was defeated two ways in the channel handler. When
the RF data was an uncorrelated fallback, control fell through to the
raw-hex and routing_info fallbacks below, which took the route from the
unrelated packet anyway; the guard had actually made that path
reachable. message.routing_info was also assigned unconditionally, and
the path command reads it. "Not attributable" is now a terminal branch
and the routing_info hand-off checks provenance.
- Same fix was incomplete for DMs: routing_info was captured and turned
into path_info before the provenance check ran, so the later check only
declined to overwrite an already-wrong value. Guarded at the source.
- Rendering could transmit for real. Capture only intercepts
send_response, but advert calls send_advert() directly and
send_response_chunked never checked capture_sink. Chunked sends are now
captured, and rendering is opt-in via BaseCommand.render_safe (default
False) instead of a denylist that cannot be complete. This also closes
the DM-only leak: schedule is not marked safe, so {cmd:schedule} can no
longer broadcast configuration to a channel.
- Multi-part rendered output was rejoined into one oversized send.
Scheduled messages are now split to the RF body budget and sent through
send_channel_messages_chunked, on character boundaries so multi-byte
text is not corrupted.
- _last_path_distance_km is instance state that was only set on success,
so an invalid path request could show the previous request's distance.
Reset at the start of every execute().
- Stale-contact retries were only counted on non-OK results, so timeouts
and exceptions left a contact eligible forever and could recreate the
storm. All failed attempts count now.
Web viewer:
- A failed config reload was reported as a successful save. The API and
UI now distinguish "saved and active" from "saved, restart needed".
- Preview count was unbounded; clamped to 1-20.
- update_ini_values is a read-modify-replace with no locking, so two
concurrent viewer saves could lose one. Serialised behind a lock.
The bar had grown to twelve items at full expansion, over half of them
configuration surfaces. Radio, Scheduled Messages, Greeter, Feeds,
Plugins and Configuration now sit behind one gear, so the top level is
about what the mesh is doing (Dashboard, Real-time, Contacts, Mesh
Graph, Multibyte, Logs) and the gear is about how the bot is set up.
Logs stays top level: it is what you reach for while watching behaviour,
not while configuring.
Also added active-page highlighting, which the bar never had. A settings
page lights up both its dropdown entry and the gear, so the current
location is still obvious once a page is one level down.
Conditional entries keep their guards, so Greeter and Feeds appear in
the menu only when enabled.
Adds a Schedule page that lists every [Scheduled_Messages] entry with its
next run time and supports add, edit and delete. Writes go to config.ini
and queue a config reload, so schedules change without restarting the
bot, which was the actual request.
It edits the config section the bot already reads rather than
introducing a database table. reload_config() already re-runs
setup_scheduled_messages() with rollback, so there is nothing to keep in
sync and the schedule command lists exactly what the page shows.
Validation runs through the same parsers the scheduler uses, so the UI
cannot accept a schedule the bot would later reject. The builder
composes cron from plain-language options and previews the next five
runs; entries the bot cannot run are listed as "Not scheduled" with the
reason instead of being hidden, since a typo that silences a message is
what an operator most needs to see. The 15-minute floor for {cmd:...}
messages is enforced at save time too.
Also relabels the radio Disconnect button to "Stop Bot" behind a
confirmation (#240). It was never a radio-only disconnect: the main loop
runs while self.connected is true, so disconnecting exits the process.
That surprised an operator running under tmux with nothing to restart
it. disconnect_radio()'s docstring now says so as well.
Two documentation gaps behind open questions.
The [External_Data] repeater_prefix_api_url comment claimed that leaving
it empty "disables prefix command functionality" (#70). That is not what
happens: the prefix command answers from the bot's own database of heard
repeaters, and the API only augments it with node counts from a wider
dataset. The wrong comment is a plausible reason the question was asked
at all. Corrected, and the JSON contract is now documented in the command
reference for anyone serving their own endpoint, since map.w0z.is is
defunct and has no drop-in replacement.
Also documented the pipx path (#222). The installer already grew a
virtualenv in July, which covered the PEP 668 half of that report, but
the unanswered part was where config.ini, the database and local/ live
under a pipx install. They are all resolved relative to the directory
containing config.ini, which means a bare `meshcore-bot` picks up
whatever is in the current directory — so the guidance is to pass an
absolute --config. Both console scripts are already smoke-tested from an
installed wheel in CI, so this path is supported rather than incidental.
find_recent_rf_data has four strategies, and the fourth returns the most
recent packet in the cache when the first three fail to correlate. That
fallback exists for SNR/RSSI timing issues, but callers were also taking
its route. So when correlation missed, a message was recorded with some
other transmission's path and hop count — the reporter's four-hop message
stored as a single direct hop via 79, which is the last hop of an
unrelated packet. Rare, because it only fires when correlation fails.
Results are now tagged with how they were matched (exact, pubkey, partial
or fallback) and rf_data_is_correlated() gates the route. The tag rides on
a shallow copy so the cache entry is never marked, and it fails closed: an
untagged dict is treated as uncorrelated.
Guarded in both handlers. The DM path was worse than the channel one — it
falls back to find_recent_rf_data() with no correlation key at all, which
can only ever return a fallback, and then overwrote message.path and
message.routing_info from it. routing_info is what the path command reads,
so a wrong route reached the user directly.
The route is now left unresolved rather than fabricated, which also stops
a bogus edge being written to the mesh graph from path_nodes that belong
to a different packet. SNR and RSSI still use the fallback as before;
mis-correlated signal figures are approximate rather than structurally
wrong, and changing them is a separate call.
Existing tests asserted the returned dict was the cache entry itself.
Identity was incidental, so they now compare contents and additionally
assert the provenance tag.
The seed check added in c1cbdf9 only guarded stale-contact cleanup, but
three other paths read the same raw last_advert and drew the same wrong
conclusion from it:
_get_repeaters_for_purging sorts by apparent age, oldest first
_get_companions_for_purging scores by days_inactive, most inactive first
purge_old_repeaters removes anything older than a cutoff
In all three an unset clock reads as maximum age, so a node that had
never been time-synced was the first candidate for eviction regardless
of whether it was active. Unlike the stale-contact path, these do not
just log — they remove contacts.
Same rule everywhere now: an unset device clock means staleness is
unknown, and unknown staleness is not grounds for removal.
Corrects the timestamp guard shipped in 5b07ee2, which assumed the
reporter's "722 days ago" contacts were genuine mid-2024 observations.
They were not.
MeshCore seeds an unset clock with a hardcoded time rather than zero:
1715770351 (15 May 2024) in VolatileRTCClock, and RTC_TIME_MIN
1772323200 (1 Mar 2026) on the NRF52 and ESP32 RTC paths. Measured from
the date in the issue, 1715770351 is exactly 722 days — every affected
contact reported the same figure because they were all sitting on the
same seed.
So those contacts were never stale. Their clocks were unset, and the bot
was reading that as extreme age: they sorted to the top of the staleness
list, consumed the whole per-sweep removal budget, and the bot kept
trying to evict nodes that may well have been active. The previous
2020-then-2024-01-01 floor sat below both seeds and never fired.
Now matched against the seeds themselves, plus anything at or below the
earliest (which still covers a raw 0 decoding to 1970). A device running
unsynced for a while reports seed + uptime and remains undetectable;
noted in the docstring rather than guessed at.
A failed remove_contact leaves the contact on the device, so the next
cleanup sweep selected it again and logged the same warning again. With
the contact list stuck near its limit the sweeps kept coming, which is
the reported flood of hundreds of "Failed to remove stale contact"
warnings that only a restart cleared — and the restart only helped
because it reset the in-memory contact list, not because anything was
fixed.
Refusals are now counted per public key. After 3 consecutive failures
the contact is excluded from future sweeps, with one summary warning
saying the list may stay near its limit and that the contact needs
removing from the companion app. A successful removal clears the count,
so a transient failure costs nothing.
Also stop treating unset and future last_seen values as staleness. An
unset timestamp parses as 1970, and since candidates are sorted by
staleness descending, those entries sorted to the top and consumed the
whole max_remove budget every sweep — starving the contacts that could
actually have been removed. A genuinely old contact (the reporter's 722
days) is still selected; that is a real observation, not a bad
timestamp.
Two guards on command placeholders in scheduled messages, neither
configurable, because both exist to protect a shared medium:
A 15-minute floor. A schedule containing {cmd:...} that fires more often
is rejected at startup with an error rather than quietly running slower.
The interval is sampled and measured by the tightest gap between
firings, so "0,1 * * * *" is correctly treated as every 60 seconds and
not as hourly. Schedules without a command placeholder are unaffected.
The command's own cooldown_seconds now applies to a render. Scheduling
is not a way around the rate a command was configured to run at. The
execution is recorded before the command runs, matching execute_commands,
so a slow or failing render cannot be retried straight past the cooldown.
Also fills documentation gaps from the preceding commits:
- --install-extras was only in the script's own --help; documented in
service-installation.md (including alongside --update-venv) and
upgrade.md.
- weather-service.md documented weather_alarm's once-a-day scheduling
with no route to more than one forecast a day, which is exactly what
people go there looking for. It now points at {cmd:wx ...} and notes
that sunrise/sunset still belong to weather_alarm.
A scheduled message can now embed the reply of any bot command:
[Scheduled_Messages]
0 6,12,18 * * * = Public:{cmd:wx Seattle}
The schedule side was never the missing piece — [Scheduled_Messages]
keys have been 5-field cron with @presets for a while, so multiple
times, intervals and @hourly were all already expressible. What was
missing was any way to get a service's output into a scheduled message:
placeholders were limited to mesh info (contact counts). That gap is
why services were growing their own schedule parsers.
CommandManager.render_command_output runs a command for its text alone.
MeshMessage.capture_sink makes that safe: when set, send_response
collects the reply and transmits nothing, checked before the
_last_response bookkeeping so a background render cannot overwrite the
response captured for a real user's command.
Degrades quietly in every failure mode — unknown, disabled, admin-only,
non-renderable, timed out or silent commands expand to nothing, so raw
{cmd:...} text is never put on the air, and a message left empty is not
sent at all. Command output is not re-scanned, so a reply containing
{cmd:...} cannot recurse. Commands that transmit directly rather than
returning text (announcements) are refused outright, since rendering
them would broadcast for real.
Bounded by [Bot] scheduled_command_timeout_seconds (default 30).
Total path distance was already a solved problem here: the test command
computes it and exposes {path_distance} through the piped-template
engine, with pipe filters and docs. The path command just had no way to
reach it, since reply_prefix went through plain str.format.
Rather than add a second distance implementation, this reuses what
exists. BaseCommand.get_standard_placeholder_fields now returns the
shared placeholder set so subclasses can extend it, the path command
renders reply_prefix through format_piped_template with an added
{path_distance}, and the sum itself uses utils.calculate_distance over
the nodes the path command has already resolved.
Distance covers sender -> each hop -> bot and is deliberately blank when
the chain cannot be measured end to end (unresolved hop, prefix
collision, missing or 0,0 coordinates, unknown sender, no configured
bot position), so a partial sum is never reported as the real distance.
Because the prefix now supports pipe filters, an empty value takes its
label with it: {path_distance|prefix_if_nonempty:📏 }
Supersedes #198, which added a parallel haversine, a show_path_distance
flag, and a distance_traveled key across 10 locales.
Add --install-extras option to install-service.sh
Amended on merge: the new non-interactive path duplicated both pip
install blocks verbatim, and sat in an elif after VENV_UPDATED_IN_PLACE,
so `-u --update-venv --install-extras` silently skipped the extras it
asked for. Factored the two installs into install_profanity_packages
and install_geocoding_packages, shared with the interactive prompts,
and reordered so --install-extras takes precedence over the in-place
skip. Fixed "profantiy"/"geodecoding" in the usage text and added the
missing CHANGELOG entry.
List only enabled commands in the cmd response
Amended on merge: the enabled check passed the builtin `bool` as
`value_type`, which is a string parameter ('str'/'bool'/'int'/...).
That fell through to the unknown-type branch, logging a warning per
command per invocation and returning the raw string, so the comparison
against "false" only matched that exact spelling — `no`, `0`, `off`,
and `False` all stayed listed. The "false" fallback also hid commands
that have no [<Name>_Command] section at all.
Switched to the idiom already used at cmd_command.py:34
(fallback=True, value_type='bool') and added regression coverage for
every configparser boolean spelling plus the no-section default.
mypy cannot track non-Noneness through the intermediate `corroborated`
boolean, so `float(snr)` in the discover-only neighbor branch was flagged
as `Any | None`. Test the value directly; `corroborated` still backs the
`signal_corroborated` field.
Add configurable PacketCapture observer name
Adds an optional [PacketCapture] observer_name used as the MQTT `origin`
for packet and status reporting, letting the observer identity differ
from the connected MeshCore device name. Unset keeps the previous
behavior. The public key backing origin_id, authentication, and topic
resolution is unchanged.
CHANGELOG conflict resolved by keeping both Unreleased entries.
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
This test suite verifies the behavior of the PacketCaptureService regarding the observer_name functionality, ensuring it correctly overrides and falls back to the device name as expected.
This update ensures that console scripts in the virtual environment do not retain outdated shebangs pointing to temporary build paths. The `--update-venv` command now rewrites shebangs in place, allowing broken installs to recover without a full rebuild. Additionally, all pip invocations now use `python -m pip` to prevent issues with stale shebangs. This change addresses issue #229 and improves the overall reliability of the service installation process.
ruff now checks the whole tree in CI and in `make lint` instead of only
modules/ and tests/. scripts/ and the root-level scripts ship to users
too and were going unlinted; the wider scope flags nothing today, which
makes now the cheap time to widen it.
Coverage floor raised 35 to 50 against an actual 53.21%.
meshcore-cli was the only dependency with no version floor. It is still
required — meshcore_cli.next_cmd backs contact and channel operations
with no equivalent in the meshcore library — so it is pinned rather than
dropped, and the reason is recorded next to it.
Removes the dead [tool.pytest.ini_options] block: pytest.ini takes
precedence, so those settings were never in effect and could only drift
from the ones that are.
The clone command still read `git clone <repository-url>` — a
placeholder in the first command a new user runs. A fix existed on main
but had never reached dev.
Adds zero-hop neighbor discovery to the packet capture entry, noting it
is off by default and spends airtime, and points the Contributing
section at CONTRIBUTING.md and SECURITY.md rather than repeating a short
list inline. Links the solar conditions provenance record from
Acknowledgments.
solar-conditions-provenance.md was in the tree but absent from the
mkdocs nav and unlinked from anywhere, so it never appeared on the docs
site. Added under a Project section and linked from the solar command
entry, where a reader would look for it.
Removes the kg7qin PR integration log — an internal development record
that was excluded from the docs build but still shipped in the repo —
along with the now-dead exclude glob.
TODO was five months stale, claiming 36.86% coverage against today's
53.21% and tracking a coverage goal already exceeded. Dropped ~180 lines
of completed items that duplicate the changelog, refreshed the remaining
coverage targets from current data rather than copying March figures
forward (graph_trace_helper 2% to 65%, hacker_command to 100%,
solar_conditions 7% to 88% all came off the list), and closed out
referenced tickets that have since been resolved. Also removed a
dangling SESSION_RESUME.md reference, a .claude/ path, and an internal
LAN broker address.
BUGS now leads with outstanding issues instead of a v0.9.0 fix log, each
one re-checked against current code rather than re-dated. BUG-005 is
rewritten: the snapshot dashboard and retention work genuinely improved
it, so the old figure no longer held. Fixed a table header that declared
five columns for four-cell rows, and noted that two archive SHAs predate
a history rewrite and no longer resolve.
CONTRIBUTING covers dev setup, reproducing all six CI jobs locally, and
the house rules a newcomer would otherwise trip over: append-only
migrations, config changes needing config.ini.example updates, new docs
pages needing an mkdocs nav entry, the ruff pin, and PRs targeting dev.
SECURITY routes reports through GitHub private vulnerability reporting
rather than an email address, with a 10-day acknowledgement and 30-day
assessment window. Scope names what this codebase actually exposes, and
explicitly puts the MeshCore protocol, RF-layer attacks, and running the
viewer without a password out of scope.
Issue forms collect the details every radio bug report needs — version,
transport, hardware, install method — and the feature form asks up front
whether a proposal spends mesh airtime. Blank issues stay enabled since
Discussions is not turned on, so they are the only route for questions.
Dependabot covers GitHub Actions weekly, where a stale or compromised
action is a real supply-chain risk. pip and npm are grouped and monthly:
runtime deps are >= ranges, so version updates are mostly floor bumps,
and security fixes arrive through Dependabot alerts regardless.
StartLimitInterval and StartLimitBurst were set under [Service], where
systemd 230 and later ignore them — they moved to [Unit] in 2016. The
unit silently fell back to the system defaults of 5 starts in 10 s, and
because RestartSec=10 spaces attempts further apart than that window,
the limiter could never trip: a bot that could not reach its radio
restarted every 10 seconds indefinitely instead of stopping in failed
state.
Moved to [Unit] as StartLimitIntervalSec=60 / StartLimitBurst=3, the
values the file already declared. Applied to both the shipped unit and
the one generated for the .deb, which had drifted the same way.
Removes 26 console.log calls, including contacts.html dumping whole data
structures and sample contact records on every render. Variables and
callback parameters that existed only to feed those logs go with them
(deviceTypes, anyNewDevices, an availableEdges debug block, and a
realtime status handler whose body was nothing but a log), so no new
no-unused-vars warnings are introduced — the count drops 60 to 59.
console.warn and console.error are kept. One console.log survives in
realtime.html: the decoder key-count line is a one-time startup message
with real diagnostic value, and removing it would mean deleting two
counters and their increments inside a loop.
Also normalizes the mesh page's user-visible "Neighbours" strings to
match the project's American spelling. The option value stays
"neighbors", so evidence-mode filtering is untouched.
login.html loads DM Sans and Outfit from fonts.googleapis.com with the
font files on fonts.gstatic.com, but neither host was in style-src or
font-src, so the browser blocked both and the login page silently fell
back to system fonts with CSP violations in the console.
The Unreleased section held everything shipping in 1.0.0 — neighbor
discovery, the rebuilt dashboard, the meshcore 2.3.8 bump, two removals
and a deprecation — while the 1.0.0 entry below it described only part of
the release. Folded them into a single dated entry, merged the duplicate
Changed headings, and reordered subsections to Keep a Changelog order.
Normalizes British spellings in the prose to match the project's
American convention.
agessaman