Fixes from a Codex review of the previous commit, plus the shared settings the rebase onto dev showed were still missing.
Settings that misstated or changed what runs:
- sports never read [Sports_Command] api_timeout (documented in the example config); the new field did nothing. It now sets the ESPN timeout.
- The path-byte requirement help on test, multitest and path said "bytes per hop"; the gate checks the path's total size (at least 2, or exactly 3).
- feed's allow_private_urls showed false while it inherited true from [Feed_Manager]. Fields can now name an inherit_section whose value they show when unset.
- notify_on_prefix_bytes was an int field for a value the service reads as a list (2,3), so the form refused a valid setting.
- The greeter's form defaults differed from the fallbacks it uses when the keys are missing. Both now come from shared constants.
- The MQTT weather limits accepted values the service clamps (payload 256 to 1 MiB, staleness 5 s to 7 days, length 64 to 4000).
Shared settings: the wx card now carries the [Weather] temperature, wind and precipitation units and the three temperature_*_format templates (with previews through format_temperature_high_low), next to wx's own [Wx_Command] unit overrides, which default to "Same as [Weather]". Two fields with one key in different sections are told apart as key@Section in the save payload; a bare key still resolves when only one field has it. An enum's "" option, like a blank number, removes the key. Packet capture brokers gain username, password, topic prefix, JWT timings and include_decoded; blank inheriting values are left out of block saves, since the service reads them with getint/getboolean when present.
Previews:
- They use the bot's saved config: a blank test format falls back to [Keywords] test, multitest lays out paths per condense_paths, and MQTT applies passthrough_max_length.
- A quoted empty template ("") counts as blank, as the commands strip the quotes first.
- A str.format width or precision over 500 isn't rendered, so `{sender:100000000}` can't make the endpoint build a huge string.
- The page invalidates an in-flight preview as soon as the field changes, and shows the server's note about fallbacks instead of guessing it.
The endpoint inventory golden gains the preview route (401 for anonymous requests when a password is set).
The Plugins page now has a field for every setting the plugins read. I compared each plugin's settings_schema against config.ini.example and the keys the code reads; fourteen plugins had gaps (greeter, trace, stats, path, wx/gwx, rain, test, multitest, sports, worldcup, cmd, feed, the webhook, weather, packet capture, earthquake and prefix-collision services) and the World Cup service had no form at all. Webhook URLs, bot tokens and channel keys use the password field. A guard test needed four [Rain_Command] keys documented in config.ini.example.
Template fields get a { } button that opens a reference of the placeholders they take, with click-to-insert. Fields rendered by the piped engine (test's response_format, path's reply_prefix) also list the filters and conditionals. Every template field previews as you type: piped fields against a 2-byte, a 1-byte and a direct sample message, so a conditional's effect shows before saving, and the str.format fields (greeter, mesh info, multitest, MQTT weather) as their plugin renders them, including what it sends when the template has a mistake. The specs and renderers live in modules/template_reference.py; the preview endpoint only renders and never saves. The greeter's channel_greetings parser and multi-part split moved into module functions so the command and the preview share them.
Fixes the new fields would otherwise have exposed:
- Clearing a number field saved `key =`, which makes getint/getfloat raise and the plugin fail to load. A cleared number now removes the key.
- The weather service raised in __init__ when only some blitz_area_* corners were set. It now needs all four and logs a warning otherwise.
- A blank MQTT weather json_template saved an empty template, so every reply said no message had arrived. Blank now saves the default.
Conflict in wx_command.py: the refactor split _get_weather_for_location_sync into _locate_*, _noaa_forecast_body and _with_noaa_alerts. The fallback now checks coverage after _locate_* and when _noaa_forecast_body returns its fetch error. _openmeteo_weather takes no address_info: a city's label comes from city_to_lat_lon again, answered from the geocoding cache the first lookup filled.
[Weather] openmeteo_fallback (default false, only with weather_provider = noaa) answers wx from Open-Meteo when NWS reports a point as outside its coverage (HTTP 400/404 from /points), in gwx's format and labels and wx's units. Timeouts and server errors keep today's error. The point goes into the existing NWS no-coverage cache, so later requests skip NWS for 24 hours. wx <place> alerts there replies that alerts are not available.
gwx's per-point body moves into _weather_for_point so wx can reuse it without geocoding twice; gwx's output is unchanged.
Codex on 495df80e: choosing the help call by signature also dropped the TypeError retry for local plugins written as get_help_text(self, message=None), which used to fall back to generic help when the message call raised, and inspect.signature itself can raise TypeError on odd callables and then take the wrong branch. help_text_for now decides by where get_help_text is defined: bundled commands (defined under modules.) are called once with the message, so an error from their body propagates as it did before 7aba432b; anything else gets exactly the old call-then-retry. Tests pin both: a bundled body's TypeError runs once, a local plugin's is retried without the message. joke and rain stay the accepted residual (no retry for a TypeError inside their help bodies).
Codex also found two inaccuracies in b569ed5d's docs: ChannelBridgeBase subclasses must also supply _open_http_session and _init_queues, and an empty [Keywords] format means no reply.
Local plugin authors had no way to learn about enabled_attr, admin_only, keywords_format_key, the get_help_text(self, message=None) signature, translated_or, run_periodic, _cancel_tasks, _subscribe/_unsubscribe_all or ChannelBridgeBase short of reading the base classes. A new "Base class hooks" section lists what each does, with the facts checked against base_command.py, base_service.py and channel_bridge_utils.py.
Each attempt waited only for its own tag, so a reply that arrived after its attempt's timeout was dropped once the retry went out. On my test mesh a three-hop trace takes about nine seconds (each repeater waits its random TX delay before forwarding), against a 2.5 s timeout, so every trace failed even though the first attempt's full reply came back.
run_trace now subscribes to TRACE_DATA once, before the first send, and takes a reply to any of its tags until the last attempt's timeout runs out. A reply that lands during the retry delay ends the trace without sending again, and subscribing before the send also closes the window where a fast reply could arrive before wait_for_event was listening.
The timeout defaults go from 1.0 s base and 0.5 s per hop to 2.0 s and 1.5 s, matching the 1.5 s the trace command and its settings schema already used. Configs that set them keep their values.
wx and gwx name the place only when it adds information: a city outside default_state or default_country, or a position. always_show_location (default false) names every place a lookup finds instead, including cities in the bot's own state and ZIP codes.
For a ZIP code, gwx now does a reverse lookup when the option is on (it skipped that since the name was never shown), and wx reuses the reverse lookup it already did for a companion position. A failed lookup leaves the reply unnamed rather than repeating the ZIP code. The prefix comes out of the message budget as before. The option is in the wx settings schema for the web viewer, the example config, the command reference and the changelog.
- A scheduled {cmd:gwx ...} or {cmd:wx ...} with a forecast and an alert slept out the TX pacing between them during capture; with bot_tx_rate_limit_seconds = 30 that passed the 30 s render timeout and the whole reply was lost. Pacing is skipped when the message has a capture_sink.
- gwx tomorrow ignored its byte budget (a Russian Reykjavik forecast was 99 bytes against 89). It now drops the precipitation amount, gusts, wind, the precipitation chance and the description, in that order. The Open-Meteo goldens give tomorrow runs a roomy budget, since their key-name translator makes the text much longer than real; trimming is tested with real catalogs in test_gwx_tomorrow_budget.
- Custom default sources (bare wx/gwx) still returned True after a refused send.
- An option without a location (wx hourly, gwx tomorrow, wx 5d) showed usage, since the fallback location was resolved before the option was parsed; the option now applies to the fallback, custom default sources included. The docs say how far each provider's forecast reaches.
- wx read its unit settings through get_config_value, whose [Weather] fallback logged a notice to move them into [Wx_Command] on every lookup (14 per reply); _unit_setting reads [Wx_Command], then [Weather], without it.
- NOAA forecasts that mention thunderstorms after sun ("Sunny then Slight Chance Showers And Thunderstorms") still got ☀️; thunderstorms now outrank sun.
- The catalog tests passed with keys removed, since a missing key differs from English; they now require each key and every wind point, with en-GB and Polish allowed to use the English letters.
A space bunny review of this branch's commits found:
- gwx's high-wind warning converted the speed to mph for every catalog, so on a km/h bot it read 49 next to a reply saying 80, and Polish (whose template said km/h) regressed in the merge. The threshold stays 30 mph; the warning shows the speed in the configured unit with a {unit} placeholder in every catalog.
- tomorrow_not_available, tomorrow_error, multiday_not_available and multiday_error were still English in seven catalogs; they are translated, and the catalog test now fails on any weather reply left in English.
- wx kept NOAA's English wind letters and English weekday letters while gwx's are translated, so one German bot showed NE and NO for the same wind. wx now translates both through the catalog (commands.wx.day_abbrev added); English output is byte-for-byte unchanged, which the goldens show (their stub translator gives English for these short labels, via tests.helpers.key_name_translate).
- Polish feels_like was ungrammatical (odczuwalna -> odczuwalnie), Russian's used a nonstandard abbreviation, and Russian gwx's MQTT subscriber error lacked wx's hint.
- The help text (commands.wx.description/usage, commands.gwx.usage) described wx as ZIP-only; it now lists cities, coordinates and the options, under 130 bytes in every language.
- The generated default config, the docs' default_state wording, _fit_tomorrow_parts' docstring and the CHANGELOG's stats claim were inaccurate.
The command reference described wx as ZIP-only and gwx as a list of fields. It now gives both commands' accepted locations, options (tomorrow, hourly, Nd, alerts), the order they fall back through when no location is given, the unit settings, when the reply names the place, what custom WXSIM and MQTT sources answer, and that both work in scheduled messages. The config example's default_state/default_country comments say how they affect the place name, the no-location comment lists default_city, and the weather service page lists the wind units that actually exist.
gwx was not render_safe, so {cmd:gwx ...} expanded to nothing, while {cmd:wx ...} already ran gwx through delegation when weather_provider = openmeteo. gwx sends every reply, including the two-part warning, through send_response, which is what render_safe requires. Tests render gwx (plain and two-part replies) through the real CommandManager capture and check nothing is transmitted.
Review follow-up. With * listed, an uncorrelated message is still answered globally when no scoped packet was heard around it; the radio's default scope does apply to sends the bot leaves unscoped; [Scheduled_Messages] needs the leading #; and the outbound log lines are quoted as they are actually printed, including that the global one is DEBUG.
Review follow-up. The scope read could raise (a bare '%' under interpolation) after the 'sent' row and mesh cooldown were reserved, and the rollback never ran, so the log showed a warning that never went on the air. The scope and body are now prepared before anything is reserved, and a failure is recorded as a failed attempt. [Region_Warnings] flood_scope is read raw and validated, because set_flood_scope's failure path sends anyway at whatever scope the radio holds. Dry runs now log the truncated body the live send would use.
Channel-delivered region warnings were hard-coded to scope='*', on the belief that a sender with no region could not hear a scoped message. That is not how MeshCore works: the companion firmware's filterRecvFloodPacket drops nothing, so a client accepts a regional flood whatever region it has set itself, and only repeaters decide how far it goes. The override only meant that a bot configured with outgoing_flood_scope_override flooded the whole mesh to tell someone not to, which is exactly what #306 reports (the log shows the override 'was not applied (explicit scope="*")').
The warning now resolves its scope like any other proactive channel send: [Region_Warnings] flood_scope, then flood_scope.<channel>, then outgoing_flood_scope_override, then global. Its body is sized with channel_body_budget so the regional overhead is accounted for, and the web viewer's byte limit reserves that overhead whenever a regional scope could apply. flood_scope = * restores the old behavior.
Region names have to be given to the bot before it can answer inside a sender's region, because the transport code is an HMAC over each packet's payload and cannot be turned back into a name. That was only spelled out in config.ini.example and a dense paragraph in configuration.md, which is how #296 came about. The new page covers how a scope works on the air, what each flood_scopes setting does to replies, the precedence for the bot's own messages, the Radio page, and the log lines that show whether a reply went out scoped.
An oversized webhook or channel body was handed to the firmware whole. The device dropped it without an ack, retries stalled the transport, and the radio looked dead. Split to the frame budget, keep links intact, and tag multi-part sends so they can be reassembled.
Reserve the wx location prefix from that same budget, and let NOAA alert polling be turned off without disabling the rest of the weather service.
Fold the Unreleased section into 1.1.0 and date it. The 1.1.0 heading was
written on 2026-09-13 but never tagged, so everything merged since belongs in
it; the section also carried two separate "### Added" headings from earlier
merges, now consolidated. All 79 entries preserved.
Extend the v1.0.0 -> v1.1.0 upgrade guide with the meshcore 2.3.14 requirement,
migration 24, and the configuration and behavior changes that landed after the
heading was first written.
The report used /n and got two literal characters. The escape works the same in
every keyword path — I checked the test command's template engine too — so the
defect is that the instructions never said "backslash" and used `test` as their
example, which is the one keyword a built-in command owns. That command renders
through a different engine and silently drops the mesh-info placeholders listed
in the same block.
Also documents the configparser continuation form, which needs no escape, and
corrects two packet-capture examples that still showed a naive local timestamp
where the bot has published UTC with a Z suffix since v1.0.0 (#276).
* feat(website): Include local commands when generating website
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* feat(website): Disabled commands are excluded from generated HTML
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* Updated CHANGELOG.md
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* fix(website): resolve enablement through the bot's own config helpers
The generated page is meant to list what the bot actually answers, so it
has to read the config the same way the bot does. Three places where it
did not:
The local `config.ini` overlay was never read. `read_config` parsed only
the base file, while the bot overlays `<local_dir_path>/config.ini` on
top of it, and that overlay is exactly where the web viewer saves a local
plugin's settings. A local command disabled from the settings page still
showed up on the site.
`is_command_enabled` derived the section name and the legacy `enabled`
spellings by hand, duplicating `BaseCommand._derive_config_section_name`
and guessing that a legacy key lives in the command's own section. Half
of them do not: `[Jokes] joke_enabled` disables `Joke_Command`. It now
calls `command_section_name` and `read_enabled`, which share the alias
table with the runtime and the settings UI.
That table was missing the same-section `[Joke_Command] joke_enabled` and
`[DadJoke_Command] dadjoke_enabled` spellings, which both commands accept
through their own fallback. The settings page read only the `[Jokes]`
form, so a bot disabled the same-section way displayed as enabled.
Also fold the duplicated local-commands-dir resolution into
`resolve_local_commands_dir`, take `bot_root` as a `MinimalBot` argument
instead of assigning it post-construction twice, and restore the `hidden`
attribute check that `generate_samples` lost when it moved to
`filter_commands`.
---------
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
Co-authored-by: agessaman <adam@gessaman.com>
* feat(command): Add contact command
The contact command allows the user to ask the bot to provide its
public key as a clickable contact. This allows the user to then DM the
bot outside of a channel.
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* fix(contact): harden matching and self_info handling, add tests and docs
Review follow-ups on the contact command:
- Match through _cleaned_content_matches so a non-matching message keeps its
original content. The raw cleanup_message_for_matching call rewrote
message.content in place during the keyword scan, which is the #267
regression every other command already avoids.
- Match against self.keywords so a configured alias works.
- Read self_info defensively (dict or object, may be absent) and validate the
public key instead of sending a literal "<None:1:None>" over the air.
- Drop the hardcoded '!' prefix strip in execute; matching already normalizes
the content, and prefixes are configurable.
- Forward skip_channel_check to the base can_execute.
- Remove copy-paste docstring leftovers from the roll command.
- Move [Contact_Command] into the alphabetical run in config.ini.example and
fix the "chanels" typo.
- Add tests, a docs/command-reference.md entry, and flesh out the changelog.
---------
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
Co-authored-by: agessaman <adam@gessaman.com>
Radio commands were serialized per method call, so a composite command held the lock for everything it awaited. A DM's send_msg_with_retry kept every other command waiting through all of its ACK timeouts (up to ~36 s with three attempts), stalling channel replies, other DMs, scheduled sends, and automatic message fetching. req_regions_sync did the same for each neighbor scope reply.
Serialize at the frame instead: wrap CommandHandler.send() on the handler instance, which is where every meshcore command writes its frame and waits for the radio's immediate reply. There is still at most one in-flight companion frame, paced as before, and library methods that call self.send are covered without a proxy. Waits for ACKs and remote responses now run outside the lock.
Add MeshCoreBot.radio_session() for short sequences that must not interleave, and use it for region-scoped channel sends so set scope, send, and restore stay together. The per-call lock never guaranteed that: a DM retry loop queued between set_flood_scope and send_chan_msg sent its flood attempts under the channel's scope.
generate_website.py can layer custom CSS on top of the built-in style chosen with --style. --embed-css appends a local file to the page's <style> block so the page stays a single file, and --link-css adds a stylesheet link after it, so both override built-in rules of equal specificity. The flags can be combined (the linked stylesheet loads last), apply to --sample pages, and an unreadable --embed-css file stops generation with an error. The website docs gain a custom CSS guide and a reference for the generated page's structure, classes, and theme variables.
Region scopes had to be hand-edited in config.ini, and the radio's own default
region could not be set from the bot at all. Both now live on the Radio page.
Node Settings gains a Default Region Scope field beside the path hash size,
which is the firmware's own setting — NodePrefs.default_scope_name and
default_scope_key, the region the radio falls back to for any send the bot does
not scope itself. Firmware that has no such command is reported as not having
answered rather than shown as an empty field, and a stored key that is not the
stored name's hash is flagged, because the radio routes by the key and the name
beside it is only a label. A build-flag default stores the bare name while
hashing the '#' form, so the comparison normalizes first.
Clearing it does not go through the meshcore library, because none of its reset
paths can: set_default_flood_scope(None) raises on len(None), "" earns
ILLEGAL_ARG from the firmware, and "*" only works because the frame is padded by
character count rather than by the name it wrote, landing one byte short of the
length the firmware reads as "a scope follows". The firmware's own contract for
clearing is a bare CMD_SET_DEFAULT_FLOOD_SCOPE, so that is what goes out. The
same padding bug is why a device scope name must be ASCII: a multi-byte
character displaces the transport key out of its field.
A separate Region Scopes card covers the bot's side — [Channels] flood_scopes as
an explicit choice between "reply whatever the scope" and an allowlist, and
outgoing_flood_scope_override. It writes config.ini, queues a hot reload, then
polls that reload and reports what the bot did with it, including saying plainly
when nothing picked it up. Per-channel flood_scope.<channel> entries are listed
read-only, since they are the reason a channel can ignore the default.
The two settings interact in a way worth stating on the page: once the bot sends
a scoped message it restores with set_flood_scope("*"), which leaves the radio in
forced-unscoped mode, so the device default stops applying until the bot scopes
another send.
Also from building it:
- flood_scopes left in [Bot] is surfaced rather than hidden. CommandManager
still honours it when [Channels] is empty, so a page that ignored it would
show "replies to every scope" while the bot enforced an allowlist — and
saving "off" hands the allowlist straight back to [Bot], which the banner now
says instead of claiming the old key stops mattering.
- outgoing_flood_scope_override = none is read as global flood on the send path,
as it already was everywhere else. send_channel_message tested the raw value
against a fixed tuple, so the lowercase spelling became the region "#none".
- Scope normalization moved to modules/flood_scope.py so the viewer, a separate
process, can validate a typed name without importing the bot's command
machinery. CommandManager delegates to it.
- Dark mode named only .border, so a .border-start divider drew Bootstrap's
light #dee2e6 onto a dark card. All four directional utilities now match.
The known-contact gate for DM delivery returns before anything is recorded, so
a bot that keeps no contacts dropped every warning and wrote no rows: the page
showed "No warnings decided yet" indefinitely beside a status card saying it
was sending, and nothing distinguished a clean mesh from one where every
warning was being discarded. Only a debug log said otherwise.
Withheld warnings are now counted per local day in bot_metadata — a counter
rather than event rows, because this fires ahead of the cooldowns that would
rate-limit rows, so logging each one would bury the decisions worth reading.
The status card shows it, and the empty log explains itself and points at
channel delivery.
Also from the verification pass:
- The percent test built its value with config.set and a doubled %, which
set() is the one input that cannot produce the bug. It now uses read_string
with a bare %, and I checked it fails against raw=False by substituting
DEFAULT_MESSAGE — the actual failure mode.
- docs still said the bot identifies itself by public key, contradicting the
section forty lines above. On this path it is name-only, and that means the
self-exemption is spoofable; the doc says so.
- delivered_today folded dry runs in with real sends, so a morning's preview
read as afternoon transmissions. previewed_today is now separate and the
figure follows the current mode.
- TRIM(MIN(channel)) so a whitespace-padded historical row cannot become the
display name for a merged channel.
**A `global` verdict now requires RF correlated to the message.** It was also
reachable through `_is_confirmed_global_flood`'s argument-from-absence route
("no scope-eligible packet in the window, therefore unscoped"), so a row that
literally said TC_FLOOD, or a scoped ADVERT that the GRP_TXT filter excluded,
came back as "no region code". That inference is fine for deciding whether a
`*` in flood_scopes authorizes a reply; it is not fine for accusing someone of
a misconfiguration. Channel messages still correlate through the payload match
(#255), so the ordinary case is unaffected.
**A `%` in the warning message no longer wedges config reload.** `_get` read
without `raw=True`, so configparser's interpolation raised, the error was
swallowed, and the bot transmitted the default wording instead. Worse, the
bot's own `_validate_config_snapshot` iterates `config.items(section)` and
would reject every hot reload until the file was hand-edited. The save endpoint
now rejects `%` outright and caps the message at 500 characters, and the read
is raw so a hand-edited value is still honored.
**One unreachable sender no longer eats the daily cap forever.** The cap counted
failed attempts but the per-sender cooldown did not, so a node the radio cannot
reach was retried every `min_unscoped_messages` messages indefinitely and no
real offender was ever warned. Both count attempts now. The regression test
fails with three sends against the old filter.
**The sender is a display name, not an identity.** MeshCore's CHANNEL_MSG_RECV
carries no public key, so `sender_pubkey` was always empty on this path and the
only identity is a prefix anyone with the channel key can forge. DM delivery
now requires a contact the radio already holds, which bounds the bot to nodes
it knows and stops failed sends spending cap slots. Two tests asserted the
opposite because the fixture supplied a pubkey the radio never sends; they now
run with what the call site actually passes, and the docs no longer claim
pubkey identity.
Also: a negative `max_warnings_per_day` fell back instead of clamping to the
"unlimited" sentinel; tallies group case-insensitively so a `#` or case change
does not split a channel; retention uses the same clock the rows are written
in; the status figure shows delivered with attempts beside it, rather than a
count of four next to "last warning: none yet"; and the channel bars are scaled
over classified traffic so their width equals the percentage printed beside
them (33.7% was drawn at 28.7%).
A channel message with no "Name: " prefix falls back to a stand-in name.
That is not a node: every such message shares the identity, so they would
accumulate a run together and then get a DM addressed to a contact that
does not exist. The hook now passes no sender for those, so they are still
counted but can never earn a warning. The literal is a named constant and a
test pins the hook's guard to it.
Also folds the channel body-budget formula into models.channel_body_limit
rather than letting the web viewer keep a fourth copy of
"max(130, 160 - len(name) - 2)"; BaseCommand and CommandManager now call it
too, and 158 becomes models.DM_BODY_LIMIT.
Closes#279.
A MeshCore client with no region configured sends every channel message as
a plain FLOOD, which every repeater on the mesh rebroadcasts. This adds two
things: free observation of how much of that the bot hears, and an opt-in
warning to the senders.
Observation classifies each channel message as scoped, global or unknown and
tallies it per channel per local day. It costs one upsert and no airtime, and
it is what lets an operator see the size of the problem before deciding to
spend airtime on it. The classification runs ahead of the flood_scopes
allowlist, because an unscoped message is exactly what that allowlist drops.
Warnings only ever fire on positive RF evidence of an unscoped FLOOD. Absence
of correlation is not proof that a sender omitted a region, so it classifies
as unknown and stays quiet. They are off by default, start in dry run, and are
fenced by min_unscoped_messages, a per-sender cooldown, a mesh-wide cooldown
and a daily cap on attempts. Dry run consumes the same budget it previews, so
the log is what going live would put on the air, not an upper bound. All three
limits read from the database, so a restart cannot release a burst.
Channel-delivered warnings go out at global scope on purpose: the recipient is
by definition outside any region the bot replies under.
New Settings -> Region Warnings page in the web viewer covers all of it. Its
dark-mode striped rows exposed a pre-existing base.html bug where Bootstrap's
light-theme text color survived on a dark row background (~1.3:1); fixed there
for every table in the app.
Add shlink support and rework response templates onto a state machine.
Merged via this branch rather than the PR head: the fork is org-owned, so
"allow edits by maintainers" does not grant push access and the review fixes
could not be pushed to #254 directly. Roger Fedor's four commits are included
with authorship intact, followed by two commits addressing the review.
Security: a shlink deployment with short_url_website unset POSTed the operator's
API key to v.gd. _normalize_base falls back to the public default and the shlink
branch guarded only on a missing key, never on a missing base. shlink now requires
an explicit base and refuses a v.gd/is.gd host outright rather than sending an
X-Api-Key there.
Correctness: _shorten_url_with_gd lost its response.ok guard when the backends were
split out, so a 502 whose body starts with http was returned as the short URL and
transmitted over RF. Restored, with a warning. The regression test that should have
caught this passed only because it left mock_resp.text as a MagicMock; it now uses
a realistic proxy maintenance body.
_shorten_url_with_shlink never checked the status either. Shlink reports failures as
RFC 7807 problem details, which parse as JSON and simply lack shortUrl, so a bad API
key was indistinguishable from an unshortenable URL at DEBUG. It now warns on a
non-OK status, a non-JSON body, and a missing shortUrl. Dropped the shortUrlSlug
fallback: that is a request field, not a response field, and returning it puts a
bare slug where a link belongs.
Timeouts and connection errors are back on their own DEBUG handler. They had fallen
through to the broad handler, whose level rose to ERROR in the same diff, so a
routine intermittent uplink logged "Unexpected error shortening URL" on every reply.
Parser: _GREEDY_ARG_FILTERS was tested before the quoted-argument branch, so
prefix_if_nonempty, the one filter already in shipped configs, could not use the
quoted syntax this PR adds. {path_distance|prefix_if_nonempty:"Dist {sender}: "}
rendered raw template text. A quote immediately after the ':' now selects the quoted
grammar; anything else stays greedy, so config.ini.example's
`prefix_if_nonempty: | Path Dist: ` keeps its pipe and its whitespace.
Blocking HTTP on the event loop: the path command rendered its reply prefix inline
from async code, so a slow shortener stalled radio RX, MQTT and every other handler
for the full 5s timeout. Added format_piped_template_async and switched the path
command to it. The test command still renders synchronously through the sync
check_keywords dispatcher; making that async is a separate change, so the filter now
warns once when it runs on the loop.
Naming: one operation should not have two names in an operator-facing DSL. shorten
and shorten_url are aliases in both response templates and feed formats, and
if_nonempty is canonical with if_notempty as an alias, so a filter chain copied
between a feed format and a response_format works either way.
Also: reduced _build_create_shlink_url to the one parameter it uses and dropped its
dead query/startswith lines, renamed the shlink POST callable from `get`, documented
the config argument on format_piped_template, stopped gating the render trace on an
unrelated parameter and logging field values (sender IDs, user phrases) with it, and
moved the changelog entry from Fixed to Added and Changed.
This commit addresses several issues related to MQTT connections. It prevents MQTT brokers from entering a reconnect storm by ensuring that the packet-capture watchdog only triggers a reconnect when necessary, avoiding duplicate CONNACKs and spurious disconnects. Additionally, it ensures that renewed MQTT auth tokens are properly enforced by implementing a clean reconnect process.
New configuration options are introduced: `mqttN_keepalive` for setting the PINGREQ interval per broker, and `mqttN_jwt_reconnect_on_renew` to control reconnect behavior after token renewal. Documentation has been updated to reflect these changes and to clarify the importance of unique client IDs for brokers in the same cluster.
{firstlast_distance|prefix_if_nonempty: | F/L Dist: } prints "| F/L Dist: N/A"
on a direct message: the distance helpers return the literal "N/A" when there
is no path to measure, prefix_if_nonempty only asks whether the value is
non-empty, and "N/A" is. Suppressing that needed a gate, and pathbytes_min was
the only one available—so it was being used as a stand-in for "did this
message take any hops", which is not what it asks. It asks how the path is
encoded, so pathbytes_min:2 also discards a one-byte multi-hop path whose
distance is real and measurable.
hops_min:N asks about the route itself. hops_min:1 drops a clause on a direct
message and nothing else:
hops_min:1 pathbytes_min:2
direct cleared cleared
1-byte 2 hops kept cleared <- the difference
2-byte 2 hops kept kept
unknown cleared cleared
An unknown hop count clears the value: a gate that cannot confirm the route
should suppress rather than guess, matching pathbytes_min.
The hop-count logic moves to utils.message_hop_count so the filter and
BaseCommand.get_hops_display_values share one implementation instead of two.
It returns None for unknown rather than 0, since a gate must not read
"cannot tell" as "direct".
Adds the 16-char MeshCore packet identity hash to the three formatters that
render user-facing templates: [Keywords] responses, the test command's
response_format, and the path command's reply_prefix. It is read only from
routing_info, which is attached solely from an RF packet correlated to the
message, so a hash from an unrelated transmission is never shown. Missing,
empty, and the 0000000000000000 error sentinel all render empty.
get_packet_hash_placeholder returns "" rather than a False sentinel. Empty
gives byte-identical output on every path—prefix_if_nonempty already tests
falsiness and str.format renders "" as nothing—whereas False would print
the literal string "False" into a mesh reply from any consumer of
get_standard_placeholder_fields that formatted it directly.
TestCommand.format_response now builds on get_standard_placeholder_fields
instead of duplicating it. That also honors the extra argument its docstring
already promised but silently ignored, and picks up the path-string hop
fallback, so {hops} reads "0" rather than "?" for a message whose hops and
routing_info are both unset but whose path says "Direct (0 hops)".
Documents the placeholder in config.ini.example, the default config that
core.py writes on a fresh install, the README and docs/path-command-config.md.
The generated config was also missing {elapsed}; added that too.
Correctness:
- {path_distance} was always blank in production. The resolution code that
builds repeater_info dropped latitude/longitude in every branch, so the
calculator could never find a coordinate. My tests passed hand-built
dicts straight to the calculator and never exercised the builder, which
is why they stayed green. Coordinates are now carried through all four
construction sites, and the new tests drive _lookup_repeater_names via
its lookup_func hook so the real builder runs.
- The #80 route guard was defeated two ways in the channel handler. When
the RF data was an uncorrelated fallback, control fell through to the
raw-hex and routing_info fallbacks below, which took the route from the
unrelated packet anyway; the guard had actually made that path
reachable. message.routing_info was also assigned unconditionally, and
the path command reads it. "Not attributable" is now a terminal branch
and the routing_info hand-off checks provenance.
- Same fix was incomplete for DMs: routing_info was captured and turned
into path_info before the provenance check ran, so the later check only
declined to overwrite an already-wrong value. Guarded at the source.
- Rendering could transmit for real. Capture only intercepts
send_response, but advert calls send_advert() directly and
send_response_chunked never checked capture_sink. Chunked sends are now
captured, and rendering is opt-in via BaseCommand.render_safe (default
False) instead of a denylist that cannot be complete. This also closes
the DM-only leak: schedule is not marked safe, so {cmd:schedule} can no
longer broadcast configuration to a channel.
- Multi-part rendered output was rejoined into one oversized send.
Scheduled messages are now split to the RF body budget and sent through
send_channel_messages_chunked, on character boundaries so multi-byte
text is not corrupted.
- _last_path_distance_km is instance state that was only set on success,
so an invalid path request could show the previous request's distance.
Reset at the start of every execute().
- Stale-contact retries were only counted on non-OK results, so timeouts
and exceptions left a contact eligible forever and could recreate the
storm. All failed attempts count now.
Web viewer:
- A failed config reload was reported as a successful save. The API and
UI now distinguish "saved and active" from "saved, restart needed".
- Preview count was unbounded; clamped to 1-20.
- update_ini_values is a read-modify-replace with no locking, so two
concurrent viewer saves could lose one. Serialised behind a lock.
Adds a Schedule page that lists every [Scheduled_Messages] entry with its
next run time and supports add, edit and delete. Writes go to config.ini
and queue a config reload, so schedules change without restarting the
bot, which was the actual request.
It edits the config section the bot already reads rather than
introducing a database table. reload_config() already re-runs
setup_scheduled_messages() with rollback, so there is nothing to keep in
sync and the schedule command lists exactly what the page shows.
Validation runs through the same parsers the scheduler uses, so the UI
cannot accept a schedule the bot would later reject. The builder
composes cron from plain-language options and previews the next five
runs; entries the bot cannot run are listed as "Not scheduled" with the
reason instead of being hidden, since a typo that silences a message is
what an operator most needs to see. The 15-minute floor for {cmd:...}
messages is enforced at save time too.
Also relabels the radio Disconnect button to "Stop Bot" behind a
confirmation (#240). It was never a radio-only disconnect: the main loop
runs while self.connected is true, so disconnecting exits the process.
That surprised an operator running under tmux with nothing to restart
it. disconnect_radio()'s docstring now says so as well.
Two documentation gaps behind open questions.
The [External_Data] repeater_prefix_api_url comment claimed that leaving
it empty "disables prefix command functionality" (#70). That is not what
happens: the prefix command answers from the bot's own database of heard
repeaters, and the API only augments it with node counts from a wider
dataset. The wrong comment is a plausible reason the question was asked
at all. Corrected, and the JSON contract is now documented in the command
reference for anyone serving their own endpoint, since map.w0z.is is
defunct and has no drop-in replacement.
Also documented the pipx path (#222). The installer already grew a
virtualenv in July, which covered the PEP 668 half of that report, but
the unanswered part was where config.ini, the database and local/ live
under a pipx install. They are all resolved relative to the directory
containing config.ini, which means a bare `meshcore-bot` picks up
whatever is in the current directory — so the guidance is to pass an
absolute --config. Both console scripts are already smoke-tested from an
installed wheel in CI, so this path is supported rather than incidental.
Two guards on command placeholders in scheduled messages, neither
configurable, because both exist to protect a shared medium:
A 15-minute floor. A schedule containing {cmd:...} that fires more often
is rejected at startup with an error rather than quietly running slower.
The interval is sampled and measured by the tightest gap between
firings, so "0,1 * * * *" is correctly treated as every 60 seconds and
not as hourly. Schedules without a command placeholder are unaffected.
The command's own cooldown_seconds now applies to a render. Scheduling
is not a way around the rate a command was configured to run at. The
execution is recorded before the command runs, matching execute_commands,
so a slow or failing render cannot be retried straight past the cooldown.
Also fills documentation gaps from the preceding commits:
- --install-extras was only in the script's own --help; documented in
service-installation.md (including alongside --update-venv) and
upgrade.md.
- weather-service.md documented weather_alarm's once-a-day scheduling
with no route to more than one forecast a day, which is exactly what
people go there looking for. It now points at {cmd:wx ...} and notes
that sunrise/sunset still belong to weather_alarm.
A scheduled message can now embed the reply of any bot command:
[Scheduled_Messages]
0 6,12,18 * * * = Public:{cmd:wx Seattle}
The schedule side was never the missing piece — [Scheduled_Messages]
keys have been 5-field cron with @presets for a while, so multiple
times, intervals and @hourly were all already expressible. What was
missing was any way to get a service's output into a scheduled message:
placeholders were limited to mesh info (contact counts). That gap is
why services were growing their own schedule parsers.
CommandManager.render_command_output runs a command for its text alone.
MeshMessage.capture_sink makes that safe: when set, send_response
collects the reply and transmits nothing, checked before the
_last_response bookkeeping so a background render cannot overwrite the
response captured for a real user's command.
Degrades quietly in every failure mode — unknown, disabled, admin-only,
non-renderable, timed out or silent commands expand to nothing, so raw
{cmd:...} text is never put on the air, and a message left empty is not
sent at all. Command output is not re-scanned, so a reply containing
{cmd:...} cannot recurse. Commands that transmit directly rather than
returning text (announcements) are refused outright, since rendering
them would broadcast for real.
Bounded by [Bot] scheduled_command_timeout_seconds (default 30).
Total path distance was already a solved problem here: the test command
computes it and exposes {path_distance} through the piped-template
engine, with pipe filters and docs. The path command just had no way to
reach it, since reply_prefix went through plain str.format.
Rather than add a second distance implementation, this reuses what
exists. BaseCommand.get_standard_placeholder_fields now returns the
shared placeholder set so subclasses can extend it, the path command
renders reply_prefix through format_piped_template with an added
{path_distance}, and the sum itself uses utils.calculate_distance over
the nodes the path command has already resolved.
Distance covers sender -> each hop -> bot and is deliberately blank when
the chain cannot be measured end to end (unresolved hop, prefix
collision, missing or 0,0 coordinates, unknown sender, no configured
bot position), so a partial sum is never reported as the real distance.
Because the prefix now supports pipe filters, an empty value takes its
label with it: {path_distance|prefix_if_nonempty:📏 }
Supersedes #198, which added a parallel haversine, a show_path_distance
flag, and a distance_traveled key across 10 locales.
Add configurable PacketCapture observer name
Adds an optional [PacketCapture] observer_name used as the MQTT `origin`
for packet and status reporting, letting the observer identity differ
from the connected MeshCore device name. Unset keeps the previous
behavior. The public key backing origin_id, authentication, and topic
resolution is unchanged.
CHANGELOG conflict resolved by keeping both Unreleased entries.
agessaman