Commit Graph
709 Commits
Author SHA1 Message Date
agessaman 4312d8a39c fix(packet_capture): default waev.app brokers to a JWT TTL they accept (#248)
waev.app rejects a token whose exp is more than an hour past its iat, so the
project-wide 24-hour default could never authenticate there — the operator saw
only a bare auth failure with no hint that the lifetime was the problem, while
the same public key worked from meshcoretomqtt.

waev.app hosts now resolve to a 3600s TTL with renewal at 3500s when no value
was configured for them, per broker or globally; a configured value still wins,
and the substitution is logged. Matching is the apex and its subdomains only, so
a lookalike host like waev.app.example.com does not qualify.
2026-09-20 10:58:09 -07:00
agessaman b5e61306b2 Merge remote-tracking branch 'origin/dev' into feat/region-code-warnings
# Conflicts:
#	CHANGELOG.md
2026-09-20 10:43:24 -07:00
Gerard Hickeyandagessaman af05c2ef7f feat(website): Include local commands in generated HTML (#287)
* feat(website): Include local commands when generating website

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>

* feat(website): Disabled commands are excluded from generated HTML

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>

* Updated CHANGELOG.md

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>

* fix(website): resolve enablement through the bot's own config helpers

The generated page is meant to list what the bot actually answers, so it
has to read the config the same way the bot does. Three places where it
did not:

The local `config.ini` overlay was never read. `read_config` parsed only
the base file, while the bot overlays `<local_dir_path>/config.ini` on
top of it, and that overlay is exactly where the web viewer saves a local
plugin's settings. A local command disabled from the settings page still
showed up on the site.

`is_command_enabled` derived the section name and the legacy `enabled`
spellings by hand, duplicating `BaseCommand._derive_config_section_name`
and guessing that a legacy key lives in the command's own section. Half
of them do not: `[Jokes] joke_enabled` disables `Joke_Command`. It now
calls `command_section_name` and `read_enabled`, which share the alias
table with the runtime and the settings UI.

That table was missing the same-section `[Joke_Command] joke_enabled` and
`[DadJoke_Command] dadjoke_enabled` spellings, which both commands accept
through their own fallback. The settings page read only the `[Jokes]`
form, so a bot disabled the same-section way displayed as enabled.

Also fold the duplicated local-commands-dir resolution into
`resolve_local_commands_dir`, take `bot_root` as a `MinimalBot` argument
instead of assigning it post-construction twice, and restore the `hidden`
attribute check that `generate_samples` lost when it moved to
`filter_commands`.

---------

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
Co-authored-by: agessaman <adam@gessaman.com>
2026-09-19 22:33:54 -07:00
Piotr Synowiec 258c9680b2 feat(hello): optionally address the sender in the reply (#292)
Adds a `[Hello_Command] include_sender` setting (default off) that names the user the hello reply is answering, so a busy channel can tell whose greeting the bot picked up.

The mention takes the place of the random human descriptor, keeping the translated sentence intact, and is dropped when it would push the reply past the channel body budget. DMs are unaffected, and channel sender names are sanitized before being echoed.
2026-09-19 21:28:19 -07:00
Gerard Hickeyandagessaman f9b1cbddb8 feat(command): Add contact command (#293)
* feat(command): Add contact command

The contact command allows the user to ask the bot to provide its
public key as a clickable contact. This allows the user to then DM the
bot outside of a channel.

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>

* fix(contact): harden matching and self_info handling, add tests and docs

Review follow-ups on the contact command:

- Match through _cleaned_content_matches so a non-matching message keeps its
  original content. The raw cleanup_message_for_matching call rewrote
  message.content in place during the keyword scan, which is the #267
  regression every other command already avoids.
- Match against self.keywords so a configured alias works.
- Read self_info defensively (dict or object, may be absent) and validate the
  public key instead of sending a literal "<None:1:None>" over the air.
- Drop the hardcoded '!' prefix strip in execute; matching already normalizes
  the content, and prefixes are configurable.
- Forward skip_channel_check to the base can_execute.
- Remove copy-paste docstring leftovers from the roll command.
- Move [Contact_Command] into the alphabetical run in config.ini.example and
  fix the "chanels" typo.
- Add tests, a docs/command-reference.md entry, and flesh out the changelog.

---------

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
Co-authored-by: agessaman <adam@gessaman.com>
2026-09-19 20:19:38 -07:00
Adam Gessaman 38b259156e chore(deps): require meshcore 2.3.14 for the send_msg_with_retry ACK fix (#294)
The ACK fix I wrote for the radio-lock work is now upstream and released
(meshcore_py#108, v2.3.13). Before it, send_msg_with_retry reported ACKed DMs as
failures: the ACK subscription was registered only after send_msg returned, so
an ACK queued right behind MSG_SENT was dispatched with no listener, and each
attempt accepted only its own ACK code, so a late ACK answering an earlier
attempt was ignored. The bot logged "no ACK received after retries" and skipped
the delivery bookkeeping for messages the recipient had in fact received. Pin
2.3.14, which also fixes send_cmd's destination-type handling.

Two fixes from re-reading the merged radio-lock change:

A region-scoped channel message now restores global flood even when
set_flood_scope raises. The restore only ran in the finally around the send, so
a set that raised left the device pinned to that region and every later send
went out under it.

Drop the __setattr__ left behind when _SerializedCommands became
_serialize_command_frames; it sat after the function's return, unreachable, and
still referenced the proxy's _commands slot.
2026-09-19 17:28:14 -07:00
Adam Gessaman 9939e22575 fix(core): release the radio lock while waiting for ACKs (#291)
Radio commands were serialized per method call, so a composite command held the lock for everything it awaited. A DM's send_msg_with_retry kept every other command waiting through all of its ACK timeouts (up to ~36 s with three attempts), stalling channel replies, other DMs, scheduled sends, and automatic message fetching. req_regions_sync did the same for each neighbor scope reply.

Serialize at the frame instead: wrap CommandHandler.send() on the handler instance, which is where every meshcore command writes its frame and waits for the radio's immediate reply. There is still at most one in-flight companion frame, paced as before, and library methods that call self.send are covered without a proxy. Waits for ACKs and remote responses now run outside the lock.

Add MeshCoreBot.radio_session() for short sequences that must not interleave, and use it for region-scoped channel sends so set scope, send, and restore stay together. The per-call lock never guaranteed that: a DM retry loop queued between set_flood_scope and send_chan_msg sent its flood attempts under the channel's scope.
2026-09-19 09:12:07 -07:00
agessaman 1a8f108ac0 fix(types): resolve mypy errors in channel secret lookup
_channel_secrets() assigned dict_items and enumerate to the same
variable, and mypy inferred Any | None for the channel_idx .get() with
a mixed Any | int default. Declare items as Iterable[tuple[int, Any]]
and hold the raw index as Any. Runtime behavior is unchanged.

Also cover the list layout meshcore actually uses, where entries
without channel_idx fall back to their position.
2026-09-17 22:01:50 -07:00
agessaman c566fa97c8 fix(channel): improve channel fetching with retries and handle empty results cleanly (#266) 2026-09-16 17:32:54 -07:00
agessaman 16cce92c46 feat(region): set region scopes from the Radio page (#283)
Region scopes had to be hand-edited in config.ini, and the radio's own default
region could not be set from the bot at all. Both now live on the Radio page.

Node Settings gains a Default Region Scope field beside the path hash size,
which is the firmware's own setting — NodePrefs.default_scope_name and
default_scope_key, the region the radio falls back to for any send the bot does
not scope itself. Firmware that has no such command is reported as not having
answered rather than shown as an empty field, and a stored key that is not the
stored name's hash is flagged, because the radio routes by the key and the name
beside it is only a label. A build-flag default stores the bare name while
hashing the '#' form, so the comparison normalizes first.

Clearing it does not go through the meshcore library, because none of its reset
paths can: set_default_flood_scope(None) raises on len(None), "" earns
ILLEGAL_ARG from the firmware, and "*" only works because the frame is padded by
character count rather than by the name it wrote, landing one byte short of the
length the firmware reads as "a scope follows". The firmware's own contract for
clearing is a bare CMD_SET_DEFAULT_FLOOD_SCOPE, so that is what goes out. The
same padding bug is why a device scope name must be ASCII: a multi-byte
character displaces the transport key out of its field.

A separate Region Scopes card covers the bot's side — [Channels] flood_scopes as
an explicit choice between "reply whatever the scope" and an allowlist, and
outgoing_flood_scope_override. It writes config.ini, queues a hot reload, then
polls that reload and reports what the bot did with it, including saying plainly
when nothing picked it up. Per-channel flood_scope.<channel> entries are listed
read-only, since they are the reason a channel can ignore the default.

The two settings interact in a way worth stating on the page: once the bot sends
a scoped message it restores with set_flood_scope("*"), which leaves the radio in
forced-unscoped mode, so the device default stops applying until the bot scopes
another send.

Also from building it:

- flood_scopes left in [Bot] is surfaced rather than hidden. CommandManager
  still honours it when [Channels] is empty, so a page that ignored it would
  show "replies to every scope" while the bot enforced an allowlist — and
  saving "off" hands the allowlist straight back to [Bot], which the banner now
  says instead of claiming the old key stops mattering.
- outgoing_flood_scope_override = none is read as global flood on the send path,
  as it already was everywhere else. send_channel_message tested the raw value
  against a fixed tuple, so the lowercase spelling became the region "#none".
- Scope normalization moved to modules/flood_scope.py so the viewer, a separate
  process, can validate a typed name without importing the bot's command
  machinery. CommandManager delegates to it.
- Dark mode named only .border, so a .border-start divider drew Bootstrap's
  light #dee2e6 onto a dark card. All four directional utilities now match.
2026-09-16 07:59:10 -07:00
agessaman 10b6b6b01d fix(region): report warnings withheld for want of a contact
The known-contact gate for DM delivery returns before anything is recorded, so
a bot that keeps no contacts dropped every warning and wrote no rows: the page
showed "No warnings decided yet" indefinitely beside a status card saying it
was sending, and nothing distinguished a clean mesh from one where every
warning was being discarded. Only a debug log said otherwise.

Withheld warnings are now counted per local day in bot_metadata — a counter
rather than event rows, because this fires ahead of the cooldowns that would
rate-limit rows, so logging each one would bury the decisions worth reading.
The status card shows it, and the empty log explains itself and points at
channel delivery.

Also from the verification pass:

- The percent test built its value with config.set and a doubled %, which
  set() is the one input that cannot produce the bug. It now uses read_string
  with a bare %, and I checked it fails against raw=False by substituting
  DEFAULT_MESSAGE — the actual failure mode.
- docs still said the bot identifies itself by public key, contradicting the
  section forty lines above. On this path it is name-only, and that means the
  self-exemption is spoofable; the doc says so.
- delivered_today folded dry runs in with real sends, so a morning's preview
  read as afternoon transmissions. previewed_today is now separate and the
  figure follows the current mode.
- TRIM(MIN(channel)) so a whitespace-padded historical row cannot become the
  display name for a merged channel.
2026-09-16 00:28:33 -07:00
agessaman 4c0f43499e fix(region): close four defects found in adversarial review
**A `global` verdict now requires RF correlated to the message.** It was also
reachable through `_is_confirmed_global_flood`'s argument-from-absence route
("no scope-eligible packet in the window, therefore unscoped"), so a row that
literally said TC_FLOOD, or a scoped ADVERT that the GRP_TXT filter excluded,
came back as "no region code". That inference is fine for deciding whether a
`*` in flood_scopes authorizes a reply; it is not fine for accusing someone of
a misconfiguration. Channel messages still correlate through the payload match
(#255), so the ordinary case is unaffected.

**A `%` in the warning message no longer wedges config reload.** `_get` read
without `raw=True`, so configparser's interpolation raised, the error was
swallowed, and the bot transmitted the default wording instead. Worse, the
bot's own `_validate_config_snapshot` iterates `config.items(section)` and
would reject every hot reload until the file was hand-edited. The save endpoint
now rejects `%` outright and caps the message at 500 characters, and the read
is raw so a hand-edited value is still honored.

**One unreachable sender no longer eats the daily cap forever.** The cap counted
failed attempts but the per-sender cooldown did not, so a node the radio cannot
reach was retried every `min_unscoped_messages` messages indefinitely and no
real offender was ever warned. Both count attempts now. The regression test
fails with three sends against the old filter.

**The sender is a display name, not an identity.** MeshCore's CHANNEL_MSG_RECV
carries no public key, so `sender_pubkey` was always empty on this path and the
only identity is a prefix anyone with the channel key can forge. DM delivery
now requires a contact the radio already holds, which bounds the bot to nodes
it knows and stops failed sends spending cap slots. Two tests asserted the
opposite because the fixture supplied a pubkey the radio never sends; they now
run with what the call site actually passes, and the docs no longer claim
pubkey identity.

Also: a negative `max_warnings_per_day` fell back instead of clamping to the
"unlimited" sentinel; tallies group case-insensitively so a `#` or case change
does not split a channel; retention uses the same clock the rows are written
in; the status figure shows delivered with attempts beside it, rather than a
count of four next to "last warning: none yet"; and the channel bars are scaled
over classified traffic so their width equals the percentage printed beside
them (33.7% was drawn at 28.7%).
2026-09-16 00:14:20 -07:00
agessaman 5ed1a5d1c7 fix(region): honest empty and count-only states on the page
An empty daily-volume chart rendered as an 84px hole with a date range under
it, which reads as broken rather than as no data; it is hidden until there is
something to plot, and the summary says so in words.

"Count only" disabled the rest of the form, which implied those values would
not be saved — they are — and stopped anyone drafting their wording before
turning warnings on. A sentence under the mode selector says it instead.
2026-09-15 22:44:38 -07:00
agessaman d0ec073bb9 fix(region): accessible chart and alert colors in both themes
Bootstrap's .text-warning is #ffc107, which is 1.6:1 on a white card — the
headline percentage and the "daily cap reached" line were close to invisible
in light mode. The chart fills had the same shape of problem in reverse: the
grey "couldn't tell" band sat at 2.1:1 against a white card and 2.9:1 against
a dark one.

Each color is now a token chosen per theme against the surface it lands on:
text clears 4.5:1, fills clear 3:1. Measured in the browser in both themes.
2026-09-15 22:43:12 -07:00
agessaman 4eca8b166e fix(region): show which channel chips are already in the allowlist
An unselected chip looked identical to a selected one, so clicking it was a
coin flip between adding and removing. The chips now reflect the field, in
both directions, and stay in step when the list is typed by hand.

Also re-read after a save rather than patching the form in place (the status
strip's budget is derived from the cap that was just changed), and stop the
reset-to-default button from blanking the message when the page never loaded.
2026-09-15 22:41:06 -07:00
agessaman 0c1aa4fd15 fix(region): never warn the synthetic "Channel User" sender
A channel message with no "Name: " prefix falls back to a stand-in name.
That is not a node: every such message shares the identity, so they would
accumulate a run together and then get a DM addressed to a contact that
does not exist. The hook now passes no sender for those, so they are still
counted but can never earn a warning. The literal is a named constant and a
test pins the hook's guard to it.

Also folds the channel body-budget formula into models.channel_body_limit
rather than letting the web viewer keep a fourth copy of
"max(130, 160 - len(name) - 2)"; BaseCommand and CommandManager now call it
too, and 158 becomes models.DM_BODY_LIMIT.
2026-09-15 22:39:34 -07:00
agessaman 147cc588e1 fix(region): reserve the warning slot before awaiting the send
Recording the decision after the send left an await between the cap check
and the row that check reads. Two channel messages in flight both passed a
cap of one and both transmitted; the new test fails with two sends against
the old ordering.

Every gate and the reservation now run with no await between them, so on the
single event loop they are atomic. A failed send corrects its own row to
'failed' and releases the mesh cooldown, but the row stays and still counts
against the daily cap: a send that reported failure may have put something on
the air before it did.

Also from a self-review pass: bound the per-sender run table (an unbounded
dict on the channel-message path is a slow leak), identify the bot's own node
by public key rather than only by the configured name, and respect
channelpause.
2026-09-15 22:36:29 -07:00
agessaman bb3bfe321b feat(region): warn senders whose channel messages carry no region code
Closes #279.

A MeshCore client with no region configured sends every channel message as
a plain FLOOD, which every repeater on the mesh rebroadcasts. This adds two
things: free observation of how much of that the bot hears, and an opt-in
warning to the senders.

Observation classifies each channel message as scoped, global or unknown and
tallies it per channel per local day. It costs one upsert and no airtime, and
it is what lets an operator see the size of the problem before deciding to
spend airtime on it. The classification runs ahead of the flood_scopes
allowlist, because an unscoped message is exactly what that allowlist drops.

Warnings only ever fire on positive RF evidence of an unscoped FLOOD. Absence
of correlation is not proof that a sender omitted a region, so it classifies
as unknown and stays quiet. They are off by default, start in dry run, and are
fenced by min_unscoped_messages, a per-sender cooldown, a mesh-wide cooldown
and a daily cap on attempts. Dry run consumes the same budget it previews, so
the log is what going live would put on the air, not an upper bound. All three
limits read from the database, so a restart cannot release a burst.

Channel-delivered warnings go out at global scope on purpose: the recipient is
by definition outside any region the bot replies under.

New Settings -> Region Warnings page in the web viewer covers all of it. Its
dark-mode striped rows exposed a pre-existing base.html bug where Bootstrap's
light-theme text color survived on a dark row background (~1.3:1); fixed there
for every table in the app.
2026-09-15 22:34:46 -07:00
agessaman 681f0dcdcc fix(core): keep services alive during radio reconnects 2026-09-15 22:03:09 -07:00
agessaman 57d8c02649 fix(weather): retry transient daily forecast failures (#264) 2026-09-15 21:48:00 -07:00
agessaman aaca69e30f fix(scheduler): tolerate short job delays (#252) 2026-09-15 21:02:32 -07:00
agessaman d3a435d9ed fix(help): preserve multiword aliases in fallback 2026-09-15 21:01:39 -07:00
agessaman 3b1edea8af fix(web): run manual backups in standalone viewer (#280) 2026-09-15 20:57:58 -07:00
agessaman a861e848af fix(commands): handle subcommand help and test escapes (#285, #277) 2026-09-15 20:57:49 -07:00
agessaman 8d941adf20 fix(path): authenticate channel RF correlation (#255) 2026-09-14 21:31:11 -07:00
Gerard Hickey c41501eb48 feat(web_viewer): manage local plugin settings (#272)
Discover local commands and services in the Plugins UI and route their settings to the local config overlay. Upgrade GitHub Actions to Node 24-capable majors and migrate frontend linting to ESLint 10 flat config. Keep command and service discovery namespaces isolated and align duplicate-name handling with runtime loading.
2026-09-13 18:36:29 -07:00
Gerard Hickey 19c3d4912a feat(web_viewer): mask plugin password fields in settings (#273)
Add a password type to settings_schema so plugin secrets render as masked inputs and never reach the browser. Stored values stay in config.ini; an empty box on save leaves an existing secret unchanged.
2026-09-13 12:57:01 -07:00
CoderNemesis b5fc971148 feat(test): localize test-command distance units (#275)
[Test_Command] distance_unit (auto/km/mi) controls {path_distance} and {firstlast_distance}. Auto follows the reply language (miles for en/en-US, kilometres otherwise, including en-GB). Elapsed times of 1s and above render as seconds to keep the ack short.
2026-09-13 12:49:21 -07:00
agessaman 1f87707ade fix(packet_capture): publish UTC in every packet time field (#278)
The published packet payload mixed clocks: "timestamp" was UTC ISO 8601
with a Z suffix, but the "time" and "date" fields beside it were rendered
from a naive datetime.now(), i.e. the host's local wall clock. A consumer
reading those two off a bot in a non-UTC zone sees a skew of exactly that
zone's UTC offset and flags the observer's clock as wrong.

Local time was never the intended reading. The original script took time
and date off the firmware log line, and mctomqtt sets the device clock
from calendar.timegm(), so upstream both fields are already UTC.

All three now render one aware UTC instant, so they cannot disagree with
each other or straddle a second boundary. _utc_iso_timestamp() takes an
optional instant to make that possible; its no-arg behavior is unchanged.
2026-09-09 22:22:26 -07:00
agessaman 5f89708461 Merge remote-tracking branch 'origin/dev' into pr262
# Conflicts:
#	CHANGELOG.md
2026-09-07 20:29:26 -07:00
agessaman 698f18dc6e fix(i18n): anchor the local catalog on local_dir_path and wire it everywhere
The local translation path defaulted to a bare `local/translations/`, resolved
against the process cwd and unrelated to `[Bot] local_dir_path`. Since
`local_dir_path` already selects where an operator's commands, service plugins
and config overlay live, the catalog belongs in that same tree. It now defaults
to `<local_dir_path>/translations`, resolved absolute against the bot root, so
relocating `local_dir_path` moves the catalog with it and the lookup no longer
depends on the working directory. An explicit `local_translation_path` still
wins.

Only the constructor at startup was passing the local path. `get_translator()`
builds and caches a translator per detected language, and `reload_config()`
builds a fresh one, and both were still constructing `Translator` with the
distributed path alone. So local overrides silently vanished from any reply that
used the sender's language, and did not survive a config reload. Both now pass
it, `reload_config()` republishes it alongside `translation_path`, and it is
snapshotted and restored on rollback. The DummyTranslator fallback sets it too,
since `get_translator()` would otherwise raise AttributeError there.

Collapsed `_deep_merge_translations` into the existing `_merge_translations`,
which already merged deeply with the primary winning. Rewrote that one to stop
mutating its input: it shallow-copied `fallback` and then recursed into the
shared sub-dicts, so merging a base language over English corrupted the cached
English catalog at every level below the first. Also replaced the two `print()`
calls on the error paths with logger calls.

Registered `local_translation_path` in the config schema next to
`translation_path`, and reverted a trailing-whitespace reflow that touched 84
lines of config.ini.example for a one-key addition.

Tests: local overlay (single-string override, added keys, local-only catalog,
missing directory), the merge no longer mutating its fallback, the default
following `local_dir_path`, an explicit setting overriding it, the overlay
reaching `bot.translator` end to end, and a reload picking up a moved path.
2026-09-07 20:28:45 -07:00
agessaman cf26da652d fix: log command failures with exception() instead of an inlined frame
Addresses the review on #243. The original change interpolated the last
traceback frame into an `err_desc` string used in two places, and the second
one was the problem: `errors.execution_error` is sent back over the mesh, so
an absolute install path went out over RF and spent airtime on the error
path, where retries are most likely.

logger.exception() puts the full traceback in the log instead of just one
frame, which is strictly more than the original wanted, and the RF reply goes
back to str(e). That also removes the `list(tb.stack)[-1]` IndexError on an
empty stack, since the frame handling is gone entirely.

Tests cover both halves: the failure is logged via exception(), and the reply
carries only the exception text with no filesystem path.

CHANGELOG entry moves under the existing `## [Unreleased]` / `### Fixed`.
2026-09-07 16:41:47 -07:00
Gerard Hickey cd54b36f7a fix: Add traceback info for failed commands
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
2026-09-07 16:35:31 -07:00
agessaman ae4fd55051 Merge pull request #254 from ComchanNet/add_support_for_shlink_message_filter
Add shlink support and rework response templates onto a state machine.

Merged via this branch rather than the PR head: the fork is org-owned, so
"allow edits by maintainers" does not grant push access and the review fixes
could not be pushed to #254 directly. Roger Fedor's four commits are included
with authorship intact, followed by two commits addressing the review.
2026-09-07 16:24:32 -07:00
agessaman 4753bf32b3 test: close coverage gaps found by differential-testing the parser rewrite
Differentially tested the current parser against the pre-PR one over every template
shape shipped in config.ini.example and docs (31 shapes x 3 field sets x 4 message
states, 372 comparisons) plus 32 adversarial inputs. That found one regression I had
introduced: reading a quote immediately after ':' as the start of a quoted argument
voided `prefix_if_nonempty:"`, whose unterminated string rejected the whole
placeholder and emitted raw template text. An unterminated quote now falls back to
greedy parsing, so a literal quote prefix behaves as it always has. Both cases are
pinned by tests.

Also covered the shorten_url alias in feed formats, which had none: accepted as a
function, chains like shorten, falls back to the original on failure, is seen through
by shorten_feed_urls so a link is not shortened twice, and does not swallow an
unrelated name that merely starts with it.

Reset the warn-once flag in the async no-warning test. It asserted warning was never
called while a preceding test could already have tripped the flag, so it would have
passed vacuously.
2026-09-07 16:16:33 -07:00
agessaman 4c76ee3dbe fix: address review findings on shlink shortener and template parser
Security: a shlink deployment with short_url_website unset POSTed the operator's
API key to v.gd. _normalize_base falls back to the public default and the shlink
branch guarded only on a missing key, never on a missing base. shlink now requires
an explicit base and refuses a v.gd/is.gd host outright rather than sending an
X-Api-Key there.

Correctness: _shorten_url_with_gd lost its response.ok guard when the backends were
split out, so a 502 whose body starts with http was returned as the short URL and
transmitted over RF. Restored, with a warning. The regression test that should have
caught this passed only because it left mock_resp.text as a MagicMock; it now uses
a realistic proxy maintenance body.

_shorten_url_with_shlink never checked the status either. Shlink reports failures as
RFC 7807 problem details, which parse as JSON and simply lack shortUrl, so a bad API
key was indistinguishable from an unshortenable URL at DEBUG. It now warns on a
non-OK status, a non-JSON body, and a missing shortUrl. Dropped the shortUrlSlug
fallback: that is a request field, not a response field, and returning it puts a
bare slug where a link belongs.

Timeouts and connection errors are back on their own DEBUG handler. They had fallen
through to the broad handler, whose level rose to ERROR in the same diff, so a
routine intermittent uplink logged "Unexpected error shortening URL" on every reply.

Parser: _GREEDY_ARG_FILTERS was tested before the quoted-argument branch, so
prefix_if_nonempty, the one filter already in shipped configs, could not use the
quoted syntax this PR adds. {path_distance|prefix_if_nonempty:"Dist {sender}: "}
rendered raw template text. A quote immediately after the ':' now selects the quoted
grammar; anything else stays greedy, so config.ini.example's
`prefix_if_nonempty: | Path Dist: ` keeps its pipe and its whitespace.

Blocking HTTP on the event loop: the path command rendered its reply prefix inline
from async code, so a slow shortener stalled radio RX, MQTT and every other handler
for the full 5s timeout. Added format_piped_template_async and switched the path
command to it. The test command still renders synchronously through the sync
check_keywords dispatcher; making that async is a separate change, so the filter now
warns once when it runs on the loop.

Naming: one operation should not have two names in an operator-facing DSL. shorten
and shorten_url are aliases in both response templates and feed formats, and
if_nonempty is canonical with if_notempty as an alias, so a filter chain copied
between a feed format and a response_format works either way.

Also: reduced _build_create_shlink_url to the one parameter it uses and dropped its
dead query/startswith lines, renamed the shlink POST callable from `get`, documented
the config argument on format_piped_template, stopped gating the render trace on an
unrelated parameter and logging field values (sender IDs, user phrases) with it, and
moved the changelog entry from Fixed to Added and Changed.
2026-09-07 15:45:49 -07:00
Adam Gessaman c9ff277fd2 Merge pull request #261 from custardpy/feat/weather-service-localization
feat(i18n): localize weather outputs and add Russian translations
2026-09-07 15:19:03 -07:00
agessaman 595f37a27b fix(i18n): address review findings on weather localization
- Extracted `modules/alert_format.py` as the single NWS alert formatter, replacing four copies of the event-type abbreviation table and two of the time compactor. `!wx alerts` and the proactive `WeatherService` broadcasts now localize from one code path, so a Russian bot no longer answers `!wx alerts` in English while its proactive alerts are Russian.
- Stopped leaking translation key paths into mesh broadcasts. `Translator.translate` returns the dotted key when a lookup misses in both the locale and the English fallback, which is right for development but reached the air in production: an unclassifiable NWS title rendered as `⚪Hazardous services.weather_service.event_types.Unknown`, an unmapped WMO code as `services.weather_service.weather_descriptions.4`, and an oddly-cased `wind_speed_unit` as `services.weather_service.wind_speed_units.KMH`. `alert_format.translate_or()` carries an English default at each site, and `WeatherService` now normalizes and validates its three `[Weather]` unit settings the way `GlobalWxCommand` already did.
- Fixed alert expiry rendering in every locale. The formatter rendered a timestamp to a string and re-parsed its own output with `(\d+)(AM|PM)` against a hardcoded English month list, so translated months took the wrong branch and truncated mid-string. Times now carry parsed parts and render through a per-locale `common.alerts.time_12h` template — the space before AM/PM was correct (Russian writes "6 дня", not "6дня"); the downstream regex was the bug.
- Restored month abbreviation. `_compact_time` iterated over abbreviations and replaced them in the string instead of mapping full names, so English stopped shortening "June 28" and Russian replaced the "Jun" inside "June", leaving a stray Latin "e" (`июнe 28`). Reuses the existing `common.date_time.month_abbreviations` rather than the duplicate `services.weather_service.months` block.
- Made `!gwx` display units follow `[Weather]` config instead of the response language. Visibility switched on `base_language != 'en'`, so `language = ru` with the default `temperature_unit = fahrenheit` printed Fahrenheit beside kilometers, and `en-GB` was forced to miles. Pressure is a locale convention rather than a metric/imperial split, so each catalog names its own via `commands.gwx.pressure_unit` — previously every non-English locale inherited mmHg from the English catalog, whose `pressure_mmhg` string contained Russian text, giving German and French users Cyrillic pressure units.
- Let localized `H`/`L` labels reach a standard install. `config.ini.example` shipped the three `temperature_*_format` keys uncommented with literal `H:`/`L:`, and a config value always beats the new locale-aware default, so a Russian bot built from the documented example still rendered `H:47°C L:33°C`. The example now uses the `{high_label}`/`{low_label}` placeholders, which were documented in the docstring but not in the file.
- Routed high/low labels through the reply's translator. `_format_high_low` passed `bot.translator`, so with `auto_detect_language` on, an English-default bot answering a Russian sender localized the rest of the line but not `H:`/`L:`. Added `BaseCommand.response_translator` for this, replacing `wx_international`'s reach into the private `_response_translator` ContextVar.
- Fixed a byte-budget overrun in `!gwx`. The guard on the extra conditions block compared a character count against a byte-derived budget while the rest of the function used `_count_display_width`; Cyrillic is two bytes per character, so the block was appended after the budget was spent.
- Reverted nine `commands.gwx` English rewordings that were not localization work, including the configuration hint in `mqtt_weather_no_subscriber` — dropping it left a mis-configured operator with no pointer to the two keys they need.
- Fixed the Russian `visibility` string, which said "км" on the miles key — the same locale/config conflation as the code bug, in the data. Shortened the Russian event-type abbreviations, which were full words consuming a quarter of the 130-byte budget at two bytes per character.
- Added `commands.wx.hourly_not_available`, missing from every catalog so `!wx hourly` printed the raw key path. Predates this branch; found while auditing every translation key the weather modules reference.
- Moved alert strings to `common.alerts.*` and wind directions to `common.wind_directions.*`, since a command and a service both read them.
2026-09-07 15:13:18 -07:00
agessaman 27d3f0847e feat: Enhance MeshMessage and command matching to preserve original content
- Addresses #267
- Added `original_content` attribute to `MeshMessage` to store the on-air body of messages, ensuring it remains unchanged during command processing.
- Updated command matching methods across various commands to utilize `_cleaned_content_matches`, which restores original content when necessary, preventing mention stripping from altering the message context.
- Implemented tests to verify that original content is preserved and correctly utilized in command execution and message handling.
2026-09-04 23:06:03 -07:00
Gerard Hickey 9f807c9260 feat(i18n): Add support for local translation files
Add the `local_tranlation_path` to the `Localization` section  of
the configuration file to allow one to specify a local translation
file outside the distributed translation files. This allows
translation files to be built for local commands without the
need of altering the distributed ones.

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
2026-09-04 21:50:02 -04:00
custardpy dec68d020d Merge remote-tracking branch 'github/dev' into feat/weather-service-localization 2026-08-30 08:15:00 +04:00
custardpy 92441d0aef fix(i18n): show gwx pressure in mmHg for metric locales
Render GWX pressure in mm Hg (rounded hPa->mmHg conversion) for
non-English locales instead of hPa, matching the metric handling used for
visibility. Also fix format_temperature_high_low so custom [Weather]
templates can use the documented {high_label}/{low_label} placeholders,
and add tests for the translator param.
2026-08-30 08:14:55 +04:00
Roger Fedor ee200670fc Add condition to skip short url creation if its an empty string 2026-08-29 16:00:49 -05:00
Roger Fedor 31fc107e5f Fixed url shortener test case and updated linting errors. Updated documentation and changelog to reflect changes. 2026-08-29 16:00:46 -05:00
Roger Fedor 938393efbd Rework response template to use a finite state machine for improved parsing capability and flexibility 2026-08-29 15:59:53 -05:00
Roger Fedor 101bc0abfe Add support for shlink and shorten_url message filter 2026-08-29 15:59:53 -05:00
Adam Gessaman 4b858e2274 fix(web_viewer): replace the Carto dark basemap with OpenFreeMap
Carto now requires an API key for basemaps.cartocdn.com and is retiring
raster tiles, so the mesh map's dark theme rendered an "API KEY REQUIRED"
watermark. OSM hosts no dark tiles of its own -- its Standard layer is
light only -- so switch the dark basemap to OpenFreeMap's vector 'dark'
style, rendered through MapLibre GL via maplibre-gl-leaflet. Leaflet and
the light OSM raster layer are unchanged, and the bridge renders into
tilePane so marker and overlay z-order is untouched.

Fall back to inverted OSM raster tiles when WebGL 2 or the bridge is
unavailable, so a failure degrades to a filtered map rather than a blank
one. The filter targets .leaflet-tile rather than .leaflet-tile-container:
the container is a 0x0 element that its absolutely-positioned tiles
overflow, so a filter there has an empty reference box and paints nothing.

CSP: drop cartocdn, allow tiles.openfreemap.org, and permit blob: workers.
MapLibre spawns its renderer in a worker from a blob: URL; without
worker-src it falls back to default-src 'self' and never starts.
2026-08-28 22:57:40 -07:00
Adam Gessaman a9f10d4dce fix(correlation): find the channel message's own RF row, not the newest (#255)
Verifying a channel message against the RF cache only ever checked the row
strategy 4 returned — the most recent packet heard. That assumes the RF log row
and the decoded CHAN event for one reception arrive back to back with nothing in
between. On a dense mesh they do not: a repeater's echo of the very same packet
is routinely logged in the gap.

The reporter's message was heard directly (SNR 13.25, 0 hops) and again via
repeater f0 185 ms later (SNR 12.0, 1 hop). Both rows carry packet hash
392926C85DCB87D0, but the check saw only the echo, disagreed on path length and
SNR, and left the route unresolved — so the bot withheld a path it had decoded
correctly and answered "No path information available in current message". The
reporter's own observation that MultiTest still reported paths is the tell:
multitest reads recent_rf_data directly and never consults the match tag, so the
radio data was there the whole time.

The cache is now searched for the row the payload matches rather than testing
just the newest one. #80's guarantee is unchanged — a route is still only ever
attributed on a positive match, never on recency — so this widens where the
check looks, not what it accepts. When more than one row matches they must
resolve to a single non-empty packet hash, which is only true of receptions of
one packet; two unrelated packets that happen to agree on all three fields stay
a fallback. A debug line now names the case where the newest row was not the
message's packet, so this failure mode is visible in logs rather than silent.

Side effect worth knowing: SNR and RSSI now come from the message's own
reception too. The reporter's message was logged at SNR 12.0 / RSSI -10, the
echo's figures, when its actual reception was 13.25 / -32.

Five tests cover it, including a reproduction built from the issue's log. Three
fail on the current code; the other two pin behaviour the fix must not break —
newest-wins among receptions of one packet, and scope_eligible_only still
filtering the search so the scope correlation cannot be handed an ineligible row.
2026-08-28 22:37:27 -07:00
Adam Gessaman 1740ca8072 fix(mqtt): resolve reconnect storm and improve token renewal handling
This commit addresses several issues related to MQTT connections. It prevents MQTT brokers from entering a reconnect storm by ensuring that the packet-capture watchdog only triggers a reconnect when necessary, avoiding duplicate CONNACKs and spurious disconnects. Additionally, it ensures that renewed MQTT auth tokens are properly enforced by implementing a clean reconnect process.

New configuration options are introduced: `mqttN_keepalive` for setting the PINGREQ interval per broker, and `mqttN_jwt_reconnect_on_renew` to control reconnect behavior after token renewal. Documentation has been updated to reflect these changes and to clarify the importance of unique client IDs for brokers in the same cluster.
2026-08-28 22:13:42 -07:00
custardpy 558e5e78b8 fix(i18n): show gwx visibility in km for metric locales 2026-08-28 10:36:42 +04:00