Integration fixes on top of the merge, in three groups.
Regressions in the shared shortener, which is not opt-in — weather alerts
call it unconditionally whenever an alert carries a link:
- Restore the `response.ok` guard on the v.gd path. Without it a 503 whose
body happens to start with `http` (a captive portal, a CDN error page) was
returned as the short URL. The test that covered this still passed because
it never set `.text`, so `MagicMock.startswith()` answered truthily; it now
supplies a body that would be accepted on a 200, so only the status check
can make it pass.
- Add the same guard to the shlink path, where a rejected API key is a 401
with a problem-details body and was indistinguishable from an empty result.
- Restore debug-level logging for `Timeout`/`ConnectionError`. A mesh node's
uplink drops out routinely and this had become `logger.error`.
- Drop the `shortUrlSlug` fallback. Shlink's create response carries
`shortUrl` and `shortCode`; `shortUrlSlug` is not in its schema, and a bare
slug is not a link — emitting one would have put `abc123` in a message.
The `shorten_url` filter did blocking HTTP inside the synchronous render
path (`process_message` → `check_keywords` → `format_response`), so a 5 s
shortener timeout stalled the radio transport along with everything else.
Making `check_keywords` async would touch 27 call sites, so instead the
network work moves ahead of the render:
- `resolve_template_async()` renders once with the filter in collection mode,
which walks the real chain so a clause already suppressed by `hops_min`
costs no request, then shortens what survived concurrently off-thread.
- The filter itself now only reads that mapping and never calls out. Asked to
render without a pre-pass it warns and drops the clause rather than block.
- On failure the clause is dropped rather than sent unshortened. A v.gd link
is ~19 bytes where the analyzer URL is ~59, against a ~158-byte budget the
prefix is subtracted from, so falling back would have turned one
transmission into two whenever the shortener was unreachable.
Smaller: rename `if_notempty` to `if_nonempty` to match the existing
`prefix_if_nonempty` (nothing ships using it yet, so there is no config to
migrate); `_build_create_shlink_url` no longer takes the long URL and API key
it never used; move the CHANGELOG entry from Fixed to Added and name the
filter it actually added; ungate the render debug log from `config`; document
that `shorten_url` is supported in `path`'s `reply_prefix` only.
Every template shipped in config.ini.example still renders identically to the
regex parser it replaced.
{firstlast_distance|prefix_if_nonempty: | F/L Dist: } prints "| F/L Dist: N/A"
on a direct message: the distance helpers return the literal "N/A" when there
is no path to measure, prefix_if_nonempty only asks whether the value is
non-empty, and "N/A" is. Suppressing that needed a gate, and pathbytes_min was
the only one available—so it was being used as a stand-in for "did this
message take any hops", which is not what it asks. It asks how the path is
encoded, so pathbytes_min:2 also discards a one-byte multi-hop path whose
distance is real and measurable.
hops_min:N asks about the route itself. hops_min:1 drops a clause on a direct
message and nothing else:
hops_min:1 pathbytes_min:2
direct cleared cleared
1-byte 2 hops kept cleared <- the difference
2-byte 2 hops kept kept
unknown cleared cleared
An unknown hop count clears the value: a gate that cannot confirm the route
should suppress rather than guess, matching pathbytes_min.
The hop-count logic moves to utils.message_hop_count so the filter and
BaseCommand.get_hops_display_values share one implementation instead of two.
It returns None for unknown rather than 0, since a gate must not read
"cannot tell" as "direct".
Adds the 16-char MeshCore packet identity hash to the three formatters that
render user-facing templates: [Keywords] responses, the test command's
response_format, and the path command's reply_prefix. It is read only from
routing_info, which is attached solely from an RF packet correlated to the
message, so a hash from an unrelated transmission is never shown. Missing,
empty, and the 0000000000000000 error sentinel all render empty.
get_packet_hash_placeholder returns "" rather than a False sentinel. Empty
gives byte-identical output on every path—prefix_if_nonempty already tests
falsiness and str.format renders "" as nothing—whereas False would print
the literal string "False" into a mesh reply from any consumer of
get_standard_placeholder_fields that formatted it directly.
TestCommand.format_response now builds on get_standard_placeholder_fields
instead of duplicating it. That also honors the extra argument its docstring
already promised but silently ignored, and picks up the path-string hop
fallback, so {hops} reads "0" rather than "?" for a message whose hops and
routing_info are both unset but whose path says "Direct (0 hops)".
Documents the placeholder in config.ini.example, the default config that
core.py writes on a fresh install, the README and docs/path-command-config.md.
The generated config was also missing {elapsed}; added that too.
Total path distance was already a solved problem here: the test command
computes it and exposes {path_distance} through the piped-template
engine, with pipe filters and docs. The path command just had no way to
reach it, since reply_prefix went through plain str.format.
Rather than add a second distance implementation, this reuses what
exists. BaseCommand.get_standard_placeholder_fields now returns the
shared placeholder set so subclasses can extend it, the path command
renders reply_prefix through format_piped_template with an added
{path_distance}, and the sum itself uses utils.calculate_distance over
the nodes the path command has already resolved.
Distance covers sender -> each hop -> bot and is deliberately blank when
the chain cannot be measured end to end (unresolved hop, prefix
collision, missing or 0,0 coordinates, unknown sender, no configured
bot position), so a partial sum is never reported as the real distance.
Because the prefix now supports pipe filters, an empty value takes its
label with it: {path_distance|prefix_if_nonempty:📏 }
Supersedes #198, which added a parallel haversine, a show_path_distance
flag, and a distance_traveled key across 10 locales.
- Implemented caching for multi-byte mesh aggregation, allowing concurrent requests to share computation and reducing CPU load.
- Updated the web viewer to coalesce live edge events and refresh the mesh graph every 30 seconds, improving responsiveness on busy networks.
- Adjusted logging behavior in the web viewer to respect the configured log level, ensuring efficient logging without excessive duplicate entries.
- Added configuration options for mesh graph caching duration, enhancing user control over performance settings.
- Implemented chunked deletion for data retention, allowing for smoother cleanup processes without monopolizing SQLite's writer lock.
- Configured retention settings to delete in batches with pauses, improving performance on SD-card installations.
- Updated Linux service installers to allocate 1GB of memory and 200% CPU, providing better resource management for Raspberry Pi workloads.
- Added detailed configuration examples for Raspberry Pi in documentation to guide users on optimal settings.
- Updated mesh graph path splitting and aggregation to run in SQLite, improving performance by avoiding Python materialization.
- Defaulted graph persistence to batched writes for new installations, reducing WAL churn and SD-card writes.
- Enhanced data retention to execute shortly after startup, independent of the nightly maintenance schedule.
- Added a table-specific index for `mesh_connections` to support window and retention queries, ensuring efficient data access.
- Updated configuration examples to include new `{hops}` and `{hops_label}` placeholders for path command replies, enhancing response detail.
- Modified `BaseCommand` to support hop count retrieval and formatting, ensuring accurate display of hop information in responses.
- Enhanced unit tests to verify correct formatting of hop-related placeholders in response messages, improving overall command functionality.
- Enhanced the command reference with detailed descriptions for `cmd`, `version`, `weather`, and `path` commands, including usage examples and configuration options.
- Added new `RandomLine` command documentation for configurable triggers.
- Updated configuration documentation to reflect the deprecation of the global `[Aliases]` section, encouraging per-command alias definitions.
- Clarified the `[Rate_Limits]` and `[Webhook]` sections in the configuration guide.
- Improved the web viewer documentation, emphasizing security practices and configuration options.
- Introduced `reply_prefix` to prepend a customizable string to path command replies, supporting placeholders for dynamic content.
- Added `minimum_path_bytes` setting to control the resolution of repeater names based on the number of bytes per hop, enhancing path command behavior.
- Updated relevant documentation and translations to reflect these new configuration options.
- Implemented unit tests to ensure correct functionality of the new features.
- Updated `meshcore` dependency version to `2.2.14` in both `pyproject.toml` and `requirements.txt`.
- Added multi-byte path support in the `PathCommand`, allowing for 1-, 2-, and 3-byte-per-hop paths.
- Enhanced `MessageHandler` to utilize `routing_info` for accurate path extraction and validation.
- Improved path extraction methods in `MultitestCommand` and `TestCommand` to prefer `routing_info` for node IDs.
- Refactored path handling logic across various commands to ensure consistent multi-byte path processing.
- Updated `MeshGraph` to support multi-resolution storage of edges, allowing prefixes of 2, 4, or 6 hex chars without truncation.
- Implemented prefix matching logic to ensure distinct links are maintained and accurately retrieved based on prefix queries.
- Refactored methods in `MessageHandler` and `PathCommand` to accommodate variable prefix lengths during graph lookups.
- Enhanced tests to validate prefix match functionality and edge management in the mesh graph.
- Added a new section in the FAQ detailing how to run meshcore-bot on a Raspberry Pi Zero 2 W, including specific configuration settings to manage memory usage effectively.
- Updated the path command configuration documentation to clarify the default values and usage of `graph_startup_load_days` and `graph_capture_enabled`, improving user understanding of these settings.
- Renamed various documentation files to use lowercase for improved consistency, including `WEB_VIEWER.md`, `DOCKER.md`, and `PATH_COMMAND_CONFIG.md`.
- Updated references in `config.ini.example`, `docker-setup.sh`, `README.md`, and multiple documentation files to reflect the new file names.
- Removed obsolete documentation files related to the Discord Bridge, Map Uploader, Packet Capture, and Weather Service, streamlining the documentation structure.
Adam Gessaman