Commit Graph
1144 Commits
Author SHA1 Message Date
agessaman de1df09d6f fix(trace): accept a late reply to an earlier attempt, and allow for repeater TX delay
Each attempt waited only for its own tag, so a reply that arrived after its attempt's timeout was dropped once the retry went out. On my test mesh a three-hop trace takes about nine seconds (each repeater waits its random TX delay before forwarding), against a 2.5 s timeout, so every trace failed even though the first attempt's full reply came back.

run_trace now subscribes to TRACE_DATA once, before the first send, and takes a reply to any of its tags until the last attempt's timeout runs out. A reply that lands during the retry delay ends the trace without sending again, and subscribing before the send also closes the window where a fast reply could arrive before wait_for_event was listening.

The timeout defaults go from 1.0 s base and 0.5 s per hop to 2.0 s and 1.5 s, matching the 1.5 s the trace command and its settings schema already used. Configs that set them keep their values.
2026-10-03 09:27:59 -07:00
Adam Gessaman 5bfe975fa8 Merge pull request #336 from agessaman/fix/transmission-tracker
fix(transmission_tracker): match a sent message only to its own packet
2026-10-03 09:22:51 -07:00
agessaman 4c1f55236e Merge remote-tracking branch 'origin/dev' into fix/transmission-tracker
# Conflicts:
#	CHANGELOG.md
2026-10-03 09:13:11 -07:00
Adam Gessaman ba94c1e3f3 Merge pull request #335 from agessaman/fix/new-contact-route
fix(message_handler): take a new contact's route from its own advert
2026-10-03 09:13:03 -07:00
agessaman db839a6192 Merge remote-tracking branch 'origin/dev' into fix/new-contact-route
# Conflicts:
#	CHANGELOG.md
2026-10-03 09:12:52 -07:00
Adam Gessaman 18e4a690e1 Merge pull request #338 from agessaman/fix/noaa-tomorrow-zero
fix(wx): keep a 0° Tomorrow period in the default NOAA reply
2026-10-03 08:56:50 -07:00
agessaman a8f4eec3ec fix(graph_trace_helper): no self-loop for a one-hop neighbor sharing our prefix; survive bad distances
Codex re-review follow-up. The one-hop shortcut wrote dd->dd both ways when the neighbor shares the bot's prefix; it now skips the update, as the multi-hop path does. A distance error (a stored infinite latitude) no longer drops the trace's edges; the edge is recorded without a distance, as before the rewrite.
2026-10-02 18:54:40 -07:00
agessaman 568b296358 fix(graph_trace_helper): tell the bot apart from a hop sharing its prefix
Keys and locations follow chain positions instead of prefixes, so a hop whose hash equals the bot's short prefix no longer takes the bot's public key or location, and an edge between two equal prefixes, which the prefix-keyed graph would store as a self-loop, is skipped.
2026-10-02 18:48:21 -07:00
agessaman a42ba576db fix(graph_trace_helper): record a trace's links in path order, with known keys only
Codex review of the bot_prefix fix; these helper defects only mattered once traces reached the graph.

- Our own trace also confirms the bot -> first hop link, which was missing. Edges follow the chain bot -> h1 ... hn -> bot with 1-based hop positions in path order, as the message path writer numbers them (they were reversed).
- A public key is stored only when its prefix belongs to one recent repeater. The location lookup still picks a position for the distance, but its pick among colliding prefixes no longer becomes the stored key (also in the one-hop case).
- bot_prefix reads the radio's current key on each access, so reconnecting or rebooting into another radio is picked up.
- extract_repeater_prefixes_from_path no longer drops a last hop equal to our prefix: a companion never forwards, so it is a repeater sharing our short prefix. On dev bot_prefix was always None, so this keeps the attribution dev already had.
2026-10-02 18:44:50 -07:00
agessaman 0ad9f4e69a fix(transmission_tracker): read the bot's prefix from the radio's reported key
bot_prefix and graph_trace_helper's bot key read meshcore.device, which meshcore_py doesn't have, so both were always None and update_mesh_graph_from_trace_data skipped every trace. Both now come from meshcore.self_info's public_key (device stays a fallback), and bot_prefix is read lazily because the tracker is built before the radio connects. The helper names the bot at the trace's hash width, so the last hop's edge matches the hops' own width, and marks 2-byte edges with prefix_bytes=2.

trace_command read update_graph_two_byte but never used it; 2-byte traces now follow it instead of update_graph_one_byte.
2026-10-02 18:32:54 -07:00
agessaman b569eb8dd7 fix(wx): keep a 0° Tomorrow period in the default NOAA reply
get_noaa_weather's Tomorrow part tested period.get('temperature', '') for truthiness, so a forecast of exactly 0 was treated as missing and the part was left out. It now uses _has_temp, as _noaa_period_str already does for Today and Tonight.
2026-10-02 18:02:34 -07:00
agessaman bf8b5c7ceb Merge remote-tracking branch 'origin/dev' into fix/new-contact-route
# Conflicts:
#	CHANGELOG.md
2026-10-02 17:33:26 -07:00
agessaman 8f99b45e44 Merge remote-tracking branch 'origin/dev' into fix/transmission-tracker
# Conflicts:
#	CHANGELOG.md
2026-10-02 17:33:26 -07:00
agessaman 2f0dd667a4 fix(message_handler): release a cancelled add's claim and match last_advert 0 exactly
Codex re-review follow-up. A cancelled add (reconnect cancelling callbacks) kept its claim, so a later NEW_CONTACT for the same advert never added; it now releases and re-raises. A last_advert of 0 is a real advert timestamp, not a missing one. The expiry test now fills the advert cache it is meant to exercise.
2026-10-02 17:33:18 -07:00
agessaman 3c76c5bf73 fix(transmission_tracker): require our radio name on a channel match and drain writes at shutdown
Codex review follow-up. A channel message from another node with the same text claimed our record, and names the sender-prefix heuristic cannot split ("Bot [RF]", "Bot: RF") never matched; the match now compares the full decrypted message with "<radio name>: <text>", falling back to the text alone before the name is known. Bot shutdown closes the tracker after disconnecting, which waits for queued repeat-count writes; any later write runs inline.
2026-10-02 17:33:18 -07:00
Adam Gessaman b6867a496a Merge pull request #334 from agessaman/fix/noaa-tonight-without-temp
fix(wx): leave out a NOAA Tonight period without a temperature instead of failing the reply
2026-10-02 17:26:38 -07:00
agessaman 201e60163c fix(message_handler): find NEW_CONTACT's advert by its timestamp, even mid-processing
Codex review follow-up. The RX handler cached an advert's RF entry only after awaiting _process_advertisement_packet, which can yield (geocoding), so a NEW_CONTACT handled meanwhile found nothing or an older advert. Adverts are now recorded in their own short cache (_advert_rf) before that await. The match also requires the advert timestamp to equal NEW_CONTACT's last_advert, so a delayed copy of an older advert no longer stands in for the newer one; without last_advert the most recent advert is used.

A failed or raising add releases its claim, so a later NEW_CONTACT for the same advert tries again.
2026-10-02 16:55:40 -07:00
agessaman 323ebedeae fix(transmission_tracker): match a sent message only to its own packet
match_packet_hash gave a pending transmission the hash of the first packet heard within its 30-second window, whatever the packet was. On a busy mesh that was usually someone else's message or advert, so its repeats were counted as ours (the web viewer's repeat counts, DARC MoWaS's ACK check, graph_trace_helper's "our trace" test) and our real repeats were not.

A transmission now records what identifies its packet (the channel index for a channel message, the recipient's key for a DM, the tag for a trace) and match_packet_hash claims it only for a packet carrying the same: a GRP_TXT that decrypts on that channel to the same text, a TXT_MSG addressed to the recipient's hash and, when the radio's key is known, from ours, or a TRACE with the same tag. The RX handler passes that evidence through _transmission_evidence, reusing the authenticated channel decode. Without evidence, match_packet_hash only looks up hashes already matched, so record_repeat and graph_trace_helper no longer claim pending transmissions.

Repeat-count writes go to a single worker thread instead of running sqlite on the event loop, and _update_command_in_database no longer inserts into sys.path on every repeat.

bot_prefix still reads meshcore.device and stays None; fixing it turns on graph_trace_helper's trace updates, which have never run, so it is a separate change.
2026-10-02 16:51:15 -07:00
agessaman 01718ff938 fix(message_handler): take a new contact's route from its own advert
handle_new_contact walked the last ten RF cache entries and used the first with routing info, whatever its type or sender. That packet's route went into the contact's out_path (and from there to add_contact in bot mode), the mesh graph and observed_paths, and its hash and signal data into tracking.

It now uses _find_advert_rf_data: the cached ADVERT whose payload starts with the contact's public key, newest advert first, and among the copies of that packet the first one heard, within rf_data_timeout. The mesh graph and observed_paths writes are gone: _process_advertisement_packet already records them for the same packet, so a correct match was counted twice.

With the correct hash, tracking reports the packet as a duplicate (the advert packet was tracked first), and bot mode skipped add_companion on that. The add gate now dedupes on the (public key, packet hash) pairs NEW_CONTACT itself added, so repeated events for one advert still add once.
2026-10-02 16:44:39 -07:00
agessaman e05df87a59 fix(wx): leave out a NOAA Tonight period without a temperature instead of failing the reply
get_noaa_weather built the Tonight part only when _noaa_period_str returned one, but the detail and append steps after it ran either way. With no temperature or short forecast, period_str was unbound and the whole reply became "Error fetching weather data"; after a night-time Today part it still held that part, which was appended a second time. The steps now sit under the same check, as in the Today and Tomorrow blocks.
2026-10-02 16:31:31 -07:00
Adam Gessaman 1009766a0c Merge pull request #333 from agessaman/feat/wx-gwx-convergence
feat(weather): converge wx and gwx on the best of each
2026-10-02 16:11:54 -07:00
agessaman 7159169eb3 Merge remote-tracking branch 'origin/dev' into feat/wx-gwx-convergence
# Conflicts:
#	CHANGELOG.md
2026-10-02 16:03:06 -07:00
agessaman f994171818 feat(weather): [Weather] always_show_location names the place in every wx/gwx reply
wx and gwx name the place only when it adds information: a city outside default_state or default_country, or a position. always_show_location (default false) names every place a lookup finds instead, including cities in the bot's own state and ZIP codes.

For a ZIP code, gwx now does a reverse lookup when the option is on (it skipped that since the name was never shown), and wx reuses the reverse lookup it already did for a companion position. A failed lookup leaves the reply unnamed rather than repeating the ZIP code. The prefix comes out of the message budget as before. The option is in the wx settings schema for the web viewer, the example config, the command reference and the changelog.
2026-10-02 07:55:18 -07:00
Adam Gessaman 5c6178dbfa Merge pull request #322 from agessaman/fix/unicode-digit-crash
fix: read superscript and circled digits in arguments as text instead of crashing
2026-10-02 07:25:27 -07:00
agessaman 66ce785e6c Merge remote-tracking branch 'origin/dev' into fix/unicode-digit-crash
# Conflicts:
#	modules/commands/alternatives/wx_international.py
#	modules/commands/wx_command.py
2026-10-02 07:19:44 -07:00
agessaman 9959cc33fc fix(gwx): moon for clear night hours in the hourly forecast
The hourly forecast still showed ☀️ for clear hours after dark. I now request Open-Meteo's hourly is_day with the hourly forecast and pick each hour's emoji from it, falling back to 06:00–18:00 local when it's absent, as the current conditions do.
2026-10-02 07:12:38 -07:00
agessaman 17a93104cc fix(gwx): label current conditions by is_day, moon at night, name tomorrow's weather
At 5:58 am wx 98258 (Open-Meteo provider) replied "Tonight: ☀️Clear … | H:68°F L:50°F | Tomorrow: 🌫️ H:67°F L:51°F". Any hour before 06:00 was labeled "Tonight", which next to today's high/low and a "Tomorrow" for the following day read like last night's forecast, and code 0 always showed a sun.

I now request Open-Meteo's is_day and use it for both the label and the emoji, falling back to 06:00–18:00 local when it's absent. Night before noon is "Overnight" (the NWS name for that period, new key in every catalog), night after noon stays "Tonight". Clear and mostly clear at night show 🌙. Tomorrow's segment adds its description when it fits and keeps the emoji-only form otherwise.

Goldens change for the new request field, Overnight on the fixtures captured after midnight, 🌙 on clear night fixtures, and tomorrow's description.
2026-10-02 07:04:52 -07:00
Adam Gessaman 46fdb1cded Merge pull request #323 from agessaman/refactor/weather-port
refactor(weather): behavior-neutral cleanup of wx, gwx, solarforecast and the weather service
2026-10-02 05:55:59 -07:00
agessaman 9c055030c7 fix(weather): alerts as an option alone, option-named sources, alert list send status
A codex re-review of the last two commits confirmed their fixes and found three gaps. wx alerts (and gwx alerts) with no location looked up a place called "alerts"; the option now applies to the fallback location like the others. A custom source named like an option (custom.wxsim.hourly or custom.mqtt_weather.hourly) was treated as the option and answered from the default source; a configured name now wins. The full alert list (wx <place> alerts) kept sending after a refused first message and reported success; it now stops and reports the refusal.
2026-10-02 01:31:37 -07:00
agessaman 1536f2a634 fix(weather): codex review of the whole branch
- A scheduled {cmd:gwx ...} or {cmd:wx ...} with a forecast and an alert slept out the TX pacing between them during capture; with bot_tx_rate_limit_seconds = 30 that passed the 30 s render timeout and the whole reply was lost. Pacing is skipped when the message has a capture_sink.
- gwx tomorrow ignored its byte budget (a Russian Reykjavik forecast was 99 bytes against 89). It now drops the precipitation amount, gusts, wind, the precipitation chance and the description, in that order. The Open-Meteo goldens give tomorrow runs a roomy budget, since their key-name translator makes the text much longer than real; trimming is tested with real catalogs in test_gwx_tomorrow_budget.
- Custom default sources (bare wx/gwx) still returned True after a refused send.
- An option without a location (wx hourly, gwx tomorrow, wx 5d) showed usage, since the fallback location was resolved before the option was parsed; the option now applies to the fallback, custom default sources included. The docs say how far each provider's forecast reaches.
- wx read its unit settings through get_config_value, whose [Weather] fallback logged a notice to move them into [Wx_Command] on every lookup (14 per reply); _unit_setting reads [Wx_Command], then [Weather], without it.
- NOAA forecasts that mention thunderstorms after sun ("Sunny then Slight Chance Showers And Thunderstorms") still got ☀️; thunderstorms now outrank sun.
- The catalog tests passed with keys removed, since a missing key differs from English; they now require each key and every wind point, with en-GB and Polish allowed to use the English letters.
2026-10-02 01:10:39 -07:00
agessaman 905c0af0cc Merge remote-tracking branch 'origin/refactor/weather-port' into feat/wx-gwx-convergence 2026-10-02 00:52:12 -07:00
agessaman e5b66b90ba refactor(weather): log a companion lookup on a bot without a database as dev did
A space bunny review (about 110,000 old-versus-new cases, two differences found) noted that the shared companion lookup returned silently when the bot had no db_manager, where dev's aurora and rain logged the AttributeError at DEBUG (and wx and gwx at WARNING). The guard is gone, so all four log it again. Unreachable in production, since the bot does not start without a database.
2026-10-02 00:52:05 -07:00
agessaman 0853739f30 fix(weather): review follow-ups: warning units, wx localization, help text
A space bunny review of this branch's commits found:
- gwx's high-wind warning converted the speed to mph for every catalog, so on a km/h bot it read 49 next to a reply saying 80, and Polish (whose template said km/h) regressed in the merge. The threshold stays 30 mph; the warning shows the speed in the configured unit with a {unit} placeholder in every catalog.
- tomorrow_not_available, tomorrow_error, multiday_not_available and multiday_error were still English in seven catalogs; they are translated, and the catalog test now fails on any weather reply left in English.
- wx kept NOAA's English wind letters and English weekday letters while gwx's are translated, so one German bot showed NE and NO for the same wind. wx now translates both through the catalog (commands.wx.day_abbrev added); English output is byte-for-byte unchanged, which the goldens show (their stub translator gives English for these short labels, via tests.helpers.key_name_translate).
- Polish feels_like was ungrammatical (odczuwalna -> odczuwalnie), Russian's used a nonstandard abbreviation, and Russian gwx's MQTT subscriber error lacked wx's hint.
- The help text (commands.wx.description/usage, commands.gwx.usage) described wx as ZIP-only; it now lists cities, coordinates and the options, under 130 bytes in every language.
- The generated default config, the docs' default_state wording, _fit_tomorrow_parts' docstring and the CHANGELOG's stats claim were inaccurate.
2026-10-02 00:48:31 -07:00
agessaman 927a3746a8 docs(weather): describe wx and gwx as they now behave
The command reference described wx as ZIP-only and gwx as a list of fields. It now gives both commands' accepted locations, options (tomorrow, hourly, Nd, alerts), the order they fall back through when no location is given, the unit settings, when the reply names the place, what custom WXSIM and MQTT sources answer, and that both work in scheduled messages. The config example's default_state/default_country comments say how they affect the place name, the no-location comment lists default_city, and the weather service page lists the wind units that actually exist.
2026-10-01 23:35:48 -07:00
agessaman 5b50229223 feat(weather): accept knots as the wind unit in wx, gwx and WXSIM
The weather service took [Weather] wind_speed_unit = kn, while wx and gwx rejected it with a warning and used mph, so the same setting gave knots in one and mph in the other. Knots now work in all of them: Open-Meteo is asked for kn, NOAA's mph and km/h values and station gusts are converted, gwx's high-wind warning converts back to mph for its threshold, and the WXSIM parser converts its km/h. The web config's wind unit choices gain m/s and knots.
2026-10-01 23:35:48 -07:00
agessaman db07f8ca6f feat(i18n): fill the weather catalog gaps in every bundled language
Nine catalogs lacked 30 of the keys wx and gwx use, so a German or Spanish bot answered with English weekday letters (M, T, W, Th), English "feels" and English MQTT and "not available" errors, and gwx and the weather service named wind directions in English. They now have translations, including the 16 wind points in local letters (Polish keeps the international ones, which the English catalog supplies). en-GB gets the English text so every catalog is complete, and Russian gets the two new hourly/custom-source replies. A test keeps each catalog covering these keys.
2026-10-01 23:30:48 -07:00
agessaman dbe4ac7028 feat(gwx): name the place only when it adds information, as wx does
gwx prefixed every reply with the resolved place. It now follows wx's policy: a city is named when it resolved to another country than [Weather] default_country or, in the US, to another state than default_state (or no default_state is set); coordinates are named when the reverse lookup found a place; a ZIP code is not named. A failed reverse lookup for coordinates used to turn the input into a label like "47.60620, -122.33210" (the longitude standing in for a country); such replies now have no label. The ZIP path drops its reverse-geocoding request, whose only use was the label.
2026-10-01 23:23:14 -07:00
agessaman 3d86b70899 fix(wx): fit NOAA's tomorrow forecast into its byte budget
format_tomorrow_forecast took max_length and never used it, so two long periods (Miami's "Chance Showers And Thunderstorms" day and night) came to 132 bytes against a 100-118 byte budget and the send path split the reply in two. It now tries the full reply, then without the night period's wind, without any wind, the first period alone, and that period without wind. Measured over the captured NOAA forecasts with the real English catalog at budgets 100-235, nothing overruns now (15 of 384 did); gwx and wx's default reply already fit. The regenerated goldens differ only in tomorrow keys, whose key-name translations are longer than real text.
2026-10-01 23:17:47 -07:00
agessaman a2f0d4c22c fix(weather): a refused first message ends the reply
Follow-up parts of a wx or gwx reply skip the per-user rate limit, since the limiter already let the first part through. When it had not (rate-limited, or the send failed), the alert after the forecast and the later parts of a multi-day forecast still went out on their own. Now a refused first part ends the reply. The multi-day sender also sent a forecast that fits one message only after dropping blank lines as a continuation, skipping the limit; the first message now always goes through it. execute returns whether the reply was sent, which the stats and web viewer record.
2026-10-01 23:12:47 -07:00
agessaman da6e699008 fix(wx): read NOAA's precipitation, gust and below-zero wording
NOAA writes "Chance of precipitation is 60%." and "with gusts as high as 25 mph", and none of the precipitation or gust patterns allowed either, so the 🌦️ and 💨 details never appeared for real forecasts (the captured Miami, Anchorage and Denver forecasts all carry them). High/low extraction took only unsigned numbers and rejected Fahrenheit values under 20, so "Low around -5." or a high of 14 lost the high/low. Both now cover NOAA's range, -65 to 130 °F or -55 to 55 °C. The regenerated goldens differ only by the newly read details, which at tight budgets now take the place of pressure.
2026-10-01 23:08:02 -07:00
agessaman 43c3006072 Merge remote-tracking branch 'origin/refactor/weather-port' into feat/wx-gwx-convergence 2026-10-01 23:00:39 -07:00
agessaman 66176a173b refactor(weather): keep wx and gwx's companion-location DEBUG lines
The codex review of this port also found that the shared companion lookup dropped the three DEBUG lines wx and gwx logged on dev (no sender key, no row for the key, the position found). get_companion_lat_lon takes a trace flag that wx and gwx pass; rain and aurora, which never logged these, stay quiet. With trace, a bot without a db_manager logs the lookup error at the caller's level again instead of returning silently.
2026-10-01 23:00:14 -07:00
agessaman 45f2512acb Merge remote-tracking branch 'origin/feat/gwx-render-safe' into feat/wx-gwx-convergence
# Conflicts:
#	CHANGELOG.md
2026-10-01 22:53:29 -07:00
agessaman 4895da3f77 Merge remote-tracking branch 'origin/fix/weather-wind-emoji' into feat/wx-gwx-convergence
Goldens conflicted; I regenerated them, and against this branch they differ only in NOAA emojis, gwx wind labels and the Phoenix width budgets the merged branch corrected. gwx's new hourly test expected the old truncating conversion (202.5° as SW); with nearest-point directions it is SSW.
2026-10-01 22:50:34 -07:00
agessaman bce38407f2 test(gwx): check that a rendered gwx reply keeps its warning part 2026-10-01 22:47:02 -07:00
agessaman e9122b2dfd fix(weather): let rain and snow outrank cloud in NOAA emojis; skip non-finite wind degrees
A review of this branch found three things. Moving the partly/mostly cloudy check ahead of rain gave forecasts like "Chance Rain Showers then Mostly Cloudy" (ten of the 79 forecasts it sampled from live NWS points) a sun-behind-cloud emoji; rain and snow now come before every cloud check, so they get 🌦️ or ❄️. A NaN or infinite wind direction made gwx's new nearest-point conversion raise; it is now left out like a null one. The Phoenix daily-width test's budgets were written for a reply one byte shorter (SW is now WSW), so they no longer pinned <= at equality; they are 330/343 now and the regenerated golden shows each segment fitting exactly at its budget.
2026-10-01 22:46:47 -07:00
agessaman 37aabe5641 Merge remote-tracking branch 'origin/feat/gwx-hourly' into feat/wx-gwx-convergence
Conflicts: NOAA hourly wind keeps this branch's full 16-point direction and fix/noaa-configured-units' speed conversion (a test pins both); gwx's hourly forecast returns (reply, {}) like the other forecast types since fix/gwx-warnings-structured, and _open_meteo_current keeps fix/gwx-missing-current's Optional return.
2026-10-01 22:42:38 -07:00
agessaman 4ab2a9ae48 Merge remote-tracking branch 'origin/fix/gwx-warnings-structured' into feat/wx-gwx-convergence
fix/gwx-missing-current returns the fetch error as a bare string when the current temperature is missing, and this branch moved that code into a method whose callers unpack (reply, current conditions). Git merged it without a conflict; the merged early return now returns the pair, so the reply is the fetch error again instead of an unpacking error.
2026-10-01 22:32:56 -07:00
agessaman f4aaa7556f Merge remote-tracking branch 'origin/fix/gwx-missing-current' into feat/wx-gwx-convergence
The distorted-London golden conflicted; I regenerated it, and it differs from this branch only in the no_current and null_temp keys the merged change means to move.
2026-10-01 22:29:46 -07:00
agessaman f597205ce4 Merge remote-tracking branch 'origin/fix/noaa-configured-units' into feat/wx-gwx-convergence
The multi-day formatter from fix/weather-multiday-days still tested the period temperature for truthiness, so with both branches a 0° day (common in Celsius) dropped out of the multi-day forecast again. It now uses _has_temp like the older formatter.
2026-10-01 22:26:35 -07:00