bump to 1.17.1 and merge upstream

.github/workflows/build.yml

@@ -24,7 +24,9 @@ env:
   # Release artifacts are still built and attached, so a manual download and
   # flash works exactly as usual.
   #
-  # 1.17.0: normal release.
+  # 1.17.1: normal release. (1.17.0 was bumped but never pushed to master, so it
+  # was never published — 1.17.1 is the first release since 1.16.8 and its notes
+  # carry the 1.17.0 content forward.)
   prerelease: "false"
 
 jobs:

releasenotes/RELEASE_NOTES_1.17.1-zephcore.md

@@ -1,8 +1,12 @@
-# ZephCore 1.17.0-zephcore
+# ZephCore 1.17.1-zephcore
 
 A **hardware release**. A new tracker board lands — the Seeed SenseCAP MeshTracker X1 — and with it the
 LR2021 radio becomes a real, on-air-validated option instead of bring-up code. Plus GPS diagnostics, an
-LED master switch, and a batch of receive-path hardening.
+LED master switch, and a batch of receive- and transmit-path hardening.
+
+> [!NOTE]
+> **1.17.0 was never published**, so this release carries everything that was written up for it. If you
+> are coming from 1.16.8, everything below is new to you.
 
 ---
 
@@ -35,6 +39,41 @@ LED master switch, and a batch of receive-path hardening.
 
 ---
 
+## Slow presets no longer lose long packets
+
+The biggest fix in this release, and it is invisible until you run a narrow bandwidth. Transmission was
+bounded by two fixed deadlines that are shorter than the airtime of many perfectly legal presets:
+
+- The **chip's own transmit timer** was pinned at 10 s. The SX126x datasheet is explicit that when this
+  timer fires the transmission is *stopped* — so on slow presets it was not a safeguard, it was a
+  truncation. A full-size packet is 17.7 s at SF10/BW31.25 and 28.6 s at SF12/BW62.5; those presets
+  were cutting every packet from ~136 bytes and ~76 bytes up, mid-air.
+- The **host's wait for "transmit finished"** was a fixed 5 s, after which the radio was yanked back
+  into receive — killing a transmission that was going perfectly well.
+
+Both now scale from the driver's own airtime calculation, floored at the old values so nothing that
+works today gets a tighter deadline. Only slow presets move. There was nothing in the log connecting
+the loss to either line, so if you have ever run SF10–SF12 at 31.25 or 62.5 kHz and seen packets simply
+not arrive, this is likely why.
+
+Alongside it: a driver that **reports a failed transmit** is now believed. Previously such a packet was
+counted as sent, so the statistics disagreed with reality.
+
+## Replies no longer dropped when `flood.max.unscoped` is low
+
+Only affects operators who have lowered `flood.max.unscoped` from its default of 64 — but on those
+meshes it looked like repeaters and room servers were ignoring requests. Two separate causes:
+
+- A reply to a **DIRECT** request went out **un-scoped**, because a direct request carries no transport
+  codes and there was no fallback. Any repeater on the return path running `flood.max.unscoped=0` then
+  dropped it at hop 0. Replies now fall back to the node's own default region scope.
+- A repeater answering a **direct login** flooded the reply even when it already had a return path
+  stored for that client. It now replies directly along that path.
+
+Ported from upstream MeshCore, with one local difference: ZephCore can tell "the request arrived
+un-scoped" apart from "the request was direct", so a requester who reached you un-scoped still gets an
+un-scoped reply rather than a scoped one they might not hear.
+
 ## New board — Seeed SenseCAP MeshTracker X1
 
 `meshtracker_x1` — nRF52840 + **Semtech LR2021**, 863–928 MHz / 22 dBm, dual-band L1+L5 GNSS,
@@ -106,14 +145,6 @@ it ever fires. The SX126x got the same fix in 1.16.x.
 Related: an error-only interrupt on the LR2021 no longer triggers a receive restart that regenerates
 the same error, which on the X1 could cost ~88 ms of deafness and a hardware reset.
 
-## Adaptive CAD now uses only levels the chip can distinguish
-
-The auto-tuning channel-detection threshold walks a range of levels, but where those levels fell
-outside the radio's own hardware limits, several of them programmed the *identical* setting — so the
-tuner was comparing physically identical rungs and reading noise as a trend. The usable window is now
-derived per-driver, so every level is a distinct configuration and the `pk` value in `get cad` is what
-the chip actually got. This mattered most on the LR2021.
-
 ## Virtual-contact fixes (companion)
 
 The built-in admin chat contact got three fixes worth knowing about:
@@ -125,6 +156,14 @@ The built-in admin chat contact got three fixes worth knowing about:
 - **"Last seen" no longer grows forever**, and the contact re-syncs to the app properly instead of
   being sent exactly once ever.
 
+## Adaptive CAD now uses only levels the chip can distinguish
+
+The auto-tuning channel-detection threshold walks a range of levels, but where those levels fell
+outside the radio's own hardware limits, several of them programmed the *identical* setting — so the
+tuner was comparing physically identical rungs and reading noise as a trend. The usable window is now
+derived per-driver, so every level is a distinct configuration and the `pk` value in `get cad` is what
+the chip actually got. This mattered most on the LR2021.
+
 ## GPS: dynamic model and a diagnostics readout
 
 - **`CONFIG_ZEPHCORE_GPS_NAV_MODE`** sets the dynamic model on CASIC modules (Quectel L76K / L76KB,
@@ -156,7 +195,13 @@ generally.
 
 ## Other changes
 
-- **Zephyr pinned to 4.4.2 final.**
+- **Zephyr pin advanced** past 4.4.2, and **Monocypher updated to 4.0.3**.
+- **Sensors behind a switched power rail now come up.** A part on a rail that is powered late cannot be
+  probed during early boot, and the rail's declared startup delay never actually runs. Such sensors are
+  now initialised on first use instead, once the rail has had the whole boot to settle. This is what
+  the X1's barometer and RTC needed; the same handling is now in place for every environment sensor.
+- **`get dc.restarts` works on companions.** It reported 0 forever regardless of what the radio was
+  doing — and the companion is the role RX duty cycle actually runs in. `clear stats` resets it.
 - **ESP32: the user button now wakes the node from light sleep** on boards using the shared PM overlay
   — previously only an incoming packet would.
 - **`get freqerr`** (LR2021 only) reports carrier frequency error measured on received packets. Purely
@@ -179,3 +224,5 @@ generally.
   which goes high-impedance on reset, so the chip raises its power-loss flag and the clock is re-set
   from the next GPS/app/CLI sync. Time is still correct in normal operation; it just isn't preserved
   across a restart.
+- **The scoped-reply and transmit-deadline fixes are not on-air validated** — both are build-verified
+  and reasoned from the datasheet, not measured on a live mesh.

zephcore/CMakeLists.txt

@@ -525,7 +525,7 @@ add_definitions(-DFIRMWARE_BUILD_EPOCH=${ZEPHCORE_BUILD_EPOCH}u)
 # release is tagged/named, and what the Mesh America catalog uses as its version
 # key — so the configurator can match a running device against the catalog. Keep
 # all four identical; the release workflow reads this value directly.
-set(ZEPHCORE_FIRMWARE_VERSION "1.17.0-zephcore")
+set(ZEPHCORE_FIRMWARE_VERSION "1.17.1-zephcore")
 add_definitions(-DFIRMWARE_VERSION="${ZEPHCORE_FIRMWARE_VERSION}")
 
 add_subdirectory(lib/monocypher)

zephcore/app/RepeaterMesh.cpp

@@ -563,26 +563,32 @@ void RepeaterMesh::sendFloodScoped(const TransportKey& scope, mesh::Packet* pkt,
 }
 
 void RepeaterMesh::sendFloodReply(mesh::Packet* packet, unsigned long delay_millis, uint8_t path_hash_size) {
-    if (recv_pkt_region && !recv_pkt_region->isWildcard()) {  // if _request_ packet scope is known, send reply with same scope
-        TransportKey scope;
-        if (region_map.getTransportKeysFor(*recv_pkt_region, &scope, 1) > 0) {
-            sendFloodScoped(scope, packet, delay_millis, path_hash_size);
-        } else {
-            sendFlood(packet, delay_millis, path_hash_size);  // send un-scoped
-        }
-    } else {
+    TransportKey req_scope;
+    bool req_scope_known = recv_pkt_region != nullptr && !recv_pkt_region->isWildcard()
+                        && region_map.getTransportKeysFor(*recv_pkt_region, &req_scope, 1) > 0;
+
+    switch (mesh::chooseReplyScope(req_scope_known, recv_pkt_unscoped_flood, !default_scope.isNull())) {
+    case mesh::REPLY_SCOPE_REQUEST:
+        sendFloodScoped(req_scope, packet, delay_millis, path_hash_size);  // same scope as the request
+        break;
+    case mesh::REPLY_SCOPE_DEFAULT:
+        // requester's scope is unknown: a DIRECT request (no transport codes), or a
+        // code that matched no Region. Un-scoped would be dropped at hop 0 by every
+        // repeater running flood.max.unscoped=0.
+        sendFloodScoped(default_scope, packet, delay_millis, path_hash_size);
+        break;
+    case mesh::REPLY_SCOPE_NONE:
         sendFlood(packet, delay_millis, path_hash_size);  // send un-scoped
+        break;
     }
 }
 
 bool RepeaterMesh::allowPacketForward(const mesh::Packet* packet) {
     if (_prefs.disable_fwd) return false;
-    if (packet->isRouteFlood()) {
-        if (packet->getPathHashCount() >= _prefs.flood_max) return false;
-        // un-scoped floods can be clamped to a lower hop limit than scoped (transport) floods
-        if (packet->getRouteType() == ROUTE_TYPE_FLOOD && packet->getPathHashCount() >= _prefs.flood_max_unscoped) return false;
-        // ADVERT floods get their own (typically tighter) hop ceiling to curb advert churn
-        if (packet->getPayloadType() == PAYLOAD_TYPE_ADVERT && packet->getPathHashCount() >= _prefs.flood_max_advert) return false;
+    if (packet->isRouteFlood()
+        && mesh::isFloodHopLimitExceeded(packet, _prefs.flood_max, _prefs.flood_max_unscoped,
+                                         _prefs.flood_max_advert)) {
+        return false;
     }
     if (packet->isRouteFlood() && recv_pkt_region == nullptr) return false;
     if (packet->isRouteFlood() && _prefs.loop_detect != LOOP_DETECT_OFF) {
@@ -676,6 +682,7 @@ mesh::DispatcherAction RepeaterMesh::onRecvPacket(mesh::Packet* pkt) {
     // Determine the request packet's region so sendFloodReply() can echo the same
     // scope. Runs for every packet (not just floods) so recv_pkt_region is cleared
     // for direct packets instead of inheriting the last flood's region.
+    recv_pkt_unscoped_flood = (pkt->getRouteType() == ROUTE_TYPE_FLOOD);
     if (pkt->getRouteType() == ROUTE_TYPE_TRANSPORT_FLOOD) {
         recv_pkt_region = region_map.findMatch(pkt, REGION_DENY_FLOOD);
     } else if (pkt->getRouteType() == ROUTE_TYPE_FLOOD) {
@@ -713,16 +720,37 @@ void RepeaterMesh::onAnonDataRecv(mesh::Packet* packet, const uint8_t* secret, c
 
         if (reply_len == 0) return;
 
-        if (packet->isRouteFlood()) {
+        /* A DIRECT request that supplied no reply path can still be answered
+         * along the out_path already stored for this client, as
+         * onPeerDataRecv() does for REQ. Flooding it instead is both wasteful
+         * and — under flood.max.unscoped=0 — silently undeliverable. */
+        ClientInfo* client = acl.getClient(sender.pub_key, PUB_KEY_SIZE);
+        bool have_out_path = client != nullptr && client->out_path_len != OUT_PATH_UNKNOWN;
+
+        switch (mesh::chooseReplyRoute(packet->isRouteFlood(),
+                                       reply_path_len != OUT_PATH_UNKNOWN, have_out_path)) {
+        case mesh::REPLY_ROUTE_PATH_RETURN: {
+            // let this sender know the path TO here, so they can use sendDirect() later
             mesh::Packet* path = createPathReturn(sender, secret, packet->path, packet->path_len,
                                                   PAYLOAD_TYPE_RESPONSE, reply_data, reply_len);
             if (path) sendFloodReply(path, SERVER_RESPONSE_DELAY, packet->getPathHashSize());
-        } else if (reply_path_len == OUT_PATH_UNKNOWN) {
-            mesh::Packet* reply = createDatagram(PAYLOAD_TYPE_RESPONSE, sender, secret, reply_data, reply_len);
-            if (reply) sendFloodReply(reply, SERVER_RESPONSE_DELAY, packet->getPathHashSize());
-        } else {
+            break;
+        }
+        case mesh::REPLY_ROUTE_DIRECT_SUPPLIED: {
             mesh::Packet* reply = createDatagram(PAYLOAD_TYPE_RESPONSE, sender, secret, reply_data, reply_len);
             if (reply) sendDirect(reply, reply_path, reply_path_len, SERVER_RESPONSE_DELAY);
+            break;
+        }
+        case mesh::REPLY_ROUTE_DIRECT_OUT_PATH: {
+            mesh::Packet* reply = createDatagram(PAYLOAD_TYPE_RESPONSE, sender, secret, reply_data, reply_len);
+            if (reply) sendDirect(reply, client->out_path, client->out_path_len, SERVER_RESPONSE_DELAY);
+            break;
+        }
+        case mesh::REPLY_ROUTE_FLOOD: {
+            mesh::Packet* reply = createDatagram(PAYLOAD_TYPE_RESPONSE, sender, secret, reply_data, reply_len);
+            if (reply) sendFloodReply(reply, SERVER_RESPONSE_DELAY, packet->getPathHashSize());
+            break;
+        }
         }
     }
 }
@@ -969,6 +997,7 @@ RepeaterMesh::RepeaterMesh(mesh::MainBoard& board, mesh::Radio& radio, mesh::Mil
     _logging = false;
     region_load_active = false;
     recv_pkt_region = nullptr;
+    recv_pkt_unscoped_flood = false;
     memset(default_scope.key, 0, sizeof(default_scope.key));
     pending_discover_tag = 0;
     pending_discover_until = 0;

zephcore/app/RepeaterMesh.h

@@ -20,6 +20,7 @@
 #include <helpers/CommonCLI.h>
 #include <helpers/MeshTimeSync.h>
 #include <helpers/RegionMap.h>
+#include <helpers/RoutingPolicy.h>
 #include <helpers/TransportKeyStore.h>
 #include <helpers/RateLimiter.h>
 #include <helpers/StatsFormatHelper.h>
@@ -98,6 +99,11 @@ class RepeaterMesh : public mesh::Mesh, public CommonCLICallbacks {
     RegionMap region_map, temp_map;
     RegionEntry* load_stack[8];
     RegionEntry* recv_pkt_region;
+    /* A null recv_pkt_region has two meanings — a DIRECT request (no transport
+     * codes at all) and an un-scoped flood our wildcard Region denies — and
+     * sendFloodReply() must treat them differently, so record the route type
+     * rather than inferring it from the pointer. */
+    bool recv_pkt_unscoped_flood;
     TransportKey default_scope;
     RateLimiter discover_limiter, anon_limiter, login_fail_limiter;
     uint32_t pending_discover_tag;

zephcore/app/RoomServerMesh.cpp

@@ -346,15 +346,23 @@ void RoomServerMesh::sendFloodScoped(const TransportKey& scope, mesh::Packet* pk
 }
 
 void RoomServerMesh::sendFloodReply(mesh::Packet* packet, unsigned long delay_millis, uint8_t path_hash_size) {
-    if (recv_pkt_region && !recv_pkt_region->isWildcard()) {  // if _request_ packet scope is known, send reply with same scope
-        TransportKey scope;
-        if (region_map.getTransportKeysFor(*recv_pkt_region, &scope, 1) > 0) {
-            sendFloodScoped(scope, packet, delay_millis, path_hash_size);
-        } else {
-            sendFlood(packet, delay_millis, path_hash_size);  // send un-scoped
-        }
-    } else {
+    TransportKey req_scope;
+    bool req_scope_known = recv_pkt_region != nullptr && !recv_pkt_region->isWildcard()
+                        && region_map.getTransportKeysFor(*recv_pkt_region, &req_scope, 1) > 0;
+
+    switch (mesh::chooseReplyScope(req_scope_known, recv_pkt_unscoped_flood, !default_scope.isNull())) {
+    case mesh::REPLY_SCOPE_REQUEST:
+        sendFloodScoped(req_scope, packet, delay_millis, path_hash_size);  // same scope as the request
+        break;
+    case mesh::REPLY_SCOPE_DEFAULT:
+        // requester's scope is unknown: a DIRECT request (no transport codes), or a
+        // code that matched no Region. Un-scoped would be dropped at hop 0 by every
+        // repeater running flood.max.unscoped=0.
+        sendFloodScoped(default_scope, packet, delay_millis, path_hash_size);
+        break;
+    case mesh::REPLY_SCOPE_NONE:
         sendFlood(packet, delay_millis, path_hash_size);  // send un-scoped
+        break;
     }
 }
 
@@ -432,6 +440,7 @@ mesh::DispatcherAction RoomServerMesh::onRecvPacket(mesh::Packet* pkt) {
     // Determine the request packet's region so sendFloodReply() can echo the same
     // scope. Runs for every packet (not just floods) so recv_pkt_region is cleared
     // for direct packets instead of inheriting the last flood's region.
+    recv_pkt_unscoped_flood = (pkt->getRouteType() == ROUTE_TYPE_FLOOD);
     if (pkt->getRouteType() == ROUTE_TYPE_TRANSPORT_FLOOD) {
         recv_pkt_region = region_map.findMatch(pkt, REGION_DENY_FLOOD);
     } else if (pkt->getRouteType() == ROUTE_TYPE_FLOOD) {
@@ -759,6 +768,7 @@ RoomServerMesh::RoomServerMesh(mesh::MainBoard& board, mesh::Radio& radio, mesh:
     _logging = false;
     region_load_active = false;
     recv_pkt_region = nullptr;
+    recv_pkt_unscoped_flood = false;
     memset(default_scope.key, 0, sizeof(default_scope.key));
 
     initNodePrefs(&_prefs);

zephcore/app/RoomServerMesh.h

@@ -21,6 +21,7 @@
 #include <helpers/CommonCLI.h>
 #include <helpers/MeshTimeSync.h>
 #include <helpers/RegionMap.h>
+#include <helpers/RoutingPolicy.h>
 #include <helpers/TransportKeyStore.h>
 #include <helpers/RateLimiter.h>
 #include <helpers/StatsFormatHelper.h>
@@ -79,6 +80,11 @@ class RoomServerMesh : public mesh::Mesh, public CommonCLICallbacks {
     RegionMap region_map, temp_map;
     RegionEntry* load_stack[8];
     RegionEntry* recv_pkt_region;
+    /* A null recv_pkt_region has two meanings — a DIRECT request (no transport
+     * codes at all) and an un-scoped flood our wildcard Region denies — and
+     * sendFloodReply() must treat them differently, so record the route type
+     * rather than inferring it from the pointer. */
+    bool recv_pkt_unscoped_flood;
     TransportKey default_scope;
     RateLimiter login_fail_limiter;
     bool region_load_active;

zephcore/boards/common/zephcore_common.conf

@@ -156,7 +156,7 @@ CONFIG_BT_DIS_FW_REV=y
 # of truth for the C side, injected as -DFIRMWARE_VERSION). This Kconfig value can't
 # read a C macro, so it must be bumped here too. Phones that read DIS and phones
 # that query CMD_DEVICE_QUERY should see the same version.
-CONFIG_BT_DIS_FW_REV_STR="1.17.0-zephcore"
+CONFIG_BT_DIS_FW_REV_STR="1.17.1-zephcore"
 CONFIG_BT_DIS_SW_REV=y
 CONFIG_BT_DIS_SW_REV_STR="Zephyr"
 CONFIG_BT_DIS_PNP=n

zephcore/helpers/RoutingPolicy.h

@@ -0,0 +1,91 @@
+/*
+ * SPDX-License-Identifier: MIT
+ * ZephCore RoutingPolicy - flood hop limits and server reply routing
+ *
+ * Ported from Arduino MeshCore fad11c90 ("Fix replies dropped when
+ * flood.max.unscoped is low", PR #3106).  Pure decision logic, no I/O: the
+ * repeater and the room server share it so the two roles cannot drift.
+ */
+
+#pragma once
+
+#include <mesh/Packet.h>
+#include <stdint.h>
+
+namespace mesh {
+
+/**
+ * Test a flood packet against the configured hop limits.
+ *
+ * @param packet              inbound flood packet (caller has already checked isRouteFlood())
+ * @param flood_max           hop ceiling for any flood packet
+ * @param flood_max_unscoped  hop ceiling for un-scoped (ROUTE_TYPE_FLOOD) packets;
+ *                            may be clamped lower than scoped/transport floods
+ * @param flood_max_advert    hop ceiling for ADVERT floods, typically tighter still
+ *                            so advert churn stays local
+ * @returns true if a limit is exceeded and the packet must not be forwarded
+ */
+inline bool isFloodHopLimitExceeded(const Packet *packet, uint8_t flood_max,
+				    uint8_t flood_max_unscoped, uint8_t flood_max_advert)
+{
+	uint8_t hops = packet->getPathHashCount();
+
+	if (hops >= flood_max) return true;
+	if (packet->getRouteType() == ROUTE_TYPE_FLOOD && hops >= flood_max_unscoped) return true;
+	if (packet->getPayloadType() == PAYLOAD_TYPE_ADVERT && hops >= flood_max_advert) return true;
+	return false;
+}
+
+/** How a server routes a reply back to the requesting client. */
+enum ReplyRoute : uint8_t {
+	REPLY_ROUTE_PATH_RETURN,      // request arrived by flood: reply with a PATH return, flooded back
+	REPLY_ROUTE_DIRECT_SUPPLIED,  // reply DIRECT, along the return path supplied in the request
+	REPLY_ROUTE_DIRECT_OUT_PATH,  // reply DIRECT, along the out_path already stored for this client
+	REPLY_ROUTE_FLOOD,            // no return path known: flood the reply
+};
+
+/**
+ * @param inbound_is_flood    the request arrived as a flood packet
+ * @param have_supplied_path  the request payload carried an explicit reply path
+ * @param have_out_path       this server already holds a stored out_path for the client
+ */
+inline ReplyRoute chooseReplyRoute(bool inbound_is_flood, bool have_supplied_path,
+				   bool have_out_path)
+{
+	if (inbound_is_flood) return REPLY_ROUTE_PATH_RETURN;
+	if (have_supplied_path) return REPLY_ROUTE_DIRECT_SUPPLIED;
+	if (have_out_path) return REPLY_ROUTE_DIRECT_OUT_PATH;
+	return REPLY_ROUTE_FLOOD;
+}
+
+/** Which transport scope a flooded reply should carry. */
+enum ReplyScope : uint8_t {
+	REPLY_SCOPE_REQUEST,  // re-use the scope the request arrived on
+	REPLY_SCOPE_DEFAULT,  // fall back to this node's default region scope
+	REPLY_SCOPE_NONE,     // send un-scoped (ROUTE_TYPE_FLOOD)
+};
+
+/**
+ * @param request_scope_known         request arrived scoped, and its Region's transport key resolved
+ * @param request_was_unscoped_flood  request arrived as an un-scoped flood
+ * @param default_scope_known         this node has a default Region with a usable transport key
+ */
+inline ReplyScope chooseReplyScope(bool request_scope_known, bool request_was_unscoped_flood,
+				   bool default_scope_known)
+{
+	if (request_scope_known) return REPLY_SCOPE_REQUEST;
+
+	/* The requester chose un-scoped, so mirror it.  Replying scoped would
+	 * change a path that works today, and repeaters not holding our default
+	 * Region would drop it anyway. */
+	if (request_was_unscoped_flood) return REPLY_SCOPE_NONE;
+
+	/* Scope unknowable: a DIRECT request carries no transport codes, or the
+	 * code matched no Region.  Un-scoped would be dropped at hop 0 by every
+	 * repeater running flood.max.unscoped=0, so use our own default. */
+	if (default_scope_known) return REPLY_SCOPE_DEFAULT;
+
+	return REPLY_SCOPE_NONE;
+}
+
+}