diff --git a/docs/cli_commands.md b/docs/cli_commands.md index 754f51201..1488c646e 100644 --- a/docs/cli_commands.md +++ b/docs/cli_commands.md @@ -1247,6 +1247,13 @@ Covers serial NMEA receivers (`MicroNMEALocationProvider`). --- +### Repeater health history + +History is queried over the mesh using the sensor-compatible `0x04` min/max/average +request; there is no history CLI command. Retention and recording interval are controlled +by `REPEATER_HISTORY_SLOTS` and `REPEATER_HISTORY_INTERVAL_SECS` at build time. +See [Repeater health history](repeater_history.md) for the channel mapping and flags. + ### Ethernet (when Ethernet support is compiled in) Ethernet support is available on RAK4631 boards with a RAK13800 (W5100S) Ethernet module. Use the `_ethernet` firmware variants (e.g. `RAK_4631_repeater_ethernet`) to enable this feature. diff --git a/docs/payloads.md b/docs/payloads.md index 493f628ce..c60a78d6c 100644 --- a/docs/payloads.md +++ b/docs/payloads.md @@ -103,7 +103,7 @@ For the common chat/server helpers in `BaseChatMesh`, the current request type v | `0x01` | get stats | get stats of repeater or room server | | `0x02` | keepalive | (deprecated) | | `0x03` | get telemetry | request node telemetry | -| `0x04` | get min/max/avg | get sensor node stats on time series data | +| `0x04` | get min/max/avg | get sensor or repeater time series statistics | | `0x05` | get acl | node ACL query | | `0x06` | get neighbors | node neighbors query | | `0x08` | subscribe | subscribe to telemetry push | @@ -140,7 +140,7 @@ Gets information about the node, possibly including the following: | req type | 1 | 0x03 (request sub type) | | permission mask | 1 | bitwise inverse mask to AND to permissions (0 = get ALL telem values) | -#### Get Min/Max/Ave (Sensor nodes) +#### Get Min/Max/Ave (Sensor and repeater nodes) | Field | Size (bytes) | Description | |----------------|--------------|------------------------------| @@ -150,6 +150,28 @@ Gets information about the node, possibly including the following: | end | 4 | ending time, seconds ago | | reserved | 2 | should be zeroes | +Requires an authenticated client with the READ_ONLY role or higher. The range includes +samples whose age is `end <= age < start`, in seconds. Request integers are little-endian. +The repeater uses exactly the sensor request and response layout. + +The response body (after decryption) is: + +| Field | Size (bytes) | Description | +|----------------|--------------|-------------| +| tag | 4 | echoed request timestamp, little-endian | +| current time | 4 | current node RTC Unix time, little-endian | +| series | variable | up to 8 series records, described below | + +Each series record contains a channel byte, an LPP type byte, then min, max, and average +in that order. Each value uses the type's Cayenne LPP size, multiplier, sign, and +big-endian encoding. There is no count byte; parse records to the end of the response. +Nonzero reserved bytes return only the tag and current time. + +Repeater history is pull-only, uses uptime internally, and omits series with no valid +samples in the requested range. Its returned current time still comes from the RTC; +an unset or corrected RTC does not affect sampling or range selection. See +[Repeater health history](repeater_history.md) for channels, retention, and build flags. + #### Subscribe to Telemetry push - (Sensor nodes) | Field | Size (bytes) | Description | diff --git a/docs/repeater_history.md b/docs/repeater_history.md new file mode 100644 index 000000000..35b37d820 --- /dev/null +++ b/docs/repeater_history.md @@ -0,0 +1,84 @@ +# Repeater health history + +The simple repeater keeps health samples in RAM and answers mesh request +`REQ_TYPE_GET_AVG_MIN_MAX` (`0x04`). A logged-in client needs READ_ONLY access or +higher. This is pull-only: recording history sends no packets. Existing GET_STATUS +and telemetry responses are unchanged. + +## Series + +| Channel | LPP type | Value recorded each interval | +|---------|----------|------------------------------| +| 200 | `LPP_VOLTAGE` (116) | Battery voltage in V, averaged from periodic reads every 15 seconds | +| 201 | `LPP_ANALOG_INPUT` (2, signed) | Radio noise floor in dBm at the end of the interval | +| 202 | `LPP_PERCENTAGE` (120) | TX airtime percentage over the elapsed interval | +| 203 | `LPP_PERCENTAGE` (120) | RX airtime percentage over the elapsed interval | +| 204 | `LPP_TEMPERATURE` (103) | MCU temperature in degrees C, when supported | + +These channels are dedicated to history. Existing self telemetry and GPS use channel 1; +`EnvironmentSensorManager` assigns at most 16 active entries to channels 2–17. Channels +200–204 therefore do not collide with current repeater telemetry. Custom sensor managers +should leave this range free. History channels do not change the channels in `0x03`. + +Battery averaging reduces the effect of short voltage dips during transmission. Noise +floor and temperature are point samples. Recording advances on fixed interval boundaries +without accumulating loop delays; the airtime denominator uses a separate actual +measurement timestamp. TX/RX values are unsigned counter deltas in +milliseconds divided by the actual elapsed interval, multiplied by 100. Percentages +use the sensor protocol's integer LPP encoding, so fractional percentages are truncated. +NaN temperatures and the radio driver's uncalibrated noise-floor sentinel (0 dBm) +are skipped. At most five series are returned, below the eight-series reply limit. + +## Query and retention + +Use the [sensor-compatible `0x04` request](payloads.md#get-minmaxave-sensor-and-repeater-nodes): +start and end are seconds ago, followed by two zero reserved bytes. For example, +`start=3600, end=0` selects samples from the last hour, including age zero and excluding +age 3600. The reply echoes the request timestamp, adds the current RTC time, and then +encodes each series as channel, LPP type, min, max, average. There is no series count. +Empty series are omitted; an empty range returns just the eight-byte header. + +Sampling and queries use monotonic uptime, accumulated across `millis()` wraps. The +RTC is used only for the response timestamp. Samples occupy a fixed interval grid +anchored at the first valid sample. Missed intervals are empty (NaN) slots; a gap +at least as long as the ring capacity expires the old ring. Late readings are assigned +to their interval boundary, so their reported age can differ by less than one interval. +Only one timestamp is stored per series. +Unsigned second differences handle seconds-counter wrap; retained samples and query +ranges must span less than one full 32-bit seconds cycle (about 136 years). + +The first sample is recorded after one interval. Each series retains the configured +number of interval slots, including missing intervals. History is lost on reboot and +is not written to flash. + +## Build flags + +| Flag | Default | Description | +|------|---------|-------------| +| `REPEATER_HISTORY_INTERVAL_SECS` | `300` | Positive recording interval in seconds | +| `REPEATER_HISTORY_SLOTS` | `288`, or `0` on STM32/RP2040 | Interval slots per series; nominally 24 hours at five-minute intervals when enabled | + +Set these using PlatformIO `build_flags`, for example `-D REPEATER_HISTORY_SLOTS=144` +for nominally 12 hours. `-D REPEATER_HISTORY_SLOTS=0` removes history members, +allocations, sampling, and handling of `0x04` entirely. STM32 and RP2040 builds default +to disabled history to conserve RAM; they can explicitly opt in with a smaller slot count. +There are no history CLI settings. + +For both nRF52 and ESP32, sample storage is `5 * slots * 4` bytes (one float per +interval), or 5,760 bytes at the defaults. Comparing enabled and disabled builds shows +168 additional bytes in the enclosing mesh object on both platforms (including padding), +for a total of 5,928 bytes plus allocator overhead for five arrays. Unsupported temperature still reserves +its series storage. Query results use temporary stack space, not retained history RAM. + +## Custom sensor sketch compatibility + +The old `examples/simple_sensor/TimeSeriesData.h` include forwards to the shared helper. +Deprecated `RTCClock*` overloads remain available for existing custom sketches. They +ignore the RTC and use a shared, wrap-safe `millis()` accumulator; callers should migrate +to explicit monotonic seconds. Call the legacy API regularly enough to observe each +`millis()` wrap (less than about 49 days between calls), and use one clock API consistently +for each series. The shared legacy clock is linked only when these overloads are used. + +The integer literal `0` is ambiguous between the seconds and pointer overloads. Use a +`uint32_t` variable (or `uint32_t(0)`) for uptime zero, and `nullptr` for a legacy null +clock argument. diff --git a/examples/simple_repeater/MyMesh.cpp b/examples/simple_repeater/MyMesh.cpp index b50d19d01..99cd35bba 100644 --- a/examples/simple_repeater/MyMesh.cpp +++ b/examples/simple_repeater/MyMesh.cpp @@ -1,5 +1,8 @@ #include "MyMesh.h" #include +#if REPEATER_HISTORY_SLOTS > 0 +#include +#endif /* ------------------------------ Config -------------------------------- */ @@ -46,6 +49,9 @@ #define REQ_TYPE_GET_STATUS 0x01 // same as _GET_STATS #define REQ_TYPE_KEEP_ALIVE 0x02 #define REQ_TYPE_GET_TELEMETRY_DATA 0x03 +#if REPEATER_HISTORY_SLOTS > 0 +#define REQ_TYPE_GET_AVG_MIN_MAX 0x04 +#endif #define REQ_TYPE_GET_ACCESS_LIST 0x05 #define REQ_TYPE_GET_NEIGHBOURS 0x06 #define REQ_TYPE_GET_OWNER_INFO 0x07 // FIRMWARE_VER_LEVEL >= 2 @@ -209,6 +215,7 @@ uint8_t MyMesh::handleAnonClockReq(const mesh::Identity& sender, uint32_t sender } int MyMesh::handleRequest(ClientInfo *sender, uint32_t sender_timestamp, uint8_t *payload, size_t payload_len) { + if (payload_len == 0) return 0; // uint32_t now = getRTCClock()->getCurrentTimeUnique(); // memcpy(reply_data, &now, 4); // response packets always prefixed with timestamp memcpy(reply_data, &sender_timestamp, 4); // reflect sender_timestamp back in response packet (kind of like a 'tag') @@ -259,6 +266,48 @@ int MyMesh::handleRequest(ClientInfo *sender, uint32_t sender_timestamp, uint8_t memcpy(&reply_data[4], telemetry.getBuffer(), tlen); return 4 + tlen; // reply_len } +#if REPEATER_HISTORY_SLOTS > 0 + if (payload[0] == REQ_TYPE_GET_AVG_MIN_MAX && payload_len >= 11 && + (sender->permissions & PERM_ACL_ROLE_MASK) >= PERM_ACL_READ_ONLY) { + uint32_t start_secs_ago, end_secs_ago; + memcpy(&start_secs_ago, &payload[1], 4); + memcpy(&end_secs_ago, &payload[5], 4); + MinMaxAvg data[8]; + int n = 0; + if (payload[9] == 0 && payload[10] == 0) { + uint32_t now_secs = uptime_millis / 1000; + battery_history.calcMinMaxAvg(now_secs, start_secs_ago, end_secs_ago, &data[n++], HISTORY_BATTERY, + LPP_VOLTAGE); + noise_history.calcMinMaxAvg(now_secs, start_secs_ago, end_secs_ago, &data[n++], HISTORY_NOISE, + LPP_ANALOG_INPUT); + tx_history.calcMinMaxAvg(now_secs, start_secs_ago, end_secs_ago, &data[n++], HISTORY_TX, + LPP_PERCENTAGE); + rx_history.calcMinMaxAvg(now_secs, start_secs_ago, end_secs_ago, &data[n++], HISTORY_RX, + LPP_PERCENTAGE); + temperature_history.calcMinMaxAvg(now_secs, start_secs_ago, end_secs_ago, &data[n], HISTORY_TEMPERATURE, + LPP_TEMPERATURE); + // Unsupported temperature sensors and empty ranges have no encodable LPP value. + if (!isnan(data[n]._avg)) n++; + } + uint8_t ofs = 4; + uint32_t now = getRTCClock()->getCurrentTime(); + memcpy(&reply_data[ofs], &now, 4); + ofs += 4; + for (int i = 0; i < n; i++) { + auto d = &data[i]; + if (isnan(d->_avg)) continue; + reply_data[ofs++] = d->_channel; + reply_data[ofs++] = d->_lpp_type; + uint8_t sz = LPPData::getDataSize(d->_lpp_type); + uint32_t mult = LPPData::getMultiplier(d->_lpp_type); + bool is_signed = LPPData::isSigned(d->_lpp_type); + ofs += LPPData::putFloat(&reply_data[ofs], d->_min, sz, mult, is_signed); + ofs += LPPData::putFloat(&reply_data[ofs], d->_max, sz, mult, is_signed); + ofs += LPPData::putFloat(&reply_data[ofs], d->_avg, sz, mult, is_signed); + } + return ofs; + } +#endif if (payload[0] == REQ_TYPE_GET_ACCESS_LIST && sender->isAdmin()) { uint8_t res1 = payload[1]; // reserved for future (extra query params) uint8_t res2 = payload[2]; @@ -862,6 +911,13 @@ void MyMesh::sendNodeDiscoverReq() { MyMesh::MyMesh(mesh::MainBoard &board, mesh::Radio &radio, mesh::MillisecondClock &ms, mesh::RNG &rng, mesh::RTCClock &rtc, mesh::MeshTables &tables) : mesh::Mesh(radio, ms, rng, rtc, *new StaticPoolPacketManager(32), tables), +#if REPEATER_HISTORY_SLOTS > 0 + battery_history(REPEATER_HISTORY_SLOTS, REPEATER_HISTORY_INTERVAL_SECS), + noise_history(REPEATER_HISTORY_SLOTS, REPEATER_HISTORY_INTERVAL_SECS), + tx_history(REPEATER_HISTORY_SLOTS, REPEATER_HISTORY_INTERVAL_SECS), + rx_history(REPEATER_HISTORY_SLOTS, REPEATER_HISTORY_INTERVAL_SECS), + temperature_history(REPEATER_HISTORY_SLOTS, REPEATER_HISTORY_INTERVAL_SECS), +#endif region_map(key_store), temp_map(key_store), _cli(board, rtc, sensors, region_map, acl, &_prefs, this), telemetry(MAX_PACKET_PAYLOAD - 4), @@ -876,6 +932,12 @@ MyMesh::MyMesh(mesh::MainBoard &board, mesh::Radio &radio, mesh::MillisecondCloc { last_millis = 0; uptime_millis = 0; +#if REPEATER_HISTORY_SLOTS > 0 + history_last_secs = history_battery_secs = history_airtime_secs = 0; + history_tx_millis = history_rx_millis = 0; + history_battery_total = 0; + history_battery_count = 0; +#endif next_local_advert = next_flood_advert = 0; dirty_contacts_expiry = 0; set_radio_at = revert_radio_at = 0; @@ -1286,7 +1348,48 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply } } +#if REPEATER_HISTORY_SLOTS > 0 +void MyMesh::recordHistory() { + uint32_t now_secs = uptime_millis / 1000; + // Average periodic reads without repeatedly enabling the battery divider. + if (uint32_t(now_secs - history_battery_secs) >= 15) { + history_battery_total += board.getBattMilliVolts(); + history_battery_count++; + history_battery_secs = now_secs; + } + uint32_t elapsed_secs = now_secs - history_last_secs; + if (elapsed_secs < REPEATER_HISTORY_INTERVAL_SECS) return; + // Advance the sampling grid without accumulating loop delays or replaying missed slots. + history_last_secs += (elapsed_secs / REPEATER_HISTORY_INTERVAL_SECS) * REPEATER_HISTORY_INTERVAL_SECS; + uint32_t airtime_elapsed_secs = now_secs - history_airtime_secs; + + if (history_battery_count > 0) { + battery_history.recordData(history_last_secs, history_battery_total / history_battery_count / 1000.0f); + } + int noise_floor = _radio->getNoiseFloor(); + if (noise_floor != 0) noise_history.recordData(history_last_secs, noise_floor); // zero means uncalibrated + uint32_t tx_millis = getTotalAirTime(); + uint32_t rx_millis = getReceiveAirTime(); + // Counters are milliseconds; unsigned subtraction also handles their wrap. + tx_history.recordData(history_last_secs, uint32_t(tx_millis - history_tx_millis) / (airtime_elapsed_secs * 10.0f)); + rx_history.recordData(history_last_secs, uint32_t(rx_millis - history_rx_millis) / (airtime_elapsed_secs * 10.0f)); + temperature_history.recordData(history_last_secs, board.getMCUTemperature()); // ignores NaN + history_airtime_secs = now_secs; + history_tx_millis = tx_millis; + history_rx_millis = rx_millis; + history_battery_total = 0; + history_battery_count = 0; +} +#endif + void MyMesh::loop() { + // Update before dispatching requests, including across millis() wrap. + uint32_t now = millis(); + uptime_millis += uint32_t(now - last_millis); + last_millis = now; +#if REPEATER_HISTORY_SLOTS > 0 + recordHistory(); +#endif #ifdef WITH_BRIDGE bridge.loop(); #endif @@ -1324,11 +1427,6 @@ void MyMesh::loop() { acl.save(_fs); dirty_contacts_expiry = 0; } - - // update uptime - uint32_t now = millis(); - uptime_millis += now - last_millis; - last_millis = now; } // To check if there is pending work diff --git a/examples/simple_repeater/MyMesh.h b/examples/simple_repeater/MyMesh.h index 7e4cfd4f2..f30eedd00 100644 --- a/examples/simple_repeater/MyMesh.h +++ b/examples/simple_repeater/MyMesh.h @@ -37,6 +37,22 @@ #include #include "RateLimiter.h" +#ifndef REPEATER_HISTORY_SLOTS +#if defined(STM32_PLATFORM) || defined(RP2040_PLATFORM) +#define REPEATER_HISTORY_SLOTS 0 +#else +#define REPEATER_HISTORY_SLOTS 288 +#endif +#endif +#ifndef REPEATER_HISTORY_INTERVAL_SECS +#define REPEATER_HISTORY_INTERVAL_SECS 300 +#endif + +#if REPEATER_HISTORY_SLOTS > 0 +#include +static_assert(REPEATER_HISTORY_INTERVAL_SECS > 0, "History interval must be positive"); +#endif + #ifdef WITH_BRIDGE extern AbstractBridge* bridge; #endif @@ -82,6 +98,16 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks { FILESYSTEM* _fs; uint32_t last_millis; uint64_t uptime_millis; +#if REPEATER_HISTORY_SLOTS > 0 + // Dedicated history channels 200..204; environment telemetry occupies 1..17. + enum { HISTORY_BATTERY = 200, HISTORY_NOISE, HISTORY_TX, HISTORY_RX, HISTORY_TEMPERATURE }; + TimeSeriesData battery_history, noise_history, tx_history, rx_history, temperature_history; + uint32_t history_last_secs, history_battery_secs, history_airtime_secs; + uint32_t history_tx_millis, history_rx_millis; + float history_battery_total; + uint32_t history_battery_count; + void recordHistory(); +#endif unsigned long next_local_advert, next_flood_advert; bool _logging; NodePrefs _prefs; diff --git a/examples/simple_sensor/SensorMesh.cpp b/examples/simple_sensor/SensorMesh.cpp index b18606681..bb747f63c 100644 --- a/examples/simple_sensor/SensorMesh.cpp +++ b/examples/simple_sensor/SensorMesh.cpp @@ -789,7 +789,9 @@ SensorMesh::SensorMesh(mesh::MainBoard& board, mesh::Radio& radio, mesh::Millise { next_local_advert = next_flood_advert = 0; dirty_contacts_expiry = 0; - last_read_time = 0; + last_read_time = uint32_t(0) - SENSOR_READ_INTERVAL_SECS; // read immediately on the first loop + last_millis = 0; + uptime_millis = 0; num_alert_tasks = 0; set_radio_at = revert_radio_at = 0; recv_pkt_region = NULL; @@ -999,6 +1001,9 @@ bool SensorMesh::getGPS(uint8_t channel, float& lat, float& lon, float& alt) { } void SensorMesh::loop() { + uint32_t now = millis(); + uptime_millis += uint32_t(now - last_millis); + last_millis = now; mesh::Mesh::loop(); if (next_flood_advert && millisHasNowPassed(next_flood_advert)) { @@ -1028,7 +1033,8 @@ void SensorMesh::loop() { } uint32_t curr = getRTCClock()->getCurrentTime(); - if (curr >= last_read_time + SENSOR_READ_INTERVAL_SECS) { + uint32_t uptime_secs = getUptimeSeconds(); + if (uint32_t(uptime_secs - last_read_time) >= SENSOR_READ_INTERVAL_SECS) { telemetry.reset(); telemetry.addVoltage(TELEM_CHANNEL_SELF, (float)board.getBattMilliVolts() / 1000.0f); // query other sensors -- target specific @@ -1069,7 +1075,7 @@ void SensorMesh::loop() { onSensorDataRead(); - last_read_time = curr; + last_read_time = uptime_secs; } // check the alert send queue diff --git a/examples/simple_sensor/SensorMesh.h b/examples/simple_sensor/SensorMesh.h index e6746d470..f1431f6f3 100644 --- a/examples/simple_sensor/SensorMesh.h +++ b/examples/simple_sensor/SensorMesh.h @@ -3,7 +3,7 @@ #include // needed for PlatformIO #include -#include "TimeSeriesData.h" +#include #if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) #include @@ -88,6 +88,7 @@ public: void sendFloodReply(mesh::Packet* packet, unsigned long delay_millis, uint8_t path_hash_size); protected: + uint32_t getUptimeSeconds() const { return uptime_millis / 1000; } // current telemetry data queries float getVoltage(uint8_t channel) { return getTelemValue(channel, LPP_VOLTAGE); } float getCurrent(uint8_t channel) { return getTelemValue(channel, LPP_CURRENT); } @@ -154,6 +155,8 @@ private: RegionEntry* load_stack[8]; TransportKey default_scope; uint32_t last_read_time; + uint32_t last_millis; + uint64_t uptime_millis; int matching_peer_indexes[MAX_SEARCH_RESULTS]; int num_alert_tasks; Trigger* alert_tasks[MAX_CONCURRENT_ALERTS]; diff --git a/examples/simple_sensor/TimeSeriesData.cpp b/examples/simple_sensor/TimeSeriesData.cpp deleted file mode 100644 index f6157f9af..000000000 --- a/examples/simple_sensor/TimeSeriesData.cpp +++ /dev/null @@ -1,45 +0,0 @@ -#include "TimeSeriesData.h" - -void TimeSeriesData::recordData(mesh::RTCClock* clock, float value) { - uint32_t now = clock->getCurrentTime(); - if (now >= last_timestamp + interval_secs) { - last_timestamp = now; - - data[next] = value; // append to cycle table - next = (next + 1) % num_slots; - } -} - -void TimeSeriesData::calcMinMaxAvg(mesh::RTCClock* clock, uint32_t start_secs_ago, uint32_t end_secs_ago, MinMaxAvg* dest, uint8_t channel, uint8_t lpp_type) const { - int i = next, n = num_slots; - uint32_t ago = clock->getCurrentTime() - last_timestamp; - int num_values = 0; - float total = 0.0f; - - dest->_channel = channel; - dest->_lpp_type = lpp_type; - - // start at most recet recording, back-track through to oldest - while (n > 0) { - n--; - i = (i + num_slots - 1) % num_slots; // go back by one - if (ago >= end_secs_ago && ago < start_secs_ago) { // filter by the desired time range - float v = data[i]; - num_values++; - total += v; - if (num_values == 1) { - dest->_max = dest->_min = v; - } else { - if (v < dest->_min) dest->_min = v; - if (v > dest->_max) dest->_max = v; - } - } - ago += interval_secs; - } - // calc average - if (num_values > 0) { - dest->_avg = total / num_values; - } else { - dest->_max = dest->_min = dest->_avg = NAN; - } -} diff --git a/examples/simple_sensor/TimeSeriesData.h b/examples/simple_sensor/TimeSeriesData.h index 6efa7834a..41bf31893 100644 --- a/examples/simple_sensor/TimeSeriesData.h +++ b/examples/simple_sensor/TimeSeriesData.h @@ -1,29 +1,4 @@ #pragma once -#include -#include - -struct MinMaxAvg { - float _min, _max, _avg; - uint8_t _lpp_type, _channel; -}; - -class TimeSeriesData { - float* data; - int num_slots, next; - uint32_t last_timestamp; - uint32_t interval_secs; - -public: - TimeSeriesData(float* array, int num, uint32_t secs) : num_slots(num), data(array), last_timestamp(0), next(0), interval_secs(secs) { - memset(data, 0, sizeof(float)*num); - } - TimeSeriesData(int num, uint32_t secs) : num_slots(num), last_timestamp(0), next(0), interval_secs(secs) { - data = new float[num]; - memset(data, 0, sizeof(float)*num); - } - - void recordData(mesh::RTCClock* clock, float value); - void calcMinMaxAvg(mesh::RTCClock* clock, uint32_t start_secs_ago, uint32_t end_secs_ago, MinMaxAvg* dest, uint8_t channel, uint8_t lpp_type) const; -}; - +// Compatibility include for custom sensor sketches; implementation lives in src/helpers. +#include diff --git a/examples/simple_sensor/main.cpp b/examples/simple_sensor/main.cpp index 66991daad..6d455f73b 100644 --- a/examples/simple_sensor/main.cpp +++ b/examples/simple_sensor/main.cpp @@ -21,13 +21,14 @@ protected: void onSensorDataRead() override { float batt_voltage = getVoltage(TELEM_CHANNEL_SELF); - battery_data.recordData(getRTCClock(), batt_voltage); // record battery + battery_data.recordData(getUptimeSeconds(), batt_voltage); // record battery alertIf(batt_voltage < 3.4f, critical_batt, HIGH_PRI_ALERT, "Battery is critical!"); alertIf(batt_voltage < 3.6f, low_batt, LOW_PRI_ALERT, "Battery is low"); } int querySeriesData(uint32_t start_secs_ago, uint32_t end_secs_ago, MinMaxAvg dest[], int max_num) override { - battery_data.calcMinMaxAvg(getRTCClock(), start_secs_ago, end_secs_ago, &dest[0], TELEM_CHANNEL_SELF, LPP_VOLTAGE); + battery_data.calcMinMaxAvg(getUptimeSeconds(), start_secs_ago, end_secs_ago, &dest[0], TELEM_CHANNEL_SELF, + LPP_VOLTAGE); return 1; } diff --git a/platformio.ini b/platformio.ini index 6a5756a30..c33105c98 100644 --- a/platformio.ini +++ b/platformio.ini @@ -175,6 +175,7 @@ build_src_filter = +<../src/Packet.cpp> +<../src/helpers/ConfigSerializer.cpp> +<../src/helpers/DynamicConfigSerializer.cpp> + +<../src/helpers/TimeSeriesData.cpp> lib_deps = google/googletest @ 1.17.0 diff --git a/src/helpers/TimeSeriesData.cpp b/src/helpers/TimeSeriesData.cpp new file mode 100644 index 000000000..371433c61 --- /dev/null +++ b/src/helpers/TimeSeriesData.cpp @@ -0,0 +1,91 @@ +#include "TimeSeriesData.h" + +#include +#include + +TimeSeriesData::TimeSeriesData(float *array, int num, uint32_t secs) + : data(array), num_slots(num > 0 ? num : 0), next(0), count(0), last_timestamp(0), interval_secs(secs), + owns_data(array == nullptr) { + if (num_slots > 0) { + if (owns_data) data = new float[num_slots]; + } +} + +TimeSeriesData::~TimeSeriesData() { + if (owns_data) delete[] data; +} + +void TimeSeriesData::recordData(uint32_t now_secs, float value) { + if (num_slots == 0 || isnan(value)) return; + uint32_t steps = 1; + if (count > 0 && interval_secs > 0) { + uint32_t elapsed = now_secs - last_timestamp; + if (elapsed < interval_secs) return; + steps = elapsed / interval_secs; + last_timestamp += steps * interval_secs; // keep a fixed sampling grid + } else { + last_timestamp = now_secs; + } + + // A long pause expires the whole ring in O(capacity), never O(elapsed time). + if (steps >= uint32_t(num_slots)) { + next = count = 0; + } else { + for (uint32_t i = 1; i < steps; i++) { + data[next] = NAN; + next = (next + 1) % num_slots; + if (count < num_slots) count++; + } + } + + data[next] = value; + next = (next + 1) % num_slots; + if (count < num_slots) count++; +} + +void TimeSeriesData::calcMinMaxAvg(uint32_t now_secs, uint32_t start_secs_ago, uint32_t end_secs_ago, + MinMaxAvg *dest, uint8_t channel, uint8_t lpp_type) const { + dest->_channel = channel; + dest->_lpp_type = lpp_type; + dest->_min = dest->_max = dest->_avg = NAN; + int num_values = 0; + float total = 0.0f; + int i = next; + uint64_t ago = uint32_t(now_secs - last_timestamp); + for (int n = 0; n < count; n++) { + i = (i + num_slots - 1) % num_slots; + if (ago >= end_secs_ago && ago < start_secs_ago && !isnan(data[i])) { + float v = data[i]; + if (num_values == 0) { + dest->_min = dest->_max = v; + } else { + if (v < dest->_min) dest->_min = v; + if (v > dest->_max) dest->_max = v; + } + total += v; + num_values++; + } + ago += interval_secs; + } + if (num_values > 0) dest->_avg = total / num_values; +} + +// Shared only by legacy callers; never consult the RTC, even after a time sync. +// At least one legacy call is required per millis() wrap (about 49 days). +static uint32_t legacyMonotonicSeconds() { + static uint32_t last_millis = 0; + static uint64_t uptime_millis = 0; + uint32_t now = millis(); + uptime_millis += uint32_t(now - last_millis); + last_millis = now; + return uptime_millis / 1000; +} + +void TimeSeriesData::recordData(mesh::RTCClock *, float value) { + recordData(legacyMonotonicSeconds(), value); +} + +void TimeSeriesData::calcMinMaxAvg(mesh::RTCClock *, uint32_t start_secs_ago, uint32_t end_secs_ago, + MinMaxAvg *dest, uint8_t channel, uint8_t lpp_type) const { + calcMinMaxAvg(legacyMonotonicSeconds(), start_secs_ago, end_secs_ago, dest, channel, lpp_type); +} diff --git a/src/helpers/TimeSeriesData.h b/src/helpers/TimeSeriesData.h new file mode 100644 index 000000000..3b7cdd47c --- /dev/null +++ b/src/helpers/TimeSeriesData.h @@ -0,0 +1,38 @@ +#pragma once + +#include + +namespace mesh { +class RTCClock; +} + +struct MinMaxAvg { + float _min, _max, _avg; + uint8_t _lpp_type, _channel; +}; + +class TimeSeriesData { + float *data; + int num_slots, next, count; + uint32_t last_timestamp, interval_secs; + bool owns_data; + +public: + TimeSeriesData(float *array, int num, uint32_t secs); + TimeSeriesData(int num, uint32_t secs) : TimeSeriesData(nullptr, num, secs) {} + ~TimeSeriesData(); + TimeSeriesData(const TimeSeriesData &) = delete; + TimeSeriesData &operator=(const TimeSeriesData &) = delete; + + // Caller supplies monotonic seconds, accumulated across millisecond clock wraps. + void recordData(uint32_t now_secs, float value); + void calcMinMaxAvg(uint32_t now_secs, uint32_t start_secs_ago, uint32_t end_secs_ago, MinMaxAvg *dest, + uint8_t channel, uint8_t lpp_type) const; + + // Compatibility for custom sensor sketches. The RTC argument is ignored. + [[deprecated("Pass monotonic seconds instead of RTCClock*")]] + void recordData(mesh::RTCClock *clock, float value); + [[deprecated("Pass monotonic seconds instead of RTCClock*")]] + void calcMinMaxAvg(mesh::RTCClock *clock, uint32_t start_secs_ago, uint32_t end_secs_ago, MinMaxAvg *dest, + uint8_t channel, uint8_t lpp_type) const; +}; diff --git a/test/test_time_series/test_time_series.cpp b/test/test_time_series/test_time_series.cpp new file mode 100644 index 000000000..eeb1aa6eb --- /dev/null +++ b/test/test_time_series/test_time_series.cpp @@ -0,0 +1,138 @@ +#include +#include +#include +#include +#include +#include + +static MinMaxAvg query(const TimeSeriesData &series, uint32_t now, uint32_t start, uint32_t end) { + MinMaxAvg result; + series.calcMinMaxAvg(now, start, end, &result, 200, 116); + return result; +} + +TEST(TimeSeriesData, RingWrapKeepsNewestSamples) { + TimeSeriesData series(3, 10); + for (uint32_t i = 1; i <= 5; i++) + series.recordData(i * 10, float(i)); + auto result = query(series, 50, 100, 0); + EXPECT_FLOAT_EQ(result._min, 3); + EXPECT_FLOAT_EQ(result._max, 5); + EXPECT_FLOAT_EQ(result._avg, 4); + EXPECT_EQ(result._channel, 200); + EXPECT_EQ(result._lpp_type, 116); +} + +TEST(TimeSeriesData, RangeIncludesEndAndExcludesStart) { + TimeSeriesData series(4, 10); + series.recordData(10, -5); + series.recordData(20, -3); + series.recordData(30, -1); + series.recordData(40, 7); + auto result = query(series, 40, 30, 10); + EXPECT_FLOAT_EQ(result._min, -3); + EXPECT_FLOAT_EQ(result._max, -1); + EXPECT_FLOAT_EQ(result._avg, -2); +} + +TEST(TimeSeriesData, EmptyAndUnwrittenRangesReturnNaN) { + TimeSeriesData series(4, 10); + EXPECT_TRUE(std::isnan(query(series, 0, 100, 0)._avg)); + series.recordData(10, 4); + for (auto range : { std::pair(100, 1), { 0, 0 }, { 1, 10 } }) { + auto result = query(series, 10, range.first, range.second); + EXPECT_TRUE(std::isnan(result._min)); + EXPECT_TRUE(std::isnan(result._max)); + EXPECT_TRUE(std::isnan(result._avg)); + } +} + +TEST(TimeSeriesData, CadenceAndNaNSamples) { + float storage[3]; + TimeSeriesData series(storage, 3, 10); + series.recordData(uint32_t(0), 2); // uptime zero is a valid first sample + series.recordData(9, 100); + EXPECT_FLOAT_EQ(query(series, 9, 100, 0)._avg, 2); + series.recordData(10, 4); + series.recordData(19, 100); + series.recordData(20, NAN); + series.recordData(20, 6); + EXPECT_FLOAT_EQ(query(series, 20, 100, 0)._avg, 4); +} + +TEST(TimeSeriesData, LargeGapsExpireOldSamplesAndDoNotStall) { + TimeSeriesData series(4, 300); + series.recordData(300, 1); + const uint32_t later = UINT32_MAX / 300 * 300; + series.recordData(later, 3); + EXPECT_FLOAT_EQ(query(series, later, 301, 0)._avg, 3); + EXPECT_FLOAT_EQ(query(series, later, UINT32_MAX, 0)._avg, 3); + series.recordData(later + uint32_t(300), 5); // seconds counter wrap + auto result = query(series, later + uint32_t(300), 301, 0); + EXPECT_FLOAT_EQ(result._min, 3); + EXPECT_FLOAT_EQ(result._max, 5); + EXPECT_FLOAT_EQ(result._avg, 4); +} + +TEST(TimeSeriesData, MissingIntervalsAreEmptyInsteadOfMovingOldSamplesForward) { + TimeSeriesData series(5, 10); + series.recordData(10, 2); + series.recordData(40, 8); + EXPECT_TRUE(std::isnan(query(series, 40, 30, 1)._avg)); + EXPECT_FLOAT_EQ(query(series, 40, 31, 0)._avg, 5); + series.recordData(52, 10); // late read belongs to the fixed slot at 50 + EXPECT_FLOAT_EQ(query(series, 52, 3, 0)._avg, 10); + EXPECT_TRUE(std::isnan(query(series, 52, 2, 0)._avg)); +} + +TEST(TimeSeriesData, QueryAgeDoesNotOverflowWhileWalkingTheRing) { + TimeSeriesData series(3, 10); + series.recordData(10, 1); + series.recordData(20, 2); + EXPECT_TRUE(std::isnan(query(series, uint32_t(20 + (UINT32_MAX - 5)), 100, 0)._avg)); +} + +TEST(TimeSeriesData, LegacyClockAPIUsesMonotonicTimeAcrossRTCAndMillisWrap) { + class Clock : public mesh::RTCClock { + public: + uint32_t time = 100000; + int reads = 0; + uint32_t getCurrentTime() override { + reads++; + return time; + } + void setCurrentTime(uint32_t value) override { time = value; } + } clock; + TimeSeriesData series(3, 1); + g_mock_millis = UINT32_MAX - 499; + series.recordData(&clock, 2); + clock.setCurrentTime(0); + g_mock_millis = 500; + series.recordData(&clock, 4); + MinMaxAvg result; + series.calcMinMaxAvg(&clock, 2, 0, &result, 1, 116); + EXPECT_FLOAT_EQ(result._avg, 3); + EXPECT_EQ(clock.reads, 0); +} + +TEST(TimeSeriesData, MillisecondCounterWrapDoesNotResetSeconds) { + TimeSeriesData series(3, 1); + uint32_t last_millis = UINT32_MAX - 499; + uint64_t uptime_millis = last_millis; + series.recordData(uint32_t(uptime_millis / 1000), 1); + uint32_t now = 500; + uptime_millis += uint32_t(now - last_millis); + series.recordData(uint32_t(uptime_millis / 1000), 3); + EXPECT_FLOAT_EQ(query(series, uint32_t(uptime_millis / 1000), 2, 0)._avg, 2); +} + +TEST(TimeSeriesData, ZeroCapacityIsEmpty) { + TimeSeriesData series(0, 10); + series.recordData(100, 1); + EXPECT_TRUE(std::isnan(query(series, 100, 1000, 0)._avg)); +} + +int main(int argc, char **argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +}