mirror of
https://github.com/mikecarper/MeshCore.git
synced 2026-09-16 15:52:50 +00:00
Enforce classic ESP32 memory budgets and safe image merging
This commit is contained in:
@@ -27,6 +27,9 @@ jobs:
|
||||
- name: Run Unit Tests
|
||||
run: pio test -e native -e native_kiss_modem -vv
|
||||
|
||||
- name: Verify ESP32 static DRAM budget
|
||||
run: python3 -B test/test_esp32_dram.py
|
||||
|
||||
- name: Verify nRF52 UF2-reset CLI coverage
|
||||
run: python3 -B test/test_nrf52_uf2reset_cli.py
|
||||
|
||||
|
||||
@@ -4011,6 +4011,12 @@ build_firmware() {
|
||||
unset MESHCORE_COMPANION_RADIO_FULL
|
||||
fi
|
||||
|
||||
if [ "$env_platform" = "ESP32_PLATFORM" ] \
|
||||
&& ! pio_env_option_contains "$pio_env_name" extra_scripts \
|
||||
"scripts/check_esp32_dram.py"; then
|
||||
append_platformio_extra_script "post:scripts/check_esp32_dram.py"
|
||||
fi
|
||||
|
||||
print_build_flags "$pio_env_name" "$env_name"
|
||||
build_status=0
|
||||
if [ "$env_platform" = "ESP32_PLATFORM" ]; then
|
||||
|
||||
@@ -1916,7 +1916,7 @@ removed. The fixed-size STM32WL FPF6 build cannot match authenticated channels.
|
||||
|
||||
**Parameters:**
|
||||
- `n`: Slot number within the table compiled for the target. Roomy ESP32 builds
|
||||
provide `1-255`; DRAM-tight classic ESP32 LoRa-OTA repeaters and nRF52/other
|
||||
provide `1-255`; classic ESP32 repeaters and nRF52/other
|
||||
normal constrained builds provide `1-31`; very-tight STM32WL repeaters provide
|
||||
`1-15`; the no-PSRAM LilyGo T-LoRa V2.1 repeater/observer provides `1-4`.
|
||||
- `channel`: `public`, a public `#channel`, or a 128/256-bit channel key in hex.
|
||||
@@ -3391,6 +3391,11 @@ set direct.retry.cr 20.0,12.0,6.0,2.0
|
||||
|
||||
**Note:** These commands are repeater-only.
|
||||
|
||||
The default capacity is 256 entries on classic ESP32, 2,048 on other ESP32
|
||||
chips, 512 on nRF52, and 64 on other platforms. Builds can override it with
|
||||
`MAX_RECENT_REPEATERS`. Classic ESP32's history uses 3,072 bytes of startup
|
||||
heap instead of 24,576 bytes, leaving more memory for the packet pool and Wi-Fi.
|
||||
|
||||
**Output order:**
|
||||
- `get recent.repeater` lists 3-byte prefixes first, then 2-byte prefixes, then 1-byte prefixes.
|
||||
- Within each prefix length, entries are sorted from highest SNR to lowest SNR.
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
# Classic ESP32 image memory budget
|
||||
|
||||
Classic ESP32 has 320 KiB of internal DRAM, but at most 160 KiB can hold
|
||||
statically allocated data. The remaining DRAM is available only through the
|
||||
runtime heap. Bluetooth, tracing, and SDK reservations further constrain the
|
||||
usable region. See Espressif's [memory types documentation](https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-guides/memory-types.html#dram-data-ram).
|
||||
|
||||
An image can therefore approach its static limit while PlatformIO reports
|
||||
less than 40% of the 320 KiB total in use. PSRAM does not expand the region used
|
||||
by ordinary internal `.data` and `.bss` allocations.
|
||||
|
||||
## Build enforcement
|
||||
|
||||
Every classic ESP32 environment inheriting `esp32_base` runs
|
||||
`scripts/check_esp32_dram.py`. The release builder also adds the script when
|
||||
an ESP32 profile replaces the inherited extra scripts. The check runs before
|
||||
image generation, merging, and uploading, including cached `nobuild` uploads.
|
||||
It requires the ELF and linker map to be present for those cached operations.
|
||||
The check script must load after `merge-bin.py` so the merge target retains its
|
||||
pre-action. A merge also requires the bootloader/partition flash layout;
|
||||
`nobuild` cannot create a bootable merged image when that layout is absent.
|
||||
|
||||
The check reads `dram0_0_seg` from the final linker map. SDK reservations are
|
||||
already reflected in that region; Bluetooth's reservation is not subtracted
|
||||
a second time. The occupied span runs from the region origin to `_heap_start`,
|
||||
including `.data`, `.bss`, `.noinit`, and alignment gaps. The allowed region is
|
||||
the smaller of the linker's region and 160 KiB.
|
||||
|
||||
Builds must leave at least **8 KiB free within that static region**. This is
|
||||
an additional project margin against near-full images, not an Espressif
|
||||
hardware limit or a prediction of free heap. A profile may require a larger
|
||||
margin with `custom_esp32_static_dram_reserve` in bytes; values below 8192 are
|
||||
rejected. Missing or unrecognized map boundaries also fail the check.
|
||||
|
||||
For the Heltec V2 build used to investigate the repeater report:
|
||||
|
||||
| Shared repeater defaults | Static span | Static region | Remaining | Result |
|
||||
| --- | ---: | ---: | ---: | --- |
|
||||
| Previous 255 scope slots | 124,020 B | 124,580 B | 560 B | Rejected |
|
||||
| Tuned 31 scope slots | 108,308 B | 124,580 B | 16,272 B | Passed |
|
||||
|
||||
These measurements remove the board's existing scope-capacity override to
|
||||
exercise the shared defaults. The normal Heltec V2 profile already used 31
|
||||
scope slots. Exact sizes vary with build flags and SDK versions.
|
||||
|
||||
To inspect an existing classic ESP32 build:
|
||||
|
||||
```sh
|
||||
python3 scripts/check_esp32_dram.py .pio/build/Heltec_v2_repeater/firmware.map
|
||||
python3 test/test_esp32_dram.py
|
||||
```
|
||||
|
||||
ESP32-S2, S3, and C-series chips have different memory maps and do not inherit
|
||||
this classic ESP32 check.
|
||||
|
||||
## Boot versus running memory
|
||||
|
||||
The build gate checks static placement needed to load and start the image.
|
||||
It never counts heap-only RAM toward that static budget. Allocations made with
|
||||
`new` or `malloc` in global constructors consume heap and are not included in
|
||||
the static span. Moving a table there changes its placement, not total RAM
|
||||
consumption.
|
||||
|
||||
Successful build checks do not prove that every runtime configuration will
|
||||
boot. Hardware validation must still cover startup allocations and the enabled
|
||||
Wi-Fi/Bluetooth services, recording minimum free internal heap and the largest
|
||||
free internal block. Available heap changes during initialization; for example,
|
||||
the Arduino 2.x repeater releases unused Bluetooth controller RAM in
|
||||
`initArduino()`, after global constructors have run.
|
||||
@@ -182,7 +182,7 @@ Capacity is selected at build time:
|
||||
|
||||
- Roomy ESP32 builds: 255 rewrite slots and 32 regionless-target slots, 10,204
|
||||
bytes RAM.
|
||||
- DRAM-tight classic ESP32 LoRa-OTA repeaters, nRF52, and other normal
|
||||
- Classic ESP32 repeaters, nRF52, and other normal
|
||||
constrained builds: 31 rewrite and regionless-target slots, 2,108 bytes RAM.
|
||||
- Very-tight STM32WL builds: 15 rewrite slots and one reusable regionless-target
|
||||
slot, 572 bytes RAM.
|
||||
@@ -190,6 +190,13 @@ Capacity is selected at build time:
|
||||
regionless-target slots, 272 bytes RAM. This minimum holds the
|
||||
three wildcard classes and one exact channel mapping.
|
||||
|
||||
Classic ESP32 uses the 31-slot default even on boards with PSRAM because these
|
||||
tables live in internal static DRAM. The separate channel-requirement table
|
||||
defaults to the same capacity and uses another 34 bytes per slot. Together,
|
||||
the 31-slot defaults save 15,712 bytes of static RAM compared with 255 slots.
|
||||
Custom builds can override `FLOOD_CHANNEL_SCOPE_SLOTS` and
|
||||
`FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS` with build flags.
|
||||
|
||||
Each rule retains its 36-byte record. A separate table holds 32-byte normalized
|
||||
names for up to the smaller of the rule count or 32 distinct regionless
|
||||
targets, except that very-tight STM32WL builds retain one reusable direct
|
||||
|
||||
@@ -11,6 +11,7 @@ Below are a few quick start guides.
|
||||
- [Filter Policy Playground](./filter_tool.md)
|
||||
- [Telemetry Decoder](./telemetry_decoder.md)
|
||||
- [CLI Availability by Firmware Build](./cli_build_matrix.md)
|
||||
- [Classic ESP32 image memory budget](./esp32_memory_budget.md)
|
||||
- [Easy LoRa OTA: ESP32 and nRF52 firmware updates](./ota_easy.md)
|
||||
- [Scripted LoRa OTA: Bash and PowerShell](./lora_ota_automation.md)
|
||||
- [nRF52 repeater OTA with external QSPI](./ota_nrf52_qspi.md)
|
||||
|
||||
@@ -35,6 +35,11 @@
|
||||
// Only repeater firmware supplies this RAM-heavy history storage.
|
||||
#if !MESH_ENABLE_RECENT_REPEATERS
|
||||
#define MAX_RECENT_REPEATERS 0
|
||||
#elif defined(CONFIG_IDF_TARGET_ESP32)
|
||||
// This history is allocated before setup(), when classic ESP32 must also
|
||||
// retain internal heap for startup, the packet pool, and WiFi. Moving the
|
||||
// old 24 KiB table off .bss alone does not reduce that heap pressure.
|
||||
#define MAX_RECENT_REPEATERS 256
|
||||
#elif defined(ESP32) || defined(ESP32_PLATFORM)
|
||||
#define MAX_RECENT_REPEATERS 2048
|
||||
#elif defined(NRF52_PLATFORM)
|
||||
@@ -196,7 +201,12 @@ struct NeighbourInfo {
|
||||
(FLOOD_PACKET_FILTER_PATH_PREFIX_HOPS_MAX * 3)
|
||||
|
||||
#ifndef FLOOD_CHANNEL_SCOPE_SLOTS
|
||||
#if defined(ESP32)
|
||||
#if defined(CONFIG_IDF_TARGET_ESP32)
|
||||
// The rewrite and require tables share this capacity. Their 255-slot
|
||||
// default leaves almost no static DRAM headroom in classic ESP32 builds;
|
||||
// PSRAM cannot hold these inline members of the global MyMesh object.
|
||||
#define FLOOD_CHANNEL_SCOPE_SLOTS 31
|
||||
#elif defined(ESP32)
|
||||
#define FLOOD_CHANNEL_SCOPE_SLOTS 255
|
||||
#elif defined(STM32_PLATFORM)
|
||||
#define FLOOD_CHANNEL_SCOPE_SLOTS 15
|
||||
|
||||
+9
-4
@@ -4,7 +4,7 @@
|
||||
|
||||
import os
|
||||
|
||||
Import("env", "projenv")
|
||||
Import("env")
|
||||
|
||||
board_config = env.BoardConfig()
|
||||
firmware_bin = "${BUILD_DIR}/${PROGNAME}.bin"
|
||||
@@ -12,8 +12,13 @@ merged_bin = os.environ.get("MERGED_BIN_PATH", "${BUILD_DIR}/${PROGNAME}-merged.
|
||||
|
||||
|
||||
def merge_bin_action(source, target, env):
|
||||
extra_images = env.Flatten(env.get("FLASH_EXTRA_IMAGES", []))
|
||||
if not extra_images:
|
||||
print("Cannot merge a bootable image without the bootloader/partition "
|
||||
"flash layout. Run the mergebin target without nobuild.")
|
||||
return 1
|
||||
flash_images = [
|
||||
*env.Flatten(env.get("FLASH_EXTRA_IMAGES", [])),
|
||||
*extra_images,
|
||||
"$ESP32_APP_OFFSET",
|
||||
source[0].get_abspath(),
|
||||
]
|
||||
@@ -35,7 +40,7 @@ def merge_bin_action(source, target, env):
|
||||
*flash_images,
|
||||
]
|
||||
)
|
||||
env.Execute(merge_cmd)
|
||||
return env.Execute(merge_cmd)
|
||||
|
||||
|
||||
env.AddCustomTarget(
|
||||
@@ -45,4 +50,4 @@ env.AddCustomTarget(
|
||||
title="Merge binary",
|
||||
description="Build combined image",
|
||||
always_build=True,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -76,6 +76,7 @@ extra_scripts =
|
||||
pre:scripts/portable_esp32_link.py
|
||||
merge-bin.py
|
||||
post:tools/mota/pio_endf.py
|
||||
post:scripts/check_esp32_dram.py
|
||||
build_flags = ${arduino_base.build_flags}
|
||||
-D ESP32_PLATFORM
|
||||
-D ENABLE_OTA=1
|
||||
|
||||
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Enforce classic ESP32's static DRAM budget before producing an image.
|
||||
|
||||
The 320 KiB DRAM total includes memory available only as runtime heap. Use
|
||||
the linked image's actual region (including SDK/BT/trace reservations), capped
|
||||
at Espressif's documented 160 KiB static limit. Keep an additional 8 KiB of
|
||||
static-region headroom as a project policy; this is not a runtime heap estimate
|
||||
or a guarantee that every peripheral/configuration will boot.
|
||||
|
||||
Run standalone with a GNU linker map, or as a PlatformIO post extra_script.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import re
|
||||
import sys
|
||||
|
||||
|
||||
STATIC_DRAM_LIMIT = 160 * 1024
|
||||
MIN_STATIC_DRAM_RESERVE = 8 * 1024
|
||||
DRAM_START = 0x3FFB0000
|
||||
DRAM_END = 0x40000000
|
||||
|
||||
|
||||
def parse_reserve(value):
|
||||
reserve = int(str(value), 0)
|
||||
if reserve < MIN_STATIC_DRAM_RESERVE:
|
||||
raise ValueError(
|
||||
f"static DRAM reserve must be at least {MIN_STATIC_DRAM_RESERVE} bytes"
|
||||
)
|
||||
return reserve
|
||||
|
||||
|
||||
def static_dram_usage(map_text):
|
||||
"""Return region length and occupied span, including alignment and .noinit."""
|
||||
try:
|
||||
memory, linked = map_text.split("Linker script and memory map", 1)
|
||||
memory = memory.split("Memory Configuration", 1)[1]
|
||||
except (ValueError, IndexError) as error:
|
||||
raise ValueError("missing GNU linker memory configuration") from error
|
||||
|
||||
regions = re.findall(
|
||||
r"^dram0_0_seg\s+(0x[0-9a-fA-F]+)\s+(0x[0-9a-fA-F]+)\s+\S+",
|
||||
memory, re.MULTILINE,
|
||||
)
|
||||
if len(regions) != 1:
|
||||
raise ValueError("expected one classic ESP32 dram0_0_seg region")
|
||||
origin, length = (int(value, 16) for value in regions[0])
|
||||
if length <= 0 or origin < DRAM_START or origin + length > DRAM_END:
|
||||
raise ValueError("invalid classic ESP32 DRAM region")
|
||||
|
||||
def symbol(name):
|
||||
values = re.findall(
|
||||
rf"^\s*(0x[0-9a-fA-F]+)\s+{re.escape(name)}\s*=",
|
||||
linked, re.MULTILINE,
|
||||
)
|
||||
if len(values) != 1:
|
||||
raise ValueError(f"expected one {name} definition in linker map")
|
||||
return int(values[0], 16)
|
||||
|
||||
data_start = symbol("_data_start")
|
||||
data_end = symbol("_data_end")
|
||||
bss_start = symbol("_bss_start")
|
||||
bss_end = symbol("_bss_end")
|
||||
heap_start = symbol("_heap_start")
|
||||
if not (origin <= data_start <= data_end <= heap_start
|
||||
and origin <= bss_start <= bss_end <= heap_start):
|
||||
raise ValueError("inconsistent static DRAM boundaries in linker map")
|
||||
|
||||
# _heap_start follows all internal static sections. Unlike summing .data
|
||||
# and .bss sizes, this also accounts for .noinit and linker alignment gaps.
|
||||
return length, heap_start - origin
|
||||
|
||||
|
||||
def check_map(map_path, reserve=MIN_STATIC_DRAM_RESERVE):
|
||||
try:
|
||||
reserve = parse_reserve(reserve)
|
||||
region_size, used = static_dram_usage(Path(map_path).read_text())
|
||||
except (OSError, UnicodeError, ValueError) as error:
|
||||
print(f"ESP32 static DRAM check failed: {error}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
limit = min(region_size, STATIC_DRAM_LIMIT)
|
||||
remaining = limit - used
|
||||
print(
|
||||
f"ESP32 static DRAM: {used:,}/{limit:,} bytes occupied, "
|
||||
f"{remaining:,} bytes free; required reserve {reserve:,} bytes "
|
||||
f"(linker region {region_size:,}, static ceiling {STATIC_DRAM_LIMIT:,})"
|
||||
)
|
||||
if remaining < reserve:
|
||||
print(
|
||||
f"ESP32 static DRAM check failed: reserve short by "
|
||||
f"{reserve - remaining:,} bytes. Reduce static tables/buffers or "
|
||||
"move suitable storage to checked heap allocations. The 320 KiB "
|
||||
"DRAM total and PSRAM do not expand this internal static budget.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def register_platformio(env):
|
||||
# ESP32-S2/S3/C-series chips have different memory maps and limits.
|
||||
if str(env.BoardConfig().get("build.mcu", "")).lower() != "esp32":
|
||||
return
|
||||
|
||||
reserve = parse_reserve(env.GetProjectOption(
|
||||
"custom_esp32_static_dram_reserve", str(MIN_STATIC_DRAM_RESERVE)
|
||||
))
|
||||
checked = None
|
||||
|
||||
def check_static_dram(source, target, env):
|
||||
nonlocal checked
|
||||
map_path = Path(env.subst("$BUILD_DIR/${PROGNAME}.map"))
|
||||
elf_path = Path(env.subst("$BUILD_DIR/${PROGNAME}.elf"))
|
||||
try:
|
||||
# Cached/nobuild uploads must still have the matching build inputs.
|
||||
elf_stat = elf_path.stat()
|
||||
map_stat = map_path.stat()
|
||||
except OSError as error:
|
||||
print(f"ESP32 static DRAM check failed: {error}", file=sys.stderr)
|
||||
return 2
|
||||
signature = (elf_stat.st_mtime_ns, elf_stat.st_size,
|
||||
map_stat.st_mtime_ns, map_stat.st_size)
|
||||
if checked == signature:
|
||||
return 0
|
||||
result = check_map(map_path, reserve)
|
||||
if result == 0:
|
||||
checked = signature
|
||||
return result
|
||||
|
||||
# checkprogsize runs on incremental builds too. The image and action hooks
|
||||
# also cover direct image generation, cached merges, and nobuild uploads.
|
||||
# Load this script after merge-bin.py: PlatformIO replaces a custom
|
||||
# target's executor when declaring it, which discards earlier pre-actions.
|
||||
# Alias nodes keep action targets distinct from files in nobuild mode.
|
||||
env.AddPreAction(env.Alias("checkprogsize"), check_static_dram)
|
||||
env.AddPreAction("$BUILD_DIR/${PROGNAME}.bin", check_static_dram)
|
||||
env.AddPreAction(env.Alias("mergebin"), check_static_dram)
|
||||
env.AddPreAction(env.Alias("upload"), check_static_dram)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("map", type=Path, help="classic ESP32 firmware.map")
|
||||
parser.add_argument(
|
||||
"--reserve", default=MIN_STATIC_DRAM_RESERVE, type=parse_reserve,
|
||||
help="required static-region headroom in bytes (minimum 8192)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
return check_map(args.map, args.reserve)
|
||||
|
||||
|
||||
try:
|
||||
Import("env") # noqa: F821 -- PlatformIO/SCons supplies Import
|
||||
except NameError:
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
else:
|
||||
register_platformio(env) # noqa: F821
|
||||
@@ -27,6 +27,7 @@ python3 test/test_replay_reset_command.py # Strict full keys and one-use r
|
||||
python3 test/test_replay_reset_integration.py # Actual repeater handler: USB/LoRa permissions and persistence ordering
|
||||
python3 test/test_regular_file_reads.py # SPIFFS phantom directories, listings, log HTTP status
|
||||
python3 test/test_esp32_full_partition.py # Full partition-preservation policy
|
||||
python3 test/test_esp32_dram.py # Classic ESP32 static limits and cached-image build gates
|
||||
python3 test/test_esp32_usb_serial_hygiene.py # Single-TTY diagnostics/NVS contract
|
||||
python3 test/test_esp32_tinyusb_role_hygiene.py # G2/room USB write coverage and bounded-list contracts
|
||||
python3 test/test_esp32_tinyusb_cooperative_output.py # Real role dump/list pumps with host C++ stubs
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Boot/static DRAM limits, independent of the larger running heap budget."""
|
||||
|
||||
import contextlib
|
||||
import configparser
|
||||
import io
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPT = ROOT / "scripts/check_esp32_dram.py"
|
||||
CHECKER = runpy.run_path(str(SCRIPT))
|
||||
|
||||
|
||||
def linker_map(used, *, origin=0x3FFBDB5C, length=0x1E6A4):
|
||||
# Include the real ESP32 Arduino 2.x region and an alignment/.noinit gap.
|
||||
# The other half of internal DRAM and PSRAM must not inflate this budget.
|
||||
return f"""Discarded input sections
|
||||
.bss.discarded 0x00000000 0x80000 unused.o
|
||||
|
||||
Memory Configuration
|
||||
Name Origin Length Attributes
|
||||
dram0_0_seg {origin:#018x} {length:#018x} rw
|
||||
extern_ram_seg 0x000000003f800000 0x0000000000400000 xrw
|
||||
|
||||
Linker script and memory map
|
||||
.dram0.data {origin + 4:#x} 0x6000
|
||||
{origin + 4:#x} _data_start = ABSOLUTE (.)
|
||||
{origin + 0x6004:#x} _data_end = ABSOLUTE (.)
|
||||
.noinit {origin + 0x6004:#x} 0x1fc
|
||||
.dram0.bss {origin + 0x6200:#x} {used - 0x6200:#x}
|
||||
{origin + 0x6200:#x} _bss_start = ABSOLUTE (.)
|
||||
{origin + used:#x} _bss_end = ABSOLUTE (.)
|
||||
.dram0.heap_start
|
||||
{origin + used:#x} 0x0
|
||||
{origin + used:#x} _heap_start = ABSOLUTE (.)
|
||||
.ext_ram.bss 0x3f800000 0x100000
|
||||
Cross Reference Table
|
||||
_heap_start libheap.a
|
||||
"""
|
||||
|
||||
|
||||
class FakeEnvironment:
|
||||
def __init__(self, directory, mcu="esp32", reserve="8192"):
|
||||
self.directory = str(directory)
|
||||
self.mcu = mcu
|
||||
self.reserve = reserve
|
||||
self.actions = {}
|
||||
|
||||
def BoardConfig(self):
|
||||
return {"build.mcu": self.mcu, "upload.maximum_ram_size": 327680}
|
||||
|
||||
def GetProjectOption(self, name, default):
|
||||
return self.reserve
|
||||
|
||||
def subst(self, value):
|
||||
return value.replace("$BUILD_DIR", self.directory).replace(
|
||||
"${PROGNAME}", "firmware"
|
||||
)
|
||||
|
||||
def AddPreAction(self, target, action):
|
||||
self.actions[target] = action
|
||||
|
||||
def Alias(self, name):
|
||||
return name
|
||||
|
||||
|
||||
class Esp32DramTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.directory = Path(self.temp.name)
|
||||
self.map = self.directory / "firmware.map"
|
||||
(self.directory / "firmware.elf").write_bytes(b"fixture ELF")
|
||||
|
||||
def check(self, text, reserve=8192):
|
||||
self.map.write_text(text)
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output), contextlib.redirect_stderr(output):
|
||||
result = CHECKER["check_map"](self.map, reserve)
|
||||
return result, output.getvalue()
|
||||
|
||||
def test_reported_38_percent_image_is_rejected(self):
|
||||
result, output = self.check(linker_map(124020))
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn("560 bytes free", output)
|
||||
self.assertIn("reserve short by 7,632 bytes", output)
|
||||
|
||||
def test_tuned_repeater_passes_with_actual_region_headroom(self):
|
||||
result, output = self.check(linker_map(108308))
|
||||
self.assertEqual(result, 0)
|
||||
self.assertIn("108,308/124,580", output)
|
||||
self.assertIn("16,272 bytes free", output)
|
||||
|
||||
def test_reserve_boundary_includes_noinit_and_alignment(self):
|
||||
self.assertEqual(self.check(linker_map(124580 - 8192))[0], 0)
|
||||
self.assertEqual(self.check(linker_map(124580 - 8191))[0], 1)
|
||||
|
||||
def test_documented_ceiling_wins_over_a_larger_linker_region(self):
|
||||
result, output = self.check(linker_map(
|
||||
160 * 1024 - 8191, origin=0x3FFB0000, length=0x2C200
|
||||
))
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn("/163,840 bytes", output)
|
||||
|
||||
def test_sdk_bluetooth_trace_reservations_reduce_the_limit(self):
|
||||
# A later start/smaller length already incorporates SDK reservations;
|
||||
# do not subtract another blanket 64 KiB from the linked region.
|
||||
result, output = self.check(linker_map(
|
||||
65536 - 8191, origin=0x3FFC0000, length=65536
|
||||
))
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn("/65,536 bytes", output)
|
||||
|
||||
def test_region_overflow_fails_even_below_160_kib(self):
|
||||
result, output = self.check(linker_map(124584))
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn("-4 bytes free", output)
|
||||
|
||||
def test_larger_reserve_is_supported_but_cannot_be_disabled(self):
|
||||
self.assertEqual(self.check(linker_map(108308), "0x4000")[0], 1)
|
||||
for reserve in (0, -1, 8191, "invalid"):
|
||||
with self.subTest(reserve=reserve):
|
||||
self.assertEqual(self.check(linker_map(108308), reserve)[0], 2)
|
||||
|
||||
def test_missing_or_inconsistent_map_never_passes(self):
|
||||
valid = linker_map(108308)
|
||||
for text in (
|
||||
"", valid.replace("dram0_0_seg", "other_region"),
|
||||
valid.replace("_heap_start =", "missing ="),
|
||||
valid + "\n 0x3ffd8270 _heap_start = ABSOLUTE (.)\n",
|
||||
valid.replace("0x3ffd8270 _heap_start", "0x3ffb0000 _heap_start"),
|
||||
linker_map(108308, origin=0x3F800000),
|
||||
):
|
||||
with self.subTest(text=text[:40]):
|
||||
self.assertEqual(self.check(text)[0], 2)
|
||||
|
||||
def test_missing_map_fails(self):
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertEqual(CHECKER["check_map"](self.map), 2)
|
||||
|
||||
def test_other_esp32_chips_do_not_inherit_classic_limits(self):
|
||||
for mcu in ("esp32s2", "esp32s3", "esp32c3", "esp32c6"):
|
||||
with self.subTest(mcu=mcu):
|
||||
env = FakeEnvironment(self.directory, mcu)
|
||||
CHECKER["register_platformio"](env)
|
||||
self.assertFalse(env.actions)
|
||||
|
||||
def test_guard_loads_after_the_custom_merge_target_is_defined(self):
|
||||
config = configparser.ConfigParser(interpolation=None)
|
||||
config.read(ROOT / "platformio.ini")
|
||||
scripts = config["esp32_base"]["extra_scripts"].split()
|
||||
self.assertGreater(
|
||||
scripts.index("post:scripts/check_esp32_dram.py"),
|
||||
scripts.index("merge-bin.py"),
|
||||
)
|
||||
|
||||
def test_build_image_merge_and_cached_upload_are_all_gated(self):
|
||||
self.map.write_text(linker_map(124020))
|
||||
env = FakeEnvironment(self.directory)
|
||||
CHECKER["register_platformio"](env)
|
||||
for target in ("checkprogsize", "$BUILD_DIR/${PROGNAME}.bin", "mergebin", "upload"):
|
||||
with self.subTest(target=target):
|
||||
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertEqual(env.actions[target]([], [], env), 1)
|
||||
|
||||
def test_cached_success_does_not_hide_a_changed_map(self):
|
||||
self.map.write_text(linker_map(108308))
|
||||
env = FakeEnvironment(self.directory)
|
||||
CHECKER["register_platformio"](env)
|
||||
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertEqual(env.actions["checkprogsize"]([], [], env), 0)
|
||||
self.map.write_text(linker_map(124020) + "\n")
|
||||
self.assertEqual(env.actions["upload"]([], [], env), 1)
|
||||
|
||||
def test_cached_upload_without_map_or_elf_fails(self):
|
||||
env = FakeEnvironment(self.directory)
|
||||
CHECKER["register_platformio"](env)
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertEqual(env.actions["upload"]([], [], env), 2)
|
||||
self.map.write_text(linker_map(108308))
|
||||
(self.directory / "firmware.elf").unlink()
|
||||
self.assertEqual(env.actions["upload"]([], [], env), 2)
|
||||
|
||||
def test_command_line_returns_failure_for_an_unsafe_image(self):
|
||||
self.map.write_text(linker_map(124020))
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(SCRIPT), str(self.map)],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("ESP32 static DRAM check failed", result.stderr)
|
||||
|
||||
|
||||
class Esp32MergeTest(unittest.TestCase):
|
||||
def load_merge(self, extra_images, exit_code=0):
|
||||
commands = []
|
||||
env = SimpleNamespace(
|
||||
BoardConfig=lambda: {"build.mcu": "esp32"},
|
||||
get=lambda name, default: extra_images,
|
||||
Flatten=lambda value: value,
|
||||
AddCustomTarget=lambda **kwargs: None,
|
||||
Execute=lambda command: commands.append(command) or exit_code,
|
||||
)
|
||||
|
||||
def import_env(*names):
|
||||
# nobuild exports env without creating projenv.
|
||||
self.assertEqual(names, ("env",))
|
||||
|
||||
namespace = runpy.run_path(
|
||||
str(ROOT / "merge-bin.py"),
|
||||
init_globals={"Import": import_env, "env": env},
|
||||
)
|
||||
return namespace["merge_bin_action"], env, commands
|
||||
|
||||
def test_missing_boot_layout_cannot_produce_app_only_merged_image(self):
|
||||
merge, env, commands = self.load_merge([])
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
self.assertEqual(merge([], [], env), 1)
|
||||
self.assertFalse(commands)
|
||||
|
||||
def test_merge_tool_failure_is_propagated(self):
|
||||
merge, env, commands = self.load_merge(
|
||||
["0x1000", "bootloader.bin", "0x8000", "partitions.bin"], 7
|
||||
)
|
||||
source = SimpleNamespace(get_abspath=lambda: "/fixture/firmware.bin")
|
||||
self.assertEqual(merge([source], [], env), 7)
|
||||
self.assertEqual(len(commands), 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -56,8 +56,7 @@ build_flags =
|
||||
; -D MESH_PACKET_LOGGING=1
|
||||
; -D MESH_DEBUG=1
|
||||
extra_scripts =
|
||||
merge-bin.py
|
||||
post:tools/mota/pio_endf.py
|
||||
${esp32_base.extra_scripts}
|
||||
build_src_filter = ${Heltec_lora32_v3.build_src_filter}
|
||||
+<helpers/ui/SSD1306Display.cpp>
|
||||
+<helpers/ota/*.cpp>
|
||||
|
||||
Reference in New Issue
Block a user