diff --git a/docs/ota_protocol.md b/docs/ota_protocol.md index 0bf099930..c8e3400c0 100644 --- a/docs/ota_protocol.md +++ b/docs/ota_protocol.md @@ -864,8 +864,21 @@ class MotaSource { }; ``` -To serve an external mota the node reads its manifest-minus-leaves + `leaves[]` into RAM (<=4 KiB for <=1024 -blocks) and streams payload blocks from the source on demand; proofs are generated from the read leaves. +Legacy source paths retain the manifest-minus-leaves and four bytes per source +block, bounded by `OTA_PROOFGEN_SCRATCH`, then stream payload blocks on demand. +Non-PSRAM ESP32 Full instead retains roots of 32-leaf groups and rereads the +selected group. Its complete workspace is bounded to 1,536 bytes while retaining +4,096 blocks and byte-identical proofs. Partial groups use the same odd-node +promotion as the full tree. Receive pipelines, capture, serving and wire formats +remain unchanged; larger custom leaf limits retain the legacy path. + +With `OTA_BOUNDED_SOURCE_PROOFS=1`, optional `read_quantum()` and +`read_metadata()` methods bound each metadata step. `SerialMotaSource` uses the +current negotiated single-response capacity and rejects a chunk that no longer +fits after a capacity change. Proof preparation yields between chunks, and +queued jobs retain their buffers across radio backpressure. See the +[V3 Full qualification](test-results/heltec_v3_full_memory_2026-10-19.md) for +allocation bounds and regression coverage. ### 10.2 The `mota-seeder` transport (`MotaSeederProto.h`) diff --git a/docs/research/companion_contact_cache.md b/docs/research/companion_contact_cache.md index 117b233ce..fbf8bddba 100644 --- a/docs/research/companion_contact_cache.md +++ b/docs/research/companion_contact_cache.md @@ -214,9 +214,28 @@ partition table. Subsequent WiFi updates must fit the new slots. An application-only update from the ordinary single-slot Full layout cannot change that layout or enable WiFi OTA. +## Shared contact index storage + +Companion and terminal-chat builds share compact contact indexes in +`BaseChatMesh`. Tables with at most 65,535 total slots use 16-bit sort and +matching-result entries; larger configurations retain the original `int` type. +Contact limits, PSRAM allocation fallback and saved/wire formats are unchanged. + +On the existing 32-bit platforms, a static 350-contact table saves 764 bytes +from these arrays; a static 100-contact table saves 264 bytes. A 350-contact +PSRAM configuration saves 96 internal bytes and 716 PSRAM bytes. Cached nRF52 +records also shrink from 96 to 92 bytes by moving runtime storage bookkeeping +into an alignment gap. The matched RAK4631 Full build saves 2,200 static RAM +bytes in total. These figures do not describe infrastructure ACL tables. + +See the [2026-10-19 qualification report](../test-results/contact_memory_2026-10-19.md) +for source-bound builds, tests and remaining board-specific memory limits. + ## Regression checks ```sh +python3 -B test/test_contact_indices.py +python3 -B test/test_companion_contact_lifetime.py python3 -B test/test_contact_cache.py python3 -B test/test_companion_uncached_storage.py python3 -B test/test_companion_contact_partial_tail.py diff --git a/docs/research/firmware_memory_budget.md b/docs/research/firmware_memory_budget.md index 942fde3f9..269204a90 100644 --- a/docs/research/firmware_memory_budget.md +++ b/docs/research/firmware_memory_budget.md @@ -231,7 +231,12 @@ rate-window counters nor struct padding can make different rules share a slot. OTA leaf capacity remains controlled by `OTA_PROOFGEN_SCRATCH`, independently of output-buffer size. The maximum source buffers are 4 KiB plus one block on ordinary nRF52, 8 KiB plus one block with SD storage, and 16 KiB plus one block -on ESP32. Generic folder sources need no output allocation for raw DATA or +on legacy ESP32 paths. Non-PSRAM ESP32 Full uses grouped source proofs instead: +the complete workspace is bounded to 1,536 bytes, while retaining the same +4,096-block source limit. The check still adds one output block and allocation +metadata; safety reserves and stacks are unchanged. PSRAM and other platforms +retain the legacy policy. Explicit larger custom leaf limits also keep the +legacy implementation. Generic folder sources need no output allocation for raw DATA or proofs; own-image raw serving retains its caller-owned compatibility buffer. Its nRF52 allowance remains in the budget even when the device encoder is off. Queued DATA, PROOF and MANIFEST responses pin their buffers through radio @@ -812,3 +817,46 @@ images fit. These results qualify linked capacity and host behavior, not physical USB/RF/MQTT integration, board boot/sleep or a sustained-load soak. The Mini Repeater remains cable-only (`--allow-no-ota`); no hardware was flashed, no release was published and existing outputs were not overwritten. + +### Shared contact RAM and RAK4631 Full recheck, 2026-10-19 UTC + +Companion and `simple_secure_chat` contact sort/match indexes now use `uint16_t` +when the complete table has at most 65,535 slots; larger configurations retain +`int`. Static 350-contact targets save 764 internal bytes; PSRAM targets save +96 internal and 716 external bytes; static 100-contact targets save 264 internal +bytes. Cached nRF52 record placement also saves four bytes per runtime contact. +Capacities, serialization, queues, stacks, features and fallback behavior remain. + +At `aed4c22e5`, matched against `31e693ec2`, RAK4631 Full saves 2,200 static RAM +bytes and 488 actual DFU application bytes. Its application is 712,112 of 712,704 +bytes (592 free); modeled RAM admission margin rises from 4,516 to 6,716. CRC +sharing and Reader packing preserve formats/text; Full features and existing SD +support remain enabled. These are linked/build capacities, not running-heap or +physical storage measurements. + +Five final-source canonical packages pass across nRF52, ESP32 PSRAM, RP2040 and +STM32. RAK3x72 USB has four actual BIN bytes free, requiring unchanged hard gates. +At that revision, V3 Full fails RAM admission on both baseline and final source; its +100-contact wrapper gains 264 bytes, reducing the deficit from 21,288 to 21,024. +The direct 350-contact V3 terminal default build passes separately; it is not a +canonical Full or release-version/preset package. No hardware was flashed. +See the [qualification report](../test-results/contact_memory_2026-10-19.md) and +its source-bound artifacts, software coverage and limits. + +### Heltec V3 Full bounded-source repair, 2026-10-19 UTC + +At `eb9259f2a`, canonical V3 Full passes with 189,616 internal bytes available +against 183,584 required: 6,032 bytes of modeled margin. The pinned NimBLE +adapter preserves 15 saved phones and 30 CCCDs. Existing Bluedroid pairings need +one-time re-pairing. Its 100 contacts, 40 channels, 256 offline frames, MQTT slot, +Full features and stack sizes remain unchanged. + +Grouped source proofs retain the same 4,096-block capacity with a lazy workspace +of 1,004 bytes, statically bounded to 1,536. The guard replaces only the old +16,384-byte leaf allocation with this bound; the output block, metadata, +transient margin and all other allowances remain additive. Legacy/PSRAM paths +and larger custom leaf limits retain the existing policy. + +The clean V3 package and constrained RAK regressions pass; physical V3 testing +still awaits USB attachment. See the [V3 qualification report](../test-results/heltec_v3_full_memory_2026-10-19.md) +for artifact hashes, tests, upgrade behavior and hardware limits. diff --git a/docs/test-results/contact_memory_2026-10-19.json b/docs/test-results/contact_memory_2026-10-19.json new file mode 100644 index 000000000..975c5f365 --- /dev/null +++ b/docs/test-results/contact_memory_2026-10-19.json @@ -0,0 +1,414 @@ +{ + "schema_version": 1, + "date_utc": "2026-10-19", + "source_commit": "aed4c22e51201f0ee3586e26e1007194a74b015d", + "baseline_source_commit": "31e693ec209109a930dcf8aeef89dacbe9116a63", + "hardware_flashed": false, + "release_published": false, + "platformio_single_process": true, + "final_matrix_complete": true, + "all_final_matrix_targets_passed": false, + "packages": [ + { + "requested_target": "RAK_4631_companion_radio_full", + "passed": true, + "exit_code": 0, + "packaged_target": "RAK_4631_companion_radio_full", + "build_profile": "full", + "pio_environment": "RAK_4631_companion_radio_full", + "elf_sha256": "f002455df52a9cdf4278461685eaee3082cc14752f741428f6dfe2dcdc2bb2a7", + "all_memory_bound_file_hashes_match": true, + "memory_manifest_sha256": "073bd0eca6c9b233fcb09080991850b61806fddea3a0af31c52b0feb8bed9762", + "files": { + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.uf2": "31d92e0c48378932ec1465bd5542ca696dcf133a9f6555fd9d513691e1fafec0", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.zip": "d99d904430de75dd5e507a530267d933b8f0bda762afe86cdb150209f4df4499", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.capabilities.json": "d74ca93392bad5ab7cdbe8e07d24a905dbb570cbea76063ad8d9ab34be9cc788", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.full", + "companion.meshtext.channel_text", + "ota.update.bluetooth", + "companion.temp_radio", + "companion.ota_cli", + "companion.usb", + "companion.bluetooth", + "companion.usb_logging", + "companion.usb_mota_source", + "companion.mota_sender", + "companion.dedicated_usb_logging", + "companion.ble_mota_source", + "logging.usb.packets", + "logging.usb.control" + ], + "capability_verification_count": 13, + "capability_verification_passed": true, + "ordinary_static_ram_bytes": 154084, + "static_sections": { + ".data": 1660, + ".bss": 152424 + }, + "static_sections_scope": "ELF .data plus ordinary .bss only; excludes separately named/reserved arenas; null when these exact sections are absent", + "available_internal_bytes": 81292, + "required_heap_bytes": 74576, + "ram_admission_margin_bytes": 6716, + "sd_fat_volume_and_files_reserve_bytes": 2048, + "application_bytes": 712112, + "application_sha256": "c2080afe8874c71eb9a5d1fb253fec06463cac8a774fc776ff46453abf2da420", + "application_source": "DFU application BIN extracted from ZIP", + "uf2_matches_DFU_application": true, + "uf2_blocks": 2782, + "initialized_ELF_flash_load_bytes_match": true, + "initialized_ELF_flash_load_bytes_compared": 712032, + "application_capacity_bytes": 712704, + "application_margin_bytes": 592 + }, + { + "requested_target": "RAK_3401_companion_radio_usb", + "passed": true, + "exit_code": 0, + "packaged_target": "RAK_3401_companion_radio_usb", + "build_profile": "auto", + "pio_environment": "RAK_3401_companion_radio_usb", + "elf_sha256": "0d4a8c90601fc50b4c44f6ee3f315cc2bdfb5b5365cec2efed9d09d54189b9e2", + "all_memory_bound_file_hashes_match": true, + "memory_manifest_sha256": "e22c31d59175cc10c4ecb4f8d04430a859d4e32b450d8f1cd50480f012fd690d", + "files": { + "RAK_3401_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.uf2": "94469781e5550ff41469a5bf80f6f4dba5fc5d2aa2f9f3e073834b06137ef3d0", + "RAK_3401_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.zip": "0709ee56d7f35ac1c9cfa8e8c997b5e2ebd5990c37d044de0e41c6e638f29587", + "RAK_3401_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.capabilities.json": "088d17a9b1de74a851c5d17807a95a27cb4622e24fa05751ff0ef753b21f5ba7", + "RAK_3401_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.auto", + "ota.update.bluetooth" + ], + "capability_verification_count": 1, + "capability_verification_passed": true, + "ordinary_static_ram_bytes": 136564, + "static_sections": { + ".data": 1536, + ".bss": 135028 + }, + "static_sections_scope": "ELF .data plus ordinary .bss only; excludes separately named/reserved arenas; null when these exact sections are absent", + "available_internal_bytes": 98812, + "required_heap_bytes": 47872, + "ram_admission_margin_bytes": 50940, + "sd_fat_volume_and_files_reserve_bytes": 0, + "application_bytes": 523864, + "application_sha256": "a7fa19297c0a9a7f34162fbf666c2a9275794e1ff15b68aa0159817fae7a3bc2", + "application_source": "DFU application BIN extracted from ZIP", + "uf2_matches_DFU_application": true, + "uf2_blocks": 2047, + "initialized_ELF_flash_load_bytes_match": true, + "initialized_ELF_flash_load_bytes_compared": 523864 + }, + { + "requested_target": "Heltec_v3_companion_radio_full", + "passed": false, + "exit_code": 1 + }, + { + "requested_target": "heltec_v4_2_v4_3_companion_radio_full_femon", + "passed": true, + "exit_code": 0, + "packaged_target": "heltec_v4_2_v4_3_companion_radio_full_femon", + "build_profile": "full", + "pio_environment": "heltec_v4_2_v4_3_companion_radio_full_femon", + "elf_sha256": "2c50274bdc350696299819962592032fab540e63f583f2909baf227f875e8219", + "all_memory_bound_file_hashes_match": true, + "memory_manifest_sha256": "6bdcb095bed1fd784169d3986df1ff4212e84ec94bc69c032a0d2581fc79591f", + "files": { + "heltec_v4_2_v4_3_companion_radio_full_femon-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.bin": "ba264695b5093c643f726d970ef7cbfb29207f86454de8f924220276d0d26d70", + "heltec_v4_2_v4_3_companion_radio_full_femon-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e-merged.bin": "f433d9275edb0b0ef470a2866f82ef948a87b7e328f2098f1e06876e8ef6641c", + "heltec_v4_2_v4_3_companion_radio_full_femon-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.capabilities.json": "08dcf0def4a66ebe628df3635cc2eb4f6e1eeea2b78abe550d1533e4601e2a26", + "heltec_v4_2_v4_3_companion_radio_full_femon-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.full", + "companion.meshtext.channel_text", + "companion.temp_radio", + "companion.ota_cli", + "companion.usb", + "companion.bluetooth", + "companion.usb_logging", + "companion.usb_mota_source", + "companion.mota_sender", + "companion.network_terminal", + "companion.wifi_ota_seeder", + "web.webconfig", + "web.unconfigured_setup_cutoff", + "companion.direct_mqtt", + "logging.usb.packets", + "logging.usb.control", + "companion.app_wifi.get_enabled", + "companion.app_wifi.set_enabled", + "companion.app_wifi.get_ssid", + "companion.app_wifi.set_ssid", + "companion.app_wifi.get_password", + "companion.app_wifi.set_password", + "companion.app_wifi.get_status", + "companion.app_wifi.get_ip", + "ota.update.wifi" + ], + "capability_verification_count": 24, + "capability_verification_passed": true, + "ordinary_static_ram_bytes": null, + "static_sections": {}, + "static_sections_scope": "ELF .data plus ordinary .bss only; excludes separately named/reserved arenas; null when these exact sections are absent", + "available_internal_bytes": 228872, + "required_heap_bytes": 194336, + "ram_admission_margin_bytes": 34536, + "sd_fat_volume_and_files_reserve_bytes": 0, + "application_bytes": 2558856, + "application_sha256": "ba264695b5093c643f726d970ef7cbfb29207f86454de8f924220276d0d26d70", + "application_source": "application BIN" + }, + { + "requested_target": "Xiao_rp2040_companion_radio_usb", + "passed": true, + "exit_code": 0, + "packaged_target": "Xiao_rp2040_companion_radio_usb", + "build_profile": "auto", + "pio_environment": "Xiao_rp2040_companion_radio_usb", + "elf_sha256": "832133d6070f4b1b86ac6a9cbd691b5004025e19f7d853ca0cab2d0755284def", + "all_memory_bound_file_hashes_match": true, + "memory_manifest_sha256": "ecd592a61157de92ceb547984baf34f7a55f1fb8b218e6fece487f4ee4897f08", + "files": { + "Xiao_rp2040_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.bin": "757ccb5adb387dab0fccca1793ed70e0713c8ff13e5801763c8a9250f102d806", + "Xiao_rp2040_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.uf2": "8d9386975f8725f2f7bbe6fb1e9a5f81dff84134c710272e95759bd7872ac56e", + "Xiao_rp2040_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.capabilities.json": "f182d520112f6667afdb45eb64197aa025df44b567dac8e370c6892de2858b34", + "Xiao_rp2040_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.auto" + ], + "capability_verification_count": 0, + "capability_verification_passed": true, + "ordinary_static_ram_bytes": 100168, + "static_sections": { + ".data": 5372, + ".bss": 94796 + }, + "static_sections_scope": "ELF .data plus ordinary .bss only; excludes separately named/reserved arenas; null when these exact sections are absent", + "available_internal_bytes": 161748, + "required_heap_bytes": 17408, + "ram_admission_margin_bytes": 144340, + "sd_fat_volume_and_files_reserve_bytes": 0, + "application_bytes": 472204, + "application_sha256": "757ccb5adb387dab0fccca1793ed70e0713c8ff13e5801763c8a9250f102d806", + "application_source": "application BIN" + }, + { + "requested_target": "RAK_3x72_companion_radio_usb", + "passed": true, + "exit_code": 0, + "packaged_target": "RAK_3x72_companion_radio_usb", + "build_profile": "auto", + "pio_environment": "RAK_3x72_companion_radio_usb", + "elf_sha256": "257b0e46be99095b1e9425bdb581b4639e5ee4f126b524ac997c4ffe123865ac", + "all_memory_bound_file_hashes_match": true, + "memory_manifest_sha256": "19c042cf40b21a068dc83a7e3177c2068167379d2f9f9c1a1a431734491de0a3", + "files": { + "RAK_3x72_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.bin": "3b45baa64a7a5a6bbcce18dc17db70c46afa4321e7fd4369fa47b45d53fa73d0", + "RAK_3x72_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.hex": "bb8198b4e06fe0133e70db53eafc33bfe2acfcafabd26b75ddec76c214b0a861", + "RAK_3x72_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-aed4c22e.capabilities.json": "5df1c74616a9fbcf4253dc5294c2367551b1f78ae19e391e7238aaa4e633edcc" + }, + "capabilities": [ + "profile.auto" + ], + "capability_verification_count": 0, + "capability_verification_passed": true, + "ordinary_static_ram_bytes": 39992, + "static_sections": { + ".data": 424, + ".bss": 39568 + }, + "static_sections_scope": "ELF .data plus ordinary .bss only; excludes separately named/reserved arenas; null when these exact sections are absent", + "available_internal_bytes": 24520, + "required_heap_bytes": 17408, + "ram_admission_margin_bytes": 7112, + "sd_fat_volume_and_files_reserve_bytes": 0, + "application_bytes": 229372, + "application_sha256": "3b45baa64a7a5a6bbcce18dc17db70c46afa4321e7fd4369fa47b45d53fa73d0", + "application_source": "application BIN", + "application_capacity_bytes": 229376, + "application_margin_bytes": 4 + } + ], + "linked_checks": { + "crypto": { + "passed": true, + "elf_sha256": "f002455df52a9cdf4278461685eaee3082cc14752f741428f6dfe2dcdc2bb2a7", + "base_table_bytes": 960, + "reader_data_bytes": 38293, + "vectors": [ + { + "rfc8032_vector": 1, + "keypair_stack_bytes": 1152, + "sign_stack_bytes": 1928 + }, + { + "rfc8032_vector": 2, + "keypair_stack_bytes": 1152, + "sign_stack_bytes": 1928 + } + ], + "zero_initialized_path_cache_bytes": 5780, + "stack_limit_bytes": 2048, + "scope": "Crypto function frames only; excludes caller/ISR/task stacks and physical timing.", + "evidence_json_sha256": "8ba582cd89bd4d182d5f9e03c70aeeddfee337b401987debcf3f481ec7c59d4e" + }, + "workspace": { + "passed": true, + "elf_sha256": "f002455df52a9cdf4278461685eaee3082cc14752f741428f6dfe2dcdc2bb2a7", + "queue_lifetimes": 4, + "workspace_address": "0x200212d4", + "protected_sd_auth_range": [ + "0x20006008", + "0x20006050" + ], + "scope": "Linked ARM empty-queue OTA acquisition/release; no hardware qualification.", + "evidence_json_sha256": "658e56af82decc3648a99ef432f547577faeb7139f2bd34e74d6744fb0d8a21d" + }, + "reader": { + "passed": true, + "elf_sha256": "f002455df52a9cdf4278461685eaee3082cc14752f741428f6dfe2dcdc2bb2a7", + "references_fully_reconstructed": 879, + "calls": 1091, + "canonical_bytes_checked": 95689, + "maximum_reply_verse_stack_bytes": 6016, + "caller_reserve_bytes": 1536, + "loop_stack_bytes": 8192, + "total_arm_instructions": 380103543, + "checker_sha256": "02d0eb4a553c9588c698ca5013c269fae16f863d864b1cac521a18fba065f1be", + "canonical_source_sha256": "916bf9d1feae0ad410d69f61e6c5f328ac6acba10651d751b5bb29719732c49e", + "partition_count": 4, + "disjoint_reference_coverage_verified": true, + "all_pages": true, + "evidence_json_sha256": "740aa99a7a514862f6e597efe56ebca47580416dae28db48016ca333d21d2fe8" + } + }, + "adjacent_native_checks": { + "Reader_tests": 16, + "Beacon_tests": 19, + "radio_profile_matrix_scenarios": 32, + "Morse_tests": 7, + "persistent_format_tests": 16 + }, + "canonical_V3_Full_failure": { + "stage": "runtime RAM admission", + "available_internal_bytes": 177408, + "required_heap_bytes": 198432, + "shortfall_bytes": 21024, + "largest_internal_region_bytes": 144640, + "matched_baseline_control": "verified preexisting RAM admission failure", + "baseline_source_commit": "31e693ec209109a930dcf8aeef89dacbe9116a63", + "baseline_available_internal_bytes": 177144, + "baseline_required_heap_bytes": 198432, + "baseline_shortfall_bytes": 21288, + "available_improvement_bytes": 264, + "profile_contact_capacity": 100, + "final_failed_ELF_hash": null, + "failed_ELF_evidence_scope": "final failed ELF was cleaned by later build; current failure is log-bound only", + "direct_terminal_control": "PASS; direct default build, not a canonical Full package", + "baseline_build_log_sha256": "f5f95ec53b9e472cdd5d46f5ec45d61de68e5eace3c2ead53dec7209e5d0f7c8", + "final_build_log_sha256": "732580a8d70c65b73023aec736a95e961da2451fc7ac9a38efeba9a091027dc4" + }, + "focused_contact_checks": { + "unittest_methods_passed": 26, + "method_breakdown": { + "cache_and_persistence_harness": 3, + "persistence_contract": 11, + "stream_contract": 7, + "index_and_layout": 2, + "lifetime_matrix": 1, + "uncached_storage_RP2040_STM32": 2 + }, + "ASan_UBSan_runtime_configurations": 24, + "runtime_configuration_breakdown": { + "cache": 3, + "index": 11, + "lifetime": 8, + "uncached": 2 + }, + "actual_ContactInfo_compile_only_layouts": 8, + "lifetime_assertions": 284, + "contact_sources_unchanged_since": "b0f14161506e2a5f6acfbb4b0b1723e7bd7689c3", + "evidence_scope": "completed tool transcript; no standalone contact log retained" + }, + "limits": [ + "Software/build/linked-ARM qualification only; no board flashed", + "RAM admission and static section totals are not running free heap or physical high-water", + "Physical SD/storage, contact/client behavior, PSRAM pressure, power and radio operation untested" + ], + "historical_failures": [ + "Intermediate b0f141615 small STM32 build failed size admission; retained separately from final-source matrix", + "Legacy Heltec_v3_companion_radio_ble has identical five WiFiSetupPortal/WiFiOtaSeeder unresolved symbols on matched baseline and intermediate revision", + "Initial V4 invocation rejected unsupported --full-exact argument before target compilation" + ], + "baseline_RAK4631_Full": { + "packaged_target": "RAK_4631_companion_radio_full", + "build_profile": "full", + "pio_environment": "RAK_4631_companion_radio_full", + "elf_sha256": "4623af75c924a830415c0a5fd186569b4bbff35a9ce1084438eafe145282058e", + "all_memory_bound_file_hashes_match": true, + "memory_manifest_sha256": "fc1dad5cab6a7deae4ea4042b26dc20231d82710e627fa6e33aef7efecf36c41", + "files": { + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-31e693ec.uf2": "6039f7fe532f121100c1734c9f292ec5f7d7c3ead50b06e7e283fe93569f3fff", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-31e693ec.zip": "ef641d49dc91db61f0a49912f191f6d81c2887a69950fa66a612f81b31f0e142", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-31e693ec.capabilities.json": "91711e0da1a8f3ba4b99c24749c1ac78f0862534f436233ff32c867ccbeb9cc9", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-31e693ec.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.full", + "companion.meshtext.channel_text", + "ota.update.bluetooth", + "companion.temp_radio", + "companion.ota_cli", + "companion.usb", + "companion.bluetooth", + "companion.usb_logging", + "companion.usb_mota_source", + "companion.mota_sender", + "companion.dedicated_usb_logging", + "companion.ble_mota_source", + "logging.usb.packets", + "logging.usb.control" + ], + "capability_verification_count": 13, + "capability_verification_passed": true, + "ordinary_static_ram_bytes": 156284, + "static_sections": { + ".data": 1660, + ".bss": 154624 + }, + "static_sections_scope": "ELF .data plus ordinary .bss only; excludes separately named/reserved arenas; null when these exact sections are absent", + "available_internal_bytes": 79092, + "required_heap_bytes": 74576, + "ram_admission_margin_bytes": 4516, + "sd_fat_volume_and_files_reserve_bytes": 2048, + "application_bytes": 712600, + "application_sha256": "520cf40ec5ada891d3fe8081b23546ce777c712383f0479292bb8fac0017d4d7", + "application_source": "DFU application BIN extracted from ZIP", + "uf2_matches_DFU_application": true, + "uf2_blocks": 2784, + "initialized_ELF_flash_load_bytes_match": true, + "initialized_ELF_flash_load_bytes_compared": 712520, + "application_capacity_bytes": 712704, + "application_margin_bytes": 104 + }, + "direct_terminal_build": { + "target": "Heltec_v3_terminal_chat", + "source_commit": "aed4c22e51201f0ee3586e26e1007194a74b015d", + "passed": true, + "scope": "direct PlatformIO default environment; not a firmware-picker Full or version/preset-qualified release package", + "contact_capacity": 350, + "elf_sha256": "1a193ae1bee829d0bf7d140310b8487d2a4671f0c85715d16cfb764850e06cb6", + "memory_manifest_sha256": "a74079788410d8cdf11e686b62f8aee80537a9743cd0590bf4caf069ab7d038d", + "available_internal_bytes": 263800, + "required_heap_bytes": 77856, + "ram_admission_margin_bytes": 185944, + "build_log_sha256": "31d410458e6e6cd87bce96019e1ce655ab698eeb5e5d720987461c7be07f3fdf" + } +} diff --git a/docs/test-results/contact_memory_2026-10-19.md b/docs/test-results/contact_memory_2026-10-19.md new file mode 100644 index 000000000..d946cee71 --- /dev/null +++ b/docs/test-results/contact_memory_2026-10-19.md @@ -0,0 +1,178 @@ + + + + +# Contact RAM savings and RAK4631 Full capacity, 2026-10-19 + +Shared contact indexes save RAM across Companion profiles and terminal chat. +RAK4631 Full additionally saves **2,200 static RAM bytes and 488 application +bytes**, preserving its existing features and SD support. Five canonical +packages and a direct terminal build pass. Heltec V3 Full failed its preexisting +RAM-admission limit at this revision; the subsequent +[V3 Full repair](heltec_v3_full_memory_2026-10-19.md) is measured separately. +**No hardware was flashed for this contact-memory campaign.** + +Baseline source: [`31e693ec209109a930dcf8aeef89dacbe9116a63`](https://github.com/mikecarper/MeshCore/commit/31e693ec209109a930dcf8aeef89dacbe9116a63). +Final source: [`aed4c22e51201f0ee3586e26e1007194a74b015d`](https://github.com/mikecarper/MeshCore/commit/aed4c22e51201f0ee3586e26e1007194a74b015d). +The [machine-readable results](contact_memory_2026-10-19.json) bind package, +ELF, memory-manifest and linked-check hashes to these revisions. + +## Shared contact savings + +`BaseChatMesh` uses `uint16_t` sort/match indexes when the complete contact table +has at most 65,535 slots; larger configurations keep `int`. API counts retain +their existing types. All Companion profiles and `simple_secure_chat` share this +implementation across nRF52, ESP32, RP2040 and STM32. Infrastructure client/ACL +tables use different record types and are outside this contact-table change. + +| Existing configuration | Internal RAM saved by indexes | PSRAM saved by indexes | +| --- | ---: | ---: | +| Static 350-contact table | 764 bytes | None | +| PSRAM 350-contact table with internal fallback | 96 bytes | 716 bytes | +| Static 100-contact table | 264 bytes | None | + +These totals include eight anonymous slots and 24 match results. PSRAM retains +its existing 24-slot internal fallback and allocation-failure behavior. Cached +nRF52 `ContactInfo` records also shrink from 96 to 92 bytes by moving the +runtime-only `storage_slot` field into an alignment gap; other record layouts +retain their sizes. Field-based serialization, contact/channel capacities, +features, queues and stack allocations remain unchanged. + +## RAK4631 Full comparison + +| Measurement | Baseline | Final | Improvement | +| --- | ---: | ---: | ---: | +| Actual application BIN extracted from DFU ZIP | 712,600 | 712,112 | 488 fewer | +| Application-region capacity | 712,704 | 712,704 | Unchanged | +| Actual flash margin | 104 | 592 | 488 more | +| Ordinary static RAM (`.data` + `.bss`) | 156,284 | 154,084 | 2,200 fewer | +| Linked heap capacity | 79,092 | 81,292 | 2,200 more | +| Modeled heap requirement | 74,576 | 74,576 | Unchanged | +| Modeled admission margin | 4,516 | 6,716 | 2,200 more | + +`.data` stays at 1,660 bytes; ordinary `.bss` falls from 154,624 to 152,424. +Separately reserved linker arenas are excluded from this section sum. The +measured reduction includes final object alignment. Linked capacities and +admission models do not measure running free heap or physical stack high-water. + +Four duplicate CRC loops share the existing implementation without changing +initial values, byte ranges or stored formats. Reader packing increases offline +Zopfli iterations from 15 to 1,000 and selects 16-bit block offsets only when +every block start fits. General 32-bit types/fallback, 4,096-byte blocks, +scratch policy and all canonical text remain unchanged. + +| Reader flash data | Baseline | Final | Saving | +| --- | ---: | ---: | ---: | +| Compressed payload | 38,366 | 38,293 | 73 bytes | +| Twenty-four block indexes | 288 | 192 | 96 bytes | +| Payload plus index | 38,654 | 38,485 | 169 bytes | + +The complete image saves 488 bytes; Reader payload plus index accounts for 169. +The Full marker and thirteen feature-verification entries are unchanged. +Existing SD support, its 2,048-byte SD/FAT runtime reserve and protected +authentication range remain intact. No physical SD operation was tested. + +## Source-bound artifacts + +| Artifact | SHA-256 | +| --- | --- | +| Baseline DFU application | `520cf40ec5ada891d3fe8081b23546ce777c712383f0479292bb8fac0017d4d7` | +| Final DFU application | `c2080afe8874c71eb9a5d1fb253fec06463cac8a774fc776ff46453abf2da420` | +| Baseline DFU ZIP | `ef641d49dc91db61f0a49912f191f6d81c2887a69950fa66a612f81b31f0e142` | +| Final DFU ZIP | `d99d904430de75dd5e507a530267d933b8f0bda762afe86cdb150209f4df4499` | +| Final ELF | `f002455df52a9cdf4278461685eaee3082cc14752f741428f6dfe2dcdc2bb2a7` | + +Every admitted package's memory-bound file hashes were recomputed. Nordic UF2 +payloads match the DFU applications, and initialized ELF flash-load bytes agree. +The JSON records the individual hashes and actual packaged selectors/profiles. + +## Software verification + +Contact coverage totals **26 Python unittest methods**: three cache/persistence +harness methods, eleven persistence-contract methods, seven stream-contract +methods, two index/layout methods, one lifetime matrix, and two RP2040/STM32 +uncached-storage methods. They execute **24 ASan/UBSan C++ configurations** +(three cache, eleven index, eight lifetime, two uncached) and separately compile +eight actual cached/uncached record layouts across the four MCU families. The +lifetime matrix checks 284 assertions. Index execution uses a small stand-in +record; separate layout checks compile the real header. Contact code is +unchanged since `b0f141615`; results are retained in the tool transcript, with +no separate contact log claimed. + +Adjacent checks pass: 16 Reader tests, 19 Beacon tests, a radio-profile matrix +with 32 scenarios, seven Morse-codec tests and sixteen persistent-format tests. +These focused counts do not establish a complete hosted CI run. + +Final linked ARM checks all bind to the ELF above: + +- Crypto: both RFC8032 vectors pass; keypair/sign frames are 1,152/1,928 bytes, + within the existing 2,048-byte function-frame bound. +- OTA workspace: four acquisition/release lifetimes pass, preserving protected + SD authentication range `0x20006008`–`0x20006050`. +- Reader: four disjoint partitions reconstruct all 879 references and every + continuation page: 95,689 canonical bytes through 1,091 calls and 380,103,543 + ARM instructions. Exact disjoint reference coverage is verified. Maximum + callee frame is 6,016 bytes; with the existing 1,536-byte caller reserve this + is 7,552 of the 8,192-byte loop-stack model. Canonical source SHA-256 remains + `916bf9d1feae0ad410d69f61e6c5f328ac6acba10651d751b5bb29719732c49e`. + +The partitioned campaign invokes the unchanged linked Reader checker with +all-page reconstruction; partitioning is not an additional release gate. +Equivalent complete reproduction against the matching ELF uses Unicorn 2.1.4: + +```sh +python scripts/check_nrf52_companion_crypto.py path/to/firmware.elf +python scripts/check_nrf52_companion_ota_workspace.py path/to/firmware.elf +python scripts/check_nrf52_companion_reader.py path/to/firmware.elf --every-page +``` + +## Final-source build matrix + +Canonical packages used `v1.18.0.1-halo-keymind-cascade-dev`, radio preset +`usa-cascade-fixed` and profile `cascade`; Full selections required OTA. +PlatformIO commands ran one at a time. For example: + +```sh +bash build.sh build-firmware RAK_4631_companion_radio_full --firmware-version v1.18.0.1-halo-keymind-cascade-dev --radio-preset usa-cascade-fixed --profile cascade --require-ota +``` + +| Requested target | Actual application bytes | Modeled RAM margin | Result | +| --- | ---: | ---: | --- | +| `RAK_4631_companion_radio_full` | 712,112 | 6,716 | PASS | +| `RAK_3401_companion_radio_usb` | 523,864 | 50,940 | PASS | +| `heltec_v4_2_v4_3_companion_radio_full_femon` | 2,558,856 | 34,536 | PASS, Full PSRAM profile | +| `Xiao_rp2040_companion_radio_usb` | 472,204 | 144,340 | PASS | +| `RAK_3x72_companion_radio_usb` | 229,372 | 7,112 | PASS; four actual flash bytes free | +| `Heltec_v3_companion_radio_full` | No admitted package | -21,024 | Verified preexisting RAM-gate failure | + +Matched baseline V3 Full also fails: 177,144 available against 198,432 required +(21,288 short), versus final 177,408 (21,024 short). This wrapper preserves 100 +contacts, gaining 264 bytes. The final failed ELF was cleaned by a later build; +the failure is bound to its build log; that ELF hash is not recorded. + +The direct `pio run -e Heltec_v3_terminal_chat` default build passes with 350 +contacts: 263,800 available, 77,856 required and 185,944 margin. ELF SHA-256 is +`1a193ae1bee829d0bf7d140310b8487d2a4671f0c85715d16cfb764850e06cb6`. +This qualifies a no-PSRAM/shared-terminal compile path; it is not a Full package +or a firmware-picker release-version/preset qualification. + +## Retained failures and limits + +Intermediate `b0f141615` failed the STM32 size check. Shared little-endian Morse +serialization and an equivalent branchless CRC loop restore the packaged fit. +STM32 static RAM +falls from 40,256 to 39,992 bytes; baseline actual BIN was 229,284/229,376 bytes +(92 free), although PlatformIO's section summary showed 444 free. Acceptance +uses actual BIN/ELF extent. The final four-byte margin and RAK4631 Full's +592-byte margin require the unchanged hard gates for subsequent versions. + +The separate legacy V3 BLE target has the same five unresolved WiFi setup/seeder +symbols on matched baseline and intermediate source. Initial disk-quota failures +and an unsupported V4 command flag remain separate failed attempts; successful +retries establish the results above. No features, capacities or budgets were +reduced to obtain these passes. There is no six-canonical-pass claim. + +These are software/build/linked-code results. No board was flashed. Physical +contact/client operation, PSRAM pressure/fallback, storage persistence, SD/radio +operation, power, timing and full-firmware ISR/SoftDevice stack high-water remain +outside this qualification. diff --git a/docs/test-results/heltec_v3_full_memory_2026-10-19.json b/docs/test-results/heltec_v3_full_memory_2026-10-19.json new file mode 100644 index 000000000..400b5b5a3 --- /dev/null +++ b/docs/test-results/heltec_v3_full_memory_2026-10-19.json @@ -0,0 +1,284 @@ +{ + "schema_version": 1, + "date_utc": "2026-10-19", + "source_commit": "eb9259f2a97113b1d482ca9610661fb24c1c8e98", + "baseline_source_commit": "31e693ec209109a930dcf8aeef89dacbe9116a63", + "all_final_builds_passed": true, + "single_process_platformio": true, + "hardware_validation": { + "performed": false, + "flashed": false, + "status": "Awaiting physical V3 USB attachment to the VM; no V3 enumerated during inventory." + }, + "v3_capacity": { + "contacts": 100, + "channels": 40, + "offline_frames": 256, + "active_ble_sessions": 1, + "saved_phone_bonds": 15, + "cccd_records": 30, + "mqtt_runtime_slots": 1 + }, + "v3_proof_workspace": { + "maximum_source_blocks": 4096, + "actual_xtensa_bytes": 1004, + "reserved_bytes": 1536, + "output_block_bytes": 2048, + "allocation_metadata_bytes": 32, + "legacy_source_leaf_bytes": 16384, + "legacy_larger_custom_limits_retained": true + }, + "v3_baseline_memory": { + "available_internal_bytes": 177144, + "required_heap_bytes": 198432, + "margin_bytes": -21288 + }, + "software_checks": { + "bounded_source_proofs": { + "python_tests": 3, + "exact_wire_proofs": 22240, + "sanitizers": [ + "ASan", + "UBSan" + ] + }, + "firmware_ram_policy": { + "python_tests": 47 + }, + "nimble_adapter_and_variant": { + "python_tests": 10, + "adapter_sanitizers": [ + "ASan", + "UBSan" + ] + }, + "legacy_ota_cold_buffers": { + "configurations": 4 + }, + "serial_deadlines": { + "python_tests": 2 + }, + "source_only_folder_resume": { + "python_tests": 2 + } + }, + "packages": [ + { + "target": "Heltec_v3_companion_radio_full", + "passed": true, + "exit_code": 0, + "elapsed_seconds": 54.22, + "elf_sha256": "3a56805b2d0216026c3710e149db43dedb677684e47ad08ebfe748094c1de55b", + "available_internal_bytes": 189616, + "required_heap_bytes": 183584, + "margin_bytes": 6032, + "memory_manifest_sha256": "dc03ea7ba74caadaf5e5fe43f64affacf14481cdadbfd37ca241eb7bb47c1548", + "memory_bound_file_hashes_verified": true, + "files": { + "Heltec_v3_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.bin": "c4f276095626560abc4c6875216c147d2fae292bf5c514e5029ef7d03454d3ce", + "Heltec_v3_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2-merged.bin": "8ab56857805925666cf9c8638aa23851bea0d36f01a9f6ca270ad5776c2a0245", + "Heltec_v3_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.capabilities.json": "d03182ad9405a2cf692687de11cc686fdf3600725d2761db3198cd7236bdba4a", + "Heltec_v3_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.full", + "companion.meshtext.channel_text", + "companion.temp_radio", + "companion.ota_cli", + "companion.usb", + "companion.bluetooth", + "companion.usb_logging", + "companion.usb_mota_source", + "companion.mota_sender", + "companion.network_terminal", + "companion.wifi_ota_seeder", + "web.webconfig", + "web.unconfigured_setup_cutoff", + "companion.direct_mqtt", + "logging.usb.packets", + "logging.usb.control", + "companion.app_wifi.get_enabled", + "companion.app_wifi.set_enabled", + "companion.app_wifi.get_ssid", + "companion.app_wifi.set_ssid", + "companion.app_wifi.get_password", + "companion.app_wifi.set_password", + "companion.app_wifi.get_status", + "companion.app_wifi.get_ip", + "ota.update.wifi" + ], + "feature_checks": 24, + "capabilities_verified": true, + "existing_recipe_reductions": [ + "companion.capacity limited to 100 contacts for runtime RAM; 256 queued frames and all Full transports retained" + ], + "components": { + "core_tasks_usb_filesystems": 24576, + "wireless_stacks": 81920, + "mqtt_startup_allowance": 24576, + "ota_source_scratch": 8192, + "browser_terminal_session": 2048, + "browser_terminal_scrollback": 4096, + "radio_packet_pool": 5120, + "ota_generic_source_buffers": 3616, + "display_pixels_and_driver": 2048, + "screen_objects_and_history": 8192, + "expanded_message_previews": 2816, + "allocation_and_transient_margin": 16384 + }, + "required_contiguous_bytes": 11008, + "largest_internal_region_bytes": 156848, + "application_bytes": 2114744, + "application_sha256": "c4f276095626560abc4c6875216c147d2fae292bf5c514e5029ef7d03454d3ce", + "application_capacity_bytes": 3342336, + "application_margin_bytes": 1227592 + }, + { + "target": "RAK_4631_companion_radio_full", + "passed": true, + "exit_code": 0, + "elapsed_seconds": 34.08, + "elf_sha256": "7f123f4c4a43200fd85fb889bb388b4f22318fd395eeafe64c014af14be3804e", + "available_internal_bytes": 81292, + "required_heap_bytes": 74576, + "margin_bytes": 6716, + "memory_manifest_sha256": "c58fdd1cf010ca8d7c5b0c33549f941482cb6ca11deb361e04c2d4770eaa4f22", + "memory_bound_file_hashes_verified": true, + "files": { + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.uf2": "b532062fec9d5fe53d490010fea26dfbad334e0ffb66a4f483def390763a31ef", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.zip": "dd58370fe7360df078b645813d633a31567780dfab02dbde5e0ec28e120b0018", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.capabilities.json": "a54fc3dc71fd31eb5ddfb8c4b7368fcec6446fe41729f05ef3408ca6be270e87", + "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.meshtext-license.txt": "ba558609720fbff1481e236462eea2f9e08a73c1f3c2fd486ea4d9e1d0a6d58f" + }, + "capabilities": [ + "profile.full", + "companion.meshtext.channel_text", + "ota.update.bluetooth", + "companion.temp_radio", + "companion.ota_cli", + "companion.usb", + "companion.bluetooth", + "companion.usb_logging", + "companion.usb_mota_source", + "companion.mota_sender", + "companion.dedicated_usb_logging", + "companion.ble_mota_source", + "logging.usb.packets", + "logging.usb.control" + ], + "feature_checks": 13, + "capabilities_verified": true, + "existing_recipe_reductions": [ + "nRF52 Full: 256 offline frames normally; 128 while mOTA borrows queue storage" + ], + "components": { + "loop_and_callback_stacks": 11264, + "core_usb_filesystems_sensors": 12288, + "sd_fat_volume_and_files": 2048, + "ota_target_name_lookup": 4112, + "bluetooth_worker_stacks": 5920, + "radio_packet_pool": 5120, + "ota_generic_source_buffers": 10272, + "ethernet_companion_worker": 4352, + "display_pixels_and_driver": 4096, + "screen_objects_and_history": 8192, + "expanded_message_previews": 2816, + "allocation_and_transient_margin": 4096 + }, + "required_contiguous_bytes": 11008, + "largest_internal_region_bytes": 81292, + "application_bytes": 712112, + "application_sha256": "2707606a9f4af071c9c4720465a7e7f17132292abc4af9eb468578d456480779", + "application_capacity_bytes": 712704, + "application_margin_bytes": 592 + }, + { + "target": "RAK_3x72_companion_radio_usb", + "passed": true, + "exit_code": 0, + "elapsed_seconds": 22.35, + "elf_sha256": "1e686c6ad520e84f2ede3f50037c5d1c4c8e4266c0e9fac605d53d67ee4bd593", + "available_internal_bytes": 24520, + "required_heap_bytes": 17408, + "margin_bytes": 7112, + "memory_manifest_sha256": "687267b1bd35019f9683b46fe52ea4c23001dc4ef1f2aa14f51a2e917a9885fc", + "memory_bound_file_hashes_verified": true, + "files": { + "RAK_3x72_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.bin": "b2770a66e75534fb48d987d2cd1bc71ea178bd7e1e720ebf0d45788efeaa7621", + "RAK_3x72_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.hex": "a583bc987801281c6c6eb5a4cb9a069e72fd4a5e7b5c3863d8df5f2b31abea58", + "RAK_3x72_companion_radio_usb-v1.18.0.1-halo-keymind-cascade-dev-eb9259f2.capabilities.json": "7977ab1c0bb8f7cb678ae675816dc3a12639d1b2e1d49d6f98bdac1b349bc86f" + }, + "capabilities": [ + "profile.auto" + ], + "feature_checks": 0, + "capabilities_verified": true, + "existing_recipe_reductions": [], + "components": { + "core_filesystems_sensors": 8192, + "radio_packet_pool": 5120, + "allocation_and_transient_margin": 4096 + }, + "required_contiguous_bytes": 5120, + "largest_internal_region_bytes": 24520, + "application_bytes": 229372, + "application_sha256": "b2770a66e75534fb48d987d2cd1bc71ea178bd7e1e720ebf0d45788efeaa7621", + "application_capacity_bytes": 229376, + "application_margin_bytes": 4 + } + ], + "rak_linked_arm_checks": { + "crypto": { + "passed": true, + "elf_sha256": "7f123f4c4a43200fd85fb889bb388b4f22318fd395eeafe64c014af14be3804e", + "base_table_bytes": 960, + "reader_data_bytes": 38293, + "vectors": [ + { + "rfc8032_vector": 1, + "keypair_stack_bytes": 1152, + "sign_stack_bytes": 1928 + }, + { + "rfc8032_vector": 2, + "keypair_stack_bytes": 1152, + "sign_stack_bytes": 1928 + } + ], + "zero_initialized_path_cache_bytes": 5780, + "stack_limit_bytes": 2048, + "scope": "Crypto function frames only; excludes caller/ISR/task stacks and physical timing." + }, + "workspace": { + "passed": true, + "elf_sha256": "7f123f4c4a43200fd85fb889bb388b4f22318fd395eeafe64c014af14be3804e", + "queue_lifetimes": 4, + "workspace_address": "0x200212d4", + "protected_sd_auth_range": [ + "0x20006008", + "0x20006050" + ], + "scope": "Linked ARM empty-queue OTA acquisition/release; no hardware qualification." + }, + "reader": { + "passed": true, + "elf_sha256": "7f123f4c4a43200fd85fb889bb388b4f22318fd395eeafe64c014af14be3804e", + "all_pages": true, + "references_fully_reconstructed": 879, + "disjoint_reference_coverage_verified": true, + "calls": 1091, + "canonical_bytes_checked": 95689, + "maximum_reply_verse_stack_bytes": 6016, + "caller_reserve_bytes": 1536, + "loop_stack_bytes": 8192, + "checker_sha256": "02d0eb4a553c9588c698ca5013c269fae16f863d864b1cac521a18fba065f1be", + "canonical_source_sha256": "916bf9d1feae0ad410d69f61e6c5f328ac6acba10651d751b5bb29719732c49e" + } + }, + "scope": "Linked capacity and host/linked-code verification; physical V3 boot, transport/load and power-loss behavior are not qualified.", + "linked_ble_store_bytes": { + "ble_store_config_our_secs": 1320, + "ble_store_config_peer_secs": 1320, + "ble_store_config_cccds": 480 + } +} diff --git a/docs/test-results/heltec_v3_full_memory_2026-10-19.md b/docs/test-results/heltec_v3_full_memory_2026-10-19.md new file mode 100644 index 000000000..3717bc735 --- /dev/null +++ b/docs/test-results/heltec_v3_full_memory_2026-10-19.md @@ -0,0 +1,96 @@ + + + + +# Heltec V3 Full memory qualification, 2026-10-19 + +Canonical V3 Full now passes its flash and runtime-RAM gates with **6,032 bytes +of modeled RAM margin**, retaining its existing features and capacities. +**Physical testing is pending:** no V3 is currently exposed to the VM, and no +board was flashed during this work. + +Measured source: [`eb9259f2a97113b1d482ca9610661fb24c1c8e98`](https://github.com/mikecarper/MeshCore/commit/eb9259f2a97113b1d482ca9610661fb24c1c8e98). +The [machine-readable evidence](heltec_v3_full_memory_2026-10-19.json) records +package hashes, allocation components, capabilities and linked ARM checks. + +## What changed + +V3 Full uses the existing NimBLE-Arduino 2.5.1 Companion adapter. It preserves +the authenticated NUS service, PIN, address/stealth controls and frame queues. +The canonical profile retains one active BLE session, **15 saved phones and +30 CCCD records** for NUS TX and Service Changed subscriptions. This does not +reuse the smaller default bond store from the optional trial configuration. + +**Bluetooth upgrade step:** existing Bluedroid bonds are not migrated. Forget +the previous pairing in the phone's Bluetooth settings and pair again after +installing this image. The PIN setting remains available as before. + +Non-PSRAM ESP32 Full also uses a bounded external-source Merkle cache. It keeps +32-leaf groups and their roots, then rereads the selected group to generate the +same wire proof. It supports the same 4,096 source blocks, folder capture, +raw/host-encoded/device-encoded serving, getter requests and transport protocols. +A proof step issues at most one metadata transaction; jobs survive partial +progress and radio backpressure. + +The complete workspace is 1,004 bytes in the 32-bit Xtensa layout, with a +production `sizeof` assertion enforcing a 1,536-byte bound. Its output block and +allocation allowance remain separate. The legacy 16 KiB source-leaf allocation +is absent on this path. PSRAM, nRF52 and other unqualified paths retain their +existing implementation, as do explicit larger custom source limits. + +The RAM check accounts for this smaller real allocation. The 16 KiB transient +margin, Bluetooth/WiFi allowances, MQTT admission, stacks and minimum floors +remain unchanged. There is no new OTA queue loan or reduction in offline frames. + +## Measured image + +Both comparisons use the canonical release-length version, Cascade profile and +USA Cascade preset. The baseline is `31e693ec2`; it already failed admission +before the shared contact-memory changes. + +| Measurement | Baseline | Final | +| --- | ---: | ---: | +| Internal capacity after linking | 177,144 | 189,616 | +| Modeled allocation requirement | 198,432 | 183,584 | +| Modeled margin | -21,288 | **6,032** | +| Application bytes | Not admitted | 2,114,744 | +| Application-slot capacity | — | 3,342,336 | +| Application-slot margin | — | 1,227,592 | + +The existing V3 Full recipe retains **100 contacts, 40 channels, 256 offline +frames and one MQTT runtime slot**. These limits were not reduced by this fix. +All 24 linked/package feature checks pass, alongside the Full profile marker. +USB, Bluetooth, simultaneous WiFi, direct MQTT, WebConfig, logging and OTA remain +in the image. Modeled capacity is not a running free-heap or load measurement. + +Final ELF SHA-256: `3a56805b2d0216026c3710e149db43dedb677684e47ad08ebfe748094c1de55b`. +Application SHA-256: `c4f276095626560abc4c6875216c147d2fae292bf5c514e5029ef7d03454d3ce`. +Every file bound by the packaged memory manifest was hash-verified. + +## Verification + +- New proof tests pass under ASan/UBSan, with 22,240 exact wire-proof comparisons, + partial/failing reads, negotiated transport-size changes, allocation failure, + cancellation, idle cleanup, folder capture, getter bytes and backpressure. +- The RAM-policy suite passes 47 tests. Ten adapter/variant tests pass, including + real adapter behavior under ASan/UBSan and the V3 DIO policy. +- Existing cold-buffer tests pass on four platform configurations. Serial + deadlines and source-only folder-resume tests each pass two test methods. +- Clean-source canonical RAK4631 Full and RAK3x72 USB also build. RAK4631 retains + 6,716 modeled RAM bytes and 592 application bytes of margin; RAK3x72 retains + 7,112 RAM bytes and four application bytes. Their existing hard gates remain. +- The final-source RAK4631 ELF passes linked RFC8032 crypto, four OTA workspace + lifetimes and every page of all 879 Reader references. Those checks bind to + its own ELF hash in the JSON rather than to an earlier contact-memory image. + +PlatformIO commands ran one at a time in a clean checkout. Reproduce the V3 +package with: + +```sh +bash build.sh build-firmware Heltec_v3_companion_radio_full --firmware-version v1.18.0.1-halo-keymind-cascade-dev --radio-preset usa-cascade-fixed --profile cascade --require-ota +``` + +Hardware acceptance still requires the physical V3: preserve its flash and +settings, then check boot, live heap under simultaneous transports, contacts, +Bluetooth pairing/reconnect, MQTT, WiFi administration, OTA and persistence. +These software results do not establish that hardware acceptance. diff --git a/docs/test-results/index.md b/docs/test-results/index.md index e39a1986a..4a0776a92 100644 --- a/docs/test-results/index.md +++ b/docs/test-results/index.md @@ -60,6 +60,8 @@ Completed validation reports, experiments, and test guides are collected here. O ## Build, storage, memory, and checklists +- [Shared contact RAM savings and RAK4631 Full qualification, 2026-10-19](contact_memory_2026-10-19.md) +- [Heltec V3 Full memory qualification, 2026-10-19](heltec_v3_full_memory_2026-10-19.md) - [Personal room mailbox checks](room_mailboxes.md) - [Stock nRF52 bootloader version audit](bootloader_version_stock_audit.md) - [CLI setting dispatch audit](cli_settings_audit.md) diff --git a/docs/test-results/nimble_companion_trial.md b/docs/test-results/nimble_companion_trial.md index 932f9fede..3fe8806af 100644 --- a/docs/test-results/nimble_companion_trial.md +++ b/docs/test-results/nimble_companion_trial.md @@ -4,6 +4,12 @@ # ESP32-S3 NimBLE Full Companion trial +Canonical Heltec V3 Full now uses the same pinned NimBLE backend, preserving +its existing 15 saved phones and both notification subscriptions per phone. +Its [Full memory qualification](heltec_v3_full_memory_2026-10-19.md) is separate +from the optional trials below. Switching from Bluedroid requires forgetting +the old phone pairing and pairing again; stored bonds are not migrated. + These optional builds use NimBLE-Arduino 2.5.1 with the existing ESP32 Arduino 2.0.17 toolchain. They are hardware qualification builds, outside the normal release matrix and firmware picker.