mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-22 22:57:59 +00:00
fix(build): pin musl thread stacks at 8 MiB and check the artifacts (#2410)
## Summary Pins musl thread stacks at 8 MiB on every link, and fixes a crash in **native Alpine builds** (`make build`, `make test`, or `go test` on an Alpine host). **The released images are not affected.** An earlier version of this description said public endpoints could crash the shipped image. That was wrong, and review caught it: see "Who is affected". ## Root cause - musl sizes every non-main thread's stack from the ELF `PT_GNU_STACK` size, with a floor of 128 KiB and a cap of 8 MiB (glibc defaults to 8 MiB). - cgo calls run on the calling thread's stack, and SQLite's `multiSelect` recurses about 850 bytes per `UNION ALL` term. - `lookupPrefixes` (up to 400 terms) and `sharedEightBytePrefixes` (up to 450 terms) overflow a 128 KiB stack and SIGSEGV the whole process. ## Who is affected | Link | `PT_GNU_STACK` without the flag | Thread stack | Wide-UNION tests | |---|---|---|---| | Alpine gcc + GNU ld (`make build` / `make test` / `go test` on Alpine) | `0` | 128 KiB | **SIGSEGV** | | zig 0.16.0, x86_64-linux-musl (Dockerfile, `crossbuild`, release job) | `0x1000000` (16 MiB) | 8 MiB | pass 3/3 | | zig 0.16.0, aarch64-linux-musl | `0x1000000` (16 MiB) | 8 MiB | pass 3/3, run on an arm64 node | So for the released images the flag pins a value that zig's default already provides. It stops being a property of a linker default, and a build check now fails if it is lost. For native Alpine links it is the fix: with the flag those tests pass. Control: a zig arm64 link with `-Wl,-z,stack-size=131072` gets 128 KiB threads and the same tests SIGSEGV in cgo on the arm64 node. ## Fix - Every musl link passes `-Wl,-z,stack-size=8388608`: the `Dockerfile`, the `Makefile` (`build` and `test` on Linux, and `crossbuild`) and the release job in `deploy.yml`. - This reserves address space only. Memory is committed as pages are touched. - **The artifacts are checked.** The Dockerfile's builder stage and the release job's "Verify release binaries" step read `PT_GNU_STACK` with `readelf` and fail unless it is exactly 8 MiB. Verified: the builder stage passes for `linux/amd64` and `linux/arm64`, and fails with `--build-arg STACK_LDFLAG=` (`/corescope-server: PT_GNU_STACK is not 8 MiB`, memsz `0x1000000`). ## Testing - `TestMuslThreadStackWideUnionQueries` (`cmd/server/musl_thread_stack_linux_test.go`, Linux only) runs both queries off the main thread and asserts exact results, so a failed chunk cannot pass as "nothing matched". Under `golang:1.27-alpine` with gcc: SIGSEGV without the flag, pass with it. On CI's glibc runners it passes either way, which is why the artifact checks above exist. - `tests/unit/test-issue-1956-release-routing.js`: the release verify step must check both binaries, and a `0x000000` or `0x1000000` header must fail the release. - `tests/unit/test-dockerfile-buildplatform.js`: all three `go build` lines carry `STACK_LDFLAG`, and the `readelf` check covers all three binaries. - `sh test-all.sh` passes. `go vet` passes for `cmd/server` on Linux with gofmt clean. - Not verified: a Windows `go vet`. The fix there is the `_linux_test.go` file name; no Windows machine was available to run it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## Merge order These four PRs were split out of #2396 and are **independent — they can merge in any order.** Verified: every pair is conflict-free, and all four merged together pass `go vet`, `go test ./...` (all 16 modules) and `sh test-all.sh`. (That combined run predates the review fixes on #2409 and #2410.) Suggested order, none of it urgent: 1. **#2408** — never lose rows during startup / a failed poll. 2. **#2407** — explicit `observers.idx` primary key. 3. **#2409** — content-hash migration moved into the ingestor. 4. **#2410** — pin musl thread stacks at 8 MiB. Released images are not affected; it fixes native Alpine builds. After these land, #2396 (Postgres) will be rebased onto master. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: efiten <erwin.fiten@gmail.com>
This commit is contained in:
6 files changed
+179
-11
No files matched your search
@@ -1167,18 +1167,21 @@ jobs:
|
||||
- name: Build corescope-decrypt (static, linux/amd64)
|
||||
run: |
|
||||
cd cmd/decrypt
|
||||
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 CC="zig cc -target x86_64-linux-musl" go build -trimpath -tags netgo,osusergo,sqlite_omit_load_extension -ldflags="-s -w -extldflags '-static -Wl,-s' -X main.version=${{ github.ref_name }}" -o ../../corescope-decrypt-linux-amd64 .
|
||||
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 CC="zig cc -target x86_64-linux-musl" go build -trimpath -tags netgo,osusergo,sqlite_omit_load_extension -ldflags="-s -w -extldflags '-static -Wl,-s -Wl,-z,stack-size=8388608' -X main.version=${{ github.ref_name }}" -o ../../corescope-decrypt-linux-amd64 .
|
||||
|
||||
- name: Build corescope-decrypt (static, linux/arm64)
|
||||
run: |
|
||||
cd cmd/decrypt
|
||||
CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC="zig cc -target aarch64-linux-musl" go build -trimpath -tags netgo,osusergo,sqlite_omit_load_extension -ldflags="-s -w -extldflags '-static -Wl,-s' -X main.version=${{ github.ref_name }}" -o ../../corescope-decrypt-linux-arm64 .
|
||||
CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC="zig cc -target aarch64-linux-musl" go build -trimpath -tags netgo,osusergo,sqlite_omit_load_extension -ldflags="-s -w -extldflags '-static -Wl,-s -Wl,-z,stack-size=8388608' -X main.version=${{ github.ref_name }}" -o ../../corescope-decrypt-linux-arm64 .
|
||||
|
||||
- name: Verify release binaries are static and runnable
|
||||
run: |
|
||||
set -e
|
||||
for f in corescope-decrypt-linux-amd64 corescope-decrypt-linux-arm64; do
|
||||
file "$f" | grep -q 'statically linked' || { echo "::error::$f is not statically linked"; exit 1; }
|
||||
# musl sizes thread stacks from PT_GNU_STACK; the build's
|
||||
# -Wl,-z,stack-size flag must have reached the artifact.
|
||||
readelf -lW "$f" | grep -Eq 'GNU_STACK .* 0x0*800000 +RW' || { echo "::error::$f: PT_GNU_STACK is not 8 MiB, musl thread stacks are not pinned"; exit 1; }
|
||||
done
|
||||
# amd64 runs natively on the runner; arm64 is only checked structurally
|
||||
# here (the container image job exercises arm64 under QEMU).
|
||||
|
||||
Reference in new issue
Block a user