Files
meshcore-analyzer/tests/unit/test-dockerfile-buildplatform.js
T
0af70cf2c1 fix(build): pin musl thread stacks at 8 MiB and check the artifacts (#2410)
## Summary

Pins musl thread stacks at 8 MiB on every link, and fixes a crash in
**native Alpine builds** (`make build`, `make test`, or `go test` on an
Alpine host).

**The released images are not affected.** An earlier version of this
description said public endpoints could crash the shipped image. That
was wrong, and review caught it: see "Who is affected".

## Root cause

- musl sizes every non-main thread's stack from the ELF `PT_GNU_STACK`
size, with a floor of 128 KiB and a cap of 8 MiB (glibc defaults to 8
MiB).
- cgo calls run on the calling thread's stack, and SQLite's
`multiSelect` recurses about 850 bytes per `UNION ALL` term.
- `lookupPrefixes` (up to 400 terms) and `sharedEightBytePrefixes` (up
to 450 terms) overflow a 128 KiB stack and SIGSEGV the whole process.

## Who is affected

| Link | `PT_GNU_STACK` without the flag | Thread stack | Wide-UNION
tests |
|---|---|---|---|
| Alpine gcc + GNU ld (`make build` / `make test` / `go test` on Alpine)
| `0` | 128 KiB | **SIGSEGV** |
| zig 0.16.0, x86_64-linux-musl (Dockerfile, `crossbuild`, release job)
| `0x1000000` (16 MiB) | 8 MiB | pass 3/3 |
| zig 0.16.0, aarch64-linux-musl | `0x1000000` (16 MiB) | 8 MiB | pass
3/3, run on an arm64 node |

So for the released images the flag pins a value that zig's default
already provides. It stops being a property of a linker default, and a
build check now fails if it is lost. For native Alpine links it is the
fix: with the flag those tests pass.

Control: a zig arm64 link with `-Wl,-z,stack-size=131072` gets 128 KiB
threads and the same tests SIGSEGV in cgo on the arm64 node.

## Fix

- Every musl link passes `-Wl,-z,stack-size=8388608`: the `Dockerfile`,
the `Makefile` (`build` and `test` on Linux, and `crossbuild`) and the
release job in `deploy.yml`.
- This reserves address space only. Memory is committed as pages are
touched.
- **The artifacts are checked.** The Dockerfile's builder stage and the
release job's "Verify release binaries" step read `PT_GNU_STACK` with
`readelf` and fail unless it is exactly 8 MiB. Verified: the builder
stage passes for `linux/amd64` and `linux/arm64`, and fails with
`--build-arg STACK_LDFLAG=` (`/corescope-server: PT_GNU_STACK is not 8
MiB`, memsz `0x1000000`).

## Testing

- `TestMuslThreadStackWideUnionQueries`
(`cmd/server/musl_thread_stack_linux_test.go`, Linux only) runs both
queries off the main thread and asserts exact results, so a failed chunk
cannot pass as "nothing matched". Under `golang:1.27-alpine` with gcc:
SIGSEGV without the flag, pass with it. On CI's glibc runners it passes
either way, which is why the artifact checks above exist.
- `tests/unit/test-issue-1956-release-routing.js`: the release verify
step must check both binaries, and a `0x000000` or `0x1000000` header
must fail the release.
- `tests/unit/test-dockerfile-buildplatform.js`: all three `go build`
lines carry `STACK_LDFLAG`, and the `readelf` check covers all three
binaries.
- `sh test-all.sh` passes. `go vet` passes for `cmd/server` on Linux
with gofmt clean.
- Not verified: a Windows `go vet`. The fix there is the
`_linux_test.go` file name; no Windows machine was available to run it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

## Merge order

These four PRs were split out of #2396 and are **independent — they can
merge in any order.** Verified: every pair is conflict-free, and all
four merged together pass `go vet`, `go test ./...` (all 16 modules) and
`sh test-all.sh`. (That combined run predates the review fixes on #2409
and #2410.)

Suggested order, none of it urgent:

1. **#2408** — never lose rows during startup / a failed poll.
2. **#2407** — explicit `observers.idx` primary key.
3. **#2409** — content-hash migration moved into the ingestor.
4. **#2410** — pin musl thread stacks at 8 MiB. Released images are not
affected; it fixes native Alpine builds.

After these land, #2396 (Postgres) will be rebased onto master.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: efiten <erwin.fiten@gmail.com>
2026-11-18 18:35:41 +01:00

64 lines
3.0 KiB
JavaScript

/* test-dockerfile-buildplatform.js — the build stage runs on the building
* machine's own architecture.
*
* BuildKit sets BUILDPLATFORM to the host doing the build. Giving the global
* ARG a default (it used to say linux/amd64) overrides that, so a native build
* on an arm64 host — a Raspberry Pi, Apple silicon — pulled the amd64 Go image
* and failed at the first RUN with "exec format error". CI never saw it: its
* arm64 image is cross-built on an amd64 runner, where the default happened to
* be right.
*/
'use strict';
const fs = require('fs');
const path = require('path');
const assert = require('assert');
const dockerfile = fs.readFileSync(path.join(__dirname, '..', '..', 'Dockerfile'), 'utf8');
const lines = dockerfile.split('\n').map((l) => l.trim()).filter((l) => l && !l.startsWith('#'));
let passed = 0, failed = 0;
function test(name, fn) {
try { fn(); passed++; console.log(` ✅ ${name}`); }
catch (e) { failed++; console.log(` ❌ ${name}: ${e.message}`); }
}
console.log('\n=== Dockerfile: build platform ===');
test('BUILDPLATFORM is never given a default', () => {
const withDefault = lines.filter((l) => /^ARG\s+BUILDPLATFORM\s*=/.test(l));
assert.deepStrictEqual(withDefault, [], 'found: ' + withDefault.join(' | '));
});
test('the Go builder stage is pinned to BUILDPLATFORM (native, no QEMU)', () => {
const from = lines.find((l) => /^FROM\b.*golang:.*\bAS\s+builder\b/i.test(l));
assert.ok(from, 'builder FROM line not found');
assert.ok(/--platform=\$BUILDPLATFORM\b/.test(from), from);
});
test('no stage hard-codes a platform', () => {
const pinned = lines.filter((l) => /^FROM\b/.test(l) && /--platform=(?!\$)/.test(l));
assert.deepStrictEqual(pinned, [], 'found: ' + pinned.join(' | '));
});
console.log('\n=== Dockerfile: musl thread stacks ===');
// The Go regression test runs on glibc in CI and cannot fail there, so the
// image build is what guards the flag: it must be on every link line, and the
// build must read it back from every binary.
test('every go build passes STACK_LDFLAG and the result is checked with readelf', () => {
const joined = dockerfile.replace(/\\\n/g, ' ');
const builds = joined.split('\n').filter((l) => /\bgo build\b/.test(l));
assert.strictEqual(builds.length, 3, 'expected server, ingestor and decrypt builds');
for (const b of builds) assert.ok(b.includes('${STACK_LDFLAG}'), 'missing STACK_LDFLAG: ' + b.trim().slice(0, 120));
assert.ok(/^ARG STACK_LDFLAG="-Wl,-z,stack-size=8388608"$/m.test(dockerfile), 'STACK_LDFLAG must be 8 MiB');
const check = joined.split('\n').find((l) => /readelf -lW/.test(l) && /GNU_STACK/.test(l));
assert.ok(check, 'no readelf PT_GNU_STACK check');
for (const bin of ['/corescope-server', '/corescope-ingestor', '/corescope-decrypt']) {
assert.ok(check.includes(bin), bin + ' is not checked');
}
assert.ok(check.includes('0x0*800000'), 'the check must require exactly 8 MiB');
});
console.log(`\n${passed} passed, ${failed} failed`);
if (failed) process.exit(1);