mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-22 17:17:56 +00:00
## Summary Pins musl thread stacks at 8 MiB on every link, and fixes a crash in **native Alpine builds** (`make build`, `make test`, or `go test` on an Alpine host). **The released images are not affected.** An earlier version of this description said public endpoints could crash the shipped image. That was wrong, and review caught it: see "Who is affected". ## Root cause - musl sizes every non-main thread's stack from the ELF `PT_GNU_STACK` size, with a floor of 128 KiB and a cap of 8 MiB (glibc defaults to 8 MiB). - cgo calls run on the calling thread's stack, and SQLite's `multiSelect` recurses about 850 bytes per `UNION ALL` term. - `lookupPrefixes` (up to 400 terms) and `sharedEightBytePrefixes` (up to 450 terms) overflow a 128 KiB stack and SIGSEGV the whole process. ## Who is affected | Link | `PT_GNU_STACK` without the flag | Thread stack | Wide-UNION tests | |---|---|---|---| | Alpine gcc + GNU ld (`make build` / `make test` / `go test` on Alpine) | `0` | 128 KiB | **SIGSEGV** | | zig 0.16.0, x86_64-linux-musl (Dockerfile, `crossbuild`, release job) | `0x1000000` (16 MiB) | 8 MiB | pass 3/3 | | zig 0.16.0, aarch64-linux-musl | `0x1000000` (16 MiB) | 8 MiB | pass 3/3, run on an arm64 node | So for the released images the flag pins a value that zig's default already provides. It stops being a property of a linker default, and a build check now fails if it is lost. For native Alpine links it is the fix: with the flag those tests pass. Control: a zig arm64 link with `-Wl,-z,stack-size=131072` gets 128 KiB threads and the same tests SIGSEGV in cgo on the arm64 node. ## Fix - Every musl link passes `-Wl,-z,stack-size=8388608`: the `Dockerfile`, the `Makefile` (`build` and `test` on Linux, and `crossbuild`) and the release job in `deploy.yml`. - This reserves address space only. Memory is committed as pages are touched. - **The artifacts are checked.** The Dockerfile's builder stage and the release job's "Verify release binaries" step read `PT_GNU_STACK` with `readelf` and fail unless it is exactly 8 MiB. Verified: the builder stage passes for `linux/amd64` and `linux/arm64`, and fails with `--build-arg STACK_LDFLAG=` (`/corescope-server: PT_GNU_STACK is not 8 MiB`, memsz `0x1000000`). ## Testing - `TestMuslThreadStackWideUnionQueries` (`cmd/server/musl_thread_stack_linux_test.go`, Linux only) runs both queries off the main thread and asserts exact results, so a failed chunk cannot pass as "nothing matched". Under `golang:1.27-alpine` with gcc: SIGSEGV without the flag, pass with it. On CI's glibc runners it passes either way, which is why the artifact checks above exist. - `tests/unit/test-issue-1956-release-routing.js`: the release verify step must check both binaries, and a `0x000000` or `0x1000000` header must fail the release. - `tests/unit/test-dockerfile-buildplatform.js`: all three `go build` lines carry `STACK_LDFLAG`, and the `readelf` check covers all three binaries. - `sh test-all.sh` passes. `go vet` passes for `cmd/server` on Linux with gofmt clean. - Not verified: a Windows `go vet`. The fix there is the `_linux_test.go` file name; no Windows machine was available to run it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## Merge order These four PRs were split out of #2396 and are **independent — they can merge in any order.** Verified: every pair is conflict-free, and all four merged together pass `go vet`, `go test ./...` (all 16 modules) and `sh test-all.sh`. (That combined run predates the review fixes on #2409 and #2410.) Suggested order, none of it urgent: 1. **#2408** — never lose rows during startup / a failed poll. 2. **#2407** — explicit `observers.idx` primary key. 3. **#2409** — content-hash migration moved into the ingestor. 4. **#2410** — pin musl thread stacks at 8 MiB. Released images are not affected; it fixes native Alpine builds. After these land, #2396 (Postgres) will be rebased onto master. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: efiten <erwin.fiten@gmail.com>
64 lines
3.0 KiB
JavaScript
64 lines
3.0 KiB
JavaScript
/* test-dockerfile-buildplatform.js — the build stage runs on the building
|
|
* machine's own architecture.
|
|
*
|
|
* BuildKit sets BUILDPLATFORM to the host doing the build. Giving the global
|
|
* ARG a default (it used to say linux/amd64) overrides that, so a native build
|
|
* on an arm64 host — a Raspberry Pi, Apple silicon — pulled the amd64 Go image
|
|
* and failed at the first RUN with "exec format error". CI never saw it: its
|
|
* arm64 image is cross-built on an amd64 runner, where the default happened to
|
|
* be right.
|
|
*/
|
|
'use strict';
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const assert = require('assert');
|
|
|
|
const dockerfile = fs.readFileSync(path.join(__dirname, '..', '..', 'Dockerfile'), 'utf8');
|
|
const lines = dockerfile.split('\n').map((l) => l.trim()).filter((l) => l && !l.startsWith('#'));
|
|
|
|
let passed = 0, failed = 0;
|
|
function test(name, fn) {
|
|
try { fn(); passed++; console.log(` ✅ ${name}`); }
|
|
catch (e) { failed++; console.log(` ❌ ${name}: ${e.message}`); }
|
|
}
|
|
|
|
console.log('\n=== Dockerfile: build platform ===');
|
|
|
|
test('BUILDPLATFORM is never given a default', () => {
|
|
const withDefault = lines.filter((l) => /^ARG\s+BUILDPLATFORM\s*=/.test(l));
|
|
assert.deepStrictEqual(withDefault, [], 'found: ' + withDefault.join(' | '));
|
|
});
|
|
|
|
test('the Go builder stage is pinned to BUILDPLATFORM (native, no QEMU)', () => {
|
|
const from = lines.find((l) => /^FROM\b.*golang:.*\bAS\s+builder\b/i.test(l));
|
|
assert.ok(from, 'builder FROM line not found');
|
|
assert.ok(/--platform=\$BUILDPLATFORM\b/.test(from), from);
|
|
});
|
|
|
|
test('no stage hard-codes a platform', () => {
|
|
const pinned = lines.filter((l) => /^FROM\b/.test(l) && /--platform=(?!\$)/.test(l));
|
|
assert.deepStrictEqual(pinned, [], 'found: ' + pinned.join(' | '));
|
|
});
|
|
|
|
console.log('\n=== Dockerfile: musl thread stacks ===');
|
|
|
|
// The Go regression test runs on glibc in CI and cannot fail there, so the
|
|
// image build is what guards the flag: it must be on every link line, and the
|
|
// build must read it back from every binary.
|
|
test('every go build passes STACK_LDFLAG and the result is checked with readelf', () => {
|
|
const joined = dockerfile.replace(/\\\n/g, ' ');
|
|
const builds = joined.split('\n').filter((l) => /\bgo build\b/.test(l));
|
|
assert.strictEqual(builds.length, 3, 'expected server, ingestor and decrypt builds');
|
|
for (const b of builds) assert.ok(b.includes('${STACK_LDFLAG}'), 'missing STACK_LDFLAG: ' + b.trim().slice(0, 120));
|
|
assert.ok(/^ARG STACK_LDFLAG="-Wl,-z,stack-size=8388608"$/m.test(dockerfile), 'STACK_LDFLAG must be 8 MiB');
|
|
const check = joined.split('\n').find((l) => /readelf -lW/.test(l) && /GNU_STACK/.test(l));
|
|
assert.ok(check, 'no readelf PT_GNU_STACK check');
|
|
for (const bin of ['/corescope-server', '/corescope-ingestor', '/corescope-decrypt']) {
|
|
assert.ok(check.includes(bin), bin + ' is not checked');
|
|
}
|
|
assert.ok(check.includes('0x0*800000'), 'the check must require exactly 8 MiB');
|
|
});
|
|
|
|
console.log(`\n${passed} passed, ${failed} failed`);
|
|
if (failed) process.exit(1);
|