mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-22 02:57:25 +00:00
## Summary Pins musl thread stacks at 8 MiB on every link, and fixes a crash in **native Alpine builds** (`make build`, `make test`, or `go test` on an Alpine host). **The released images are not affected.** An earlier version of this description said public endpoints could crash the shipped image. That was wrong, and review caught it: see "Who is affected". ## Root cause - musl sizes every non-main thread's stack from the ELF `PT_GNU_STACK` size, with a floor of 128 KiB and a cap of 8 MiB (glibc defaults to 8 MiB). - cgo calls run on the calling thread's stack, and SQLite's `multiSelect` recurses about 850 bytes per `UNION ALL` term. - `lookupPrefixes` (up to 400 terms) and `sharedEightBytePrefixes` (up to 450 terms) overflow a 128 KiB stack and SIGSEGV the whole process. ## Who is affected | Link | `PT_GNU_STACK` without the flag | Thread stack | Wide-UNION tests | |---|---|---|---| | Alpine gcc + GNU ld (`make build` / `make test` / `go test` on Alpine) | `0` | 128 KiB | **SIGSEGV** | | zig 0.16.0, x86_64-linux-musl (Dockerfile, `crossbuild`, release job) | `0x1000000` (16 MiB) | 8 MiB | pass 3/3 | | zig 0.16.0, aarch64-linux-musl | `0x1000000` (16 MiB) | 8 MiB | pass 3/3, run on an arm64 node | So for the released images the flag pins a value that zig's default already provides. It stops being a property of a linker default, and a build check now fails if it is lost. For native Alpine links it is the fix: with the flag those tests pass. Control: a zig arm64 link with `-Wl,-z,stack-size=131072` gets 128 KiB threads and the same tests SIGSEGV in cgo on the arm64 node. ## Fix - Every musl link passes `-Wl,-z,stack-size=8388608`: the `Dockerfile`, the `Makefile` (`build` and `test` on Linux, and `crossbuild`) and the release job in `deploy.yml`. - This reserves address space only. Memory is committed as pages are touched. - **The artifacts are checked.** The Dockerfile's builder stage and the release job's "Verify release binaries" step read `PT_GNU_STACK` with `readelf` and fail unless it is exactly 8 MiB. Verified: the builder stage passes for `linux/amd64` and `linux/arm64`, and fails with `--build-arg STACK_LDFLAG=` (`/corescope-server: PT_GNU_STACK is not 8 MiB`, memsz `0x1000000`). ## Testing - `TestMuslThreadStackWideUnionQueries` (`cmd/server/musl_thread_stack_linux_test.go`, Linux only) runs both queries off the main thread and asserts exact results, so a failed chunk cannot pass as "nothing matched". Under `golang:1.27-alpine` with gcc: SIGSEGV without the flag, pass with it. On CI's glibc runners it passes either way, which is why the artifact checks above exist. - `tests/unit/test-issue-1956-release-routing.js`: the release verify step must check both binaries, and a `0x000000` or `0x1000000` header must fail the release. - `tests/unit/test-dockerfile-buildplatform.js`: all three `go build` lines carry `STACK_LDFLAG`, and the `readelf` check covers all three binaries. - `sh test-all.sh` passes. `go vet` passes for `cmd/server` on Linux with gofmt clean. - Not verified: a Windows `go vet`. The fix there is the `_linux_test.go` file name; no Windows machine was available to run it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## Merge order These four PRs were split out of #2396 and are **independent — they can merge in any order.** Verified: every pair is conflict-free, and all four merged together pass `go vet`, `go test ./...` (all 16 modules) and `sh test-all.sh`. (That combined run predates the review fixes on #2409 and #2410.) Suggested order, none of it urgent: 1. **#2408** — never lose rows during startup / a failed poll. 2. **#2407** — explicit `observers.idx` primary key. 3. **#2409** — content-hash migration moved into the ingestor. 4. **#2410** — pin musl thread stacks at 8 MiB. Released images are not affected; it fixes native Alpine builds. After these land, #2396 (Postgres) will be rebased onto master. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: efiten <erwin.fiten@gmail.com>
207 lines
9.4 KiB
Docker
207 lines
9.4 KiB
Docker
# syntax=docker/dockerfile:1
|
|
# Build stage always runs natively on the builder's arch ($BUILDPLATFORM) and
|
|
# cross-compiles to $TARGETOS/$TARGETARCH. No QEMU for compilation.
|
|
#
|
|
# The SQLite driver is github.com/mattn/go-sqlite3, which is cgo, so the Go
|
|
# toolchain alone can no longer cross-compile this: it needs a C compiler that
|
|
# can target the other architecture. `zig cc` is that compiler. Targeting musl
|
|
# makes the result fully static (see -extldflags below), so the runtime stage
|
|
# has no libc dependency on the base image at all.
|
|
#
|
|
# BUILDPLATFORM is set automatically by BuildKit (the default builder since
|
|
# Docker 23, and what buildx uses) to the machine doing the build. Declare it
|
|
# without a default: a default here overrides BuildKit's value, so a native
|
|
# build on an arm64 host (a Raspberry Pi, Apple silicon) pulled the amd64 Go
|
|
# image and failed with "exec format error" at the first RUN.
|
|
ARG BUILDPLATFORM
|
|
FROM --platform=$BUILDPLATFORM golang:1.27-alpine AS builder
|
|
|
|
ARG APP_VERSION=unknown
|
|
ARG GIT_COMMIT=unknown
|
|
# musl sizes every non-main thread's stack from the ELF PT_GNU_STACK size
|
|
# (floor 128 KiB, cap 8 MiB). cgo calls into SQLite run on the calling thread's
|
|
# stack, and a deep query (multiSelect recursing ~850 B per UNION ALL term)
|
|
# overflows 128 KiB and SIGSEGVs the process. zig's linker already writes a
|
|
# 16 MiB PT_GNU_STACK, so these binaries got 8 MiB threads before this flag
|
|
# existed; GNU ld writes 0, which is where the crash was seen (a native Alpine
|
|
# gcc link). The flag pins the size so it no longer rests on a linker default,
|
|
# and the readelf check after the builds fails the image if it is ever lost.
|
|
# Reserved address space, not resident memory. Go rejects this flag in
|
|
# #cgo LDFLAGS, so it must go in -extldflags.
|
|
ARG STACK_LDFLAG="-Wl,-z,stack-size=8388608"
|
|
ARG BUILD_TIME=unknown
|
|
# Provided by buildx for multi-arch builds
|
|
ARG TARGETOS
|
|
ARG TARGETARCH
|
|
|
|
# Keep these in step with the Makefile: netgo/osusergo preserve the pure-Go
|
|
# resolver and user lookup the binaries had under CGO_ENABLED=0, and
|
|
# sqlite_omit_load_extension drops the dlopen path so -static links cleanly.
|
|
ENV GO_BUILD_TAGS=netgo,osusergo,sqlite_omit_load_extension \
|
|
ZIG_GLOBAL_CACHE_DIR=/tmp/zig-cache
|
|
|
|
# Pinned zig, checksum-verified. Arch comes from `uname -m` rather than
|
|
# TARGETARCH because this stage is pinned to BUILDPLATFORM — it has to work when
|
|
# someone builds on an arm64 machine too.
|
|
ARG ZIG_VERSION=0.16.0
|
|
# binutils: readelf, for the thread-stack check after the builds.
|
|
RUN apk add --no-cache curl xz binutils && \
|
|
case "$(uname -m)" in \
|
|
x86_64) ZA=x86_64; ZSHA=70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00 ;; \
|
|
aarch64) ZA=aarch64; ZSHA=ea4b09bfb22ec6f6c6ceac57ab63efb6b46e17ab08d21f69f3a48b38e1534f17 ;; \
|
|
*) echo "unsupported builder arch $(uname -m)" >&2; exit 1 ;; \
|
|
esac && \
|
|
curl -sSLo /tmp/zig.tar.xz "https://ziglang.org/download/${ZIG_VERSION}/zig-${ZA}-linux-${ZIG_VERSION}.tar.xz" && \
|
|
echo "${ZSHA} /tmp/zig.tar.xz" | sha256sum -c - && \
|
|
mkdir -p /opt/zig && tar -xJf /tmp/zig.tar.xz -C /opt/zig --strip-components=1 && \
|
|
ln -s /opt/zig/zig /usr/local/bin/zig && rm /tmp/zig.tar.xz && \
|
|
zig version
|
|
|
|
# zigcc resolves TARGETARCH to a zig target triple once, so the three build
|
|
# steps below stay readable and cannot disagree with each other.
|
|
RUN printf '%s\n' '#!/bin/sh' \
|
|
'case "$TARGETARCH" in' \
|
|
' amd64) t=x86_64-linux-musl ;;' \
|
|
' arm64) t=aarch64-linux-musl ;;' \
|
|
' *) echo "unsupported TARGETARCH=$TARGETARCH" >&2; exit 1 ;;' \
|
|
'esac' \
|
|
'exec zig cc -target "$t" "$@"' > /usr/local/bin/zigcc && chmod +x /usr/local/bin/zigcc
|
|
ENV CC=zigcc CGO_ENABLED=1
|
|
|
|
# Build server
|
|
WORKDIR /build/server
|
|
COPY cmd/server/go.mod cmd/server/go.sum ./
|
|
COPY internal/geofilter/ ../../internal/geofilter/
|
|
COPY internal/sigvalidate/ ../../internal/sigvalidate/
|
|
COPY internal/packetpath/ ../../internal/packetpath/
|
|
COPY internal/dbconfig/ ../../internal/dbconfig/
|
|
COPY internal/dbschema/ ../../internal/dbschema/
|
|
COPY internal/prunequeue/ ../../internal/prunequeue/
|
|
COPY internal/perfio/ ../../internal/perfio/
|
|
COPY internal/mbcapqueue/ ../../internal/mbcapqueue/
|
|
COPY internal/lora/ ../../internal/lora/
|
|
COPY internal/users/ ../../internal/users/
|
|
COPY internal/channel/ ../../internal/channel/
|
|
COPY internal/mailer/ ../../internal/mailer/
|
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
|
--mount=type=cache,target=/go/pkg/mod \
|
|
--mount=type=cache,target=/tmp/zig-cache \
|
|
go mod download
|
|
COPY cmd/server/ ./
|
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
|
--mount=type=cache,target=/go/pkg/mod \
|
|
--mount=type=cache,target=/tmp/zig-cache \
|
|
GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
|
|
go build -trimpath -tags ${GO_BUILD_TAGS} \
|
|
-ldflags "-s -w -extldflags '-static -Wl,-s ${STACK_LDFLAG}' -X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" \
|
|
-o /corescope-server .
|
|
|
|
# Build ingestor
|
|
WORKDIR /build/ingestor
|
|
COPY cmd/ingestor/go.mod cmd/ingestor/go.sum ./
|
|
COPY internal/channel/ ../../internal/channel/
|
|
COPY internal/geofilter/ ../../internal/geofilter/
|
|
COPY internal/sigvalidate/ ../../internal/sigvalidate/
|
|
COPY internal/packetpath/ ../../internal/packetpath/
|
|
COPY internal/dbconfig/ ../../internal/dbconfig/
|
|
COPY internal/dbschema/ ../../internal/dbschema/
|
|
COPY internal/prunequeue/ ../../internal/prunequeue/
|
|
COPY internal/perfio/ ../../internal/perfio/
|
|
COPY internal/mbcapqueue/ ../../internal/mbcapqueue/
|
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
|
--mount=type=cache,target=/go/pkg/mod \
|
|
--mount=type=cache,target=/tmp/zig-cache \
|
|
go mod download
|
|
COPY cmd/ingestor/ ./
|
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
|
--mount=type=cache,target=/go/pkg/mod \
|
|
--mount=type=cache,target=/tmp/zig-cache \
|
|
GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
|
|
go build -trimpath -tags ${GO_BUILD_TAGS} \
|
|
-ldflags "-s -w -extldflags '-static -Wl,-s ${STACK_LDFLAG}'" \
|
|
-o /corescope-ingestor .
|
|
|
|
# Build decrypt CLI
|
|
WORKDIR /build/decrypt
|
|
COPY cmd/decrypt/go.mod cmd/decrypt/go.sum ./
|
|
COPY internal/channel/ ../../internal/channel/
|
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
|
--mount=type=cache,target=/go/pkg/mod \
|
|
--mount=type=cache,target=/tmp/zig-cache \
|
|
go mod download
|
|
COPY cmd/decrypt/ ./
|
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
|
--mount=type=cache,target=/go/pkg/mod \
|
|
--mount=type=cache,target=/tmp/zig-cache \
|
|
GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
|
|
go build -trimpath -tags ${GO_BUILD_TAGS} \
|
|
-ldflags "-s -w -extldflags '-static -Wl,-s ${STACK_LDFLAG}' -X main.version=${APP_VERSION}" \
|
|
-o /corescope-decrypt .
|
|
|
|
# The thread-stack size must be in every binary's PT_GNU_STACK (see
|
|
# STACK_LDFLAG). The Go test for it runs on glibc in CI, where it passes with
|
|
# or without the flag, so the artifact is checked here.
|
|
RUN for f in /corescope-server /corescope-ingestor /corescope-decrypt; do \
|
|
readelf -lW "$f" | grep -Eq 'GNU_STACK .* 0x0*800000 +RW' || \
|
|
{ echo "$f: PT_GNU_STACK is not 8 MiB, musl thread stacks are not pinned" >&2; readelf -lW "$f" | grep GNU_STACK >&2; exit 1; }; \
|
|
done
|
|
|
|
# Runtime image
|
|
FROM alpine:3.24
|
|
|
|
RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy wget
|
|
|
|
# Account for the Go services when a deployment opts into running them
|
|
# unprivileged with RUN_AS_UID=1000, the usual choice. Creating it costs
|
|
# nothing when it goes unused, and saves the entrypoint creating one at start.
|
|
#
|
|
# Nothing in the image is given to it here. -H matters for that: without it
|
|
# busybox adduser creates the home directory itself, and /app would be
|
|
# corescope-owned mode 2755 — the setgid bit then giving gid 1000 to /app/data
|
|
# and to every file written into it, which Docker copies into each new named
|
|
# volume. The home field still points at /app; only the directory is left to
|
|
# the WORKDIR below, root-owned as in earlier releases.
|
|
RUN addgroup -S -g 1000 corescope && \
|
|
adduser -S -u 1000 -G corescope -H -h /app -s /sbin/nologin corescope
|
|
|
|
# Default: the services run as root, exactly as in earlier releases. The
|
|
# entrypoint overrides this when RUN_AS_UID asks for an unprivileged account.
|
|
ENV CORESCOPE_SERVICE_USER=root
|
|
|
|
WORKDIR /app
|
|
|
|
# Go binaries (statically linked; they do not use this image's libc)
|
|
COPY --from=builder /corescope-server /corescope-ingestor /corescope-decrypt /app/
|
|
|
|
# Frontend assets + config
|
|
COPY public/ ./public/
|
|
COPY config.example.json channel-rainbow.json ./
|
|
|
|
# Bake git commit SHA — manage.sh and CI write .git-commit before build
|
|
# Default to "unknown" if not provided
|
|
RUN echo "unknown" > .git-commit
|
|
|
|
# Supervisor + Mosquitto + Caddy config
|
|
COPY docker/supervisord-go.conf /etc/supervisor/conf.d/supervisord.conf
|
|
COPY docker/supervisord-go-no-mosquitto.conf /etc/supervisor/conf.d/supervisord-no-mosquitto.conf
|
|
COPY docker/supervisord-go-no-caddy.conf /etc/supervisor/conf.d/supervisord-no-caddy.conf
|
|
COPY docker/supervisord-go-no-mosquitto-no-caddy.conf /etc/supervisor/conf.d/supervisord-no-mosquitto-no-caddy.conf
|
|
COPY docker/mosquitto.conf /etc/mosquitto/mosquitto.conf
|
|
COPY docker/Caddyfile /etc/caddy/Caddyfile
|
|
|
|
# Data directory
|
|
RUN mkdir -p /app/data /var/lib/mosquitto /data/caddy && \
|
|
chown -R mosquitto:mosquitto /var/lib/mosquitto
|
|
|
|
# Entrypoint
|
|
COPY docker/entrypoint-go.sh /entrypoint.sh
|
|
# Sourced by the entrypoint, so deliberately not executable.
|
|
COPY docker/select-service-user.sh /select-service-user.sh
|
|
RUN chmod +x /entrypoint.sh
|
|
|
|
EXPOSE 80 443 1883
|
|
|
|
VOLUME ["/app/data", "/data/caddy"]
|
|
|
|
ENTRYPOINT ["/entrypoint.sh"]
|