mirror of
https://github.com/agessaman/MeshCore.git
synced 2026-08-29 09:48:52 +00:00
ci: add beta release channel workflow
Publishes a parallel observer firmware channel that cannot cross-contaminate production. Manual dispatch only, so the branch is chosen in the Actions UI rather than hardcoded here. Channel separation (each of these is load-bearing, not cosmetic): - OTA_MANIFEST_BASE_URL -> beta nodes only ever read beta manifests. This is the one that actually keeps devices on-channel. - Separate RELEASE_TAG: the publish step prunes all but KEEP_BUILDS hashes WITHIN its tag, so a shared tag would make each channel delete the other's assets. - Separate build counter: shared counters would interleave and make OTA's "N behind" comparison meaningless. - Separate staticPath via a derived config-beta.json. FIRMWARE_VERSION deliberately matches production: the OTA logic treats a different base version as "always an update", so channels must be separated by manifest URL, not base version. OTA_CHANNEL_TAG marks the embedded version instead (v1.16.0.N-observer-beta-<hash>) so `ver` identifies the channel. config-beta.json is derived per build rather than committed - a checked-in copy would be a 56-entry duplicate of config.json that goes stale as devices are added. Deriving keeps the beta device list identical by construction. Two verify steps fail the build rather than publish firmware that would OTA itself onto production: one checks the beta URL is baked into a binary (and the production URL is not), one checks the generated manifests use the beta host. Production's changelog and docs sync steps are omitted - those rewrite site-wide content the production channel owns. The flasher commit is scoped to the beta manifest dir and counter for the same reason. Also adds OTA_CHANNEL_TAG support to build.sh. Safe for OTA version parsing: ota_parseVersion() reads to the first '-' and ota_extractHash() takes the token after the last, so an extra tag between them changes neither. Verified on a real build: v1.16.0.7-observer-beta-36831271.
This commit is contained in:
agessaman
@@ -0,0 +1,272 @@
|
||||
name: Build MQTT Observer Firmwares (BETA channel)
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
# Manual dispatch only, deliberately. Unlike the production workflow this has no
|
||||
# push trigger: the beta channel exists to publish a specific branch on purpose,
|
||||
# and dispatching lets you pick that branch in the Actions UI without this file
|
||||
# hardcoding an (often short-lived) branch name.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
# Shared with build-observer-firmwares.yml and sync-flasher-content.yml so the
|
||||
# workflows never push to the flasher repo at the same time.
|
||||
concurrency:
|
||||
group: flasher-publish
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
# MUST stay equal to the production channel's FIRMWARE_VERSION. The observer's
|
||||
# OTA comparison treats a different base version as "always an update", so a
|
||||
# distinct base here would make every beta node think it is permanently behind.
|
||||
# Channels are separated by manifest URL, not by base version.
|
||||
FIRMWARE_VERSION: v1.16.0
|
||||
|
||||
# Beta-only rolling release. A separate tag is required, not cosmetic: the
|
||||
# publish step prunes all but the KEEP_BUILDS most recent build hashes within
|
||||
# its tag, so sharing production's tag would make each channel delete the
|
||||
# other's assets.
|
||||
RELEASE_TAG: observer-mqtt-beta-latest
|
||||
|
||||
# The channel itself. Firmware fetches <OTA_MANIFEST_BASE>/<OTA_VARIANT>.json,
|
||||
# so this URL is what keeps beta nodes on beta.
|
||||
OTA_MANIFEST_BASE_URL: https://observer.gessaman.com/beta/v
|
||||
# Marks the embedded version, e.g. v1.16.0.3-observer-beta-abc1234, so `ver`
|
||||
# identifies the channel. Does not affect OTA version parsing.
|
||||
OTA_CHANNEL_TAG: beta
|
||||
|
||||
# Beta's own build counter, so the two channels' build numbers never interleave.
|
||||
COUNTER_URL: https://observer.gessaman.com/observer-beta-build-counter.json
|
||||
COUNTER_FILE: observer-beta-build-counter.json
|
||||
|
||||
# Where beta artifacts live in the flasher repo. MANIFEST_DIR must correspond to
|
||||
# OTA_MANIFEST_BASE_URL's path, and STATIC_PATH must be a host/route serving the
|
||||
# beta GitHub release (see cloudflare-worker).
|
||||
MANIFEST_DIR: beta/v
|
||||
STATIC_PATH: https://observer-fw-beta.gessaman.com
|
||||
|
||||
jobs:
|
||||
|
||||
enumerate:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
matrix: ${{ steps.split.outputs.matrix }}
|
||||
build_number: ${{ steps.buildnum.outputs.n }}
|
||||
steps:
|
||||
- name: Clone Repo
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Split observer envs into shards
|
||||
id: split
|
||||
shell: bash
|
||||
run: |
|
||||
SHARDS=14
|
||||
ENVS=$(grep -rhoE '^\[env:[^]]*observer_mqtt\]' platformio.ini variants/*/platformio.ini \
|
||||
| sed -E 's/^\[env:(.*)\]$/\1/' | sort -u)
|
||||
echo "Discovered envs:"; echo "$ENVS"
|
||||
MATRIX=$(echo "$ENVS" | awk -v n="$SHARDS" '
|
||||
{ shard[NR % n] = shard[NR % n] " " $0 }
|
||||
END { for (i = 0; i < n; i++) { sub(/^ /, "", shard[i]);
|
||||
printf "{\"idx\":%d,\"envs\":\"%s\"}\n", i, shard[i] } }' \
|
||||
| jq -cs .)
|
||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Compute beta build number
|
||||
id: buildnum
|
||||
shell: bash
|
||||
run: |
|
||||
# Same scheme as production but off the BETA counter, so the channels
|
||||
# increment independently.
|
||||
CUR=$(curl -fsSL "$COUNTER_URL" 2>/dev/null || echo '{}')
|
||||
PREV_BASE=$(echo "$CUR" | jq -r '.baseVersion // ""')
|
||||
PREV_BUILD=$(echo "$CUR" | jq -r '.build // 0')
|
||||
if [ "$PREV_BASE" = "$FIRMWARE_VERSION" ]; then
|
||||
N=$((PREV_BUILD + 1))
|
||||
else
|
||||
N=1
|
||||
fi
|
||||
echo "Base $FIRMWARE_VERSION; previous beta build $PREV_BUILD (base $PREV_BASE) -> N=$N"
|
||||
echo "n=$N" >> "$GITHUB_OUTPUT"
|
||||
|
||||
build:
|
||||
needs: enumerate
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
shard: ${{ fromJSON(needs.enumerate.outputs.matrix) }}
|
||||
steps:
|
||||
- name: Clone Repo
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Cache PlatformIO Toolchains
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.platformio/packages
|
||||
~/.platformio/platforms
|
||||
key: pio-toolchains-${{ runner.os }}-${{ hashFiles('platformio.ini') }}
|
||||
restore-keys: |
|
||||
pio-toolchains-${{ runner.os }}-
|
||||
|
||||
- name: Setup Build Environment
|
||||
uses: ./.github/actions/setup-build-environment
|
||||
|
||||
- name: Build Shard ${{ matrix.shard.idx }}
|
||||
env:
|
||||
FIRMWARE_BUILD_NUMBER: ${{ needs.enumerate.outputs.build_number }}
|
||||
# These two are what make the output a beta build; build.sh reads both.
|
||||
OTA_MANIFEST_BASE_URL: ${{ env.OTA_MANIFEST_BASE_URL }}
|
||||
OTA_CHANNEL_TAG: ${{ env.OTA_CHANNEL_TAG }}
|
||||
run: /usr/bin/env bash build.sh build-firmware ${{ matrix.shard.envs }}
|
||||
|
||||
- name: Verify beta channel is baked in
|
||||
shell: bash
|
||||
run: |
|
||||
# Fail fast rather than publish firmware that would OTA itself onto the
|
||||
# production channel. Checks one built binary actually carries the beta
|
||||
# manifest URL and does NOT carry the production one.
|
||||
BIN=$(find .pio/build -name firmware.elf | head -1)
|
||||
if [ -z "$BIN" ]; then echo "no ELF found to verify" >&2; exit 1; fi
|
||||
if ! strings "$BIN" | grep -qF "$OTA_MANIFEST_BASE_URL"; then
|
||||
echo "ERROR: beta manifest base missing from $BIN" >&2; exit 1
|
||||
fi
|
||||
if strings "$BIN" | grep -qE 'https://observer\.gessaman\.com/v"?$'; then
|
||||
echo "ERROR: production manifest base present in a beta build" >&2; exit 1
|
||||
fi
|
||||
echo "OK: $BIN carries $OTA_MANIFEST_BASE_URL"
|
||||
|
||||
- name: Upload Shard Artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: fw-${{ matrix.shard.idx }}
|
||||
path: out
|
||||
if-no-files-found: error
|
||||
|
||||
release:
|
||||
needs: [enumerate, build]
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Clone Repo
|
||||
uses: actions/checkout@v4
|
||||
# Shallow on purpose — see the production workflow: `git rev-parse --short`
|
||||
# must produce the same abbreviation build.sh used for the asset filenames.
|
||||
|
||||
- name: Download All Shard Artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: artifacts
|
||||
|
||||
- name: Flatten into out/
|
||||
run: |
|
||||
mkdir -p out
|
||||
find artifacts -type f -name '*.bin' -exec cp -f {} out/ \;
|
||||
find artifacts -type f -name '*.partsig' -exec cp -f {} out/ \;
|
||||
echo "Collected binaries:"; ls -1 out
|
||||
|
||||
- name: Compute Short SHA
|
||||
id: sha
|
||||
run: echo "short=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Publish to Beta Rolling Release
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
if ! gh release view "$RELEASE_TAG" >/dev/null 2>&1; then
|
||||
gh release create "$RELEASE_TAG" --prerelease \
|
||||
--title "MQTT Observer Firmwares (BETA)" \
|
||||
--notes "Rolling BETA build. Separate channel from observer-mqtt-latest; beta nodes only OTA within this channel."
|
||||
fi
|
||||
|
||||
gh release upload "$RELEASE_TAG" $(find out -maxdepth 1 -type f ! -name '*.partsig') --clobber
|
||||
|
||||
KEEP_BUILDS=2
|
||||
keep_hashes=$(gh release view "$RELEASE_TAG" --json assets \
|
||||
-q '.assets[] | "\(.createdAt) \(.name)"' \
|
||||
| sort -r \
|
||||
| while read -r _ts name; do
|
||||
printf '%s' "$name" | grep -oiE '[0-9a-f]{7,40}(-merged)?\.bin$' | grep -oiE '^[0-9a-f]{7,40}'
|
||||
done \
|
||||
| awk '!seen[$0]++' | head -n "$KEEP_BUILDS")
|
||||
echo "Retaining build hashes:"; echo "$keep_hashes"
|
||||
gh release view "$RELEASE_TAG" --json assets -q '.assets[].name' \
|
||||
| while read -r asset; do
|
||||
ah=$(printf '%s' "$asset" | grep -oiE '[0-9a-f]{7,40}(-merged)?\.bin$' | grep -oiE '^[0-9a-f]{7,40}' || true)
|
||||
if [ -n "$ah" ] && grep -qxF "$ah" <<<"$keep_hashes"; then
|
||||
continue
|
||||
fi
|
||||
gh release delete-asset "$RELEASE_TAG" "$asset" --yes || true
|
||||
done
|
||||
|
||||
- name: Checkout Flasher Repo
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: agessaman/flasher.meshcore.io
|
||||
token: ${{ secrets.FLASHER_DISPATCH_TOKEN }}
|
||||
path: flasher
|
||||
|
||||
- name: Generate Beta Manifests
|
||||
env:
|
||||
BUILD_NUMBER: ${{ needs.enumerate.outputs.build_number }}
|
||||
run: |
|
||||
# config-beta.json is derived here and NOT committed: it would otherwise
|
||||
# be a 56-entry duplicate of config.json that silently goes stale as
|
||||
# devices are added. Deriving it per build keeps the beta device list
|
||||
# identical to production by construction. Only the slim manifests are
|
||||
# published.
|
||||
python3 - <<'PY'
|
||||
import json, os, pathlib
|
||||
src = pathlib.Path("flasher/config.json")
|
||||
cfg = json.loads(src.read_text())
|
||||
old = cfg["staticPath"]
|
||||
cfg["staticPath"] = os.environ["STATIC_PATH"]
|
||||
pathlib.Path("config-beta.json").write_text(json.dumps(cfg, indent=2))
|
||||
print(f"derived config-beta.json: staticPath {old} -> {cfg['staticPath']}")
|
||||
PY
|
||||
|
||||
python3 flasher/scripts/update-firmware.py \
|
||||
"${{ steps.sha.outputs.short }}" \
|
||||
--config config-beta.json
|
||||
|
||||
mkdir -p "flasher/$MANIFEST_DIR"
|
||||
python3 flasher/scripts/gen-slim-manifests.py \
|
||||
--config config-beta.json \
|
||||
--out-dir "flasher/$MANIFEST_DIR" \
|
||||
--base-version "$FIRMWARE_VERSION" \
|
||||
--build "$BUILD_NUMBER" \
|
||||
--partsig-dir out
|
||||
|
||||
printf '{\n "baseVersion": "%s",\n "build": %s\n}\n' \
|
||||
"$FIRMWARE_VERSION" "$BUILD_NUMBER" > "flasher/$COUNTER_FILE"
|
||||
echo "Beta build $FIRMWARE_VERSION.$BUILD_NUMBER"
|
||||
|
||||
- name: Verify beta manifests point at the beta channel
|
||||
run: |
|
||||
# Guards against a beta manifest handing out a production download URL.
|
||||
SAMPLE=$(find "flasher/$MANIFEST_DIR" -name '*.json' | head -1)
|
||||
echo "sample: $SAMPLE"; cat "$SAMPLE"
|
||||
if ! grep -qF "$STATIC_PATH" "$SAMPLE"; then
|
||||
echo "ERROR: beta manifest does not use $STATIC_PATH" >&2; exit 1
|
||||
fi
|
||||
|
||||
# NOTE: production's "Generate Changelog" and "Sync Docs into Flasher" steps
|
||||
# are deliberately omitted. Those rewrite site-wide content (CHANGELOG.md,
|
||||
# MQTT_IMPLEMENTATION.md, ...) that the production channel owns; a beta build
|
||||
# must not overwrite them.
|
||||
|
||||
- name: Commit & Push Beta Artifacts
|
||||
working-directory: flasher
|
||||
run: |
|
||||
# Scoped add: beta only ever touches its manifest dir and its counter, so
|
||||
# a stray edit elsewhere in the flasher checkout can never be published
|
||||
# by this workflow.
|
||||
git add -A "$MANIFEST_DIR" "$COUNTER_FILE"
|
||||
if git diff --cached --quiet; then
|
||||
echo "No beta changes to commit."
|
||||
exit 0
|
||||
fi
|
||||
git config user.name "meshcore-bot"
|
||||
git config user.email "noreply@gessaman.com"
|
||||
git commit -m "Update BETA observer firmware to ${{ steps.sha.outputs.short }} (build ${FIRMWARE_VERSION}.${{ needs.enumerate.outputs.build_number }})"
|
||||
git push
|
||||
Reference in New Issue
Block a user