* core: add prepaid name codes and length bounds * store: run the names migration after badges * store: issue and revoke prepaid name codes * store: document issuing prepaid name codes * store: move the ABI helpers to Names.Abi * store: read the [names] and [chain] settings * store: add the names chain client and reads * store: refuse group name codes while names are off * store: search names and preview links * store: serve each site on its own port * scripts: deploy the badges and names sites * store: document search and the two sites * store: add the drop_taken_codes migration * store: read the names codes_only setting * store: add the names store and order signatures * store: take name orders by card, crypto or code * store: document name orders and codes-only * store: add the names transaction sender * store: commit and register queued orders * store: reveal and retry an owner's order * store: document the gas key * store: add the name balance migrations * store: read crypto units, rates and confirmations * store: keep a taken order's money as a balance * store: spend a balance on another name * store: record refunds with //refund * store: document balances and refunds * store: add the agreed terms migration * store: read the terms and community names * store: answer community and terms in search * store: sign and store the agreed terms * store: document the terms and community names * store: add the invoiced units migration * store: read the crypto an invoice asks * store: credit late and overpaid crypto by share * store: document crediting late crypto by share * store: add the record changes migration * store: find an owner's orders and move one * store: relay owners' link edits on chain * store: document link edits * store: move isLoopbackHost to Config * store: trust dev issuer keys on the local chain * scripts: build the names site from a submodule * scripts: add the dev setup and contract run * store: document the dev setup and names image * store: send a Stripe idempotency key for names * tests: wait until a relay lets go of its edit * store: refuse a link edit deadline under 10 min * store: count reverted link edits in the year limit * store: require https for a remote chain node * store: refuse zero chain confirmations * store: set up the dev stack from the names branch * store: rerun a failed unpaid cleanup next pass * store: never replace a succeeded Stripe intent * store: refuse an ini setting written as key: value * store: note the webapp submodule's pinned branch * store: document confirmLanded judging by ownerOf * store: update the names webapp submodule * store-service: register names under the name policy, every name accepting its terms on chain (#7889) * store: accept each name's terms on chain under the name policy * ci: run the reduced build on sh/names-deploy * store: address review: policy re-read, terms reads and startup log * store: never credit an unconfirmed extra * store: alert money sent to a paid name invoice * store: refuse to start on another chain's node * store: keep orders while the allowance is used up * store: replace a stalled transaction after 3 blocks * store: judge a name taken at confirmation depth * store: bind the template's listener to loopback * store: scan only the ids of active name orders * store: refuse unknown [names] and [chain] settings * store: fetch the webapp submodule in upgrade steps * store: delete definitions only tests use * store: lock only name invoices at settlement * store: retry an unexpired commitment as usual * store: share the helpers defined twice * store: test route limits on the running service * store: fix tests that could not fail * ci: test the store service on Postgres * store: count IPv6 clients by their /64 * store: sweep invoices read alone past a list cap * store: cap checkouts of all clients together * store: cap the long polls held at once * store: stop the old badge project at upgrade * store: keep search lookups free of previews * store: read the ini as UTF-8 in any locale * ci: run Postgres store tests in a UTF-8 locale * store: cap the transactions one order sends * store: cancel a card intent after three declines * store: refuse a name price under a cent * store: send each transaction with gas headroom * store: cap searches and read terms once per block * store: never log the group join link * store: mask the database password at startup * store: run one instance per database * store: limit name codes to admins, 100 a day * store: refuse HighSpeed settlement with names * store: no unverified receipts with a trusted key * store: cap the log lines of refused webhooks * store: check Apple certificates at the signed date * store: refuse a malformed Stripe publishable key * store: check the owner before a refund * store: fold the names migrations into one * store: read onion name links through the proxy * store: refund money after paid once confirmed * store: limit the allowance wait to a dry run * store: bind loopback when upgrading * ci: cache and widen the store Postgres job * store: fix stale and duplicate names tests * store: fix the submodule path in the upgrade * store: refuse unpaid name codes on the CLI * store: apply the review's names test list * store: finish the review's names test list * store: restore the base registration tests * store: pass link hosts that do not resolve here * store: refuse a move once its sends are used up * store: take the shared checkout at the provider * store: hold a search slot for chain reads only * store: state which refunds are safe on a reference * store: check a failed order's name before refusing its move (#8060) * store: test and document the shared checkout cap for name orders * store: check a failed order's name on chain before refusing its move * store: test name links whose host lookup fails * store: test that a search holds its slot through the chain reads only * store: restore the signed transaction check in the chain tests * store: remove the chain tests' copy of the signed transaction check * store: correct the search slot texts and the README's balance endpoints * docs: system and sequence diagrams of the names service (#8068) --------- Co-authored-by: sh-agents <350651734+sh-agents@users.noreply.github.com> Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com> Co-authored-by: brenzi-agent <brenzi-agent@brenzi.ch> Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
12 KiB
CI on GitHub's runners
Three workflows build the apps from a branch's own code and test them, only when asked: ios-sim (the iOS app in a simulator on a macOS runner), desktop-linux (the desktop app on a Linux runner) and android-emu (the Android app in an emulator on a Linux runner). Each runs a scripted smoke test and, on request, a Claude session that follows a test plan, and a test backend (relay, directory, names server, peers). Details: iOS, desktop, Android.
Reduced build
GitHub runs the reduced build of build.yml on a dispatch (workflow_dispatch), on each push to a draft pull request and on a push to master or stable in a fork. Once the pull request is marked ready for review, GitHub runs the full matrix, as on a push to master or stable in simplex-chat/simplex-chat.
On a push to master or stable in a fork, the release steps and tests are skipped; build-linux saves the -O0 build caches, and on master the Docker image cache, that the fork's dispatches restore.
On a draft pull request, the tests are selected by a fenced block with the info string ci-tests in the description, one hspec match pattern per line; the block is read at each push. / runs the whole suite; without a block, the reduced build runs no tests.
Dispatch
curl -fsS -X POST \
-H "Authorization: Bearer $GH_TOKEN" \
-H "Accept: application/vnd.github+json" \
https://api.github.com/repos/<owner>/simplex-chat/actions/workflows/build.yml/dispatches \
-d '{"ref": "<branch>", "inputs": {"tests": "/SimpleX chat client/group tests/\n/Message batching/"}}'
Endpoint: https://docs.github.com/en/rest/actions/workflows#create-a-workflow-dispatch-event. Inputs:
| Input | Default | Value |
|---|---|---|
tests |
empty | hspec match patterns, one per line (\n in JSON); / runs the whole suite, and an empty value builds without tests |
base |
master |
the branch that the changes are compared with:stable for a branch cut from stablethe parent branch for a stacked branch |
When a newer dispatch on the same branch starts build-linux or build-ios, GitHub cancels the unfinished job of the older one (cancel-in-progress).
Results:
- Run and jobs:
GET /repos/<owner>/simplex-chat/actions/runs?head_sha=<commit>&event=workflow_dispatchreturns the run id;GET /repos/<owner>/simplex-chat/actions/runs/<run id>/jobsreturns each step with its start, end and conclusion. - Job log:
GET /repos/<owner>/simplex-chat/actions/jobs/<job id>/logsreturns a redirect to a text file, valid for 1 minute (https://docs.github.com/en/rest/actions/workflow-jobs#download-job-logs-for-a-workflow-run). The test summary is printed as "Finished in N seconds" and "N examples, N failures".
Scopes
The reduced build runs one Linux job, build-linux (ubuntu-22.04 x86_64), for every scope except ios, and one macOS job, build-ios (macos-latest), for ios.
core
- Globs:
BUILD_CHANGES["haskell"]in packages_release.py. - CI, in build-linux:
- Docker image (step "Build and cache Docker image");
- caches (the steps "Restore ... from cache");
cabal build -j --enable-tests --ghc-options=-O0(step "Build CLI");- simplex-chat-test with the selected tests (step "Run tests"), each attempt limited to 35 minutes by
timeout; an attempt that reaches the limit fails, and "Attempt timed out." is printed.
- Local:
The selection runs as in CI with scripts/ci/run-tests.sh, which appends the patterns of
cp scripts/cabal.project.local.linux cabal.project.local cabal build --enable-tests --ghc-options=-O0 mkdir -p tests/tmp cabal test simplex-chat-test --ghc-options=-O0 --test-option=-m --test-option='/SimpleX chat client/group tests/'TESTSto the test binary:TESTS='/SimpleX chat client/group tests/' scripts/ci/run-tests.sh "$(cabal list-bin simplex-chat-test --ghc-options=-O0)"--ghc-options=-O0matches the reduced build; the library and test-suite stanzas of simplex-chat.cabal set -O2. Test directories are created under tests/tmp (tmpBracketin tests/Test.hs).
kotlin
- Globs:
BUILD_CHANGES["kotlin"]in packages_release.py. - CI: Gradle in the Docker container of build-linux (step "Build and test Kotlin apps", matrix field
kotlin). - Local:
JDK 17 and an Android SDK in ANDROID_HOME are required, as installed by Dockerfile.build.
cd apps/multiplatform ./gradlew --no-daemon --max-workers=1 :common:desktopTest :desktop:compileKotlinJvm -x :desktop:cmakeBuildAndCopy :android:compileGoogleDebugKotlin :android:testFossDebugUnitTest
ios
- Globs:
- apps/ios/**
- scripts/ci/ios-stub-libs.sh
- CI: build-ios: stub libraries, then xcodebuild for the iOS Simulator without signing.
- Local, on macOS:
scripts/ci/ios-stub-libs.sh, then the xcodebuild command of the step "Build iOS app" of build-ios.
packages
- Globs:
- packages/simplex-chat-nodejs/**
- packages/simplex-chat-client/types/typescript/**
- packages/simplex-chat-python/**
- scripts/ci/package-tests.sh
- CI, in build-linux:
- Node 22 and Python 3.12 (steps "Setup Node.js" and "Setup Python");
- libsimplex, when
CORE_SOURCESof packages_release.py also changed (step "Build library"); package-tests.sh build linux-x86_64andpackage-tests.sh test(step "Package tests (pull request)").
- With libsimplex built, SIMPLEX_LIBS_DIR is set to it for the nodejs tests; the python tests run with it unset (package-tests.sh). Without SIMPLEX_LIBS_DIR, the pinned release libraries are downloaded (packages/simplex-chat-nodejs/src/libs.ts).
- Local:
scripts/ci/package-tests.sh build linux-x86_64 scripts/ci/package-tests.sh test
badge-web
- Globs: apps/simplex-store-service/web/. These paths are also in the core globs (apps/simplex-store-service/), and web/src/catalog.ts is read by core tests (tests/Bots/StoreService/CatalogTests.hs, WebTests.hs).
- CI, in build-linux: core build and tests, then
scope-tests.sh badge-web(step "Test changed scopes"). - Selected core tests:
/SimpleX store service/, which includes/SimpleX store service e2e/. - Local:
cd apps/simplex-store-service/web npm ci npm test
ci-scripts
- Globs:
- scripts/ci/packages_release.py
- scripts/ci/test_packages_release.py
- CI, in build-linux:
scope-tests.sh ci-scripts(step "Test changed scopes"). - Local:
python3 scripts/ci/test_packages_release.py; Python 3.11 or newer is required for tomllib (packages_release.py).
store service on Postgres
- Job: test-store-postgres (ubuntu-22.04 x86_64), on a pull request that changes the store service, its tests,
tests/Test.hs,src/Simplex/Chat/PaymentService,src/Simplex/Chat/Badges,cabal.project,simplex-chat.cabalor build.yml, and on every dispatch. Production runs the store service on Postgres only, and the SQLite jobs do not cover it. - CI: the build-linux Docker image, a
postgres:17container on the host network,cabal buildwith+client_postgres, then simplex-chat-test with--fail-on=empty -m "Supporter badges" -m "SimpleX store service" -m "StoreService names". - Local, with a Postgres 17 on 127.0.0.1 that trusts the
postgresuser and allows 1500 connections:cabal build --builddir dist-postgres --constraint 'simplexmq +client_postgres' --constraint 'simplex-chat +client_postgres' simplex-chat-test mkdir -p tests/tmp PGHOST=127.0.0.1 LANG=C.UTF-8 "$(cabal list-bin --builddir dist-postgres --constraint 'simplexmq +client_postgres' --constraint 'simplex-chat +client_postgres' simplex-chat-test)" -m "Supporter badges" -m "SimpleX store service" -m "StoreService names"
Scopes lacking CI tests
| Scope | Paths | CI |
|---|---|---|
| ios | apps/ios/** | compile only |
| types package | packages/simplex-chat-client/types/typescript/** | compile only |
| website | website/ images/ blog/ docs/ PRIVACY.md |
build and deploy on master pushes (web.yml) |
| ts-client | packages/simplex-chat-client/typescript/** | - |
| webrtc | packages/simplex-chat-webrtc/** | - |
| mcp | packages/simplex-chat-mcp/** | - |
| calculator-bot | apps/simplex-calculator-bot/** | - |
| support-bot | apps/simplex-support-bot/** | - |
| support-bot-light | apps/simplex-support-bot-light/** | - |
| prices | scripts/prices/** | - |
| nft | eth/nft/** | - |
CI coverage of such a scope is added in the pull request that changes it (Adding CI coverage).
Adding CI coverage
For a scope lacking CI tests, in the pull request that changes it:
- scripts/ci/packages_release.py: an entry
"<scope>": ["<glob>", ...]in SCOPE_CHANGES. - build.yml,
on.pull_request.paths: the same globs; the two are compared intest_build_yml_runs_for_the_paths_of_its_buildsof test_packages_release.py. - scripts/ci/scope-tests.sh: a
<scope>)case with the build and test commands, and the scope name in the usage line. - scripts/ci/test_packages_release.py, ChangesTest: a case asserting that the scope is printed in
scopesfor a change to one of its files. - This file, "Scopes": a subsection for the scope, in place of its row in "Scopes lacking CI tests".
- Checks:
python3 scripts/ci/test_packages_release.py, the new case of scope-tests.sh locally, then a dispatch of the reduced build.
The scope commands run in build-linux after the core tests, with Node 22 and Python 3.12 (steps "Setup Node.js" and "Setup Python"). For a scope that needs other versions, a setup step is added before "Test changed scopes", conditioned on the scope name in needs.maybe-release.outputs.scopes. A scope that needs macOS is a BUILD_CHANGES entry with a separate maybe-release output, as ios is, and a step in build-ios; the if of build-ios includes that output.
Starting a run
Pushes start none of the app workflows: a run is started by hand, on any branch, with inputs that are off by default and combine.
| Input | iOS | Desktop | Android |
|---|---|---|---|
| no input | scripted test, the app uploaded | scripted test, the app uploaded | scripted test, the app uploaded |
claude |
scripted test, then Claude | scripted test, then Claude | scripted test, then Claude |
backend |
also the test backend | also the test backend | also the test backend |
release |
— | also the latest release's onboarding, to tell a branch's failure from the runner's | also the latest release's onboarding, to tell a branch's failure from the runner's |
armv7a |
— | — | also the core library for 32-bit Arm (armeabi-v7a), built only |
In the Actions tab: ios-sim, desktop-linux or android-emu → Run workflow, choose the branch and tick the inputs. With the GitHub CLI, each input is a field:
gh workflow run ios-sim.yml --ref <branch>
gh workflow run ios-sim.yml --ref <branch> -f claude=true
gh workflow run desktop-linux.yml --ref <branch> -f claude=true -f backend=true -f release=true
gh workflow run android-emu.yml --ref <branch> -f claude=true
A newer run of a workflow on the same branch cancels the older one.
Each run uploads the app it built, for manual tests, as an artifact kept 14 days: ios-sim-app (SimpleX.app.tar.gz, for the iOS simulator on an Apple-silicon Mac), desktop-app (desktop-app.tar.gz, for Linux x86-64) and android-app (the debug APK, arm64-v8a only). The run's page lists it under Artifacts; gh run download <run id> -n <artifact> downloads it.
An agent that runs these workflows in a fork follows app-tests-agent.md.
How long a run takes
| Run | iOS | Desktop | Android |
|---|---|---|---|
| Scripted test, builds reused | about 25 min, plus the wait for a macOS runner, which the team's builds share (minutes to an hour) | about 10 min | about 15 min |
| The core rebuilt, after Haskell changes | 30–45 min more | about 25 min more | about 27 min more |
| With the test backend | 20–45 min more when it is rebuilt, then about 10 min to start | about 30 min more when it is rebuilt, then a few minutes to start | as desktop, whose backend it shares |
| With Claude on the smoke plan | 10–15 min more | about 4 min more | about 5 min more |
The Claude job
It needs the GitHub environment app-sim-claude with the secret CLAUDE_CODE_OAUTH_TOKEN, a Claude subscription token. The environment's branch rule alone decides which branches may use it: on a branch it doesn't allow, a run that asks for Claude fails at the Claude job's environment check, and the scripted test runs as usual. Each run spends the subscription's usage limits; the job summary gives its tokens and cost estimated at API prices.