ui do runs several helper commands in one call and stops at the first that fails, so an agent on the runner needs fewer turns for predictable sequences. The helper unsets CLAUDE_CODE_OAUTH_TOKEN first, so nothing it starts inherits the agent's credentials.
A phrase that is exactly the text now wins over one that only contains it (the alert's Search button under "Search in Directory?", the Directory header next to the Search in Directory row), and words are compared by their joined characters, so OCR splitting "#ci_group" still matches. New commands: ui find prints a text's centre and box; ui avatar measures the profile image left of a list row's name, saves that area, and with a colour checks that it dominates.
Run 38715611090 still stopped at the profile screen: averaging 4x4 blocks and taking a 5x5-block median leaves large bold text in the estimate (the title read as a dark background) and lets the white labels lift a grey disabled button over the threshold. A map coarse enough for titles would swallow 55-pixel buttons, so each word's background is now the median of the rows 2 to 4 pixels above and below its box, read from the 8-bit grayscale screenshot, which also feeds both OCR passes. Checked offline on the screenshots and TSVs of runs 38712854507 and 38714550750: the first screen, the profile screen with its title and grey button, and the branch chat list with the typed text and the directory row all read correctly.
In run 38714550750 the background map judged the white title area dark and the grey "Create profile" button light, so both were dropped, and onboarding stopped at the profile screen on both apps. ImageMagick's Q16 build writes PGM samples as 16 bits unless told -depth 8, while the reader takes one byte per pixel. The reader now also refuses a map without 8-bit samples.
Run 38712854507 got through onboarding on both apps, but on the branch app the typed "ci smoke" was lost: the dark-text pass read it as "ci smokel" with confidence 6, below the cutoff, while the light-text pass read noise ("OKC!") at the same place, and its "i)" split "Search in Directory". The light pass reads the gaps between dark letters as text, and the dark pass reads noise inside the coloured buttons, so neither confidence nor a pass order decides well. A median of the downscaled screenshot gives the background luminance; a word counts only from the dark pass on light backgrounds and from the light pass on darker ones, with no confidence cutoff. Phrases are now built from neighbouring words, so a stray word elsewhere on the same row no longer splits one ("Be" and "free" were).
Run 38712060819 showed that one image for both polarities smears page text: the morphological open treats the white gaps between dark letters as light text. Dark text now comes from the local threshold of run 38707461053, which read the grey and dark onboarding text; light text, such as labels on coloured buttons, from the difference from an open, which leaves no ring at the button edge. The words of both passes are merged by position, keeping the more confident reading.
In run 38711351160 the normalised image had no rings and showed the button labels, grey and dark text all dark on white, but the fixed 80% threshold turned every pixel with over 20% contrast black, so letters merged into blobs ("Be free" read as "Be tires"). With the background now uniformly white, tesseract's own threshold separates the text, so the image goes to it in grayscale.
Run 38707461053 read the grey onboarding text after the local threshold, but not the white labels on the blue buttons: in the negative image the threshold drew a thick ring around each button, and tesseract output nothing inside it. The text image is now the difference of the screenshot from its morphological close and open, so dark and light text both become dark on white and step edges between regions cancel; one tesseract pass instead of two. Each shot keeps that image and tesseract's TSV in UI_OUT/ocr.
In run 38702315040 the app started under Xvfb and showed onboarding, but tesseract's single global threshold, pulled down by the black screen outside the window and the illustration, lost the grey text and the white labels on the blue buttons, so "Create your profile" was never found. A local adaptive threshold (-lat) on the enlarged grayscale image and on its negative turns both into dark text on white. Each shot now also keeps the two images OCR read, in UI_OUT/ocr.
First iteration on f/ci-desktop: the core job builds libsimplex.so with scripts/desktop/build-lib-linux.sh on ubuntu-24.04 (GHC 9.6.3, direct-sqlcipher +openssl, no VLC) and caches it by source hash, with the cabal store cached separately. The app job builds :desktop:createDistributable with Corretto 17 and the repo's assets, runs :common:desktopTest, then starts the app under Xvfb and openbox with software rendering and drives onboarding to the chat list and typing in the search field with the ui helper (xdotool input, tesseract OCR to find text), saving numbered screenshots, the text read from them and a step log as artifacts.
spacedandy-agents-simplex