mirror of
https://github.com/simplex-chat/simplex-chat.git
synced 2026-11-04 17:17:39 +00:00
core, ui: resize images in the core for sharper previews and images (#7941)
* WIP: Haskell impl of resizeImageToStrSize * WIP: add FFI export wrapper * WIP: switch ios resizeImageToStrSizeSync to backend * WIP: fix resizeImageToStrSizeSync * WIP: add JNI wrappers * Add non-base64 resize and cWriteImage * WIP: add chat_write_image * fix resizeToStr * fixes * fix syntax, raise jpeg quality back * add chatWriteImage * Fix cfArgs in tests * Use jpeg-turbo encoder * add nix pins * bump jpeg-turbo * add CI deps * add encoding funs to android lib * bump jpeg-turbo for 20.04 fix * jpeg-turbo fix for 8.10 * android fixes * Bump jpeg-turbo * set static flag for jpeg-turbo * Fix nix for aarch64/android * Fix android C api * Fix resize in android * Bump hs-libjpeg-turbo * target correct override * override android lib from android pkgs * add macos build tweaks * bump hs-jpeg-turbo for 8.10 fixes * more build fixes * make ios compile * Always use pkgconfig for libjpeg-turbo * WIP: debug libjpeg in ubuntu pkg-config CI * Add pkg-config to dockerfiles * Add libturbojpeg to dockerfiles * Use -dev version of libturbojpeg in dockerfiles * Install pkg-config in windows CI * Disable flag for custom libturbojpeg linking options Workarounds relocation error on x86_64-linux with gcc: /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libturbojpeg.a(turbojpeg.c.o): relocation R_X86_64_TPOFF32 against `errStr' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: failed to set dynamic section sizes: bad value * WIP: debug libjpeg in windows pkg-config CI * WIP: debug libjpeg package in windows CI * core: image resize module and buffer FFI * ci: link libjpeg-turbo statically on all platforms * ui: resize images in the core * core: rename image resize functions to chat_resize_image and chat_resize_image_uri * ci: restore whitespace in build.yml * ci: build libjpeg-turbo 3.1.4.1 from a submodule * core: hs-jpeg-turbo from github.com/simplex-chat * nodejs: resize images * python: resize images * core: hs-jpeg-turbo from gitlab.com, the GitHub import lacks LFS objects * core: generate /img previews * ci: link libjpeg-turbo on Windows without pkg-config * ci: build libjpeg-turbo on Windows with the GHC toolchain * docs: plan for image resizing in the core * docs: update image resizing plan * ios: prevent crash when wallpaper image cannot be resized * android, desktop: raise an error when an image cannot be resized * docs: update spec references for image resizing * docker: build the CLI image from a clone without submodules * docs: apply review fixes to image resizing plan * docker: build the CLI from a git worktree with the submodule present * docs: apply second review to image resizing plan * docs: point image resizing plan at the rewritten commit * core: choose image format by transparency in chat_resize_image * nodejs: resize images to the app sizes * python: resize images to the app sizes * nodejs, python: describe the core modules as used for image resizing * docs: add license notices of the image libraries * ci: describe the avatar method without attributing it to the apps * ci, ios, nix: remove the libjpeg-turbo build outside cabal * python: name resize_image in the core module text * nodejs, python: describe resizing to the app image sizes * core: refuse images larger than the supported size * core: resize to JPEG images with a side above 65500 pixels * docs: record image resizing decisions for the bundled libjpeg-turbo * core, docs: build libjpeg-turbo and stb from the fork submodules * docs: record the fork pins in the image resizing plan * core: resize images to at most 4320 pixels per side, refuse above 16384 * docs: record the image side limits in the image resizing plan * core: pin the rebased hs-jpeg-turbo fork * core: pin the hs-jpeg-turbo fork without package.yaml * docs: plan scaled JPEG decoding and EXIF orientation of images * docs: copy the JuicyPixels-stbir licence of the pinned fork * core: pin the hs-jpeg-turbo fork with scaled JPEG decoding * core: decode JPEG at the display size, up to 65500 pixels per side * core: rotate JPEG images by their EXIF orientation * docs: refer the image plan to the JPEG decoding code * docs: record the approved image processing decisions * core: limit JPEG decoding memory to 1024 MB outside iOS and Android * docs: name JPEG decoding in the libjpeg-turbo notice * docs: plan image resizing from files and the decisions of 26 Oct * core: scale images in one Catmull-Rom pass with sharpening * docs: plan image sizes at a fixed quality * core: resize images at a fixed quality to fill the size limit * core: resize images from a file path or a file descriptor * ios, android, desktop: resize picked images from their files * core: encode resized images with mozjpeg * tests: check descriptor closing by the file it refers to * core: raise JPEG quality only for a full-size image below 97% of the limit * ci: name the mozjpeg submodule in the Windows cleanup comments * docs: refer the image plan to the current code * docs: refer the image plan to the code merged with master * core: export every image resizing function from each library * core: close the file descriptor when its handle cannot be created * docs: count the C sources of mozjpeg with the arithmetic decoder * core: use the reviewed hs-jpeg-turbo and JuicyPixels-stbir forks * ci: build the armv7a Android core library on request * core: round image heights half up in the size search * ios: describe the file image resizing functions in SimpleX.h * ci: pick, preview and send images in the Claude app tests * core: decode PNG images with libspng and refuse other formats * core: encode PNG with libspng, keeping the smaller filter choice * ci: pick, preview and send images in the scripted app smoke tests * flake: build the PNG decoder package in nix builds * ios: remove the copy of a picked image after sending it * core: refuse image files larger than the decode memory limit * core: scale a JPEG image to its largest size once per resize * core: bisect JPEG quality without a step function * core: reduce the memory used to sharpen and to decode PNG images * android, desktop: pass the image format to the core as an integer * android, desktop: fuse the preview conditions of picked media * android, desktop, ci: check that the core resized the smoke test images * python: remove the docstrings of the explicit image resizing functions * docs: add the second review fixes to the image plan * ci: tap the iOS attachment button by its accessibility label * core: keep the colour profiles of resized images * docs: add the colour profiles to the image plan * ci: wait up to 30 s for the new group screens in the iOS smoke test * ci: pick the Android smoke test images from their own photo folders * ci: tap the iOS photo picker by position, as its tree is not shown * ci: pick the test card and the logo by their iOS picker positions * core: update JuicyPixels-stbir with the stb_image_resize licence terms * core: write a fixed Display P3 profile for Display P3 images only * core: do not keep the input image in memory for its colour profile * core: read the JPEG segments once for the orientation and the profile * core: read an image file into one buffer of its size * docs: add the third review fixes to the image plan * core: resize images with Display P3 or no profile, not profile bytes * ci: check the pixel sizes of the images sent in the iOS smoke test * core: update hs-jpeg-turbo with PNG chunk limits and encoder fixes * docs: list Compact ICC Profiles, libspng and miniz in the dependencies * android, desktop: reword the picked media preview comment * docs: refer the image plan to the code merged with master * ci: render the iOS snapshots before the images case * ci: dismiss the snapshot tests' camera alert before the iOS images case * core: update hs-jpeg-turbo with tests and the JuicyPixels-spng synopsis * core: pin hs-jpeg-turbo and JuicyPixels-stbir in simplex-chat repositories * update ios core library * core, ui: accept only JPEG and PNG image data * core: update JuicyPixels-stbir * docs: plan for the iOS copy of picked images --------- Co-authored-by: IC Rainbow <aenor.realm@gmail.com> Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com> Co-authored-by: Alexandre Esteves <alexfmpe@proton.me>
This commit is contained in:
118 files changed
+4227
-348
No files matched your search
@@ -15,6 +15,10 @@ on:
|
||||
description: Also test the latest released app, to tell a branch's failure from the runner's
|
||||
type: boolean
|
||||
default: false
|
||||
armv7a:
|
||||
description: Also build the core library for 32-bit Arm (armeabi-v7a)
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -41,6 +45,7 @@ jobs:
|
||||
run-claude: ${{ steps.jobs.outputs.claude }}
|
||||
run-backend: ${{ steps.jobs.outputs.backend }}
|
||||
run-release: ${{ steps.jobs.outputs.release }}
|
||||
run-armv7a: ${{ steps.jobs.outputs.armv7a }}
|
||||
claude-backend: ${{ steps.scope.outputs.backend }}
|
||||
core: ${{ steps.names.outputs.core }}
|
||||
core-run: ${{ steps.core.outputs.run-id }}
|
||||
@@ -53,10 +58,12 @@ jobs:
|
||||
CLAUDE: ${{ inputs.claude }}
|
||||
BACKEND: ${{ inputs.backend }}
|
||||
RELEASE: ${{ inputs.release }}
|
||||
ARMV7A: ${{ inputs.armv7a }}
|
||||
run: |
|
||||
echo "claude=$CLAUDE" | tee -a "$GITHUB_OUTPUT"
|
||||
echo "backend=$BACKEND" | tee -a "$GITHUB_OUTPUT"
|
||||
echo "release=$RELEASE" | tee -a "$GITHUB_OUTPUT"
|
||||
echo "armv7a=$ARMV7A" | tee -a "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
@@ -173,6 +180,29 @@ jobs:
|
||||
retention-days: 30
|
||||
if-no-files-found: error
|
||||
|
||||
core-armv7a:
|
||||
name: core library for Android armv7a
|
||||
needs: plan
|
||||
if: needs.plan.outputs.run-armv7a == 'true'
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 150
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
filter: tree:0
|
||||
|
||||
- name: Show runner
|
||||
run: |
|
||||
lsb_release -ds
|
||||
nproc
|
||||
free -h
|
||||
df -h /
|
||||
|
||||
- name: Build core library
|
||||
run: scripts/ci/android/build-core.sh ~/core/android-core-armeabi-v7a.tar.gz armv7a
|
||||
|
||||
app:
|
||||
name: Android app in the emulator
|
||||
needs: [plan, core, backend]
|
||||
@@ -364,6 +394,14 @@ jobs:
|
||||
- name: Boot emulator
|
||||
run: scripts/ci/android/emulator.sh start "$RUNNER_TEMP/claude-test/screens"
|
||||
|
||||
- name: Add test images
|
||||
run: |
|
||||
adb push tests/fixtures/test.jpg /sdcard/Pictures/test-card.jpg
|
||||
adb push tests/fixtures/logo-large-rgba.png /sdcard/Pictures/simplex-logo.png
|
||||
for image in test-card.jpg simplex-logo.png; do
|
||||
adb shell am broadcast -a android.intent.action.MEDIA_SCANNER_SCAN_FILE -d "file:///sdcard/Pictures/$image"
|
||||
done
|
||||
|
||||
- name: Reach the test backend
|
||||
if: steps.backend.outcome == 'success'
|
||||
run: scripts/ci/android/emulator.sh reverse 5223
|
||||
|
||||
@@ -1499,8 +1499,8 @@ jobs:
|
||||
if: steps.restore-build.outcome == 'success' && steps.restore-build.outputs.cache-matched-key == ''
|
||||
run: echo "::warning title=Cache miss::${{ steps.keys.outputs.dist }}-${{ env.opt }}-${{ env.branch }}"
|
||||
|
||||
# rm -rf dist-newstyle/src/{direct-sq,simplexmq}* is here because of the bug in cabal's dependency which prevents second build from finishing
|
||||
# (simplexmq is removed because cabal cannot delete its read-only git submodule pack files - blst, libbbs - on Windows)
|
||||
# rm -rf dist-newstyle/src/{direct-sq,simplexmq,hs-jpeg-t,JuicyPixe}* is here because of the bug in cabal's dependency which prevents second build from finishing
|
||||
# (simplexmq, hs-jpeg-turbo and JuicyPixels-stbir are removed because cabal cannot delete their read-only git submodule pack files - blst, libbbs, mozjpeg, stb - on Windows)
|
||||
- name: Build CLI
|
||||
id: windows_cli_build
|
||||
shell: msys2 {0}
|
||||
@@ -1516,10 +1516,10 @@ jobs:
|
||||
echo " extra-lib-dirs: $openssl_windows_style_path" >> cabal.project.local
|
||||
|
||||
scripts/ci/prune-old-builds.sh dist-newstyle
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq*
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq* dist-newstyle/src/hs-jpeg-t* dist-newstyle/src/JuicyPixe*
|
||||
sed -i "s/, unix /--, unix /" simplex-chat.cabal
|
||||
cabal build -j --enable-tests ${{ env.ghc_options }}
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq*
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq* dist-newstyle/src/hs-jpeg-t* dist-newstyle/src/JuicyPixe*
|
||||
path=$(cabal list-bin simplex-chat | tail -n 1)
|
||||
echo "bin_path=$path" >> $GITHUB_OUTPUT
|
||||
echo "bin_hash=$(echo SHA2-256\(${{ matrix.cli_asset_name }}\)= $(openssl sha256 $path | cut -d' ' -f 2))" >> $GITHUB_OUTPUT
|
||||
@@ -1642,7 +1642,7 @@ jobs:
|
||||
scripts/desktop/build-lib-windows.sh
|
||||
cd apps/multiplatform
|
||||
./gradlew -Psimplex.assets.dir=../../assets packageMsi
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq*
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq* dist-newstyle/src/hs-jpeg-t* dist-newstyle/src/JuicyPixe*
|
||||
path=$(echo $PWD/release/main/msi/*imple*.msi | sed 's#/\([a-z]\)#\1:#' | sed 's#/#\\#g')
|
||||
echo "package_path=$path" >> $GITHUB_OUTPUT
|
||||
echo "package_hash=$(echo SHA2-256\(${{ matrix.desktop_asset_name }}\)= $(openssl sha256 $path | cut -d' ' -f 2))" >> $GITHUB_OUTPUT
|
||||
@@ -1684,9 +1684,9 @@ jobs:
|
||||
export PATH=$PATH:/c/ghcup/bin:$(echo /c/tools/ghc-*/bin || echo)
|
||||
scripts/desktop/prepare-openssl-windows.sh
|
||||
scripts/ci/prune-old-builds.sh dist-newstyle
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq*
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq* dist-newstyle/src/hs-jpeg-t* dist-newstyle/src/JuicyPixe*
|
||||
cabal build -j --enable-tests --ghc-options=-O0
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq*
|
||||
rm -rf dist-newstyle/src/direct-sq* dist-newstyle/src/simplexmq* dist-newstyle/src/hs-jpeg-t* dist-newstyle/src/JuicyPixe*
|
||||
|
||||
- name: Save -O0 build directory to cache
|
||||
if: ${{ !cancelled() && steps.build-o0.outcome == 'success' && steps.restore-build-o0.outputs.cache-hit != 'true' }}
|
||||
|
||||
@@ -407,6 +407,12 @@ jobs:
|
||||
- name: Start the app
|
||||
run: scripts/ci/desktop-linux/smoke-test.sh "$RUNNER_TEMP/app/simplex" "$RUNNER_TEMP/claude-test/screens" start
|
||||
|
||||
- name: Add test images
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/claude-test/images"
|
||||
cp tests/fixtures/test.jpg "$RUNNER_TEMP/claude-test/images/test-card.jpg"
|
||||
cp tests/fixtures/logo-large-rgba.png "$RUNNER_TEMP/claude-test/images/simplex-logo.png"
|
||||
|
||||
# Claude's Bash is limited to single ui commands by the guard hook in claude-test.sh. Claude Code's credential
|
||||
# scrub stays off: on Linux it forces its whole Bash sandbox (bubblewrap, socat), which ui isn't set up for.
|
||||
- name: Run Claude
|
||||
|
||||
@@ -266,7 +266,7 @@ jobs:
|
||||
run: scripts/ci/ios-sim/install-axe.sh
|
||||
|
||||
- name: Run smoke test
|
||||
run: scripts/ci/ios-sim/smoke-test.sh "$RUNNER_TEMP/DerivedData/Build/Products/Debug-iphonesimulator/SimpleX.app" "$RUNNER_TEMP/smoke-test"
|
||||
run: scripts/ci/ios-sim/smoke-test.sh "$RUNNER_TEMP/DerivedData/Build/Products/Debug-iphonesimulator/SimpleX.app" "$RUNNER_TEMP/smoke-test" images
|
||||
|
||||
- name: Stop test backend
|
||||
if: ${{ !cancelled() && env.START_BACKEND == 'true' }}
|
||||
@@ -354,6 +354,14 @@ jobs:
|
||||
- name: Run smoke test
|
||||
run: scripts/ci/ios-sim/smoke-test.sh "$RUNNER_TEMP/SimpleX.app" "$RUNNER_TEMP/claude-test/smoke-test"
|
||||
|
||||
- name: Add test images
|
||||
run: |
|
||||
images=(tests/fixtures/test.jpg tests/fixtures/logo-large-rgba.png)
|
||||
if sips -r 90 -s format heic tests/fixtures/test.jpg --out "$RUNNER_TEMP/test-card-portrait.heic"; then
|
||||
images+=("$RUNNER_TEMP/test-card-portrait.heic")
|
||||
fi
|
||||
xcrun simctl addmedia "$SIM_UDID" "${images[@]}"
|
||||
|
||||
- name: Install Claude Code
|
||||
run: |
|
||||
npm install -g "@anthropic-ai/claude-code@$CLAUDE_CODE_VERSION"
|
||||
|
||||
Reference in new issue
Block a user