Files
simplex-chat/tests/ImageTests.hs
T
fbca2cae81 core, ui: resize images in the core for sharper previews and images (#7941)
* WIP: Haskell impl of resizeImageToStrSize

* WIP: add FFI export wrapper

* WIP: switch ios resizeImageToStrSizeSync to backend

* WIP: fix resizeImageToStrSizeSync

* WIP: add JNI wrappers

* Add non-base64 resize and cWriteImage

* WIP: add chat_write_image

* fix resizeToStr

* fixes

* fix syntax, raise jpeg quality back

* add chatWriteImage

* Fix cfArgs in tests

* Use jpeg-turbo encoder

* add nix pins

* bump jpeg-turbo

* add CI deps

* add encoding funs to android lib

* bump jpeg-turbo for 20.04 fix

* jpeg-turbo fix for 8.10

* android fixes

* Bump jpeg-turbo

* set static flag for jpeg-turbo

* Fix nix for aarch64/android

* Fix android C api

* Fix resize in android

* Bump hs-libjpeg-turbo

* target correct override

* override android lib from android pkgs

* add macos build tweaks

* bump hs-jpeg-turbo for 8.10 fixes

* more build fixes

* make ios compile

* Always use pkgconfig for libjpeg-turbo

* WIP: debug libjpeg in ubuntu pkg-config CI

* Add pkg-config to dockerfiles

* Add libturbojpeg to dockerfiles

* Use -dev version of libturbojpeg in dockerfiles

* Install pkg-config in windows CI

* Disable flag for custom libturbojpeg linking options

Workarounds relocation error on x86_64-linux with gcc:
  /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libturbojpeg.a(turbojpeg.c.o): relocation R_X86_64_TPOFF32 against `errStr' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: failed to set dynamic section sizes: bad value

* WIP: debug libjpeg in windows pkg-config CI

* WIP: debug libjpeg package in windows CI

* core: image resize module and buffer FFI

* ci: link libjpeg-turbo statically on all platforms

* ui: resize images in the core

* core: rename image resize functions to chat_resize_image and chat_resize_image_uri

* ci: restore whitespace in build.yml

* ci: build libjpeg-turbo 3.1.4.1 from a submodule

* core: hs-jpeg-turbo from github.com/simplex-chat

* nodejs: resize images

* python: resize images

* core: hs-jpeg-turbo from gitlab.com, the GitHub import lacks LFS objects

* core: generate /img previews

* ci: link libjpeg-turbo on Windows without pkg-config

* ci: build libjpeg-turbo on Windows with the GHC toolchain

* docs: plan for image resizing in the core

* docs: update image resizing plan

* ios: prevent crash when wallpaper image cannot be resized

* android, desktop: raise an error when an image cannot be resized

* docs: update spec references for image resizing

* docker: build the CLI image from a clone without submodules

* docs: apply review fixes to image resizing plan

* docker: build the CLI from a git worktree with the submodule present

* docs: apply second review to image resizing plan

* docs: point image resizing plan at the rewritten commit

* core: choose image format by transparency in chat_resize_image

* nodejs: resize images to the app sizes

* python: resize images to the app sizes

* nodejs, python: describe the core modules as used for image resizing

* docs: add license notices of the image libraries

* ci: describe the avatar method without attributing it to the apps

* ci, ios, nix: remove the libjpeg-turbo build outside cabal

* python: name resize_image in the core module text

* nodejs, python: describe resizing to the app image sizes

* core: refuse images larger than the supported size

* core: resize to JPEG images with a side above 65500 pixels

* docs: record image resizing decisions for the bundled libjpeg-turbo

* core, docs: build libjpeg-turbo and stb from the fork submodules

* docs: record the fork pins in the image resizing plan

* core: resize images to at most 4320 pixels per side, refuse above 16384

* docs: record the image side limits in the image resizing plan

* core: pin the rebased hs-jpeg-turbo fork

* core: pin the hs-jpeg-turbo fork without package.yaml

* docs: plan scaled JPEG decoding and EXIF orientation of images

* docs: copy the JuicyPixels-stbir licence of the pinned fork

* core: pin the hs-jpeg-turbo fork with scaled JPEG decoding

* core: decode JPEG at the display size, up to 65500 pixels per side

* core: rotate JPEG images by their EXIF orientation

* docs: refer the image plan to the JPEG decoding code

* docs: record the approved image processing decisions

* core: limit JPEG decoding memory to 1024 MB outside iOS and Android

* docs: name JPEG decoding in the libjpeg-turbo notice

* docs: plan image resizing from files and the decisions of 26 Oct

* core: scale images in one Catmull-Rom pass with sharpening

* docs: plan image sizes at a fixed quality

* core: resize images at a fixed quality to fill the size limit

* core: resize images from a file path or a file descriptor

* ios, android, desktop: resize picked images from their files

* core: encode resized images with mozjpeg

* tests: check descriptor closing by the file it refers to

* core: raise JPEG quality only for a full-size image below 97% of the limit

* ci: name the mozjpeg submodule in the Windows cleanup comments

* docs: refer the image plan to the current code

* docs: refer the image plan to the code merged with master

* core: export every image resizing function from each library

* core: close the file descriptor when its handle cannot be created

* docs: count the C sources of mozjpeg with the arithmetic decoder

* core: use the reviewed hs-jpeg-turbo and JuicyPixels-stbir forks

* ci: build the armv7a Android core library on request

* core: round image heights half up in the size search

* ios: describe the file image resizing functions in SimpleX.h

* ci: pick, preview and send images in the Claude app tests

* core: decode PNG images with libspng and refuse other formats

* core: encode PNG with libspng, keeping the smaller filter choice

* ci: pick, preview and send images in the scripted app smoke tests

* flake: build the PNG decoder package in nix builds

* ios: remove the copy of a picked image after sending it

* core: refuse image files larger than the decode memory limit

* core: scale a JPEG image to its largest size once per resize

* core: bisect JPEG quality without a step function

* core: reduce the memory used to sharpen and to decode PNG images

* android, desktop: pass the image format to the core as an integer

* android, desktop: fuse the preview conditions of picked media

* android, desktop, ci: check that the core resized the smoke test images

* python: remove the docstrings of the explicit image resizing functions

* docs: add the second review fixes to the image plan

* ci: tap the iOS attachment button by its accessibility label

* core: keep the colour profiles of resized images

* docs: add the colour profiles to the image plan

* ci: wait up to 30 s for the new group screens in the iOS smoke test

* ci: pick the Android smoke test images from their own photo folders

* ci: tap the iOS photo picker by position, as its tree is not shown

* ci: pick the test card and the logo by their iOS picker positions

* core: update JuicyPixels-stbir with the stb_image_resize licence terms

* core: write a fixed Display P3 profile for Display P3 images only

* core: do not keep the input image in memory for its colour profile

* core: read the JPEG segments once for the orientation and the profile

* core: read an image file into one buffer of its size

* docs: add the third review fixes to the image plan

* core: resize images with Display P3 or no profile, not profile bytes

* ci: check the pixel sizes of the images sent in the iOS smoke test

* core: update hs-jpeg-turbo with PNG chunk limits and encoder fixes

* docs: list Compact ICC Profiles, libspng and miniz in the dependencies

* android, desktop: reword the picked media preview comment

* docs: refer the image plan to the code merged with master

* ci: render the iOS snapshots before the images case

* ci: dismiss the snapshot tests' camera alert before the iOS images case

* core: update hs-jpeg-turbo with tests and the JuicyPixels-spng synopsis

* core: pin hs-jpeg-turbo and JuicyPixels-stbir in simplex-chat repositories

* update ios core library

* core, ui: accept only JPEG and PNG image data

* core: update JuicyPixels-stbir

* docs: plan for the iOS copy of picked images

---------

Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: Alexandre Esteves <alexfmpe@proton.me>
2026-10-30 12:14:51 +00:00

459 lines
22 KiB
Haskell

{-# LANGUAGE OverloadedStrings #-}
module ImageTests where
import Codec.Picture (DynamicImage (..), Image (..), Pixel8, PixelCMYK8, PixelRGB16 (..), PixelRGB8 (..), PixelRGBA16 (..), PixelRGBA8 (..), convertRGB8, convertRGBA8, decodeImage, dynamicMap, encodeBitmap, encodeGifImage, generateImage, pixelAt)
import Codec.Picture.JPEGTurbo (encodeRGB)
import Codec.Picture.Jpg (encodeDirectJpegAtQualityWithMetadata, encodeJpegAtQuality)
import Codec.Picture.Png (encodePng)
import Codec.Picture.Types (convertImage, extractLumaPlane, pixelFold)
import Control.Monad (forM_, when)
import Control.Monad.Except (runExceptT)
import Data.Bits (complement, shiftR, testBit, xor)
import qualified Data.ByteString as B
import qualified Data.ByteString.Base64.Lazy as LB64
import qualified Data.ByteString.Lazy.Char8 as LB
import Data.Either (isLeft, isRight)
import qualified Data.Text as T
import Data.Word (Word32)
import Simplex.Chat.Image
import Simplex.Chat.Library.Commands (fixedImagePreview, imagePreview)
import Simplex.Chat.Types (ImageData (..))
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Crypto.File (CryptoFile (..))
import qualified Simplex.Messaging.Crypto.File as CF
import System.FilePath ((</>))
import Test.Hspec
import UnliftIO.STM (atomically)
import UnliftIO.Temporary (withTempDirectory)
imageTests :: Spec
imageTests = do
describe "data URI" $ do
it "should resize JPEG to JPEG" $ testDataUri "tests/fixtures/test.jpg" "data:image/jpg;base64,"
it "should resize PNG with transparency to PNG" $ testDataUri "tests/fixtures/logo-large-rgba.png" "data:image/png;base64,"
it "should resize opaque PNG to JPEG" $ testDataUri "tests/fixtures/preview-issue1.png" "data:image/jpg;base64,"
it "should convert 16-bit images to 8 bits" testDataUri16Bit
describe "image data" $ do
it "should reduce JPEG size" testResizeJpeg
it "should reduce PNG size" testResizePng
it "should fail when image does not fit" testImageTooLarge
it "should refuse image with a side above 16384 pixels" testImageSideLimit
it "should halve image to at most 4320 pixels per side" testDisplaySide
it "should fill 97-100% of the limit" testFill
it "should raise JPEG quality when the image fits at its size" testFullSizeQuality
describe "image preview" $ do
it "should generate preview of an image file" $ testFilePreview $ CryptoFile "tests/fixtures/test.jpg" Nothing
it "should generate preview of an encrypted image file" testEncryptedFilePreview
it "should keep the fixed preview when the file is not an image" $
imagePreview (CryptoFile "tests/fixtures/test.pdf" Nothing) `shouldReturn` fixedImagePreview
describe "JPEG decoding" $ do
it "should decode JPEG at the display size" testJpegDisplaySize
it "should decode JPEG colour spaces" testJpegColourSpaces
it "should refuse unsupported and malformed JPEG" testJpegRefused
it "should apply EXIF orientation of JPEG" testJpegOrientation
describe "PNG decoding" $ do
it "should decode PNG as JuicyPixels does" testPngDecoding
it "should refuse malformed PNG and other formats" testPngRefused
describe "scaling" $ do
it "should scale and sharpen as the research code" testScaleResearch
it "should keep an image of the same size" testScaleSameSize
it "should scale transparency in premultiplied alpha" testScaleTransparency
describe "colour profiles" $ do
it "should write the Display P3 profile for Display P3 JPEG" testJpegProfile
it "should write the Display P3 profile for Display P3 PNG" testPngProfile
it "should drop colour profiles other than Display P3" testProfileDropped
it "should drop the Display P3 profile above a tenth of the limit" testProfileAboveLimit
it "should ignore inconsistent JPEG colour profiles" testJpegProfileInconsistent
testDataUri :: FilePath -> LB.ByteString -> IO ()
testDataUri path prefix = do
Right img <- decodeImage <$> B.readFile path
Right uri <- resizeImageDataUri 14000 img CPNone
LB.length uri `shouldSatisfy` (<= 14000)
let (uriPrefix, body) = LB.splitAt (LB.length prefix) uri
uriPrefix `shouldBe` prefix
Right resized <- pure $ decodeImage . LB.toStrict =<< LB64.decode body
imageSize resized `shouldKeepAspect` imageSize img
testDataUri16Bit :: IO ()
testDataUri16Bit = do
let rgb = ImageRGB16 $ generateImage (\x y -> PixelRGB16 (fromIntegral x * 512) (fromIntegral y * 512) 0) 128 128
rgba = ImageRGBA16 $ generateImage (\x y -> PixelRGBA16 (fromIntegral x * 512) (fromIntegral y * 512) 0 32768) 128 128
Right jpegUri <- resizeImageDataUri 14000 rgb CPNone
jpegUri `shouldSatisfy` ("data:image/jpg;base64," `LB.isPrefixOf`)
Right pngUri <- resizeImageDataUri 14000 rgba CPNone
Just body <- pure $ LB.stripPrefix "data:image/png;base64," pngUri
Right (ImageRGBA8 _) <- pure $ decodeImage . LB.toStrict =<< LB64.decode body
pure ()
testResizeJpeg :: IO ()
testResizeJpeg = do
Right img <- decodeImage <$> B.readFile "tests/fixtures/test.jpg"
Right jpeg <- resizeImage IFJpeg 10000 img CPNone
LB.length jpeg `shouldSatisfy` (<= 10000)
LB.take 2 jpeg `shouldBe` "\xFF\xD8"
Right resized <- pure $ decodeImage $ LB.toStrict jpeg
fst (imageSize resized) `shouldSatisfy` (< 1280)
imageSize resized `shouldKeepAspect` imageSize img
testResizePng :: IO ()
testResizePng = do
Right img <- decodeImage <$> B.readFile "tests/fixtures/logo-large-rgba.png"
Right png <- resizeImage IFPng 10000 img CPNone
LB.length png `shouldSatisfy` (<= 10000)
Right resized@(ImageRGBA8 _) <- pure $ decodeImage $ LB.toStrict png
fst (imageSize resized) `shouldSatisfy` (< 1630)
imageSize resized `shouldKeepAspect` imageSize img
testImageTooLarge :: IO ()
testImageTooLarge = do
let img = ImageRGBA8 $ generateImage (\_ _ -> PixelRGBA8 0 0 0 0) 1 1
resizeImage IFPng 10 img CPNone >>= (`shouldSatisfy` isLeft)
resizeImage IFJpeg 10 img CPNone >>= (`shouldSatisfy` isLeft)
testImageSideLimit :: IO ()
testImageSideLimit = do
let blank w h = ImageRGBA8 $ generateImage (\_ _ -> PixelRGBA8 0 0 0 0) w h
resizeAll img = sequence [resizeImage IFPng 100000 img CPNone, resizeImage IFJpeg 100000 img CPNone, resizeImageDataUri 100000 img CPNone]
forM_ [blank 16384 4, blank 4 16384] $ \img ->
resizeAll img >>= (`shouldSatisfy` all isRight)
forM_ [blank 16385 4, blank 4 16385] $ \img ->
resizeAll img >>= (`shouldSatisfy` all isLeft)
testDisplaySide :: IO ()
testDisplaySide =
forM_ [((8640, 8), (4320, 4)), ((8, 8640), (4, 4320)), ((4320, 4), (4320, 4))] $ \((w, h), size) -> do
let opaque = ImageRGB8 $ generateImage (\_ _ -> PixelRGB8 0 0 0) w h
transparent = ImageRGBA8 $ generateImage (\_ _ -> PixelRGBA8 0 0 0 0) w h
forM_ [opaque, transparent] $ \img -> do
Right image <- resizeImage (imageFormat img) 261120 img CPNone
decodedSize image `shouldBe` Right size
forM_ [12500, 14000] $ \limit -> do
Right uri <- resizeImageDataUri limit img CPNone
(decodedSize =<< LB64.decode (LB.drop 1 $ LB.dropWhile (/= ',') uri)) `shouldBe` Right size
where
decodedSize = fmap imageSize . decodeImage . LB.toStrict
testFill :: IO ()
testFill = do
forM_ ["tests/fixtures/test.jpg", "tests/fixtures/preview-issue1.png"] $ \path -> do
Right img <- decodeImage <$> B.readFile path
Right uri <- resizeImageDataUri 14000 img CPNone
LB.length uri `shouldSatisfy` filled 14000
Right image <- resizeImage IFJpeg 20000 img CPNone
LB.length image `shouldSatisfy` filled 20000
where
filled limit n = n <= limit && fromIntegral n >= 0.97 * (fromIntegral limit :: Double)
testFullSizeQuality :: IO ()
testFullSizeQuality = do
let rgb = generateImage (\x y -> PixelRGB8 (fromIntegral $ x * 4) (fromIntegral $ y * 5) (fromIntegral $ x * y)) 64 48
q65 <- encodeRGB 65 Nothing rgb
q95 <- encodeRGB 95 Nothing rgb
let limit = fromIntegral $ (B.length q65 + B.length q95) `div` 2
Right jpeg <- resizeImage IFJpeg limit (ImageRGB8 rgb) CPNone
LB.length jpeg `shouldSatisfy` (\n -> n > fromIntegral (B.length q65) && n <= limit)
imageSize <$> decodeImage (LB.toStrict jpeg) `shouldBe` Right (64, 48)
testFilePreview :: CryptoFile -> IO ()
testFilePreview f = do
ImageData preview <- imagePreview f
T.length preview `shouldSatisfy` (<= 14000)
preview `shouldSatisfy` ("data:image/jpg;base64," `T.isPrefixOf`)
testEncryptedFilePreview :: IO ()
testEncryptedFilePreview = withTempDirectory "tests/tmp" "image-preview" $ \dir -> do
args <- atomically . CF.randomArgs =<< C.newRandom
let f = CryptoFile (dir </> "test.jpg") (Just args)
Right () <- runExceptT . CF.writeFile f . LB.fromStrict =<< B.readFile "tests/fixtures/test.jpg"
testFilePreview f
testJpegDisplaySize :: IO ()
testJpegDisplaySize =
forM_ sizes $ \(w, (w', h')) ->
forM_ [((w, 16), (w', h')), ((16, w), (h', w'))] $ \((iw, ih), size) -> do
jpeg <- encodeRGB 90 Nothing $ generateImage (\_ _ -> PixelRGB8 128 128 128) iw ih
Right (img, _) <- decodeResizable jpeg
resizedSize img `shouldReturn` size
when (w <= 8641) $ do
Right full <- pure $ decodeImage jpeg
resizedSize full `shouldReturn` size
where
sizes = [(4320, (4320, 16)), (4321, (2160, 8)), (8640, (4320, 8)), (8641, (4320, 8)), (17280, (4320, 4)), (17281, (4320, 4)), (17282, (4320, 4)), (34561, (4320, 2)), (65500, (4094, 1))]
resizedSize img = do
Right image <- resizeImage IFJpeg 261120 img CPNone
either fail (pure . imageSize) $ decodeImage $ LB.toStrict image
testJpegColourSpaces :: IO ()
testJpegColourSpaces = do
let rgb = generateImage (\x y -> PixelRGB8 (fromIntegral $ x * 4) (fromIntegral $ y * 5) 128) 64 48
progressive <- encodeRGB 90 Nothing rgb
Right (ImageRGB8 _, _) <- decodeResizable progressive
Right (ImageRGB8 _, _) <- decodeResizable $ baselineJpeg rgb
Right (ImageY8 _, _) <- decodeResizable $ LB.toStrict $ encodeDirectJpegAtQualityWithMetadata 90 mempty $ extractLumaPlane rgb
Right (ImageCMYK8 _, _) <- decodeResizable $ LB.toStrict $ encodeDirectJpegAtQualityWithMetadata 90 mempty (convertImage rgb :: Image PixelCMYK8)
pure ()
testJpegRefused :: IO ()
testJpegRefused =
forM_ inputs $ \input -> isLeft <$> decodeResizable input `shouldReturn` True
where
jpeg = baselineJpeg $ generateImage (\_ _ -> PixelRGB8 128 128 128) 64 48
sof = B.length $ fst $ B.breakSubstring "\xFF\xC0" jpeg
patchAt i bytes = B.take i jpeg <> B.pack bytes <> B.drop (i + length bytes) jpeg
sides w h = patchAt (sof + 5) $ concatMap (\v -> [fromIntegral (v `shiftR` 8), fromIntegral v]) [h, w :: Int]
inputs =
[ patchAt (sof + 4) [12],
patchAt (sof + 1) [0xC3],
sides 65501 16,
sides 16385 16384,
B.init jpeg,
B.take sof jpeg <> "\0" <> B.drop sof jpeg
]
testJpegOrientation :: IO ()
testJpegOrientation = do
jpeg <- encodeRGB 100 Nothing $ generateImage quadrant 64 32
forM_ orientations $ \(orientation, size, colours) -> do
Right (img, _) <- decodeResizable $ withExif (odd orientation) 8 orientation jpeg
imageSize img `shouldBe` size
quadrants img `shouldBe` colours
Right resized <- resizeImage IFJpeg 261120 img CPNone
"Exif" `B.isInfixOf` LB.toStrict resized `shouldBe` False
Right (img, _) <- decodeResizable $ withExif True 1000 6 jpeg
quadrants img `shouldBe` [red, green, blue, white]
where
quadrant x y = case (x < 32, y < 16) of
(True, True) -> red
(False, True) -> green
(True, False) -> blue
(False, False) -> white
orientations :: [(Int, (Int, Int), [PixelRGB8])]
orientations =
[ (1, (64, 32), [red, green, blue, white]),
(2, (64, 32), [green, red, white, blue]),
(3, (64, 32), [white, blue, green, red]),
(4, (64, 32), [blue, white, red, green]),
(5, (32, 64), [red, blue, green, white]),
(6, (32, 64), [blue, red, white, green]),
(7, (32, 64), [white, green, blue, red]),
(8, (32, 64), [green, white, red, blue])
]
red = PixelRGB8 255 0 0
green = PixelRGB8 0 255 0
blue = PixelRGB8 0 0 255
white = PixelRGB8 255 255 255
quadrants img =
let rgb = convertRGB8 img
(w, h) = (imageWidth rgb, imageHeight rgb)
snap (PixelRGB8 r g b) = PixelRGB8 (level r) (level g) (level b)
level c = if c > 127 then 255 else 0
in [snap $ pixelAt rgb x y | y <- [h `div` 4, 3 * h `div` 4], x <- [w `div` 4, 3 * w `div` 4]]
withExif :: Bool -> Int -> Int -> B.ByteString -> B.ByteString
withExif littleEndian ifdOffset orientation jpeg = B.take 2 jpeg <> "\xFF\xE1" <> bigEndian 2 (8 + B.length tiff) <> "Exif\0\0" <> tiff <> B.drop 2 jpeg
where
tiff = B.concat [if littleEndian then "II" else "MM", word 2 42, word 4 ifdOffset, word 2 1, word 2 0x0112, word 2 3, word 4 1, word 2 orientation, word 2 0, word 4 0]
word n = (if littleEndian then B.reverse else id) . bigEndian n
testPngDecoding :: IO ()
testPngDecoding = do
transparent <- B.readFile "tests/fixtures/logo-large-rgba.png"
Right (img@(ImageRGBA8 rgba), CPNone) <- decodeResizable transparent
Right reference <- pure $ decodeImage transparent
imageData rgba `shouldBe` imageData (convertRGBA8 reference)
IFPng <- pure $ imageFormat img
opaque <- B.readFile "tests/fixtures/preview-issue1.png"
Right (ImageRGB8 rgb, CPNone) <- decodeResizable opaque
Right reference' <- pure $ decodeImage opaque
imageData rgb `shouldBe` imageData (convertRGB8 reference')
testPngRefused :: IO ()
testPngRefused =
forM_ inputs $ \(input, err) ->
either Just (const Nothing) <$> decodeResizable input `shouldReturn` Just err
where
inputs =
[ (pngShortImageData, "IDAT stream too short"),
(pngSideAbove16384, "image width exceeds user limit"),
(LB.toStrict $ encodeGifImage $ generateImage (\_ _ -> 0) 16 16, "unsupported image format"),
(LB.toStrict $ encodeBitmap $ generateImage (\_ _ -> PixelRGB8 0 0 0) 16 16, "unsupported image format")
]
pngShortImageData :: B.ByteString
pngShortImageData = pngWithSides 2048 2048 $ blankPng 2048 1
pngSideAbove16384 :: B.ByteString
pngSideAbove16384 = pngWithSides 65537 65537 $ blankPng 1 1
blankPng :: Int -> Int -> B.ByteString
blankPng w h = LB.toStrict $ encodePng (generateImage (\_ _ -> 0) w h :: Image Pixel8)
pngWithSides :: Int -> Int -> B.ByteString -> B.ByteString
pngWithSides w h png = B.take 8 png <> pngChunk "IHDR" (bigEndian 4 w <> bigEndian 4 h <> B.take 5 (B.drop 24 png)) <> B.drop 33 png
pngWithIccProfile :: B.ByteString -> B.ByteString -> B.ByteString
pngWithIccProfile profile png = B.take 33 png <> pngChunk "iCCP" ("test\0\0" <> zlibStored profile) <> B.drop 33 png
pngChunk :: B.ByteString -> B.ByteString -> B.ByteString
pngChunk name body = bigEndian 4 (B.length body) <> name <> body <> bigEndian 4 (fromIntegral $ crc32 $ name <> body)
crc32 :: B.ByteString -> Word32
crc32 = complement . B.foldl' (\c b -> iterate step (c `xor` fromIntegral b) !! 8) 0xffffffff
where
step c = if testBit c 0 then 0xedb88320 `xor` shiftR c 1 else shiftR c 1
zlibStored :: B.ByteString -> B.ByteString
zlibStored s = "\x78\x01\x01" <> B.reverse (bigEndian 2 n) <> B.reverse (bigEndian 2 $ 65535 - n) <> s <> bigEndian 4 (b * 65536 + a)
where
n = B.length s
(a, b) = B.foldl' (\(x, y) w -> let x' = (x + fromIntegral w) `mod` 65521 in (x', (y + x') `mod` 65521)) (1, 0) s
withIccProfile :: [B.ByteString] -> B.ByteString -> B.ByteString
withIccProfile chunks jpeg = B.take 2 jpeg <> B.concat ["\xFF\xE2" <> bigEndian 2 (14 + B.length c) <> "ICC_PROFILE\0" <> c | c <- chunks] <> B.drop 2 jpeg
iccChunk :: Int -> Int -> B.ByteString -> B.ByteString
iccChunk n count profile = B.pack [fromIntegral n, fromIntegral count] <> profile
bigEndian :: Int -> Int -> B.ByteString
bigEndian n v = B.pack [fromIntegral (v `shiftR` (8 * i)) | i <- [n - 1, n - 2 .. 0]]
testScaleResearch :: IO ()
testScaleResearch = do
Right src <- decodeImage <$> B.readFile "tests/fixtures/preview-issue1.png"
Right (ImageRGB8 expected) <- decodeImage <$> B.readFile "tests/fixtures/preview-issue1-400x200-sharpened.png"
let scaled = scaleRGB8 (convertRGB8 src) (400, 200)
imageSize (ImageRGB8 scaled) `shouldBe` (400, 200)
pixelFold (\n x y p -> if p == pixelAt expected x y then n else n + 1) (0 :: Int) scaled `shouldBe` 0
testScaleSameSize :: IO ()
testScaleSameSize = do
let rgb = generateImage (\x y -> PixelRGB8 (fromIntegral $ x * 4) (fromIntegral $ y * 5) 128) 64 48
rgba = generateImage (\x y -> PixelRGBA8 (fromIntegral $ x * 4) (fromIntegral $ y * 5) 128 (fromIntegral $ x + y)) 64 48
imageData (scaleRGB8 rgb (64, 48)) `shouldBe` imageData rgb
imageData (scaleRGBA8 rgba (64, 48)) `shouldBe` imageData rgba
testScaleTransparency :: IO ()
testScaleTransparency = do
let scaled = scaleRGBA8 (generateImage pixel 64 64) (40, 40)
visible = [p | y <- [0 .. 39], x <- [0 .. 39], let p@(PixelRGBA8 _ _ _ a) = pixelAt scaled x y, a > 0]
filter (\(PixelRGBA8 r g b _) -> (r, g, b) /= (255, 0, 0)) visible `shouldBe` []
any (\(PixelRGBA8 _ _ _ a) -> a < 255) visible `shouldBe` True
where
pixel x y
| (x - 32) * (x - 32) + (y - 32) * (y - 32) < 400 = PixelRGBA8 255 0 0 255
| even (x + y) = PixelRGBA8 0 255 0 0
| otherwise = PixelRGBA8 0 0 255 0
testJpegProfile :: IO ()
testJpegProfile = do
displayP3 <- B.readFile displayP3File
jpeg <- encodeRGB 90 Nothing gradient
let (start, rest) = B.splitAt 100 p3Profile
Right (img, profile) <- decodeResizable $ withIccProfile [iccChunk 2 2 rest, iccChunk 1 2 start] jpeg
profile `shouldBe` CPDisplayP3
Right (_, profile') <- decodeResizable $ withIccProfile [iccChunk 1 1 $ iccProfile "RGB " $ 33759 + 2 : drop 1 p3Primaries] jpeg
profile' `shouldBe` CPDisplayP3
Right jpegData <- resizeImage IFJpeg 261120 img profile
LB.toStrict jpegData `shouldSatisfy` (displayP3 `B.isInfixOf`)
Right pngData <- resizeImage IFPng 261120 img profile
forM_ [jpegData, pngData] $ \resized -> resizedProfile resized `shouldReturn` profile
testPngProfile :: IO ()
testPngProfile = do
let png = pngWithIccProfile p3Profile $ LB.toStrict $ encodePng transparentGradient
Right (img, profile) <- decodeResizable png
profile `shouldBe` CPDisplayP3
forM_ [IFJpeg, IFPng] $ \format -> do
Right resized <- resizeImage format 261120 img profile
resizedProfile resized `shouldReturn` profile
Right uri <- resizeImageDataUri 14000 img profile
uri `shouldSatisfy` ("data:image/png;base64," `LB.isPrefixOf`)
uriProfile uri `shouldReturn` profile
testProfileDropped :: IO ()
testProfileDropped = do
jpeg <- encodeRGB 90 Nothing gradient
Right (img, CPNone) <- decodeResizable jpeg
Right without <- resizeImage IFJpeg 261120 img CPNone
forM_ profiles $ \profile -> do
Right (img', profile') <- decodeResizable $ withIccProfile [iccChunk 1 1 profile] jpeg
profile' `shouldBe` CPNone
resizeImage IFJpeg 261120 img' profile' `shouldReturn` Right without
where
profiles =
[ iccProfile "RGB " srgbPrimaries,
iccProfile "RGB " $ 33759 + 3 : drop 1 p3Primaries,
iccProfile "GRAY" p3Primaries,
patch 128 (bigEndian 4 9) p3Profile,
patch 160 (bigEndian 4 209) p3Profile
]
srgbPrimaries = [28576, 14581, 912, 25239, 46983, 6361, 9375, 3972, 46787]
patch i bytes profile = B.take i profile <> bytes <> B.drop (i + B.length bytes) profile
testProfileAboveLimit :: IO ()
testProfileAboveLimit = do
let img = ImageRGB8 gradient
forM_ [IFJpeg, IFPng] $ \format -> do
Right without <- resizeImage format 4799 img CPNone
resizeImage format 4799 img CPDisplayP3 `shouldReturn` Right without
Right kept <- resizeImage format 4800 img CPDisplayP3
resizedProfile kept `shouldReturn` CPDisplayP3
Right uriWithout <- resizeImageDataUri 6399 img CPNone
resizeImageDataUri 6399 img CPDisplayP3 `shouldReturn` Right uriWithout
Right uri <- resizeImageDataUri 6400 img CPDisplayP3
uriProfile uri `shouldReturn` CPDisplayP3
testJpegProfileInconsistent :: IO ()
testJpegProfileInconsistent = do
jpeg <- encodeRGB 90 Nothing gradient
let (start, rest) = B.splitAt 100 p3Profile
forM_
[ [iccChunk 1 2 start],
[iccChunk 1 2 start, iccChunk 1 2 rest],
[iccChunk 1 2 start, iccChunk 2 3 rest],
[iccChunk 0 1 start],
[iccChunk 1 1 ""],
["\1"]
]
$ \chunks -> do
Right (_, profile) <- decodeResizable $ withIccProfile chunks jpeg
profile `shouldBe` CPNone
gradient :: Image PixelRGB8
gradient = generateImage (\x y -> PixelRGB8 (fromIntegral $ x * 4) (fromIntegral $ y * 5) 128) 64 48
transparentGradient :: Image PixelRGBA8
transparentGradient = generateImage (\x y -> PixelRGBA8 (fromIntegral $ x * 4) (fromIntegral $ y * 5) 128 (fromIntegral $ 255 - x)) 64 48
displayP3File :: FilePath
displayP3File = "src/Simplex/Chat/Image/DisplayP3-v4.icc"
p3Profile :: B.ByteString
p3Profile = iccProfile "RGB " p3Primaries
p3Primaries :: [Int]
p3Primaries = [33759, 15807, -69, 19135, 45367, 2745, 10296, 4363, 51385]
iccProfile :: B.ByteString -> [Int] -> B.ByteString
iccProfile colourSpace primaries = header <> bigEndian 4 3 <> B.concat entries <> B.concat tags
where
header = bigEndian 4 228 <> B.replicate 12 0 <> colourSpace <> "XYZ " <> B.replicate 104 0
entries = [sig <> bigEndian 4 (168 + 20 * k) <> bigEndian 4 20 | (k, sig) <- zip [0 ..] ["rXYZ", "gXYZ", "bXYZ"]]
tags = ["XYZ \0\0\0\0" <> B.concat (map (bigEndian 4) $ take 3 $ drop (3 * k) primaries) | k <- [0 .. 2]]
resizedProfile :: LB.ByteString -> IO ColourProfile
resizedProfile = fmap (either (const CPNone) snd) . decodeResizable . LB.toStrict
uriProfile :: LB.ByteString -> IO ColourProfile
uriProfile = either (const $ pure CPNone) resizedProfile . LB64.decode . LB.drop 1 . LB.dropWhile (/= ',')
baselineJpeg :: Image PixelRGB8 -> B.ByteString
baselineJpeg = LB.toStrict . encodeJpegAtQuality 90 . convertImage
imageSize :: DynamicImage -> (Int, Int)
imageSize = dynamicMap (\img -> (imageWidth img, imageHeight img))
shouldKeepAspect :: (Int, Int) -> (Int, Int) -> Expectation
shouldKeepAspect (w, h) (w', h') = abs (w * h' - h * w') `shouldSatisfy` (<= w' + h')