core, ui: resize images in the core for sharper previews and images (#7941)

* WIP: Haskell impl of resizeImageToStrSize

* WIP: add FFI export wrapper

* WIP: switch ios resizeImageToStrSizeSync to backend

* WIP: fix resizeImageToStrSizeSync

* WIP: add JNI wrappers

* Add non-base64 resize and cWriteImage

* WIP: add chat_write_image

* fix resizeToStr

* fixes

* fix syntax, raise jpeg quality back

* add chatWriteImage

* Fix cfArgs in tests

* Use jpeg-turbo encoder

* add nix pins

* bump jpeg-turbo

* add CI deps

* add encoding funs to android lib

* bump jpeg-turbo for 20.04 fix

* jpeg-turbo fix for 8.10

* android fixes

* Bump jpeg-turbo

* set static flag for jpeg-turbo

* Fix nix for aarch64/android

* Fix android C api

* Fix resize in android

* Bump hs-libjpeg-turbo

* target correct override

* override android lib from android pkgs

* add macos build tweaks

* bump hs-jpeg-turbo for 8.10 fixes

* more build fixes

* make ios compile

* Always use pkgconfig for libjpeg-turbo

* WIP: debug libjpeg in ubuntu pkg-config CI

* Add pkg-config to dockerfiles

* Add libturbojpeg to dockerfiles

* Use -dev version of libturbojpeg in dockerfiles

* Install pkg-config in windows CI

* Disable flag for custom libturbojpeg linking options

Workarounds relocation error on x86_64-linux with gcc:
  /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libturbojpeg.a(turbojpeg.c.o): relocation R_X86_64_TPOFF32 against `errStr' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: failed to set dynamic section sizes: bad value

* WIP: debug libjpeg in windows pkg-config CI

* WIP: debug libjpeg package in windows CI

* core: image resize module and buffer FFI

* ci: link libjpeg-turbo statically on all platforms

* ui: resize images in the core

* core: rename image resize functions to chat_resize_image and chat_resize_image_uri

* ci: restore whitespace in build.yml

* ci: build libjpeg-turbo 3.1.4.1 from a submodule

* core: hs-jpeg-turbo from github.com/simplex-chat

* nodejs: resize images

* python: resize images

* core: hs-jpeg-turbo from gitlab.com, the GitHub import lacks LFS objects

* core: generate /img previews

* ci: link libjpeg-turbo on Windows without pkg-config

* ci: build libjpeg-turbo on Windows with the GHC toolchain

* docs: plan for image resizing in the core

* docs: update image resizing plan

* ios: prevent crash when wallpaper image cannot be resized

* android, desktop: raise an error when an image cannot be resized

* docs: update spec references for image resizing

* docker: build the CLI image from a clone without submodules

* docs: apply review fixes to image resizing plan

* docker: build the CLI from a git worktree with the submodule present

* docs: apply second review to image resizing plan

* docs: point image resizing plan at the rewritten commit

* core: choose image format by transparency in chat_resize_image

* nodejs: resize images to the app sizes

* python: resize images to the app sizes

* nodejs, python: describe the core modules as used for image resizing

* docs: add license notices of the image libraries

* ci: describe the avatar method without attributing it to the apps

* ci, ios, nix: remove the libjpeg-turbo build outside cabal

* python: name resize_image in the core module text

* nodejs, python: describe resizing to the app image sizes

* core: refuse images larger than the supported size

* core: resize to JPEG images with a side above 65500 pixels

* docs: record image resizing decisions for the bundled libjpeg-turbo

* core, docs: build libjpeg-turbo and stb from the fork submodules

* docs: record the fork pins in the image resizing plan

* core: resize images to at most 4320 pixels per side, refuse above 16384

* docs: record the image side limits in the image resizing plan

* core: pin the rebased hs-jpeg-turbo fork

* core: pin the hs-jpeg-turbo fork without package.yaml

* docs: plan scaled JPEG decoding and EXIF orientation of images

* docs: copy the JuicyPixels-stbir licence of the pinned fork

* core: pin the hs-jpeg-turbo fork with scaled JPEG decoding

* core: decode JPEG at the display size, up to 65500 pixels per side

* core: rotate JPEG images by their EXIF orientation

* docs: refer the image plan to the JPEG decoding code

* docs: record the approved image processing decisions

* core: limit JPEG decoding memory to 1024 MB outside iOS and Android

* docs: name JPEG decoding in the libjpeg-turbo notice

* docs: plan image resizing from files and the decisions of 26 Oct

* core: scale images in one Catmull-Rom pass with sharpening

* docs: plan image sizes at a fixed quality

* core: resize images at a fixed quality to fill the size limit

* core: resize images from a file path or a file descriptor

* ios, android, desktop: resize picked images from their files

* core: encode resized images with mozjpeg

* tests: check descriptor closing by the file it refers to

* core: raise JPEG quality only for a full-size image below 97% of the limit

* ci: name the mozjpeg submodule in the Windows cleanup comments

* docs: refer the image plan to the current code

* docs: refer the image plan to the code merged with master

* core: export every image resizing function from each library

* core: close the file descriptor when its handle cannot be created

* docs: count the C sources of mozjpeg with the arithmetic decoder

* core: use the reviewed hs-jpeg-turbo and JuicyPixels-stbir forks

* ci: build the armv7a Android core library on request

* core: round image heights half up in the size search

* ios: describe the file image resizing functions in SimpleX.h

* ci: pick, preview and send images in the Claude app tests

* core: decode PNG images with libspng and refuse other formats

* core: encode PNG with libspng, keeping the smaller filter choice

* ci: pick, preview and send images in the scripted app smoke tests

* flake: build the PNG decoder package in nix builds

* ios: remove the copy of a picked image after sending it

* core: refuse image files larger than the decode memory limit

* core: scale a JPEG image to its largest size once per resize

* core: bisect JPEG quality without a step function

* core: reduce the memory used to sharpen and to decode PNG images

* android, desktop: pass the image format to the core as an integer

* android, desktop: fuse the preview conditions of picked media

* android, desktop, ci: check that the core resized the smoke test images

* python: remove the docstrings of the explicit image resizing functions

* docs: add the second review fixes to the image plan

* ci: tap the iOS attachment button by its accessibility label

* core: keep the colour profiles of resized images

* docs: add the colour profiles to the image plan

* ci: wait up to 30 s for the new group screens in the iOS smoke test

* ci: pick the Android smoke test images from their own photo folders

* ci: tap the iOS photo picker by position, as its tree is not shown

* ci: pick the test card and the logo by their iOS picker positions

* core: update JuicyPixels-stbir with the stb_image_resize licence terms

* core: write a fixed Display P3 profile for Display P3 images only

* core: do not keep the input image in memory for its colour profile

* core: read the JPEG segments once for the orientation and the profile

* core: read an image file into one buffer of its size

* docs: add the third review fixes to the image plan

* core: resize images with Display P3 or no profile, not profile bytes

* ci: check the pixel sizes of the images sent in the iOS smoke test

* core: update hs-jpeg-turbo with PNG chunk limits and encoder fixes

* docs: list Compact ICC Profiles, libspng and miniz in the dependencies

* android, desktop: reword the picked media preview comment

* docs: refer the image plan to the code merged with master

* ci: render the iOS snapshots before the images case

* ci: dismiss the snapshot tests' camera alert before the iOS images case

* core: update hs-jpeg-turbo with tests and the JuicyPixels-spng synopsis

* core: pin hs-jpeg-turbo and JuicyPixels-stbir in simplex-chat repositories

* update ios core library

* core, ui: accept only JPEG and PNG image data

* core: update JuicyPixels-stbir

* docs: plan for the iOS copy of picked images

---------

Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: Alexandre Esteves <alexfmpe@proton.me>
This commit is contained in:
authored and GitHub committed 2026-10-30 12:14:51 +00:00
1 parent 1491d0cff8
commit fbca2cae81
118 files changed
+4227 -348

No files matched your search

+1 -1
View File
@@ -134,7 +134,7 @@ The library docs are [here](./docs/README.md).
Library provides these modules:
- [bot](./docs/Namespace.bot.md): a simple declarative API to run a chat-bot with a single function call. It automates creating and updating of the bot profile, address and bot commands shown in the app UI.
- [api](./docs/Namespace.api.md): an API to send chat commands and receive chat events to/from chat core. You need to use it in bot event handlers, and for any other use cases.
- [core](./docs/Namespace.core.md): a low level API to the core library - the same that is used in desktop clients. You are unlikely to ever need to use this module directly.
- [core](./docs/Namespace.core.md): a low level API to the core library - the same that is used in desktop clients. You are unlikely to ever need to use this module directly, except for image resizing.
- [util](./docs/Namespace.util.md): useful functions for chat events and types.
+112 -27
View File
@@ -36,6 +36,8 @@ struct Library {
chat_read_file_fn chat_read_file = nullptr;
chat_encrypt_file_fn chat_encrypt_file = nullptr;
chat_decrypt_file_fn chat_decrypt_file = nullptr;
chat_resize_image_fn chat_resize_image = nullptr;
chat_resize_image_uri_fn chat_resize_image_uri = nullptr;
};
static Library lib;
@@ -136,6 +138,45 @@ static bool RequireLoaded(Env env) {
return false;
}
template <typename F>
static bool RequireExport(Env env, F fn, const std::string& name, const std::string& feature) {
if (fn != nullptr) return true;
Error::New(env, "loaded libsimplex does not export " + name + "; " + feature + " needs a newer libsimplex").ThrowAsJavaScriptException();
return false;
}
static bool GetInt32Arg(Env env, const Value& value, const std::string& name, int& result) {
Number number = value.As<Number>();
result = number.Int32Value();
if (static_cast<double>(result) == number.DoubleValue()) return true;
RangeError::New(env, "Expected 32-bit integer " + name).ThrowAsJavaScriptException();
return false;
}
struct BufferArg {
char* data;
int len;
};
static bool GetBufferArg(Env env, const Value& value, BufferArg& result) {
size_t len;
if (value.IsArrayBuffer()) {
ArrayBuffer ab = value.As<ArrayBuffer>();
result.data = static_cast<char*>(ab.Data());
len = ab.ByteLength();
} else {
TypedArray view = value.As<TypedArray>();
result.data = static_cast<char*>(view.ArrayBuffer().Data()) + view.ByteOffset();
len = view.ByteLength();
}
if (len > static_cast<size_t>(INT_MAX)) {
RangeError::New(env, "Buffer is too large").ThrowAsJavaScriptException();
return false;
}
result.len = static_cast<int>(len);
return true;
}
Value Load(const CallbackInfo& args) {
Env env = args.Env();
if (args.Length() < 1 || !args[0].IsString()) {
@@ -173,6 +214,8 @@ Value Load(const CallbackInfo& args) {
return env.Undefined();
}
Resolve(h, "chat_migrate_init_queue", l.chat_migrate_init_queue);
Resolve(h, "chat_resize_image", l.chat_resize_image);
Resolve(h, "chat_resize_image_uri", l.chat_resize_image_uri);
Resolve(h, "hs_thread_done", l.hs_thread_done);
lib = l;
haskell_init();
@@ -527,10 +570,7 @@ Value ChatMigrateInit(const CallbackInfo& args) {
Value ChatMigrateInitQueue(const CallbackInfo& args) {
Env env = args.Env();
if (!RequireLoaded(env)) return env.Undefined();
if (lib.chat_migrate_init_queue == nullptr) {
Error::New(env, "loaded libsimplex does not export chat_migrate_init_queue; queue size needs a newer libsimplex").ThrowAsJavaScriptException();
return env.Undefined();
}
if (!RequireExport(env, lib.chat_migrate_init_queue, "chat_migrate_init_queue", "queue size")) return env.Undefined();
if (args.Length() < 4 || !args[0].IsString() || !args[1].IsString() || !args[2].IsString() || !args[3].IsNumber()) {
TypeError::New(env, "Expected three string arguments and number").ThrowAsJavaScriptException();
return env.Undefined();
@@ -539,12 +579,8 @@ Value ChatMigrateInitQueue(const CallbackInfo& args) {
std::string path = args[0].As<String>().Utf8Value();
std::string key = args[1].As<String>().Utf8Value();
std::string confirm = args[2].As<String>().Utf8Value();
Number queue_size_arg = args[3].As<Number>();
int queue_size = queue_size_arg.Int32Value();
if (static_cast<double>(queue_size) != queue_size_arg.DoubleValue()) {
RangeError::New(env, "Expected 32-bit integer queue size").ThrowAsJavaScriptException();
return env.Undefined();
}
int queue_size;
if (!GetInt32Arg(env, args[3], "queue size", queue_size)) return env.Undefined();
Function cb;
Promise promise = CreatePromiseAndCallback(env, cb);
@@ -657,27 +693,14 @@ Value ChatWriteFile(const CallbackInfo& args) {
chat_ctrl ctrl = FromChatCtrlBigInt(args[0]);
std::string path = args[1].As<String>().Utf8Value();
char* data;
size_t len;
if (args[2].IsArrayBuffer()) {
ArrayBuffer ab = args[2].As<ArrayBuffer>();
data = static_cast<char*>(ab.Data());
len = ab.ByteLength();
} else {
TypedArray view = args[2].As<TypedArray>();
data = static_cast<char*>(view.ArrayBuffer().Data()) + view.ByteOffset();
len = view.ByteLength();
}
if (len > static_cast<size_t>(INT_MAX)) {
RangeError::New(env, "Buffer is too large").ThrowAsJavaScriptException();
return env.Undefined();
}
BufferArg buffer;
if (!GetBufferArg(env, args[2], buffer)) return env.Undefined();
Function cb;
Promise promise = CreatePromiseAndCallback(env, cb);
auto execute_fn = [ctrl, path, data, len](ResultAsyncWorker* worker) {
char* c_res = lib.chat_write_file(ctrl, path.c_str(), data, static_cast<int>(len));
auto execute_fn = [ctrl, path, buffer](ResultAsyncWorker* worker) {
char* c_res = lib.chat_write_file(ctrl, path.c_str(), buffer.data, buffer.len);
HandleCResult(worker, c_res, "chat_write_file");
};
@@ -785,6 +808,66 @@ Value ChatDecryptFile(const CallbackInfo& args) {
return promise;
}
Value ChatResizeImage(const CallbackInfo& args) {
Env env = args.Env();
if (!RequireLoaded(env)) return env.Undefined();
if (!RequireExport(env, lib.chat_resize_image, "chat_resize_image", "image resizing")) return env.Undefined();
if (args.Length() < 3 || !(args[0].IsArrayBuffer() || args[0].IsTypedArray()) || !args[1].IsNumber() || !args[2].IsNumber()) {
TypeError::New(env, "Expected ArrayBuffer or Uint8Array, number (maxSize), number (png)").ThrowAsJavaScriptException();
return env.Undefined();
}
BufferArg image;
if (!GetBufferArg(env, args[0], image)) return env.Undefined();
int max_size;
if (!GetInt32Arg(env, args[1], "maxSize", max_size)) return env.Undefined();
int png;
if (!GetInt32Arg(env, args[2], "png", png)) return env.Undefined();
Function cb;
Promise promise = CreatePromiseAndCallback(env, cb);
auto execute_fn = [image, max_size, png](ResultAsyncWorker* worker) {
char* c_res = lib.chat_resize_image(image.data, image.len, max_size, png);
HandleCResult(worker, c_res, "chat_resize_image");
};
ResultAsyncWorker* worker = new ResultAsyncWorker(cb, std::move(execute_fn));
worker->KeepAlive(args[0].As<Object>());
worker->Queue();
return promise;
}
Value ChatResizeImageUri(const CallbackInfo& args) {
Env env = args.Env();
if (!RequireLoaded(env)) return env.Undefined();
if (!RequireExport(env, lib.chat_resize_image_uri, "chat_resize_image_uri", "image resizing")) return env.Undefined();
if (args.Length() < 2 || !(args[0].IsArrayBuffer() || args[0].IsTypedArray()) || !args[1].IsNumber()) {
TypeError::New(env, "Expected ArrayBuffer or Uint8Array, number (maxSize)").ThrowAsJavaScriptException();
return env.Undefined();
}
BufferArg image;
if (!GetBufferArg(env, args[0], image)) return env.Undefined();
int max_size;
if (!GetInt32Arg(env, args[1], "maxSize", max_size)) return env.Undefined();
Function cb;
Promise promise = CreatePromiseAndCallback(env, cb);
auto execute_fn = [image, max_size](ResultAsyncWorker* worker) {
char* c_res = lib.chat_resize_image_uri(image.data, image.len, max_size);
HandleCResult(worker, c_res, "chat_resize_image_uri");
};
ResultAsyncWorker* worker = new ResultAsyncWorker(cb, std::move(execute_fn));
worker->KeepAlive(args[0].As<Object>());
worker->Queue();
return promise;
}
Object Init(Env env, Object exports) {
exports.Set("load", Function::New(env, Load));
auto* receivers = new Receivers();
@@ -804,6 +887,8 @@ Object Init(Env env, Object exports) {
exports.Set("chat_read_file", Function::New(env, ChatReadFile));
exports.Set("chat_encrypt_file", Function::New(env, ChatEncryptFile));
exports.Set("chat_decrypt_file", Function::New(env, ChatDecryptFile));
exports.Set("chat_resize_image", Function::New(env, ChatResizeImage));
exports.Set("chat_resize_image_uri", Function::New(env, ChatResizeImageUri));
return exports;
}
@@ -35,4 +35,10 @@ typedef char *(*chat_encrypt_file_fn)(chat_ctrl ctrl, const char *fromPath, cons
// chat_decrypt_file returns null-terminated string with the error message
typedef char *(*chat_decrypt_file_fn)(const char *fromPath, const char *key, const char *nonce, const char *toPath);
// chat_resize_image returns base64 of at most maxSize bytes of PNG (png > 0), JPEG (png == 0), or PNG for an image with transparency, otherwise JPEG (png < 0), or an empty string
typedef char *(*chat_resize_image_fn)(const char *data, int len, int maxSize, int png);
// chat_resize_image_uri returns a data URI of at most maxSize characters, or an empty string
typedef char *(*chat_resize_image_uri_fn)(const char *data, int len, int maxSize);
#endif /* simplex_h */
@@ -7,7 +7,7 @@
# core
A low level API to the core library - the same that is used in desktop clients.
You are unlikely to ever need to use this module directly.
You are unlikely to ever need to use this module directly, except for image resizing.
## Namespaces
@@ -17,6 +17,7 @@ You are unlikely to ever need to use this module directly.
## Enumerations
- [ImageSize](core.Enumeration.ImageSize.md)
- [MigrationConfirmation](core.Enumeration.MigrationConfirmation.md)
## Classes
@@ -45,6 +46,9 @@ You are unlikely to ever need to use this module directly.
- [chatMigrateInit](core.Function.chatMigrateInit.md)
- [chatReadFile](core.Function.chatReadFile.md)
- [chatRecvMsgWait](core.Function.chatRecvMsgWait.md)
- [chatResizeImage](core.Function.chatResizeImage.md)
- [chatResizeImageUri](core.Function.chatResizeImageUri.md)
- [chatSendCmd](core.Function.chatSendCmd.md)
- [chatWriteFile](core.Function.chatWriteFile.md)
- [loadLibrary](core.Function.loadLibrary.md)
- [resizeImage](core.Function.resizeImage.md)
@@ -6,7 +6,7 @@
# Class: ChatAPIError
Defined in: [src/core.ts:117](../core.ts#L117)
Defined in: [src/core.ts:154](../core.ts#L154)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:117](../core.ts#L117)
> **new ChatAPIError**(`message`, `chatError?`): `ChatAPIError`
Defined in: [src/core.ts:118](../core.ts#L118)
Defined in: [src/core.ts:155](../core.ts#L155)
#### Parameters
@@ -44,7 +44,7 @@ Defined in: [src/core.ts:118](../core.ts#L118)
> **chatError**: `ChatError` \| `undefined` = `undefined`
Defined in: [src/core.ts:118](../core.ts#L118)
Defined in: [src/core.ts:155](../core.ts#L155)
***
@@ -52,7 +52,7 @@ Defined in: [src/core.ts:118](../core.ts#L118)
> **message**: `string`
Defined in: [src/core.ts:118](../core.ts#L118)
Defined in: [src/core.ts:155](../core.ts#L155)
#### Inherited from
@@ -6,7 +6,7 @@
# Class: ChatInitError
Defined in: [src/core.ts:141](../core.ts#L141)
Defined in: [src/core.ts:187](../core.ts#L187)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:141](../core.ts#L141)
> **new ChatInitError**(`message`, `dbMigrationError`): `ChatInitError`
Defined in: [src/core.ts:142](../core.ts#L142)
Defined in: [src/core.ts:188](../core.ts#L188)
#### Parameters
@@ -44,7 +44,7 @@ Defined in: [src/core.ts:142](../core.ts#L142)
> **dbMigrationError**: [`DBMigrationError`](core.TypeAlias.DBMigrationError.md)
Defined in: [src/core.ts:142](../core.ts#L142)
Defined in: [src/core.ts:188](../core.ts#L188)
***
@@ -52,7 +52,7 @@ Defined in: [src/core.ts:142](../core.ts#L142)
> **message**: `string`
Defined in: [src/core.ts:142](../core.ts#L142)
Defined in: [src/core.ts:188](../core.ts#L188)
#### Inherited from
@@ -6,7 +6,7 @@
# Interface: ErrorMigration
Defined in: [src/core.ts:174](../core.ts#L174)
Defined in: [src/core.ts:220](../core.ts#L220)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:174](../core.ts#L174)
> **dbFile**: `string`
Defined in: [src/core.ts:176](../core.ts#L176)
Defined in: [src/core.ts:222](../core.ts#L222)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:176](../core.ts#L176)
> **migrationError**: [`MigrationError`](core.TypeAlias.MigrationError.md)
Defined in: [src/core.ts:177](../core.ts#L177)
Defined in: [src/core.ts:223](../core.ts#L223)
***
@@ -34,7 +34,7 @@ Defined in: [src/core.ts:177](../core.ts#L177)
> **type**: `"errorMigration"`
Defined in: [src/core.ts:175](../core.ts#L175)
Defined in: [src/core.ts:221](../core.ts#L221)
#### Overrides
@@ -6,7 +6,7 @@
# Interface: ErrorNotADatabase
Defined in: [src/core.ts:169](../core.ts#L169)
Defined in: [src/core.ts:215](../core.ts#L215)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:169](../core.ts#L169)
> **dbFile**: `string`
Defined in: [src/core.ts:171](../core.ts#L171)
Defined in: [src/core.ts:217](../core.ts#L217)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:171](../core.ts#L171)
> **type**: `"errorNotADatabase"`
Defined in: [src/core.ts:170](../core.ts#L170)
Defined in: [src/core.ts:216](../core.ts#L216)
#### Overrides
@@ -6,7 +6,7 @@
# Interface: ErrorSQL
Defined in: [src/core.ts:180](../core.ts#L180)
Defined in: [src/core.ts:226](../core.ts#L226)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:180](../core.ts#L180)
> **dbFile**: `string`
Defined in: [src/core.ts:182](../core.ts#L182)
Defined in: [src/core.ts:228](../core.ts#L228)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:182](../core.ts#L182)
> **migrationSQLError**: `string`
Defined in: [src/core.ts:183](../core.ts#L183)
Defined in: [src/core.ts:229](../core.ts#L229)
***
@@ -34,7 +34,7 @@ Defined in: [src/core.ts:183](../core.ts#L183)
> **type**: `"errorSQL"`
Defined in: [src/core.ts:181](../core.ts#L181)
Defined in: [src/core.ts:227](../core.ts#L227)
#### Overrides
@@ -6,7 +6,7 @@
# Interface: InvalidConfirmation
Defined in: [src/core.ts:161](../core.ts#L161)
Defined in: [src/core.ts:207](../core.ts#L207)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:161](../core.ts#L161)
> **type**: `"invalidConfirmation"`
Defined in: [src/core.ts:162](../core.ts#L162)
Defined in: [src/core.ts:208](../core.ts#L208)
#### Overrides
@@ -6,7 +6,7 @@
# Interface: InvalidQueueSize
Defined in: [src/core.ts:165](../core.ts#L165)
Defined in: [src/core.ts:211](../core.ts#L211)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:165](../core.ts#L165)
> **type**: `"invalidQueueSize"`
Defined in: [src/core.ts:166](../core.ts#L166)
Defined in: [src/core.ts:212](../core.ts#L212)
#### Overrides
@@ -8,4 +8,4 @@
> **Tag** = `"invalidConfirmation"` \| `"invalidQueueSize"` \| `"errorNotADatabase"` \| `"errorMigration"` \| `"errorSQL"`
Defined in: [src/core.ts:155](../core.ts#L155)
Defined in: [src/core.ts:201](../core.ts#L201)
@@ -0,0 +1,35 @@
[**simplex-chat**](README.md)
***
[simplex-chat](README.md) / [core](Namespace.core.md) / ImageSize
# Enumeration: ImageSize
Defined in: [src/core.ts:173](../core.ts#L173)
App image size limit
## Enumeration Members
### Image
> **Image**: `"image"`
Defined in: [src/core.ts:174](../core.ts#L174)
***
### MessagePreview
> **MessagePreview**: `"messagePreview"`
Defined in: [src/core.ts:176](../core.ts#L176)
***
### ProfilePreview
> **ProfilePreview**: `"profilePreview"`
Defined in: [src/core.ts:175](../core.ts#L175)
@@ -6,7 +6,7 @@
# Enumeration: MigrationConfirmation
Defined in: [src/core.ts:126](../core.ts#L126)
Defined in: [src/core.ts:163](../core.ts#L163)
Migration confirmation mode
@@ -16,7 +16,7 @@ Migration confirmation mode
> **Console**: `"console"`
Defined in: [src/core.ts:129](../core.ts#L129)
Defined in: [src/core.ts:166](../core.ts#L166)
***
@@ -24,7 +24,7 @@ Defined in: [src/core.ts:129](../core.ts#L129)
> **Error**: `"error"`
Defined in: [src/core.ts:130](../core.ts#L130)
Defined in: [src/core.ts:167](../core.ts#L167)
***
@@ -32,7 +32,7 @@ Defined in: [src/core.ts:130](../core.ts#L130)
> **YesUp**: `"yesUp"`
Defined in: [src/core.ts:127](../core.ts#L127)
Defined in: [src/core.ts:164](../core.ts#L164)
***
@@ -40,4 +40,4 @@ Defined in: [src/core.ts:127](../core.ts#L127)
> **YesUpDown**: `"yesUpDown"`
Defined in: [src/core.ts:128](../core.ts#L128)
Defined in: [src/core.ts:165](../core.ts#L165)
@@ -0,0 +1,31 @@
[**simplex-chat**](README.md)
***
[simplex-chat](README.md) / [core](Namespace.core.md) / chatResizeImage
# Function: chatResizeImage()
> **chatResizeImage**(`image`, `maxSize`, `format`): `Promise`\<`Buffer`\<`ArrayBufferLike`\>\>
Defined in: [src/core.ts:122](../core.ts#L122)
Resize image to JPEG or PNG of at most maxSize bytes
## Parameters
### image
`ArrayBuffer` \| `Uint8Array`\<`ArrayBufferLike`\>
### maxSize
`number`
### format
`"jpeg"` \| `"png"`
## Returns
`Promise`\<`Buffer`\<`ArrayBufferLike`\>\>
@@ -0,0 +1,27 @@
[**simplex-chat**](README.md)
***
[simplex-chat](README.md) / [core](Namespace.core.md) / chatResizeImageUri
# Function: chatResizeImageUri()
> **chatResizeImageUri**(`image`, `maxSize`): `Promise`\<`string`\>
Defined in: [src/core.ts:115](../core.ts#L115)
Resize image to a data URI of at most maxSize characters
## Parameters
### image
`ArrayBuffer` \| `Uint8Array`\<`ArrayBufferLike`\>
### maxSize
`number`
## Returns
`Promise`\<`string`\>
@@ -0,0 +1,73 @@
[**simplex-chat**](README.md)
***
[simplex-chat](README.md) / [core](Namespace.core.md) / resizeImage
# Function: resizeImage()
## Call Signature
> **resizeImage**(`image`, `size`): `Promise`\<`Buffer`\<`ArrayBufferLike`\>\>
Defined in: [src/core.ts:129](../core.ts#L129)
Resize image to the app size limit: JPEG or PNG for an image, a data URI for a preview
### Parameters
#### image
`ArrayBuffer` \| `Uint8Array`\<`ArrayBufferLike`\>
#### size
[`Image`](core.Enumeration.ImageSize.md#image)
### Returns
`Promise`\<`Buffer`\<`ArrayBufferLike`\>\>
## Call Signature
> **resizeImage**(`image`, `size`): `Promise`\<`string`\>
Defined in: [src/core.ts:130](../core.ts#L130)
Resize image to the app size limit: JPEG or PNG for an image, a data URI for a preview
### Parameters
#### image
`ArrayBuffer` \| `Uint8Array`\<`ArrayBufferLike`\>
#### size
[`ProfilePreview`](core.Enumeration.ImageSize.md#profilepreview) \| [`MessagePreview`](core.Enumeration.ImageSize.md#messagepreview)
### Returns
`Promise`\<`string`\>
## Call Signature
> **resizeImage**(`image`, `size`): `Promise`\<`string` \| `Buffer`\<`ArrayBufferLike`\>\>
Defined in: [src/core.ts:131](../core.ts#L131)
Resize image to the app size limit: JPEG or PNG for an image, a data URI for a preview
### Parameters
#### image
`ArrayBuffer` \| `Uint8Array`\<`ArrayBufferLike`\>
#### size
[`ImageSize`](core.Enumeration.ImageSize.md)
### Returns
`Promise`\<`string` \| `Buffer`\<`ArrayBufferLike`\>\>
@@ -6,7 +6,7 @@
# Interface: APIResult\<R\>
Defined in: [src/core.ts:112](../core.ts#L112)
Defined in: [src/core.ts:149](../core.ts#L149)
## Type Parameters
@@ -20,7 +20,7 @@ Defined in: [src/core.ts:112](../core.ts#L112)
> `optional` **error?**: `ChatError`
Defined in: [src/core.ts:114](../core.ts#L114)
Defined in: [src/core.ts:151](../core.ts#L151)
***
@@ -28,4 +28,4 @@ Defined in: [src/core.ts:114](../core.ts#L114)
> `optional` **result?**: `R`
Defined in: [src/core.ts:113](../core.ts#L113)
Defined in: [src/core.ts:150](../core.ts#L150)
@@ -6,7 +6,7 @@
# Interface: CryptoArgs
Defined in: [src/core.ts:136](../core.ts#L136)
Defined in: [src/core.ts:182](../core.ts#L182)
File encryption key and nonce
@@ -16,7 +16,7 @@ File encryption key and nonce
> **fileKey**: `string`
Defined in: [src/core.ts:137](../core.ts#L137)
Defined in: [src/core.ts:183](../core.ts#L183)
***
@@ -24,4 +24,4 @@ Defined in: [src/core.ts:137](../core.ts#L137)
> **fileNonce**: `string`
Defined in: [src/core.ts:138](../core.ts#L138)
Defined in: [src/core.ts:184](../core.ts#L184)
@@ -6,7 +6,7 @@
# Interface: UpMigration
Defined in: [src/core.ts:215](../core.ts#L215)
Defined in: [src/core.ts:261](../core.ts#L261)
## Properties
@@ -14,7 +14,7 @@ Defined in: [src/core.ts:215](../core.ts#L215)
> **upName**: `string`
Defined in: [src/core.ts:216](../core.ts#L216)
Defined in: [src/core.ts:262](../core.ts#L262)
***
@@ -22,4 +22,4 @@ Defined in: [src/core.ts:216](../core.ts#L216)
> **withDown**: `boolean`
Defined in: [src/core.ts:217](../core.ts#L217)
Defined in: [src/core.ts:263](../core.ts#L263)
@@ -6,7 +6,7 @@
# Interface: MTREDifferent
Defined in: [src/core.ts:236](../core.ts#L236)
Defined in: [src/core.ts:282](../core.ts#L282)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:236](../core.ts#L236)
> **downMigrations**: `string`[]
Defined in: [src/core.ts:238](../core.ts#L238)
Defined in: [src/core.ts:284](../core.ts#L284)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:238](../core.ts#L238)
> **type**: `"different"`
Defined in: [src/core.ts:237](../core.ts#L237)
Defined in: [src/core.ts:283](../core.ts#L283)
#### Overrides
@@ -6,7 +6,7 @@
# Interface: MTRENoDown
Defined in: [src/core.ts:231](../core.ts#L231)
Defined in: [src/core.ts:277](../core.ts#L277)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:231](../core.ts#L231)
> **dbMigrations**: `string`[]
Defined in: [src/core.ts:233](../core.ts#L233)
Defined in: [src/core.ts:279](../core.ts#L279)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:233](../core.ts#L233)
> **type**: `"noDown"`
Defined in: [src/core.ts:232](../core.ts#L232)
Defined in: [src/core.ts:278](../core.ts#L278)
#### Overrides
@@ -8,4 +8,4 @@
> **Tag** = `"noDown"` \| `"different"`
Defined in: [src/core.ts:225](../core.ts#L225)
Defined in: [src/core.ts:271](../core.ts#L271)
@@ -6,7 +6,7 @@
# Interface: MEDowngrade
Defined in: [src/core.ts:204](../core.ts#L204)
Defined in: [src/core.ts:250](../core.ts#L250)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:204](../core.ts#L204)
> **downMigrations**: `string`[]
Defined in: [src/core.ts:206](../core.ts#L206)
Defined in: [src/core.ts:252](../core.ts#L252)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:206](../core.ts#L206)
> **type**: `"downgrade"`
Defined in: [src/core.ts:205](../core.ts#L205)
Defined in: [src/core.ts:251](../core.ts#L251)
#### Overrides
@@ -6,7 +6,7 @@
# Interface: MEUpgrade
Defined in: [src/core.ts:199](../core.ts#L199)
Defined in: [src/core.ts:245](../core.ts#L245)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:199](../core.ts#L199)
> **type**: `"upgrade"`
Defined in: [src/core.ts:200](../core.ts#L200)
Defined in: [src/core.ts:246](../core.ts#L246)
#### Overrides
@@ -30,4 +30,4 @@ Defined in: [src/core.ts:200](../core.ts#L200)
> **upMigrations**: [`UpMigration`](core.Interface.UpMigration.md)[]
Defined in: [src/core.ts:201](../core.ts#L201)
Defined in: [src/core.ts:247](../core.ts#L247)
@@ -6,7 +6,7 @@
# Interface: MigrationError
Defined in: [src/core.ts:209](../core.ts#L209)
Defined in: [src/core.ts:255](../core.ts#L255)
## Extends
@@ -18,7 +18,7 @@ Defined in: [src/core.ts:209](../core.ts#L209)
> **mtrError**: [`MTRError`](core.TypeAlias.MTRError.md)
Defined in: [src/core.ts:211](../core.ts#L211)
Defined in: [src/core.ts:257](../core.ts#L257)
***
@@ -26,7 +26,7 @@ Defined in: [src/core.ts:211](../core.ts#L211)
> **type**: `"migrationError"`
Defined in: [src/core.ts:210](../core.ts#L210)
Defined in: [src/core.ts:256](../core.ts#L256)
#### Overrides
@@ -8,4 +8,4 @@
> **Tag** = `"upgrade"` \| `"downgrade"` \| `"migrationError"`
Defined in: [src/core.ts:193](../core.ts#L193)
Defined in: [src/core.ts:239](../core.ts#L239)
@@ -8,4 +8,4 @@
> **DBMigrationError** = [`InvalidConfirmation`](core.DBMigrationError.Interface.InvalidConfirmation.md) \| [`InvalidQueueSize`](core.DBMigrationError.Interface.InvalidQueueSize.md) \| [`ErrorNotADatabase`](core.DBMigrationError.Interface.ErrorNotADatabase.md) \| [`ErrorMigration`](core.DBMigrationError.Interface.ErrorMigration.md) \| [`ErrorSQL`](core.DBMigrationError.Interface.ErrorSQL.md)
Defined in: [src/core.ts:147](../core.ts#L147)
Defined in: [src/core.ts:193](../core.ts#L193)
@@ -8,4 +8,4 @@
> **MTRError** = [`MTRENoDown`](core.MTRError.Interface.MTRENoDown.md) \| [`MTREDifferent`](core.MTRError.Interface.MTREDifferent.md)
Defined in: [src/core.ts:220](../core.ts#L220)
Defined in: [src/core.ts:266](../core.ts#L266)
@@ -8,4 +8,4 @@
> **MigrationError** = [`MEUpgrade`](core.MigrationError.Interface.MEUpgrade.md) \| [`MEDowngrade`](core.MigrationError.Interface.MEDowngrade.md) \| [`MigrationError`](core.MigrationError.Interface.MigrationError.md)
Defined in: [src/core.ts:187](../core.ts#L187)
Defined in: [src/core.ts:233](../core.ts#L233)
+46
View File
@@ -109,6 +109,43 @@ function cryptoArgsResult(res: string): CryptoArgs {
}
}
/**
* Resize image to a data URI of at most maxSize characters
*/
export async function chatResizeImageUri(image: ArrayBuffer | Uint8Array, maxSize: number): Promise<string> {
return resizedImage(await simplex.chat_resize_image_uri(image, maxSize))
}
/**
* Resize image to JPEG or PNG of at most maxSize bytes
*/
export async function chatResizeImage(image: ArrayBuffer | Uint8Array, maxSize: number, format: "jpeg" | "png"): Promise<Buffer> {
return resizeImageData(image, maxSize, format === "png" ? 1 : 0)
}
/**
* Resize image to the app size limit: JPEG or PNG for an image, a data URI for a preview
*/
export function resizeImage(image: ArrayBuffer | Uint8Array, size: ImageSize.Image): Promise<Buffer>
export function resizeImage(image: ArrayBuffer | Uint8Array, size: ImageSize.ProfilePreview | ImageSize.MessagePreview): Promise<string>
export function resizeImage(image: ArrayBuffer | Uint8Array, size: ImageSize): Promise<Buffer | string>
export async function resizeImage(image: ArrayBuffer | Uint8Array, size: ImageSize): Promise<Buffer | string> {
switch (size) {
case ImageSize.Image: return resizeImageData(image, 261120, -1)
case ImageSize.ProfilePreview: return chatResizeImageUri(image, 12500)
case ImageSize.MessagePreview: return chatResizeImageUri(image, 14000)
}
}
async function resizeImageData(image: ArrayBuffer | Uint8Array, maxSize: number, png: number): Promise<Buffer> {
return Buffer.from(resizedImage(await simplex.chat_resize_image(image, maxSize, png)), "base64")
}
function resizedImage(res: string): string {
if (res === "") throw new Error("image cannot be decoded or does not fit maxSize")
return res
}
export interface APIResult<R> {
result?: R
error?: T.ChatError
@@ -130,6 +167,15 @@ export enum MigrationConfirmation {
Error = "error"
}
/**
* App image size limit
*/
export enum ImageSize {
Image = "image",
ProfilePreview = "profilePreview",
MessagePreview = "messagePreview"
}
/**
* File encryption key and nonce
*/
+1 -1
View File
@@ -12,7 +12,7 @@ export * as api from "./api"
/**
* A low level API to the core library - the same that is used in desktop clients.
* You are unlikely to ever need to use this module directly.
* You are unlikely to ever need to use this module directly, except for image resizing.
*/
export * as core from "./core"
+2
View File
@@ -10,3 +10,5 @@ export function chat_write_file(ctrl: bigint, path: string, buffer: ArrayBuffer
export function chat_read_file(path: string, key: string, nonce: string): Promise<Buffer>
export function chat_encrypt_file(ctrl: bigint, fromPath: string, toPath: string): Promise<string>
export function chat_decrypt_file(fromPath: string, key: string, nonce: string, toPath: string): Promise<string>
export function chat_resize_image(image: ArrayBuffer | Uint8Array, maxSize: number, png: number): Promise<string>
export function chat_resize_image_uri(image: ArrayBuffer | Uint8Array, maxSize: number): Promise<string>
+1 -1
View File
@@ -1,7 +1,7 @@
// The add-on is downloaded from the libsimplex release, so it is loaded at runtime by core.loadLibrary.
const NOT_LOADED = "libsimplex is not loaded, call core.loadLibrary(backend) first"
const BINDINGS = ["chat_migrate_init", "chat_migrate_init_queue", "chat_close_store", "chat_send_cmd", "chat_recv_msg_wait",
"chat_write_file", "chat_read_file", "chat_encrypt_file", "chat_decrypt_file"]
"chat_write_file", "chat_read_file", "chat_encrypt_file", "chat_decrypt_file", "chat_resize_image", "chat_resize_image_uri"]
let addon
@@ -1,6 +1,8 @@
import {execFile, spawnSync} from "child_process";
import * as crypto from "crypto";
import * as fs from "fs";
import * as path from "path";
import * as zlib from "zlib";
import {core} from "../src/index";
import * as libs from "../src/libs";
import * as simplex from "../src/simplex";
@@ -140,10 +142,102 @@ describe("Core tests", () => {
await expect(core.chatEncryptFile(ctrl, path.join(tmpDir, "unknown.txt"), encryptedPath)).rejects.toThrow();
await expect(core.chatDecryptFile(path.join(tmpDir, "unknown.txt"), cryptoArgs, decryptedPath)).rejects.toThrow();
await stopAndClose(ctrl);
});
describe("image resizing", () => {
const notImage = Buffer.from("not an image");
const resizeError = "image cannot be decoded or does not fit maxSize";
const oldLibraryError = (name: string) => `loaded libsimplex does not export ${name}; image resizing needs a newer libsimplex`;
let imageResizing = false;
beforeAll(async () => {
const error = await core.chatResizeImageUri(notImage, 14000).then(() => "", (e: Error) => e.message);
imageResizing = error !== oldLibraryError("chat_resize_image_uri");
});
function noisePng(width: number, height: number, alpha = false): Buffer {
const rowLength = width * (alpha ? 4 : 3) + 1;
const rows = crypto.randomBytes(rowLength * height);
for (let y = 0; y < height; y++) rows[y * rowLength] = 0;
const header = Buffer.alloc(13);
header.writeUInt32BE(width, 0);
header.writeUInt32BE(height, 4);
header.set([8, alpha ? 6 : 2, 0, 0, 0], 8);
const chunk = (type: string, data: Buffer) => {
const body = Buffer.concat([Buffer.from(type), data]);
const length = Buffer.alloc(4);
length.writeUInt32BE(data.length);
const crc = Buffer.alloc(4);
crc.writeUInt32BE(zlib.crc32(body));
return Buffer.concat([length, body, crc]);
};
const signature = Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]);
return Buffer.concat([signature, chunk("IHDR", header), chunk("IDAT", zlib.deflateSync(rows)), chunk("IEND", Buffer.alloc(0))]);
}
it("should resize image to data URI", async () => {
const resize = core.chatResizeImageUri(noisePng(256, 256), 14000);
if (!imageResizing) return expect(resize).rejects.toThrow(oldLibraryError("chat_resize_image_uri"));
const uri = await resize;
expect(uri.startsWith("data:image/jpg;base64,")).toBe(true);
expect(uri.length).toBeLessThanOrEqual(14000);
});
it("should resize the view of a Uint8Array to JPEG and PNG", async () => {
const image = noisePng(256, 256);
const view = Buffer.concat([Buffer.alloc(3), image]).subarray(3);
const resize = core.chatResizeImage(view, 20000, "jpeg");
if (!imageResizing) return expect(resize).rejects.toThrow(oldLibraryError("chat_resize_image"));
const jpeg = await resize;
expect(jpeg.subarray(0, 3)).toEqual(Buffer.from([0xff, 0xd8, 0xff]));
expect(jpeg.length).toBeLessThanOrEqual(20000);
const png = await core.chatResizeImage(new Uint8Array(image).buffer, 20000, "png");
expect(png.subarray(0, 8)).toEqual(image.subarray(0, 8));
expect(png.length).toBeLessThanOrEqual(20000);
});
it("should resize image to each ImageSize", async () => {
const image = noisePng(512, 512);
const resize = core.resizeImage(image, core.ImageSize.Image);
if (!imageResizing) return expect(resize).rejects.toThrow(oldLibraryError("chat_resize_image"));
const jpeg = await resize;
expect(jpeg.subarray(0, 3)).toEqual(Buffer.from([0xff, 0xd8, 0xff]));
expect(jpeg.length).toBeLessThanOrEqual(261120);
const profile = await core.resizeImage(image, core.ImageSize.ProfilePreview);
expect(profile.startsWith("data:image/jpg;base64,")).toBe(true);
expect(profile.length).toBeLessThanOrEqual(12500);
const preview = await core.resizeImage(image, core.ImageSize.MessagePreview);
expect(preview.startsWith("data:image/jpg;base64,")).toBe(true);
expect(preview.length).toBeLessThanOrEqual(14000);
});
it("should resize image with transparency to PNG for each ImageSize", async () => {
const image = noisePng(256, 256, true);
const resize = core.resizeImage(image, core.ImageSize.Image);
if (!imageResizing) return expect(resize).rejects.toThrow(oldLibraryError("chat_resize_image"));
const png = await resize;
expect(png.subarray(0, 8)).toEqual(image.subarray(0, 8));
expect(png.length).toBeLessThanOrEqual(261120);
const profile = await core.resizeImage(image, core.ImageSize.ProfilePreview);
expect(profile.startsWith("data:image/png;base64,")).toBe(true);
expect(profile.length).toBeLessThanOrEqual(12500);
const preview = await core.resizeImage(image, core.ImageSize.MessagePreview);
expect(preview.startsWith("data:image/png;base64,")).toBe(true);
expect(preview.length).toBeLessThanOrEqual(14000);
});
it("should reject data that is not an image", async () => {
await expect(core.chatResizeImage(notImage, 20000, "jpeg")).rejects.toThrow(imageResizing ? resizeError : oldLibraryError("chat_resize_image"));
});
it("should reject maxSize outside 32-bit integers", async () => {
const error = imageResizing ? "Expected 32-bit integer maxSize" : oldLibraryError("chat_resize_image_uri");
await expect(core.chatResizeImageUri(noisePng(8, 8), 2 ** 31)).rejects.toThrow(error);
});
});
it("should not block the libuv pool while receiving", async () => {
const ctrl = await core.chatMigrateInit(dbPath, "key", core.MigrationConfirmation.YesUp);
const receives = [1, 2, 3, 4].map(() => core.chatRecvMsgWait(ctrl, 500_000));
@@ -38,7 +38,7 @@ describe("native load", () => {
try { simplex[name](); console.log(name + ": no error") } catch (e) { console.log(name + ": " + e.message) }
}`)
const bindings = ["chat_close_store", "chat_decrypt_file", "chat_encrypt_file", "chat_migrate_init", "chat_migrate_init_queue",
"chat_read_file", "chat_recv_msg_wait", "chat_send_cmd", "chat_write_file"]
"chat_read_file", "chat_recv_msg_wait", "chat_resize_image", "chat_resize_image_uri", "chat_send_cmd", "chat_write_file"]
const expected = bindings.map(name => `${name}: libsimplex is not loaded, call core.loadLibrary(backend) first\n`).join("")
expect(out).toBe(`${expected}status=0 signal=null`)
})