dandri/simplex-chat
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplex-chat.git synced 2026-05-15 05:56:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

update github content (#519)

Browse Source 
* update github content

* update comparison

* update link

* move message_views.sql to scripts

* move section

* move news section

* typos

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* update readme

* update readme

* update readme

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
This commit is contained in:
Evgeny Poberezkin
2022-04-11 22:29:08 +01:00
committed by GitHub
parent 0a17f5c491
commit af471d0077
12 changed files with 377 additions and 341 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CLI.md
+224
View File
@@ -0,0 +1,224 @@
# SimpleX Chat terminal (console) app for Linux/MacOS/Windows
## Table of contents
- [Terminal chat features](#terminal-chat-features)
- [Installation](#🚀-installation)
- [Download chat client](#download-chat-client)
- [Linux and MacOS](#linux-and-macos)
- [Windows](#windows)
- [Build from source](#build-from-source)
- [Using Docker](#using-docker)
- [Using Haskell stack](#using-haskell-stack)
- [Usage](#usage)
- [Running the chat client](#running-the-chat-client)
- [How to use SimpleX chat](#how-to-use-simplex-chat)
- [Groups](#groups)
- [Sending files](#sending-files)
- [User contact addresses](#user-contact-addresses)
- [Access chat history](#access-chat-history)
## Terminal chat features
- 1-to-1 chat with multiple people in the same terminal window.
- Group messaging.
- Sending files to contacts and groups.
- User contact addresses - establish connections via multiple-use contact links.
- Messages persisted in a local SQLite database.
- Auto-populated recipient name - just type your messages to reply to the sender once the connection is established.
- Demo SMP servers available and pre-configured in the app - or you can [deploy your own server](https://github.com/simplex-chat/simplexmq#using-smp-server-and-smp-agent).
- No global identity or any names visible to the server(s), ensuring full privacy of your contacts and conversations.
- Two layers of E2E encryption (double-ratchet for duplex connections, using X3DH key agreement with ephemeral Curve448 keys, and NaCl crypto_box for SMP queues, using Curve25519 keys) and out-of-band passing of recipient keys (see [How to use SimpleX chat](#how-to-use-simplex-chat)).
- Message integrity validation (via including the digests of the previous messages).
- Authentication of each command/message by SMP servers with automatically generated Ed448 keys.
- TLS 1.3 transport encryption.
- Additional encryption of messages from SMP server to recipient to reduce traffic correlation.
Public keys involved in key exchange are not used as identity, they are randomly generated for each contact.
See [Encryption Primitives Used](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md#encryption-primitives-used) for technical details.
<a name="🚀-installation"></a>
## 🚀 Installation
### Download chat client
#### Linux and MacOS
To **install** or **update** `simplex-chat`, you should run the install script. To do that, use the following cURL or Wget command:
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/install.sh | bash
```
```sh
wget -qO- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/install.sh | bash
```
Once the chat client downloads, you can run it with `simplex-chat` command in your terminal.
Alternatively, you can manually download the chat binary for your system from the [latest stable release](https://github.com/simplex-chat/simplex-chat/releases) and make it executable as shown below.
```sh
chmod +x <binary>
mv <binary> ~/.local/bin/simplex-chat
```
(or any other preferred location on `PATH`).
On MacOS you also need to [allow Gatekeeper to run it](https://support.apple.com/en-us/HT202491).
#### Windows
```sh
move <binary> %APPDATA%/local/bin/simplex-chat.exe
```
### Build from source
> **Please note:** to build the app use source code from [stable branch](https://github.com/simplex-chat/simplex-chat/tree/stable).
#### Using Docker
On Linux, you can build the chat executable using [docker build with custom output](https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs):
```shell
$ git clone git@github.com:simplex-chat/simplex-chat.git
$ cd simplex-chat
$ git checkout stable
$ DOCKER_BUILDKIT=1 docker build --output ~/.local/bin .
```
> **Please note:** If you encounter `` version `GLIBC_2.28' not found `` error, rebuild it with `haskell:8.10.4-stretch` base image (change it in your local [Dockerfile](Dockerfile)).
#### Using Haskell stack
Install [Haskell stack](https://docs.haskellstack.org/en/stable/README/):
```shell
curl -sSL https://get.haskellstack.org/ | sh
```
and build the project:
```shell
$ git clone git@github.com:simplex-chat/simplex-chat.git
$ cd simplex-chat
$ git checkout stable
$ stack install
```
## Usage
### Running the chat client
To start the chat client, run `simplex-chat` from the terminal.
By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and two SQLite database files `simplex_v1_chat.db` and `simplex_v1_agent.db` are initialized in it.
To specify a different file path prefix for the database files use `-d` command line option:
```shell
$ simplex-chat -d alice
```
Running above, for example, would create `alice_v1_chat.db` and `alice_v1_agent.db` database files in current directory.
Three default SMP servers are hosted on Linode - they are [pre-configured in the app](https://github.com/simplex-chat/simplex-chat/blob/stable/src/Simplex/Chat/Options.hs#L42).
If you deployed your own SMP server(s) you can configure client via `-s` option:
```shell
$ simplex-chat -s smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@smp.example.com
```
Base64url encoded string preceding the server address is the server's offline certificate fingerprint which is validated by client during TLS handshake.
You can still talk to people using default or any other server - it only affects the location of the message queue when you initiate the connection (and the reply queue can be on another server, as set by the other party's client).
Run `simplex-chat -h` to see all available options.
### How to use SimpleX chat
Once you have started the chat, you will be prompted to specify your "display name" and an optional "full name" to create a local chat profile. Your display name is an alias for your contacts to refer to you by - it is not unique and does not serve as a global identity. If some of your contacts chose the same display name, the chat client adds a numeric suffix to their local display name.
The diagram below shows how to connect and message a contact:
<div align="center">
<img align="center" src="images/how-to-use-simplex.svg">
</div>
Once you've set up your local profile, enter `/c` (for `/connect`) to create a new connection and generate an invitation. Send this invitation to your contact via any other channel.
You are able to create multiple invitations by entering `/connect` multiple times and sending these invitations to the corresponding contacts you'd like to connect with.
The invitation can only be used once and even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established. See agent protocol for explanation of [invitation format](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md#connection-request).
The contact who received the invitation should enter `/c <invitation>` to accept the connection. This establishes the connection, and both parties are notified.
They would then use `@<name> <message>` commands to send messages. You may also just start typing a message to send it to the contact that was the last.
Use `/help` in chat to see the list of available commands.
### Groups
To create a group use `/g <group>`, then add contacts to it with `/a <group> <name>`. You can then send messages to the group by entering `#<group> <message>`. Use `/help groups` for other commands.
![simplex-chat](./images/groups.gif)
> **Please note**: the groups are not stored on any server, they are maintained as a list of members in the app database to whom the messages will be sent.
### Sending files
You can send a file to your contact with `/f @<contact> <file_path>` - the recipient will have to accept it before it is sent. Use `/help files` for other commands.
![simplex-chat](./images/files.gif)
You can send files to a group with `/f #<group> <file_path>`.
### User contact addresses
As an alternative to one-time invitation links, you can create a long-term address with `/ad` (for `/address`). The created address can then be shared via any channel, and used by other users as a link to make a contact request with `/c <user_contact_address>`.
You can accept or reject incoming requests with `/ac <name>` and `/rc <name>` commands.
User address is "long-term" in a sense that it is a multiple-use connection link - it can be used until it is deleted by the user, in which case all established connections would still remain active (unlike how it works with email, when changing the address results in people not being able to message you).
Use `/help address` for other commands.
![simplex-chat](./images/user-addresses.gif)
### Access chat history
SimpleX chat stores all your contacts and conversations in a local SQLite database, making it private and portable by design, owned and controlled by user.
You can view and search your chat history by querying your database. Run the below script to create message views in your database.
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/scripts/message_views.sql | sqlite3 ~/.simplex/simplex_v1_chat.db
```
Open SQLite Command Line Shell:
```sh
sqlite3 ~/.simplex/simplex_v1_chat.db
```
See [Message queries](./SQL.md) for examples.
> **Please note:** SQLite foreign key constraints are disabled by default, and must be **[enabled separately for each database connection](https://sqlite.org/foreignkeys.html#fk_enable)**. The latter can be achieved by running `PRAGMA foreign_keys = ON;` command on an open database connection. By running data altering queries without enabling foreign keys prior to that, you may risk putting your database in an inconsistent state.
**Convenience queries**
Get all messages from today (`chat_dt` is in UTC):
```sql
select * from all_messages_plain where date(chat_dt) > date('now', '-1 day') order by chat_dt;
```
Get overnight messages in the morning:
```sql
select * from all_messages_plain where chat_dt > datetime('now', '-15 hours') order by chat_dt;
```
docs/COMPARISON.md
+61
View File
@@ -0,0 +1,61 @@
# SimpleX platform - motivation and comparison
## Problems
Existing chat platforms and protocols have some or all of the following problems:
- Lack of privacy of the user profile and contacts (meta-data privacy).
- No protection (or only optional protection) of [E2EE][1] implementations from MITM attacks via provider.
- Unsolicited messages (spam and abuse).
- Lack of data ownership and protection.
- Complexity of usage for all non-centralized protocols to non-technical users.
The concentration of the communication in a small number of centralized platforms makes resolving these problems quite difficult.
## Proposed solution
Proposed stack of protocols solves these problems by making both messages and contacts stored only on client devices, reducing the role of the servers to simple message relays that only require authorization of messages sent to the queues, but do NOT require user authentication - not only the messages but also the metadata is protected becuse users do not have any identifiers assiged to them - unlike with any other platforms.
See [SimpleX whitepaper](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for more information on platform objectives and technical design.
## Comparison with other protocols
| | SimpleX chat | Signal, big platforms | XMPP, Matrix | P2P protocols |
| :--------------------------------------------- | :----------------: | :-------------------: | :-------------: | :-------------: |
| Requires user identifiers | No = private | Yes<sup>1</sup> | Yes<sup>2</sup> | Yes<sup>3</sup> |
| Possibility of MITM | No = secure | Yes<sup>4</sup> | Yes | Yes |
| Dependence on DNS | No = resilient | Yes | Yes | No |
| Single operator or network | No = decentralized | Yes | No | Yes<sup>5</sup> |
| Central component or other network-wide attack | No = resilient | Yes | Yes<sup>2</sup> | Yes<sup>6</sup> |
1. Usually based on a phone number, in some cases on usernames.
2. DNS based.
3. Public key or some other globally unique ID.
4. If operator’s servers are compromised.
5. While P2P networks and cryptocurrency-based networks are distributed, they are not decentralized - they operate as a single network, with a single namespace of user addresses.
6. P2P networks either have a central authority or the whole network can be compromised - see the next section.
## Comparison with [P2P][9] messaging protocols
There are several P2P chat/messaging protocols and implementations that aim to solve privacy and centralisation problem, but they have their own set of problems that makes them less reliable than the proposed design, more complex to implement and analyse and more vulnerable to attacks.
1. [P2P][9] networks use some variant of [DHT][10] to route messages/requests through the network. DHT implementations have complex designs that have to balance reliability, delivery guarantee and latency. The proposeddesign has both better delivery guarantees and lower latency (the message is passed multiple times in parallel, through one node each time, using servers chosen by the recipient, while in P2P networks the message is passed through `O(log N)` nodes sequentially, using nodes chosen by the algorithm).
2. The proposed design, unlike most P2P networks, has no global user identitifiers of any kind, even temporary.
3. P2P itself does not solve [MITM attack][2] problem, and most existing solutions do not use out-of-band messages for the initial key exchange. The proposed design uses out-of-band messages or, in some cases, pre-existing secure and trusted connections for the initial key exchange.
4. P2P implementations can be blocked by some Internet providers (like [BitTorrent][11]). The proposed design is transport agnostic - it can work over standard web protocols, and the servers can be deployed on the same domains as the websites.
5. All known P2P networks are likely to be vulnerable to [Sybil attack][12], because each node is discoverable, and the network operates as a whole. Known measures to reduce the probability of the Sybil attack either require a centralized component or expensive [proof of work][13]. The proposed design, on the opposite, has no server discoverability - servers are not connected, not known to each other and to all clients. The SimpleX network is fragmented and operates as multiple isolated connections. It makes network-wide attacks on SimpleX network impossible - even if some servers are compromised, other parts of the network can operate normally, and affected clients can switch to using other servers without losing contacts or messages.
6. P2P networks are likely to be vulnerable to [DRDoS attack][14]. In the proposed design clients only relay traffic from known trusted connection and cannot be used to reflect and amplify the traffic in the whole network.
[1]: https://en.wikipedia.org/wiki/End-to-end_encryption
[2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
[9]: https://en.wikipedia.org/wiki/Peer-to-peer
[10]: https://en.wikipedia.org/wiki/Distributed_hash_table
[11]: https://en.wikipedia.org/wiki/BitTorrent
[12]: https://en.wikipedia.org/wiki/Sybil_attack
[13]: https://en.wikipedia.org/wiki/Proof_of_work
[14]: https://www.usenix.org/conference/woot15/workshop-program/presentation/p2p-file-sharing-hell-exploiting-bittorrent
docs/SQL.md
+39
View File
@@ -0,0 +1,39 @@
# Accessing message history via SQL queries
You can run queries against `direct_messages`, `group_messages` and `all_messages` (or their simpler alternatives `direct_messages_plain`, `group_messages_plain` and `all_messages_plain`), for example:
```sql
-- you can put these or your preferred settings into ~/.sqliterc
-- to persist across sqlite3 client sessions
.mode column
.headers on
.nullvalue NULL
-- simple views into direct, group and all_messages
-- with user's messages deduplicated for group and all_messages;
-- only 'x.msg.new' ("new message") chat events - filters out service events;
-- msg_sent is 0 for received, 1 for sent
select * from direct_messages_plain;
select * from group_messages_plain;
select * from all_messages_plain;
-- query other details of your chat history with regular SQL, for example:
-- files you offered for sending
select * from direct_messages where msg_sent = 1 and chat_msg_event = 'x.file';
-- everything catherine sent related to cats
select * from direct_messages where msg_sent = 0 and contact = 'catherine' and msg_body like '%cats%';
-- all correspondence with alice in #team
select * from group_messages where group_name = 'team' and contact = 'alice';
-- aggregate your chat data
select contact_or_group, num_messages from (
select
contact as contact_or_group, count(1) as num_messages
from direct_messages_plain group by contact
union
select
group_name as contact_or_group, count(1) as num_messages
from group_messages_plain group by group_name
)
order by num_messages desc;
```
docs/protocol/types.ts
+74
View File
@@ -0,0 +1,74 @@
// x. namespace is for chat messages transmitted inside SMP agent MSG
type MemberMessageType =
| "x.grp.info" // group profile information or update
| "x.grp.off" // disable group
| "x.grp.del" // group deleted
| "x.grp.mem.new" // new group member
| "x.grp.mem.acl" // group member permissions (ACL)
| "x.grp.mem.leave" // group member left
| "x.grp.mem.off" // suspend group member
| "x.grp.mem.on" // enable group member
| "x.grp.mem.del" // group member removed
type ProfileMessageType =
| "x.info" // profile information or update
| "x.info.grp" // information about group in profile
| "x.info.con" // information about contact in profile
type NotificationMessageType = "x.msg.read"
type OpenConnMessageType =
| "x.open.grp" // open invitation to the group
| "x.open.con" // open invitation to the contact
type ContentMessageType =
| "x.msg.new" // new message
| "x.msg.append" // additional part of the message
| "x.msg.del" // delete message
| "x.msg.update" // update message
| "x.msg.fwd" // forward message
| "x.msg.reply" // reply to message
// TODO namespace for chat messages transmitted as other agent messages
type DirectMessageType =
| ProfileMessageType
| NotificationMessageType
| OpenConnMessageType
| ContentMessageType
type GroupMessageType = MemberMessageType | DirectMessageType
type ContentType =
| "c.text"
| "c.html"
| "c.image"
| "c.audio"
| "c.video"
| "c.doc"
| "c.sticker"
| "c.file"
| "c.link"
| "c.form"
| "c.poll"
| "c.applet"
// the type of message data transmitted inside SMP agent MSG
interface MessageData<T extends GroupMessageType> {
type: T
sent: Date
data: unknown
}
interface DirectMessageData<T extends DirectMessageType> extends MessageData<T> {}
interface GroupMessageData<T extends GroupMessageType> extends MessageData<T> {
msgId: number
parents: ParentMessage[]
}
interface ParentMessage {
memberId: Uint8Array
msgId: number
msgHash: Uint8Array
}
docs/rfcs/2022-01-26-mobile-app.md
+167
View File
@@ -0,0 +1,167 @@
# Porting SimpleX Chat to mobile
## Background and motivation
We have code that "works", the aim is to keep platform differences in the core minimal and get the apps to market faster.
### SimpleX platform design
See [overview](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for overall platform design and objectives, it is worth reading the introduction. The diagram copied from this doc:
```
User's Computer Internet Third-Party Server
------------------ | ---------------------- | -------------------------
| |
SimpleX Chat | |
| |
+----------------+ | |
| Chat App | | |
+----------------+ | |
| SimpleX Agent | | |
+----------------+ -------------- TLS ---------------- +----------------+
| SimpleX Client | ------ SimpleX Messaging Protocol ------> | SimpleX Server |
+----------------+ ----------------------------------- +----------------+
| |
```
- SimpleX Servers only pass messages, we don't need to touch that for the app
- SimpleX clients talk to the servers, we won't use them directly
- SimpleX agent is used from chat, we won't use it directly from the app
- Chat app will expose API to the app to communicate with everything, including DB and network.
### Important application modules
Modules of simplexmq package used from simplex-chat:
- a [functional API in Agent.hs]([Agent.hs](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent.hs#L38)) to send messages and commands
- TBQueue to receive messages and notifications (specifically, [subQ field of AgentClient record in Agent/Client.hs](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent/Client.hs#L72))
- [types from Agent/Protocol.hs](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent/Protocol.hs)).
This package has its [own sqlite database file](https://github.com/simplex-chat/simplexmq/tree/master/migrations) - as v1 was not backwards compatible migrations are restarted - where it stores all encryption and signing keys, shared secrets, servers and queue addresses - effectively it completely abstracts the network away from chat application, providing an API to manage logical duplex connections.
Simplex-chat library is what we will use from the app:
- command type [ChatCommand in Chat.hs](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat.hs#L72) that UI can send to it
- UI sends these commands via TBQueue that `inputSubscriber` reads in forever loop and sends to `processChatCommand`. There is a hack that `inputSubscriber` not only reads commands but also shows them in the view, depending on the commands.
- collection of [view functions in Chat/View.hs](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat/View.hs) to reflect all events in view.
This package also creates its own [database file](https://github.com/simplex-chat/simplex-chat/tree/master/migrations) where it stores references to agent connections managed by the agent, and how they map to contacts, groups, and file transmissions.
## App design options and questions
### Sending chat commands from UI and receiving them in Haskell
Possible options:
- function (exported via FFI) that receives strings from UI and decodes them into ChatCommand type, then sending this command to `processChatCommand`. This option requires a single function in C header file, but also requires encoding in UI and decoding in Haskell.
- multiple functions exported via FFI each sending different command to `processChatCommand`. This option requires multiple functions in header file and multiple exports from Haskell.
Overall, the second option seems a bit simpler and cleaner, if we agree to go this route we will refactor `processChatCommand` to expose its parts that process different commands as independent functions.
On another hand, it might be easier to grow chat API if this is passed via a single function and serialized as strings (e.g. as JSON, to have it more universal) - it would also might give us an API for a possible future chat server that works with thin, UI-only clients.
In both cases, we should split `processChatCommand` (or the functions it calls) into a separate module, so it does not have code that is not used from the app.
**Proposal**
Use option 2 to send commands from UI to chat, encoding/decoding commands as strings with a tag in the beginning (TBC binary, text or JSON based - encoding will have to be replicated in UI land; both encoding and decoding is needed in Haskell land to refactor terminal chat to use this layer as well, so we have a standard API for all implementations).
This function would have this type:
```haskell
sendRequest :: CString -> IO CString
```
to allow instant responses.
One more idea. This function could be made to match REST semantics that would simplify making chat into a REST chat server api:
```haskell
sendRequest :: CString -> CString -> CString -> CString -> IO CString
sendRequest verb path qs body = pure ""
```
### Sending messages and notifications from Haskell to UI
Firstly, we have to refactor the existing code so that all functions in [View.hs](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat/View.hs) are passed to `processChatCommand` (or the functions for each command, if we go with this approach) as a single record containing all view functions.
The current code from View.hs will not be used in the mobile app, it is terminal specific; we will create a separate connector to the UI that has the same functions in a record - these functions communicate to the UI.
Again, there are two similar options how this communication can happen:
- UIs would export multiple functions however each platform allows it, as C exports, and they would be all imported in Haskell. This option feels definitely worse, as it would have to be maintained in both iOS and Android separately for exports, and in Haskell for imports, resulting in lots of boilerplate.
- UIs would export one function that receives strings (e.g. JSON encoded) with the messages and notifications, there will be one function in Haskell to send these JSON. All required view functions in Haskell land would simply send different strings into the same function.
In this case the second option seems definitely easier, as even with simple terminal UI there are more view events than chat commands (although, given different mobile UI paradigms some of these events may not be needed, but some additional events are likely to be addedd, that would be doing nothing for terminal app).
**Proposal**
Encode messages and notifications as JSON, but instead of exporting the function from UI (which would have to be done differently from different platforms), have Haskell export function `receiveMessage` that would be blocking until the next notification or message is available. UI would handle it in a simple loop, on a separate thread:
```haskell
-- CString is serialized JSON (ToJSON serialized datatype from haskell)
receiveMessage :: IO CString ()
```
To convert between Haskell and C interface:
```haskell
type CJSON = CString
toCJSON ToJSON a => a -> CJSON
toCJSON = ...
-- Haskell interface
send :: ToJSON a => String -> IO a
recv :: ToJSON a => IO a
-- C interface
c_send :: CString -> IO CJSON
c_recv :: IO CJSON
```
### Accessing chat database from the UI
Unlike terminal UI that does not provide any capabilities to access chat history, mobile UI needs to have access to it.
Two options how it can be done:
- UI accesses database directly via its own database library. The upside of this approach is that it keeps Haskel core smaller. The downside is that sqlite is relatively bad with concurrent access. In Haskell code we allowed some concurrency initially, having the pool limited to few concurrent connection, but later we removed concurrency (by limiting pool size to 1), as otherwise it required retrying to get transaction locks with difficult to set retry time limits, and leading to deadlocks in some cases. Also mobile sqlite seems to be compiled with concurrency disabled, so we would have to ship app with our own sqlite (which we might have to do anyway, for the sake of full text search support). We could use some shared semaphore in Haskell to obtain database lock, but it adds extra complexity...
- UI accesses database via Haskell functions. The upside of this is that there would be no issues with concurrency, and chat schema would be "owned" by Haskell core, but it requires either a separate serializable protocol for database access or multiple exported functions (same two options as before).
However bad the second option is, it seems slightly better as at least we would not have to duplicate sql quiries in iOS and Android. But this is the trade-off I am least certain of...
**Proposal**
Use the same `sendRequest` function to access database.
Additional idea: as these calls should never mutate chat database, they should only query the state, and as these functions will not be needed for terminal UI, I think we could export it as a separate function and have all necessary queries/functions in a separate module, e.g.:
```haskell
-- params and result are JSON encoded
chatQuery :: CString -> IO CString
chatQuery params = pure ""
```
On another hand, if we go with REST-like `sendRequest` then it definitely should be the only function to access chat and database state.
### UI database
UI needs to have its own storage to store information about user settings in the app and, possibly, which chat profiles the user has (each would have its own chat/agent databases).
### Chat database initialization
Currently it is done in an ad hoc way, during the application start ([`getCreateActiveUser` function](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat.hs#L1178)), we could either expose this function to accept database name or just check on the start and initialize database with the default name in case it is not present.
### Multiple profiles in the app
All user profiles are stored in the same database. The current schema allows multiple profiles, but the current UI does not. We do not need to do it in the app MVP.
## Notifications
We don't need it in the first version - it is out of scope of releasable MVP - but we need to think a bit ahead how it will be done so it doesn't invalidate the design we settle on.
There is no reliable background execution, so the only way to receive messages when the app is off is via notifications. We have added notification subscriptions to the low protocol layer so that Haskell core would receive function call when notification arrives to the native part and receive and process messages and communicate back to the local part that would show a local notification on the device:
```
Push notification -> Native -> Haskell ... process ... -> Native -> Local notification
```
Notifications are the main reason why we will need to store multiple profiles in the same database file - when notification arrives we do not know which profile it is for, it only has server address and queue ID, and if different profiles were in different databases we would either had to have a single table mapping queues to profiles or lookup multiple databases - both options seem worse than a single database with multiple profiles.
For the rest we would just use the same approaches we would use for UI/Haskell communications - probably a separate functions to receive notifications to Haskell, and the same events to be sent back.
docs/rfcs/2022-02-10-deduplicate-contact-requests.md
+19
View File
@@ -0,0 +1,19 @@
# Deduplicate contact requests
1. add nullable fields `via_contact_uri_hash` and `xcontact_id` to `connections`
2. when joining (Connect -> SCMContact)
- generate and save random `xcontact_id`
- save hash of `AConnectionRequestUri` when joining via contact uri
(AConnectionRequestUri -> ConnectionRequestUri -> CRContactUri)
- send random identifier in `XContact` as `Maybe XContactId`
- check for repeat join - if connection with such `via_contact_uri_hash` has contact notify user
- check for repeat join - check in connections if such contact uri exists, if yes use same identifier; the rest of request can (should) be regenerated, e.g. new server, profile
can be required
3. add nullable field `xcontact_id` to `contact_requests` and to `contacts` (* for auto-acceptance)
4. on contact request (processUserContactRequest)
- save identifier
- \* check if `xcontact_id` is in `contacts` - then notify this contact already exists
- when saving check if contact request with such identifier exists, if yes update `contact_request`
(`invId`, new profile)
- ? remove old invitation - probably not necessarily, to be done in scope of connection expiration
- return from Store whether request is new or updated (Bool?), new chat response for update or same response
docs/rfcs/2022-02-24-servers-configuration.md
+29
View File
@@ -0,0 +1,29 @@
# Server configuration
- in agent:
- Agent.Env.SQLite - move smpServers from AgentConfig to Env, make it TVar; keep "initialSmpServers" in AgentConfig?
- Agent - getSMPServer to read servers from Env and choose a random server
- Agent - new functional api - "useServers"
- ~~Agent.Protocol - new ACommand?~~
- chat core:
- db:
- new table `smp_servers`, server per row, same columns as for agent. Have rowid for future
- getServers method
- update - truncate and rewrite
- ChatCommand GetServers - new ChatResponse with list of user SMPServers, it may be empty if default are used
- ChatCommand SetServers - ChatResponse Ok (restore default servers is empty set servers list)
- agent config is populated using getServers, if it's empty default are used
- mobile chat:
- mobileChatOpts to be populated with initial servers on init (getServers or default if empty)
- in ui:
- view in settings
- GetServers on view open to populate
- Confirm buttons, Restore button - destructive - clears user servers and default are used
- validation
- validation on submit, error with server's string
- ~~TBD real-time validation~~
- ~~fastest is validation on submit without detailed error?~~
- ~~maybe even faster - alternatively have 3 fields for entry per server - fingerprint, host, port - and build server strings (still validate to avoid hard crash?)?~~
- terminal chat:
- if -s option is given, these servers are used and getServers is not used for populating agentConfig
- if -s option is not provided - same as in mobile - getServers or default if empty
docs/rfcs/2022-03-02-avatars.md
+14
View File
@@ -0,0 +1,14 @@
# Include (Optional) Images in User Profiles
1. Add SQL migration for database in `src/Simplex/Chat/Migrations`
- This will touch `contact_profiles` and `group_profiles`
2. Add field to `User` in `Types.hs` allowing for null entry using `Maybe`
3. Extend parsing in `Chat.hs` under `chatCommandP :: Parser ChatCommand`
4. Update `UpdateProfile` in `Chat.hs` to accept possible display picture and implement an `APIUpdateProfile` command which accepts a JSON string `/_profile{...}` which will add the image to a profile.
5. Connect up to Android and iOS apps (new PRs)
Profile images will be base 64 encoded images. We can use the `base64P` parser to process them and pass them as JSON.
docs/rfcs/2022-03-02-number-chat-items.md
+61
View File
@@ -0,0 +1,61 @@
# Message replies and chat item sequential numbers
## Problem
Many chat features require referring to the previous chat items in the same conversation:
- item editing
- item deletion
- item reply (with quoting)
- delivery/read receipts
- any interactive features mutating chat item state
- group message integrity via DAG
The most in-demand feature is replies.
## Proposed solution
As group message integrity is needed not for chat items, but for messages, the updated proposal is to introduce a random, non-sequential message id, unique per conversation and per sender.
All above features would rely on this ID, e.g. reply would use the ID of the message that created the item.
We will add an optional property `msgId` into all chat messages (not only visible to the users) and `msgRef` into messages that need to reference other messages.
`msgId` property is a base64 encoded 12 byte binary
JTD for quoting messages:
```yaml
definitions:
msgRef:
discriminator: type
mapping:
direct:
properties:
msgId: type: string
sentAt: type: datetime
sent: type: boolean # true if it is in reference to the item that the sender of the message originally sent, false for references to received items
group:
properties:
msgId: type: string
sentAt: type: datetime
memberId: type: string # base64 member ID of the sender known to all group members for group chats
content:
properties:
type: type: string
text: type: string
properties:
msgId: string
event: enum: ["x.msg.new"]
params:
properties:
content: ref: content
quote:
properties:
content: ref: content
msgRef: ref: msgRef
```
This format ensures that replies with quoting show as normal messages on the clients that do not support showing quotes (`quote` property will be ignored).
The only feature that would not work in case chatItem/chatItemRef is missing is navigating to the message to which the message is in reply to.