# SMP agent groups

## Problems

- device/user profile synchronisation
- group communication

Both use cases can be facilitated by message broadcasts between a group of SMP agents.

## Solution: symmetric groups as part of SMP agent protocol

The proposed approach does not scale to large groups, as each agent has to broadcast the messages of their clients to all other agents in the group. While for large group it is more effective to have a server managing the list of group members, it introduces the challenges with key distribution, privacy etc.

This proposal contains the set of additional SMP agent commands and message envelopes to provide a low level abstraction for group communication.

The groups are symmetric, all agents who are members of the group have equal rights and can add and remove members and leave group. Higher level protocol can manage the permissions of the different users, user and group profiles, using reserved fields for passing arbitrary information about the groups and the members.

All the information about the groups is stored only in the agents.

## Group message integrity

Two approaches are possible:

1. Each agent on a regular interval sends to all agents the sender IDs and the digests of the last messages from all agents they communicate with.
To avoid `(n-1)*n` messages for each group integrity verification, IDs and digests from all senders can be compacted in one messages:

```
broadcast: %s"CHECK" SP memId1 ":" msgId1 ":" hash1 ":" status1 SP memId2 ":" msgId2 ":" hash2 ":" status2
```

A side question is that we currently do not support large agent messages; possibly we should support messages larger than SMP block size to simplify this and other scenarios, similarly to how websockets protocol does it. There might still be a limit to how large the full message can be. That probably requires re-thinking of how messages are managed and separation of message reception from the servers and message delivery to the clients, but this is likely to be required anyway for when we start running the agent in the background.

There may be two situations to consider:

1) The recipient of this verification message can have the same message as the last one or they can have more messages received - both such scenarios are ok and do not necessarily indicate a problem, as they just might be slightly ahead in receiving the messages. They would make the last matching message as ok, and the latter messages would remain unknown - it does not indicate the lack of integrity of the group, although if the next integrity check without the same messages does (this needs to be clarified).
2) The recipient of this verification message can have fewer messages received. This situation can be resolved in several ways:
- try to retrieve all messages from the queue that is behind. It might work, but it may be that the sender is simply trying to send messages because the network is down.
- wait until the next integrity check and only report integrity violation if during the next integrity check they still cannot reconcile the previous integrity check. This is probably an acceptable compromise.

2. Each agent sends message receipts to sending agents - message receipts would contain a signature of the message hashes. In the agent uses the same verification key for each member of the group, as considered below, these receipts can be re-broadcasted to other agents (again, grouping them in one message to avoid `(n-1)*n` messages) as a proof that the messages were delivered.

Comparing with the first approach, there are pros and cons:
- pros:
  - the sender would only send such integrity check messages when they have send a message to all parties, thus avoiding the situation when some messages might have been not yet sent (or failed to send and they are retrying).
- cons:
  - integrity check message mush contain a signature per member, so it would be substantially larger.
  - agents must use the same verification key for all members in the group, complicating the group and connections management.
  - in the same way as in the first scenario, some delivery notifications can be arbitrarily delayed.

With either approach, each agent should probably track all the receipts of all messages from all agents. 

Overall the first approach seems better. It shows who received which messages and the only case where the lack of integrity would be reported if the message with the same order number is different or some messages are skipped to some recipients (i.e. they broadcast integrity violation for some of the senders).

## Agent commands and messages syntax

- command `G:gId? NEW` - create group (response is `G:gId OK`, or `ERR` if this group already exists)
- command `C:cId INTRO G:gId gInfo` - invite existing connection to a group
- message `C:cId REQ G:invID gInfo` - invitation to join the group
- command `G:gId? ACPT G:invId` - accept invitation (response is `G:gId OK`)
- message `G:gId CON C:cId` - 2 connections created with some group member (both for group and direct messages)
- message `G:gId MEM [C:cId]` - connection created with all group members for a given member or current client
- command `G:gId SEND msg` - send message to group
- message `G:gId SENT [C:cId] msgId` - notification that the message is sent and its internal ID
- message `G:gId MSG C:cId msgdata` - received group message from cId, msgdata is the same set of parameters as in `MSG`
- command `G:gId ACK msgId` - acknowledge message reception by the client
- message `G:gId RCVD [C:cId] msgId status` - message delivery notification
- command `G:gId LEAVE` - leave the group
- message `G:gId LEFT [C:cId]` - connection cId left the group
- command `G:gId REM C:cId` - remove group member (response is `G:gId OK`, followed by `REMD` notification)
- message `G:gId REMD C:cId [C:cId]` - member removed (who, by whom - if it's not the current user)
- message `G:gId OUT C:cId` - you are removed
- message `G:gId EMPTY` - all members left the group and it is now empty
- command `G:gId DEL` - delete the group (response is `G:gId OK`)
- message `G:gId DELD [C:cId]` - group deleted (by whom, if it's not the current user)
- command `G:bId LS` - list connections in the group, response is `MS space_separated_connections`
- message `G:bId MS space_separated_connections`

## Agent message envelopes syntax (group-specific)

- `GROUP C:mid G:inv gInfo` - invitation to join the group
- `MEM C:mid` - confirmation that member connected to all members
- `LEFT` - notification that member left the group
- `OUT` - you are removed from the group
- `REM C:mid` - remove member mid from the group
- `REMD C:mid` - confirmation that member is removed
- `DEL` - group is deleted
- `DELD` - confirmation that group is deleted

## Protocol costs

- Adding a member:
  - GROUP - `1 + connection_cost`
  - [introductions](./2021-05-23-introduction.md) - `(5 + 2 * connection_cost) * (n - 1)`
  - MEM - `n - 1`
  - total - `6n - 5 + (2n - 1) * connection_cost` agent messages, where connection cost is 5 messages (2 * HELLO, 2 * MSG - confirmation, 1 * REPLY ) agent messages, so the total is `16n - 10`.

- Sending a message:
  - MSG - (n-1)
  - RCVD - (n-1)
  - total - `2n-2` messages

- Member leaves a group:
  - LEFT - (n-1)
  - total -`n-1` messages

- Member is removed by another member:
  - OUT - 1
  - REM - (n-1)
  - REMD - (n-1)
  - total `2n-1` messages

- Group is deleted:
  - DEL - (n-1)
  - DELD - (n-1)
  - total - `2n-2` messages

- Group integrity verification:
  - TODO

## Questions

1. Message verification keys. Agents use separate server and encryption key for each connection, but there can be a value of having the same message verification key used for all members in the group. E.g., a member can validate to other group members that the message was delivered to all group members by sending signed message receipts they receive from the agent (the second approach to group integrity verification above). This would mean that for each message `n*(n-1)` messages will be send, although signed receipts can be grouped to reduce this number. It can be done periodically, rather than on each message, as described [here](https://signal.org/blog/private-groups/).

The [sequence digram for group operations](https://mermaid-js.github.io/mermaid-live-editor/#[encoded data, 6346 characters]), the source is [here](./groups.mmd).

![sequence digram for group operations](https://mermaid.[encoded data, 6348 characters])

## Relevant external documents

[Signal group protocol](https://signal.org/blog/private-groups/)

[mpOTR](https://cypherpunks.ca/~iang/pubs/mpotr.pdf)

[Threema whitepaper](https://threema.ch/press-files/cryptography_whitepaper.pdf)
