Commit Graph

25829 Commits

Author SHA1 Message Date
Jason Robinson 21d08a060c Optimize calculating left room id's for incremental sync 2026-06-05 19:37:05 +03:00
Jason Robinson e90096ef57 Use a "per user profile updates" tracking table
This makes writes heavier when profile updates happen, but reduces the effort to produce an incremental sync response by not needing to look whether users share rooms.
2026-06-05 19:17:12 +03:00
Jason Robinson 877074f38e Lint tests/handlers/test_sync.py 2026-06-05 15:30:23 +03:00
Jason Robinson d02150fd84 Remove code that accidentally came here from https://github.com/element-hq/synapse/pull/19542/changes#r2913827732 and was then removed in https://github.com/element-hq/synapse/pull/19398 2026-06-05 15:19:47 +03:00
Jason Robinson 28587d7907 Fix the lazy loading test cases 2026-06-05 13:49:27 +03:00
Jason Robinson e6c25e4930 Add profile_updates_sequence sequence to synapse_port_db.py 2026-06-04 22:29:05 +03:00
Jason Robinson 2d9f943034 Collect lazy loaded members for profile updates in sync response from events 2026-06-04 22:04:41 +03:00
Jason Robinson 301e9c1ba7 ProfileUpdate.action should never be None 2026-06-04 13:10:29 +03:00
Jason Robinson 36f0f64f62 Add "action" to profile updates stream
This allows distinguishing between "a field was updated" or "something else happened". In this case, we want to know if the user left a room. Seeing this action while building the sync response we can then check if the user is still a member of the same rooms as the syncing user, and if not, we can tell the client to remove their profile.
2026-06-04 11:26:50 +03:00
Jason Robinson b61f748928 Fix some broken tests due to sync token changes 2026-06-03 18:08:36 +03:00
Jason Robinson bd40add5d5 Add some tests for profile handler updating profile_updates stream table and waking up notifiers 2026-06-03 13:55:22 +03:00
Jason Robinson 1a59c0ae0f Only return profiles for initial sync that have events when using lazy loading 2026-06-02 21:38:16 +03:00
Jason Robinson 1ae91c87d0 Fix get_profile_data_for_users if user does not have custom fields 2026-06-02 21:36:00 +03:00
Jason Robinson e236ddb32b Add SyncProfileUpdatesTestCase
For initial sync
2026-06-02 21:35:34 +03:00
Jason Robinson 391fd2b62a Merge branch 'develop' into anoa/msc4429 2026-06-01 16:22:20 +03:00
Jason Robinson d254167e19 Clarify in _generate_sync_entry_for_profile_updates that only local profiles will be returned 2026-06-01 16:04:21 +03:00
Jason Robinson 3cf7555f84 Clarify profile updates sync response with a comment 2026-06-01 16:01:30 +03:00
dependabot[bot] 6c3ba2205b Bump idna from 3.11 to 3.15 (#19790)
Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.15.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's
changelog</a>.</em></p>
<blockquote>
<h2>3.15 (2026-05-12)</h2>
<ul>
<li>Enforce DNS-length cap on individual labels early in
<code>check_label</code>,
short-circuiting contextual-rule processing for oversized input
while staying compatible with UTS 46 usage.</li>
<li>Tidy core helpers: hoist bidi category sets to module-level
frozensets (avoiding per-codepoint list construction), simplify
length checks, and reuse the shared <code>_unicode_dots_re</code> from
<code>idna.core</code> in the codec module.</li>
<li>Use <code>raise ... from err</code> for proper exception chaining
and
switch internal string formatting to f-strings.</li>
<li>Allow <code>flit_core</code> 4.x in the build backend.</li>
<li>Expand the ruff lint set (flake8-bugbear, flake8-simplify,
pyupgrade, perflint) and apply the surfaced fixes; pin lint CI
to Python 3.14.</li>
<li>Add Dependabot configuration for GitHub Actions.</li>
<li>Convert README and HISTORY from reStructuredText to Markdown.</li>
<li>Reference CVE-2026-45409 for the 3.14 advisory in place of the
initial GHSA identifier.</li>
</ul>
<p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for
contributions to this release.</p>
<h2>3.14 (2026-05-10)</h2>
<ul>
<li>Removed opportunity to process long inputs into quadratic
time by rejecting oversize inputs up-front. Closes a bypass
of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li>
</ul>
<p>Thanks to Stan Ulbrych for reporting the issue.</p>
<h2>3.13 (2026-04-22)</h2>
<ul>
<li>Correct classification error for codepoint U+A7F1</li>
</ul>
<h2>3.12 (2026-04-21)</h2>
<ul>
<li>Update to Unicode 17.0.0.</li>
<li>Issue a deprecation warning for the transitional argument.</li>
<li>Added lazy-loading to provide some performance improvements.</li>
<li>Removed vestiges of code related to Python 2 support, including
segmentation of data structures specific to Jython.</li>
</ul>
<p>Thanks to Rodrigo Nogueira for contributions to this release.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a>
Release 3.15</li>
<li><a
href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a>
Pre-release 3.15rc0</li>
<li><a
href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from
kjd/convert-docs-to-markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a>
Convert README and HISTORY from reStructuredText to Markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from
kjd/dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a>
Merge branch 'master' into
dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from
StanFromIreland/patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a>
Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions
group</li>
<li><a
href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a>
Merge branch 'master' into patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from
StanFromIreland/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/kjd/idna/compare/v3.11...v3.15">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=idna&package-manager=pip&previous-version=3.11&new-version=3.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/element-hq/synapse/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-01 14:45:41 +02:00
dependabot[bot] 71e07d4c75 Bump hashicorp/vault-action from 3.4.0 to 4.0.0 (#19804)
Bumps
[hashicorp/vault-action](https://github.com/hashicorp/vault-action) from
3.4.0 to 4.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/vault-action/releases">hashicorp/vault-action's
releases</a>.</em></p>
<blockquote>
<h2>v4.0.0</h2>
<h2>4.0.0 (May 12, 2026)</h2>
<p>Improvements:</p>
<ul>
<li>Bump node runtime from node20 to node24 <a
href="https://redirect.github.com/hashicorp/vault-action/pull/604">GH-604</a></li>
<li>Fix leading slash in secret paths causing HTTP 400 errors (e.g.
<code>/cubbyhole/test</code> → <code>v1/cubbyhole/test</code> instead of
<code>v1//cubbyhole/test</code>)</li>
<li>bump jsrsasign from 11.1.0 to 11.1.3</li>
<li>bump body-parser from 1.20.3 to 1.20.5</li>
<li>bump qs from 6.13.0 to 6.15.1</li>
<li>bump http-errors from 2.0.0 to 2.0.1</li>
<li>bump minimatch from 3.1.2 to 3.1.5</li>
<li>bump underscore from 1.13.4 to 1.13.8</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/vault-action/blob/main/CHANGELOG.md">hashicorp/vault-action's
changelog</a>.</em></p>
<blockquote>
<h2>4.0.0 (May 12, 2026)</h2>
<p>Improvements:</p>
<ul>
<li>Bump node runtime from node20 to node24 <a
href="https://redirect.github.com/hashicorp/vault-action/pull/604">GH-604</a></li>
<li>Fix leading slash in secret paths causing HTTP 400 errors (e.g.
<code>/cubbyhole/test</code> → <code>v1/cubbyhole/test</code> instead of
<code>v1//cubbyhole/test</code>)</li>
<li>bump jsrsasign from 11.1.0 to 11.1.3</li>
<li>bump body-parser from 1.20.3 to 1.20.5</li>
<li>bump qs from 6.13.0 to 6.15.1</li>
<li>bump http-errors from 2.0.0 to 2.0.1</li>
<li>bump minimatch from 3.1.2 to 3.1.5</li>
<li>bump underscore from 1.13.4 to 1.13.8</li>
</ul>
<h2>3.4.0 (June 13, 2025)</h2>
<p>Bugs:</p>
<ul>
<li>replace all dot chars during normalization (<a
href="https://redirect.github.com/hashicorp/vault-action/pull/580">hashicorp/vault-action#580</a>)</li>
</ul>
<p>Improvements:</p>
<ul>
<li>Prevent possible DoS via polynomial regex (<a
href="https://redirect.github.com/hashicorp/vault-action/pull/583">hashicorp/vault-action#583</a>)</li>
</ul>
<h2>3.3.0 (March 3, 2025)</h2>
<p>Features:</p>
<ul>
<li>Wildcard secret imports can use <code>**</code> to retain case of
exported env keys <a
href="https://redirect.github.com/hashicorp/vault-action/pull/545">GH-545</a></li>
</ul>
<h2>3.2.0 (March 3, 2025)</h2>
<p>Improvements:</p>
<ul>
<li>Add retry for jwt auth login to fix intermittent login failures <a
href="https://redirect.github.com/hashicorp/vault-action/pull/574">GH-574</a></li>
</ul>
<h2>3.1.0 (January 9, 2025)</h2>
<p>Improvements:</p>
<ul>
<li>fix wildcard handling when field contains dot <a
href="https://redirect.github.com/hashicorp/vault-action/pull/542">GH-542</a></li>
<li>bump body-parser from 1.20.0 to 1.20.3</li>
<li>bump braces from 3.0.2 to 3.0.3</li>
<li>bump cross-spawn from 7.0.3 to 7.0.6</li>
<li>bump micromatch from 4.0.5 to 4.0.8</li>
</ul>
<p>Features:</p>
<ul>
<li><code>secretId</code> is no longer required for approle to support
advanced use cases like machine login when <code>bind_secret_id</code>
is false. <a
href="https://redirect.github.com/hashicorp/vault-action/pull/522">GH-522</a></li>
<li>Use <code>pki</code> configuration to generate certificates from
Vault <a
href="https://redirect.github.com/hashicorp/vault-action/pull/564">GH-564</a></li>
</ul>
<h2>3.0.0 (February 15, 2024)</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/hashicorp/vault-action/commit/892a26828f195e65540a40b4768ae4571f51ebfc"><code>892a268</code></a>
Update copywrite headers for v.4.0.0 release (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/607">#607</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/a7ffa26e2e6ede175ca2e4f7dec54e78425d6936"><code>a7ffa26</code></a>
Prepare for release v4.0.0 (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/606">#606</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/a049f0183861f1dbbd996f64b48335487cc968db"><code>a049f01</code></a>
[COMPLIANCE] Add/Update Copyright Headers (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/605">#605</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/95977a3e2387e93244aaae1232de66fc47b379a3"><code>95977a3</code></a>
Adding team-vault-consumption as CODEOWNERS (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/600">#600</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/7e48e563b6a9b4b0ba8b028c5ee89c41a8ae2671"><code>7e48e56</code></a>
Upgrade Node.js to 24 and update dependencies (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/604">#604</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/79632e33d6953d190b940ffa440bf97821cabd80"><code>79632e3</code></a>
[COMPLIANCE] Add Copyright and License Headers (Batch 1 of 1) (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/589">#589</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/734c523c4fbdb289cdf26dd2dc177f3627d1e140"><code>734c523</code></a>
README.md: Removing jwtGithubAudience default (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/590">#590</a>)</li>
<li><a
href="https://github.com/hashicorp/vault-action/commit/2c5827061f1ad91ca97897d6257ebe638e033699"><code>2c58270</code></a>
[Compliance] - PR Template Changes Required (<a
href="https://redirect.github.com/hashicorp/vault-action/issues/586">#586</a>)</li>
<li>See full diff in <a
href="https://github.com/hashicorp/vault-action/compare/4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b...892a26828f195e65540a40b4768ae4571f51ebfc">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=hashicorp/vault-action&package-manager=github_actions&previous-version=3.4.0&new-version=4.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-01 14:38:08 +02:00
Jason Robinson 54c299d091 Clarify newsfile that this pr is for local users only 2026-06-01 15:24:16 +03:00
Jason Robinson aeeb9336a6 Restrict initial sync profiles to local users for now
We don't have federated profile updates (yet), so use a cheaper database call to fetch only local users for the initial sync.

This mirrors incremental sync as currently there are only local profiles being pushed into the profile updates stream table.
2026-06-01 15:22:01 +03:00
Richard van der Hoff 2aef6c33a8 Add logging to help diagnose missing to-device messages (#19801)
Some attempts to debug
https://github.com/element-hq/synapse/issues/19795.

---------

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Eric Eastwood <erice@element.io>
2026-06-01 11:34:12 +01:00
Jason Robinson c082da4033 Allow a msc4429_enabled config alias outside of the experimental config 2026-06-01 12:35:16 +03:00
Jason Robinson 4cf5e098a5 Clarify _generate_initial_sync_entry_for_profile_updates is for local users only currently 2026-06-01 12:25:35 +03:00
Jason Robinson 4c2bbbce20 Use a "many" insert for add_profile_updates 2026-06-01 12:25:31 +03:00
Jason Robinson 3444aef480 Improve docstrings of get_updated_profile_updates and get_profile_updates_for_fields 2026-06-01 11:57:15 +03:00
Jason Robinson 4697e73880 Simplify collecting profile field updates for sync
Change `get_profile_data_for_users` to compile a friendly dictionary, so we don't need to special if/else case when collecting data for sync. The sync methods just treat every field identically, without displayname or avatar_url having any specific importance.
2026-05-29 15:02:16 +03:00
shcherbak 306d8b23bd Feat/gcp json formatter (#19775)
Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
2026-05-29 10:57:33 +00:00
Jason Robinson 92bfed7c18 Don't return null's in initial sync for profiles with no value for the asked field 2026-05-29 12:40:49 +03:00
Jason Robinson 632c7ea42c Add a docstring, remove premature optimization 2026-05-29 12:06:15 +03:00
dependabot[bot] c45096e7e8 Bump sigstore/cosign-installer from 4.1.1 to 4.1.2 in the minor-and-patches group (#19803)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-29 09:00:39 +00:00
Jason Robinson 56a3647b8f Remove unnecessary upgrade notes for profile updates stream writer 2026-05-29 11:58:07 +03:00
Jason Robinson 352efe18c9 Fix replication of profile updates to the right stream writer 2026-05-29 11:55:03 +03:00
Jason Robinson c30e0d9103 Also replicate DELETE profile fields to the right stream worker 2026-05-28 22:26:58 +03:00
Jason Robinson e4c35cef26 Add missing value to docstring 2026-05-28 22:13:54 +03:00
Jason Robinson df1b587332 Add ReplicationProfileSetFieldValue endpoint
If we get PUT for profile fields on an instance that isn't the profile updates stream writer, we'll route the request to be finished on the correct stream writer via http replication.
2026-05-28 22:10:06 +03:00
Tulir Asokan a6b8a19ac7 Add workaround for missing events in _should_perform_remote_join (#19730) 2026-05-28 17:44:17 +00:00
dependabot[bot] 0a6aed8df2 Bump rand from 0.9.2 to 0.9.4 (#19687)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-28 17:35:45 +00:00
dependabot[bot] 65ddc7118a Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.34.0 to 1.43.0 in /complement (#19673)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-28 17:33:26 +00:00
Jason Robinson 6ec8ee2a5e Mount ProfileFieldRestServlet on all Synapse instances
But only allow PUT/DELETE from the `profile_updates` worker (which defaults to being `main`).
2026-05-28 12:52:25 +03:00
Jason Robinson 2fa20e0fee Only support unstable prefix for profile_fields in API filters 2026-05-27 16:36:47 +03:00
Olivier 'reivilibre f9f08db4c3 Changelog tweaks v1.154.0rc1 2026-05-27 14:27:37 +01:00
Jason Robinson fcbceee2fc Don't return anything for users without profiles in _generate_initial_sync_entry_for_profile_updates 2026-05-27 16:25:51 +03:00
Jason Robinson 7e964a9206 user_id: UserID in ProfileUpdatesStreamRow
Also add docstrings for the fields.
2026-05-27 16:21:48 +03:00
Jason Robinson 4e820ca969 Multiline string for _get_updated_profile_updates_txn 2026-05-27 16:18:41 +03:00
Jason Robinson c02e6a6181 Only pass field names to add_profile_updates 2026-05-27 16:14:47 +03:00
Jason Robinson 386958cbda Add a timestamp column to profile_updates
Also add some comments to some fields.
2026-05-27 15:57:48 +03:00
Jason Robinson 343d4dffe0 Drop profile_updates_fk_users foreign key
The profile updates table should also allow supporting federated user profile updates.
2026-05-27 15:39:58 +03:00
Olivier 'reivilibre 82d66fa0f1 1.154.0rc1 2026-05-27 12:27:43 +01:00
Tulir Asokan 0e39c0c8f6 Fix policy server signature merging again (#19797)
Fixes #19796
2026-05-26 14:12:20 -05:00