dandri/synapse
1
0 Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2026-10-22 21:54:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

deploy: 615d96ad6e

Browse Source
This commit is contained in:
H-Shay committed 2022-05-09 21:43:32 +00:00
1 parent 3de68fe69c
commit ba63260d28
4 files changed
+192 -120
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

develop/print.html
+95 -59
View File
@@ -15911,40 +15911,23 @@ more CPU and RAM, and make use of <a href="usage/administration/../../workers.ht
to make use of multiple CPU cores / multiple machines for your homeserver.</p>
<div style="break-before: page; page-break-before: always;"></div><h2 id="some-useful-sql-queries-for-synapse-admins"><a class="header" href="#some-useful-sql-queries-for-synapse-admins">Some useful SQL queries for Synapse Admins</a></h2>
<h2 id="size-of-full-matrix-db"><a class="header" href="#size-of-full-matrix-db">Size of full matrix db</a></h2>
<p><code>SELECT pg_size_pretty( pg_database_size( 'matrix' ) );</code></p>
<pre><code class="language-sql">SELECT pg_size_pretty( pg_database_size( 'matrix' ) );
</code></pre>
<h3 id="result-example"><a class="header" href="#result-example">Result example:</a></h3>
<pre><code>pg_size_pretty
----------------
6420 MB
(1 row)
</code></pre>
<h2 id="show-top-20-larger-rooms-by-state-events-count"><a class="header" href="#show-top-20-larger-rooms-by-state-events-count">Show top 20 larger rooms by state events count</a></h2>
<pre><code class="language-sql">SELECT r.name, s.room_id, s.current_state_events
FROM room_stats_current s
LEFT JOIN room_stats_state r USING (room_id)
ORDER BY current_state_events DESC
LIMIT 20;
</code></pre>
<p>and by state_group_events count:</p>
<pre><code class="language-sql">SELECT rss.name, s.room_id, count(s.room_id) FROM state_groups_state s
LEFT JOIN room_stats_state rss USING (room_id)
GROUP BY s.room_id, rss.name
ORDER BY count(s.room_id) DESC
LIMIT 20;
</code></pre>
<p>plus same, but with join removed for performance reasons:</p>
<pre><code class="language-sql">SELECT s.room_id, count(s.room_id) FROM state_groups_state s
GROUP BY s.room_id
ORDER BY count(s.room_id) DESC
LIMIT 20;
</code></pre>
<h2 id="show-top-20-larger-tables-by-row-count"><a class="header" href="#show-top-20-larger-tables-by-row-count">Show top 20 larger tables by row count</a></h2>
<pre><code class="language-sql">SELECT relname, n_live_tup as rows
FROM pg_stat_user_tables
<pre><code class="language-sql">SELECT relname, n_live_tup AS &quot;rows&quot;
FROM pg_stat_user_tables
ORDER BY n_live_tup DESC
LIMIT 20;
</code></pre>
<p>This query is quick, but may be very approximate, for exact number of rows use <code>SELECT COUNT(*) FROM &lt;table_name&gt;</code>.</p>
<p>This query is quick, but may be very approximate, for exact number of rows use:</p>
<pre><code class="language-sql">SELECT COUNT(*) FROM &lt;table_name&gt;;
</code></pre>
<h3 id="result-example-1"><a class="header" href="#result-example-1">Result example:</a></h3>
<pre><code>state_groups_state - 161687170
event_auth - 8584785
@@ -15967,37 +15950,15 @@ users_in_public_rooms - 364059
device_lists_stream - 326903
user_directory_search - 316433
</code></pre>
<h2 id="show-top-20-rooms-by-new-events-count-in-last-1-day"><a class="header" href="#show-top-20-rooms-by-new-events-count-in-last-1-day">Show top 20 rooms by new events count in last 1 day:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, COUNT(e.event_id) cnt FROM events e
LEFT JOIN room_stats_state r USING (room_id)
WHERE e.origin_server_ts &gt;= DATE_PART('epoch', NOW() - INTERVAL '1 day') * 1000 GROUP BY e.room_id, r.name ORDER BY cnt DESC LIMIT 20;
</code></pre>
<h2 id="show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month"><a class="header" href="#show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month">Show top 20 users on homeserver by sent events (messages) at last month:</a></h2>
<pre><code class="language-sql">SELECT user_id, SUM(total_events)
FROM user_stats_historical
WHERE TO_TIMESTAMP(end_ts/1000) AT TIME ZONE 'UTC' &gt; date_trunc('day', now() - interval '1 month')
GROUP BY user_id
ORDER BY SUM(total_events) DESC
LIMIT 20;
</code></pre>
<h2 id="show-last-100-messages-from-needed-user-with-room-names"><a class="header" href="#show-last-100-messages-from-needed-user-with-room-names">Show last 100 messages from needed user, with room names:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, e.event_id, e.type, e.content, j.json FROM events e
LEFT JOIN event_json j USING (room_id)
LEFT JOIN room_stats_state r USING (room_id)
WHERE sender = '@LOGIN:example.com'
AND e.type = 'm.room.message'
ORDER BY stream_ordering DESC
LIMIT 100;
</code></pre>
<h2 id="show-top-20-larger-tables-by-storage-size"><a class="header" href="#show-top-20-larger-tables-by-storage-size">Show top 20 larger tables by storage size</a></h2>
<pre><code class="language-sql">SELECT nspname || '.' || relname AS &quot;relation&quot;,
pg_size_pretty(pg_total_relation_size(C.oid)) AS &quot;total_size&quot;
FROM pg_class C
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
pg_size_pretty(pg_total_relation_size(c.oid)) AS &quot;total_size&quot;
FROM pg_class c
LEFT JOIN pg_namespace n ON (n.oid = c.relnamespace)
WHERE nspname NOT IN ('pg_catalog', 'information_schema')
AND C.relkind &lt;&gt; 'i'
AND c.relkind &lt;&gt; 'i'
AND nspname !~ '^pg_toast'
ORDER BY pg_total_relation_size(C.oid) DESC
ORDER BY pg_total_relation_size(c.oid) DESC
LIMIT 20;
</code></pre>
<h3 id="result-example-2"><a class="header" href="#result-example-2">Result example:</a></h3>
@@ -16022,8 +15983,75 @@ public.state_groups - 160 MB
public.device_lists_remote_cache - 124 MB
public.state_group_edges - 122 MB
</code></pre>
<h2 id="show-top-20-larger-rooms-by-state-events-count"><a class="header" href="#show-top-20-larger-rooms-by-state-events-count">Show top 20 larger rooms by state events count</a></h2>
<p>You get the same information when you use the
<a href="usage/administration/../../admin_api/rooms.html#list-room-api">admin API</a>
and set parameter <code>order_by=state_events</code>.</p>
<pre><code class="language-sql">SELECT r.name, s.room_id, s.current_state_events
FROM room_stats_current s
LEFT JOIN room_stats_state r USING (room_id)
ORDER BY current_state_events DESC
LIMIT 20;
</code></pre>
<p>and by state_group_events count:</p>
<pre><code class="language-sql">SELECT rss.name, s.room_id, COUNT(s.room_id)
FROM state_groups_state s
LEFT JOIN room_stats_state rss USING (room_id)
GROUP BY s.room_id, rss.name
ORDER BY COUNT(s.room_id) DESC
LIMIT 20;
</code></pre>
<p>plus same, but with join removed for performance reasons:</p>
<pre><code class="language-sql">SELECT s.room_id, COUNT(s.room_id)
FROM state_groups_state s
GROUP BY s.room_id
ORDER BY COUNT(s.room_id) DESC
LIMIT 20;
</code></pre>
<h2 id="show-top-20-rooms-by-new-events-count-in-last-1-day"><a class="header" href="#show-top-20-rooms-by-new-events-count-in-last-1-day">Show top 20 rooms by new events count in last 1 day:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, COUNT(e.event_id) cnt
FROM events e
LEFT JOIN room_stats_state r USING (room_id)
WHERE e.origin_server_ts &gt;= DATE_PART('epoch', NOW() - INTERVAL '1 day') * 1000
GROUP BY e.room_id, r.name
ORDER BY cnt DESC
LIMIT 20;
</code></pre>
<h2 id="show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month"><a class="header" href="#show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month">Show top 20 users on homeserver by sent events (messages) at last month:</a></h2>
<p>Caution. This query does not use any indexes, can be slow and create load on the database.</p>
<pre><code class="language-sql">SELECT COUNT(*), sender
FROM events
WHERE (type = 'm.room.encrypted' OR type = 'm.room.message')
AND origin_server_ts &gt;= DATE_PART('epoch', NOW() - INTERVAL '1 month') * 1000
GROUP BY sender
ORDER BY COUNT(*) DESC
LIMIT 20;
</code></pre>
<h2 id="show-last-100-messages-from-needed-user-with-room-names"><a class="header" href="#show-last-100-messages-from-needed-user-with-room-names">Show last 100 messages from needed user, with room names:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, e.event_id, e.type, e.content, j.json
FROM events e
LEFT JOIN event_json j USING (room_id)
LEFT JOIN room_stats_state r USING (room_id)
WHERE sender = '@LOGIN:example.com'
AND e.type = 'm.room.message'
ORDER BY stream_ordering DESC
LIMIT 100;
</code></pre>
<h2 id="show-rooms-with-names-sorted-by-events-in-this-rooms"><a class="header" href="#show-rooms-with-names-sorted-by-events-in-this-rooms">Show rooms with names, sorted by events in this rooms</a></h2>
<p><code>echo &quot;select event_json.room_id,room_stats_state.name from event_json,room_stats_state where room_stats_state.room_id=event_json.room_id&quot; | psql synapse | sort | uniq -c | sort -n</code></p>
<p><strong>Sort and order with bash</strong></p>
<pre><code class="language-bash">echo &quot;SELECT event_json.room_id, room_stats_state.name FROM event_json, room_stats_state \
WHERE room_stats_state.room_id = event_json.room_id&quot; | psql -d synapse -h localhost -U synapse_user -t \
| sort | uniq -c | sort -n
</code></pre>
<p>Documentation for <code>psql</code> command line parameters: https://www.postgresql.org/docs/current/app-psql.html</p>
<p><strong>Sort and order with SQL</strong></p>
<pre><code class="language-sql">SELECT COUNT(*), event_json.room_id, room_stats_state.name
FROM event_json, room_stats_state
WHERE room_stats_state.room_id = event_json.room_id
GROUP BY event_json.room_id, room_stats_state.name
ORDER BY COUNT(*) DESC
LIMIT 50;
</code></pre>
<h3 id="result-example-3"><a class="header" href="#result-example-3">Result example:</a></h3>
<pre><code> 9459 !FPUfgzXYWTKgIrwKxW:matrix.org | This Week in Matrix
9459 !FPUfgzXYWTKgIrwKxW:matrix.org | This Week in Matrix (TWIM)
@@ -16035,13 +16063,21 @@ public.state_group_edges - 122 MB
43601 !iNmaIQExDMeqdITdHH:matrix.org | Riot Web/Desktop
</code></pre>
<h2 id="lookup-room-state-info-by-list-of-room_id"><a class="header" href="#lookup-room-state-info-by-list-of-room_id">Lookup room state info by list of room_id</a></h2>
<pre><code class="language-sql">SELECT rss.room_id, rss.name, rss.canonical_alias, rss.topic, rss.encryption, rsc.joined_members, rsc.local_users_in_room, rss.join_rules
FROM room_stats_state rss
LEFT JOIN room_stats_current rsc USING (room_id)
WHERE room_id IN (WHERE room_id IN (
'!OGEhHVWSdvArJzumhm:matrix.org',
'!YTvKGNlinIzlkMTVRl:matrix.org'
)
<p>You get the same information when you use the
<a href="usage/administration/../../admin_api/rooms.html#room-details-api">admin API</a>.</p>
<pre><code class="language-sql">SELECT rss.room_id, rss.name, rss.canonical_alias, rss.topic, rss.encryption,
rsc.joined_members, rsc.local_users_in_room, rss.join_rules
FROM room_stats_state rss
LEFT JOIN room_stats_current rsc USING (room_id)
WHERE room_id IN ( WHERE room_id IN (
'!OGEhHVWSdvArJzumhm:matrix.org',
'!YTvKGNlinIzlkMTVRl:matrix.org'
);
</code></pre>
<h2 id="show-users-and-devices-that-have-not-been-online-for-a-while"><a class="header" href="#show-users-and-devices-that-have-not-been-online-for-a-while">Show users and devices that have not been online for a while</a></h2>
<pre><code class="language-sql">SELECT user_id, device_id, user_agent, TO_TIMESTAMP(last_seen / 1000) AS &quot;last_seen&quot;
FROM devices
WHERE last_seen &lt; DATE_PART('epoch', NOW() - INTERVAL '3 month') * 1000;
</code></pre>
<div style="break-before: page; page-break-before: always;"></div><p>This blog post by Victor Berger explains how to use many of the tools listed on this page: https://levans.fr/shrink-synapse-database.html</p>
<h1 id="list-of-useful-tools-and-scripts-for-maintenance-synapse-database"><a class="header" href="#list-of-useful-tools-and-scripts-for-maintenance-synapse-database">List of useful tools and scripts for maintenance Synapse database:</a></h1>
develop/searchindex.js
+1 -1
View File
@@ -1 +1 @@
Object.assign(window.search, {"doc_urls":["welcome_and_overview.html#introduction","welcome_and_overview.html#installing-and-using-synapse","welcome_and_overview.html#developing-on-synapse","welcome_and_overview.html#donating-to-synapse-development","welcome_and_overview.html#reporting-a-security-vulnerability","setup/installation.html#installation-instructions","setup/installation.html#choosing-your-server-name","setup/installation.html#installing-synapse","setup/installation.html#prebuilt-packages","setup/installation.html#installing-as-a-python-module-from-pypi","setup/installation.html#setting-up-synapse","setup/installation.html#using-postgresql","setup/installation.html#tls-certificates","setup/installation.html#client-well-known-uri","setup/installation.html#email","setup/installation.html#registering-a-user","setup/installation.html#setting-up-a-turn-server","setup/installation.html#url-previews","setup/installation.html#troubleshooting-installation","postgres.html#using-postgres","postgres.html#install-postgres-client-libraries","postgres.html#set-up-database","postgres.html#synapse-config","postgres.html#tuning-postgres","postgres.html#porting-from-sqlite","postgres.html#overview","postgres.html#using-the-port-script","postgres.html#troubleshooting","postgres.html#alternative-auth-methods","postgres.html#fixing-incorrect-collate-or-ctype","postgres.html#fixing-inconsistent-sequences-error","reverse_proxy.html#using-a-reverse-proxy-with-synapse","reverse_proxy.html#homeserver-configuration","reverse_proxy.html#reverse-proxy-configuration-examples","reverse_proxy.html#nginx","reverse_proxy.html#caddy-v1","reverse_proxy.html#caddy-v2","reverse_proxy.html#apache","reverse_proxy.html#haproxy","reverse_proxy.html#relayd","reverse_proxy.html#health-check-endpoint","reverse_proxy.html#synapse-administration-endpoints","setup/forward_proxy.html#using-a-forward-proxy-with-synapse","setup/forward_proxy.html#configure","setup/forward_proxy.html#connection-types","setup/forward_proxy.html#troubleshooting","turn-howto.html#overview","turn-howto.html#requirements","turn-howto.html#coturn-setup","turn-howto.html#initial-installation","turn-howto.html#configuration","turn-howto.html#synapse-setup","turn-howto.html#troubleshooting","delegate.html#delegation-of-incoming-federation-traffic","delegate.html#well-known-delegation","delegate.html#external-server","delegate.html#serving-a-well-knownmatrixserver-file-with-synapse","delegate.html#srv-dns-record-delegation","delegate.html#delegation-faq","delegate.html#when-do-i-need-delegation","delegate.html#should-i-use-a-reverse-proxy-for-federation-traffic","upgrade.html#upgrading-synapse","upgrade.html#rolling-back-to-older-versions","upgrade.html#upgrading-to-v1590","upgrade.html#device-name-lookup-over-federation-has-been-disabled-by-default","upgrade.html#deprecation-of-the-synapseappappservice-worker-application-type","upgrade.html#upgrading-to-v1580","upgrade.html#groupscommunities-feature-has-been-disabled-by-default","upgrade.html#upgrading-to-v1570","upgrade.html#changes-to-database-schema-for-application-services","upgrade.html#upgrading-to-v1560","upgrade.html#open-registration-without-verification-is-now-disabled-by-default","upgrade.html#groupscommunities-feature-has-been-deprecated","upgrade.html#change-in-behaviour-for-postgresql-databases-with-unsafe-locale","upgrade.html#upgrading-to-v1550","upgrade.html#synctl-script-has-been-moved","upgrade.html#compatibility-dropped-for-mjolnir-131-and-earlier","upgrade.html#upgrading-to-v1540","upgrade.html#legacy-structured-logging-configuration-removal","upgrade.html#upgrading-to-v1530","upgrade.html#dropping-support-for-webclient-listeners-and-non-https-web_client_location","upgrade.html#stablisation-of-msc3231","upgrade.html#time-based-cache-expiry-is-now-enabled-by-default","upgrade.html#deprecation-of-capability-orgmatrixmsc3283","upgrade.html#removal-of-user_may_create_room_with_invites","upgrade.html#upgrading-to-v1520","upgrade.html#twisted-security-release","upgrade.html#upgrading-to-v1510","upgrade.html#deprecation-of-webclient-listeners-and-non-https-web_client_location","upgrade.html#upgrading-to-v1500","upgrade.html#dropping-support-for-old-python-and-postgres-versions","upgrade.html#upgrading-to-v1470","upgrade.html#removal-of-old-room-admin-api","upgrade.html#deprecation-of-the-user_may_create_room_with_invites-module-callback","upgrade.html#upgrading-to-v1450","upgrade.html#changes-required-to-media-storage-provider-modules-when-reading-from-the-synapse-configuration-object","upgrade.html#upgrading-to-v1440","upgrade.html#the-url-preview-cache-is-no-longer-mirrored-to-storage-providers","upgrade.html#upgrading-to-v1430","upgrade.html#the-spaces-summary-apis-can-now-be-handled-by-workers","upgrade.html#upgrading-to-v1420","upgrade.html#removal-of-old-room-admin-api-1","upgrade.html#user-interactive-authentication-fallback-templates-can-now-display-errors","upgrade.html#removal-of-out-of-date-email-pLine truncated
Object.assign(window.search, {"doc_urls":["welcome_and_overview.html#introduction","welcome_and_overview.html#installing-and-using-synapse","welcome_and_overview.html#developing-on-synapse","welcome_and_overview.html#donating-to-synapse-development","welcome_and_overview.html#reporting-a-security-vulnerability","setup/installation.html#installation-instructions","setup/installation.html#choosing-your-server-name","setup/installation.html#installing-synapse","setup/installation.html#prebuilt-packages","setup/installation.html#installing-as-a-python-module-from-pypi","setup/installation.html#setting-up-synapse","setup/installation.html#using-postgresql","setup/installation.html#tls-certificates","setup/installation.html#client-well-known-uri","setup/installation.html#email","setup/installation.html#registering-a-user","setup/installation.html#setting-up-a-turn-server","setup/installation.html#url-previews","setup/installation.html#troubleshooting-installation","postgres.html#using-postgres","postgres.html#install-postgres-client-libraries","postgres.html#set-up-database","postgres.html#synapse-config","postgres.html#tuning-postgres","postgres.html#porting-from-sqlite","postgres.html#overview","postgres.html#using-the-port-script","postgres.html#troubleshooting","postgres.html#alternative-auth-methods","postgres.html#fixing-incorrect-collate-or-ctype","postgres.html#fixing-inconsistent-sequences-error","reverse_proxy.html#using-a-reverse-proxy-with-synapse","reverse_proxy.html#homeserver-configuration","reverse_proxy.html#reverse-proxy-configuration-examples","reverse_proxy.html#nginx","reverse_proxy.html#caddy-v1","reverse_proxy.html#caddy-v2","reverse_proxy.html#apache","reverse_proxy.html#haproxy","reverse_proxy.html#relayd","reverse_proxy.html#health-check-endpoint","reverse_proxy.html#synapse-administration-endpoints","setup/forward_proxy.html#using-a-forward-proxy-with-synapse","setup/forward_proxy.html#configure","setup/forward_proxy.html#connection-types","setup/forward_proxy.html#troubleshooting","turn-howto.html#overview","turn-howto.html#requirements","turn-howto.html#coturn-setup","turn-howto.html#initial-installation","turn-howto.html#configuration","turn-howto.html#synapse-setup","turn-howto.html#troubleshooting","delegate.html#delegation-of-incoming-federation-traffic","delegate.html#well-known-delegation","delegate.html#external-server","delegate.html#serving-a-well-knownmatrixserver-file-with-synapse","delegate.html#srv-dns-record-delegation","delegate.html#delegation-faq","delegate.html#when-do-i-need-delegation","delegate.html#should-i-use-a-reverse-proxy-for-federation-traffic","upgrade.html#upgrading-synapse","upgrade.html#rolling-back-to-older-versions","upgrade.html#upgrading-to-v1590","upgrade.html#device-name-lookup-over-federation-has-been-disabled-by-default","upgrade.html#deprecation-of-the-synapseappappservice-worker-application-type","upgrade.html#upgrading-to-v1580","upgrade.html#groupscommunities-feature-has-been-disabled-by-default","upgrade.html#upgrading-to-v1570","upgrade.html#changes-to-database-schema-for-application-services","upgrade.html#upgrading-to-v1560","upgrade.html#open-registration-without-verification-is-now-disabled-by-default","upgrade.html#groupscommunities-feature-has-been-deprecated","upgrade.html#change-in-behaviour-for-postgresql-databases-with-unsafe-locale","upgrade.html#upgrading-to-v1550","upgrade.html#synctl-script-has-been-moved","upgrade.html#compatibility-dropped-for-mjolnir-131-and-earlier","upgrade.html#upgrading-to-v1540","upgrade.html#legacy-structured-logging-configuration-removal","upgrade.html#upgrading-to-v1530","upgrade.html#dropping-support-for-webclient-listeners-and-non-https-web_client_location","upgrade.html#stablisation-of-msc3231","upgrade.html#time-based-cache-expiry-is-now-enabled-by-default","upgrade.html#deprecation-of-capability-orgmatrixmsc3283","upgrade.html#removal-of-user_may_create_room_with_invites","upgrade.html#upgrading-to-v1520","upgrade.html#twisted-security-release","upgrade.html#upgrading-to-v1510","upgrade.html#deprecation-of-webclient-listeners-and-non-https-web_client_location","upgrade.html#upgrading-to-v1500","upgrade.html#dropping-support-for-old-python-and-postgres-versions","upgrade.html#upgrading-to-v1470","upgrade.html#removal-of-old-room-admin-api","upgrade.html#deprecation-of-the-user_may_create_room_with_invites-module-callback","upgrade.html#upgrading-to-v1450","upgrade.html#changes-required-to-media-storage-provider-modules-when-reading-from-the-synapse-configuration-object","upgrade.html#upgrading-to-v1440","upgrade.html#the-url-preview-cache-is-no-longer-mirrored-to-storage-providers","upgrade.html#upgrading-to-v1430","upgrade.html#the-spaces-summary-apis-can-now-be-handled-by-workers","upgrade.html#upgrading-to-v1420","upgrade.html#removal-of-old-room-admin-api-1","upgrade.html#user-interactive-authentication-fallback-templates-can-now-display-errors","upgrade.html#removal-of-out-of-date-email-pLine truncated
develop/searchindex.json
+1 -1
View File
@@ -1 +1 @@
{"doc_urls":["welcome_and_overview.html#introduction","welcome_and_overview.html#installing-and-using-synapse","welcome_and_overview.html#developing-on-synapse","welcome_and_overview.html#donating-to-synapse-development","welcome_and_overview.html#reporting-a-security-vulnerability","setup/installation.html#installation-instructions","setup/installation.html#choosing-your-server-name","setup/installation.html#installing-synapse","setup/installation.html#prebuilt-packages","setup/installation.html#installing-as-a-python-module-from-pypi","setup/installation.html#setting-up-synapse","setup/installation.html#using-postgresql","setup/installation.html#tls-certificates","setup/installation.html#client-well-known-uri","setup/installation.html#email","setup/installation.html#registering-a-user","setup/installation.html#setting-up-a-turn-server","setup/installation.html#url-previews","setup/installation.html#troubleshooting-installation","postgres.html#using-postgres","postgres.html#install-postgres-client-libraries","postgres.html#set-up-database","postgres.html#synapse-config","postgres.html#tuning-postgres","postgres.html#porting-from-sqlite","postgres.html#overview","postgres.html#using-the-port-script","postgres.html#troubleshooting","postgres.html#alternative-auth-methods","postgres.html#fixing-incorrect-collate-or-ctype","postgres.html#fixing-inconsistent-sequences-error","reverse_proxy.html#using-a-reverse-proxy-with-synapse","reverse_proxy.html#homeserver-configuration","reverse_proxy.html#reverse-proxy-configuration-examples","reverse_proxy.html#nginx","reverse_proxy.html#caddy-v1","reverse_proxy.html#caddy-v2","reverse_proxy.html#apache","reverse_proxy.html#haproxy","reverse_proxy.html#relayd","reverse_proxy.html#health-check-endpoint","reverse_proxy.html#synapse-administration-endpoints","setup/forward_proxy.html#using-a-forward-proxy-with-synapse","setup/forward_proxy.html#configure","setup/forward_proxy.html#connection-types","setup/forward_proxy.html#troubleshooting","turn-howto.html#overview","turn-howto.html#requirements","turn-howto.html#coturn-setup","turn-howto.html#initial-installation","turn-howto.html#configuration","turn-howto.html#synapse-setup","turn-howto.html#troubleshooting","delegate.html#delegation-of-incoming-federation-traffic","delegate.html#well-known-delegation","delegate.html#external-server","delegate.html#serving-a-well-knownmatrixserver-file-with-synapse","delegate.html#srv-dns-record-delegation","delegate.html#delegation-faq","delegate.html#when-do-i-need-delegation","delegate.html#should-i-use-a-reverse-proxy-for-federation-traffic","upgrade.html#upgrading-synapse","upgrade.html#rolling-back-to-older-versions","upgrade.html#upgrading-to-v1590","upgrade.html#device-name-lookup-over-federation-has-been-disabled-by-default","upgrade.html#deprecation-of-the-synapseappappservice-worker-application-type","upgrade.html#upgrading-to-v1580","upgrade.html#groupscommunities-feature-has-been-disabled-by-default","upgrade.html#upgrading-to-v1570","upgrade.html#changes-to-database-schema-for-application-services","upgrade.html#upgrading-to-v1560","upgrade.html#open-registration-without-verification-is-now-disabled-by-default","upgrade.html#groupscommunities-feature-has-been-deprecated","upgrade.html#change-in-behaviour-for-postgresql-databases-with-unsafe-locale","upgrade.html#upgrading-to-v1550","upgrade.html#synctl-script-has-been-moved","upgrade.html#compatibility-dropped-for-mjolnir-131-and-earlier","upgrade.html#upgrading-to-v1540","upgrade.html#legacy-structured-logging-configuration-removal","upgrade.html#upgrading-to-v1530","upgrade.html#dropping-support-for-webclient-listeners-and-non-https-web_client_location","upgrade.html#stablisation-of-msc3231","upgrade.html#time-based-cache-expiry-is-now-enabled-by-default","upgrade.html#deprecation-of-capability-orgmatrixmsc3283","upgrade.html#removal-of-user_may_create_room_with_invites","upgrade.html#upgrading-to-v1520","upgrade.html#twisted-security-release","upgrade.html#upgrading-to-v1510","upgrade.html#deprecation-of-webclient-listeners-and-non-https-web_client_location","upgrade.html#upgrading-to-v1500","upgrade.html#dropping-support-for-old-python-and-postgres-versions","upgrade.html#upgrading-to-v1470","upgrade.html#removal-of-old-room-admin-api","upgrade.html#deprecation-of-the-user_may_create_room_with_invites-module-callback","upgrade.html#upgrading-to-v1450","upgrade.html#changes-required-to-media-storage-provider-modules-when-reading-from-the-synapse-configuration-object","upgrade.html#upgrading-to-v1440","upgrade.html#the-url-preview-cache-is-no-longer-mirrored-to-storage-providers","upgrade.html#upgrading-to-v1430","upgrade.html#the-spaces-summary-apis-can-now-be-handled-by-workers","upgrade.html#upgrading-to-v1420","upgrade.html#removal-of-old-room-admin-api-1","upgrade.html#user-interactive-authentication-fallback-templates-can-now-display-errors","upgrade.html#removal-of-out-of-date-email-pushers","upgrade.html#upgradiLine truncated
{"doc_urls":["welcome_and_overview.html#introduction","welcome_and_overview.html#installing-and-using-synapse","welcome_and_overview.html#developing-on-synapse","welcome_and_overview.html#donating-to-synapse-development","welcome_and_overview.html#reporting-a-security-vulnerability","setup/installation.html#installation-instructions","setup/installation.html#choosing-your-server-name","setup/installation.html#installing-synapse","setup/installation.html#prebuilt-packages","setup/installation.html#installing-as-a-python-module-from-pypi","setup/installation.html#setting-up-synapse","setup/installation.html#using-postgresql","setup/installation.html#tls-certificates","setup/installation.html#client-well-known-uri","setup/installation.html#email","setup/installation.html#registering-a-user","setup/installation.html#setting-up-a-turn-server","setup/installation.html#url-previews","setup/installation.html#troubleshooting-installation","postgres.html#using-postgres","postgres.html#install-postgres-client-libraries","postgres.html#set-up-database","postgres.html#synapse-config","postgres.html#tuning-postgres","postgres.html#porting-from-sqlite","postgres.html#overview","postgres.html#using-the-port-script","postgres.html#troubleshooting","postgres.html#alternative-auth-methods","postgres.html#fixing-incorrect-collate-or-ctype","postgres.html#fixing-inconsistent-sequences-error","reverse_proxy.html#using-a-reverse-proxy-with-synapse","reverse_proxy.html#homeserver-configuration","reverse_proxy.html#reverse-proxy-configuration-examples","reverse_proxy.html#nginx","reverse_proxy.html#caddy-v1","reverse_proxy.html#caddy-v2","reverse_proxy.html#apache","reverse_proxy.html#haproxy","reverse_proxy.html#relayd","reverse_proxy.html#health-check-endpoint","reverse_proxy.html#synapse-administration-endpoints","setup/forward_proxy.html#using-a-forward-proxy-with-synapse","setup/forward_proxy.html#configure","setup/forward_proxy.html#connection-types","setup/forward_proxy.html#troubleshooting","turn-howto.html#overview","turn-howto.html#requirements","turn-howto.html#coturn-setup","turn-howto.html#initial-installation","turn-howto.html#configuration","turn-howto.html#synapse-setup","turn-howto.html#troubleshooting","delegate.html#delegation-of-incoming-federation-traffic","delegate.html#well-known-delegation","delegate.html#external-server","delegate.html#serving-a-well-knownmatrixserver-file-with-synapse","delegate.html#srv-dns-record-delegation","delegate.html#delegation-faq","delegate.html#when-do-i-need-delegation","delegate.html#should-i-use-a-reverse-proxy-for-federation-traffic","upgrade.html#upgrading-synapse","upgrade.html#rolling-back-to-older-versions","upgrade.html#upgrading-to-v1590","upgrade.html#device-name-lookup-over-federation-has-been-disabled-by-default","upgrade.html#deprecation-of-the-synapseappappservice-worker-application-type","upgrade.html#upgrading-to-v1580","upgrade.html#groupscommunities-feature-has-been-disabled-by-default","upgrade.html#upgrading-to-v1570","upgrade.html#changes-to-database-schema-for-application-services","upgrade.html#upgrading-to-v1560","upgrade.html#open-registration-without-verification-is-now-disabled-by-default","upgrade.html#groupscommunities-feature-has-been-deprecated","upgrade.html#change-in-behaviour-for-postgresql-databases-with-unsafe-locale","upgrade.html#upgrading-to-v1550","upgrade.html#synctl-script-has-been-moved","upgrade.html#compatibility-dropped-for-mjolnir-131-and-earlier","upgrade.html#upgrading-to-v1540","upgrade.html#legacy-structured-logging-configuration-removal","upgrade.html#upgrading-to-v1530","upgrade.html#dropping-support-for-webclient-listeners-and-non-https-web_client_location","upgrade.html#stablisation-of-msc3231","upgrade.html#time-based-cache-expiry-is-now-enabled-by-default","upgrade.html#deprecation-of-capability-orgmatrixmsc3283","upgrade.html#removal-of-user_may_create_room_with_invites","upgrade.html#upgrading-to-v1520","upgrade.html#twisted-security-release","upgrade.html#upgrading-to-v1510","upgrade.html#deprecation-of-webclient-listeners-and-non-https-web_client_location","upgrade.html#upgrading-to-v1500","upgrade.html#dropping-support-for-old-python-and-postgres-versions","upgrade.html#upgrading-to-v1470","upgrade.html#removal-of-old-room-admin-api","upgrade.html#deprecation-of-the-user_may_create_room_with_invites-module-callback","upgrade.html#upgrading-to-v1450","upgrade.html#changes-required-to-media-storage-provider-modules-when-reading-from-the-synapse-configuration-object","upgrade.html#upgrading-to-v1440","upgrade.html#the-url-preview-cache-is-no-longer-mirrored-to-storage-providers","upgrade.html#upgrading-to-v1430","upgrade.html#the-spaces-summary-apis-can-now-be-handled-by-workers","upgrade.html#upgrading-to-v1420","upgrade.html#removal-of-old-room-admin-api-1","upgrade.html#user-interactive-authentication-fallback-templates-can-now-display-errors","upgrade.html#removal-of-out-of-date-email-pushers","upgrade.html#upgradiLine truncated
develop/usage/administration/useful_sql_for_admins.html
+95 -59
View File
@@ -148,40 +148,23 @@
<h2 id="some-useful-sql-queries-for-synapse-admins"><a class="header" href="#some-useful-sql-queries-for-synapse-admins">Some useful SQL queries for Synapse Admins</a></h2>
<h2 id="size-of-full-matrix-db"><a class="header" href="#size-of-full-matrix-db">Size of full matrix db</a></h2>
<p><code>SELECT pg_size_pretty( pg_database_size( 'matrix' ) );</code></p>
<pre><code class="language-sql">SELECT pg_size_pretty( pg_database_size( 'matrix' ) );
</code></pre>
<h3 id="result-example"><a class="header" href="#result-example">Result example:</a></h3>
<pre><code>pg_size_pretty
----------------
6420 MB
(1 row)
</code></pre>
<h2 id="show-top-20-larger-rooms-by-state-events-count"><a class="header" href="#show-top-20-larger-rooms-by-state-events-count">Show top 20 larger rooms by state events count</a></h2>
<pre><code class="language-sql">SELECT r.name, s.room_id, s.current_state_events
FROM room_stats_current s
LEFT JOIN room_stats_state r USING (room_id)
ORDER BY current_state_events DESC
LIMIT 20;
</code></pre>
<p>and by state_group_events count:</p>
<pre><code class="language-sql">SELECT rss.name, s.room_id, count(s.room_id) FROM state_groups_state s
LEFT JOIN room_stats_state rss USING (room_id)
GROUP BY s.room_id, rss.name
ORDER BY count(s.room_id) DESC
LIMIT 20;
</code></pre>
<p>plus same, but with join removed for performance reasons:</p>
<pre><code class="language-sql">SELECT s.room_id, count(s.room_id) FROM state_groups_state s
GROUP BY s.room_id
ORDER BY count(s.room_id) DESC
LIMIT 20;
</code></pre>
<h2 id="show-top-20-larger-tables-by-row-count"><a class="header" href="#show-top-20-larger-tables-by-row-count">Show top 20 larger tables by row count</a></h2>
<pre><code class="language-sql">SELECT relname, n_live_tup as rows
FROM pg_stat_user_tables
<pre><code class="language-sql">SELECT relname, n_live_tup AS &quot;rows&quot;
FROM pg_stat_user_tables
ORDER BY n_live_tup DESC
LIMIT 20;
</code></pre>
<p>This query is quick, but may be very approximate, for exact number of rows use <code>SELECT COUNT(*) FROM &lt;table_name&gt;</code>.</p>
<p>This query is quick, but may be very approximate, for exact number of rows use:</p>
<pre><code class="language-sql">SELECT COUNT(*) FROM &lt;table_name&gt;;
</code></pre>
<h3 id="result-example-1"><a class="header" href="#result-example-1">Result example:</a></h3>
<pre><code>state_groups_state - 161687170
event_auth - 8584785
@@ -204,37 +187,15 @@ users_in_public_rooms - 364059
device_lists_stream - 326903
user_directory_search - 316433
</code></pre>
<h2 id="show-top-20-rooms-by-new-events-count-in-last-1-day"><a class="header" href="#show-top-20-rooms-by-new-events-count-in-last-1-day">Show top 20 rooms by new events count in last 1 day:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, COUNT(e.event_id) cnt FROM events e
LEFT JOIN room_stats_state r USING (room_id)
WHERE e.origin_server_ts &gt;= DATE_PART('epoch', NOW() - INTERVAL '1 day') * 1000 GROUP BY e.room_id, r.name ORDER BY cnt DESC LIMIT 20;
</code></pre>
<h2 id="show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month"><a class="header" href="#show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month">Show top 20 users on homeserver by sent events (messages) at last month:</a></h2>
<pre><code class="language-sql">SELECT user_id, SUM(total_events)
FROM user_stats_historical
WHERE TO_TIMESTAMP(end_ts/1000) AT TIME ZONE 'UTC' &gt; date_trunc('day', now() - interval '1 month')
GROUP BY user_id
ORDER BY SUM(total_events) DESC
LIMIT 20;
</code></pre>
<h2 id="show-last-100-messages-from-needed-user-with-room-names"><a class="header" href="#show-last-100-messages-from-needed-user-with-room-names">Show last 100 messages from needed user, with room names:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, e.event_id, e.type, e.content, j.json FROM events e
LEFT JOIN event_json j USING (room_id)
LEFT JOIN room_stats_state r USING (room_id)
WHERE sender = '@LOGIN:example.com'
AND e.type = 'm.room.message'
ORDER BY stream_ordering DESC
LIMIT 100;
</code></pre>
<h2 id="show-top-20-larger-tables-by-storage-size"><a class="header" href="#show-top-20-larger-tables-by-storage-size">Show top 20 larger tables by storage size</a></h2>
<pre><code class="language-sql">SELECT nspname || '.' || relname AS &quot;relation&quot;,
pg_size_pretty(pg_total_relation_size(C.oid)) AS &quot;total_size&quot;
FROM pg_class C
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
pg_size_pretty(pg_total_relation_size(c.oid)) AS &quot;total_size&quot;
FROM pg_class c
LEFT JOIN pg_namespace n ON (n.oid = c.relnamespace)
WHERE nspname NOT IN ('pg_catalog', 'information_schema')
AND C.relkind &lt;&gt; 'i'
AND c.relkind &lt;&gt; 'i'
AND nspname !~ '^pg_toast'
ORDER BY pg_total_relation_size(C.oid) DESC
ORDER BY pg_total_relation_size(c.oid) DESC
LIMIT 20;
</code></pre>
<h3 id="result-example-2"><a class="header" href="#result-example-2">Result example:</a></h3>
@@ -259,8 +220,75 @@ public.state_groups - 160 MB
public.device_lists_remote_cache - 124 MB
public.state_group_edges - 122 MB
</code></pre>
<h2 id="show-top-20-larger-rooms-by-state-events-count"><a class="header" href="#show-top-20-larger-rooms-by-state-events-count">Show top 20 larger rooms by state events count</a></h2>
<p>You get the same information when you use the
<a href="../../admin_api/rooms.html#list-room-api">admin API</a>
and set parameter <code>order_by=state_events</code>.</p>
<pre><code class="language-sql">SELECT r.name, s.room_id, s.current_state_events
FROM room_stats_current s
LEFT JOIN room_stats_state r USING (room_id)
ORDER BY current_state_events DESC
LIMIT 20;
</code></pre>
<p>and by state_group_events count:</p>
<pre><code class="language-sql">SELECT rss.name, s.room_id, COUNT(s.room_id)
FROM state_groups_state s
LEFT JOIN room_stats_state rss USING (room_id)
GROUP BY s.room_id, rss.name
ORDER BY COUNT(s.room_id) DESC
LIMIT 20;
</code></pre>
<p>plus same, but with join removed for performance reasons:</p>
<pre><code class="language-sql">SELECT s.room_id, COUNT(s.room_id)
FROM state_groups_state s
GROUP BY s.room_id
ORDER BY COUNT(s.room_id) DESC
LIMIT 20;
</code></pre>
<h2 id="show-top-20-rooms-by-new-events-count-in-last-1-day"><a class="header" href="#show-top-20-rooms-by-new-events-count-in-last-1-day">Show top 20 rooms by new events count in last 1 day:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, COUNT(e.event_id) cnt
FROM events e
LEFT JOIN room_stats_state r USING (room_id)
WHERE e.origin_server_ts &gt;= DATE_PART('epoch', NOW() - INTERVAL '1 day') * 1000
GROUP BY e.room_id, r.name
ORDER BY cnt DESC
LIMIT 20;
</code></pre>
<h2 id="show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month"><a class="header" href="#show-top-20-users-on-homeserver-by-sent-events-messages-at-last-month">Show top 20 users on homeserver by sent events (messages) at last month:</a></h2>
<p>Caution. This query does not use any indexes, can be slow and create load on the database.</p>
<pre><code class="language-sql">SELECT COUNT(*), sender
FROM events
WHERE (type = 'm.room.encrypted' OR type = 'm.room.message')
AND origin_server_ts &gt;= DATE_PART('epoch', NOW() - INTERVAL '1 month') * 1000
GROUP BY sender
ORDER BY COUNT(*) DESC
LIMIT 20;
</code></pre>
<h2 id="show-last-100-messages-from-needed-user-with-room-names"><a class="header" href="#show-last-100-messages-from-needed-user-with-room-names">Show last 100 messages from needed user, with room names:</a></h2>
<pre><code class="language-sql">SELECT e.room_id, r.name, e.event_id, e.type, e.content, j.json
FROM events e
LEFT JOIN event_json j USING (room_id)
LEFT JOIN room_stats_state r USING (room_id)
WHERE sender = '@LOGIN:example.com'
AND e.type = 'm.room.message'
ORDER BY stream_ordering DESC
LIMIT 100;
</code></pre>
<h2 id="show-rooms-with-names-sorted-by-events-in-this-rooms"><a class="header" href="#show-rooms-with-names-sorted-by-events-in-this-rooms">Show rooms with names, sorted by events in this rooms</a></h2>
<p><code>echo &quot;select event_json.room_id,room_stats_state.name from event_json,room_stats_state where room_stats_state.room_id=event_json.room_id&quot; | psql synapse | sort | uniq -c | sort -n</code></p>
<p><strong>Sort and order with bash</strong></p>
<pre><code class="language-bash">echo &quot;SELECT event_json.room_id, room_stats_state.name FROM event_json, room_stats_state \
WHERE room_stats_state.room_id = event_json.room_id&quot; | psql -d synapse -h localhost -U synapse_user -t \
| sort | uniq -c | sort -n
</code></pre>
<p>Documentation for <code>psql</code> command line parameters: https://www.postgresql.org/docs/current/app-psql.html</p>
<p><strong>Sort and order with SQL</strong></p>
<pre><code class="language-sql">SELECT COUNT(*), event_json.room_id, room_stats_state.name
FROM event_json, room_stats_state
WHERE room_stats_state.room_id = event_json.room_id
GROUP BY event_json.room_id, room_stats_state.name
ORDER BY COUNT(*) DESC
LIMIT 50;
</code></pre>
<h3 id="result-example-3"><a class="header" href="#result-example-3">Result example:</a></h3>
<pre><code> 9459 !FPUfgzXYWTKgIrwKxW:matrix.org | This Week in Matrix
9459 !FPUfgzXYWTKgIrwKxW:matrix.org | This Week in Matrix (TWIM)
@@ -272,13 +300,21 @@ public.state_group_edges - 122 MB
43601 !iNmaIQExDMeqdITdHH:matrix.org | Riot Web/Desktop
</code></pre>
<h2 id="lookup-room-state-info-by-list-of-room_id"><a class="header" href="#lookup-room-state-info-by-list-of-room_id">Lookup room state info by list of room_id</a></h2>
<pre><code class="language-sql">SELECT rss.room_id, rss.name, rss.canonical_alias, rss.topic, rss.encryption, rsc.joined_members, rsc.local_users_in_room, rss.join_rules
FROM room_stats_state rss
LEFT JOIN room_stats_current rsc USING (room_id)
WHERE room_id IN (WHERE room_id IN (
'!OGEhHVWSdvArJzumhm:matrix.org',
'!YTvKGNlinIzlkMTVRl:matrix.org'
)
<p>You get the same information when you use the
<a href="../../admin_api/rooms.html#room-details-api">admin API</a>.</p>
<pre><code class="language-sql">SELECT rss.room_id, rss.name, rss.canonical_alias, rss.topic, rss.encryption,
rsc.joined_members, rsc.local_users_in_room, rss.join_rules
FROM room_stats_state rss
LEFT JOIN room_stats_current rsc USING (room_id)
WHERE room_id IN ( WHERE room_id IN (
'!OGEhHVWSdvArJzumhm:matrix.org',
'!YTvKGNlinIzlkMTVRl:matrix.org'
);
</code></pre>
<h2 id="show-users-and-devices-that-have-not-been-online-for-a-while"><a class="header" href="#show-users-and-devices-that-have-not-been-online-for-a-while">Show users and devices that have not been online for a while</a></h2>
<pre><code class="language-sql">SELECT user_id, device_id, user_agent, TO_TIMESTAMP(last_seen / 1000) AS &quot;last_seen&quot;
FROM devices
WHERE last_seen &lt; DATE_PART('epoch', NOW() - INTERVAL '3 month') * 1000;
</code></pre>
</main>