mirror of
https://github.com/element-hq/synapse.git
synced 2026-08-29 09:58:34 +00:00
Warn about skipping reactor metrics when using unknown reactor type (#19383)
Spawning from not seeing any reactor metrics in the Grafana dashboard in some load tests, noticing `python_twisted_reactor_tick_time_bucket` is `0` in Prometheus, following it back to Synapse and seeing that we don't warn about skipping reactor metrics in all cases (when using an unknown reactor type). A follow-up to this would be to actually figure out how to instrument the `ProxiedReactor` or why `ProxiedReactor` is being chosen in the first place and see if we can get it to use a more normal type 🤔 ### Reproduction instructions 1. Using the Complement scripts **with workers**: `WORKERS=1 ./scripts-dev/complement.sh ./tests/csapi` 1. `docker logs complement_csapi_dirty_hs1 2>&1 | grep -i "reactor"` 1. With these changes, notice `Skipping configuring ReactorLastSeenMetric: unexpected reactor type: <__main__.ProxiedReactor object at 0x7fc0adaaea50>` and `Twisted reactor: ProxiedReactor` 1. Cleanup: - `docker stop $(docker ps --all --filter "label=complement_context" --quiet)` - `docker rm $(docker ps --all --filter "label=complement_context" --quiet)` I'm unable to reproduce with the normal Synapse images or `complement-synapse` without workers. They all use `Twisted reactor: EPollReactor` <details> <summary>Checking <code>docker/Dockerfile-workers</code></summary> 1. Build the Docker image for Synapse: `docker build -t matrixdotorg/synapse -f docker/Dockerfile . && docker build -t matrixdotorg/synapse-workers -f docker/Dockerfile-workers .` ([docs](https://github.com/element-hq/synapse/blob/7a24fafbc376b9bffeb3277b1ad4aa950720c96c/docker/README-testing.md#building-and-running-the-images-manually)) 1. Start Synapse: ``` docker run -d --name synapse \ --mount type=volume,src=synapse-data,dst=/data \ -e SYNAPSE_SERVER_NAME=my.docker.synapse.server \ -e SYNAPSE_REPORT_STATS=no \ -e SYNAPSE_ENABLE_METRICS=1 \ -p 8008:8008 \ -p 9469:9469 \ matrixdotorg/synapse-workers:latest ``` 1. `docker logs synapse 2>&1 | grep -i "reactor"` 1. Says `Twisted reactor: EPollReactor` </details>
This commit is contained in:
@@ -0,0 +1 @@
|
||||
Warn about skipping reactor metrics when using unknown reactor type.
|
||||
@@ -158,12 +158,17 @@ try:
|
||||
else:
|
||||
# E.g. this does not support the (Windows-only) ProactorEventLoop.
|
||||
logger.warning(
|
||||
"Skipping configuring ReactorLastSeenMetric: unexpected asyncio loop selector: %r via %r",
|
||||
"Skipping configuring reactor metrics: unexpected asyncio loop selector: %r via %r",
|
||||
selector,
|
||||
asyncio_loop,
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"Skipping configuring reactor metrics: unexpected reactor type: %r",
|
||||
reactor,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("Configuring ReactorLastSeenMetric failed: %r", e)
|
||||
logger.warning("Configuring reactor metrics failed: %r", e)
|
||||
|
||||
|
||||
if wrapper:
|
||||
|
||||
Reference in New Issue
Block a user