Use beautifulsoup4 instead of lxml for URL previews (#19301)

Use `beautifulsoup4` instead of `lxml` for URL previews. This offers
some nicer APIs when parsing HTML and avoids using `libxml`, [which is
unmaintained](https://gitlab.gnome.org/GNOME/libxml2/-/commit/9c80a89af2fdf4f853892f84e46580f4902658ba).

I haven’t done a full regression against commonly previewed sites, but I
expect this will give similar (or better) results.

beautiulsoup also handles decoding the charset for us, which is less
custom code.

---------

Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
This commit is contained in:
Patrick Cloke
2026-10-02 15:46:34 +01:00
committed by GitHub
co-authored by Andrew Morgan Andrew Morgan
parent 5a8c4b3990
commit 4f5524043c
14 changed files with 428 additions and 655 deletions
+1 -5
View File
@@ -312,7 +312,7 @@ Installing prerequisites on CentOS or Fedora Linux:
```sh
sudo dnf install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
libwebp-devel libxml2-devel libxslt-devel libpq-devel \
libwebp-devel libxslt-devel libpq-devel \
python3-virtualenv libffi-devel openssl-devel python3-devel
sudo dnf group install "Development Tools"
```
@@ -638,10 +638,6 @@ This is critical from a security perspective to stop arbitrary Matrix users
spidering 'internal' URLs on your network. At the very least we recommend that
your loopback and RFC1918 IP addresses are blacklisted.
This also requires the optional `lxml` python dependency to be installed. This
in turn requires the `libxml2` library to be available - on Debian/Ubuntu this
means `apt-get install libxml2-dev`, or equivalent for your OS.
### Backups
Don't forget to take [backups](../usage/administration/backups.md) of your new server!