Use beautifulsoup4 instead of lxml for URL previews (#19301)

Use `beautifulsoup4` instead of `lxml` for URL previews. This offers
some nicer APIs when parsing HTML and avoids using `libxml`, [which is
unmaintained](https://gitlab.gnome.org/GNOME/libxml2/-/commit/9c80a89af2fdf4f853892f84e46580f4902658ba).

I haven’t done a full regression against commonly previewed sites, but I
expect this will give similar (or better) results.

beautiulsoup also handles decoding the charset for us, which is less
custom code.

---------

Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

.github/workflows/tests.yml

@@ -483,7 +483,7 @@ jobs:
       - run: |
           sudo apt-get -qq update
           sudo apt-get -qq install build-essential libffi-dev python3-dev \
-            libxml2-dev libxslt-dev xmlsec1 zlib1g-dev libjpeg-dev libwebp-dev
+            libxslt-dev xmlsec1 zlib1g-dev libjpeg-dev libwebp-dev
 
       - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
@@ -535,7 +535,7 @@ jobs:
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
       # Install libs necessary for PyPy to build binary wheels for dependencies
-      - run: sudo apt-get -qq install xmlsec1 libxml2-dev libxslt-dev
+      - run: sudo apt-get -qq install xmlsec1 libxslt-dev
       - uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
         with:
           python-version: ${{ matrix.python-version }}