Unicode strings with encoding declaration are not supported (some mails are not properly displayed)

[mirtouf]

Hi,

my setup is

• OS Type: Debian
• OS Version: 11
• Database Type: MySQL
• Database version: 10.5.23
• Modoboa: 2.3.2
• installer used: No
• Webserver: Nginx

Steps to reproduce

Some mails in the webmail cannot be displayed

Current behavior

Web interface is throwing an error (see #8 for another example)

ValueError at /webmail/getmailcontent Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.Request Method: GET Request URL: https://mail-admin.mirtouf.fr/webmail/getmailcontent?mbox=INBOX&mailid=4216&links=1 Django Version: 4.2.8 Exception Type: ValueError Exception Value: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration. Exception Location: src/lxml/parser.pxi, line 1911, in lxml.etree._parseMemoryDocument Raised during: modoboa_webmail.views.getmailcontent Python Executable: /usr/bin/uwsgi-core Python Version: 3.9.2 Python Path: ['.', '', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/modoboa/env/lib/python3.9/site-packages'] Server time: Fri, 13 Sep 2024 18:50:51 +0000

Traceback
`Environment:

Request Method: GET
Request URL: https://mail-admin.mirtouf.fr/webmail/getmailcontent?mbox=INBOX&mailid=4216&links=1

Django Version: 4.2.8
Python Version: 3.9.2
Installed Applications:
('django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.sites',
'django.contrib.staticfiles',
'reversion',
'ckeditor',
'ckeditor_uploader',
'oauth2_provider',
'corsheaders',
'rest_framework',
'rest_framework.authtoken',
'drf_spectacular',
'phonenumber_field',
'django_otp',
'django_otp.plugins.otp_totp',
'django_otp.plugins.otp_static',
'django_rename_app',
'django_rq',
'modoboa',
'modoboa.core',
'modoboa.lib',
'modoboa.admin',
'modoboa.transport',
'modoboa.relaydomains',
'modoboa.limits',
'modoboa.parameters',
'modoboa.dnstools',
'modoboa.policyd',
'modoboa.maillog',
'modoboa.dmarc',
'modoboa.pdfcredentials',
'modoboa.imap_migration',
'modoboa.postfix_autoreply',
'modoboa.sievefilters',
'modoboa_radicale',
'modoboa_webmail',
'modoboa_contacts',
'webpack_loader')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'x_forwarded_for.middleware.XForwardedForMiddleware',
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'modoboa.core.middleware.LocalConfigMiddleware',
'modoboa.lib.middleware.AjaxLoginRedirect',
'modoboa.lib.middleware.CommonExceptionCatcher',
'modoboa.lib.middleware.RequestCatcherMiddleware')

Traceback (most recent call last):
File "/home/modoboa/env/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/home/modoboa/env/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/modoboa/env/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
return view_func(request, *args, **kwargs)
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa/admin/lib.py", line 46, in wrapped_f
return f(request, *args, **kwargs)
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa_webmail/views.py", line 553, in getmailcontent
"mailbody": email.body if email.body else ""
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa_webmail/lib/imapemail.py", line 168, in body
bodyc = getattr(self, "post_process%s" % self.mformat)(bodyc)
File "/home/modoboa/env/lib/python3.9/site-packages/modoboa/lib/email_utils.py", line 229, in _post_process_html
html = lxml.html.fromstring(content)
File "/home/modoboa/env/lib/python3.9/site-packages/lxml/html/init.py", line 873, in fromstring
doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
File "/home/modoboa/env/lib/python3.9/site-packages/lxml/html/init.py", line 759, in document_fromstring
value = etree.fromstring(html, parser, **kw)
File "src/lxml/etree.pyx", line 3264, in lxml.etree.fromstring

File "src/lxml/parser.pxi", line 1911, in lxml.etree._parseMemoryDocument

Exception Type: ValueError at /webmail/getmailcontent
Exception Value: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.`

Expected behavior

Mail is displayed properly or at least without such big errors

Video/Screenshot link (optional)