Headline
CVE-2023-38686: Enable TLS certificate validation by default for SMTP/IMAP/FTP/POP/NNTP protocols · Issue #91826 · python/cpython
Sydent is an identity server for the Matrix communications protocol. Prior to version 2.5.6, if configured to send emails using TLS, Sydent does not verify SMTP servers’ certificates. This makes Sydent’s emails vulnerable to interception via a man-in-the-middle (MITM) attack. Attackers with privileged access to the network can intercept room invitations and address confirmation emails. This is patched in Sydent 2.5.6. When patching, make sure that Sydent trusts the certificate of the server it is connecting to. This should happen automatically when using properly issued certificates. Those who use self-signed certificates should make sure to copy their Certification Authority certificate, or their self signed certificate if using only one, to the trust store of your operating system. As a workaround, one can ensure Sydent’s emails fail to send by setting the configured SMTP server to a loopback or non-routable address under one’s control which does not have a listening SMTP server.
Feature or enhancement
I was surprised that Python does not verify hostnames by default for the stdlib modules for SMTP, IMAP, FTP, POP and NNTP. I believe the “insecure by default” behavior is no longer appropriate even for those protocol (at least SMTP and IMAP, I’m not too familiar with the rest - but even there, I suppose if an user asks for a secure connection, it should be secure).
In PEP 476 (Enabling certificate verification by default for stdlib http clients, 2014), certificate verification was enabled by default for HTTPS, with the rationale that:
The failure to do these checks means that anyone with a privileged network position is able to trivially execute a man in the middle attack against a Python application using either of these HTTP clients, and change traffic at will.
and
The “S” in “HTTPS” stands for secure. When Python’s users type “HTTPS” they are expecting a secure connection, and Python should adhere to a reasonable standard of care in delivering this. Currently we are failing at this, and in doing so, APIs which appear simple are misleading users.
When asked, many Python users state that they were not aware that Python failed to perform these validations, and are shocked.
and
The failure of various applications to note Python’s negligence in this matter is a source of regular CVE assignment
That PEP only improved that situation for HTTPS, stating that:
This PEP only proposes requiring this level of validation for HTTP clients, not for other protocols such as SMTP.
This is because while a high percentage of HTTPS servers have correct certificates, as a result of the validation performed by browsers, for other protocols self-signed or otherwise incorrect certificates are far more common. Note that for SMTP at least, this appears to be changing and should be reviewed for a potential similar PEP in the future:
https://www.facebook.com/notes/protect-the-graph/the-current-state-of-smtp-starttls-deployment/1453015901605223
https://www.facebook.com/notes/protect-the-graph/massive-growth-in-smtp-starttls-deployment/1491049534468526
Unfortunately, it seems to be very difficult to find more recent data about how many SMTP/IMAP/… servers in the wild present an invalid certificate - all I could find is that according to Google, adoption of outgoing encryption in general went up from ~75% when the PEP was written to ~90% now, and inbound encryption went up from ~57% to ~87%.
However, there seems to be a strong consensus to treat this kind of thing as a vulnerability in clients, even as early as 2007. Some examples:
- CWE - CWE-297: Improper Validation of Certificate with Host Mismatch (4.6)
- CVE - CVE-2007-5770: The (1) Net::ftptls, (2) Net::telnets, (3) Net::imap, (4) Net::pop, and (5) Net::smtp libraries in Ruby 1.8.5 and 1.8.6 do not verify that the commonName (CN) field in a server certificate matches the domain name in a request sent over SSL, which makes it easier for remote attackers to intercept SSL transmissions via a man-in-the-middle attack or spoofed web site, different components than CVE-2007-5162.
- NVD - CVE-2009-3766: mutt_ssl.c in mutt 1.5.16 and other versions before 1.5.19, when OpenSSL is used, does not verify the domain name in the subject’s Common Name (CN) field of an X.509 certificate, which allows man-in-the-middle attackers to spoof SSL servers via an arbitrary valid certificate.
- CVE - CVE-2011-4318: Dovecot 2.0.x before 2.0.16, when ssl or starttls is enabled and hostname is used to define the proxy destination, does not verify that the server hostname matches a domain name in the subject’s Common Name (CN) of the X.509 certificate, which allows man-in-the-middle attackers to spoof SSL servers via a valid certificate for a different hostname.
- CVE - CVE-2011-1429: Mutt does not verify that the smtps server hostname matches the domain name of the subject of an X.509 certificate, which allows man-in-the-middle attackers to spoof an SSL SMTP server via an arbitrary certificate, a different vulnerability than CVE-2009-3766.
- CVE - CVE-2012-2993: Microsoft Windows Phone 7 does not verify the domain name in the subject’s Common Name (CN) field of an X.509 certificate, which allows man-in-the-middle attackers to spoof an SSL server for the (1) POP3, (2) IMAP, or (3) SMTP protocol via an arbitrary valid certificate.
- CVE - CVE-2013-0308: The imap-send command in GIT before 1.8.1.4 does not verify that the server hostname matches a domain name in the subject’s Common Name (CN) or subjectAltName field of the X.509 certificate, which allows man-in-the-middle attackers to spoof SSL servers via an arbitrary valid certificate.
- CVE - CVE-2014-7273 / CVE - CVE-2014-7274: The IMAP-over-SSL implementation in getmail 4.0.0 through 4.43.0 does not verify X.509 certificates from SSL servers, which allows man-in-the-middle attackers to spoof IMAP servers and obtain sensitive information via a crafted certificate.
and more recently:
- CVE - CVE-2020-1758: A flaw was found in Keycloak […], where it does not perform the TLS hostname verification while sending emails using the SMTP server. […]
- CVE - CVE-2020-9488: Improper validation of certificate with host mismatch in Apache Log4j SMTP appender. […]
- CVE - CVE-2020-13163: em-imap 0.5 uses the library eventmachine in an insecure way […]. The hostname in a TLS server certificate is not verified.
- NVD - CVE-2020-15047: MSA/SMTP.cpp in Trojita […] ignores certificate-verification errors, which allows man-in-the-middle attackers to spoof SMTP servers.
- No enforcement of STARTTLS. OfflineIMAP/offlineimap#669
- NVD - CVE-2021-44549: Apache Sling Commons Messaging Mail provides a simple layer on top of JavaMail/Jakarta Mail for OSGi to send mails via SMTPS. To reduce the risk of “man in the middle” attacks additional server identity checks must be performed when accessing mail servers. For compatibility reasons these additional checks are disabled by default in JavaMail/Jakarta Mail. The SimpleMailService in Apache Sling Commons Messaging Mail 1.0 lacks an option to enable these checks for the shared mail session. A user could enable these checks nevertheless by accessing the session via the message created by SimpleMessageBuilder and setting the property mail.smtps.ssl.checkserveridentity to true. Apache Sling Commons Messaging Mail 2.0 adds support for enabling server identity checks and these checks are enabled by default. (quoted in full as probably similar backwards compat concerns like in Python)
Previous discussion
- PEP 476, see above
- My Tweet, where @vstinner encouraged me to open an issue (and perhaps later PR) about this
- SSL releated deprecation for 3.6 #72209 (2016), where @tiran proposed: make ftplib, imaplib, nntplib, pop3lib, smtplib etc. validate certs by default., but @vstinner objected (pointing to the concerns in PEP 476), and so did @ncoghlan. In August 2021, @kousu pointed out that this is still the case.
Related news
Red Hat Security Advisory 2024-8887-03 - An update for Openshift Jenkins is now available for Red Hat Product OCP Tools 4.13. Red Hat Product Security has rated this update as having a security impact of important. A Common Vulnerability Scoring System base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link in the References section. Issues addressed include bypass and denial of service vulnerabilities.
Red Hat Security Advisory 2024-8886-03 - An update for Openshift Jenkins is now available for Red Hat Product OCP Tools 4.12. Red Hat Product Security has rated this update as having a security impact of important. A Common Vulnerability Scoring System base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link in the References section. Issues addressed include bypass and denial of service vulnerabilities.
Red Hat Security Advisory 2024-8885-03 - An update for Openshift Jenkins is now available for Red Hat Product OCP Tools 4.14. Red Hat Product Security has rated this update as having a security impact of important. A Common Vulnerability Scoring System base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link in the References section. Issues addressed include bypass and denial of service vulnerabilities.
Red Hat Security Advisory 2024-8884-03 - An update for Openshift Jenkins is now available for Red Hat Product OCP Tools 4.15. Red Hat Product Security has rated this update as having a security impact of important. A Common Vulnerability Scoring System base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link in the References section. Issues addressed include bypass and denial of service vulnerabilities.
Gentoo Linux Security Advisory 202402-16 - Multiple vulnerabilities have been discovered in Apache Log4j, the worst of which can lead to remote code execution. Versions less than or equal to 1.2.17 are affected.
IBM QRadar SIEM 7.5 is vulnerable to information exposure allowing a delegated Admin tenant user with a specific domain security profile assigned to see data from other domains. This vulnerability is due to an incomplete fix for CVE-2022-34352. IBM X-Force ID: 266808.
## Impact If configured to send emails using TLS, Sydent does not verify SMTP servers' certificates. This makes Sydent's emails vulnerable to interception via a [man-in-the-middle (MITM) attack](https://en.wikipedia.org/wiki/Man-in-the-middle_attack). Attackers with privileged access to the network can intercept room invitations and address confirmation emails. CVSS 3.1 overall score: 3.3 - [AV:A/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:N/CR:L/IR:L/AR:X/MAV:A/MAC:H/MPR:N/MUI:N/MS:C/MC:L/MI:L/MA:N](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:A/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:N/CR:L/IR:L/AR:X/MAV:A/MAC:H/MPR:N/MUI:N/MS:C/MC:L/MI:L/MA:N&version=3.1) _Reported by Martin Schobert, [Pentagrid AG](https://pentagrid.ch/)._ ### Details Sydent can be configured to send emails over a TLS-encrypted socket by setting ```yaml email: tlsmode: "TLS" # or the legacy value "SSL" ``` in its config file. Alternatively it can be configured to use [Opportunistic TLS](https://en.wikipedia.or...
Dell Streaming Data Platform prior to 1.4 contains Open Redirect vulnerability. An attacker with privileges same as a legitimate user can phish the legitimate the user to redirect to malicious website leading to information disclosure and launch of phishing attacks.
Dell PowerStore, versions prior to 3.0.0.0, contains an OS Command Injection vulnerability in PowerStore T environment. A locally authenticated attacker could potentially exploit this vulnerability, leading to the execution of arbitrary OS command on the PowerStore underlying OS. Exploiting may lead to a system take over by an attacker.
Vulnerability in the Oracle Java SE, Oracle GraalVM Enterprise Edition product of Oracle Java SE (component: JNDI). Supported versions that are affected are Oracle Java SE: 7u331, 8u321, 11.0.14, 17.0.2, 18; Oracle GraalVM Enterprise Edition: 20.3.5, 21.3.1 and 22.0.0.2. Easily exploitable vulnerability allows unauthenticated attacker with network access via multiple protocols to compromise Oracle Java SE, Oracle GraalVM Enterprise Edition. Successful attacks of this vulnerability can result in unauthorized update, insert or delete access to some of Oracle Java SE, Oracle GraalVM Enterprise Edition accessible data. Note: This vulnerability applies to Java deployments, typically in clients running sandboxed Java Web Start applications or sandboxed Java applets, that load and run untrusted code (e.g., code that comes from the internet) and rely on the Java sandbox for security. This vulnerability can also be exploited by using APIs in the specified Component, e.g., through a web service ...
Apache Log4j2 2.0-beta9 through 2.15.0 (excluding security releases 2.12.2, 2.12.3, and 2.3.1) JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. From version 2.16.0 (along with 2.12.2, 2.12.3, and 2.3.1), this functionality has been completely removed. Note that this vulnerability is specific to log4j-core and does not affect log4net, log4cxx, or other Apache Logging Services projects.
Vulnerability in the Oracle Database Enterprise Edition Unified Audit component of Oracle Database Server. Supported versions that are affected are 12.1.0.2, 12.2.0.1 and 19c. Easily exploitable vulnerability allows high privileged attacker having Local Logon privilege with network access via Oracle Net to compromise Oracle Database Enterprise Edition Unified Audit. Successful attacks of this vulnerability can result in unauthorized update, insert or delete access to some of Oracle Database Enterprise Edition Unified Audit accessible data. CVSS 3.1 Base Score 2.7 (Integrity impacts). CVSS Vector: (CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:N/I:L/A:N).
Fetchmail before 6.4.22 fails to enforce STARTTLS session encryption in some circumstances, such as a certain situation with IMAP and PREAUTH.
In Alpine before 2.25, untagged responses from an IMAP server are accepted before STARTTLS.
Vulnerability in the MySQL Server product of Oracle MySQL (component: Server: DML). Supported versions that are affected are 5.7.33 and prior. Easily exploitable vulnerability allows high privileged attacker with network access via multiple protocols to compromise MySQL Server. Successful attacks of this vulnerability can result in unauthorized ability to cause a hang or frequently repeatable crash (complete DOS) of MySQL Server. CVSS 3.1 Base Score 4.9 (Availability impacts). CVSS Vector: (CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:N/I:N/A:H).
Vulnerability in the Java SE, Java SE Embedded, Oracle GraalVM Enterprise Edition product of Oracle Java SE (component: Libraries). Supported versions that are affected are Java SE: 7u291, 8u281, 11.0.10, 16; Java SE Embedded: 8u281; Oracle GraalVM Enterprise Edition: 19.3.5, 20.3.1.2 and 21.0.0.2. Difficult to exploit vulnerability allows unauthenticated attacker with network access via multiple protocols to compromise Java SE, Java SE Embedded, Oracle GraalVM Enterprise Edition. Successful attacks require human interaction from a person other than the attacker. Successful attacks of this vulnerability can result in unauthorized creation, deletion or modification access to critical data or all Java SE, Java SE Embedded, Oracle GraalVM Enterprise Edition accessible data. Note: This vulnerability applies to Java deployments that load and run untrusted code (e.g., code that comes from the internet) and rely on the Java sandbox for security. CVSS 3.1 Base Score 5.3 (Integrity impacts). CV...
Vulnerability in the Oracle VM VirtualBox product of Oracle Virtualization (component: Core). The supported version that is affected is Prior to 6.1.18. Easily exploitable vulnerability allows high privileged attacker with logon to the infrastructure where Oracle VM VirtualBox executes to compromise Oracle VM VirtualBox. While the vulnerability is in Oracle VM VirtualBox, attacks may significantly impact additional products. Successful attacks of this vulnerability can result in unauthorized access to critical data or complete access to all Oracle VM VirtualBox accessible data. CVSS 3.1 Base Score 6.0 (Confidentiality impacts). CVSS Vector: (CVSS:3.1/AV:L/AC:L/PR:H/UI:N/S:C/C:H/I:N/A:N).
Vulnerability in the MySQL Server product of Oracle MySQL (component: InnoDB). Supported versions that are affected are 8.0.21 and prior. Easily exploitable vulnerability allows high privileged attacker with network access via multiple protocols to compromise MySQL Server. Successful attacks of this vulnerability can result in unauthorized ability to cause a hang or frequently repeatable crash (complete DOS) of MySQL Server. CVSS 3.1 Base Score 4.9 (Availability impacts). CVSS Vector: (CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:N/I:N/A:H).
Vulnerability in the Oracle Database - Enterprise Edition component of Oracle Database Server. Supported versions that are affected are 12.1.0.2, 12.2.0.1, 18c and 19c. Easily exploitable vulnerability allows high privileged attacker having DBA role account privilege with network access via Oracle Net to compromise Oracle Database - Enterprise Edition. While the vulnerability is in Oracle Database - Enterprise Edition, attacks may significantly impact additional products. Successful attacks of this vulnerability can result in unauthorized update, insert or delete access to some of Oracle Database - Enterprise Edition accessible data. CVSS 3.1 Base Score 4.1 (Integrity impacts). CVSS Vector: (CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:C/C:N/I:L/A:N).