Headline
GHSA-wpqr-jcpx-745r: Incorrect handling of invalid surrogate pair characters
Impact
What kind of vulnerability is it? Who is impacted?
Anyone parsing JSON from an untrusted source is vulnerable.
JSON strings that contain escaped surrogate characters not part of a proper surrogate pair were decoded incorrectly. Besides corrupting strings, this allowed for potential key confusion and value overwriting in dictionaries.
Examples:
# An unpaired high surrogate character is ignored.
>>> ujson.loads(r'"\uD800"')
''
>>> ujson.loads(r'"\uD800hello"')
'hello'
# An unpaired low surrogate character is preserved.
>>> ujson.loads(r'"\uDC00"')
'\udc00'
# A pair of surrogates with additional non surrogate characters pair up in spite of being invalid.
>>> ujson.loads(r'"\uD800foo bar\uDC00"')
'foo bar𐀀'
Patches
Has the problem been patched? What versions should users upgrade to?
Users should upgrade to UltraJSON 5.4.0.
From version 5.4.0, UltraJSON decodes lone surrogates in the same way as the standard library’s json module does, preserving them in the parsed output:
>>> ujson.loads(r'"\uD800"')
'\ud800'
>>> ujson.loads(r'"\uD800hello"')
'\ud800hello'
>>> ujson.loads(r'"\uDC00"')
'\udc00'
>>> ujson.loads(r'"\uD800foo bar\uDC00"')
'\ud800foo bar\udc00'
Workarounds
Is there a way for users to fix or remediate the vulnerability without upgrading?
Short of switching to an entirely different JSON library, there are no safe alternatives to upgrading.
For more information
If you have any questions or comments about this advisory:
- Open an issue in UltraJSON
Impact
What kind of vulnerability is it? Who is impacted?
Anyone parsing JSON from an untrusted source is vulnerable.
JSON strings that contain escaped surrogate characters not part of a proper surrogate pair were decoded incorrectly. Besides corrupting strings, this allowed for potential key confusion and value overwriting in dictionaries.
Examples:
# An unpaired high surrogate character is ignored. >>> ujson.loads(r’"\uD800"’) ‘’ >>> ujson.loads(r’"\uD800hello"’) ‘hello’
# An unpaired low surrogate character is preserved. >>> ujson.loads(r’"\uDC00"’) ‘\udc00’
# A pair of surrogates with additional non surrogate characters pair up in spite of being invalid. >>> ujson.loads(r’"\uD800foo bar\uDC00"’) ‘foo bar𐀀’
Patches
Has the problem been patched? What versions should users upgrade to?
Users should upgrade to UltraJSON 5.4.0.
From version 5.4.0, UltraJSON decodes lone surrogates in the same way as the standard library’s json module does, preserving them in the parsed output:
>>> ujson.loads(r’"\uD800"’) ‘\ud800’ >>> ujson.loads(r’"\uD800hello"’) ‘\ud800hello’ >>> ujson.loads(r’"\uDC00"’) ‘\udc00’ >>> ujson.loads(r’"\uD800foo bar\uDC00"’) ‘\ud800foo bar\udc00’
Workarounds
Is there a way for users to fix or remediate the vulnerability without upgrading?
Short of switching to an entirely different JSON library, there are no safe alternatives to upgrading.
For more information
If you have any questions or comments about this advisory:
- Open an issue in UltraJSON
References
- GHSA-wpqr-jcpx-745r
Related news
Gentoo Linux Security Advisory 202403-3 - Multiple vulnerabilities have been discovered in UltraJSON, the worst of which could lead to key confusion and value overwriting. Versions greater than or equal to 5.4.0 are affected.
Ubuntu Security Notice 6629-3 - USN-6629-1 fixed vulnerabilities in UltraJSON. This update provides the corresponding updates for Ubuntu 20.04 LTS. It was discovered that UltraJSON incorrectly handled certain input with a large amount of indentation. An attacker could possibly use this issue to crash the program, resulting in a denial of service. Jake Miller discovered that UltraJSON incorrectly decoded certain characters. An attacker could possibly use this issue to cause key confusion and overwrite values in dictionaries. It was discovered that UltraJSON incorrectly handled an error when reallocating a buffer for string decoding. An attacker could possibly use this issue to corrupt memory.
Ubuntu Security Notice 6629-2 - USN-6629-1 fixed vulnerabilities in UltraJSON. This update provides the corresponding updates for Ubuntu 20.04 LTS. It was discovered that UltraJSON incorrectly handled certain input with a large amount of indentation. An attacker could possibly use this issue to crash the program, resulting in a denial of service. Jake Miller discovered that UltraJSON incorrectly decoded certain characters. An attacker could possibly use this issue to cause key confusion and overwrite values in dictionaries. It was discovered that UltraJSON incorrectly handled an error when reallocating a buffer for string decoding. An attacker could possibly use this issue to corrupt memory.
Ubuntu Security Notice 6629-1 - It was discovered that UltraJSON incorrectly handled certain input with a large amount of indentation. An attacker could possibly use this issue to crash the program, resulting in a denial of service. Jake Miller discovered that UltraJSON incorrectly decoded certain characters. An attacker could possibly use this issue to cause key confusion and overwrite values in dictionaries. It was discovered that UltraJSON incorrectly handled an error when reallocating a buffer for string decoding. An attacker could possibly use this issue to corrupt memory.
Red Hat Security Advisory 2022-8864-01 - UltraJSON is an ultra fast JSON encoder and decoder. Issues addressed include a double free vulnerability.
An update for python-ujson is now available for Red Hat OpenStack Platform 16.1.9 (Train) for Red Hat Enterprise Linux (RHEL) 8.2. Red Hat Product Security has rated this update as having a security impact of Moderate. A Common Vulnerability Scoring System (CVSS) base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link(s) in the References section.This content is licensed under the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/). If you distribute this content, or a modified version of it, you must provide attribution to Red Hat Inc. and provide a link to the original. Related CVEs: * CVE-2022-31116: python-ujson: improper decoding of escaped surrogate characters may lead to string corruption, key confusion or value overwriting * CVE-2022-31117: python-ujson: Potential double free of buffer during string decoding
An update for python-ujson is now available for Red Hat OpenStack Platform 16.2.4 (Train) for Red Hat Enterprise Linux (RHEL) 8.4. Red Hat Product Security has rated this update as having a security impact of Moderate. A Common Vulnerability Scoring System (CVSS) base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link(s) in the References section.This content is licensed under the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/). If you distribute this content, or a modified version of it, you must provide attribution to Red Hat Inc. and provide a link to the original. Related CVEs: * CVE-2022-31116: python-ujson: improper decoding of escaped surrogate characters may lead to string corruption, key confusion or value overwriting * CVE-2022-31117: python-ujson: Potential double free of buffer during string decoding
UltraJSON is a fast JSON encoder and decoder written in pure C with bindings for Python 3.7+. Affected versions were found to improperly decode certain characters. JSON strings that contain escaped surrogate characters not part of a proper surrogate pair were decoded incorrectly. Besides corrupting strings, this allowed for potential key confusion and value overwriting in dictionaries. All users parsing JSON from untrusted sources are vulnerable. From version 5.4.0, UltraJSON decodes lone surrogates in the same way as the standard library's `json` module does, preserving them in the parsed output. Users are advised to upgrade. There are no known workarounds for this issue.