CVE-2016-9052: TALOS-2016-0266 || Cisco Talos Intelligence Group

Summary

An exploitable stack-based buffer overflow vulnerability exists in the querying functionality of Aerospike Database Server 3.10.0.3. A specially crafted packet can cause a stack-based buffer overflow in the function as_sindex__simatch_by_iname resulting in remote code execution. An attacker can simply connect to the port to trigger this vulnerability.

Tested Versions

Aerospike Database Server 3.10.0.3

Product URLs

https://github.com/aerospike/aerospike-server/tree/3.10.0.3

CVSSv3 Score

9.8 - CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H

CWE

CWE-121 - Stack-based Buffer Overflow

Details

Aerospike Database Server is both a distributed and scalable NoSQL database that is used as a back-end for scalable web applications that need a key-value store. With a focus on performance, it is multi-threaded and retains its indexes entirely in ram with the ability to persist data to a solid-state drive or traditional rotational media.

When processing a packet from the client, the server will execute the thr_demarshal function. After accepting a connection on the socket, the server will read the header from the packet and check it’s protocol type. If its protocol type specifies that the packet is compressed (PROTO_TYPE_AS_MSG_COMPRESSED), it will decompress it with zlib and then continue to process the packet [1]. Later, when the protocol type is PROTO_TYPE_AS_MSG the server will pass the packet to the thr_tsvc_process_or_enqueue function [2].

as/src/base/thr_demarshal.c:389
void *
thr_demarshal(void *unused)
{
...
    // Demarshal transactions from the socket.
...
        // Iterate over all events.
        for (i = 0; i < nevents; i++) {
...
                // If pointer is NULL, then we need to create a transaction and
                // store it in the buffer.
                if (fd_h->proto == NULL) {
...
                    // Do a preliminary read of the header into a stack-
                    // allocated structure, so that later on we can allocate the
                    // entire message buffer.
                    if (0 >= (n = cf_socket_recv(sock, &proto, sizeof(as_proto), MSG_WAITALL))) {
                        cf_detail(AS_DEMARSHAL, "proto socket: read header fail: error: rv %d sz was %d errno %d", n, sz, errno);
                        goto NextEvent_FD_Cleanup;
                    }
...
                // Check for a finished read.
                if (0 == fd_h->proto_unread) {
...
                    // Check if it's compressed.
                    if (tr.msgp->proto.type == PROTO_TYPE_AS_MSG_COMPRESSED) {      // [1]
...
                    }
...
                    // Either process the transaction directly in this thread,
                    // or queue it for processing by another thread (tsvc/info).
                    if (0 != thr_tsvc_process_or_enqueue(&tr)) {                    // [2]
                        cf_warning(AS_DEMARSHAL, "Failed to queue transaction to the service thread");
                        goto NextEvent_FD_Cleanup;
                    }

The thr_tsvc_process_or_enqueue function will first read the namespace from the packet then and check to see the data is configured to be stored in memory by calling the as_msg_peek_data_in_memory function [1]. If the namespace is undefined or configured to not be stored in memory, the function will continue by calling into process_transaction [2]. In order to trigger this particular vulnerability, this is the path that must be taken.

as/src/base/thr_tsvc.c:497
int
thr_tsvc_process_or_enqueue(as_transaction *tr)
{
    // If transaction is for data-in-memory namespace, process in this thread.
    if (g_config.allow_inline_transactions &&
            g_config.n_namespaces_in_memory != 0 &&
                    (g_config.n_namespaces_not_in_memory == 0 ||
                            as_msg_peek_data_in_memory(&tr->msgp->msg))) {  // [1]
        process_transaction(tr);                                            // [2]
        return 0;
    }
...

Inside the process_transaction function, the server will use the string defined by the AS_MSG_FIELD_TYPE_NAMESPACE field to determine the namespace. Once the namespace is discovered, the server will determine what type of transaction request is being made. This is done by checking which fields are defined. After determining the transaction is a multi-record type by checking that the AS_MSG_FIELD_BIT_KEY and AS_MSG_FIELD_BIT_DIGEST_RIPE fields are not included [1], the transaction will be checked to see if it’s a batched transaction. This is done by checking to see if the AS_MSG_FIELD_BIT_DIGEST_RIPE_ARRAY is included [2]. Finally after checking for those fields, the function will call as_transaction_is_query [3]. The as_transaction_is_query function will then check for the AS_MSG_FIELD_BIT_INDEX_RANGE field being set and if so will pass execution to the as_query function at [4].

as/src/base/thr_tsvc.c:71
void
process_transaction(as_transaction *tr)
{
...
    // All transactions must have a namespace.
    as_msg_field *nf = as_msg_field_get(m, AS_MSG_FIELD_TYPE_NAMESPACE);
...
    as_namespace *ns = as_namespace_get_bymsgfield(nf);
...
    if (as_transaction_is_multi_record(tr)) {           // [1] \
...
        if (as_transaction_is_batch_direct(tr)) {       // [2] \
...
        }
        else if (as_transaction_is_query(tr)) {         // [3] \
            // Query.
...
            if (as_query(tr, ns) != 0) {                // [4]
...


\ [1]
as/include/base/transaction.h:265
static inline bool
as_transaction_is_multi_record(const as_transaction *tr)
{
    return  (tr->msg_fields & (AS_MSG_FIELD_BIT_KEY | AS_MSG_FIELD_BIT_DIGEST_RIPE)) == 0 &&
            (tr->from_flags & FROM_FLAG_BATCH_SUB) == 0;
}

\ [2]
as/include/base/transaction.h:272
static inline bool
as_transaction_is_batch_direct(const as_transaction *tr)
{
    // Assumes we're already multi-record.
    return (tr->msg_fields & AS_MSG_FIELD_BIT_DIGEST_RIPE_ARRAY) != 0;
}

\ [3]
as/include/base/transaction.h:265
static inline bool
as_transaction_is_query(const as_transaction *tr)
{
    // Assumes we're already multi-record.
    return (tr->msg_fields & AS_MSG_FIELD_BIT_INDEX_RANGE) != 0;
}

At the beginning of the as_query function, the application will hand-off the transaction to the query_setup function [1]. Inside this function, the server will ensure that the requested namespace has a secondary index associated with it by calling the as_sindex_ns_has_sindex [2]. After this is determined, the namespace and the packet itself will be passed to the as_sindex_from_msg function [3].

as/src/base/thr_query.c:2856
int
as_query(as_transaction *tr, as_namespace *ns)
{
    if (tr) {
        QUERY_HIST_INSERT_DATA_POINT(query_txn_q_wait_hist, tr->start_time);
    }

    as_query_transaction *qtr;
    int rv = query_setup(tr, ns, &qtr);                             // [1] \
...
\
as/src/base/thr_query.c:2686
static int
query_setup(as_transaction *tr, as_namespace *ns, as_query_transaction **qtrp)
{
...
    bool has_sindex   = as_sindex_ns_has_sindex(ns);                // [2]
    if (!has_sindex) {
        tr->result_code = AS_PROTO_RESULT_FAIL_INDEX_NOTFOUND;
        cf_debug(AS_QUERY, "No Secondary Index on namespace %s", ns->name);
        goto Cleanup;
    }

    // TODO - still lots of redundant msg field parsing (e.g. for set) - fix.
    if ((si = as_sindex_from_msg(ns, &tr->msgp->msg)) == NULL) {    // [3]
        cf_debug(AS_QUERY, "No Index Defined in the Query");
    }

Upon receiving the namespace and the packet, the server will extract the fields identified by the enumerations AS_MSG_FIELD_TYPE_INDEX_NAME [1] and AS_MSG_FIELD_TYPE_SET [2] from the packet. Afterwards, the string stored in the AS_MSG_FIELD_TYPE_INDEX_NAME field is then passed to the as_sindex_lookup_by_iname function [3].

as/src/base/secondary_index.c:2591
as_sindex *
as_sindex_from_msg(as_namespace *ns, as_msg *msgp)
{
    cf_debug(AS_SINDEX, "as_sindex_from_msg");
    as_msg_field *ifp  = as_msg_field_get(msgp, AS_MSG_FIELD_TYPE_INDEX_NAME);              // [1]
    as_msg_field *sfp  = as_msg_field_get(msgp, AS_MSG_FIELD_TYPE_SET);                     // [2]

...
    iname         = cf_strndup((const char *)ifp->data, as_msg_field_get_value_sz(ifp));

    as_sindex *si = as_sindex_lookup_by_iname(ns, iname, AS_SINDEX_LOOKUP_FLAG_ISACTIVE);   // [3]

Both the ns representing the namespace and the iname variable containing the secondary index name extracted from the packet will then be passed to the as_sindex_lookup_by_iname. This server will continue to hand-off these arguments through the function call at [2] and then at [3].

as/src/base/secondary_index.c:1068
as_sindex *
as_sindex_lookup_by_iname(as_namespace *ns, char * iname, char flag)
{
    return as_sindex__lookup(ns, iname, NULL, -1, 0, 0, NULL, flag);                                // [1] \
}
\
as/src/base/secondary_index.c:1058
as_sindex *
as_sindex__lookup(as_namespace *ns, char *iname, char *set, int binid, as_sindex_ktype type,
                        as_sindex_type itype, char * path, char flag)
{
    SINDEX_GRLOCK();
    as_sindex *si = as_sindex__lookup_lockfree(ns, iname, set, binid, type, itype, path, flag);     // [2] \
    SINDEX_GUNLOCK();
    return si;
}
\
as/src/base/secondary_index.c:1003
as_sindex *
as_sindex__lookup_lockfree(as_namespace *ns, char *iname, char *set, int binid,
                                as_sindex_ktype type, as_sindex_type itype, char * path, char flag)
{
...
    int simatch   = -1;
    as_sindex *si = NULL;
    // If iname is not null then search in iname hash and store the simatch
    if (iname) {
        simatch   = as_sindex__simatch_by_iname(ns, iname);                                         // [3]
    }
...

Finally the as_sindex__simatch_by_iname function will be called with the namespace and secondary index name from the packet. This function will write the index name to the iname buffer which has a maximum size of 0x100 bytes [1]. The index name is then written into the buffer using snprintf and a buffer length based on the strlen of the string within the packet [2]. Due to the server using the length of the string from the packet as the bounds for the buffer instead of the size of the buffer itself, a stack-based buffer overflow can be made to occur.

as/include/base/datamodel.h:129
#define AS_ID_INAME_SZ 256

as/src/base/secondary_index.c:982
int
as_sindex__simatch_by_iname(as_namespace *ns, char *idx_name)
{
    int simatch = -1;
    char iname[AS_ID_INAME_SZ]; memset(iname, 0, AS_ID_INAME_SZ);       // [1]
    snprintf(iname, strlen(idx_name) + 1, "%s", idx_name);              // [2]
    int rv      = shash_get(ns->sindex_iname_hash, (void *)iname, (void *)&simatch);
    cf_detail(AS_SINDEX, "Found iname simatch %s->%d rv=%d", iname, simatch, rv);

    if (rv) {
        return -1;
    }
    return simatch;
}

Crash Information

# gdb -q -p `systemctl status aerospike.service | grep 'Main PID' | cut -d: -f2- | cut -d' ' -f2`
...

(gdb) b as_sindex__simatch_by_iname
Breakpoint 5 at 0x506331: file base/secondary_index.c, line 985.

(gdb) c
Continuing.
[Switching to Thread 0x7f8d0cf77700 (LWP 43832)]

Breakpoint 5, as_sindex__simatch_by_iname (ns=0x7f8d92983010,
    idx_name=0x7f8d040c4300 'A' <repeats 200 times>...)
    at base/secondary_index.c:985
985             int simatch = -1;
(gdb) next
986             char iname[AS_ID_INAME_SZ]; memset(iname, 0, AS_ID_INAME_SZ);
(gdb) next
987             snprintf(iname, strlen(idx_name) + 1, "%s", idx_name);
(gdb) p sizeof(iname)
$1 = 0x100
(gdb) p strlen(idx_name)+1
$2 = 0x201
(gdb) dq $rbp L2
7f8d0cf73460 | 00007f8d0cf734c0 0000000000506497 | .4.......dP.....
(gdb) next
988             int rv      = shash_get(ns->sindex_iname_hash, (void *)iname, (void *)&simatch);
(gdb) dq $rbp L1
7f8d0cf73460 | 4242424242424242 4242424242424242 | BBBBBBBBBBBBBBBB
(gdb) finish
Run till exit from #0  as_sindex__simatch_by_iname (ns=0x4242424242424242,
    idx_name=0x4242424242424242 <Address 0x4242424242424242 out of bounds>)
    at base/secondary_index.c:988
Warning:
Cannot insert breakpoint 0.
Error accessing memory address 0x4242424242424242: Input/output error.

Exploit Proof-of-Concept

To execute the proof-of-concept (note: this is only provided to the vendor), simply extract and run it as follows:

$ python poc hostname:3000 $namespace
Trying to connect to hostname:3000
Sending 0x232 byte packet... done.

A client packet for Aerospike server has the following structure. The first 2 bytes describe the protocol version and the protocol type. The version must be 0x02, where the protocol type can be one of two values. If AS_COMPRESSED_MSG(0x04) is specified, then the contents of data are zlib-encoded. Otherwise, the AS_MSG(0x03) value is used. The size of this data is defined by the sz field which is a 48-bit unsigned integer.

<class aspie.as_proto_s>
[0] <instance aspie.proto_version 'version'> v2(0x2)
[1] <instance aspie.proto_type 'type'> AS_MSG(0x3)
[2] <instance uint48_t 'sz'> +0x00000000022e (558)
[8] <instance aspie.as_msg_s 'data'> "\x00\x00\x00\x00\x00\x00\x00 ..skipped ~538 bytes.. \x42\x42\x42\x42\x42\x42\x42"

The contents of the data field has the following structure. In order to submit a message that passes the checks at as_transaction_is_multi_record and as_transaction_is_query, There simply needs to be a field with the NAMESPACE(0x0) id, one with an INDEX_RANGE(0x16) id, and no fields that use the BIT_KEY(2) or BIT_DIGEST_RIPE(4) identifiers. The field that is being used to overflow with is using the INDEX_NAME(0x15) id. This means that there must be at least three fields defined and thus the uint16_t field n_fields must be set to 0x0003 or more.

<class aspie.as_msg_s> 'data'
[8] <instance uint8_t 'header_sz'> +0x00 (0)
[9] <instance aspie.AS_MSG_INFO1 'info1'> {bits=8} (0x00, 8)
[a] <instance aspie.AS_MSG_INFO2 'info2'> {bits=8} (0x00, 8)
[b] <instance aspie.AS_MSG_INFO3 'info3'> {bits=8} (0x00, 8)
[c] <instance uint8_t 'unused'> +0x00 (0)
[d] <instance uint8_t 'result_code'> +0x00 (0)
[e] <instance uint32_t 'generation'> +0x00000000 (0)
[12] <instance uint32_t 'record_ttl'> +0x00000000 (0)
[16] <instance uint32_t 'transaction_ttl'> +0x00000000 (0)
[1a] <instance uint16_t 'n_fields'> +0x0003 (3)
[1c] <instance uint16_t 'n_ops'> +0x0000 (0)
[1e] <instance array(aspie.as_msg_field_s,3) 'fields'> aspie.as_msg_field_s[3] "\x00\x00\x00\x09\x00\x58\x58 ..skipped ~516 bytes.. \x42\x42\x42\x42\x42\x42\x42"
[236] <instance array(aspie.as_msg_op_s,0) 'ops'> aspie.as_msg_op_s[0] ""

Af offset 0x1e of the packet is the definition of fields. This is an array of fields that provide options for the type of request that is being made. The field identified by NAMESPACE(0x0) contains a namespace that supports the configuration defined above.

<class aspie.as_msg_field_s> '0'
[1e] <instance uint32_t 'field_sz'> +0xXXXXXXXX (X)
[22] <instance aspie.AS_MSG_FIELD_TYPE 'type'> NAMESPACE(0x0)
[23] <instance aspie.as_msg_namespace_s<char_t> 'data'> ...

One of the requirements is that an INDEX_RANGE(0x16) field must be defined. This begins at offset 0x2b of the packet generated by the proof-of-concept.

<class aspie.as_msg_field_s> '1'
[2b] <instance uint32_t 'field_sz'> +0x00000002 (2)
[2f] <instance aspie.AS_MSG_FIELD_TYPE 'type'> INDEX_RANGE(0x16)
[30] <instance aspie.as_msg_index_range_s 'data'> "\x00"

The last field that is used to overflow the 0x100 byte buffer has the identifier of INDEX_NAME(0x15). As long as the length defined in field_sz is larger than 0x200 (exclusive) and the contents of data contains the same number of bytes, this vulnerability is being triggered.

<class aspie.as_msg_field_s> '2'
[31] <instance uint32_t 'field_sz'> +0x00000201 (513)
[35] <instance aspie.AS_MSG_FIELD_TYPE 'type'> INDEX_NAME(0x15)
[36] <instance aspie.as_msg_index_name_s<char_t> 'data'> ...

Timeline

2016-12-23 - Vendor Disclosure
2017-01-09 - Public Release

Discovered by the Cisco Talos Team.