You can try to hide your firmware from Kelly Patterson, but she’ll find it (and break it)

Monday, September 11, 2023 08:09

How her work illustrates the difference Talos’ vulnerability research team makes

When Kelly Patterson first learned how to code by making small programs in her high school class, she preferred breaking her creations to building them.

She’d make a game and then spend double the time debugging that same code, looking for holes in her work.

Today, she’s always looking for what’s wrong with other people’s code, whether that be in a wireless router, IoT speaker, or an open-source software stack that dates back to 1991.

Patterson (né Leuschner) is one of the researchers that make up Talos’ Vulnerability Discovery team, a group of reverse-engineers, penetration testers and general expert coders who look for vulnerabilities in firmware, software and hardware and help the creators fix those issues.

Patterson and her teammates are responsible for helping to disclose and patch more than 200 security vulnerabilities a year, some of which affect devices used in thousands of households around the world, and others that support everything from industrial control systems to critical infrastructure.

Specifically for Patterson, she enjoys looking at hardware and its accompanying firmware. She began her IT career as a systems engineer but quickly found that she was more interested in debugging what she was working on, so she started pursuing projects outside of the office that allowed her to reverse-engineer code and talk about it publicly. This eventually led her to Talos, which was specifically attractive because it allowed her to be a researcher full-time.

“I like to spread the word that these bugs are still out there and we’re finding them, proving that we haven’t ‘solved’ security completely,” Patterson said.

One of her first and most memorable projects at Talos was looking at a series of programmable logic controllers (PLCs) made by WAGO, a German company specializing in automation solutions. She teamed up with other researchers to approach these devices from different angles, trying to dissect what attack surface existed, exactly. By the time their research was public, they had found several critical vulnerabilities in two WAGO PLCs that could allow a remote, unauthenticated attacker to execute arbitrary code on the devices or cause a denial of service by sending specially crafted packets.

Patterson performing with her improv troupe.

Patterson specifically focused on the cloud-connected portion of the software, which opened her eyes to the attack surface that cloud storage and communication presents.

“That was fun research for me. It helped open my eyes to the fact that the cloud is an attack vector for embedded devices,” she said. “Myself, Carl [Hurd] and Patrick [DeSantis] split it up and came at it from different angles — it was a great device to research because it was so customizable and had a huge attack surface.”

That research led her to look at more industrial control systems and internet-of-things devices that are virtually always on and talking to the network. Right now she’s examining various open-source software stacks that ICS environments typically use and has multiple fuzzers running to search for potential code vulnerabilities.

As Patterson puts it, not all code bugs are vulnerabilities, but all vulnerabilities are bugs — it’s up to her to determine if a bug could allow a bad actor to carry out any undesirable action. Once she confirms a vulnerability exists, her team reports it to the vendor (all adhering to Cisco’s third-party vulnerability disclosure policy) and works with them to create a patch. Then, she also has to confirm that that patch works and fixes the issue, which isn’t always the case.

“I think a lot of times vendors patch the bugs we find, but I think it provides a way for other developers to look into what kind of bugs they’re hiding and accidentally actually designing and creating into their products. Hopefully, this information helps them so other vendors’ bugs don’t end up in the wild,” she said.

When she starts any research into a particular product, her endgame is usually to gain access to the firmware. Many times, vendors will do everything they can to hide the firmware, so she’ll develop a method to intercept a firmware update or look for ways she can physically access the device’s inner workings to exploit any vulnerabilities that exist there.

“I try to think of things that I know are commonly used and would have a large impact if they were compromised. In the past, that’s been a lot of ICS,” Patterson said. “Or maybe it’s a new architecture or framework that I haven’t worked with before.”

All this research has made Patterson slightly more paranoid than the average user — she always opts for the “dumb” version of any appliance or electronic she brings into her home to limit the number of devices connected to her home network. But she doesn’t balk at using certain IoT devices like home assistants or smart speakers, either, because she trusts certain manufacturers’ internal testing teams who look for vulnerabilities before a product is released.

Though she’s now four-plus years into her career as a vulnerability researcher, Patterson was not always sure she was going to stay on this path.

During the COVID-19 pandemic, she elected to leave her role at Talos to be a stay-at-home mom while her children couldn’t attend school in person.

“I was scared — it was a really tough time,” Patterson said. “And I hadn’t ever been solely responsible for caring for my children all the time. But it ended up being a really good experience.”

Patterson spent about 21 months away from work, during which she questioned whether she wanted to keep at her vulnerability research or look down another IT-related career path. After taking on some personal projects and home and mulling it over, she decided to re-apply for an opening at Talos and re-joined the vulnerability research team in September 2022 part-time.

She currently works a half-time schedule to balance her duties as a mom and her work.

“That was a lot of personal growth for me to figure out a work-life balance, which I never was forced to do before until [crap] hit the fan,” Patterson said.

Patterson has several interests outside of work, including rock climbing.

Patterson said her current goals are built around writing and creating new tools she can use to examine firmware and software, she says she often learns more from the process of building those tools than examining the vulnerabilities themselves.

And while she does enjoy breaking apart routers every now and then, she said she views her work creating fuzzers and other tools for examining code as part of the larger vulnerability puzzles her team works on.

"That’s the major benefit of working on a team is that we all have our own specialties,” she said. “It can feel challenging because you can feel like you get stuck, but that’s where other people come in to make suggestions or push you over the hump.”