Deep Dive: Protecting Against Container Threats in the Cloud

A deep dive into securing containerized environments and understanding how they present unique security challenges.

Containers are self-contained pods representing complete, portable application environments. They contain everything an application needs to run, including binaries, libraries, configuration files and dependencies (Docker and Amazon Elastic, for instance, are two of the more well-known offerings).

Multiple containers can run on a shared infrastructure and use the same operating system kernel, but they’re abstracted from that layer and have little contact with the underlying hosting resources (which could be, for example, a public cloud instance).

[Editor’s Note: This article was originally published in the free Threatpost eBook “Cloud Security: The Forecast for 2022.” In it we explore organizations’ top risks and challenges, best practices for defense, and advice for security success in such a dynamic computing environment, including handy checklists. Please download the FREE eBook for the full story]

The benefits of running cloud-based containers are varied and include the ability to easily spin applications up and down for users (think “write once, run everywhere” – a big boon for companies managing pandemic-related remote footprints). They also offer major infrastructure cost savings compared with managing applications on owned-and-operated servers or on virtual machines. They also provide increased agility by supporting DevOps goals.

Containers are also easy to manage, thanks to orchestration engines such as Kubernetes. Admins can use orchestration to manage containerized apps and services at scale in a centralized fashion, pushing out automatic updates, isolating any failing containers and the like.

As a result, container adoption is at an all-time high, with companies of all sizes looking to embrace the technology. In just one example, a survey from the Cloud Native Computing Foundation (CNCF) found that 83 percent of respondents were using Kubernetes in production in 2020, up from 78 percent the previous year and just 58 percent in 2018.

As adoption increases, so does the interest of cybercriminals. A June Red Hat survey found that a whopping 94 percent of respondents had suffered a Kubernetes security incident over the previous 12 months.

“Kubernetes attacks are actually quite common, especially given how popular the container orchestration software is,” said Trevor Morgan, product manager at comforte AG. “The array of threats to Kubernetes environments is quite broad.”

He added, “Whether they are state-sponsored agents trying to undermine other political entities or are part of a gang or individual effort to steal for financial gain, the common denominator is always sensitive information. If threat actors can get to sensitive information, they can leverage it to build more complete data subject profiles (to then use for nefarious purposes), to hold information for ransom, and to weaponize it in any number of ways. And don’t underestimate the sheer value of chaos that this all can create. They thrive in environments of fear and chaos.”

****Containers in Cyberattack Sights****

As an example of how popular targeting vulnerable cloud infrastructure has become, Akamai security researcher Larry Cashdollar recently set up a simple Docker container honeypot, just to see what kind of notice it might attract from the wider web’s cadre of cyberattackers. The results were head-turning: The honeypot was used for four different criminal campaigns in the span of 24 hours.

Cashdollar had implemented SSH protocol for encryption and implemented a “guessable” root password. Since it was running a standard cloud container configuration, it wouldn’t stand out on the web as an obvious honeypot, he explained. Instead, it would simply look like a vulnerable cloud instance.

The attacks were varied in terms of their goals: One campaign tried to use the container as a proxy to tap into Twitch streams or access other services, another attempted a botnet infection, another performed cryptomining, and the last effort involved running a work-from-home scam.

As these examples show, “profit is still the primary motivation for cybercriminals targeting containers,” explained Mark Nunnikhoven, distinguished cloud strategist at Lacework. “Malicious actors strive to retrieve access to resources or data they can convert into a profit. Resources like CPU time and bandwidth can be resold to other criminals for underground services, or even can be used to mine cryptocurrency directly. Data can always be sold or ransomed. These motivations don’t change in an environment that heavily leverages containers.”

****Misconfiguration: The Most-Common Container Risk Factor****

Container technology, like other types of infrastructure, can be compromised in a number of different ways – however, misconfiguration reigns atop the initial-access leaderboard. According to a recent Gartner analysis, through 2025, more than 99 percent of cloud breaches will have a root cause of customer misconfigurations or mistakes.

“Containers are often deployed in sets and in very dynamic environments,” Nunnikhoven explained. “The misconfiguration of access, networking and other settings can lead to an opportunity for cybercriminals.”

Trevor Morgan, product manager at comforte AG, noted that companies, especially smaller companies, are generally using default configuration settings vs. more sophisticated and granular configuration capabilities: “Basic misconfigurations or accepting default settings that are far less secure than customized settings.”

That can lead to big (and expensive) problems. For instance, last June the “Siloscape” malware was discovered, which is the first known malware to target Windows containers. It breaks out of Kubernetes clusters to plant backdoors, raid nodes for credentials or even hijack an entire database hosted in a cluster. Its main purpose, Palo Alto Networks Unit 42 researchers said, is opening “a backdoor into poorly configured Kubernetes clusters in order to run malicious containers.”

Configuration woes often extend beyond the containers themselves. Last July, for example, Kubernetes clusters were seen being attacked via misconfigured Argo Workflows instances.

Argo Workflows is an open-source, container-native workflow engine for orchestrating parallel jobs on Kubernetes – to speed up processing time for compute-intensive jobs like machine learning and big-data processing. Malware operators were taking advantage of publicly available dashboards that didn’t require authentication for outside users, according to an analysis from Intezer, in order to drop cryptominers into the cloud.

Compromised Container Images

Nunnikhoven noted that beyond misconfiguration, compromised images or layers are the next most important risk to containers. Images are pre-made, static files with executable code that can create a container on a computing system. They can be made available via open-source repositories for easy deployment.

“Lacework Labs has seen several occurrences of cybercriminals compromising containers either through malware implants or cryptomining programs being pre-installed in the image,” he explained. “When a team deploys those images, the attacker then gains access to the resources of the victim.”

A related case involves a bug found in 2020 in the Containerd runtime tool, which manages the complete container lifecycle of its host system. The bug (CVE-2020-15157) was located in the container image-pulling process, according to Gal Singer, researcher at Aqua. Adversaries could exploit it by building dedicated container images designed to steal the host’s token when they were pulled into a project. Then, they could use the token to take over a cloud project.

Similarly, a denial-of-service issue in one of the Go libraries that Kubernetes is based on (CVE-2021-20291) was found to be triggered by placing a malicious image inside a registry. The DoS condition was created when that image was pulled from the registry by an unsuspecting user.

Bug Parade

The next problem area arises from vulnerabilities, both known and zero-day issues. Several container bugs were identified in 2021, but perhaps the most disconcerting was “Azurescape.”

Unit 42 researchers discovered a chain of exploits that could allow a malicious Azure user to infiltrate other customers’ cloud instances within Microsoft’s multitenant container-as-a-service offering. This critical crossaccount container takeover was described as a “nightmare scenario for the public cloud.”

“Azurescape is evidence that they’re more real than we’d like to think,” according to Unit 42. “Cloud providers invest heavily in securing their platforms, but it’s inevitable that unknown zero-day vulnerabilities would exist and put customers at risk.”

****Best Practices for Container Defense****

Containerized environments can present unique challenges for observability and in the application of security controls, Nunnikhoven noted, but following a layered security approach can help.

“Given the speed of change and the scale of these environments, organizations must be able to quickly analyze the operational data looking for abnormal behaviors,” he said. “The traditional approach of having a list of ‘bad’ things to look for won’t work in a container-based environment.”

To protect one’s Kubernetes assets, users should implement a laundry list of best practices, researchers advised:

• Keep cluster infrastructure patched;
• Avoid default configurations;
• Employ strong passwords;
• Refrain from sending privileged service accounts tokens to anyone but the API server to prevent attackers from masquerading as the token owner;
• Enable the “BoundServiceAccountTokenVolume” feature: When a pod terminates, its token is no longer valid, minimizing the impact of token theft;
• Deploy policy enforcers to monitor and prevent suspicious activity within clusters, especially service accounts or nodes that query the SelfSubjectAccessReview or SelfSubjectRulesReview APIs for their permissions;
• Pull container images from reputable sources, stored in secured repositories, tagged and signed with trust certificates. When new versions become available, archive outdated versions from the repositories;
• Evaluate orchestrators for least-privilege configurations to ensure that movements within CI/CD are authenticated, logged and monitored;
• Be holistic: Create a consolidated view of risk across cloud-application environments as well as traditional IT infrastructure;
• Have data-analysis tooling in place and an automated runbook that can react to the results of that analysis;
• Provide the context and information to your security analysts to make a timely and informed decision, and then run the appropriate automated response; and
• Protect data at ingress and egress.

“As containers multiply, so does the attack surface open up, which gives more entryways into the company’s operational environment,” said comforte AG’s Morgan. “Learn from reported breaches and other incidents. They are not just situations that happen to other companies – your business right now may be sustaining an attack somewhere, perhaps on your container environment. Assume that that’s the case and act accordingly to audit, assess and bolster your defensive posture. The fallout is much more expensive and certainly is damaging to your organization as a whole.”

Moving to the cloud? Discover emerging cloud-security threats along with solid advice for how to defend your assets with our FREE downloadable eBook****, “Cloud Security: The Forecast for 2022.” We explore organizations’ top risks and challenges, best practices for defense, and advice for security success in such a dynamic computing environment, including handy checklists.