Headline
Microsoft Response to Distributed Denial of Service (DDoS) Attacks against HTTP/2
Summary Summary Beginning in September 2023, Microsoft was notified by industry partners about a newly identified Distributed Denial-of-Service (DDoS) attack technique being used in the wild targeting HTTP/2 protocol. This vulnerability (CVE-2023-44487) impacts any internet exposed HTTP/2 endpoints. As an industry leader, Microsoft promptly opened an investigation and subsequently began working with industry partners for a coordinated disclosure and mitigation plan.
Summary Summary
Beginning in September 2023, Microsoft was notified by industry partners about a newly identified Distributed Denial-of-Service (DDoS) attack technique being used in the wild targeting HTTP/2 protocol. This vulnerability (CVE-2023-44487) impacts any internet exposed HTTP/2 endpoints. As an industry leader, Microsoft promptly opened an investigation and subsequently began working with industry partners for a coordinated disclosure and mitigation plan. Microsoft recommends customers follow the guidance provided in this blog to ensure your services are hardened and protected against this DDoS attack technique.
This DDoS attack, known as ‘HTTP/2 Rapid Reset’, leverages a flaw in the implementation of HTTP/2. Microsoft promptly created mitigations for IIS (HTTP.sys), .NET (Kestrel), and Windows, which were part of Microsoft Security Updates released on Oct 10th, 2023.
While this DDoS has the potential to impact service availability, it alone does not lead to the compromise of customer data, and at this time we have seen no evidence of customer data being compromised.
Attack Details Attack Details
This HTTP/2 vulnerability allows malicious actors to launch a DDoS attack targeting HTTP/2 servers. The attack sends a set number of HTTP requests using HEADERS followed by RST_STREAM and repeating this pattern to generate a high volume of traffic on the targeted HTTP/2 servers. By packing multiple HEADERS and RST_STREAM frames in a single connection, attackers can cause a significant increase in the request per second and high CPU utilization on the servers that eventually can cause resource exhaustion.
Protecting your services from CVE-2023-44487 Protecting your services from CVE-2023-44487
This HTTP DDoS activity is primarily targeted at layer 7 rather than layer 3 or 4. Microsoft hardened layer 7 protections in our web service implementations and patched services to better protect customers from the impact of these DDoS attacks. While these tools and techniques are highly effective at mitigating the majority of disruptions, Microsoft consistently reviews the performance of its hardening capabilities and incorporates learnings into refining and improving their effectiveness.
Microsoft services used for hosting web applications have applied security updates to provide mitigations against this attack.
Microsoft recommends customers that are self-hosting web applications patch web servers/proxies using Windows update or Open-Source Software (OSS) fixes for CVE-2023-44487 as quickly as possible to protect their environments. Affected products requiring customer action have been released in our Microsoft Security Update Guide.
Microsoft recommends enabling Azure Web Application Firewall (WAF) on Azure Front Door or Azure Application Gateway to further improve security posture. WAF rate limiting rules are effective in providing additional protection against these attacks. Review recommendation section or this blog for more details.
Microsoft recommends restricting internet access to your web applications where possible.
If you are unable to apply the appropriate patches and your web application is not protected by WAF on Azure Front Door or Application Gateway, consider disabling HTTP2 on your web services. Please note disabling the HTTP2 protocol in your environment should be a deliberate decision, carefully assessing its potential impact on your products and services, as it can significantly influence performance and user experience.
Recommendations – Layer 7 DDoS Protection Tips Recommendations – Layer 7 DDoS Protection Tips
Microsoft recommends using layer 7 protection services such as Azure Web Application Firewall (WAF) (available with Azure Front Door, Azure Application Gateway) to protect web applications. Review the following mitigation guidance to reduce the impact of layer 7 DDoS attack: Application DDoS protection.
If using Azure WAF on Azure Front Door or Application Gateway:
- Use the bot protection managed rule set provides protection against known bad bots. For more information, see Configuring bot protection on Azure Front Door and Configuring bot protection on Application Gateway.
- IP addresses and ranges that you identify as malicious should be blocked. For more information, see examples at Create and use custom rules for Azure Front Door and Create and use v2 custom rules for Application Gateway.
- Traffic from outside a defined geographic region, or within a defined region, should be blocked, rate limited, or redirected to a static webpage. For more information, see examples at Create and use custom rules for Azure Front Door and Create and use v2 custom rules for Application Gateway.
- Create rate limiting custom rules to automatically block and rate limit HTTP or HTTPS attacks that have known signatures. For more information, see examples at Create rate limiting custom rules for Azure Front Door and Create rate limiting custom rules for Application Gateway.
If using Azure API Management, in addition to fronting with WAF, customers should enable the following configuration:
- Deploy your API Management instance into Virtual Network and enable DDOS protection
- Apply L7 rate limit policy to block excessive HTTP traffic.
Azure and M365 Azure and M365
Per the Cloud Shared Responsibility Model, Microsoft will be applying the appropriate security updates to the Microsoft managed SaaS and PaaS services as well as IaaS auto patched services using Safe Deployment Practices. Additionally, these services are also protected by DDoS services. Customers using IaaS services that are on a manual patching are encouraged to apply the updates based on the guidance outlined above.
Microsoft follows Coordinated Vulnerability Disclosure (CVD), which systematically and responsibly manages the discovery, reporting, and remediation of security vulnerabilities. CVD allows us to collaborate with researchers and the wider security community in a way that prioritizes user security and system integrity. By following a coordinated approach, we can work with researchers and industry partners to ensure that potential vulnerabilities are addressed before they’re made public, reducing the risk of exploitation.
Part of any robust security posture is working with researchers to help find vulnerabilities, so we can fix any findings before details of the vulnerabilities are broadly available and misused. We want to thank Amazon, Cloudflare, and Google who reported this vulnerability under Coordinated Vulnerability Disclosure (CVD). We also want to thank the partners in the open-source community that worked with the Microsoft Security Response Center (MSRC) to develop fixes and help keep Microsoft customers safe.
References References
- Application DDoS protection - Azure Web Application Firewall | Microsoft Learn
- DDoS Mitigation with Microsoft Azure Front Door | Azure Blog | Microsoft Azure
- 2022 in review: DDoS attack trends and insights | Microsoft Security Blog
- Joint CISA FBI MS-ISAC Guide on Responding to DDoS Attacks and DDoS Guidance for Federal Agencies | CISA
Related news
Red Hat Security Advisory 2024-3352-03 - An update for etcd is now available for Red Hat OpenStack Platform 16.2. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2024-0766-03 - Red Hat OpenShift Container Platform release 4.15.0 is now available with updates to packages and images that fix several bugs and add enhancements. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2024-0304-03 - Updated images are now available for Red Hat Advanced Cluster Security 3.74. The updated images includes bug and security fixes.
Red Hat Security Advisory 2023-7704-03 - Red Hat OpenShift Virtualization release 4.14.1 is now available with updates to packages and images that fix several bugs and add enhancements. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-7478-01 - Red Hat OpenShift Container Platform release 4.11.54 is now available with updates to packages and images that fix several bugs.
Red Hat Security Advisory 2023-7334-01 - An update for rh-varnish6-varnish is now available for Red Hat Software Collections. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-6243-01 - An update for openshift-gitops-kam is now available for Red Hat OpenShift GitOps 1.10.
Red Hat Security Advisory 2023-6022-01 - An update for the varnish:6 module is now available for Red Hat Enterprise Linux 8.2 Advanced Update Support, Red Hat Enterprise Linux 8.2 Telecommunications Update Service, and Red Hat Enterprise Linux 8.2 Update Services for SAP Solutions. Issues addressed include a denial of service vulnerability.
Cloudflare on Thursday said it mitigated thousands of hyper-volumetric HTTP distributed denial-of-service (DDoS) attacks that exploited a recently disclosed flaw called HTTP/2 Rapid Reset, 89 of which exceeded 100 million requests per second (RPS). "The campaign contributed to an overall increase of 65% in HTTP DDoS attack traffic in Q3 compared to the previous quarter," the web infrastructure
Red Hat Security Advisory 2023-6061-01 - Red Hat OpenShift Pipelines 1.12.1 has been released. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5979-01 - Updated Satellite 6.12 packages that fixes important security bugs and several regular bugs are now available for Red Hat Satellite. Issues addressed include a code execution vulnerability.
Red Hat Security Advisory 2023-5971-01 - An update for osp-director-agent-container, osp-director-downloader-container, osp-director-operator-bundle-container, and osp-director-operator-container is now available for Red Hat OpenStack Platform 17.1.1. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5970-01 - An update for collectd-libpod-stats is now available for Red Hat OpenStack Platform 17.1.1. Issues addressed include a denial of service vulnerability.
Ubuntu Security Notice 6427-2 - USN-6427-1 fixed a vulnerability in .NET. This update provides the corresponding update for .NET 8. It was discovered that the .NET Kestrel web server did not properly handle HTTP/2 requests. A remote attacker could possibly use this issue to cause a denial of service.
Red Hat Security Advisory 2023-5930-01 - An update for varnish is now available for Red Hat Enterprise Linux 9.0 Extended Update Support. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5869-01 - An update for the nodejs:18 module is now available for Red Hat Enterprise Linux 8. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5863-01 - An update for grafana is now available for Red Hat Enterprise Linux 8. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5837-01 - nghttp2 contains the Hypertext Transfer Protocol version 2 client, server, and proxy programs as well as a library implementing the HTTP/2 protocol in C. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5809-01 - Red Hat Ansible Automation Platform provides an enterprise framework for building, deploying and managing IT automation at scale. IT Managers can provide top-down guidelines on how automation is applied to individual teams, while automation developers retain the freedom to write tasks that leverage existing knowledge without the overhead. Ansible Automation Platform makes it possible for users across an organization to share, vet, and manage automation content by means of a simple, powerful, and agentless language.
Red Hat Security Advisory 2023-5801-01 - Migration Toolkit for Runtimes 1.2.1 Images. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5783-01 - Red Hat JBoss Web Server is a fully integrated and certified set of components for hosting Java web applications. It is comprised of the Apache Tomcat Servlet container, JBoss HTTP Connector, the PicketLink Vault extension for Apache Tomcat, and the Tomcat Native library. This release of Red Hat JBoss Web Server 5.7.5 serves as a replacement for Red Hat JBoss Web Server 5.7.4. This release includes bug fixes, enhancements and component upgrades, which are documented in the Release Notes, linked to in the References section. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5710-01 - .NET is a managed-software framework. It implements a subset of the .NET framework APIs and several new APIs, and it includes a CLR implementation. New versions of .NET that address a security vulnerability are now available. The updated versions are .NET 6.0 to SDK 6.0.123 and Runtime 6.0.23. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5780-01 - A security update for Camel Extensions for Quarkus 2.13.3 is now available. The purpose of this text-only erratum is to inform you about the security issues fixed. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5769-01 - nghttp2 contains the Hypertext Transfer Protocol version 2 client, server, and proxy programs as well as a library implementing the HTTP/2 protocol in C. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5766-01 - nghttp2 contains the Hypertext Transfer Protocol version 2 client, server, and proxy programs as well as a library implementing the HTTP/2 protocol in C. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5765-01 - Node.js is a software development platform for building fast and scalable network applications in the JavaScript programming language. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5714-01 - nginx is a web and proxy server supporting HTTP and other protocols, with a focus on high concurrency, performance, and low memory usage. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5713-01 - nginx is a web and proxy server supporting HTTP and other protocols, with a focus on high concurrency, performance, and low memory usage. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5711-01 - nginx is a web and proxy server supporting HTTP and other protocols, with a focus on high concurrency, performance, and low memory usage. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5709-01 - .NET is a managed-software framework. It implements a subset of the .NET framework APIs and several new APIs, and it includes a CLR implementation. New versions of .NET that address a security vulnerability are now available. The updated versions are .NET 7.0 to SDK 7.0.112 and Runtime 7.0.12. Issues addressed include a denial of service vulnerability.
Red Hat Security Advisory 2023-5708-01 - .NET is a managed-software framework. It implements a subset of the .NET framework APIs and several new APIs, and it includes a CLR implementation. New versions of .NET that address a security vulnerability are now available. The updated versions are .NET 6.0 to SDK 6.0.123 and Runtime 6.0.23. Issues addressed include a denial of service vulnerability.
Debian Linux Security Advisory 5522-2 - The patch to address CVE-2023-44487 (Rapid Reset Attack) was incomplete and caused a regression when using asynchronous I/O (the default for NIO and NIO2). DATA frames must be included when calculating the HTTP/2 overhead count to ensure that connections are not prematurely terminated.
Plus, many of the world’s largest cloud providers are warning of a vulnerability that attackers exploited in August to launch the largest distributed denial-of-service attack on record.
Cisco Talos is actively tracking the novel distributed denial-of-service (DDoS) attacks cloud services provider Cloudflare disclosed earlier this week. The techniques described in Cloudflare’s blog post resulted in a record-breaking DDoS attack and could facilitate much larger attacks in the future. CVE-2023-44487 CVE-2023-44487, a vulnerability in the
A malicious HTTP/2 client which rapidly creates requests and immediately resets them can cause excessive server resource consumption. While the total number of requests is bounded by the http2.Server.MaxConcurrentStreams setting, resetting an in-progress request allows the attacker to create a new request while the existing one is still executing. With the fix applied, HTTP/2 servers now bound the number of simultaneously executing handler goroutines to the stream concurrency limit (MaxConcurrentStreams). New requests arriving when at the limit (which can only happen after the client has reset an existing, in-flight request) will be queued until a handler exits. If the request queue grows too large, the server will terminate the connection. This issue is also fixed in golang.org/x/net/http2 for users manually configuring HTTP/2. The default stream concurrency limit is 250 streams (requests) per HTTP/2 connection. This value may be adjusted using the golang.org/x/net/http2 package; see ...
Debian Linux Security Advisory 5522-1 - Several security vulnerabilities have been discovered in the Tomcat servlet and JSP engine.
By Deeba Ahmed Google, Cloudflare, and AWS Disclosed Digital History’s Largest Ever DDoS Attack- Courtesy HTTP/2 Zero-day. This is a post from HackRead.com Read the original post: Google, Cloudflare, and AWS Disclose Largest DDoS Attack in History
Microsoft has released its Patch Tuesday updates for October 2023, addressing a total of 103 flaws in its software, two of which have come under active exploitation in the wild. Of the 103 flaws, 13 are rated Critical and 90 are rated Important in severity. This is apart from 18 security vulnerabilities addressed in its Chromium-based Edge browser since the second Tuesday of September. The two
Microsoft today issued security updates for more than 100 newly-discovered vulnerabilities in its Windows operating system and related software, including four flaws that are already being exploited. In addition, Apple recently released emergency updates to quash a pair of zero-day bugs in iOS.
swift-nio-http2 is vulnerable to a denial-of-service vulnerability in which a malicious client can create and then reset a large number of HTTP/2 streams in a short period of time. This causes swift-nio-http2 to commit to a large amount of expensive work which it then throws away, including creating entirely new `Channel`s to serve the traffic. This can easily overwhelm an `EventLoop` and prevent it from making forward progress. swift-nio-http2 1.28 contains a remediation for this issue that applies reset counter using a sliding window. This constrains the number of stream resets that may occur in a given window of time. Clients violating this limit will have their connections torn down. This allows clients to continue to cancel streams for legitimate reasons, while constraining malicious actors.
Amazon Web Services (AWS), Cloudflare, and Google on Tuesday said they took steps to mitigate record-breaking distributed denial-of-service (DDoS) attacks that relied on a novel technique called HTTP/2 Rapid Reset. The layer 7 attacks were detected in late August 2023, the companies said in a coordinated disclosure. The cumulative susceptibility to this attack is being tracked as CVE-2023-44487,
The HTTP/2 protocol allows a denial of service (server resource consumption) because request cancellation can reset many streams quickly, as exploited in the wild in August through October 2023.