CrowdStrike Will Give Customers Control Over Falcon Sensor Updates

Source: Ascannio via Shutterstock

CrowdStrike will give customers more control over how they deploy content updates to the company’s Falcon sensor endpoint security technology following the recent incident that saw a faulty update crash more than 8.5 million Windows systems worldwide.

The beleaguered security vendor — which is the target of two lawsuits over the incident already — has implemented new features to its platform to support the capability with additional functionality planned for the future.

Multiple Changes

The update is one of several changes CrowdStrike has implemented following the completion of a root cause analysis (RCA) of the July 19 incident. In an Aug. 7 update, CrowdStrike announced other changes it has made to ensure something similar does not happen in the future. The changes include new content configuration system test procedures, additional deployment layers and acceptance checks for its content configuration system, and new validation checks for its updates.

CrowdStrike has also asked two independent third-party security vendors to review the code for its Falcon sensor technology and of the company’s quality control and release processes for the product. “We are using the lessons learned from this incident to better serve our customers,” CrowdStrike CEO George Kurtz said in a statement that accompanied its RCA. “To this end, we have already taken decisive steps to help prevent this situation from repeating and to help ensure that we — and you — become even more resilient.”

CrowdStrike’s problems started with a July 19 content update for a new Falcon sensor capability that the security vendor first rolled out in February 2024. The automatically deployed update caused Windows systems worldwide to crash and created enormous disruptions for organizations across multiple sectors, including airlines, financial services, healthcare, manufacturing, and government. In many cases, systems admins had to manually restart computers, which meant that it took days for numerous organizations to restore services fully.

CrowdStrike has already become the target of at least two class-action lawsuits over the incident — one on behalf of the company’s shareholders and the other on behalf of affected businesses. Many others, including Delta Air Lines, are expected to sue CrowdStrike over related outage costs in coming days and months.

Parameter Count Mismatch

The security vendor has identified a parameter count mismatch between what its Falcon sensor product expected and what the July 19 content configuration update actually contained as the root cause for the problems. The update was for a Falcon sensor feature that CrowdStrike rolled out in February to detect and provide insights into new attack techniques that exploit specific Windows mechanisms. Falcon sensor uses a specific template with a predefined set of 20 separate input fields to deliver this specific capability.

CrowdStrike’s content configuration update on July 19 provided 21 input fields rather than the 20 fields the sensor expected. “In this instance, the mismatch resulted in an out-of-bounds memory read, causing a system crash,” CrowdStrike said.

While the security vendor introduced the template with the mismatched parameter count in February, its analysis showed it slipped past multiple layers of build validation and testing. No one caught the discrepancy during the sensor release test process, during stress tests of the template, or even during initial real-world deployments. In part, this was because the test processes and initial deployments used a “wildcard matching criteria” — meaning they accepted any value or no value at all — for the extra input field’s parameter.

The July 19 update used a non-wildcard matching criterion for the July 21 parameter, which meant the sensor had to contend with data for a field it did not expect. “The Content Interpreter expected only 20 values,” CrowdStrike said. “Therefore, the attempt to access the 21st value produced an out-of-bounds memory read beyond the end of the input data array and resulted in a system crash.”

About the Author

Jai Vijayan is a seasoned technology reporter with over 20 years of experience in IT trade journalism. He was most recently a Senior Editor at Computerworld, where he covered information security and data privacy issues for the publication. Over the course of his 20-year career at Computerworld, Jai also covered a variety of other technology topics, including big data, Hadoop, Internet of Things, e-voting, and data analytics. Prior to Computerworld, Jai covered technology issues for The Economic Times in Bangalore, India. Jai has a Master’s degree in Statistics and lives in Naperville, Ill.