4 Big Mistakes to Avoid in OT Incident Response

When it comes to incident response, many organizations assume that the training and preparation they have done for the IT side of their networks extends to their operational technology/industrial control systems environments. While they may appear to share many similarities, OT security requires different protocols, objectives, analysis, forensics, and other security methods. What works successfully in an IT network may be problematic for conducting incident response (IR) in an OT/ICS environment, where availability and safety of operations must be maintained.

Organizations must understand the key differences to navigate around gaps and pitfalls that could prevent a successful OT/ICS IR scenario.

These are the most common areas we see organizations need some help with during our incident response simulation assessments.

Shutting Down First, Asking Questions Later

In an IT environment, isolating or shutting down the system to prevent further infection is a normal and relatively common practice. In an OT environment, isolation or shutting down a system has much more substantial ramifications.

For example, in the Colonial Pipeline attack, they responded to an attack in the IT systems. To ensure it did not spread to OT systems, they shut those down as well, even though the ransomware did not reach the OT level. The move was in line with their culture of safety. But a shutdown of OT systems has a higher cost per minute of downtime and requires longer to get running again. An IT server can be taken offline and a new one created and stood up in about a day if the entire IT department works on it. In Colonial’s case, it took five days after the ransom was paid to stand the pipeline’s OT network backup to regular operations.

Stopping the Attack and Starting Remediation Too Early

IT security personnel in a security operations center (SOC) traditionally do not have intimate knowledge of the OT systems, and OT equipment is much more sensitive to data transfer. The SOC team members may get an alert of something happening in the OT system, and if they jump to conduct a scan for vulnerabilities, they can easily overwhelm the system, denial-of-service style. That means it will cease to function and cause a massive loss in productivity. It also requires the OT engineer to go in and fix the mess, which can be an unnecessary source of additional frustration during an already stressful time.

Lack of Alignment on Ultimate Responsibility

Because IT and OT teams are so separate, sometimes no one has responsibility for OT from a day-to-day perspective. If a SOC is responsible for the security of all IT systems in the company, does that include OT systems? OT systems aren’t IT systems, so one could say no. OT systems have IT assets, though, so are those included? Are those OT assets or IT assets? There is a problem from the moment of responsibility. You have IT professionals without knowledge or experience with the OT assets, and you have OT professionals without a security mindset or understanding of cyber-risk. And neither likes being dictated to by the other.

Misunderstanding What the Other Side Is Trying to Accomplish

One story told by a customer perfectly sums up this point. They talked about how their SOC used to send security reports to the OT engineers to have them patch, fix, or change something in the OT systems. Whenever the OT professionals received this report, they would promptly throw it in the trash. Whether this was from a lack of understanding on either the OT or IT side is unclear, but what is clear is that the communication between the teams was not just insufficient — it was actively opposed.

So how can organizations overcome these challenges?

Communication and compromise are crucial to building trust. Instead of the security professionals dictating what needs to occur in an OT system, they should work with the OT professionals to find compromises that do not hinder OT goals too much while still allowing the network to maintain security. It’s vital to proactively bring both groups together to work towards common security objectives.

CISOs and security managers can implement structured training programs that simulate how cybersecurity and incident response teams from both IT and OT disciplines would work together during a real-life incident response event. By putting the right people from both sides in the right seats and creating a shared learning path, you invite communication and collaboration between the groups. The more they can practice detecting, responding, and remediating attacks on critical infrastructure together, the better they can understand how to work together better in real life.

At the end of the day, they are both after the same thing: keeping the organization and its people safe from industrial cyberattacks.