The Evolution Of Security Operations Automation
We’ve reached a point where automation in security operations (SecOps) is a necessity. We’re simply not keeping up. For example, the recent NotPetya malware attack was entirely automated. From initial infection to credential harvesting to lateral movement to system destruction, the entire attack lifecycle was automated. Take that in for a moment — fully automated and sophisticated attacks penetrating companies across the globe. This is not the future, this is today. There’s reason to worry and reason to move forward.
In SecOps, we’ve spent years putting people in front of our attempts to solve our security challenges. In some ways, we’ve won and failed in many others. Most importantly, we’ve learned. What used to be people, process, and technology, is becoming technology with some human reasoning and process flow embedded. We now understand the domain and how to solve several of our toughest challenges by augmenting with machine intelligence.
Every organization today struggles with effective and efficient security operations.
This post is about how we’re evolving with automation throughout SecOps core processes to make better decisions and speed detection and recovery time.
Collection of relevant data, including telemetry to conduct investigations
Collection of detection telemetry is well understood and heavily automated. We had a need to automatically retrieve data for correlation and analysis. The answer was plumbing and aggregation with Security Information and Event Management (SIEM) platforms. This technology transformed ad-hoc data collection and monitoring into the Security Operations Center (SOC) found in many organizations today.
Even though this area of SecOps is well understood and automated, we are continuing to grow automation in this area in several ways. For example, the improvements to Windows Event Forwarding and contextual information like vulnerability scan data or employee information. Consider how difficult it is to retrieve the right DNS logs in many enterprises for investigation. Even though there are areas needing improvement, there has been significant progress over the past several decades and this is the most understood and progressed for automation.
Analysis of data to understand situations
Over time, this area of SecOps has developed by making better alerts for an analyst and automating some of the human analyst workflows. Unfortunately, to make a better alert, many times we simply filtered more and more until all difficult to manage, but valuable, events of interest were removed. Does your SOC look at low and medium Network IPS events? Probably not. Do they provide value? I can say with certainty the answer is yes. We applied advanced (or what we thought was advanced) correlation logic, trending and machine learning, to produce a better alert for a human analyst. Then we bolted on the automation of common human analyst research actions to speed up our investigation. But, we were still relying on a human analyst to interpret and investigate alerts.
Fortunately, we learn and technology improves. And, over time, we evolve with better solutions for our industry. We now understand how humans reason through intrusion analysis and how other domains can help resolve some of our toughest challenges. This area of SecOps is incredibly exciting and we’re seeing the emergence of expert systems that don’t just provide another alert or automate a step but perform a full investigation entirely autonomously at machine speed, consistency, and scale. The same reasoning a human can perform now can be done by a machine. There is no need to filter with an event funnel. There is no tiredness or memory loss. There is no bias or inconsistency.
Even though incredible results have been produced (not thought possible even several years ago), it’s no surprise that human analysts are still needed. Machines are great for specific use cases, but humans provide creativity and have access to data a machine may not. There should be significant growth in areas where machines reason across many use cases for intrusion detection. Also, expect hunt operations to augment machines for use cases where a machine does not know how to reason or where data is difficult to collect.
Make decisions based on investigations
After you or a machine completes an investigation, a decision is made. That decision is based on analysis, and again, just like a human, a machine can reason through data to make an informed decision. Traditionally, humans have made security incident decisions, but this is another area that is changing with expert systems. Information access, processing, and memory capacity are orders of magnitude greater with a machine. As a security analyst, would you remember all the host names and accounts you observed an hour, a day, a week or a month ago? Shift logs help humans, but typically these logs are specific to only those things that were very interesting at the time and are often based on 0.0001% of available information from the start.
These same expert systems can not only access an incredible amount of information through automated searches and questions to make an informed decision, they can precisely reason to determine the likelihood of a malicious and actionable incident. With advanced math, these expert systems can even learn and adjust their reasoning and understanding of a specific environment or domain.
Of course, for those areas where we still need hunt operations for investigations, hunt analysts are needed to make decisions. Machines are not for every security use case and not for every decision.
Take action to resolve situations
As we move through the core processes of SecOps and see its evolution through automation, the action taken by incident response is, by far, the most difficult to automate completely. This is not because of the difficulty automating actions (there are many vendors in the automation and orchestration space), but because of the difficulty in taking corrective action based completely on the decision of a machine, and a machine alone. Automating blocking of applications or traffic would make many uncomfortable, at least for the foreseeable future. Automation is assisting the human analyst by speeding time to recovery. However, in many cases, a human-in-the-loop is required to give the signal to proceed with an automated action. Human analysts, primarily incident responders, are required in this area and likely will be for years to come.
Our industry has come a long way in automating SecOps and you can expect to see significant progress in the future. This will result in a better depth of analysis, wider coverage, greater consistency, improved accuracy and greater reductions in time to detection and recovery.
Humans are still very much needed in hunt operations and incident response, but machines are finding their place in making the SecOps organization more effective and efficient through automation. Consider your organization and how automation can move your SecOps program forward.