Ripping off the Bandage: How AI is Changing the SOC Maturity Model

The introduction of virtual analysts, artificial intelligence and other advanced technologies into the Security Operations Center (SOC) is changing how we should think about maturity models. AI is replacing traditional human tasks, and when those tasks are automated the code effectively becomes the procedure. Is that a -1 or a +10 for security operations? Let’s discuss that.

To see the big picture here, we should review what a maturity model is and why we are using them for formal security operations. A maturity model is a process methodology that drives good documentation, repeatability, metrics and continuous improvement. The assumption being that these are a proxy for effectiveness and efficiency. The most common model used in Security Operations is a variant of the Carnegie Mellon, Capability Maturity Model for Integration (CMMI). Many process methods focus on defect management, this is even more evident in the CMMI since it originated in the software industry.

In the early 2000’s, we started using CMMI at IBM, Big Blue insisted that we couldn’t offer a commercial service that wasn’t on a maturity path and they had adopted CMMI across the entire company at that point. We had, at that time, what seemed like a never-ending series of failures in our security monitoring services, and for each failure a new “bandage” in the form of a process or procedure was applied. After a few years we had an enormous list of processes and procedures, each connected to the other in a PERT chart of SOC formality. Most of these “bandages” were intended to provide guidance and support to analysts as they conducted security monitoring and to prevent predictable failures, so we could offer a consistent and repeatable service across shifts and customers.

To understand this better, let’s look at the 5 levels of the CMMI model:

  1. Initial (ad hoc)
  2. Managed (can be repeated)
  3. Defined (is repeated)
  4. Measured (is appropriately measured)
  5. Self-optimizing (measurements leads to improvements)

This well-defined approach seemed to be perfect. It allowed us to take junior analysts and empower them to have a consistent level of service delivery. We could repeat ourselves across customers. We might not deliver the most effective results, but we could at least be reasonably consistent. As it turns out, people don’t like working in such structured roles because there’s little room for creativity or curiosity. Not surprisingly, this gave rise to the 18-24 month security analyst turn-over phenomenon. Many early analysts came from help desk positions and were escaping “call resolution” metrics in the first place.

Our application of SOC maturity morphed over the years from solving consistency problems into consistently repeating the wrong things because they could be easily measured. When failures happened, we were now in the habit of applying the same “bandages” over and over.  Meanwhile, the bad guys had moved on to new and better attack techniques. I have seen security operations teams follow maturity guidelines right down a black hole, when for example, a minor SIEM content change can take months, not the few hours it should take.

According to the HPE Security Operations Maturity report, the industry median maturity score is 1.4, or slightly better than ad-hoc. I’m only aware of 2 SOCs in the world that are CMMI 3.0.  So, while across the industry we are measuring our repeatability and hoping that it equates to effectiveness and efficiency, we are still highly immature, and this is reflected in the almost daily breaches being reported. You can also see this in the multi-year sine wave of SOC capability many organizations experience; it goes something like this:

  1. Breach
  2. Response
  3. New SOC or SOC rebuild
  4. Delivery challenges
  5. Maturity program
  6. Difficulty articulating ROI
  7. Cost reductions
  8. Outsourcing
  9. Breach
  10. Repeat

With a virtual analyst, your SOC can now leap to CMMI level 5 for what was traditionally a human-only task. An AI-based virtual analyst, like the Respond Analyst, conducts deep analysis in a consistent fashion and learns rationally from experience. This approach provides effective monitoring in real time and puts EVERY SINGLE security-relevant event under scrutiny. Not only that, you liberate your people from rigorous process control, and allow them to hunt for novel or persistent attackers using their creativity and curiosity.

This will tip the balance towards the defender and we need all the help we can get!

When Currency is Time, Spend it Threat Hunting

“Time is what we want most, but what we use worst.”
– William Penn

How many valuable cybersecurity tasks have you put aside due to the pressures of time? Time is currency and we spend it every moment we’re protecting our enterprises.

When we are constantly tuning, supporting and maintaining our security controls or chasing down an alert from an MSSP, only to discover it’s yet another false positive, we spend precious currency. When we create new correlation logic in our SIEM or decide which signatures to tune down to lower the volume of events to make it more manageable for our security team, we spend precious currency. When we analyze events from a SIEM to determine if they’re malicious and actionable or if a SIEM rule needs additional refinement, we spend precious currency. When we hire and train new analysts to cover churn, then watch them leave for a new opportunity – we waste currency and the investment hurts.

You can spend your “currency” doing pretty much anything, which is a blessing and a curse. We can (and do) waste an inordinate amount of time going down rabbit holes chasing false positives. We are forced to make choices: do we push back a request while we investigate the MSSP escalations or do we delay an investigation to provide the service agility the enterprise requires?

Both options are important, and both need addressing; forcing us to make a choice. In our gut we think the escalation is another false positive, but as cybersecurity professionals; we wait for the sword of Damocles to fall. It’s only a matter of time before one of these escalations is related to the thing we worry about most in our environments. Either way, something gets delayed…. hopefully just lunch.

Basing decisions on what we can neglect is reactive and unsustainable. It’s a matter of time until we choose to postpone the wrong thing.

We need to use our time more wisely.

Organizations need to spend precious “currency” focusing on higher value tasks, like threat hunting, that motivate their talent and provide value to the organization. But also need to maintain two hands on the wheel of lower value tasks that still need attention.

Organizations should implement automation tools to focus on the lower-value, repetitive tasks such as high-volume network security monitoring. Generating and receiving alerts from your security controls is easy, making sense and determining if they’re malicious and actionable is a different story. The decision to escalate events is typically inconsistent and heavily relies on the analyst making the decision. Factor in the amount of time required to gather supporting evidence and then make a decision, while doing this an additional 75 times an hour. As a defender, you don’t have enough “currency of time” to make consistent, highly-accurate decisions. Security analysts tasked with monitoring high-noise, low-signal event feeds is a misallocation of time that only leads to a lack of job satisfaction and burnout.

There is another way.

Employing Respond Analyst is like adding a virtual team of expert, superhuman analysts and will allow your team to, bring their talent and expertise to threat hunting. Adding Respond Analyst allows your talent to focus on higher value tasks and more engaging work so you can combat analyst burnout, training drains, and churn.

“Fake News” Must Learn to Regulate Itself!

Digital interaction has surpassed interpersonal interaction — even grandmothers primarily interact with their grandchildren via iPad.  Almost everything we do can be done via an App or online, before long that’ll be the only way we can do anything. We have a word for this and that is “critical infrastructure,” and as a society we need to be aware of how this information infrastructure can be used against us or to control us.

We are recklessly sprinting into an unmapped digital future. Crashing traditional business models that have endured for hundreds of years, and inventing new ones along the way. Losing and gaining new jobs at break-neck speed. The only plan being what is profitable and possible.

Think about how much digital marketers, cyber-criminals and foreign intelligence services know about us from the constant data breaches and our daily digital life. We use our phones to conduct and document almost every aspect of modern life.  Thus, you can easily map most of the dimensions of human personality and then use them to present attractive options to consumers or to bend and target messages in inappropriate ways. Our social discourse is so bad right now that we are unreliable verifiers of truth. We may need to leave it up to the machines. Or another way to put this: artificial intelligence may be an imperative for the future of our society and our world.

The dangers a post-truth digital world pose to human institutions are hard to quantify, scary to contemplate, and easily observable in the world around us, every day. Be it election maskirovka or plain old “fake news” and no matter your political or personal lens, human institutions are no longer able to reliably regulate truth.

Truth is going to have to self-regulate. That is a phenomenal application for artificial intelligence. “Computer! Solve the equation: What’s the likelihood something is true, given everything else you know about it? And a little help with the spin too, please.”

These days saying you use artificial intelligence is already almost like saying you use the Java programming language, mundane. There are a dizzying array of AI algorithms. Each is optimized to some specific application or purpose, understanding how and when to apply them requires subject matter expertise and deep math but it’s doable. In fact, that’s what we do here at Respond Software, we teach mathematics what we used to teach humans about catching hackers, and thus free today’s analysts from monitoring a boring but critical console so they can use their creativity and curiosity to provide better security for our digital businesses and lives.

If the digital future has to fend for itself or defend its own integrity, we are in the process of teaching it how. The agreement, “We’ll keep you turned on, and you save us from ourselves.”

Join our growing community! Subscribe to our newsletter, the "First Responder Notebook," delivered straight to your inbox.