SOC Events Per Analyst Hour aka (EPAH)

Early in my corporate career, I was promoted to the first SOC manager for IBM’s Managed Security Services. Prior to building that SOC, our security monitoring services were provided by the NOC.

This was back in 2000, the early days of security operations, before we’d figured it all out.
This SOC team was tasked with monitoring security events in real-time to determine which were important enough to be escalated to our customers for deeper investigation.

Within weeks, the team pointed out to me that they were completely overwhelmed by the volume of events.  In fact, when I pushed back on the team they said, “You try it!”  So, I did, and it turned out that 2,400 events per hour will make you crazy in less than 20 minutes (and they knew that).  This led me to the question, “What is too much volume for a SOC analyst and how do you measure it?”  This is when I began measuring Events Per Analyst Hour (EPAH).

Now that I have been measuring humans analyzing events for almost 20 years, the “green zone” for EPAH is between 75 and 150.   And, those events can usually be collapsed into 12-15 different simultaneous potential attacks to analyze.  An experienced analyst can usually handle around 150 events and a junior analyst will struggle with 75 events.

To demonstrate the challenge this limitation poses for Security Operations monitoring, let’s look at the hard numbers to bring this into focus:

  1. A mature SOC provides 24 x 7 x 365 coverage
  2. It takes a minimum of 10 FTE (really 12 with management) to cover that at 1 analyst per hour
  3. IDS/IPS alone can produce 100+M events in any given month
  4. Many other data sources are just as loud, so assume 300-500M events per month for many enterprise SOCs
  5. At 500M events per month, that equals 694,444 events per analyst hour

Clearly that scenario doesn’t work.  So, if you implement an event funnel to reduce the above by 4,629x down to 150 EPAH, what did you miss?  I’m not sure, how about you?

When the capacity of humans doing security monitoring is exceeded, it’s obvious.  On the other hand, what was ignored and missed, that should have been spotted?  Here’s the catch — you won’t know until it’s too late.  While EPAH is a critical measure of what human analysts are capable of, it doesn’t tell you anything about what you ignored.  This is the fundamental limitation of console monitoring.

Even if humans were perfectly effective at monitoring (see my previous article “How Human Factors Hurt the SOC”) catching capable attackers is nearly impossible due to the small amount of total possible attack traffic actually observed.  This is borne out by the consistent truth that most attacks are caught when new stolen items show up in the darknet.