At Recon InfoSec we have the honor of working with some of the best security operations, incident response, and threat hunting teams in the world: Fortune 100 companies, military cyber protection teams, global incident response firms, “3 letter agencies,” and “Big 4” professional services companies.
We have completed thousands of hours training these teams, and our experience shows us that nearly all security operations teams commit time and resources to training. But, the primary factor which distinguishes great teams is how they train.
Great security operations teams:
- Measure the value of training by its impact on Operational Capability.
- Focus the majority of their training on building team experience.
The Training Challenge
The growing challenge facing the Information Security community and the corresponding talent shortage are widely recognized. Nowhere is this more acute than in security operations centers and incident response teams.
Given these challenges, what is the best way to make these teams better? Is it more classes? More certificates? More labs and workshops? More Red vs. Blue capture-the-flag games? There is a bewildering catalog of offerings from an industry that is well known for its smoke and mirrors.
Many teams take an “all training is good training” approach. Unfortunately, this leads to the all-too-common feeling that they are spending a lot on training and certifications but not getting any better at security operations.. Enterprise Security is notoriously broad, complex and dynamic. There are literally hundreds of topics, tools, and techniques that are relevant. But how much of it leads to measurable improvements in team performance? Do all training dollars produce the same benefit? Are traditional classes more valuable than hands-on workshops? Is there more benefit to a set of labs or a new certificate? What role does on-the-job training have? What knowledge should be required of new hires? What knowledge should be developed in-house?
Knowledge, Skills and Operational Capability
The core of information security operations, incident response, and threat hunting is practical, technical problem solving. It is practical because incidents require analysts with hands-on skills. It is technical in that it requires extensive, specialized knowledge.
For great teams, Knowledge, Skills, and Operational Capabilities are important and necessary, but they differ significantly in how they are developed, when they are developed, and their overall value to the organization. The model below captures the relationship between the three.
Knowledge is what we understand.
- Knowledge is the breadth and depth of relevant information that we understand.
- For enterprise security this knowledge includes computing concepts, hardware, operating systems, protocols and packets, wireless network security, virtualization, cloud computing, internet protocols, etc.
- Knowledge is most often acquired through reading, textbooks, lectures, videos, quizzes, and tests.
- There are a myriad of good sources of security operations knowledge including college courses, Cybrary, and SANS.
- Foundational knowledge is usually a prerequisite for a security operations job. It offers the most benefit to entry level practitioners early in their careers.
Skills are what we can do.
- Skills are the breadth and depth of the relevant actions we can take.
- It is the application of knowledge.
- For enterprise security, this manifests as proficiencies in log aggregation platforms, search queries, analytical tools, operating systems, scripting languages, network and firewall configuration, etc.
- Skills are most often acquired through workshops and labs via application of knowledge.
- There are many good sources of security operations skills including Chris Sanders’ courses, vendor-specific courses, and building open source labs.
- Skills are the “building blocks” of practical problem solving. They are the “tools in your toolbox” that need to be maintained and added to throughout a career.
Operational Capabilities are the problems we can solve.
- Operational capability is the breadth and depth of the practical, technical problems we can solve.
- For enterprise security, operational capability means how well we can detect, investigate, respond, and remediate threats to our enterprise environment.
- Operational Capabilities are developed through experience.
- The best methods for developing this type of judgement is a combination of mentoring, on the job training, and ranges.
- Operational Capabilities are the value that security operations teams deliver to their organizations.
This 3 layer model is exceptionally valuable in diagnosing training needs, developing training plans, and allocating training resources.
Focus on Operational Capability
So how do the best security operations teams approach training? Do they invest in more classes? Do they host more workshops and labs? Do they establish good mentoring programs, structure on the job training, and run regular range exercises? The short answer is yes, they do all those things.
But it is more insightful to note that they only do those things when they have identified specific deficiencies in their Operational Capability tied directly to those activities. Operational Capability is the value the security operations team brings to their organization, it is the goal of their daily operations, and it is the measure of the effectiveness of their training programs.
They don’t invest in classes for the sake of certifications. They only invest in classroom training when the team has knowledge deficiencies that are limiting their Operational Capabilities. They regularly invest time and money in workshops and labs to build skills that have demonstrated utility. Most importantly, they realize the best way to improve Operational Capability is proactively, through experience.
Invest in Experience
Having run the Recon Network Defense Range for hundreds of the best incident responders and threat hunters around the world has taught us the importance of focusing on Operational Capability and team experience. We design all of our courses and Range Days around those objectives, focus our internal training this way, and we would encourage all security operations teams to do the same.
The best security operations teams focus their training budgets on mentoring, on-the-job-training, and high-fidelity simulations. Not only does experience build and improve Operational Capability, but it also helps assess the gaps in both knowledge and skills proactively.
KEY TAKE-AWAY
To optimize your security operations:
- Measure the value of training by its impact on Operational Capability.
- Focus the majority of training on building team experience.