The threat landscape faced by organizations is forever changing, but one constant in the battle against online menaces has been the security operations center. However, while most businesses see the SOC as a critical part of their cybersecurity strategies, there's growing dissatisfaction with what they're getting for their investment.
One study, conducted by the Ponemon Institute and released in January, found that only slightly more than half of organizations (51%) were satisfied with the effectiveness of their SOCs in detecting attacks.
Another troubling development is the rising perception that the return on investment from a SOC is getting worse. The Ponemon study found that more than half of organizations (51%) felt that way, compared with 44% in 2019.
The study, based on a survey of 16,841 IT and IT security practitioners that have a SOC, found that organizations spend an average of $2.86 million annually on their in-house SOC. That cost increases significantly, to $4.44 million, if SOC functions are outsourced to a managed security service provider.
To be more effective, organizations need to spend more money, the researchers found. An average of $3.5 million was spent on highly effective SOCs, compared with an average of $1.96 million on SOCs with very low effectiveness.
But spending is just the beginning. Security teams understand it's how you spend it that matters. TechBeacon talked to top experts for guidance. Here are the top SecOps team challenges—and best practices for dealing with them.
1. Cost of complexity
If organizations are looking for a rapid ROI from their SOCs and technologies such as SEIM and security orchestration, automation, and response (SOAR), it's easy to understand why they might be disappointed.
Dan Lamorena, vice president for marketing at FireEye, said ROI can take time.
"Traditional SIEMs and SOARs require a ton of work by professional services or in-house engineering teams to get up and running so it's going to take some time before ROI shows. There can be a trough of disillusionment that occurs while that's happening."
—Dan Lamorena
Indeed, the Ponemon study found a connection between complexity and SOC effectiveness. Nearly three-quarters of the organizations surveyed (74%) acknowledged difficulty managing their SOCs because of their complexity. "As a result," the researchers wrote, "only about half of respondents (51%) say their organizations are highly effective in detecting attacks."
A survey of Fortune 1,000 companies released in February by CardinalOps, maker of an AI-powered threat coverage optimization platform, found that many of the rules and policies written for SIEMs are ineffective. For example, researchers found that an average of 25% of SIEM rules are broken and will never fire, primarily due to fields that are not extracted correctly or log sources that are not sending the required data.
In addition, they discovered that 15% of SIEM rules lead to 95% of the tickets handled by a SOC, demonstrating that a small percentage of noisy rules overwhelm SOC analysts with distracting false positive alerts.
Anton Chuvakin, a former Gartner analyst working on security strategy at Google Cloud, said that for many organizations, buying security technologies seems to be a much easier task than utilizing them and operationalizing them.
“In fact, there is a lot more guidance on 'Which tool to buy?' and 'How to buy security right?' than on how to actually make use of the tool in a particular environment.”
—Anton Chuvakin
2. Staffing challenges
The Ponemon report also found that difficulties with hiring, retaining, and paying SOC staff is contributing to ROI dissatisfaction. Because of shortages in expert people, it explained, the cost of staffing SOCs continues to rise. The average pay for a Tier 1 analyst is $102,315, an it's expected to keep going up.
Nearly half of the organizations surveyed by researchers (45%) predicted salaries to jump an average of 29% in 2020. The report said that more than half the costs of running a SOC are labor-related, with the average cost of maintaining a SOC being around $3 million — $1.46 million for labor, and $1.4 million for everything else.
Tim Wade, technical director for the CTO team at Vectra Networks, a provider of automated threat management solutions, said that finding and retaining talented security analysts are among the top pain points cited by security leaders.
Skilled analysts are difficult to find, and the supply seems to outstrip the demand by an order of magnitude. "This has been discussed ad nauseam," he noted.
"[Perhaps] more interesting is that most security operations center tooling is ineffective and leads to a combination of alert fatigue and unresolved, frustratingly dead-end investigations—which increases analyst burnout."
—Tim Wade
Burnout is a significant problem, the Ponemon study found as well. Some 70% of survey participants agreed that SOC analysts burn out quickly because of the high-pressure environment they're in and the crushing workload they're carrying.
Mark Manglicmot, vice president of security services at Arctic Wolf, a provider of concierge cybersecurity services, said that with many organizations doing security operations themselves, analysts struggle from being inundated with "a tsunami of alerts that they have to respond to."
"If analysts don't have enough coverage on their network, they may not be able to determine the root cause of a threat, so there's a lingering attacker presence in their environment. Then they end up playing whack-a-mole instead of holistically responding to attacks."
—Mark Manglicmot
Meanwhile, changes in working conditions due to COVID-19 have dialed up stress and workload levels for SOC staffers, said Charles Herring, CTO and co-founder of WitFoo, maker of a diagnostic security operations platform. "SOC analysts must work from home and don’t have access to some of the tools and tactics they once had."
"Protecting a centralized network gives them the option of blocking IP addresses, shutting down network ports, and physically performing forensics on devices inside of the organizations. The business changes created by COVID have reduced data and limited response tactics, making traditional SOC procedures obsolete."
—Charles Herring
Chris Hazelton, director of security solutions at Lookout, a provider of mobile phishing solutions, said it was a matter of doing the math to see how this is playing out.
"Before COVID-19, SOCs were focused on securing one or more offices. Now, SOCs are focused on securing hundreds or thousands of home offices."
—Chris Hazelton
The Ponemon report also noted that stress and workload contribute to turnover, which impacts SOC effectiveness. Almost two-thirds of the participants in the study told researchers that the time spent finding and training analysts to fill vacancies had a significant impact on the ability of SOC staff involved in the process to perform their other duties.
Keeping pace with turnover is challenging for organizations, the report added. It takes an average of nearly eight months to bring a new analyst online—3.5 months to find someone, and another 3.8 months to train them—while for every four analysts hired, three leave the organization during the same period.
While the pandemic may be contributing to turnover by increasing stress and workload levels, it may also be acting as a damper on it, too, said A.N. Ananth, president of Netsurion, a cybersecurity-as-a-service provider.
"Some employees are nervous about moving because they're uncertain about what the environment will be. That's helped us, because people who would have normally left have stayed put. It's also helped us because we're not restricted to hiring in a particular geography."
—A.N. Ananth
Ananth said the firm previously only considered people in the same geography as the SOC. "Now everyone is remote, so that doesn't matter anymore," he said.
Brandon Hoffman, chief information security officer at Netenrich, a provider of IT, cloud, and cybersecurity operations and services, said there are drawbacks for SOC analysts confined to their homes.
"Most SOC workers are used to a large chunk of remote work but still having the ability to come together with different members or teams for deeper triage. This has to be done remotely now, and the collaboration technology today doesn’t really allow for a comparable interactive experience, making interactive triage or war room more difficult."
—Brandon Hoffman
Hoffman noted that it was also more difficult to perform deeper hunting or triage in cases where physical access to the system is needed or strongly preferred.
3. Build in cyber resiliency
Despite the stress COVID-19 has imposed on SOC operations, some organizations have managed to minimize the pandemic's impact on their security operations. A recent study by Cisco found that those organizations share some common characteristics that contribute to their resiliency:
- They had a proactive tech-refresh strategy emphasizing frequent upgrades to best-of-breed IT and security technologies.
- They had adequate security staffing levels and invested in their people through role-based training programs.
- They kept top executives informed through clear reporting on the activities and effectiveness of the security program.
Wade Baker, a partner with the Cyentia Institute, a cybersecurity research firm, said the technology approach was critical to achieving cyber resilience.
"We interpret these results to suggest that an organization’s ability to maintain resiliency through unexpected events like the COVID-19 pandemic is strongly dependent upon a modern, high-performance tech stack maintained by capable personnel with strong accountability from organizational leadership."
—Wade Baker
While organizations may be disappointed in the bang they're getting from their SOC bucks, as the Ponemon/Respond study points out, the centers remain important to many organizations' security strategy.
Uri May, CEO of Hunters, an open XDE threat hunting company, said the time is now to to think about automation and better use of the tools and talent you have in place.
"Organizations have a lot of security tools and massive amounts of telemetry, but SOC processes are still manual. Most of SOC analyst time is still spent in rules writing, investigating alerts, and trying to figure out the root cause of incidents."
—Uri May
A modern SOC is key
The new generation of SOC technologies is being designed to solve these issues. Security tools are connected and the telemetry is seamlessly ingested from IT and security tools. Artificial intelligence and machine learning are replacing rules-based detection and IR, and remediation are built into workflow and have better automation.
"The SOC isn't going away, but technology and operations in the SOC will be different."
—Dan Lamorena
Tools for detection and response need to pull data from many sources, the FireEye executive continued, and new solutions need to be cloud-native. "We also need to recognize that the workforce is changing, and we need to bring new talent into the cybersecurity space and be more accepting of people who may not have the perfect background but can be trained up to meet our needs."
Keep learning
Learn from your SecOps peers with TechBeacon's State of SecOps 2021 Guide. Plus: Download the CyberRes 2021 State of Security Operations.
Get a handle on SecOps tooling with TechBeacon's Guide, which includes the GigaOm Radar for SIEM.
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed on cyber resilience with TechBeacon's Guide. Plus: Take the Cyber Resilience Assessment.
Put it all into action with TechBeacon's Guide to a Modern Security Operations Center.
