DEEP DIVE INTO CYBER REALITY

SECURITY EFFECTIVENESS REPORT 2020

 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 3

Deep Dive Into Cyber Reality

This report focuses on an analysis of security controls effectiveness across the
multiple stages of attack lifecycles within 11 global industries. To gather data, our
experts executed thousands of tests comprised of real attacks, specific malicious
behaviors, and actor-attributed techniques and tactics. The report data provides
measured evidence of leading enterprise production environments across network,
email, endpoint and cloud-based security controls.

Our findings confirm the concern held by many security practitioners:

Security controls are not performing as expected.

Mandiant Security Validation1 experts generated evidence that current security

control configurations cannot consistently protect enterprises from elevated cyber
threat risks as previously assumed. This directly impacts core business objectives
such as continuity of business operations, security of corporate assets, delivering
evidence of regulatory compliance, and controls optimization.

1. Formerly Verodin Security Instrumentation Platform (SIP)

4 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

Table of Contents

Cyber Effectiveness as a Business Metric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

The Challenge of Measuring Security Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

The Impact of Macro Trends on Security Effectiveness. . . . . . . . . . . . . . . . . . . . . . . . . 8

Details on Seven Critical Security Challenges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

How to Improve Security Effectiveness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 5

Cyber Effectiveness
as a Business Metric
Measuring the effectiveness of and justifying the investment in security controls
has become a key performance metric for enterprises because boards of
directors and CEOs are expected to provide verifiable proof that business assets
are protected from the fallout of a potential breach. However, as organizations
begin to address cyber risk as a business problem, they also continue to manage
security as an IT function. This dynamic exposes the misalignment between IT,
which owns infrastructure, and the security team, which owns the cyber security
controls and processes that protect the business. Our experts have found that this
disconnect increases the need for security teams to generate reliable evidence of
effectiveness.

Security leaders report that they need to be able to confidently answer important
questions, such as:

• How effective are my security controls?

• How quickly can I assess the relevance of threat intelligence or my exposure to a
likely attack?
• How well do I stop data leakage and protect data integrity?
• How can I simplify and standardize my security stack?
• What evidence can I provide with key security metrics for my executives?

 
6 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

The Challenge of Measuring

Security Effectiveness
CIOs and CISOs continue to report the importance of being vigilant as they
validate and test security architectures. The challenges and complexities of having
unique environments, multiple teams and constant change requires that their
security programs evolve continuously. Security teams need a way to continuously
measure and monitor controls to capture quantitative evidence of security gaps
so they can demonstrate with evidence the ability to reduce risk and improve the
organization’s overall security posture.

The statistics outlined in this report were generated through careful analysis of
thousands of attack behaviors. These attack behaviors were executed in enterprise
production environments supporting over 900 million consumers, and against 123
market-leading security technologies, such as network, email, endpoint and cloud
solutions.

Industry verticals
11 123
Market-leading
security technologies

Enterprise
Production Environments

900 million
consumers affected
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 7

It is alarming that
9%
alerts are only generated for
of attacks

Our experts discovered:

• Security tools perform differently from one environment to the next

• Size of an organization has not proven to correlate to security effectiveness
• There is a disconnect between security team assumptions, expectations and
reality when we compare the effectiveness of organizations’ ability to alert,
block and detect threats

53% Missed

9% Alerted

26% Detected

33% Prevented

Definitions of Figure 1. Aggregated data for attack interactions. Total is greater than 100% because alerted
is a subset of detected and attacks can be either or both detected and prevented.
Attack Interactions
Missed An attack that
was not prevented or
detected. Many organizations are performing below their predicted levels of effectiveness.
Alerted Event raised to The data (Fig. 1) shows that many companies find a discrepancy between their
an analyst or response expected capabilities and the measured results. On average, they detect only 26%
level, typically through of attacks and prevent 33% of them, which provides an opportunity to optimize
a SIEM. their investments. It is alarming that alerts are only generated for 9% of attacks.
Detected Security
control creates an Altogether, this has a negative impact on incident response because SIEMs and
event identifying an other technologies responsible for triggering alerts cannot deliver a high level of
attack. fidelity to both prioritize and address security concerns.
Prevented Security
control successfully
blocks an attack.
8 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

The Impact of Macro Trends on

Security Effectiveness
Cloud
Moving workloads to cloud environments is commonplace today which introduces
security risks to the enterprise. Organizations and experts alike have highlighted
how this move complicates visibility and the ability to validate that controls such as
network segmentation and credential management operate as intended in a hybrid
model. Tests have shown that misconfigurations can expose data to the public
when new instances are created and policies are set incorrectly. Corporate assets
are also susceptible to risk when controls for specific business network zones are
accidentally bypassed due to misconfigurations.

Disconnect Between IT and Security

While security teams are responsible for protecting organizational assets, they do
not always have the corresponding operational authority or visibility into decisions
or changes being made that impact the infrastructure. This disconnect results
in “environmental drift,” which causes the organization’s risk posture to change
unexpectedly. In the absence of continuous validation of controls, this can put the
organization in a precarious position.

Technology Overload and Movement to Standardization of Controls

While our research suggests that on average, enterprises have 30-50 different
security tools, data in this report comes from organizations that can exceed
that number. This highlights the need to produce evidence of a specific
tool’s contribution to the overall security posture—evidence that supports
standardization of security controls and divestiture of technologies that no longer
add value.
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 9

Impact of Data Leakage

Organizations continue to highlight that protection of data and intellectual
property has become central to business objectives. Corporate data protection,
integrity and access are directly aligned with competitive advantage and valuation.
Evidence of an organization’s ability to protect data is a fundamental requirement
of reporting to boards of directors and executives. The rise in activity from nation
states, criminal actors and hacktivists, combined with the rapid growth of data and
the complexities of corporate networks, reinforce the need to constantly test and
validate specific controls and policies.

Host-based Controls
An over-reliance on host-based controls, which can be associated with a lack
of visibility into status of security controls, may cause additional exposure for
organizations.
10 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

Details on Seven Critical

Security Challenges
From Reconnaissance to Lateral Movement
Several attacker techniques and tactics are associated with challenges most
commonly found in enterprise environments when conducting testing through
security validation. They are analyzed here, and include real-world examples.

Security tools are often configured to address such challenges but may be poorly
optimized. The most common reasons for poor optimization include:

• Deployed under default “out-of-the-box” configurations

• Lack of resources to tune and tweak post-deployment
• Security events not making it to the SIEM
• Inability to force controls testing
• Unexpected changes or drift in the underlying infrastructure

ATT When we asked security executives, “How do you believe your controls are

&CK performing in each focus area?” many found that after executing an initial iteration
of testing, their production environments performed well below expectations
Operationalize Threat against these challenges:
Intelligence
The MITRE Adversarial • Reconnaissance
Tactics, Techniques &
• Infiltrations and ransomware
Common Knowledge
(ATT&CK) framework • Policy evasion
has emerged as a key
resource for security teams • Malicious file transfer
attempting the process of
• Command and control
defending against threat
actors. Technologies • Data exfiltration
designed to test or validate
security defenses offer new • Lateral movement
means to operationalize
threat intelligence. Security
teams can leverage
ATT&CK to perform gap
assessments on their
defenses and discover what
needs improvement.
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 11

4%
of reconnaissance activity
generated an alert

Reconnaissance
After testing network traffic, organizations reported only 4% of reconnaissance
activity generated an alert. This exposes the risk associated with misconfigured
controls, resulting in higher risks of successful scanning and profiling as well as a
high percentage of missed early stage attack tactics.

54% Missed

4% Alerted

26% Detected

37% Prevented

Common Causes
• Network segmentation misconfiguration
• Lack of internal security control points—inside network traffic is not monitored
the same way
• Inability to distinguish reconnaissance from normal network monitoring

Example
A Fortune 500 company leveraging security validation discovered an inadvertently
misconfigured proxy that was responsible for maintaining segmentation across
two regulated systems. This misconfiguration enabled communications between
networks and exposed a portion of the company’s critical internal business
network. With continuous validation in place, the security team was immediately
alerted on this change and the company quickly restored segmentation and
addressed exposure.
12 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

Controls did not prevent or detect detonation

within their environment

68%
of the time.

Infiltrations and Ransomware

After testing against infiltration and ransomware tactics, organizations reported
their controls did not prevent or detect detonation within their environment 68%
of the time.

68% Missed

7% Alerted

21% Detected

35% Prevented

Common Causes
• Deployed under default “out-of-the-box” configurations
• Unknown fail-open conditions in security controls
• Outdated or poorly maintained signatures

Example
During an initial testing period within a government entity, the security team
identified that their network firewall blocked only 24% of executed attacks. Using
detailed information that identified the attack patterns and behaviors, the security
team was able to work with the client’s vendor to optimize the firewall and
increase attack blocking capability to 74%.
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 13

65% of the time, security environments were

not able to prevent or detect the approaches beng tested.

Policy Evasion
When executing evasive focused attack techniques to bypass policies, 65% of the
time, security environments were not able to prevent or detect the approaches
being tested.

65% Missed

15% Alerted

25% Detected

31% Prevented

Common Causes
• Outdated classification categories
• Limited network monitoring on expected protocols
• Inadequate tracking and communication of changes for one-off exceptions

Example
A Fortune 500 company leveraged security validation to continuously monitor
for changes causing environmental drift, and the investigating team discovered
that data was not being delivered to the SIEM. After analyzing test results,
they discovered that syslogs were being sent over UDP instead of TCP and a
misconfigured load balancer was dropping all UDP traffic. As a result, events were
not being sent to the SIEM and correlation rules did not trigger alerts to initiate the
incident response process. The ability to test this with real attack actions exposed
this scenario and allowed the company’s security team to remove the risk.
14 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

48% of the time, controls in place

could not prevent or detect this stage
of the attack lifecycle.

Malicious File Transfer

When executing techniques and tactics associated with the delivery and
movement of malicious files, 48% of the time, controls in place could not prevent
or detect this stage of the attack lifecycle.

48% Missed

23% Alerted

29% Detected

37% Prevented

Common Causes
• Unaware of vendor removal of malware signatures
• Misconfiguration of existing security controls
• Under-resourced or aging sandboxing techniques and technologies

Example
An insurance provider leveraged security validation to test various network zones,
including areas designated as hardened. Test results provided evidence that 35%
of malicious file transfers attempted were allowed by the company’s security
tools and no alerts were generated in the SIEM for attempts that were detected
and prevented. Continuous security validation identified misconfigurations, and
this discovery resulted in the rapid optimization of security tools to minimize risk
exposure going forward.
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 15

97% of the behaviors executed

did not have a corresponding alert generated
in the SIEM.

Command and Control

Of the tested command and control activities, 97% of the behaviors executed did
not have a corresponding alert generated in the SIEM.

39% Missed

3% Alerted

26% Detected

40% Prevented

Common Causes
• Outdated or missing site classification
• Lack of SSL inspection
• Security events not making it to the SIEM

Example
To rationalize significant security investments and identify areas for divestiture, a
critical infrastructure customer in the energy sector leveraged security validation.
The team’s testing efforts identified areas of overlap in capabilities, inefficiencies in
product expectations and gaps in overall security posture. The findings provided
evidence to support cost reductions in endpoint technologies, correct alerting
gaps to the SIEM and deliver improved executive reporting through a third-party
analytics platform.
16 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

Exfiltration techniques and tactics were successful

67%
of the time.

Data Exfiltration
Data leakage and protection remains a top concern for CISOs, but exfiltration
techniques and tactics were successful 67% of the time during initial testing.

67% Missed

11% Alerted

31% Detected

29% Prevented

Common Causes
• Unknown fail-open conditions in security controls
• Lack of SSL inspection
• Misconfiguration of existing security controls
• Under-resourced sandboxing technologies or outdated signatures

Example
A Fortune 1000 company testing data loss prevention (DLP) policies and the
ability to stop data leakage observed that its next-generation firewall was not
blocking techniques used to exfiltrate data. security validation exposed the gap
and identified the misconfigured device. Further analysis showed that the firewall
vendor disabled detection capabilities in the latest release without making it
widely known to customers. With this new awareness, the company reconfigured
firewall policies and restored detection, prevention and alerting capabilities.
 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 17

54% of the techniques and tactics

used to execute testing of lateral movement are missed.

Lateral Movement
Lateral movement is an essential tactic to infiltration of a network. Fifty-four
percent of the techniques and tactics used to execute testing of lateral movement
are missed, and 96% of the behaviors executed did not have a corresponding alert
generated in the SIEM.

54% Missed

4% Alerted

FPO 26% Detected

37% Prevented

Common Causes
• Network segmentation misconfiguration
• Lack of internal security control points—inside network traffic is not monitored
the same way
• Inability to distinguish administrative behaviors from malicious activities

Example
A large private healthcare provider had concerns about APT41, a specific actor
reported to be actively targeting the healthcare industry. Leveraging security
validation with integrated threat intelligence, the company discovered that its
network security controls did not detect or prevent known techniques and
tactics associated with attacks used by APT41. This exposed the company to dual
espionage, criminal activity and over 46 different malware families. The testing
results enabled the team to proactively optimize their controls and ensure they
were prepared to defend against this adversary.
18 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

How to Improve Security

Effectiveness
Organizations often state that before using controls testing and configuration
validation platforms, they had to answer these questions:

• How effective are my security controls?

• How quickly can I assess the relevance of threat intelligence or my exposure to
a likely attack?
• How well do I stop data leakage and protect data integrity?
• How can I simplify and standardize my security stack?
• What evidence can I provide with key security metrics for my executives?

Security validation can quantify the actual effectiveness of security controls

because it provides continuous monitoring of unexpected changes or drift in
underlying infrastructure that may impact the performance of security controls.
As a result, you can gain the information you need to proactively outmaneuver
attackers.

Effective cyber security requires implementing an enterprise platform that

automates the key fundamentals of continuous security validation in order to
maintain a strong defensive posture and proactively reduce risk.

To improve cyber security effectiveness we recommend the implementation of

automated processes for continuous security validation (Fig. 2).

Continuous Validation

Assess Optimize Rationalize Monitor

Create a Baseline Establish and Reduce Costs Be Vigilant About
Maintain Effectiveness Environmental Changes

Figure 2. Process of continuous validation.

 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020 19

Fundamentals of Security Validation

Adversary Coverage
• Tests on both adversary techniques and technical attacks
• New content delivered quickly as threat actors evolve
• Coverage across adversary attack vectors—email, endpoint, and network
• Customizable content to maximize test relevancy for your organization

Validation Automation and Outcomes

• Infrastructure discovery and visibility
• Continually tests efficacy at scale
• Ability to execute an attack once or on a periodic and continual basis
• Exercises external and internal security controls across all network paths and
directions
• Graphical dashboards of the results of security effectiveness validation
• Reporting that includes quantitative results of attacks that demonstrate for
security leaders the overall results of security effectiveness validation

Business Metrics
• Provides metrics to assess business risk and value of investments

Enterprise Readiness
• Proven in large complex environments
• Backed by a global support team and customer success program

• Deploys safely in live production environments

• Deploys on-premises or in the cloud and available as customer managed,

co-managed, or fully managed
20 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

Conclusion

Companies are at much greater risk than they realize. As organizations—from the
C-suite and board of directors down to those on the front lines of cyber defense—
struggle to strengthen cyber hygiene and minimize risk, it has become imperative
that organizations validate security effectiveness.

Organizations make significant investments in security infrastructure, hire and train

teams and put processes in place to protect critical assets. But our research shows
that without evidence of security performance, those organizations are operating
on assumptions that don’t match reality and leave them with significant risk.

The best way for your organization to combat this disconnect is to validate the
effectiveness of your security program through ongoing, automated assessment,
optimization and rationalization. This will enable you to minimize cyber risk across
your entire organization by protecting not only critical assets but also brand
reputation and economic value.
22 SPECIAL REPORT SECURITY EFFECTIVENESS REPORT 2020

To learn more about FireEye, visit: www.FireEye.com

FireEye, Inc. About FireEye, Inc. About Mandiant Solutions

601 McCarthy Blvd. Milpitas, CA 95035 FireEye is the intelligence-led security The cyber landscape continues to grow in complexity as
408.321.6300/877.FIREEYE (347.3393) company. Working as a seamless, scalable adversaries become increasingly sophisticated and rapidly
[email protected] extension of customer security operations, morph their tactics. To outmaneuver motivated attackers,
FireEye offers a single platform that organizations need continuous security validation
blends innovative security technologies, technology powered by timely and relevant intelligence.
nation-state grade threat intelligence, and Mandiant Solutions, part of FireEye, brings together the
world-renowned Mandiant® consulting. world’s leading threat intelligence and frontline incident
©2020 FireEye, Inc. All rights reserved. FireEye
is a registered trademark of FireEye, Inc. All With this approach, FireEye eliminates the response data with continuous security validation to arm
other brands, products, or service names are complexity and burden of cyber security organizations with the tools needed to increase security
or may be trademarks or service marks of their
respective owners. for organizations struggling to prepare for, effectiveness and reduce business risk while protecting
M-EXT-RT-US-EN-000287-01 prevent and respond to cyber attacks. their reputation and economic value.