Skip to content

Beyond MITRE ATT&CK Coverage: The Importance of Data Completeness in SOCs

Original URL

Introduction

The focus in many Security Operations Centers (SOCs) is heavily weighted towards MITRE ATT&CK® coverage, which is understandable given its importance as an indicator of detection capabilities. However, equal, if not greater, emphasis should be placed on the data required to ensure these detections function effectively. This article explores the critical balance between rule completeness and data completeness for trustworthy and high-performing threat detection.

Main Points

The Symbiotic Relationship of Rule and Data Completeness

  • Effective detection relies on both rule completeness (coverage and accuracy of detection rules) and data completeness (coverage and data quality).
  • Trustworthy detections are achieved only when both rule and data aspects are robust. A detection rule might exist for a MITRE ATT&CK technique, but if the SOC lacks the relevant data or if the data is of poor quality, the detection will fail.

The Pitfalls of Over-Reliance on MITRE ATT&CK Coverage

  • Treating MITRE ATT&CK coverage as a mere checklist can create a false sense of security. The same technique can manifest in numerous ways, and detecting only one variation does not guarantee complete protection.
  • Organizations should focus on which procedures for a given ATT&CK technique are reliably detected and identify the gaps in their coverage. The framework is a valuable guideline for testing, validation, and continuous improvement, not a scoreboard.

The Data Completeness Imperative

  • Collecting and storing data without a clear purpose is wasteful. SOCs must define what data is needed to detect prioritized techniques, aligning data collection with specific detection goals.
  • Continuous validation of detections is essential to ensure the pipeline functions as expected and that the detections actually fire when relevant activity occurs.

Visibility and the Cost-Effectiveness Threshold

  • Achieving 100% visibility is unrealistic and not necessarily cost-effective. Organizations should strive for excellent visibility into a significant portion of their environment rather than poor visibility across the board.
  • A data lake strategy, combined with XDR and SIEM solutions, can help balance cost-effectiveness with completeness, allowing for the collection of data that may not be immediately analyzed but can be leveraged later.

The Importance of a Risk-Based Approach

  • Rather than aiming for blanket MITRE ATT&CK coverage, SOCs should adopt a risk-based approach. Start with the desired detection outcomes and work backward to identify the necessary logs and context.
  • This approach ensures that detection efforts are relevant to business risks and that analysts focus on meaningful alerts rather than being overwhelmed by noise.

Detection Resilience and Addressing Detection Gaps

  • Detection gaps pose a greater problem than false positives. Optimizing solely for precision while neglecting recall leaves the organization vulnerable.
  • AI-driven SOCs can reduce the burden of false positives, freeing up analysts to focus on addressing detection gaps, even if it means employing less precise detections.

Integrating Offense with Defense

  • Defense alone is insufficient. Organizations should integrate offensive strategies with defensive measures to disrupt the adversary's supply chain within their network.
  • Focus prevention and hardening efforts where they can inflict the most pain on attackers, prioritizing actions based on threat models and concentrating on critical assets.

Continuous Threat Exposure Management

  • Traditional defenses are not keeping pace with adaptive adversaries. Continuous Threat Exposure Management (CTEM) enables security teams to prioritize and remediate critical gaps.
  • CTEM transforms data into actionable insights, allowing teams to strengthen password and data theft defenses and measure the impact of their risk reduction efforts.

Pentest ROI Beyond Vulnerability Counts

  • The return on investment (ROI) of penetration testing should not be measured solely by the number of vulnerabilities identified. Instead, focus on metrics that matter to the board, such as the percentage of findings remediated, time-to-fix for high-severity issues, and recurring exposure trending.
  • The true measure of pentest ROI is how much harder it is to break into the organization today compared to last quarter, showcasing the effectiveness of security improvements.

Conclusion

While MITRE ATT&CK coverage is a valuable metric, SOCs must avoid treating it as the ultimate goal. A balanced approach that prioritizes both rule completeness and data completeness, coupled with a risk-based strategy and continuous validation, is essential for building a robust and effective security posture. Embracing strategies like CTEM and focusing on meaningful metrics will ensure that security efforts translate into tangible risk reduction and improved resilience against evolving threats.