By: Kevin Gonzalez, VP of Security, Operations, and Data at Anvilogic

‍

The cybersecurity landscape is constantly evolving. Threat actors continue to find new ways to exploit vulnerabilities, leveraging sophisticated methods to bypass traditional defenses, while Security Operations Centers (SOCs) are constantly just trying to keep up with their methods, and this dynamic shift has made the importance of foundational detection engineering practices more critical than ever.

Before more advanced frameworks can take center stage, organizations must first establish robust detection engineering and threat research practices that serve as the building blocks for effective detection and response. 

‍

Limitations of Solely Relying on ‘Protection’

Relying solely on the 'protection' granted by Endpoint Detection and Response (EDR) vendors is not a sustainable strategy in today's modern threat landscape. Attackers continually develop new EDR detection bypass techniques, leveraging generative AI to innovate ways of evading traditional protective controls. Moreover, many organizations cannot fully enable EDR protection features due to concerns about disrupting critical business processes, which further undermines the effectiveness of these solutions.

Generative AI is also accelerating the pace at which adversaries adapt their methods, making sole reliance on standardized, out-of-the-box EDR solutions increasingly inadequate. These limitations highlight the need for a more foundational, proactive approach that includes robust detection engineering tailored to the organization’s specific environment and threat landscape. Practical detection engineering ensures that detections are adaptable, relevant, and capable of addressing the unique challenges of modern adversaries, enabling organizations to stay one step ahead of emerging threats.

‍

The Challenges with Machine Learning on Raw Telemetry

Traditional machine learning techniques applied to raw telemetry have been largely unsuccessful in effectively replacing detection engineering efforts. User and Entity Behavior Analytics (UEBA) products attempted to fill this gap but have often failed. These systems are extremely difficult to maintain and update and often lack the contextual information required to classify a threat accurately. Even when alerts are raised, analysts spend more time determining the alerts' validity and meaning than focusing on the actual response efforts. This inefficiency has highlighted the critical need for purpose-built detection engineering that is supplemented, rather than replaced, by machine learning and analytics. 

Foundational detection engineering and threat research are not optional–they are necessities that pave the way for new detection paradigms to be implemented where data science techniques are applied in a manner where they are most effective. You can read more about one of these detection paradigms, the Detection Engineering Escalation & Recommendations (DEER) framework, here.

‍

The Importance of Detection Engineering

Detection engineering is the heart of any SOC. It is the practice of building detection logic that identifies indicators of attack (IoAs) or indicators of compromise (IoCs) within an environment. Foundational detection engineering provides the initial signals that fuel alert generation and informs subsequent analysts of events potentially worth triage, escalation, and response. 

For any organization to be successful against the ever-evolving threat landscape, foundational detection engineering practices must be solid and purpose-driven. These practices include:

• Atomic-Level Detections: The creation of detailed, specific detections that identify key actions taken by adversaries. These are not broad, generalized rules but focused detections that can provide high-quality signals. There are two main types of atomic-level detections: behavioral and signature.
• Detection-as-Code (DaC): Implementing software engineering principles in your detection engineering pipeline ensures structure in process and implementation with version-controlled logic, deployment pipelines, change control, auditing, and consistency across environments. 
• Contextual Enrichments: Effective detections go beyond just generating an alert; they also include enrichment, such as tagging relevant contextual information (e.g., use case information, MITRE techniques, threat group associations, and associated entity enrichments) that help SOC analysts to quickly profile threats and make informed decisions.
• Structured Detection Outputs: The output from foundational detection practices is standardized, context-rich data that serves as the basis for triage. Ensuring consistent naming conventions, data structures, and content in the detection outputs facilitates easier downstream analysis.

‍

Threat Research: The Engine Behind Detection

Detection engineering does not work in a vacuum. Effective detections require timely and accurate threat intelligence. This is where threat research plays a crucial role. The threat research team analyzes emerging threats, adversary behaviors, and evolving techniques to provide valuable intelligence that forms the foundation of detection engineering efforts. 

Threat research focuses on:

• Threat Modeling: Systematically identifying and evaluating potential threats to the organization’s environment. By understanding the likely attack paths and the methods adversaries might use, threat researchers can prioritize and inform detection efforts that address specific weaknesses, ultimately helping to strengthen the organization’s defensive posture.
• Adversary Technique Analysis: Researching and documenting adversary tactics, techniques, and procedures (TTPs) to inform detection efforts. This knowledge is crucial for creating effective atomic detections that address specific behaviors.
• Providing Indicators of Attack & Compromise: Threat researchers generate IoAs and IoCs that detection engineers can use for behavioral and signature detections. Behavioral detections are derived from the kinds of TTPs adversaries leverage for exploitation and compromise based on IoAs. Signature detections are based on artifacts that adversaries leave behind post-exploitation–these are based on IoCs.
• Guiding Detection Content Development: The threat research team collaborates with detection engineers to ensure that the developed detection content aligns with the most recent threat landscape and organizational needs. This collaboration ensures that the detection efforts are proactive, forward-looking, and applicable.
• Documentation: Document the specific use case that each detection is meant to address. Clear documentation helps ensure that the detection logic remains aligned with its intended output, assists detection engineers and triage analysts in understanding the intent of each detection, and simplifies future update and tuning efforts.

Read and subscribe to our Forge reports to discover the latest threat trends and adversarial tactics alongside ready-to-deploy detection. 

‍

Detection Engineering: The Foundation

The detection engineering team takes the insights from threat research and translates them into actionable detection logic. Their work forms the critical foundation of an organization’s ability to detect and respond to a particular threat while providing the building blocks for more advanced detections and workflows. The team is responsible for:

• Building Atomic-Level Detections: Translating threat research derivatives into highly specific detection rules that can generate high-quality alerts. These detections are primarily categorized into behavioral and signature-based detections. They ensure that individual components of an attack are identified early, allowing for rapid escalation and the ability to piece them together to see the broader attack path of a threat. 
• Enrichment and Tagging: Effective detections must be enriched with contextual data that provides meaning to the alert, helping analysts understand the threat and respond effectively. This enrichment turns basic telemetry into actionable information for response and higher-level detections.
• Building Sequential Detections: Using the enriched outputs of atomic-level detections to build sequential detection logic to identify more complex attack chains. This approach allows for the detection of defined multi-step attacks and provides a clearer picture of the threat’s activities for a more immediate response.
• Testing and Quality Assurance: Ensuring the detection logic is rigorously tested within lab environments to validate its effectiveness. This includes testing detections against known attack simulations to ensure they accurately identify threats without generating excessive false positives.
• Continuous Tuning and Improvements: Detection engineers also play a key role in tuning and improving detection logic based on analyst feedback. This iterative improvement process is critical in ensuring the effectiveness of detection rules over time. 

‍

Empowering SOCs with Purpose-Built Detection Engineering 

The foundational practices of detection engineering and threat research are essential to an organization’s ability to effectively identify and respond to threats. Relying solely on out-of-the-box EDR solutions or traditional machine learning approaches has proven insufficient in the face of attackers' evolving adversarial techniques and the increased use of generative AI. By focusing on detailed, atomic-level detections, enrichment, structured detection outputs, and proper testing, organizations create a proactive and resilient approach to threat detection. This foundation paves the way for more sophisticated detections and workloads, enabling organizations to stay ahead of adversaries and minimize the impact of potential incidents.

The journey towards effective threat response begins with strong, adaptable, and purpose-built detection engineering practices. With a robust detection foundation, organizations are well-positioned to enhance their defenses, reduce alert fatigue, and ensure that analysts can focus on what truly matters–protecting the organization from real threats.

‍

Watch our on-demand Detection Engineering Dispatch episode for a deep dive into the detection engineering role and discover essential skills, mindset, and strategies for success from Chris Black, Sr. Detection Engineer at NBCUniversal.