AI SOC Analysts — LLMs Find a Home in the Security Org
| Author | Shayan Shafii (Scale Venture Partners) |
|---|---|
| Published | September 24, 2024 |
| Format | Article |
| Original | Read the original |
Security Operations Center work is investigative, high-volume and high-turnover, which is exactly the shape of task that LLMs and agent frameworks can now do, so nearly every SOC will soon run an “AI Analyst,” and the open question is whether buyers want full autonomy or a copilot.
AI SOC Analysts — LLMs Find a Home in the Security Org
Published September 24, 2024.
Key claims
- The SOC is under-resourced against AI-enabled attackers. Threat actors adopted LLMs first (hyper-personalized phishing, LLM-generated malicious scripts, pre-built attack agents) while defenders kept “yesterday’s tooling.” Shafii predicts nearly every SOC will be assisted by an AI Analyst, by analogy to GitHub Copilot becoming “a several hundred million ARR business.”
- Three tiers of analyst. Tier 1 triages every alert to a yes/no verdict; Tier 2 runs the deeper root-cause investigation and writes the report for auditors and regulators; Tier 3 does proactive threat hunting. Tiers 1 and 2 are reactive, Tier 3 is proactive.
- Tier 1 is the pain point, quantified. The average SOC processes ~4,500 alerts a day and still handles only ~50% of them. Investigations are expected inside 30 minutes of the alert, with some SLAs as low as five minutes. Tier 1 analysts last one to two years in the role, so institutional knowledge is constantly lost.
- The technical unlock is flexibility. Rule-based automation could not follow a clue off-script. LLMs can extract variables from unstructured alert text, generate step-by-step plans, and write bespoke API calls or SQL. Investigations that took 30–60 minutes of human time now take an AI Analyst seconds.
- A six-step agent workflow. Comprehend the alert; generate an initial investigation plan (task planning, citing ReAct and LangChain); execute tasks against security tools (Okta, CrowdStrike, Palo Alto Networks), internal knowledge bases (Slack, Jira, Confluence, ServiceNow) and external sources (the National Vulnerability Database); recursively adjust the plan like a depth-first search (e.g. 5,000 login attempts from one IP in an hour prompts “were any successful?”); reach a verdict; and show its work, which buyers backtest against historical alerts.
- A crowded market map. Fourteen startups named: Dropzone AI, Prophet Security, Seven AI, Exaforce, Culminate Security, Radiant Security, AirMDR, Qevlar AI, Command Zero, Legion Security, Conifers AI, XBOW, Runsybil and Specular. AI pentesting tools are placed in the Tier 3 bucket because the buyers may merge.
- Open question one is autonomy. Founders and buyers split on software-only vs. human-in-the-loop. Shafii frames it with the five levels of self-driving autonomy (Autonomous Vehicles): if technology is the bottleneck, it will be overcome; if buyers want human eyes, Tier 1 analysts become reviewers of the software’s decisions and Tier 2 gets a copilot.
- Differentiation comes from accuracy, MTTR and integrations. Accuracy improves locally (organizational context in one customer, raising switching costs) and globally (network effects across customers), as with the last generation of ML detection companies (CrowdStrike, SentinelOne, Abnormal Security). Mean time to recovery is the metric SOC teams live by, and depth of read/write integrations is “make-or-break” in purchasing.
- Incumbents are a threat but may only ship “cruise control.” CrowdStrike Charlotte AI, SentinelOne Purple AI, Microsoft Copilot for Security and Tines Workbench are where SOC managers try AI first. Shafii’s two doubts: a bolted-on chatbot is not Level 5 autonomy, and CISOs want an independent analyst with visibility across endpoint, identity and cloud rather than more vendor lock-in. Tines Workbench is singled out for cross-domain visibility.
Notable quotes
“In other words, they’ve been bringing a SIEM to an AI fight.”
“I believe that nearly every SOC on Earth will be soon assisted by some form of AI Analyst.”
“Recent studies have suggested that the average SOC processes around ~4,500 alerts per day. SOC managers try to size their teams to meet this workload, but the volume is so great that the average SOC still only processes about ~50% of the alerts produced on any given day.”
“With this context in mind, it’s no surprise that Tier 1 SOC Analysts are among the highest-turnover jobs in the enterprise.”
“Investigations that previously required 30-60 minutes of a person’s time can now be completed by an AI Analyst in seconds.”
“Just like math class, if you don’t show your work, you don’t get any points.”
“In the self-driving car framework, they would be more akin to ‘cruise control’ than Level 5 Autonomy.”
“SOCs are expensive, manual, and imperfect, but most importantly, they are business critical.”
Charts and figures



How it connects
- Security Operations Center (SOC) and SIEM are the concept pages for the system this piece wants to automate; Cybersecurity is the broader category.
- AI Agents: the plan, execute, re-plan, conclude loop described here is a concrete agent architecture in a domain where the output (a verdict plus an audit trail) is checkable.
- GitHub Copilot is the explicit analogy for the adoption curve, and Autonomous Vehicles supplies the autonomy-levels frame for the copilot-vs-autopilot debate.
- CrowdStrike, SentinelOne and Microsoft appear twice, as integration targets and as incumbents shipping their own AI investigators, the platform vs. best-of-breed tension in miniature.
- Scale Venture Partners is the firm; Shayan Shafii the author.