What does it actually take to automate security operations when you're processing 7 trillion log lines daily and a single missed threat could compromise billions of users? Michael Sinno, Director of Detection & Response at Google, explains how his team handles this with less than 1% requiring human intervention through strategic AI implementation. He explores Google's methodical approach to AI autonomy, including fine-tuned models trained on golden datasets, validation through overseer agents, and the critical distinction between traditional automation and agentic AI that exercises judgment.

Michael also discusses groundbreaking work with Sec-Gemini and Timesketch that enables forensic analysis to surface attack patterns humans would never detect manually. Michael shares concrete metrics like reducing executive incident notifications from 30 minutes to 90 seconds, achieving 95% precision in ticket deduplication, and automating vulnerability coordination from hours to minutes.

Topics discussed:

• Processing 7 trillion log lines daily with less than 1% of a million annual tickets requiring human intervention at Google

• Strategic evolution from AI-assisted to AI-led to autonomous security operations using fine-tuned models and golden datasets

• Building modular detection agents as pluggable components that can be combined like Legos for specific security use cases

• Implementing quality assurance through overseer agents that review other agents' work to ensure precision in security decisions

• Reducing executive incident notifications from 30 minutes to 90 seconds using AI-powered summarization and context gathering

• Achieving 95% precision in ticket deduplication while managing the trade-off between precision and 38% recall rates

• Integrating Sec-Gemini with Timesketch to surface attack patterns in forensic investigations that humans would never find manually

• Shifting from traditional detection and response to infer-and-interrupt models that contain threats immediately before escalation

• Automating vulnerability coordination workflows from hours to minutes through AI-powered data collection and impact analysis

• Distinguishing between traditional automation and agentic AI that exercises judgment rather than following if-then logic

• Setting a stretch goal of 70% automation in operations work while focusing humans on novel and complex security challenges

• Measuring success through time-to-mitigation metrics and evaluating AI performance against human baseline capabilities

Listen to more episodes:

Apple

Spotify

YouTube

Website

What if the real risk isn't adopting AI agents, but refusing to? James Nettesheim, CISO & Head of Enterprise Technology at Block, argues that principled risk-taking beats playing it safe. James shares Block's journey co-designing the Model Context Protocol with Anthropic and building Goose, their open-source general-purpose agent that enables anyone in the company to write security detections using natural language.

James also explores Block's Binary Intelligent Triage system achieving 99.9% accuracy, their data safety levels framework, and practical strategies for balancing autonomous AI capabilities with human oversight. James offers candid insights about implementing AI security principles, the evolution from tool experts to domain experts, and why open source remains fundamental to Block's mission of economic empowerment and technological innovation.

Topics discussed:

• Co-designing of MCP with Anthropic and developing of Goose as an open-source general-purpose AI agent

• Implementing prompt injection defenses and adversarial AI concepts to harden Goose against malicious instructions and attacks

• Rolling out AI responsibly through data safety levels modeled after CDC bio-contamination protocols for sensitive data handling

• Democratizing detection engineering by enabling anyone at Block to write detections using natural language

• Achieving 40% of new detections created with AI assistance through recipes, playbooks, and automated tuning capabilities

• Building Binary Intelligent Triage system that analyzes historical alerts and investigations to achieve 99.9% automated triage accuracy

• Balancing autonomous AI capabilities with human oversight, requiring PR reviews and maintaining accountability for agent-generated code

• Transitioning from tool expertise to domain expertise as the future skill set needed for detection and response professionals

• Block's commitment to open source development driven by economic empowerment mission and desire to build accessible financial tools

Listen to more episodes:

Apple

Spotify

YouTube

Website

Ryan Glynn, Staff Security Engineer at Compass, has a practical AI implementation strategy for security operations. His team built machine learning models that removed 95% of on-call burden from phishing triage by combining traditional ML techniques with LLM-powered semantic understanding.

He also explores where AI agents excel versus where deterministic approaches still win, why tuning detection rules beats prompt-engineering agents, and how to build company-specific models that solve your actual security problems rather than chasing vendor promises about autonomous SOCs.

Topics discussed:

• Language models excel at documentation and semantic understanding of log data for security analysis purposes
• Using LLMs to create binary feature flags for machine learning models enables more flexible detection engineering
• Agentic SOC platforms sometimes claim to analyze data they aren't actually querying accurately in practice
• Tuning detection rules directly proves more reliable than trying to prompt-engineer agent analysis behavior
• Intent classification in email workflows helps automate triage of forwarded and reported phishing attempts effectively
• Custom ML models addressing company-specific burdens can achieve 95% reduction in analyst workload for targeted problems
• Alert tagging systems with simple binary classifications enable better feedback loops for AI-assisted detection tuning
• Context gathering costs in security make efficiency critical when deploying AI agents across diverse data sources
• Query language complexity across SIEM platforms creates challenges for general-purpose LLM code generation capabilities
• Explainable machine learning models remain essential for security decisions requiring human oversight and accountability

Listen to more episodes:

Apple

Spotify

YouTube

Website