Beyond The Pilot: Enterprise AI in Action cover art

Beyond The Pilot: Enterprise AI in Action

Beyond The Pilot: Enterprise AI in Action

Written by: VentureBeat
Listen for free

About this listen

 AI gets real here. On “Beyond the Pilot,” top business execs share what actually happens after the AI proof of concept — from infrastructure and org design to wins, failures, and ROI. Not theory, but deep dives into how they scaled AI that works.© 2025 VentureBeat. All rights reserved. Politics & Government
Episodes
  • LexisNexis on Why Standard RAG Fails in Law

    LexisNexis on Why Standard RAG Fails in Law
    Feb 18 2026
    On February 2nd, a single plugin wiped nearly $800 billion off the enterprise software market. Wall Street is terrified that AI agents are about to eat the legal industry's lunch. But LexisNexis isn't scared—they're building the moat. In this episode of Beyond the Pilot, Min Chen (Chief AI Officer, LexisNexis) reveals the sophisticated architecture they built to counter the "LLM wrapper" revolution. Moving beyond standard RAG, Min breaks down their move to "GraphRAG", their deployment of Agentic workflows (using Planner and Reflection agents), and why they created a proprietary "Usefulness Score" because standard accuracy metrics weren't good enough for lawyers. AI Gets Real Here. No theory, just the execution roadmap for deploying AI in a zero-error environment. In this episode, we cover: The "Dangerous RAG" Problem: Why semantic search fails in professional domains (retrieving "relevant" but overruled cases) and how "Point of Law" knowledge graphs fix it. The "Usefulness" Metric: The 8 sub-metrics LexisNexis uses (including Authority, Comprehensiveness, and Fluency) to grade AI quality. Agentic ROI: How deploying a "Planner Agent" to break down complex questions increased answer usefulness by 20%. The "Reflection Agent": Using a secondary agent to critique and refine drafts in real-time. Hallucination Detection: Why you should never rely on an LLM to judge its own hallucinations (and the deterministic code they use instead). ⏱️ TIMESTAMPS 00:00 - Intro: The $800 Billion AI Threat to Legal Tech 02:18 - Min Chen’s Journey: From Feature Engineering to Chief AI Officer 05:55 - Why Standard RAG Fails in Law (and How GraphRAG Fixes It) 10:40 - "Accuracy" is a Vanity Metric: The 8-Point Usefulness Score 14:20 - The "Auto-Eval" Framework: Human-in-the-Loop at Scale 16:40 - The Secret Sauce: Don't Use LLMs to Detect Hallucinations 21:15 - Agentic AI: How "Planner Agents" Drove a 20% Gain 22:00 - The "Reflection Agent": Self-Critique Loops for Drafting 30:30 - Distillation: Balancing Cost, Speed, and Quality 32:45 - Min’s Advice: Don't Build the Product First (Build the Metrics) Presented by Outshift by Cisco Outshift is Cisco’s emerging tech incubation engine and driver of Agentic AI, quantum, and next-gen infrastructure. Learn more at outshift.cisco.com. About VentureBeat: VentureBeat equips enterprise technology leaders with the clearest, expert guidance on AI – and on the data and security foundations that turn it into working reality. 🔗 CONNECT WITH US Subscribe to our Newsletters for technical breakdowns: https://venturebeat.com/newsletters Visit VentureBeat: Venturebeat.com . . . Subscribe to VentureBeat: / @VentureBeat . . Subscribe to the full podcast here: Apple: https://podcasts.apple.com/us/podcast/venturebeat/id1839285239 Spotify: https://open.spotify.com/show/4Zti73yb4hmiTNa7pEYls4 YouTube: https://www.youtube.com/VentureBeat Learn more about your ad choices. Visit megaphone.fm/adchoices
    Show More Show Less
    36 mins
  • Mastercard's 160 Billion Transactions: AI's Biggest Test

    Mastercard's 160 Billion Transactions: AI's Biggest Test
    Feb 4 2026
    While most of the world is still running GenAI pilots, Mastercard is running AI inference on 160 billion transactions a year—with a hard latency limit of 50 milliseconds per score. In this episode of Beyond the Pilot, Johan Gerber (EVP of Security Solutions) and Chris Merz (SVP of Data Science) open the hood on one of the world's largest production AI systems: Decision Intelligence Pro. They reveal how they moved beyond legacy rules engines to build Recurrent Neural Networks (RNNs) that act as "inverse recommenders"—predicting legitimate behavior faster than the blink of an eye. AI Gets Real Here. This isn't just about defense. Johan and Chris detail how they are taking the fight to criminals by leveraging Generative AI to engage scammers with "honeypots," expose mule accounts, and map fraud networks globally. In this episode, we cover: The 50ms Inference Challenge: How Mastercard optimized their RNNs to score transactions at a peak rate of 70,000 per second. "Scamming the Scammers": How GenAI agents are being used to automate honeypot conversations and extract mule account data. The "Inverse Recommender" Architecture: Why Mastercard treats fraud detection as a recommendation problem (predicting the next likely merchant). Org Design for Scale: The "Data Science Engineering Requirements Document" (DSERD) Chris used to align four separate engineering teams. The Hybrid Infrastructure: Why moving to Databricks and the cloud was necessary to cut innovation cycles from months to hours. 🚀 CHAPTERS 00:00 - Intro: 160 Billion Transactions & 50ms Decisions 02:08 - Thinking Like a Criminal: Johan’s Law Enforcement Background 06:22 - Org Design: Why AI is the "Middle Lane" of Engineering 11:00 - The Scale: 70k Transactions Per Second 15:47 - Decision Intelligence Pro: The "Inverse Recommender" RNN 23:00 - The "Lego Block" Strategy: Aligning Data Science & Engineering 33:00 - Infrastructure: Why Cloud/Databricks was Non-Negotiable 37:00 - GenAI Offensive: Threat Hunting & "Scamming the Scammers" 46:40 - "Honeypots" and Detecting Mule Accounts 52:00 - Advice for Technical Leaders: Talent & Prioritization Presented by Outshift by Cisco Outshift is Cisco’s emerging tech incubation engine and driver of Agentic AI, quantum, and next-gen infrastructure. Learn more at outshift.cisco.com. About VentureBeat: VentureBeat equips enterprise technology leaders with the clearest, expert guidance on AI – and on the data and security foundations that turn it into working reality. 🔗 CONNECT WITH US Subscribe to our Newsletters for technical breakdowns: https://venturebeat.com/newsletters Visit VentureBeat: Venturebeat.com . . . Subscribe to VentureBeat: / @VentureBeat . . Subscribe to the full podcast here: Apple: https://podcasts.apple.com/us/podcast/venturebeat/id1839285239 Spotify: https://open.spotify.com/show/4Zti73yb4hmiTNa7pEYls4 YouTube: https://www.youtube.com/VentureBeat Learn more about your ad choices. Visit megaphone.fm/adchoices
    Show More Show Less
    56 mins
  • Inside LinkedIn’s AI Engineering Playbook

    Inside LinkedIn’s AI Engineering Playbook
    Jan 21 2026
    While the rest of the industry chases massive models, LinkedIn quietly achieved a major engineering breakthrough by going small. In this episode of Beyond the Pilot, Erran Berger (VP of Product Engineering, LinkedIn) opens the "cookbook" on how they distilled massive 7B parameter models down to ultra-efficient 600M parameter "student" models—scaling AI to 1.2 billion users without breaking the bank. AI Gets Real Here. This isn't theory. Erran details the exact architecture, the "Multi-Teacher" distillation process, and the organizational shift that forced Product Managers to write evals instead of specs. In this episode, we cover: The Distillation Pipeline: How to train a 7B "Teacher" and distill it to a 1.7B intermediate and 0.6B "Student" for production. Synthetic Data Strategy: Using GPT-4 to generate the "Golden Dataset" for training. Multi-Teacher Architecture: Why they separated "Product Policy" and "Click Prediction" into different teacher models to solve alignment issues. 10x Efficiency Hacks: Specific techniques (Pruning, Quantization, Context Compression) that slashed latency. Org Design: Why the "Eval First" culture is the new requirement for AI engineering teams. 🚀 CHAPTERS 00:00 - Intro: LinkedIn's Massive "Small Model" Feat 04:00 - Why Commercial Models Failed at LinkedIn Scale 08:00 - The "Product Policy" Funnel & Synthetic Data Generation 12:00 - The Pipeline: 7B → 1.7B → 600M Parameters 19:00 - The "Multi-Teacher" Breakthrough (Relevance vs. Clicks) 23:00 - How They Achieved 10x Latency Reduction (Pruning/Compression) 31:00 - Changing the Culture: Why PMs Must Write Evals 35:00 - The "Bright Green Matrix": Measuring Success & Future Roadmap Presented by Outshift by Cisco Outshift is Cisco’s emerging tech incubation engine and driver of Agentic AI, quantum, and next-gen infrastructure. Learn more at outshift.cisco.com. About VentureBeat: VentureBeat equips enterprise technology leaders with the clearest, expert guidance on AI – and on the data and security foundations that turn it into working reality. 🔗 CONNECT WITH US Subscribe to our Newsletters for technical breakdowns: https://venturebeat.com/newsletters Visit VentureBeat: Venturebeat.com . . . Subscribe to VentureBeat: / @VentureBeat . . Subscribe to the full podcast here: Apple: https://podcasts.apple.com/us/podcast/venturebeat/id1839285239 Spotify: https://open.spotify.com/show/4Zti73yb4hmiTNa7pEYls4 YouTube: https://www.youtube.com/VentureBeat #EnterpriseAI #LLMDistillation #LinkedInEngineering #SmallLanguageModels #AIArchitecture #TechLeadership Learn more about your ad choices. Visit megaphone.fm/adchoices
    Show More Show Less
    41 mins
No reviews yet