AI Engineering Podcast cover art

AI Engineering Podcast

AI Engineering Podcast

Written by: Tobias Macey
Listen for free

This show is your guidebook to building scalable and maintainable AI systems. You will learn how to architect AI applications, apply AI to your work, and the considerations involved in building or customizing new models. Everything that you need to know to deliver real impact and value with machine learning and artificial intelligence.© 2024 Boundless Notions, LLC.
Episodes
  • What Do Humans Do Now? Rethinking Software Engineering in the Age of AI Agents

    What Do Humans Do Now? Rethinking Software Engineering in the Age of AI Agents
    Oct 15 2026
    Summary In this episode, I spoke with Jonathan Ellis about how AI agents are changing the economics of software development and forcing teams to rethink where human time is best spent. We explored his argument that, now that code generation is dramatically faster, traditional human-centered code review is becoming a poor use of attention. Instead, Jonathan makes the case for shifting review earlier in the lifecycle toward specifications, RFCs, and architecture decisions, while investing more heavily in automated validation, static analysis, fuzzing, and other quality gates that can help verify whether software does what it is supposed to do regardless of how the code was written. Jonathan also shared lessons from building Brokk AI’s tooling, including Bifrost for fast, agent-friendly static analysis and Mjolnir for coordinating multiple coding agents across harnesses and subscriptions. We discussed the emerging need for better agent-oriented collaboration systems, the risks of blindly outsourcing judgment to AI, and the challenge of training newer engineers in a world where agents can do much of the implementation work. Overall, this conversation examines what remains uniquely valuable for humans as agents take over more of the mechanics of software delivery: setting goals, making product and design decisions, building better feedback loops, and deciding what is worth building in the first place. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systemsYour host is Tobias Macey and today I'm interviewing Jonathan Ellis about how to make the best use of human time when using agents to build softwareInterview IntroductionHow did you get involved in machine learning?Agentic coding has become much more viable this year, which dramatically shifts the bottlenecks in software delivery. What are the ways that you are seeing that impact engineering teams?As an industry we have built up numerous expectations and "best practices" around the software lifecycle, with many assumptions about the need for human review before sending code into production. How are you seeing those expectations impact teams who have shifted to agentic coding?Advanced teams have been working toward removing the human bottleneck in the validation and deployment path for several years now. What are the lessons that we can all learn from them?What are the new tools or practices that we need to build or adopt to gain the confidence that we need to deploy AI-written code to end-users?What are the highest leverage points of the software lifecycle that we can focus our limited human attention and review processes?What are you building that helps us work with agents to produce better software and build confidence in it?What lessons can we learn from the changes happening in software engineering that we can apply to the design and development of other agentic workflows?What are the most interesting, innovative, or unexpected ways that you have seen Bifrost/Mjolnir/etc. used?What are the most interesting, unexpected, or challenging lessons that you have learned while working on tools for agentic software engineering?When is the Brokk suite the wrong choice?What do you have planned for the future of Bifrost/Mjolnir/etc.?Contact Info LinkedInParting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story.To help other people find the show please leave a review on iTunes and tell your friends and co-workers.Links BifrostMjolnirStop Doing Code Review Blog PostSlopCopIntelliJADR == Architecture Decision RecordRFC == Request for CommentsWitan Context - Agent-native memory, tasks, and projects (built by Tobias)CoverityFuzz TestingApache CassandraAgent SkillsOpenAI Article - Using Exec Plans To Drive Agents For Longer TasksVectorize Hindsight SWE-BenchDeep-SWECursor BenchThe intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
    Show More Show Less
    1 hr and 8 mins
  • How to Evaluate RAG Systems When Ground Truth Keeps Changing

    How to Evaluate RAG Systems When Ground Truth Keeps Changing
    Oct 7 2026
    Summary In this episode Ofer Mendelevitch shares what it really takes to evaluate RAG systems in production when your source data is incomplete, constantly changing, or difficult to validate against a clean ground truth. He explores how RAG has evolved from simple “chat with your PDF” demos into enterprise-grade retrieval systems that require robust ingestion pipelines, hybrid search, reranking, multimodal support, access controls, and refresh strategies for large and dynamic document collections. Ofer also explains why evaluation becomes one of the hardest parts of the stack, particularly when retrieval quality, generation quality, and system behavior can all shift as you change chunking, models, prompts, or ranking components. He digs into practical approaches for offline and online evaluation, including synthetic question generation, sampling production traffic, and LLM-as-a-judge techniques such as UMBRELA and Auto Nuggetizer. Along the way, he discusses the rise of agentic RAG, context engineering, MCP-connected tools, text-to-SQL systems, and the growing role of coding agents in building and maintaining AI pipelines. Ofer closes with his perspective on where retrieval, memory, multimodal systems, and agent orchestration are headed next, along with a reminder that the biggest gaps in AI today are often not just technical, but educational and organizational as well. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systemsYour host is Tobias Macey and today I'm interviewing Ofer Mendelevitch about how to evaluate your RAG systems when you have incomplete or changing data to grade againstInterviewIntroductionHow did you get involved in machine learning?RAG was one of the first concepts to gain traction in the early explosion of LLMs and their application to real-world problems. How would you characterize its role in the current landscape?RAG is conceptually straight-forward, but can be quite complex to do well. What are some of the common pitfalls that teams encounter when bringing a retrieval-oriented LLM service into production?What are some of the predominant methods to avoid regressions or increased error rates as the data, code, and models change in a given deployment?There are numerous software projects and companies that offer various forms of evaluation/validation for RAG workloads. What are the factors that teams should be evaluating when selecting which one(s) to adopt?Especially when working with data that is subject to change (user-contributed content, changing products, news-based data, etc.), what are the techniques that teams can use to be sure that the generated responses based on that information is accurate?As models have become more powerful the idea of "agentic RAG" started circulating. How is that different from the first iteration of RAG systems?What new pressures and challenges does that prevent?What are the ways that it can improve or simplify a content-based generative system?How do you think about the differentiation of RAG and "context engineering"? What are the most interesting, innovative, or unexpected ways that you have seen teams address the production needs of RAG systems?What are the most interesting, unexpected, or challenging lessons that you have learned while working on RAG evaluation?How do you see the future of RAG, context engineering, and data-driven agents changing in the near to medium term?Contact InfoLinkedInParting QuestionFrom your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?LinksNaive BayesHands on RAG for Production (affiliate link)RAG == Retrieval Augmented GenerationVectaraHybrid SearchTF-IDF == Term-Frequency Inverse Document FrequencyBM25UMBRELAAutoNuggetizerAgentic RAGMalloyThe intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
    Show More Show Less
    1 hr and 3 mins
  • Harness Engineering for Reliable, Governed AI Agents

    Harness Engineering for Reliable, Governed AI Agents
    Sep 19 2026
    Summary In this episode Nikunj Bajaj, co-founder and CEO of TrueFoundry, talks about the challenge of building reliable agents on top of inherently variable foundation models. He explores the idea of the agent harness as everything around the model engine: memory and context management, tool and MCP integration, sandboxed code execution, permissions, observability, and guardrails. Nikunj explained how TrueFoundry approaches enterprise AI as a centralized control plane for token traffic, while TrueForge provides an open source, vendor-neutral harness for building agents without locking teams into a proprietary stack. He talks about the complexity of model routing across both the harness and gateway layers, the tradeoffs between unified APIs and provider-specific capabilities, and the architectural decisions that affect cost, accuracy, and reliability. He highlighted the importance of benchmark-driven validation, durable execution, human-in-the-loop workflows, sandbox security, and especially agent identity and least-privilege access across chains of agents. Overall, this was a deep look at harness engineering as an emerging discipline, and why organizations need more than isolated point solutions if they want to operate agents safely, efficiently, and at scale. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systemsYour host is Tobias Macey and today I'm interviewing Nikunj Bajaj about the challenges of keeping your agents reliable when the underlying models aren't and how your harness can helpInterview IntroductionHow did you get involved in AI?Can you describe what TrueForge is and the story behind it?A substantial amount of focus over the past few months has been on "harness engineering". What are the key aspects of the harness?What are the key challenges of building a harness that works well?How do you define what "works well" means given the massive variety of potential use-cases and operating contexts for an agent?Given the proliferation of harnesses, what are the key factors that someone should be looking at to determine which one to use?What are your opinions on why TypeScript and JS runtimes are such a popular choice for these harnesses?What are the key characteristics that you see in harnesses across different language ecosystems?Can you describe the implementation and architecture of TrueForge?How have the design and goals of the project evolved since you first started working on it?What are the elements of a harness that contribute the most to the cost efficiency, accuracy, and reliability (or lack thereof)?The general wisdom suggests that changing a model while keeping the same prompts for a given use-case can have wildly different outcomes. How accurate is that assumption today?What are the most interesting, innovative, or unexpected ways that you have seen TrueForge used?What are the most interesting, unexpected, or challenging lessons that you have learned while working on TrueForge?When is TrueForge the wrong choice?What do you have planned for the future of TrueForge?Contact Info LinkedInParting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story.To help other people find the show please leave a review on iTunes and tell your friends and co-workers.Links TrueFoundryTrueForgeFPGA == Field-Programmable Gate ArrayAI GatewayHarness EngineeringMCP == Model Context ProtocolA2A == Agent-to-Agent ProtocolPi Agent HarnessvLLM Semantic RouterTesting PyramidThe intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
    Show More Show Less
    1 hr and 4 mins
adbl_web_anon_alc_button_suppression_t1
No reviews yet