Season's Savings | ₹5/mo for 3 months

₹199/month after first 3 months. Renews automatically. Terms apply.
This is the worst AI tutor we'll ever have | DataCamp cover art

This is the worst AI tutor we'll ever have | DataCamp

This is the worst AI tutor we'll ever have | DataCamp

Listen for free

View show details
About this listen
DataCamp tested an early version of its AI tutor against its classic courses, and the tutor won: willingness to pay, engagement and satisfaction were all higher. It was also easily ten to twenty times more expensive per hour of learning than DataCamp's previous setup. Financially, Jonathan says, shipping the better product would have made the business worse.He expects that cost to fall until it looks like the cost of serving DataCamp's interactive courses. The only question, he says, is whether that takes four to eight weeks or another year.He's building the tutor on that bet.Jonathan Cornelissen is co-founder and CEO of DataCamp and is joined by Yusuf Saber, DataCamp's Chief AI Officer and the founder of Optima, the learning company DataCamp acquired. They walk Conor through how the tutor works: teaching behaviors it has to show on cue, evals that test each one repeatedly, a mix of models picked for speed and cost, and a latency budget measured in milliseconds.We cover:- How DataCamp turns what great human teachers do into about 30 core behaviors, and how the tutor gives feedback: one main issue at a time, hints phrased as questions, and asking before it reveals a solution- The A/B test against DataCamp Classic, and why Jonathan calls today's tutor the worst one DataCamp will ever have- The models behind the tutor: Claude Sonnet handles about 70% of the work, Gemini Flash used to handle much of the fast work, and a cheaper tier is starting to take on more- Why open models keep failing on speed, hosting and caching, even after one outperformed DataCamp's best model at the time- Going from almost nothing to a run rate of over a million hours of learning a year, and why the same three-hour course can take one learner half an hour and another six hours- Pre-generating responses while the learner thinks, and why a stronger model lets them aim for cache hit rates in the mid-90s- What Bloom's tutoring research says, why doing better on a test isn't proof of learning, and what happens to junior roles when AI takes the grunt workConnect with Jonathan Cornelissen:LinkedIn: https://www.linkedin.com/in/jonathan-cornelissen/Twitter/X: https://x.com/CornelissenJoDataCamp: https://www.datacamp.comDataCamp on X: https://x.com/datacampConnect with Yusuf Saber:LinkedIn: https://www.linkedin.com/in/yusuf-saber-ba380b15/Connect with Chain of Thought host Conor Bronsdon:Newsletter: https://newsletter.chainofthought.show/Twitter/X: https://x.com/ConorBronsdonLinkedIn: https://www.linkedin.com/in/conorbronsdon/YouTube: https://www.youtube.com/@ConorBronsdonMore episodes: https://chainofthought.showChapters:(0:00) Better product, worse economics(0:59) How the tutor helps a struggling learner(7:09) From static courses to AI tutoring(9:25) Multi-agent architecture behind the tutor(13:10) Scaling costs and testing open models(18:30) Economics of cost per learning hour(22:13) What great looks like for DataCamp(25:33) Latency budgets and precomputed lesson plans(32:53) Measuring learning outcomes with Bloom's research(39:20) AI skills gap for early-career workers(43:22) Evals built on tutor behavior expectations(50:12) Closing thoughts on how skills are changingThanks to CoralBricks for sponsoring this episode. CoralBricks serves open models through one API, fast and at low cost: DeepSeek V4.1 Flash streams at over 300 tokens a second. Get $5 in credits on signup, plus another $5 on your first purchase: https://www.coralbricks.ai/r/Z6V2REThanks to Walrus, presenting sponsor of season four of Chain of Thought. Walrus Memory gives AI agents portable, verifiable memory that carries context across apps, sessions, and other agents. Get started at https://walrus.xyz/cotThanks to Inngest, presenting sponsor of season four of Chain of Thought. Agents in production run long - they call models and wait on APIs and people. But the longer agents run, the more they break. Inngest handles that with durable execution. You build your agent as steps in TypeScript, Python, or Go. When a step fails, Inngest retries it with exponential backoff, and completed steps are saved and skipped. Try it out: https://inngest.link/cot-podThanks to WisdomAI for sponsoring this episode. In WisdomAI's latest benchmark, Claude Code alone topped out at 73% accuracy; WisdomAI closed the gap, delivering accurate answers 3x faster at about a third of the cost. Get "The Blueprint for Trusted AI Analytics": https://wisdom.ai/benchmark
adbl_web_anon_alc_button_suppression_t1
No reviews yet