The stories referenced in this episode
- Hackers use WormGPT to launch BEC attacks
- FraudGPT crafts fraud SMS messages and websites targeting banks
- The Hong Kong deepfake video call fraud, and how cybercrime has "removed the guardrails" on AI
- The Pikesville, Maryland athletic director deepfake audio case
- Researchers show Google's Gemini can be manipulated into leaking system prompts and giving harmful instructions
- Canadian cybersecurity officials on AI being used for hacking and misinformation
- Employees enter sensitive company data into GenAI prompts far more often than they realize
- Malwarebytes overview of AI cybersecurity risks, including virtual-kidnapping voice-clone scams and a documented ChatGPT chat-history leak between users
Further reading on AI and deception generally
- AI systems are learning to lie and deceive, scientists say (Washington Times)
- How smart is AI, really? A comparison to a child's reasoning (Washington Post)
Additional background on how hackers use AI
- 7 ways AI can be used by hackers to steal personal information
- The growing threat: how hackers use AI to attack companies
- Rise of the machines: emerging AI hacking tools
Book referenced
- Practical AI Security: A Hands-On Guide, by Harriet Farlow — see Practical AI Security - Harriet Farlow for full notes. Farlow's "3 D's" framework (Disrupt, Deceive, Disclose) is a useful mental model for sorting any AI security story you encounter: is this attack trying to break the system, trick it into a wrong decision, or leak something private?
Free practice resources mentioned in the source material (search by name)
- Lakera's Gandalf — a free, gamified way to see prompt manipulation firsthand
- MITRE ATLAS — the public knowledge base of real-world AI attack techniques
Jul 4 20268 mins
Jul 3 20264 mins