Fireside Chat – How Bad Actors AbUse LLMs The Mechanisms That Enable Misuse and What We Can Do About

This talk explores the ways in which malicious actors exploit Generative AI systems, addressing issues such as prompt injection, hallucination-driven supply chain attacks, and the creation of malware with large language models (LLMs). The speaker delves into the mechanics of LLMs and their vulnerabilities, particularly in applications powered by technologies like ChatGPT, Azure OpenAI, and various open-source models. Actionable defense strategies are discussed, focusing on building secure AI features through principles such as least privilege, prompt hardening, and careful model selection. Real-world instances demonstrating the use of Generative AI for capabilities like auto-triage and text-to-SQL are examined, along with the potential threats to these systems and methods for proactive defense.