Building AI assistants for DSLs experiences and findings from Langium AI

This talk covers the process of building AI assistants for domain-specific languages (DSLs) using Langium AI, as presented by Benjamin Wilson from TypeFox. The session explores evaluation-driven development and practical implementation workflows, addressing key challenges such as enabling large language models (LLMs) to reliably generate and interpret DSL programs, especially under conditions of limited training data. The speaker discusses the iterative workflow of defining objectives, selecting a model stack, and refining evaluations to guide development. He emphasizes the role of synthetic data generation in expanding coverage and improving both the DSL and the AI assistant, ultimately demonstrating that structured evaluation and tool integration can enhance performance beyond traditional fine-tuning methods.