Small LLMs at the Edge are the engine for open source scalable AI agents

This talk, presented by PhD Luca Bianchi from Overnet, delves into the potential of small language models running on edge hardware as a foundation for scalable open-source AI agents. It challenges the traditional belief that effective AI systems rely solely on large, proprietary models and cloud infrastructure, proposing a shift to smaller, specialised models that can be deployed efficiently on commodity hardware. The speaker discusses techniques such as distillation and supervised fine-tuning that enhance performance in targeted tasks, while maintaining compliance with regulations like the EU AI Act by ensuring data sovereignty through local deployment. Furthermore, the session explores how open-weight models foster customisation and interoperability among AI systems, ultimately providing a comprehensive blueprint for building effective AI solutions that prioritise control over data and reduce reliance on external providers.