About this talk
This talk focuses on redefining the interaction with AI models by moving away from treating them as simplistic black-box REST endpoints. The speaker highlights the capabilities introduced with JDK 25 that allow developers to connect directly with large language models, image classifiers, and object detection algorithms using the Foreign Function and Memory API. Attendees will learn how to map tensor buffers to Java MemorySegment, manage execution providers for optimized CPU/GPU inference, and create self-contained Java applications. The session also delves into Project Babylon’s code reflection, enabling the expression of model logic in Java code that can efficiently be analyzed and converted to various accelerator backends. This approach aims to enhance AI model development by creating expressive and testable inference solutions directly in Java.
