About this talk
This talk discusses the integration of AI assistance in modeling tools, specifically focusing on Papyrus Web and Season. The speakers explain the necessity for AI support in addressing the complexities of modern software development, including safety, liability, and collaborative processes among various stakeholders. They propose employing model-driven engineering to facilitate compliance and automation in development workflows, detailing the iterative refinement processes that incorporate AI to enhance development and testing. Significant emphasis is placed on the implementation of an agent-based model to enable interaction with UML models through a new API system, enhancing user experience and fostering collaboration. The talk also introduces ongoing projects that explore AI integration within different domains such as robotics and automotive software, aiming to improve the trustworthiness and usability of modeling tools.
Full transcript
[music] >> So, I'd like to welcome you to our talk about AI assistant modeling with Papyrus web and Season. And also bit with Capella. And the talk will be given together with my colleague Julia Bonai and myself Anskar Radermacher and we also use some material from our colleagues and our lab. So, we are here a brief overview of the agenda and we start with the motivation. Why
do we need AI support for modeling? well, as you all know, software development has become more and more complex. You have to deal with additional aspects like product liability, functional safety, real-time certification issues and as we had heard today this morning, there is also um the cyber safety which is becoming more and more important and you want to develop these systems with mature process. So, I have
shown here a simplified diagram from Ebert et al. And So, you need to support some methodologies to develop these systems since you're dealing with different concerns. You also want collaboration support because there are different stakeholders that help you to develop these systems. And in order to all these requirements related to uh you need to have traceability. You have to make sure that your requirements are covered, that
you uh demonstrate your conformity with regulations and automation maintenance. And for all these tasks, you want to have some suitable automation. uh you could Well, you can use model-driven engineering uh because it supports well these different concerns. You can have different models for these aspects. But uh in order to produce good models, this requires a lot of expertise. And therefore, this automation is something which uh can
really be useful also to to become an expert in tool usage might be in a something which prevents people to use these tools. So, how can AI help? And uh well, we are in our lab uh um we are doing applied research and we are look looking at the different ways uh how can I AI support us to be integrated into the engineering processes, uh for which
tasks, which artifacts within the models we can use that. And uh if you are having a complex system, for instance, in the software-defined vehicle domain, um we have to deploy it on a hardware. We also we also need hardware models. And um uh as I already said, there's this a collaboration uh between different stakeholders to define such a complex system. And we can imagine that AI is
also one of the participants that uh collaborate on a model. So, which are the interaction mechanisms that we can use? very important question is if you get some AI results, we have to evaluate them. We have to have confidence in them. We are have a suitable process to integrate these results and remain confident in the produced software systems. And one question that you might ask is, well,
we have these powerful AI mechanisms, these everyday more powerful AI models, couldn't I just uh take all my requirements, um safety, security, and some design goals, and then with a very elaborated prompt ask my AI to generate the whole system? Well, you can do that. For simple systems, this might work, but we have this verification analysis of everything is really conformant with the requirements, and this is
just not realistic to do for complex system because you don't have any intermediate results. You have directly the executable and deployable code. And if you are not happy with the results, that's difficult to analyze. Um then you can just refine your your prompt, but it's difficult to obtain such a system and at the same time demonstrate the conformity with regulation. So, instead we propose model-driven engineering process.
And well, this might look like a V process where the right hand is missing but it's rather that we are well we are proposing to integrate mechanisms like shift left and and shift right so that we have at each level here already some validation steps. So, when we are looking at the requirements, we have some refinements of the requirements where AI can already help us. So, we
have the feedback and analysis on this level. We have the model driven specification, functional specification, design document. And we can go to code generation and deployment and at each level we can have some devops that put tests in the loop, that put analysis tools in the loop. by this way this iterative refinement, we will arrive to create that fulfills all the requirements and corresponds to regulation. an
AI coach can help in this process, automate some of the aspects and thus help us create a a good system. And by doing all these steps, we master the architecture. We have a separate intermediate design document for the We can run safety and security analysis on this and we can run verification and validation on the immediate models. And this is a teaming up between human and AI
we can have uh, specialized agents for specific tasks like, uh, completion and analysis. We also have, well, with respect to design uh, knowledge, these can be uh, covered by means of well-known architectures, well-known solutions, design patterns, and we have an ongoing PhD thesis to analysis analysis this. And, uh, we can also generate test cases as has been done in a in a project with a with a
partner. So, uh, this is the basic process that we like to do, and we want to do that in the context of some modeling tools that we have developed. Um, initially, uh, as I was saying, since the early 2010, uh, years, there uh, an Eclipse-based project that is called Papyrus. So, this was, uh, or still is a UML-based modeler for the, uh, Eclipse desktop. So, it's based
on, uh, open standards, the OMG standard UML. And it's, uh, highly customizable. Uh, it can be adapted to a specific domain. We can, for instance, with extensions, support code generation. Uh, there is uh, Papyrus software designer. There's also a Papyrus version for robotics. And this is an open open-source Eclipse-based tool, and you can download it at the webpage here. What you see here, some elements like the
model explorer and, uh, diagrams, yeah, in this case, representing a state machine. something which is very new is that we have started, uh, um, uh, migration of this tool to the web. So, there is the Papyrus variant for the It's uh, accessible in the browser, so it supports collaborative modeling. It's also configurable and extensible, and it's based on Sirius web collaboration that we have uh with Obeo.
And um here the interaction with your team members can be done with uh Matrix element. So, it's another open-source tool that we integrated into our solution. uh here's another tool that is uh the result is the result of a collaboration with Obeo. You see here SysON, a SysML V2 uh based modeling tool um that is based on the same technology as Papyrus web, so on Sirius web
from Obeo, and you can download it here. And uh you have probably I forgot to mention that my colleague is very proud of that we have now a sandbox version of SysML Papyrus web. So, that you can directly try out or after the session to to get your models. And with respect to AI integration, we we first had the idea that we uh if if user has
some prompts to ask some queries about the model, then we pass by uh uh small textual format of the UML model achieve with PlantUML. so, whenever you post a post a question in in the prompt, um then I implicitly this uh text version of the UML model was available, and you get a response related to that, but you still had to still integrate the responses from the
model. And a new thing is that we now are have uh an agent based model that uh uh no longer relies on on the textual representation of the model, but we are relying on an pilot module that is able to interact with the UML model. So, it's an an AI agent that is able to use an MCB protocol to interact the UML model. it's piloted by an
orchestrator that is called Numeribus and uh all of this will be detailed later by my colleague. >> The presentation of this AI assistant will be done in two parts. Uh in the first part, I will show you the result of our research, the demonstration of Numeribus as Ansgar mentioned. And in the second part, I will explain the different steps we worked on to get these results. So,
this is Papyrus Web semantic view, the graphical view, and the AI view called Numeribus. >> We >> You first need to say to the AI where you're working on. So, I Sorry, the presentation is in French. I will translate each each part of this presentation if it loads. >> Yeah, if it's not loading, we can uh we can run directly the downloaded. Yeah. >> Okay, so I
say I'm working on the Pizzaria uh project because it's a Pizzaria representation. Uh and in the Pizzaria uh document will be able to uh found this model and to describe it. So, yeah. It founds all of the the classes, ingredients, clients, command. But, you can also uh ask the AI more details about this this model. So, what this model represents? This model represents uh gestion of a
Pizzaria with the the client class contain properties like You can ask the AI to create new classes. Here, I ask to add the class uh delivery. So, it creates a class in the semantic view and in the graphical view. It calls the tool from Papillot. I will describe this later. And you can you can also ask the AI to add multiple multiply elements at the same time
like three new properties in this new class like name, uh first name, and address. You can ask AI to delete the last changes. I made a mistake. Please delete last modifications. And here the The new properties have been deleted. And you can also ask the AI to add links between classes. Here, add a link between pizza and ingredients, and it creates the link So, now on the
architecture of this workflow, uh to be able to make the modification thanks to AI, we need to create an API. We call this API Pilot because it pilots And this API is completely independent from Papyrus Web. It works on a completely different um server. It can access models uh on projects. So, it can retrieve all the components from this project. And it can modify the semantic and
the graphical parts of uh Pap- the project in Papyrus Web. Uh more than 30 different commands are available in uh Papyrus Web in Pilot, sorry. And uh this API uh acts as a GraphQL client connected to server and ask for queries, changes, or subscriptions. So, we got we've got an API, but this API can be directly called by an LLM. We need to change this API API
to uh MCP server. Uh so, that's the second version as of Pilot, which is Pilot V2. And each of the REST API commands are now MCP tools. Papyrus hosts the MCP server part and it's accessible from an MCP client. And the MCP tools created by this MCP server are usable in any AI agents. Uh we also I can show you a uh demonstration of Papyrus. >> Yeah.
>> Double click. Doesn't work. >> All right. >> This one. Okay. So on the left we have the Papyrus web view and in the right we have Postman. Um to see what the MCP server contains. Here we can see all of the MCP tools. So for example, get all projects. It will give the as the result of the projects in our Papyrus web session. And you can
call each tools with this button. In this part we do it manually without LLM uh for now. So when you run get all projects, you get a JSON with the name of the project, the ID of this project. And it's act directly on the Papyrus side on the left. So we create a class. We will ask the MCP to rename this class. So, you can use the
MCP server manually. Um the final step was to connect these tools to an agentic workflow. Uh AI agents are LLM's model with access to tools. It allows them to reason and to act on the workspace. An agentic workflow is a sequence of specialized agents. Here in our workspace uh on our workflow, we've got two specialized agents, the planner and the executor. The planner will think about the
sections sections actions to call from from the MCP tool. It analyzes analyzes the user queries and such as create a new class or create a all uh model to understand what it needs to do and it generates a plan from this user query. The second agent is the executor. The agent has access to the plan from the planner and will execute the tools on Papel on Papelot.
And it answers the response in it answered the user query with natural language or it can ask to more precision about the user query. For example, it will not create a class if it doesn't know what the class should represent. What technical What What technical stack we are using? The programming language is Python. We have created web app with a Streamlit Python library which easily transform Python
code to interfaces. For privacy issues issues, we are constrained to use LM from the CEA which is Codestral from Mistral. But the structure is made to be able to use any kind of LLM such as Claude or Gemini. The agents are created with LangChain. It's an open source library to structure the usage of AI agents. It helps with the use of prompt tools and memory for each
agent. Our agent communicate through the A2A protocol from Google. The The agents are coded in two different files. And you can choose your preferred language to code the this agents. And even if they are not the same in the same language, they will be able to communicate together. And we give the agent access to the MCP tools as I mentioned before. The architecture is standard and modular.
So, this first architecture is very basic, but in the future we can add easily new agents, new specialized Um I talk about human and agent collaboration, but the purpose of PipeRider web is the human human collaboration. So, we integrate Numerics, the agentic workflow, as a bot in Matrix element. Uh I'm going to talk about the Matrix as a collaborative chat for each project. So, you can call
the Numerics bot in the chat. It will modify the the model as requested. And every members of these chats have to the user query and to the agent re- response. Uh about next steps, uh because the new releases of Sirius uh the GraphQL schema called by PipeRider called by PipeRider uh change frequently. So, now we need to adapt it manually at each new version. And uh for
the next part of the project, we plan to automate the handling of Sirius GraphQL to make this uh automatic. Uh thanks to JN generative AI. Um another project is to use specialized agent, as I mentioned before, for other we can imagine adding uh robotic specialized agents in the PipeRider for robotic. in PipeRider for robotic. >> Mhm. >> And we can imagine also adding agent for mhm creating
a documentation automatically with the model. to extend human-machine interaction, we plan to add more possibilities uh to Numera B such as integrating a voice-to-text. It could be useful in a meeting to just ask Numera B to add um components in the model. But it's also more inclusive. it's a more inclusive way to use Papyrus uh for people who don't have possibility to use a keyboard, for example.
>> Yeah, thank you, Julie. Uh to address one of the problems that we had before, well, there is this work that we have already shown with our agent-based approach. But at the beginning I identified two uh uh two issues. That was uh these modeling tools uh that we developed in the past uh are not very easy to use. So, it's uh divi- it requires a certain time
to get familiar with these models. uh then there's also some need for automation. So, if if I want to create the class with some members, there's a lot of clicks I have to do. And if I can just describe either orally uh with voice-to-text or um with the keyboard saying, "Okay, I want a class having these attributes um or any certain domain add me these standard attributes."
Then I can really profit from the automation. And uh these two goals are to simplify the tool usage and to guide and assist the user is a goal that we share with Obeo who's also developing modeling tools. So, the idea was to combine the strengths of both partners which are model driven engineering, the associated tools, the frameworks, notably Sirius Web. And the methodologies like Cadia. And then
add the AI assistance to it to simplify the tool usage and to provide for automation. And we're doing that in several small projects focusing on certain aspects. Like my colleague will now show you. >> Um, this project under joint laboratory. The first one is the intelligent onboarding assistant for MVC tools. the plan is that the AI agent provide interactive guidance, explanations, and suggestions when you load project.
And it should be tailored to the user models. The assistant is personalized based on the user profile considering their experience, their job, and the context of the model. The output will not be the same for a novice student user than for a senior engineering engineering engineer. And it adapts to the user progress. Here is an example of how this onboarding agent works. In this demo, Alice creates
a model. it's blurry, but it's car modelization with the car with engine and seats, for example. She might leave the this work for a while due to holidays or other projects. And when she returns, the unbonding agents uh provides her a summary of the project. The output may look like this. Uh this project is about modeling a car. the central element is the car, and the last
it presents the last update of this project. The the last update is the addition of the braking system in the with the security requirements. As I mentioned, the output is tailored to the user profile. So, if Bob, a new member, a new novice project member, uh join the project, he will receive a totally different outputs. is much more detailed because Bob never saw the model before. So,
it describes uh each class, an engine, which is responsible for the boring, a passenger compartment. it's the same subject, but it's much more detailed for Bob. Um here is a toy example. It's very simple uh model, but we can imagine how useful it could be uh when you join a very complex and dense project to have a a summary of this project. Uh then, if Bob modifies
the project, here we can see that Bob is adding an indicator. Uh when Alice comes back to the project, she will receive a summary of the modification made by Bob. And here it's blurry, too, but uh the AI output is since your last last connection yesterday, the user Bob has added a wiper and an indicator. this uh onboarding assistant is adapt on user profile, so uh Alice
as an expert already know the project. She only needs a quick reminder. And Bob as a new member of the project requires a much more detailed summary. This onboarding agent simplifies and accelerates the integration process for new project members. And it helps uh users quickly get back on track after a a break on the project and enhances collaboration by summarizing the changes made by co-workers. >> Yes,
thank you, Julie. I'd like to come back uh uh conclude uh our talk with uh some description of ongoing work. So, we like to customize our approach for different domains. There is for instance a European project as here in which we uh apply these processes to the robotic And we are also uh having uh projects in the context of software-defined vehicles uh and manufacturing. Uh I will
come back to these uh later. uh we will uh then evaluate the trustworthiness of the uh AI systems assistants with which is very important because uh um well, we are dealing with domains in which safety plays an important role. And if you blindly trust the AI agent, uh then uh we could have very problematic uh results. So, one one advantage is that we are already at a
relatively high level of abstraction here in the modeling. So, we can evaluate the intermediate results, but we can also imagine that we integrate some hallucination detection mechanisms on which my colleagues work. And that we can put well, stronger emphasis on a human in the loop. Other colleagues of mine are also working on the specification of the operational design domain, which is important in in the automotive domain,
so that we know under which condition a system is supposed to work. And if these conditions are not met, that we can automatically remind the user to to specify some countermeasures for for these edge cases. in in all the cases where we are using LLMs, we need good data sets. And for code, this is not a big problem, but for models, there are not many data sets
with proven results. And something which is interesting that that we are in a joint project called Code Commons with the software heritage archive. we have access to to huge amount of of code and models, and we start to to analyze them to get also better modeling with respect to the integration of safety and security, we are collaborating in the SDV projects Halfer SDV, Shift to SDV. For
instance, is Technical University of Munich, is creating test case scenarios based on AI assistance and that is that also the partner AVL is doing. We have Andreas Ecker here and see audience the head of the half for STV project. So AI is playing a certain role in these projects and also modeling tools to have these intermediate results that you can evaluate. And well there are other supporting
projects we already mentioned the joint lab and see half for STV projects and they are shift STV projects which are chip C GU projects and uh are maybe in the future the AI for STV are coordinated with an additional project called federate. There are there is the ST projects in the robotics There is error head tools another project focusing on tooling specifically in the AI context. there's
code comments where you get to the software heritage archive. And France is also supporting these aspects. LLMs for EU initiative. I think I will quickly skip some of the publications we have done. Well there's There is some work on design patterns, some work on agentic AI. I think you will will get you will get access to the slides later and I see these aspects. So thank you
very much for attending today and you can ask questions now and you can also visit us later at the booth number six here depicted on on slide Thank you for your attention. >> [applause] >> I think for the presentation. I noticed that you guys said that you had already a REST API, but you took the time to translate everything to MCP >> Right. >> What was the
motive behind that and probably some improvement, right? >> Yeah, the motive well, I can also my colleague can also answer. The motive is that if you have an AI agent it's easier for for the to to to to integrate it with an MCP protocol than directly the REST API. >> Yes. >> Okay, was there any performance gain or cheaper to use because of the tokens usage that
we use or something? >> Um yeah, but it's was just just not not directly possible I think to call a REST API from an AI agent or do you have some complementary information on that, Julie? >> Um it's it's just easier to use the MCP tools and the the agents can understand more easily MCP tool than how to make a request to >> There's a standardized way
to describe the API and what it what it supports. If you use MCP, so it was more more native way to do it. >> Okay, you're right. I mean MCP describes that you have to say what the method does and what the parameters mean, so >> Thank you. >> Other questions? Okay, then. Thank you and visit us at the booth. >> [music]
