About this talk
In this talk, Adam Najj shares his extensive experience with the European Commission, where he has spent over nine years as a senior Drupal and AI integration architect. He discusses the shift of the European Commission's web presence towards open-source solutions, particularly the usage of Drupal across its numerous websites. Najj highlights the evolution of the Commission's open-source strategy from basic internal use to a more comprehensive approach that mandates the sharing of code and prioritizes open-source solutions. He elaborates on the challenges and achievements in migrating various websites to a flexible, decoupled architecture with reusable components. The talk also touches on the integration of AI tools to enhance content creation and management processes within the Commission. Ultimately, he emphasizes the importance of collaboration, transparency, and community engagement in developing digital solutions that serve the public sector effectively.
Full transcript
Okay. So, welcome to today's presentation. Uh, using Drupal doesn't make you open source. Hello there. Um, so I will um talk about a bit of um um story from uh Europe's largest public sector platform from the European Commission. I'm a technical person. So, um I will talk a bit uh about history uh legislation and uh procurement but uh my um experience is uh based on what I
did there. So, uh we'll touch a lot of uh technical points. So, let me introduce myself. My name is Adam Najj. Um I was born in Hungary and I'm a senior Drupal and AI integration architect. Um I'm consulting for the European Commission for over 9 years and um I've been contributing to Drupal for more than a decade and um I've been a drummer for 20 years. So
um I love playing music. uh I was playing uh in many bands, shows and festivals and um I have never been in the US before for so thank you very much for having me here. So um just before I go on, how many of you know what's what the European Commission is? uh for those who don't um think of it as an executive branch of the European
Union, in the European Union, um there are 27 member states and u around 32,000 staff with uh 44 departments. That means um yeah that's um that's that's a lot of websites and we we call these departments directorates general uh I will refer to them as DGs uh from now on and basically the commission um proposes legislation manages EU policies and runs the day-to-day business of the European
Union. Um so the total websites are around 700 uh these are including static sites um Java and even the Drupal ones. Um yeah I put there the URL to that points to the website of the European Commission or you can use the QR code if you want to check it out. So uh that's um 330 websites on a single platform. We have uh 24 official languages uh
but the platform supports uh over 27 languages and all are Drupal. Uh basically the European Commission's web presence is one of the largest Drupal deployments uh in the world but um yeah the most interesting part of this story has nothing to do with the numbers. So I will go a bit into um a history start um uh from the beginning and uh yeah this uh this is
a um um it's a imaginary uh character working at the the web platform at the commission. Uh so in 2000 the commission first um open-source strategy uh was adopted and um it was about mostly internal use like um choosing when open source made sense. Uh so over the next decade um Linux became the recommended server operating system. Apache started powering up the Europa.eu domain. So open source
was gra gaining ground but on very low level and mostly infrastructure. So um a decade later the first Drupal 6 websites appeared. These were u mostly individual um teams for u um separate uh agencies and and DGs. So by um 2011 the strategy went a bit further. So open source became the preferred choice for new information systems developed from that point on. So two years later, Drupala
was the standard for the commission's communication websites. Um so the first distribution uh was next Europa. Uh it was built on uh Drupal 7 and it was a multisight platform. So um the decision was made to move from documentum to Drupal 7 as part of a major digital transformation campaign at that time and um sites were migrated onto the next Europa platform which grew to 160 sites.
So the strategy by then was talking about um uh community participation contribution but in practice the way we build things um hadn't really changed. So um as you may know um multicight has its limits. So, and anyone who has worked with one at scale knows them. And yeah, so I we we I have I use a to little bit um show the drama to elevate [laughter] the
the the urgency here. Um so what happened is uh we started to have the first cracks uh on this. So you can't upgrade one site without uh upgrading all of them and uh you can't roll out a patch to only five sites. Um you you or hold on uh apart uh some some other site. So deployments were all or nothing at this point. Security vulnerability in one
module um affects every site at once. um a bad update can take down the whole fleet. Um and high traffic on one side can slow down all the others. Um so and as soon as the the site starts diverging in uh what they need different integrations, different content models, the shared code base becomes a liability, not an advantage. Um so um testing new releases meant testing against
every site on the platform. Deploying security patches was slow and risky and the cost efficiency what was supposed to come from sharing uh one code base wasn't there anymore. So a single Drupa 7 site uh was pretty expensive to to start with. you needed some site building, some development and and also the hosting was uh costly and not every site could fit into this setup. So I
I was working um um at the time on the main uh website of the European Commission. So like um some DGs like the Digiccom for communication ran their own standalone Drupal sites alongside the platform and um that main site of the commission was considered the most important one. So um and when even with the the multi-sight uh each site owner um build their thing their own way.
So we when we were working on that site, we had already pretty complex business processes implemented uh content types for news article, events, publication, um call for tenders like uh uh you name it and and other sites were having the same. So you could represent like you you saw news represented in different ways not not only visually but like um when when even the content was repeated.
So then different teams they were working on the same thing fixing same bugs um uh over and over. So um there was no sharing. And so that was uh uh a major um drawback from working in silos. So that was really the deeper problem here and um none of it was shared basically. Then um so we adopted open source but what we built on top of it
was kind of proprietary. The strategy said um yeah use open source. Uh so we were using it uh but we were using it um the same way as we had used everything before. So um they believe that it helps uh cutting cost because there is no licensing fee and uh we tick the box that uh we are open source at the same time. So it sounded good.
Then um the shift uh happened during the Drupal 8 release and it was um a big undertake. So we pledged staying with Drupal and we decided to migrate the 160 sites on Drupal 8. well uh and well instead of the multicide model we we um we worked on a a loosely coupled uh reusable component approach like a decoupled approach. Um we created a new SAS products to
serve the corporate web presence called um Europa web publishing platform but we didn't migrate all the 160 sites overnight. Um so we started with a limited number and built the platform as you went. So we we isolated like around um 60 sites that had uh really the the the corporate need and and basic functionalities where we could build from. So really the first step was uh understanding
what we actually needed and that meant hundreds of meetings with SH stakeholders across the commission and for every feature we had to ask is this a corporate need um that every site should have is is it an extra some sites can opt into or um it's just too custom to belong in a shared platform at all. So those distinctions had to be made clearly and um they
weren't always easy conversations. So we architects we wanted to do all this the open source way and so we pushed from bottom up towards higher management um the architectural governance. So we created the open Europa initiative to strengthen the adoption of the methodology with uh established reusable technical governance guidelines uh kind of code of conduct. The code of conduct that we demanded from our community members to
learn, adopt and respect. And then open Europa library was born collecting fully pledged reusable solutions for the European institutions. We built the vet platform and the open Europa components feature by feature. Each time um corporate need was identified, it it became a uh shared uh like authentication uh multilingual workflow, translation u integration, theming, content types, listing pages, editorial overflows, you name it. So the library grew organically
and um driven by real requirements and not by um deciding up front what needs to exist there. So over the time more sites migrated onto the platform as it matured. So after the 60 sites, we started to to to contact other DGs and and prepare the the onboarding and the migration process and identify the extra features that needs to uh be implemented and and yeah to to
build um the component base and also the the Europa web publishing platform as a SAS product itself. So um Europe open Europa is not a distribution you install and get everything. It's a collection of building blocks. Each component has its own project, its own release cycle. Um each one can be adopted independently. So if an agency needs an authentication or multilingual support but nothing else, they for
example they wouldn't fit the the the platform then they could those they could take those two uh components and um leave the rest to build on top everything on is on GitHub and um everything is open source. So now it looks pretty simple but uh you will see more on that later how it looks today. Um so in 2022 the commission published its uh think open strategy
with six principles transform share contribute secure and stay in control. It was um part of a broader push towards digital sovereignity reducing dependency on um technology that the institution can't inspect adapt or share. So by then open Europa was already living those principles and um strategy just formalized and and what the practice had already started. Um and then in 2024 um something bigger happened. Uh the European
Union passed the interoperable Europe act. This is a regulation um not a recommendation, not a strategy, not a um um paper, proper EU law. And uh it says that uh if a public sector body builds a digital solution, it has to share it including the source code with any other public body that asks. And also um it says that when choosing solutions, open source must be prioritized
where it's equivalent in functionality, cost and security. And there's there's now even a portal for this um on Drupal naturally and it hosts the open source um observatory an EUwide catalog of uh reusable um and it's the single entry point for everything related to crossber interoperability. So for us um this was a validation what we've been doing with open Europa um sharing components publishing source code and
contributing to the ecosystem that's now the legal baseline for every public sector administration in Europe. We didn't build open Europa um because of uh regulation but the regulation exists because the approach works. So, um yeah, but a little bit about um legislation. Um legislation doesn't write code and it doesn't write contracts. Uh uh it's it's been um I've been in public sector um for long enough to
to know if the contract says deliver feature X and nothing about how you often get a fork or a proprietary solution. That's why uh open source being prioritized matters because it if if it says deliver feature X in a way that can be maintained contributed upstream you get something the whole ecosystem can use. But uh the reality is that um still proprietary wins often by default because
uh procurement processes reward uh vendor name recognition. Uh framework contracts favor large consult consultancies and uh open source doesn't have a sales team to showing up to bid. Um also public sector buyers often don't know how to evaluate open source. Uh they compare license costs where open source wins of course but miss the total picture maintenance sustainability who actually keeps the code alive. So vendors can package
and resell open source without contributing back. Um undercutting those who invest in the ecosystem. So there is no mechanism to distinguish a contributor from a taker in a tender. Uh so if I could wish more uh open source should be the default not just a priority. That's um public money and therefore public code principle. If tax payers fund it, the code should be open. Uh upstream contributions
should be actual deliverables, not nice to have in a contract. Uh and when you evaluate a supplier, look at their track record. Uh are they actually contributing to the projects they sell? um that tells you more than any proposal document. Uh right now um there is nothing in a tender that um distinguishes a company maintaining modules upstream from one that forks and walks away. Um and and
that's um that's a challenge. So the act says share the strategy says and um basically this gives the space and the mandate to do it for us in the open because previously when we were working on Drupal 7 or we were contributing fixes we couldn't attribute it to the our client the commission it had to be secret we we couldn't say okay this we we are doing
it for the commission and and really the um the open source strategy as it evolved and uh the act um it gave us the the mandate to do this and now it's possible. So that means um well yeah we today we we build um things in the open. Um and but the longer answer is to how we build today um that we changed how we make architecture
decisions. So uh we don't just publish our own modules. We take ownership of what we depend on and this is crucial. If we use a contributed module in production, uh it's part of our um scope uh it's part of our ecosystem. So it will define the quality of our product. So uh that's why we we report issues. We we write patches. We add tests. We offer co-maintainerships.
We work in the open. Uh we use the issue queue to do new implementations even contributed modules that we that we create. We we mirror internal tickets as uh issues in the issue queue uh for transparency. So yeah, not we we don't do it because we set out to but that's what happens when you treat the ecosystem as part of your own infrastructure and for us uh
it works. Um so basically when a new feature request comes in the first question is always does something already exist? If a contributed modules cover most of what we need, we extend it. We contribute back to it. We help maintain it. Um, we don't fork it. We we don't rebuild it from scratch. If it sometimes the functionality is there and it misses tests, then we write tests.
Then then we can be sure that okay this is up to our standards. There are um there are no bad update uh that could uh break our sites. So if nothing exists and we need to build something new, we design it uh project agnostic whenever possible that means uh configuration over hard coding, plug-in systems, you know the drill. Um however not every part of uh the feature
can be designed this way at all times and those become like our uh really specific uh features. But we try to uh clearly separate and what can be useful for the community uh contributed. So the mindset is there and when the architecture is right contribution is what comes the other end. Um so I don't know how visible is this. uh I tried to take a screenshot of
uh at at least partially from current state of uh open Europa components. So what you see here is a collection of uh modules and libraries. The blue ones are developed for the European institutions um like uh content type related modules uh multilingual editorial features uh AI features user interface uh yeah like service like um specific service integrations. uh the green ones are uh contrib modules and the
yellow ones are contri modules that either we contributed or we help maintain. so in numbers uh today the European Commission has 100 people active on Drupal.org. We support 120 projects. Um and we we maintain these uh contributed modules uh in many aspects. Um sometimes um just uh help the the like as a co- maintainer help um reviewing issues uh merge request etc. Um and we have um
130 Drupal developers across the commission and the EU agencies across the the whole European institution ecosystem. Of course, none of this works without testing. We aim for full automatic test coverage on every component. As I said in the beginning of the presentation, I'm a technical person. So I will give um my the my technical um aspects uh to this. So yeah work we work with current test
functional test existing existing site test that can run against full fully installed platform. Uh every pull request we run through CI and nightly builds to catch regressions before they reach productions. Many time many times we we catch uh country modules being released overnight and we have a nightly build and then it fails and then we report issue and then we we have to uh either lock the
version temporarily uh while the the fix gets in or just use the patch that we we provide. So it's been valuable to us and yeah in in in general um um every ticket goes through double technical review and um no bug fix uh can go in without test otherwise it's just a workaround for us. So um about about little bit about the community aspect here. So um
you can have the best architecture and the best tools but if people don't talk to each other um you are back to silos and we've been there and that's why we um we created the Drupal community of practice. It brings together Drupal developers, architects, um decision makers and teams across the European Commission and EU agencies. People who were previously building the same things without knowing about each
other uh now have a shared space to communicate and operate. In practice that means we we have dedicated time for Drupal contributions. We have expert talks where we uh share what we built and learned hackathons and channel for sharing best practices across projects. So we also hold regular short meetings where community members can ask technical questions to the core team so they get updates on the road
map and the reasoning behind decisions. So we try to involve everybody. Um so it's not the governance body, it's where the culture of uh working in the open actually lives. Um but what about AI? So yeah, I I promised um I'm not going to talk much about AI and this is going to be like a a break from all the buzz that's going on in this u
fantastic conference. so much expertise but yeah so what about AI um to to give you some highlights um from the Drupal community of practice we are contributing to the Drupal AI together with freely give and drop solid we or organize free uh webinar series bringing Drupal AI into your DNA uh that's open to the whole community in 2020 24 we ran our first Drupal AI um with
Amazon Web Services. I led the team uh Ainstein and u this is a that's uh that's our um imaginary character uh who works at the commission because um uh the content creators um are having daily challenges when for example you have the European med medicine agency having uh research papers and they have to publish them and they need to summarize it up what's in those research papers
and it's uh very scientific. So uh that's why we created the AI summarized document module that uh helps you to summarize PDF documents in any tone in any length you wish and with that uh my team won the first prize during the hackathon and it was the beginning of the the Drupal AI module. So we we we had a freely give and team of Jamie u we
were um collecting uh bugs and issues uh together to to contribute to Drupal AI and um this January together with Frederick from drop solid we organized Drupal for gov EU a full day conference at the commission during uh opensource week the European open source over 100 people uh policy makers uh architects, digital leaders, uh national governments and EU agencies. Uh it's been a vibrant uh conference day.
Also this January we did the second round of the Drupal AI uh hackathon uh this time uh with MRAI uh 80 people, nine teams. I led the team called uh token burners and we resolved 40 issues. Uh we contributed two modules uh uh flow drop agents and flow drop no sessions. So basically what we what we worked on um it was challenge to uh improve the the
life of the content editors and really speed up the process of the validation. um and to get uh the best quality because now you are not writing not necessarily only writing content for uh uh visitors but you know AI um you write for AI as well. So um it was an we dreamed an agentic uh solution that helps the the editors to uh validate their content against
accessibility SEO fact checking that's very important uh at the commission and overall content health like semantics vocabulary etc. And this uh agentic solution you could use it on demand uh as as you go creating your content or uh with the chrome job from the background you could have all your content running through this uh and basically you have a uh a scoring system and um with the
dashboard view and you have an overall view of how's your uh content on website um how much of need fixing because the legislation changed and now uh it's not up to date. So with this solution we won the the first prize again. Um at least uh I had uh I was fortunate enough to to lead uh um both teams but um uh they were different members and
um yeah uh we also contributed uh modules um together um with the folks from uh the Princeton University um for the editorial accessibility checker module. It's a kind of integration to with the Drupal AI and basically it um allows you to ask um an AI model for accessibility fixes uh um that you might have. So it can fix uh text or markup missing uh um titles or
um um yeah h HTML uh table head u things like this. It's also contributed. So the Drupal community of practice is doing what it's meant to do giving people the space and time to contribute. So one of the questions I get often is how do you justify the cost of and um my always my honest answer is that we we don't because we don't have to we
uh we know the cost of not doing this um was much higher and um with the new model uh of course thanks to the SAS solution uh we reduced cost by 70%. This constitutes to uh multi millions of euros. Um spinning up a new website used to take months of site building development uh but now it just takes uh a pipeline and 10 minutes. uh teams used
to pay separately for the same feature investing uh for the same bug fix. Now um that doesn't happen anymore. You build a component once in open Europa and you basically give it to all all all the sites. The same for bug fix. We fix a bug in a contributed modules then the whole ecosystem benefits. So um maintaining the modules for the community is not a separate budget
line for us. It's um it's protecting our own production environment essentially. Um so the the testing infrastructure that makes the contribution safe is the same infrastructure that keeps our platform stable. It's the same work. it's not separate. Um, but I don't want to leave the impression that we have all figured out. We don't uh but we have seen that the impact has been huge for the community
uh in a positive way in the most positive way. So um for us it's really challenging to maintain all these modules ac across multiple Drupal core versions especially when uh you lose a body field between uh two minor versions. Uh yeah, so we we need to still support Drupal 10, Drupal 11 and uh all of this um and that's been quite challenging. Some of our institutions are
still running order versions. So uh we have to um support them but we are trying to follow the the same um release plan and release cycle as as um Drupal core does. So by the end of life we will drop support uh on those versions. So uh there's a con constant tension between what an ind individual stakeholder want and what the platform can sustain. Uh so every
DG have specific needs. Um finding the balance between those customizations and keeping the SAS product uh maintainable. It's something we navigate every day. And then there's the uh pace of innovation. Um especially lately uh in the Drupal AI and new initiatives and new tools move fast. Keeping up with that while working inside an institution uh that don't move uh with the same speed, it's often uh it's
its own kind of challenge. uh data and digital sovereignity is a central aspect of the current landscape. Uh but vendor locking makes it uh difficult uh to adapt and of course procurement is a constant topic that uh comes up. So often contracts and policies get formulated by parties who lack the complete picture of how open source landscape works. So as um as a closing thought um yeah
um uh 330 websites on one platform 24 and we started with a handful of Drupal six sites and strategy that said uh use open source. uh it took us over a decade uh and dropping a a monolith uh approach and um complete rethink of how we build software to get here. Uh we are not done but uh the mindset is contribution is what comes out the other
So um that was uh my little story about what we do and u how we uh approach um uh building sites for the European Uh thank uh thank you very much for being here and I would like to thank you the organizers for making this amazing conference possible. There are so many knowledgeable people here and experts. Uh, it's been a blast. So, it's time for questions, I
guess. >> Thank you. >> So, I've got a question. Um, you're not Well, you might be the first. Um, I mean, Australia has done this as well with Govc CNS. Um, they were more active in promoting this initially. Like, we've been talking about this publicly for a decade. We've got GovHub in Georgia that is doing this as well. So, um, That's that's interesting. I think that I've
learned today that South Carolina has a model. There's a private sector company that's doing something similar with private sector companies, but but it seems like this is a this should be the norm for government and it isn't. Um but but how much is there any collaboration happening between between the different um companies that are trying or sorry different organizations that are building um building these platforms for
multi-governmental like there there's a role for collaboration beyond just what you're doing should scale to every member of the European Union and every country in the world. >> Yeah. So basically the question is um it's about um if if all all these um different um countries and continents who already did something similar as us um do they in any way uh collaborate or or share? Well, what
I can say is um is yes and no because there is there is no um clear um methodology or even um um clear idea on on what's happening in in each country and and their own landscape and I think u sharing this is the first step and and um the second to for example having these conferences uh that we did in in Europe for uh Drupal for
gov it's been it's been great because there we had uh uh people from the UK and um and and I think these kind of forums are really um giving them the the necessary space and uh time to to do all these things. So what I see is also with this um hackathon. Um actually it was uh this this hackathon that we organized was was the first uh
that uh I was attending and there were business people involved meaning the we as the different teams had business analysts uh uh project managers and decision makers and first I thought how you do a hackathon like it's very technical but I had to realize that having these people on board and and fighting with us for for a common goal and really being part of this buzz for
multiple days. It's been it's been it's been great because they they went out and they they spread the word in in in their own departments and and really pushed uh um a lot towards um really embracing AI new technologies because even though we work uh in a very bureaucratic institution and um you know the gears are rolling quite slow it's it's very important to to still be
relevant to today's technologies. So I think these um these events and and um and conferences are so important and um and I would love to hear more about such uh initiatives even even from from from other continents. Um and um I I did this presentation uh with the hope that I can give some insights and uh relevant u information to um the US um public sector and
uh their participants. That would be my answer for the Um any other questions? comment because of course European Commission is a big part of our European Drupal con uh with Rosa Sabina and her team there and I would love eventually to see maybe like a smaller hackathon done even there on the ground at Drupal. >> Yeah. Yeah. So yeah um definitely definitely I I would also love
like uh not not only uh with AI but uh also for cyber security uh also buck hunts. So this is something that we are planning to do in the future to to have a kind of bug hunt uh and bug bounty uh hackathons uh for for Drupal or or any other open source uh technology. >> Go ahead. Yeah. So the question is how do we spread the
word about all the work? Uh >> and try to involve more people. Yeah. um about this the the the commission and the European institutions in general it's it's still like uh I would say it's really the beginning so after the uh the interoperable Europe act and that really gave the push for this um and we recently started to even have the possibility for the commission to sponsor
an event like uh Dupac on Europe. We had our stand there last year and it was surreal. People were surprised and and for us it was an amazing experience to be there and and for we are really doing these baby steps uh towards that and and have people approaching us and asked how the European Commission what it has to do with Drupal. Um, and it's funny because
in in in open source uh with and on on the legislatory levels um and um in people when they talk about open source they have no idea about Drupal. They don't and and basically it powers the the whole corporate web presence of the European Union and they don't know about Drupal. Imagine but on the other hand in Drupal they don't know that the the European Commission is
now really a major contributor. It's uh one of the top contributors uh at the moment. So um yeah that's why I think this community of practice and and we we create a a lending group where you can join yourself and and we try to to disseminate information uh success stories and uh even we invite external experts to to give talks and we try to do collaboration with
uh with European experts uh in in in the open source community. So this is what we are trying to to do for this. You're welcome. Any other question? Um we still have we still have uh two minutes so plenty of time. >> What's next? >> Yes. >> Yes. So uh I think um for us to to be really part of this uh uh it's it's something that
we want to keep up this momentum. It's been a very important mission for us because we see the impact for for the European citizens and the work that we do for them and that we do for the community. And um basically for us now uh major focus is uh integrating AI workflows uh into into um the commission platform to support business processes that can that could be
simplified by AI. uh but but mostly to give the tools in to the hands of the people who work on daily basis uh like the editors on on the platform. So this is now a big undertaking that we we are working on to we don't want to replace positions or anything. We just want to make sure that uh uh the people involved they have the superpowers thanks
to AI that it can deliver and that's a major focus besides uh cyber security and data and digital sovereignity. That's uh that's what's next for us and uh we are trying to to work in that landscape. So really uh be part of this momentum with uh the Drupal AI initiative and um yeah just give more and more visibility and space space to this kind of work out
there and um involve the the broader community not not only Drupal but we have talks with uh with the PHP community uh different experts from the field And um yeah now now we we have the mandate to do that. So we want to keep up and really grow in this way. All right. So if you like the presentation you can give a feedback. Even if you didn't
I would love to hear your feedback. So thank you very much. That's that's all for today for me. [laughter] Thank you.
