About this talk
In this talk, Sander Bouwmeester describes the Backstage Builder experience at Adyen, focusing on how Backstage serves as a framework for internal developer portals. He explains how Backstage facilitates organization, enabling engineers to manage complexities by creating a cohesive internal platform. Key features such as the catalog, scaffolder, and modular plugins are highlighted as essential tools for enhancing development speed and quality. The speaker also discusses the transition from a centralized to a decentralized approach, emphasizing the importance of leadership buy-in, team collaboration, and user discovery for both end users and builders. Additionally, he mentions the challenges faced during this transformation, including technical expertise and deployment autonomy, while providing insights into the improvements made over two years.
Full transcript
All right. Yeah, everybody can hear me. Welcome everybody. My name is Sander Bouwmeester. I've been an engineer at Adyen for the past 2 and 1/2 years. Welcome to this talk about the Backstage Builder experience. I really appreciate you being here because I realize that at the same time there are three talks about AI and I won't talk about AI as much. So, let's go. So, for those
of you that are not as familiar with Adyen, we are a global financial technology platform, which means that we offer financial services to leading businesses like McDonald's, Uber, and Spotify. Yeah, if you're interested in what we're doing, we also have a booth here at the conference today and tomorrow. You can talk to engineers like me to get to know more about our tech stack or about what
we do. And to give an intro to our talk today, let's actually first talk about what Backstage is. So, as I described here, it's a framework that actually acts as a single pane of glass for your engineering organization. Um As your organization skills and the amount of developers, you yeah, want to untangle your complexity by building a platform. And while you build this platform, you can build
a internal developer portal on top of it to act as a unified gateway into this platform. And this is what Backstage offers you, like this internal developer portal that sits on top of your internal developer Now, as I said, it's a framework. And what is important about that is when you start with Backstage, you for instance run this beautiful command at the top to create an application
actually. So, what you get when you run this command out of the box is a front-end application, a back-end application, and yeah, a database technology of your choice like Postgres. So, you get a React app, a Node.js back-end, and for instance a PostgreSQL uh, integration. Now, with the Backstage framework, there is a few key features that ship for you out of the box and I'll highlight a
few. So, one of them is the catalog. The catalog acts as a phonebook for all your software. Engineers define YAML files in their repositories to identify, "Hey, this team owns this application. Here you can find the documentation. This is the description of the thing." And this information is all gathered into the catalog of Backstage to service exactly this information to users so that they can quickly find,
"Hey, who owns this service?" or "Where can I find the the documentation?" A second very powerful feature is the scaffolder. So, whenever you're building out this internal developer platform, you will develop standards. And standards, for instance, to create a database or to request an S3 bucket. And to make it as easy as possible for platform engineers to expose that within your Backstage application, you can leverage the
scaffolder in which you can declare a YAML file in which you define the form, the workflow that needs to be executed whenever you, for instance, request a database. Thirdly, and it's the most important feature of today, it's the plugins. So, Backstage is set up quite modular as a From it's it has a bunch of plugins out of the box like the catalog and the scaffolder as I
uh, just showed you. Um, but besides that, it also has community plugins. So, people within the community have open-sourced their plugins for you to use in your Backstage instance. But besides that, and that's the most important one of today, it's the in-house plugins. So, if you have a specific need for your organization to service something within Backstage, you can build your own plugins. And these plugins can
basically house everything that you want. Like you can show anything that you want or process anything within the Backstage back end that you would that you would want. So, my main point of the talk today is that your internal developer portal has a higher chance to succeed in your organization if you allow your builders of the portal to deliver with speed and quality. And that's not an
idea I came up with on my own. There is this talk by Stanley of Spotify who open sourced Backstage from Backstage Con 2025, so last year, in which he talks about how you can go from a very centralized approach with Backstage where one team is managing the whole thing and building the whole application to a more decentralized approach where you have the still the core team kind
of helping the other teams in the organization to come to your Backstage instance and deliver value, building plugins, etc. And on this journey from a very centralized approach to a more decentralized approach, one of the things that Stanley also highlight in this talk is to ensure that you establish golden standard for building within Backstage. Because you have to imagine that Backstage itself is kind of a platform.
Like other people come in there to, yeah, build out this portal. And I want to re-emphasize that point by letting you imagine that Backstage is at the center of your platform engineering organization. So, you can imagine that above it are these end users that come to the portal to achieve their day-to-day tasks like request a database or see who owns a certain service. But underneath are actually
these platform engineers, the builders, that come to your platform and want to expose, yeah, their tooling to the organization. And so, to highlight the challenges that we encountered on our two-year journey of going from a very empty portal to a inner sourced platform, let me showcase you uh, yeah, Launchpad, which is our internal backstage instance at ABN. So, as you can see 2 years ago, there's not
that much content in the sidebar. There's a beautiful note at the top that says it's a work in progress. And this was basically started by a CICD engineer within our organization to explore like hey, what what could backstage offer for us? If we take that two two years later to today, you'll see that in the sidebar we have a lot more content. You'll see the catalog and
request resource which is yeah, the catalog and scaffolder that I talked about before. But in the development section, you also see a lot of content such as code health, firewall manager and release and deploy. What's important to understand is that all those items under development are custom built. So, they are not community plugins or core features of backstage. This was all developed by people from within our
organization and contributed to the And one more thing I want you to pay attention to is the documentation section. The internal knowledge part you can ignore, but under developer kits, that's something that I will come back later on in the talk. because let's go into the challenges that we faced in this journey of two years. And the first challenge is more about getting the basics right. So,
when we started two and a half years ago with a CICD engineer that kind of POC'd yeah, backstage framework, we realized that we actually first needed leadership buy-in to showcase the value of what an internal developer portal could yeah, do for ABN as a company. and once we got that right, we realized we needed to build out a team where you actually need the expertise of both
product tech and UX like the triad to really leverage all the different expertises to now once you have a team established, it's also important to realize that as I mentioned before, you have these end users. They come to the platform and you need to do user discovery and interviews with them to figure out like what are the frictions that we need to solve for them. But I
want to emphasize that it is as important to do so for the builders. They will have frictions in their journey for building out for Backstage and you need to go talk to them, have interviews and understand the friction that they have in building for this So once we had a team set up and had were very in the centralized approach and on this journey towards the decentralized
approach, um we encountered that we had a lot of uh knowledge about how to build with Backstage and and use this uh framework, but people in ABN outside of our team did not. And there's a feature of Backstage that we used called Tech Docs that we used to uh solve for this. So Tech Docs is a feature where yeah, as it says, docs like code, you have
your documentation next to your code and you can then expose it within Backstage. Let me also quickly showcase that to you. I promised you that I would come back to the developer kit. And one of the things that we do is leverage this Tech Docs feature to surface in this platform or in our portal yeah, a set of guides on how to build for Backstage within ABN.
How do you run it locally? How do you create a UI plugin? How do you connect uh to a backend API if you have one uh uh built out? so again, we're leveraging Tech Docs to to make that possible. And another cool feature that was introduced if the slides want to load. Yes. Um it's the the Backstage CLI. So Backstage uh uh ships with a CLI which
has a bunch of useful commands, for instance, to run testing, linting, et cetera. But it also comes with a command to scaffold a new plugin. in a previous version of Backstage, it was not possible to um adjust that output. So you were bound to whatever Backstage had put as the content of that However, since Backstage 1.36, you can actually provide templates to the CLI, um which allows
you to influence whatever is output when you run, "Hey, give me a new uh plug-in." And we use that heavily to kind of educate engineers whenever they are uh creating a new plug-in to showcase to them, "Hey, this is how you should do data fetching. This is how you render a table. Uh you immediately follow all our standards from day one uh instead of encountering them along
the way." The third challenge that we encountered is after we kind of, yeah, solved this uh tribal knowledge, is front-end expertise. So, you have to imagine these platform engineers, infrastructure developers, yeah, they come to build like custom plug-ins, but yeah, they've never worked with React before. Logically so. Um and what we encountered that Backstage does ship with a lot of default things around testing, linting, uh uh
how to build, et cetera, but that we still had to go over a set of yeah, let's say generic front-end concerns like how do you interact with URL state parameters? How do you do data fetching uh so that you enable uh things like polling, caching, retrying, um so we went over all those shared concerns and uh uh introduced libraries such as TanStack to do that uh uh
uh standardization of the data fetching. Um but another part where people struggled is what should I use to build things? And of course, we have that uh scaffolder template or the the uh CLI template, excuse me. Um but another way to showcase to developers what they should use to build is Storybook. And let me also quickly So, within Storybook, we showcase to developers, besides the developer kit
of uh the develop kit shows them guides of, "Hey, this is how you quickly set something up." But Storybook really shows them, "Hey, these are the different components that you can use to build up a page." Here I'm showing you an example of how to do tables. Tables are a very complex UI component. and you need to think about things like pagination, different column sizing, you name
it. And we show a very elaborate guide of how to interact with a table in Launchpad or Backstage. Um also show them the code that they can copy-paste into their code to yeah, start leveraging tables in this case. Um yeah. Then we are at the last challenge of the day, which is autonomy. So once you have made everything as easy as possible for them, you want to
be able to let them deploy and develop autonomously without our centralized team being there in the middle as a dependency. to start with deployment, what you have to realize is when you generate this Backstage application is that you can deploy it however you want. We currently still do it in a very monolithic way, so it's one front-end application and one back-end application. However, if you want to,
you can also go for micro front-ends and microservices, but that yeah, requires you to put in the work to um enable that. So we are still in a very monolithic approach, which means that yeah, before actually before we did things with Argo CD and Kubernetes, you were dependent on us to deploy the application and we did actually two things. One is we leveraged Argo CD Git generators
to create ephemeral environments. And by that I mean that if you created a merge request on our open source Backstage, um it would allow you to add a preview label, which in turn spawns up this Kubernetes environment with your whole own yeah, Backstage instance so that you can validate your changes before going to main or going to production with your own complete dedicated environment with all the
data. Another part is that we started allowing people actually yeah, deploy without us. So they could create a Git tag, it would do the whole build, they could deploy to pre-production environments themselves like testing and staging. And for production, yes, they would still require an approval of us because we still have this very monolithic build. Once we move to things like micro front ends and micro services,
you can imagine that we would also allow them to just go ahead and deploy their own thing. So the last part is about development. We've talked about the deployment part, but they were also still relied on us when it came to creating these merge requests. Every single merge request needed to be reviewed by us, get an approval, and then they could merge it and roll it out
themselves. But the next level to that is that they can also just open an MR on their own plugins, merge it, and get it shipped end-to-end. We are not fully there yet, but we are leveraging tools like SonarQube and ESLint to for instance give them a set of linting rules that will validate that they for instance use the components that we offer them, that they don't go
and use tables that yeah, we said hey, you need to use these standards. Don't go use your own table. Of course, we are not in the perfect situation either. We have next challenges, and let me name four of them to to close off the talk. So the first one is compatibility. As we rolled out these new front end standards around the URL URL state parameters or data
fetching, we saw that over this two-year journey, the newer plugins, they're all in the latest standards, but the older plugins tend to become yeah, kind of legacy. They don't move with this new standard that we rolled out. Are you going to migrate all of them like yourself? Maybe not. Yeah, we we kind of struggle still with this. You can maybe use tools like open rewrite or code
mods to to solve this, but we we haven't really found the the sweet spot there yet. Another one is pipeline speed. So, as I mentioned, we have this monolithic build, and as the application grows with all these people contributing to it, you'll see that the amount of test grows, the amount of things that need to be linted grows, and so the build gets longer and longer and
longer. which in turn then can get solved by micro frontends where you can build only the changes that are done by a developer in a merge request. Thirdly, what we also see is I I talked before about the importance of this UX, product, and tech triad. And we did try to get that right from the start. However, even though we had UX, we still saw that people
had isolated user journeys. Like all these people like create these own plugins within isolation. So, they don't think holistically about how does this all fit together in an actual user journey. this is something we also still struggle with today. Um but I just want to emphasize, if you're going to start with Backstage, involve UX as early as possible, and also think about is there for instance an
internal design system that you want to leverage, or do you want to go for what Backstage has out of the box? Lastly, we have AI, of course. Yes, there it is. I am talking about it. as we see more and more code being generated by AI, of course, we also need guide these AIs by providing skills, rules, workflows in order to work with this Backstage repository. That's
it for today. So, as I said, your chance to succeed if your builders can deliver with speed and quality. We use all these tools and processes to enable our engineers, and are still on a a journey, of course, to to get there. I thank you so much for your attention, and if you have any questions, I am
