Webit 2026: Physical AI: Solving the next frontier in Robotics

This talk covers the advancements and challenges in the field of physical AI, particularly in robotics. The speaker, Nikolai Nicolov, highlights the need for machines to not only process language and data but also to comprehend their physical surroundings and interact with them. He argues that developing physical intelligence is far more complex than achieving reasoning capabilities seen in large language models. Nicolov discusses the evolution of robotics, contrasting traditional hand-engineered systems with emerging generalist models that can adapt to diverse environments and tasks. He emphasizes the importance of human data and simulation in training these intelligent systems, and suggests that hardware-agnostic solutions will prevail in the competitive landscape of robotics.

We now move from digital intelligence to physical intelligence. For years, AI has transformed how we search, write, analyze, predict, and automate information. But the next frontier is far more complex, bringing intelligence into the physical world. Robotics is where AI must understand not only language and data, but space, movement, uncertainty, objects, environments, and real world consequences. This is the frontier of physical AI where machines begin to perceive,

reason, and act in the world around us. The challenge is not only to build smarter robots. It is to build systems that can operate safely, adaptively, and reliably in the complexity of real life. Please welcome Nikolai Nicolov, researcher at Insight, with his keynote, Physical AI: Solving the Next Frontier in Robotics. Hello everyone. I'm Nikolai Nikolov. I'm currently a researcher at Insight which is a institute here in

Sophia affiliated with Sophia University and also ETHZIC. um the which is a Swiss uh university and today I'm going to talk about physical AI um and I'm going to focus on like what's the next frontier in robotics but to to understand this we need to go through a brief history so I will also go through a brief history and I've s I I've I have structured my

presentation as a series of somewhat contrarian views about how the future of robotics is going to develop. So brace with me. I'm going to start with the most mild one in the beginning which is that the LM that LLMs that we see today they only solved language but they didn't solve intelligence. And to illustrate my point better, uh if we look through history, at some point history

of life, like at some point a meimbus and like the most simple organisms formed, then it took us around 3.7 billion years to reach to apes to evolve there. Then another 20 million years to evolve to humans. another five million years to develop reasoning and language and then another 150k years to evolve to where we are today now. And if I compra compress this timeline in 24

hours, so like we are going to see that it took us only 60 seconds to develop and to develop these language and reasoning capabilities. However, it took us tw 12 20 hours to the hardware and the physical intelligence that we need to operate in this real world. So, I'm arguing that physical intelligence is much more difficult to develop compared to reasoning and language. So, let's talk about

a bit more about robotics and let me show you what are the difficulties there. And going to my next contrarian view, it's that generalist robotics models will wipe out the vertical solutions or the vertically integrated uh models. Let me show you what I mean by that. So handgineered robotics was how we were doing robotics since like before before AI before uh before AI become more more prevalent

and started working. And what's the problem with that? First of all, it's fragile. Like you need to program for a specific environment over months or years and your robot is going to work only in that environment and only for that task, then there's a z there's zero common sense. If the model sees something that it hasn't been that hasn't been hardcoded into it, if it sees an

edge case, it's not able to adapt. It doesn't have this ability of humans to adapt to new envir to to adapt to unseen situations. And it's and as I said is again like environment dependent. So like you change you put this robot in a different environment, it's no longer working. This is a video from the famous DARPA challenge. It's from 10 years ago. Uh these are the

best solutions that people managed to develop back then. These were has these are hand engineered robotics like it cannot even walk in a stable way. It's they're struggling a lot. Okay. Can we push this further now? And yes, the answer is yes. like um this is a video from a famous company Boston Dynamics. They managed to push this further. So like as you can see the robots

here are a lot more stable. They can now perform a lot more complex in a way skills. But again the problem is that this is handineered and this is like for this specific for this specific environment. This kind of solution take around 12 months at least to develop and requires multiple a team of multiple experts to build something like this. And I'm arguing that like the that

generalist robotics models are going to be able to do to solve these problems. And these are videos. They're extremely fresh. They're like one month old some of them. the demonstrating what we have been able to achieve with these generalist robotics models which can adapt to different environments and uh perform skills that they haven't seen before. So you can see like for example the robots are able to

walk without a problem. They're able to tie a zip tie. This is extremely difficult or or hold something between their fingers. This is the extremely difficult thing to teach robots to do. And to back up my reasoning that be the ones that are going to win the challenge that that are going to win in the end. I'm going to draw a lesson for autonomous from autonomous driving.

I was actually working before in autonomous driving at this company wave and back in 2018 when we started this is what we were able to achieve. So we were using end to end learning to learn to teach a car to drive. As you can see, like it's fairly simplistic, not very impressive. Like it's just following some road in a uh in the countryside. And it that's not

impressive. That's not the same as the real world as the city that we observe today. But fast forwarding until last year, Wave was able to show that it can drive in 500 different cities around the world. This was the first time any company was able to do that. So let's talk about a bit more about these generalist robotics models and what's special about them. Um the first

question is yeah how it works and the answer is you train on a lot of data. So you need millions of hours of data. This is any a big big number. And some of the success stories we've seen over the years uh alpha go openai uh first they did da 2 which is very uh much less popular than such gp today and the other example is again

like wave what they were able to achieve in autonomous driving and now the big question is okay how do you get this for robotics one way is tele operation So here on the videos you see basically we have a human who's teleoperating the robot basically performing a task through the robot embodiment and we are going to collect the movements of the robot and we are going to

then try to learn from it. The good thing is that this is easy. Like one famous professor says, tell operation is the sugar of robotics. But as you know, sugar is not good for you long term. So this turns out to be extremely expensive. It is unable to scale. You require one or two people per robot and you're very limited but by what you can achieve because

you have to develop these different environments where you collect this data and it's very slow like it's the robot brain that you develop is extremely slow because the human who's performing the task through the robot embodiment is not fast either. What's the other way? Humans. So naturally there are like many humans in the world. Maybe we can learn something from them. So they perform a lot of

tasks in daily. Maybe we can record their movements. So the good thing is that this is fairly easy to collect at large scale which is what matters which matters big time. But it's hard to learn directly from it because there's a difference. There's a gap between the human embodiment and the robot embodiment. And one step towards bridging this gap was developing these devices in the in white

that the human is holding here. They're called uh um or universal manipulation interface. So this is basically like the embodiment of where the task is is becoming a lot more similar to what the robot sees when we deploy it. And the last way to to get data is simulation. So example here we put a car in a simulator it starts driving and we start recording the data.

Cool. But the first of all it's large scale so that's good. But the problem is that it's extremely difficult to reproduce the physics of the real world. We we cannot copy and model the physics of the real world in simulation. And second, it's that it's very it's actually impossible to reproduce the structural complexity and the diversity in the real world. So like when we train something in

simulation and try to transfer it in the real world, it cannot work because it hasn't seen what is actually the real world. It's what it has seen in simulation is completely different. So let's look at the different branches and I'm I'm going to connect this in a second. So I've briefly touched about this already. So locomotion this is first the skill of how we are how skill

of balancing. So like for a dog or a humanoid how we can balance like we are not just figuring we're not figuring out where to walk yet. We're just trying to balance and stay on our legs. Then the second branch is navigation. Now we are looking at the problem where are we going to walk like we're going to walk safely such that we don't crash into obstacles

and we don't cause danger to other uh other people in the environment. So autonomous driving is here an example and the last branch is manipulation. So this is about how do we manipulate the objects in the real world and how do we interact with them I would consider actually walkotion 90% solved today navigation is getting close very very close but manipulation is the one which is unsolved

so everyone today is asking how are we going to solve this problem of manipulation. And that brings me to my third contrarian view that the solution for one of the branches will not be the same for the others. And let me show you a bit more why. So what we what worked in Wcomotion if we start looking at it, it was that it was simulation surprisingly actually.

what they did as I mentioned there is a problem that uh there is a gap between the simulation and the real world. So the method that they use they randomize sim the dynamics in the simulation and now you're actually start seeing randomly the dynamics of the real world during this training um by randomizing the properties of the robot the terrains and the physics and you use reinforcement

learning as a method to train this. So this is if I translate this this is basically you a trial and error. So you try something, you see if it works and you learn from it and people were able to make this like to build these very capable systems using this uh approach. So like in the video on the right here you can see how the system even

adapts if if you injure one of the legs of the of the robot. So first it might not work but then okay it's able to balance similar to how humans work. Here it's locking the wheels of the of the robot and it instead of sliding it starts walking which is an very impressive kind of skill. Moving on to navigation. What worked there? The data was simulation. No,

it didn't that didn't work at all. What worked was tele operation. So you have a a lot of humans uh driving through the real world and collecting this data and we are the method that worked okay we are going to take this data we're going to copy what these humans do and we're are going to slightly improve by trial and error afterwards and what it was able

to achieve I already mentioned this able to drive 500 cities around the which is the big big question now today the data is it going to be operation personally I don't think so a lot of companies are actually pouring a lot of money today in this daily operation data but I don't think that's going to be the way that works is going to be human data yes

I strongly think so because it's we're to build humanoids that are working in the environments that we have already built for humans in our homes. We can learn a lot from the from us, from ourselves as humans. Simulation, are we going to need it? Yes. I I'll actually argue that we're going to need it. And this is going to be for this dextrous five finger um robots.

And the methods are going to be very kind of similar to what we saw in the previous branches. And here are you can see what we are what these methods are able to achieve today. Very different skills with different kind of complexity. I'm going to skip this quickly in the interest of time and I'm going to go to my last controversial point about hardware. So I would

argue that hardware agnostic solutions will bit build the will beat the full stack solutions which in incorporate both hardware and software. And to make my point here, I'm going to again take a lesson from from autonomous driving. Why I'm talking about this is because like a lot of companies today are betting on this like they're building both hardware and software. So the lesson from autonomous driving is

that if we take one of the leading companies Whimo uh this Google's uh spin out uh they they're probably very ahead of in the game and they have only 3,000 cars operating in the world today. They're building both hardware and software. Tesla, another example, They have 7 million cars in operation today. In comparison, the cars in the in the world today are 1.6 billion. So they have

barely touched even 1% of the total number of cars in the real world. In comparison, Wave who are already building um only softwarebased solutions, they are now going to brands such as Mark, Mercedes, Stalantis, Nissan. They're making contracts with them and they can directly put their software on their cars. So like there the market is operating in a way where it's the hardware is already the hardware

market is already captured. So to scale and to build the solution which actually scales you need to be only software and go to the people who already have operations in the real So my last finally my takeaways generalist robotics models will be the kinks and why once they're one once they work we're going to see a market for vertical applications which are built on top how we

are going to solve the manipul solve manipulation the big questions today human data in my opinion together with simulation data and lastly hardware agnostic solutions will rule the market but we will not be the first one to deploy. Thank you.