About this talk
This talk explores the untapped potential of web development, emphasizing that many existing web projects can be enhanced using the capabilities already available in browsers. The speaker, Laura Brooke, discusses how websites can transcend traditional interactions like clicks and scrolling, suggesting that modern browsers equipped with various sensors can create more immersive web spaces. By utilizing APIs for device motion, orientation, and voice recognition, developers can foster environments that respond dynamically to user actions and context. The difference between a web page and a web space is highlighted, where the latter integrates sensory inputs to create shared experiences. Laura encourages experimentation with simple hardware and JavaScript to unlock new possibilities in interactive design.
Full transcript
Welcome. Hi everyone. Um I'm Laura Brooke. I run a small studio in London called Hyper critical sir. We build interactive things that live somewhere between websites and a piece of furniture. For the next 30 minutes, I'm going to be talking about how websites Well, the websites that we're already shipping are often more capable than we give them credit to be or allow them to be. Not with
new tools, but with the ones we already have access to. We can actually unlock a whole world of possibilities to experiment and play with. So, um something to think about. How many inputs does your project currently respond to? It's click, it's scroll, it's keyboard, maybe hover. But most of us are shipping something that just responds to a few. So, yeah, click, scroll, touch, keyboard. But the browser
sitting on a phone with nine sensors in it, a microphone, a camera, a gyroscope, and it's an accelerator, ambient light, proximity, and we're just using four. It's worth thinking about that for a minute. Because the gap between the physical world and your JavaScript is very small. The capability has been available for us to us for years. The APIs are pretty stable. They're shipping in the browsers you
have open right now. The thing in the way isn't the technology. It's us making the time to explore and experiment. So, the APIs that let us play, um, generic sensors, device motion, device orientation, web speech. None of this is based on native apps or specialist SDKs. If you have it if you have the ability to write an event listener, you can use every single one of these
right now. Device motion, device orientation, they're available everywhere. Web speech synthesis works everywhere. Uh, recognition is Chromium only. Generic sensor is the newest most powerful, but that's currently Chromium Um, but it still covers the majority of users. we've been shipping web pages when perhaps we should be building web spaces. the web page is a metaphor taken from print. We took it online in the early '90s, and
we've never really questioned it. But if devices can sense light, motion, orientation, proximity, and voice, it knows where you are, it knows who you are, how you're moving, and what you're saying. Then, what you're building isn't really a page, it's a space. So, some of the key differences between pages and spaces, um, a page is a thing you look at. A space is something that you're in.
A page has visitors. A space has occupants. A page responds to clicks. A space responds to conditions. A space changes depending on who's in and what time it is, and whether the lights are on. It's the same web stack, but a whole set of different possibilities, which in the past might have felt daunting, but with the assistance of AI, there's so much you can do right now
with the code that you're already developing. And we have an opportunity to build in the time that it used to take us to build pages. So, web spaces are already here. Um what makes them work isn't the technology, it's what they make people feel. They create shared experiences, they trigger real emotion. They turn passive visitors into active participants. So, for example, members clubs where you your card
recognizes who you are and personalizes the room. Gallery spaces where the whole exhibition responds to how long you stand in front of a piece, your movement through the space. Restaurants where boards changes uh with menus based on the time of day and how busy it is. Formula 1 cars where digital panels update based on what's happening in the race with different advertisements, et cetera. Brands are creating
content for websites every day. If those brands have physical spaces, that content can really come to life in those spaces. So, JavaScript is at the center of it all. Um On one end, you have built-in sensors with the the browser that are available right now with no extra hardware, just APIs. They can detect motion, orientation, light, voice. In the middle, a camera plus something like MediaPipe, Google's
open source framework for real-time computer vision in the browser, which can detect pose, hand tracking, gaze, etc. On the far end, external hardware over web sockets, microcontrollers, RFID readers, NFC tags, physical buttons, weight sensors, proximity any real-world input can be talking to your web project. Hardware is getting smaller and costs are falling. Screens are getting bigger, thinner, more more flexible. Projections can be sensed. So, um now
I'm going to attempt to do a quick uh demo to show you sort of just how easy it is to get some external hardware talking to your web browser. Hoping this will work. Cool. So, here we have some metaballs that um are just taken from a web project. And here we have I don't know if you can see them, some uh eggs, large eggs. And so, by
moving these, theoretically, let's try this one. I might have to do a quick uh reshuffle Oh. The problem is that I'm not on the same network as the eggs. So, I'm just going to switch that now. Bear with me. So. Okay. So, I'm going to try one more time. So, yeah. We can see now that we've got metaballs that respond to movement. this one's maybe falling asleep.
Let me check. >> [laughter] >> This one seems to have gone to sleep for some reason. possibly the battery, but the point is that you can sort of create devices that really easily interact with screens or anything that you you choose to display your web content on. yeah, so >> Yes. So, um finally back to the right place. Yes, so the idea is that without a click,
without a tap, the browser is able to understand what was happening. yeah, for really a very cheap price, you can buy microcontrollers. They have Wi-Fi on board, which allows you to then use a web socket to to read what they're sensing and send it back to the browser. So, yeah, what was inside the the eggs? Just a really simple ESP32 connected to a small battery pack. A
small bit of code programmed in Arduino. And then just a very simple uh node web socket server that I was running on my machine. And yeah, from here a web app can react to data from any source, um triggering a video, swapping a layout, animating a panel. Pushing to a CMS. The visual layer is what we have a lot of understanding of, and this is just about
extending it to that. Um so, yeah, it's just um headless API first, the same stack, but just a set of different triggers, infinite surfaces. Um the architecture is so simple that it you may as well experiment and have some fun. Um yeah, so really really just the same assets, new surfaces. So, your clients already have um video, photography, editorial content, and a lot of them will have
physical spaces as well. Obviously, this is a very basic prototype to show you the art of the possible, but this is the kind of thing that you can do in half a day to a And if that's possible, what can you do with a bit more time available? For example, if you had a um if you had a coffee shop as one of your clients, you could
have some coffee on display, attach a sensor to it, and then a screen nearby starts playing video content that previously you'd only used on the on their website, and that triggers when someone picks up a um packet of coffee, something like that. I mean, the the possibilities are really endless with what you can do. Um and actually, it's just uh small bit of JavaScript and a sensor
making it possible. So, um yeah, none of this should really be without consideration. There are a lot of factors that we should consider. For example, accessibility. We want to make sure that anybody can use things and get benefit from them. So, if something is dependent on someone being able to lift something up, then it should also be by a click or something like that. Um The other
thing is privacy and security. if you're using cameras and microphones, you should really ask permission. You be sure not to store anything that you don't necessarily need to store. I think it's also important to look at the security around it, any potential risks, and making sure that the person that's using or in the space is able to understand how that data is being used. assets. So, yeah,
I mean the assets, important to understand that if you're building this into a CMS, you need to be thinking about including intent. It's not necessarily a layout thing, but it is how your content is going to be used in different contexts. Um the final thing is is around resolution and and surface. So, a web space might render on a 4K wall, um a watch, a projector, or
a phone. Design the system, not the screen. Um make sure that if you're commissioning video, that you're commissioning it in the sizes that you're likely to need to use it in. Same with photography. Um vectors are our friends really because obviously they they scale. Um But yeah, the the um what I would say is that web spaces aren't the future, they're the present that's waiting to happen
and waiting for us to build. And yeah, I think I'm finishing quite a lot earlier than planned, but I'd love to um chat to anyone that's interested in this area. Anyone that's already building things, anyone that wants to build things. I'll also be sharing the demo files. So, if if you want to receive those, then please do sign up to my newsletter or drop me an email
and I'll I'll send those through to you so you can have a a play around, too. Thank you. Thanks. >> [applause]
