“Browser Grooves - Building a Groovebox, Looper and Sampler for the Browser” – Алекс Миланов

This talk presents the development of music software applications that model hardware devices for music creation and performance. The speaker, a developer and musician, discusses a series of apps he has built, including a groove box that simulates a digital audio workstation and a sampler inspired by classic devices like the Akai MPC. He shares insights into the use of web audio and MIDI technologies, exploring functionalities such as sequencers, synth engines, and integration with MIDI controllers. The session highlights the challenges of latency in audio software and the optimizations he has implemented to ensure a seamless user experience. The speaker encourages collaboration and community engagement in the ongoing development of these open-source tools.

Alexander Groove Box. >> [applause] >> Check one, two. Check one, two. Uh, just a quick question. Uh, is there anyone who doesn't understand Bulgarian in the room? There's one person. Okay. I'm the risk. Super. Yeah. Okay. Uh browser groups. This is actually version four of this uh lecture because this is these apps are being developed. Um actually maybe it's version five because uh in 2018 I gave

a presentation here at Openfest for the first app. Uh so there's a little bit of history of of that. Uh the other name of this lecture is I didn't have money to buy hardware so I wrote software. Uh what are we talking about? So um these are basically apps that model u music hardware and I'm going to go a little bit uh into details. Uh I have

different motivation uh uh for for this work. One is uh of course exploration. Uh the other thing is uh uh who was yesterday at the jam dusty. uh I'm I'm organizing uh different events here in Sophia u uh around music tech BG and also a lot a lot of them are also in it the hacker space which is one of the partners here of the open fest

uh and um in uh at least before I I bought now I have hardware but before I bought it I also used these apps uh to jam and of course building my own tools Um, I have some videos. So, I'm something between a developer and musician. So, I experiment with a lot of things. Also, some acoustic instruments like hopefully you'll see. So, only acoustic instruments. Uh I'm

not sure how uh if we have enough time to to play every video to the end, but uh just to give you a snippet of what uh the things that I I do. Uh this is one of the jam sessions from last year. I wanted to add uh a clip from yesterday, but I didn't have enough time to edit it, but uh it will be in YouTube

or for sure you can see uh what happened. This is a clip from Jam at from last So, uh if I had a fader, I could uh when I stop it, I could fade out. That's uh that's u something to improve on my uh presentation app which is actually also wrote this uh this app building my own tools. can you guys hear me well? >> Okay, good.

>> This is uh presenting [laughter] the jam station at one event I think in Sweden. Uh as you see there is also my MIDI controller there. I created connected it to a my armbarm band uh which has a accelerometer and ETG like u um gesture uh handler let's say and I managed to create to connect the accelerometer to the jam station and you can create the different

effects uh with uh like movement. So this is like exploring in the direction of performance. Okay. And uh uh for a couple years while I was building that I was promoting my my own framework web- based framework. Uh and um in my my opinion it was much straightforward than uh the others. right now. I mean, I still use sorry, I still use it. Uh, but I'm not

pushing it that much, but just to to give you an example, this is basically my framework uh working on top of uh the the slides up working on top of the framework. So you have actions that uh modify the state which is reflected in the UI and you have a action structure and you can easily create uh a new action. Just a second one. Then you change

the UI. after I'm done. Anyhow, uh this is the the problem of uh live uh presentation and uh writing everything yourself. Uh so we create a new change that to this change it here. >> Okay. Cool. So, we did like a pair coding uh live. [sighs] Cool. Uh so, uh I'm not going to go through the timeline. I mentioned that u uh I did some of the

things uh and some of my interests. So, I'm going to skip that. Uh let's look at the apps. So the first app uh that I started I was uh talking about uh it being a D and actually a digital audio workstation kind of simulates a lot of different uh hardware that was used to produce music like in the in late 70s 80s. Uh so they're actually those

modeled hardware and studio equipments. Uh but at this stage I can call it a groove box. what it what the groove box is is basically it has a sequencer. It has some kind of synth engine as maybe some kind of drum machine or a sampler. Um and currently in its current state it has uh the UI interface is heavily inspired by at least u a version of

foss that they used 25 years ago where you have uh where you directly interact with the drum sequence. So for example if you play just a second [music] and then you can For some reason this pointer when it's full screen there is some kind of misalignment in full screen mode. Okay. So basically you can create your own drum pattern. This is the sequencer elements and the drum

machine in one. The other uh point is it has a synth. You can use a MIDI keyboard or you can directly interact with the screen. I have a tablet. So, uh there's different ways you can uh record things. And for example, I'll I'll just make a short recording. All right. This is the first app I created. Uh it uh these two elements. It has a session manager.

So technically you can build a whole track with different parts. Uh you can have different MIDI devices that interact with it. And uh I've also used it to control other MIDI devices. So you can use it as a sequencer that you can can connect multiple uh devices. The synth engine is uh a little uh is not that complex. It has two oscillators that create the sound. It

has a square wave and south yeah south wave uh has uh uh several uh yeah you can uh control the the shape of the uh the sound or attack decay sustain. Now if I just see if I can uh so technically you can create different types of sounds uh directly in in the app. So you can have a baseline, you can have uh some kind of strings

uh for harmony and some uh like a bell like for melody. So we have some some ability to to control the uh the sound. Okay, we have have some time. Uh currently uh there are uh some samples that can you can also use like a small sample library. I want to expand it uh that you can uh get sounds from free sound or record your own sounds

like you can do with one of the other apps but this is for when I have time to implement that or if somebody really wants that then uh for me it's u additional motivation to implement uh this feature. Uh what else? inspiration from E studio uh the original app. Uh yeah uh I mentioned that also I integrated with different uh devices like the armband. and yeah I

mostly used it uh a lot to experiment and uh to connect it with other devices and it was my main uh how should you say it uh main tool to to sequence music and uh explore different things for a while. Uh as I said you can quickly sketch some kind of uh uh a melody or a track. Uh you can use uh for example you can um

have uh all time signatures. It has a uh for example you can make we can make this in 78 So this is Axel F78. So you can definitely use it to if you want to uh learn the guitar or some other instruments, you can uh create very easily some kind of basic harmony as some progression and you can jam on top of that. uh rhythm exploration. I

mentioned that uh because it supports oldtime signatures and I said MIDI clock centerpiece and you can uh connect other devices uh through MIDI uh as I've created right now one device but yeah can do more uh current works um the piano row currently u uh is just for visualizing and it would be nice if you can just draw the notes directly. Uh there are some issues. Uh

if you heard when I press pray there is u a delay of one sometimes uh of one uh bar and uh I would definitely want to make it uh uh responsive interface so that you can use it on the phone. Um I mentioned that I would like to integrate functions from the other apps like sampling and uh some uh audio loops uh would be uh it would

be a huge step in going to the uh digital audio workstation route. Uh yeah there are some uh open um uh open source uh file um conventions or music uh uh like music XML is uh for uh notation and D project is something that is used in Bitwick and some other um Dows to share uh uh project data between them. Uh and right now I've uh you

have VCF like so we have a low pass and high pass filter. You have an LFO I need to and reverb. So you have uh some tools that can shape shape your I think this was making it like a more uh but um uh currently you can only toggle it on and off and I would like to have it uh to be able to add uh specific

effect to specific tracks and uh it would be nice uh to to allow for third party plugins. So if somebody wants to create some kind of effect and integrate it to be able to to add it to to the uh especially now with uh web assembly where you can compile C code to to something that runs in the browser. Uh [sighs] uh well I was given the

demo uh all of these apps are hosted in my GitHub account. Uh so Alex-billanov and you'll be able to find them. I'm not sure if uh I'm going to be able to create a link for this presentation uh in somewhere in the site of openfest but this uh al also can be published in my GitHub. Um all right so this is the first app. Uh by the

way do you guys have any questions about it? >> uh I think it we may start with something specific and uh at some point of course uh if uh there there's a community and people want to use it definitely u there'll be direction to integrate like a common ways to share plugins. Yeah. yeah. So this is the first and the main app that I'm developing. I'm kind

of hearing an echo. I don't know if Yeah. Cool. Uh so the second app is the whooper. uh yesterday uh if if you were there the actually the whooper uh the inspiration for this app was the centerpiece of the of the gem um it's a way uh what inspired me just a uh is this uh RC505 loop station by by Boston Roland which was uh uh quite

popular back in the day especially for beatbox battles where people for uh creating uh tracks on the on the fly and so uh I I was watching a lot of these battles and I couldn't afford uh this device so I said how hard would be to create this one uh and yeah mega creative name the first is jam station the second is GS loop station and very

creative with naming things um It has four loop tracks and uh uh you can technically create connect several instruments at the same time. So when I'm talking about this, I think it would be nice to to be able to to see it. So uh instead of five, uh I created uh with four tracks because uh this controller has four sets of pads. Right now it's not working

but technically I create so that you can control it even with this device. Um and uh the interaction is that uh right now it's uh connected to the default uh input but technically you can create connect the guitar microphone or another instrument and you can uh loop or record things together at the same time. So let's uh let me give you an example of how how it

works. Check check check and check one two. Uh, can you guys uh Is there a sound from the check? Check. Check. One, two. Uh, give me a second. Let me see if it's connected. Oh, it's uh it's my uh [groaning] it's my fault. Uh the laptop went to standby and it disconnected. So, check check. Cool. Well, it happens. Uh so, let me try to create a loop.

let's clear everything. So that's uh a simple loop. Uh and if I uh had connected it to the uh uh MIDI MIDI uh device, it will be much uh uh more like a direct performance. Right now I had to click on the or touch the screen and it was not very uh direct or I haven't practiced it. Maybe if I practice with the touch interface it will

be better. So this is uh the second this is the second app. it's uh uh it's functioning uh as you expect. Uh there are some things like quantizing the recording because sometimes it starts earlier or or later uh and it makes it so that it's kind of synced uh uh between all the uh different uh tracks uh and uh yeah. Do you have any questions about this

app? Maybe I will. Yeah. Yeah. Yeah. That's a good idea. Uh it of course the RC505 has a lot more functions. This is why I'm also using it instead of this. But uh definitely uh it's uh [groaning] uh and actually uh some of this functionality are going to be integrated also in the other apps. Uh and uh it's and all of them are open source. So if

you guys have some ideas how you we can use them or uh if you want to add something like there, please contact me and uh open the app, clone it, look at the source code, uh create a like a bad review. No, I'm joking. Uh but yeah, it's u it's something that I decided to share at some point. So all of them are in GitHub with open

licenses. So yeah semifunctional looper. Yeah. Uh oh yeah I I uh yeah I I created uh I used used to use it with the foot switch which is uh which allows me to loop and use the guitar at the same time. It's yeah it is fun. Yeah. Effect chains. Ah mobile devices. This is like one of my top to-dos for all the apps. And of course it's

in the it's hosted in Maybe I'll create a website uh at some point for these apps. Uh yeah, it's not top priority but it will be nice to have the information in one place. The last app. Okay, I have a little bit of time. Uh sampler. This is uh so the first app was modeling a groove box. The second was modeling uh looper and the third one

is modeling a sampler. And uh sampler um if you don't know what it is, it's basically a lot of hip-hop artists uh use this type of software to to create uh their tracks. Um the M MPC I think uh was one of the uh top devices that was used for that. and this kind of models that uh yeah uh inspiration from Akai MPC series. I own uh

SP 404 and uh I kicked off this uh um app uh in a hackathon in uh Abi Road Studios several years ago uh before the pandemic. Um and uh interesting part about it is there was an audio common project which integrated free sound and other similar projects to uh create a platform where you can search for sounds that are uh have creative comments or CC0 or others

that you have to attribute uh the the creator. a grid interface as as you would imagine like an MPC to have like a row of 4x4. It has uh search panel where you can search for sounds for example from free Let's search for kicks. Then you can for some reason the HDMI connection kind just give me a second. Yeah, probably the same person samples multiple times. Let's

see. This is kind of better. And uh you can uh play the sounds and if you like a certain sound, you can add it to the grid. Or another option that you have is you can record or sample some of your instruments. Like yesterday I brought some u uh Cuban percussion or in this case you can sample your voice like we have a kick maybe a snare.

And of course I I don't want to have like a gap uh in the start of the sound. we have some basic wave editing uh or really basic. So you can either uh uh crop the the sound or you can flip it or you can only crop it at the moment I guess. [laughter] So it's uh as it is uh it's basically creates your pallet of different

sounds and uh at this stage and you can uh use a MIDI controller to to finger drum at the moment. So what is the future of this uh direct sampling workflow such as explore sample uh potential use case is u uh if you have uh as some people including myself have different sampling devices or drum machines. It would be nice to have one uh like a sample

library where you can organize your sounds and uh pick them and then you can uh export them to a format that is for a specific device. Uh but currently it's not uh it's far from there. uh the current works and I added it in the previous uh presentation that reintegrate the free sound API because after the audio commerce project finished or the the financing for it uh

it stopped working and uh yeah and but uh now there is uh uh connection to free sound again. Um would be nice to have some sequencing logic like in the jam station you can uh sequence uh pre-made uh uh drum sounds but here uh you can just uh sample the sounds directly and uh create the sequence. It would be nice if you uh can uh have this

uh function uh directly in the app and it's it's like the groove box and uh sampler have different types of interface. So um groove box kind of uh incorporates a little bit of the interaction as a sampler but sampler first like the SP 404 or the MPC series is a different way you can create sound. So it's even though we are integrating I'll be integrating functionalities from

app one app to the other uh it's still a plus because uh all of them have different interface to interact with um it would be nice I think the the looping of the the samples will be something that uh uh can be done quickly for example you can as as in a looper you can create like a basic beat or load something that loops and then you

can finger drum on top of So this this is on the on the horizon for this app. well I'm not sure about this is why I wrote maybe uh splice integration or other sample providers. Currently it's free sound. It's fine for a lot of cases but for some cases uh uh it would be nice maybe to have other sources as well. And as as I mentioned, it

would be really nice if it's uh if you have functionality to to manage your whole sample library and be able to import and export samples from uh different hardware. Uh again, it's uh hosted in my GitHub Uh do you guys have any questions about you can use them together. Uh I haven't explored it that much but basically all of them are currently running. I think there was

some limitations if uh they're not in the active tab. Uh there might be some some uh Yeah. >> Yeah. I'm going to I'm going to try something. Uh just a second. >> [music] >> Give me a second. I I'll You gave me an interesting idea. >> It's too far from this. Um >> Oh. Oh. Oh. Look. >> I uh [groaning] you you're giving me uh an interesting

uh separate >> Uh every tab is uh is a separate thing. >> Uh uh but uh no you you won't be able I I understood like uh you can create um I probably can uh connect the monitor of the uh browser to as an input uh but uh Yeah, sadly doesn't right now doesn't work that way. But uh yeah. Yeah, that's that's a limitation that uh if

it if I compile these apps as a desktop apps >> then yes. >> Yeah. Yeah. Uh but uh technically uh I can like right now I was sampling this but for example I uh not the best but just for demonstration purposes. Yeah. So if you really want to do it, you can create something and and use them together. Uh and uh the jam station if you create

like um um how should you say it? Uh there is uh medi uh through ports. Uh you can technically uh uh put the output of the sequencer to the through port and uh put the input of the uh sampler uh again to the through port. So you have to have this uh and in Linux you can there are some commands that you can create uh multiple through

ports. So this is a way you can uh connect them together within the same machine. So technically yes I haven't worked too much into integrating them but uh and with the browser is for the sound especially it's uh I don't know if uh um it's on the on the radar of Google or other browser developers to have every tab as a single app with inputs and outputs.

uh but it can be uh like with electron and other uh tools you can package these apps as a desktop apps and then it's uh you can do definitely do that. Uh so this is the third app. Um I'll find a way if I can publish this because we have some more things uh that I wanted to share like uh web audio. So let's go a little

bit through web audio. Um there's a little bit web audio and web MIDI. Uh it's an interesting uh API. Uh you have a audio context uh which uh gives you a lot of things. Uh and uh it's the node that you connect to all the other nodes. And I I created this uh modular syn picture because it's uh this is a way you can uh interact with

web audio. we have basically have different nodes that are connected to each other. Uh the context is the the the main node and uh uh to create an oscillator or audio source uh or oscillator source basically you use this context.create oscillator and then you can oscillator connect context destination. So it's really straightforward. Uh and like an example it's uh like the the pure waves is not the

best thing for your ears. Uh but is is basically you can see the uh the code behind it. Then uh we we connected it. Then we have uh uh we start uh uh and stop functions and uh then I uh oh it's self-explanatory. So after two seconds stop the sound uh or and disconnect it. this was not the Okay. So what do I change? Ah okay. So

it's uh then I uh this is an example to use a sawtooth wave and a specific frequency. I think it's uh a or la. Sorry for that. It's like a pure south south wave. Uh maybe you can uh lower the the sound a little bit for this type uh this part of the presentation. uh because I'm uh it's if if you don't uh a gain uh note

which you can create the uh uh the volume then uh it's okay one minute uh it's a very uh not that pleasant uh yeah and this is the the the fourth I think node that you can use and ah okay here I So it's an example of how uh you can uh uh modify the sound. And uh here I sequenced the simple melody very children song type

of thing. But as you see with with a little bit of code uh you can use web audio to uh create uh uh different uh possibilities or write software with it. Uh there is some more slides for uh uh web MIDI and uh for input and output uh in the browser but uh definitely uh I I'll try to publish it someplace so that uh you can use

it as a tutorial. How can you uh get in touch with uh with me in GitHub alex-millanov all the apps are hosted there and uh Alex_millanof is my Instagram account where I post uh uh clips and things like that and you can DM me there as well. uh community uh music techbg is the thing that uh uh like the umbrella that we organize events and jams and

things like that and initlab is the hacker space in Sophia where we a lot of our events uh take place and it's uh like a partner for this conference do do we have time for more questions or uh yes so we have a few minutes more for some questions if anybody has anything to ask, uh, he can go to the microphone in the middle so every everyone

can hear. So, hi, thanks for the really nice presentation. Uh, so one of the very diff most difficult thing when you're writing software for audio and music in particular is latency. >> Yeah. And uh so I'm very impressed that you you seem to have very low latency even if you're developing in in basically JavaScript here. So do you have to do anything special here to to reduce

latency and have a nice experience? >> Thank you for the question. Yes, it is. uh and uh I've done some optimizations uh for some of the software uh for the looper is is actually the ones that uh if you if you have latency you cannot uh loop uh effectively or you can't loop like if it's uh over a certain like milliseconds then your brain you cannot process

it. Uh so this is why currently it the the sound does not go through the browser because if it goes through the browser there's already higher latency that uh than you can work with. So I've uh as you guys saw I I routed it directly through Linux like uh there is this Katya software which I got the input from the mixer to the output of the HDMI

so that I can perform looping uh right I mean if you ask Google or you have contact there to optimize this probably that but I I imagine that it's not their high priority. >> Thank you very much. But there are workarounds as you saw. >> Okay. So sadly we've we're a bit over our time. So if if anyone else has any other questions, you can meet Alex

Mu outside of the hall. And that's pretty much it. Please a round of applause for his presentation. Yeah, guys, if you have any ideas, uh, these are the links. Alex_millof in Instagram or GitHub. Yeah, thank you.