Music with Artificial Intelligence


Hey it’s Andrew Huang! there are some people who believe that the development of artificial Intelligence and machine learning will lead to humanity’s demise which maybe so maybe no, I’m asking the same question I’m always asking Which is: will it music? this video is a collaboration with a team at Google called magenta whose focus is on exploring How we can create art and music with machine learning They got in touch a little while ago and showed me a bunch of things they were working on and asked if there was anything I would want to do with it, and I was really drawn to a project of theirs called N-SYNTH, which stands for neural synthesizer It’s super exciting to me because it’s an entirely new form of synthesis rather than manipulating sounds like most synthesizers N-SYNTH manipulates data it analyzes audio files learns how all that data relates to itself and then Produces new audio files from scratch and this opens up some interesting Possibilities so for instance if we have this vibraphone sound and this guitar sound We can ask and synth to generate a new sound that is 50% of each It’s interpolating between the raw data of both sounds at the level of individual samples and when I say samples I don’t mean like this is a drum sample or this is a flute sample I Mean like the building blocks of digital audio You know how a video is made up of different frames for instance these are all the frames You just saw of me saying the word frames in this video You’re seeing 24 frames per second, and it makes for a pretty smooth recreation of all of my movement With audio, it’s similar You need all these tiny little Individual samples of sound that get strung together to make what you hear except with audio you need a little bit more right now You’re hearing 48,000 samples per second our ears just need that much more resolution to perceive digital audio with a quality that’s close to real life and Synth contextually generates one sample at a time based on the previous few thousand samples as well as all the audio It’s been trained on and n synth sounds a little grittier because they opted for a lower sample rate of 16,000 samples per second because that is still a ton of sample To generate so you can play with n synth right in your browser I’ll put a link in the description since I had access to magentas resources though I wanted to do some experimenting the incent algorithm was trained on pitched material. It was trained on notes 300,000 notes from a thousand different instruments. I like to mess with stuff so when the folks at Google asked If there was anything I’d like to do I said two things what if we feed it? Percussion and what if we feed it a bunch of completely random sounds that people send me on Twitter And that’s what we did I sent in about a hundred drum sounds from my own sample packs And I also got about a hundred submissions from people on Twitter, and I didn’t give people any guidelines We ended up with sounds from instruments household objects voices animals The magenta team crossed every combination of two drum sounds and every combination of two random Twitter sounds leading to just shy of 9,000 new sounds and yes there will be a sample back The algorithm did a really good job with the drums. I just got a whole bunch of variations on drum sounds Where things really got interesting is when we combined two sounds that were completely different from each other? Here’s a Stylophone? And a 3d printer. This is what they sound like crossed What if we take this baby goat and? combine that with that 3d printer Oops just summoned Satan here’s one more example. We’ve got a fountain Frost with a string scrape This combination is really interesting Doesn’t that just sound like an electric guitar this type of sound happened fairly often And I’m wondering if it’s a byproduct of ensign having to deal with much more complex sounds than single notes dealing with a lot more Frequencies at once but still trying to interpret that in a pitched way because it was trained on pitched material Maybe I’m gonna end this video by sharing a piece of music I made using only these sounds that ends synth generated the drum ones and the Twitter ones I started by just trying to listen to them all and I think I got through almost a Thousand before giving up on that idea so I made some selects I organized them in Ableton based on how they sound and then I made this track basically by just Moving them around putting them in different orders stacking them in different combinations listening the whole time Seeing what worked and how things flowed and I ended up using somewhere between two and three hundred of these Sounds in the final piece if you’re interested I talk more about my process in a video on natin friends channel Which I’ll link to I named this track rainbow gram Which is what they call all these cool-looking sound visualizations on the N synth website That’s a whole other tangent that we’re not going to take right now. Here’s the music

100 Replies to “Music with Artificial Intelligence”

  1. I love that track! Would love to hear a revamp with higher quality sounds, but still using the base sounds.

  2. Funnny that you can hear that the samples themselves are low resolution, but the effects (esp. reverb and other kinda stuff) that Andrew puts on them sort of masks/glosses over that and makes them sound clean altogether

  3. Man i love your vids. I cant believe i havent watched them sooner. Your getting me off my instruments and back on the computer😂. Is there any episodes on your team behind you and their involvement?

  4. I'm going to claim that a 320 kbps mp3 file delivers a better audio quality than a 16 kHz wave file (at a much lower size), and that therefore this algorithm would have much better results working with mp3 files.

    Does anybody know if this project was open source, or however if I can find further info somewhere?

  5. If you take 9000 sounds and choose 50 of them, you make a track, then this is the merit of not artificial intelligence. You chose these 50 sounds from a huge amount. With the same success, we can say that the garbage dump created something meaningful, if someone suddenly managed to find something useful in the trash.

  6. Now I really want Andrew to make a track using all the pokemon sounds. Not the repeating their name pokemon sounds, the 8 bit sounds from the games. 🙂

  7. Umm… real producer been layering and blending different sounds for decades. Sad you cant create your own music so you gotta get AI to do it for you, yet you still dont have a hit single.

  8. the intro kinda just soumded like i was listening to it through 1$ earbuds but the beat itself was sick af so maybe if you made a second ai that took music recorded in low quality and tries to make the soumd clearer it would go from 7/10 to 9/10

    edit, watched the rest of the video, i misunderstood the title. didnt realise only the individual sounds where ai-ed. no wonder the rythym is so on point, that part was andrew lol

  9. I like this, but you should check out Orb Composer – it's designed for composers and lets you have more creative control and makes complete tracks. Just listen to some of the orchestral stuff they've put out:

    https://www.youtube.com/watch?v=acLzFzBwIxY&list=PLmNpr_PCZR-v4h5HEZGH3AYyJNDdp82HB

  10. Think about it, when I’m like 50 people are gonna be all crazy for the “vintage” sound of the original n-synth. Now that all of the n-synth models are 48,000 samples per second, the gritty sound of the 16,000 samples per second synths are unobtainable.

  11. You make great tracks but this is easily my favorite so far, crazy beautiful dark organic sounding texture, just badass

  12. When you talk about samples, you talk about video frames as if one frame is one sample, when actually one pixel is one sample. At 1080p, that is over 2 million samples per frame. At 24fps, the number of image samples per second is over a thousand times larger than the number of audio samples.

    P.S.: Well, technically, each pixel is three samples, one for each color channel, which makes this all even more ridiculous.

  13. What a fantastic video. The slap got me laughing, the summoning Satan will make me laugh for a week and the song is RAD.

  14. Hi I just downloaded the sample pack and I'm trying to download them. I got the link and followed it to the samples but there doesn''t seem to be anyway to download them and I tried with firefox and windows explorer. I'm sure I'm missing something simple but please help because I just want to download the samples.

  15. question: what if you got 1000 base sounds, numbered them, then combined each number pair (1 & 2, 3 & 4 ect) to get 500 sounds, numbered them, did it again to get to 250, did it again ect ect until you were left with one sound. What would that sound be like?

  16. google could turn this into a module. Get it working with real time inputs. Two sound input ports. One sound output. Boom. Fucking put that in your eurorack

Leave a Reply

Your email address will not be published. Required fields are marked *