Lemonaide is a state of the art AI Company that creates MIDI and audio plugins that generate inspirational ideas (such as melodies and chord progressions in any key). They are partnered with BeatStars, an online marketplace for electronic music producers and beat makers, where they sell access to their Generative AI MIDI models.
In 2024, they partnered with a second company, Sauceware, to release a new plugin called Spawn with audio visualization and a substantially larger collection of sounds. The original Lemonaide app has a small monthly subscription fee, whereas Spawn comes with a one-time purchase fee.
Whether you’re stuck in a creative rut or looking to experiment with new styles, Lemonaide makes sure you never run out of ideas. They’ve achieved a high quality, human sound with rolling chord articulation and catchy singable melodies. The model generates 4 and 8 bar phrases in a single key, appealing to sample-based producers in search of a quick starting point.
Lemonaide’s Fairly Trained AI MIDI models
Lemonaide began with a home-brewed base model called Seeds, with four different moods to choose from. In 2024 they released a handful of fine-tuned AI MIDI models, called the Collab Club, in partnership with Grammy-winning producers and chart-topping artists:
- Kato On The Track: Billboard-charting producer with credits for Joyner Lucas, E-40, and Ice Cube.
- KXVI: Grammy-nominated R&B/Soul producer with credits for SZA, Future, and DJ Khaled.
- DJ Pain 1: Multi-platinum Neo Soul/Hip Hop producer for Nipsey Hussle, 50 Cent, and Ludacris.
- Mantra: Pop hitmaker for Rihanna, Dua Lipa, and Bad Bunny.
Each model is designed to reflect the nuances of its genre, giving you access to styles crafted by industry pros. The Collab Club models are royalty-free for selling beats and placements with fewer than 1,000,000 streams. For major placements, Lemonaide provides an easy clearing process to ensure your projects remain hassle-free.
Lemonaide is certified by Fairly Trained, a non-profit initiative certifying companies that use ethically sourced training material in their AI model datasets. This certification aims to protect artists from unauthorized use of their work, addressing concerns about AI-generated content’s origins and its impact on human creativity
This model incentivizes content creators by allowing them to generate income from their creative work while maintaining clear boundaries for when licensing terms come into play. It’s a form of ensuring creators are compensated if the AI-generated content is commercially successful.To learn more about this topic, check out the MIDI.ORG article on ethical AI MIDI software.
Built-in virtual instruments and the DAW bridge
Lemonaide’s original product includes a handful of built-in virtual instruments including space pads, electric keys, pain piano, billboard piano, and synth strings. You can audition MIDI seeds with any of those instruments before dragging them into your DAW. They also provide a DAW bridge to enable playback with virtual instruments from your personal collection.
Their latest product, Spawn, includes hundreds of curated instrument presets designed to work together seamlessly. Here’s a quick summary of what they offer:
- Bass: Deep sub-bass, mid-bass, and plucked basslines for rhythmic foundation.
- Keys & Piano: Versatile piano, electric keys, and organ sounds for harmonic richness.
- Synth: Synth keys, leads, and pads for modern, dynamic soundscapes.
- Strings & Mallet: Lush string layers, percussive mallet sounds, and steel drums for unique textures.
- Brass & Woodwinds: Bold brass, airy flutes, and shimmering bells for melodic accents.
- Guitar & Pluck: Acoustic and electric guitar tones, along with sharp plucks for rhythmic melodies.
- Soundscapes: Atmospheric and ambient layers to create depth and atmosphere in your tracks.
Spawn’s prompt interface includes a variety of sonic qualities and effect presets as well. Choose from descriptive properties like aggressive, airy, ambient, analog, bright, clean, complex, deep, dirty, distorted, dry, evolving, ethnic, filtered, harsh, huge, lush, processed, punch, simple, spacey, sub, underwater, vinyl, and wobble.
Those prompts guide the MIDI generation, but your control over the music doesn’t end there. Spawn includes additional effect layers like reverb, delay, chorus, distortion, and flanger. Granular control over generative music is precisely what’s been missing from other state of the art text-to-music generators like Suno and Udio.
An interview with the Lemonaide Team
What inspires a group of independent musicians and software developers to go all in on an AI MIDI product like this? I wanted to understand their greatest challenges as well as their biggest wins. So we interviewed their co-founders Michael Jacobs and Anirudh Mani along with Senior Research Scientist Julian Lenz to learn more.
Ezra: What inspired you to start an AI MIDI company?
MJ: It actually all started in my career as a rapper. I fell in love with creating music at age 11 (a lot of my musical inspiration was created out of a lot of Trauma I dealt with as a kid). I uploaded several music videos to YouTube which caught pretty solid steam back in the day.
After spending countless hours making music, I also decided to get into technology out of the goal of simply helping my family escape financial poverty. I ended up going to college for Technology, and spent 5 years at Google learning more about Cloud Computing and AI.
After learning the impact / potential AI has, I decided it would be awesome to create a Hip-Hop EP that was Co-Produced by AI. And from there, the inspiration continued to snowball into realizing, it would be awesome to make helpful tools for musicians using the unique inspirational value AI can provide.
Ani: As MJ was playing with Magenta and other tools, and building our initial offering of “Lemonaid”, I was a Research Scientist at Amazon’s Alexa group working on speech audio related research problems during the day, and experimenting with AI MIDI models for music at night as a very serious hobby, primarily to build something interesting for my own music.
When MJ and I crossed paths, it was serendipitous. Personally, I never thought I’d start a company, but I realized that co-founding “Lemonaide” was the best way for me to express my passion and skills for pushing AI research forward when applied to music, something I also went to Grad school for at Carnegie Mellon.
Growing up in a household obsessed with Hindustani Classical music in India, and learning piano and production at a very early stage, I see myself as an artist first, and a researcher second. I believe this instilled and solidified in me the ethical principles that we now practice at Lemonaide everyday – always building keeping the Artist in the center.
Ezra: What have been some of the greatest challenges you’ve faced so far?
MJ: It always starts with the training data. Using Pre-trained MIDI models only got us so far, but we very quickly realized in order to build truly meaningful algorithms, we would need to ethically source high quality MIDI from real human musicians that care about their craft, in order for our AI models to generate things that seem truly useful to the musician.
Outside of the training data, it also has to do with building custom MIDI algorithms that have the ability to learn the differences and patterns within the training data that make the music what it is. These are things like truly capturing velocity, strumming, offset timing – the list goes on, this work is detailed in this paper we published this past year.
Julian: The single biggest challenge I see is understanding exactly how people would like to interact with ML MIDI systems. The ‘old’ system is, “here’s 20 pre-made MIDI files, now go make this into a song”. Deep learning opens up so many new possibilities, and we believe that most of them in the MIDI realm haven’t been explored yet.
From a birds-eye view, we see from the rise of LLM chatbots that people love interactive systems that can be personalized to their exact task and creative/professional style. So, what is the MIDI version of that? This challenge is both technical and creative; and I think there is an opportunity to really redefine how people interact with MIDI in the future.
Another more practical challenge is that of data quantity. We are really proud of being Fairly Trained, which means every piece of our training data is legally cleared. But from the ML side, this of course means that we are working with datasets much smaller than a typical modern AI company.
To put it bluntly, I don’t think companies like OpenAI, Suno or Anthropic could make their type of models if they had to account for all of the data. So this puts a really fun challenge on the deep learning side, where we have to use every trick in the bag since we can’t just rely on scale.
Finally, there is an open challenge of getting models that know just how to break the ‘right’ rules, musically speaking. Most MIDI models, from Magenta days up until more recent state of the art versions, are pretty diatonic and well-behaved. Of course you can under-train them, or push the temperature, so they just get really weird outputs. But musically speaking, there is that beautiful gray zone where just a few rules are broken – the place where musicians like Stravinsky, Frank Zappa and Thelonius Monk thrive. It’s a huge challenge but I think we are on the right path.
Ani: One of the earliest challenges we were facing was difficulty in striking the balance between a truly generalizable MIDI model versus a musically interesting MIDI model, as we had limited MIDI data. We took an ensemble of models approach to provide a rounded experience for our user during inference, and in parallel continued to collect ethically sourced MIDI data directly from some amazing artists, and were able to overcome this hurdle pretty soon after.
At some point in the last year we also realized that there was a need to increase the overall quality of our MIDI output by capturing more expressive details, which are especially important for a genre like hiphop where the swing matters a lot.
This led to our research led by Julian on introducing a new MIDI tokenization technique called PerTok which captures such granular details while reducing sequence length up to 59% and vocabulary size up to 95% for polyphonic, monophonic and rhythmic tasks.
Our paper (https://arxiv.org/abs/2410.02060) was also published at ISMIR this year, and this research work is integral to the quality of outputs that our users love from our products Seeds, Collab Club and Spawn.
Ezra: What’s the most rewarding part of running a MIDI company?
MJ: One of the coolest things we are so proud of is the Collab Club. Being able to partner with Grammy Nominated, Billboard producers, meet with them on a weekly basis for over a 6-month period – collect their data, train algorithms with their feedback, define a monthly revenue share business model, and then deploy that to consumers who are looking for inspirational tooling. This is by far one of my favorite videos of one of our artists using their own model and highlighting the journey.
Ani: Lemonaide is an AI company and MIDI is our first love. ‘Controllability’ in AI modeling for music is a widely discussed topic and we believe MIDI modeling is going to be a key part of that conversation moving forward.
As MJ mentioned, everyday we cross paths with people that we adore and look up to as artists ourselves, and to be able to build something for other artists and help them is the most rewarding feeling ever.
Collab Club is one such example, where we built AI MIDI models with artists in their style, and now they are the ones who get the biggest share of earnings from these models. Lemonaide will continue to grow and evolve, but something that remains a constant for us is safeguarding the interests of the Artist while navigating this new uncertain world.
Community and Support
Lemonaide fosters a thriving community of producers and artists through its Discord channel and blog resources, offering tutorials, insights, and a space for collaboration. Whether you’re troubleshooting or sharing your latest creation, the Lemonaide community is there to support you.
Check out the Lemonaide and Spawn websites to learn more.
This article was written by Ezra Sandzer-Bell, founder at AudioCipher Technologies.