Google’s Music Transformer- State of the Art in Music AI?

Google has been doing amazing work in music AI and recently they posted demos created by their Music Transformer. The goal was to generate longer pieces of music that had more coherence because the model was using relative attention. 

We found that by using relative attention, which explicitly modulates attention based on how far apart two tokens are, the model is able to focus more on relational features. Relative self-attention also allows the model to generalize beyond the length of the training examples, which is not possible with the original Transformer model.

by  Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu and Douglas Eck.

 The following three examples were created by Music Transformer, an attention-based neural network.  We won’t even get into the question of who owns the copyright to these pieces of music because it makes our head hurt. Remember all of this comes from the neural networks being trained by MIDI files from the e-competition recorded on Yamaha Disklaviers. 

To explain how this relative attention works Google created a video displaying the relative attention as “arcs showing which notes in the past are informing the future.” 

There are other possibilities for Music Transformer.   Here are two versions  of Twinkle Twinkle Little Star. 

Here we trained a Music Transformer model to map heuristically-extracted melody to performance, and then asked it to play the Twinkle Twinkle Little Star melody (with chords unspecified):

 by Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu and Douglas Eck.

In this next case, the AI model was given the chords to Hotel California.  You can see the core technology has tons of potential for helping musicians to be creative in the future.   Artificial Intelligence will soon be another tool in our creative palette. 

 For more technical details you can read the actual paper or go to the original blog post.