Want to make music with your friends on Internet. This tutorial from Andrew Bales will show you how to build an app that allows you to collaborate with friends to make music in real-time!
Try out a live demo: https://midi-collaboration.herokuapp.com/
Check out the code: https://github.com/agbales/web-midi-collaboration
by Andrew Bales
As you can see, playing a note lights up a key in pink and displays the input data in the list. If another user joins the session, their input will light up blue and their data will appear in the list as blue entries.
We’ll break the process into 5 parts:
- Creating an App with Express.js
- Connecting a Midi Controller
- Adding Socket.io for Collaboration
- Styling
- Deploying to Heroku
Step 1: Creating an Express.js App
We’ll get started by making a directory and initializing the project with these three terminal commands:
mkdir midi-collaboration
cd midi-collaboration
npm initThe NPM utility will ask for a bit of information to set up the package.json file. Feel free to provide that info or just hit enter to leave these fields blank for now.
Next, add Express:
npm install express --saveIn package.json, you will now see Express included as a dependency.
In the root folder, make server.js with:
touch server.js
In server.js, add the following:
var express = require('express');
var app = express();
const port = process.env.PORT || 8080;
var server = app.listen(port);
app.use(express.static('public'));
This creates an instance of Express and sets the port that defaults to 8080. app.use instructs express to serve up files from the ‘public’ folder. If you ran the server now, you’d get an error. That’s because we don’t yet have a public folder!
Public folder
Let’s make that public folder:
mkdir public
cd public
Inside, we’ll make index.html, index.js, and a CSS folder containing style.css:
touch index.html
touch index.js
mkdir css
cd css
touch style.css
The ‘public’ folder should look like this:
public
--> index.html
--> index.js
css
--> style.css
Index.html should link to style.css and index.js. So let’s add the following:
<!-- /public/index.html -->
<html>
<head>
<title>Midi Collaboration</title>
<link rel="stylesheet" href="/css/style.css">
</head>
<body>
<div>
<h1>MIDI Collaboration</h1>
<ul id="midi-data">
</ul>
</div>
</body>
<script src="/index.js" type="text/javascript"></script>
</html>
This makes a simple header followed by an empty unordered list — this is where we’ll log our midi data. After the body, it references index.js. For now, let’s add a simple console.log to index.js so that we’re sure it’s working properly:
// public/index.js
console.log('index.js is connected!');
Update package.json
Finally, we want to update package.json so that we can run our server with the terminal command ‘npm start’. Add the following line to “scripts”:
"start": "node server.js"
Your package.json should look like this:
{
"name": "remaking-midi-maker",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node server.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"express": "^4.15.3"
}
Make sure you’re in the midi-collaboration folder and start up your app:
npm start
Great! It’s now hosted at http://localhost:8080/
Step 2: Connecting your Midi Controller
We’ll use the Web Midi API to connect a USB midi controller to the app. If you’re seeing ‘index.js is connected!’ in your console, let’s replace the console.log with:
// public/javascript/index.js
var context = new AudioContext();
var oscillators = {};
var midi, data;
if (navigator.requestMIDIAccess) {
navigator.requestMIDIAccess({
sysex: false
}).then(onMIDISuccess, onMIDIFailure);
} else {
console.warn("No MIDI support in your browser");
}
function onMIDISuccess(midiData) {
console.log(midiData);
midi = midiData;
var allInputs = midi.inputs.values();
for (var input = allInputs.next(); input && !input.done; input = allInputs.next()) {
input.value.onmidimessage = onMIDImessage;
}
}
function onMIDIFailure() {
console.warn("Not finding a MIDI controller");
}
If your midi controller is hooked up, you should see the console.log for the MIDIAccess object:
In onMIDISuccess(), a for loop listens for midi messages. Right now it should be causing an error in the console. Why? Because we haven’t defined what to do when it receives a midi message.
Let’s create the onMIDImessage function referenced in the loop:
function onMIDImessage(messageData) {
var newItem = document.createElement('li');
newItem.appendChild(document.createTextNode(messageData.data));
newItem.className = 'user-midi';
document.getElementById('midi-data').prepend(newItem);
This function creates a new <li> element. It appends a text node with our midi data. It adds a css class of user-midi (this will be important later). Finally, it adds that new list item to the unordered list with the id “midi-data”.
Pretty cool, eh?
But what do these numbers mean? Also: where’s the sound?
MIDI Protocol
MIDI Protocol is a rabbit hole all its own, but for our purposes you can understand the numbers with a simple chart:
On / Off → 144 / 128
Pitch → 0–127
Velocity → 0–127
When working with this data, we’ll treat the first number like an on/off switch. 144 = on, 128 = off. The second number is the range of pitches. The final velocity input could also be understood as volume in our usage here.
If you’d like a more in-depth look at midi, here’s a good place to start.
Sound
The MIDI data is not the sound, but a set of directions: ON/OFF, PITCH, VELOCITY. We’ll need to make our own synth that can turn this information into musical tones.
First, let’s convert those value sets from an array into a ‘note’ object that we can pass to our sound player. In onMIDImessage function, add:
var d = messageData.data; // Example: [144, 60, 100]
var note = {
on: d[0],
pitch: d[1],
velocity: d[2]
}
play(note);
Above, the variable ‘d’ is assigned the incoming data: an array of three numbers. Those three values are accessed by their index and assigned as values for the object properties. The note object is then passed to the play function. Let’s write that function:
function play(note){
switch(note.on) {
case 144:
noteOn(frequency(note.pitch), note.velocity);
break;
case 128:
noteOff(frequency(note.pitch), note.velocity);
break;
}
function frequency(note) {
return Math.pow(2, ((note - 69) / 12)) * 440;
}
function noteOn(frequency, velocity) {
var osc = oscillators[frequency] = context.createOscillator();
osc.type = 'sawtooth';
osc.frequency.value = frequency;
osc.connect(context.destination);
osc.start(context.currentTime);
}
function noteOff(frequency, velocity) {
oscillators[frequency].stop(context.currentTime);
oscillators[frequency].disconnect();
}
}
This function checks to see if the note is on (144) or off (128) and triggers the appropriate command. Both noteOn and noteOff reference two global variables that we established at the top of index.js to handle the starting and stopping of sound.
The frequency function is used to convert the midi note number into a hertz frequency. If you’re curious, you can read more about it here.
Your app should now play like a synth. Huzzah!
Step 3: Adding Socket.io for Collaboration
Now it’s time for the fun stuff: syncing multiple users in a session! Socket.io will help us do that by enabling ‘real-time bidirectional event-based communication.’ That means we can send and receive midi messages as they’re triggered on any browser as they occur. Add Socket.io to the project:
npm install socket.io --save
You’ll also need to add this inside the head of index.html:
<!-- public/index.html -->
<script src="/https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.0.3/socket.io.js"></script>
Note: I’m using version 2.0.3 — be sure your versions of Socket.io match your package.json and html script. You might want this link to the CDN.
For Socket.io to connect users we need to assure that:
1) Users can both send and receive notes
2) The server listens for user input and broadcasts it to other users
User send/receive
First, let’s make sure the user emits a signal every time they play a note. This can be accomplished by opening index.js and adding a single line to the function onMIDImessage. Be sure to add this after the note object has been defined.
// public/index.js (inside onMIDImessage)
socket.emit('midi', note);
We also want to play any external notes that come in from other users. Inside index.js, add the following:
// public/index.js
var socket = io();
socket.on('externalMidi', gotExternalMidiMessage);
function gotExternalMidiMessage(data) {
var newItem = document.createElement('li');
newItem.appendChild(document.createTextNode('Note: ' + data.pitch + ' Velocity: ' + data.velocity));
newItem.className = "external-midi";
document.getElementById('midi-data').prepend(newItem);
playNote(data);
}
When we receive an ‘externalMidi’ message from the server, it triggers gotExternalMidiMessage. This function should look familiar — in fact, we could refactor this later, but for now we’ll repeat code for clarity. It displays the external note in the view in a manner that’s almost identical to how we treat midi input from our own keyboard. However, we’ve given the <li> a class name ‘external-midi’. This will be important in a moment when we add styles to differentiate between our midi input and that of outside users.
Finally, the note is passed to the player to trigger a sound.
Server
Now let’s make a bridge between users. We want to handle any incoming signals and pass them to any other sessions.
In server.js, require Socket.io and add the function newConnection. This will be triggered when the server gains a new connection.
// server.js
var socket = require('socket.io');
var io = socket(server);
io.sockets.on('connection', newConnection);
function newConnection(socket) {
console.log('new connection from:' + socket.id);
socket.on('midi', midiMsg);
function midiMsg(data) {
socket.broadcast.emit('externalMidi', data);
}
}
The newConnection console.log will appear in the terminal every time a new connection is made. We also have socket.on listening for messages from any user and then triggering midiMsg, which broadcasts that data to every user except the original user.
With that, we’re all set!
Step 3: Styling
If you’re just interested in seeing your notes differentiated from other players, you can take a shortcut here and simply add these classes to your style.css:
// public/css/sytle.css
.user-midi {
color: green;
}
.external-midi {
color: blue;
}
That’s it! Now you will see your own input in green and any external signal in blue. Feel free to skip to the next step and deploy your app to Heroku!
If you’d like to create a keyboard interface, let’s keep rolling:
Build a keyboard
This keyboard we’ll make builds off of @baileyparker’s CodePen project. We’ll make some design and functionality changes to fit our usage. To start, open up this pen in a separate window:
HTML
The div with the class ‘piano’ houses all of our keys, which are labeled with a variety of selectors. Follow the html structure in the pen and paste the piano div and all its keys into your document (/public/index.html).
CSS
This is a big update for our styles. We’re importing a google font, assigning a background image to the body, and finally giving colors to differentiate between .user-midi (pink) and .external-midi (blue) signals. The ul and li elements have been styled so that they’ll slant back at a pleasing angle.
The keyboard styling takes up the remainder of the CSS. Worth noting here are the ‘active’ classes like ‘.piano-key-natural:active’ or ‘.piano-key-natural-external:active’. These are triggered by the Javascript when a note is played. If it matches the data number, the CSS will activate that key to be pink for your notes and blue for any external input.
When you copy the CSS from the pen into your project’s style sheet (/public/style.css), be sure to follow the notes included inside. Most importantly, you’ll need to update the path to the background.
JS
You’ll do the same thing for the Javascript: cut and paste it into /public/index.js below the code we’ve written. Again, it is important to read and follow the few comments within the code.
This code will manage midi controller connections. But you’ll want to focus in on two functions: onMidiMessage & updateKeyboard. The former handles local input and applies the appropriate class to light up the keys. The latter does the same thing (but with a different class) for external messages.
To get external notes to light up the keyboard, we need to return to the function gotExternalMidiMessage. After we call play(), add the following code:
var msg = { }
msg.data = [];
msg.data.push(data.on);
msg.data.push(data.pitch);
msg.data.push(data.velocity);
updateKeyboard(data);
The keyboard needs a specific kind of object data structure to update, so we create msg and fill it with data from our note. That msg object is passed to updateKeyboard which lights up the keys in blue for that external signal.
You should now see your keys light up pink for your own input! When we connect to Heroku add more users, you’ll see those external midi messages light up in blue!
Step 4: Deploy to Heroku
Instead of walking through each step here, I’ll direct you towards the comprehensive documentation at Heroku. They’ve done a great job reviewing options for deployment:
I’m excited about this collaborative app. It has been a real thrill hooking it up and inviting friends from other states to log on and play music! Looking back, I also see ways that the data could be passed more efficiently and areas where the design could be enhanced — it’s not friendly to smaller screens, for instance.
I’d love to hear how you’re using the app, and I hope it has given you a better understanding of how to combine Express.js, Socket.io, and the Web MIDI API!