I have been working on a project for a while. I am converting Super Nintendo (SNES) SPC audio files in to MIDI files, mainly using a program called spc2midi 0.023j
It creates the MIDI files by interpreting the input to the SNES sound chip (SPC700) in to MIDI data. An SPC file is a rip of a specific game's sound code and samples packaged to be playable in an audio player. This is not a direct 1:1 MIDI conversion, as the SPC700 uses custom sound programs per game, so spc2midi has to do some wizardry to get decent results, like running a FFT transform on the audio samples in an SFC file to make a best guess at the correct pitch for instruments. It also does its best to determine if a sound should be a melodic instrument (more than one pitch in the original file) or percussion (single, repeated pitch). Fortunately, many other things do generally match up, like volume control, attack, decay, and, perhaps very helpfully, the audio is placed on a maximum of 8 channels.
The process is not perfect, and the conversion software is old and hasn't been updated since 2004. I have to load each game audio file individually, make notes of which memory addresses each file uses for instruments, then cross-reference with the actual SPC file in my player of choice (Winamp, Timbres Of Heaven soundfont) to try to find a MIDI instrument that best matches up with the sound from the game. I have a spreadsheet of most of the games I wish to convert, with 7/13 complete for data. Some games the instruments need the pitch manually adjusted per track (one track the pitch might be fine, another might require a drop of 60 semitones). A few games I have a tossup between which instrument to use for a particular address. It has been a very time-consuming process. Each SNES instrument needs to be properly selected as MIDI instrument or percussion (not always correctly auto-determined), its attack and decay enabled, actual MIDI instrument assigned, pitch and/or volume set (if required), and the song length needs to be set for the export. Nothing can be saved as a preset for the next file, so it all has to be manually redone every time.
Now to the actual question:
** Recommendations on what I can do to clean up these files, and make them more presentable in a MIDI editor?
My current process will be as follows:
0 - Figure out all of the instrumentation
1 - Export MIDI file using spc2midi
2 - Repair MIDI file using Anvil Studio's repair option
This removes some unnecessary note off events, fixes some out of range values, and a few other things
3 - Delete all extra unused channels from the MIDI file in Anvil Studio
And that's what I have at the moment. The resulting files sound good, but they are visually not ideal for editing, as they do not line up to normal measures on a piano roll in an editor. I tried using Anvil Studio's "quantize" option, but it didn't seem to make a difference. There was a comment when I posted a work-in-progress file in a previous thread about an excessive amount of pitch bending, but that is inherent to the way that particular game sound engine was programmed, and may be a common "feature" of these files when they are finished.
This process also kicked off my desire to write my MIDI instrument changer, quattj's MIDI-MIS (which I started working on in March, and still have some work to do on, since I went back to work full time over the summer so it fell by the wayside. And my actual ORIGINAL intent was to write my OWN spc to MIDI utility, but then I learned what I didn't know :p ). I will be using my instrument changer to make further instrument changes on the "finished" files, as spc2midi can only assign GM instruments, and I have found I can get better results using GS or XG instruments. For example, many SPC tracks use a Euro Hit sound, which is only available on higher bank numbers in place of the standard Orchestra Hit, and some games will sound a lot better using alternate drum kits. (This will require me to finish my GS/XG SYSEX Reset insertion code)
I'm including a screenshot of spc2midi English translation (I'm using the Japanese version because it is newer) and my spreadsheet so you can see some of the process. The numbers on the left are the memory addresses of the instruments. The numbers in the grid are manual pitch settings. I'll also follow up with a set of MIDI files if I finish one of the games any time soon. I've included the first track from the game from the spreadsheet so you can see the current "end result" (both SPC and MIDI). When all is said and done, I plan on creating mp3s from the finished files and uploading them to YouTube, plus uploading the actual MIDI files (and maybe spreadsheet) on my Google Drive for other people to use/modify/enjoy.
If you are interested to see the inner workings of an SPC file, this player is very nice/accurate, and has a bunch of different display modes that can show you pretty much everything real-time, and you can visually see while it's playing how things can be interpreted in to MIDI.
https://github.com/dgrfactory/spcplay
with a huge archive of SPC files at
https://www.zophar.net/music/nintendo-snes-spc
And if you want to get Super in to it (see what I did there??)
spc file specs
http://snesmusic.org/files/spc_file_format.txt
and SPC700 specs and reference
https://en.wikibooks.org/wiki/Super_NES_Programming/Loading_SPC700_programs
https://wiki.superfamicom.org/spc700-reference
Ahh, and the "Tempo" in the spc2midi screenshot is actually the pitch.
And I would like to know how I can add a fade to the end of each MIDI file, as that option was never implemented in the converter, and most songs should fade out at the end.
Hmm, and it seems there is still an issue I need to work around for a few of the tracks (songs).
Some things have volume too low (track 7a) so pieces of the music are missing (tuba) because the volume ends up becoming zero. To compensate, I raised the volume of all the instruments to 70 in the converter, which until a short while ago seemed to be a good solution. I am finding now, however, that doing so loses many volume differences that SHOULD be present, because all of the volume levels now get pegged at max instead of having variation. I tried backing it off, but even back down at 30 some instruments are maxing out and being squashed, so losing dynamic range, and any lower and I lose that one instrument again. It appears to be a problem with the emulation that occurs in the converter. The missing tuba sound should be mostly center-balanced in the stereo signal, but only the left side makes noise on part of it, at maybe half volume of the other tuba part that plays properly.
Is there some way I can use an editor (and recommendations) on fixing this tuba part? I'm guessing I'd need to isolate the instrument somehow and boost the volume and pan the sound more to center only on that part.
I can mute all of the other instruments in the converter and have a MIDI file that only plays the tuba, but combining it back in to the other file is not something I think I am currently skilled enough to be able to do, unless perhaps I put it on a new track in the MIDI file?
Hi Jason, I'll be interested to hear how much success you have converting SPC to MIDI. That is a hero's journey. 😀
It seems your questions are mostly about finding a powerful MIDI file editor, which is strangely hard to come by.
If you are on Windows, I recommend Domino or Sekaiju. These are old Japanese programs from the DTM era, but with some modern English translations.
Here is Domino in English:
https://github.com/Hans5958/Domino-English-Translation/releases
This program takes some getting used to, but it is very powerful with MIDI. You'll have access to the raw MIDI event list and can perform batch operations.
For getting the notes to line up to a note grid, your best bet is to just guess and check by doing a very precise Time Stretch (see attachment) until notes line up, and then do a quantize.
As for the fade out... this should be as simple as automating the Channel Volume (CC 07).
[quotePost id=13662]Hi Jason, I'll be interested to hear how much success you have converting SPC to MIDI. That is a hero's journey. 😀
[/quotePost]
Haha, don't I know it :p
I currently have the instrumentation selection for 14 games 90-100% complete, and working on another 5, plus others I haven't started yet.
I created full MIDI file sets for 3 games so far (Axelay, Battletoads & Double Dragon, and Prince Of Persia), plus a handful of random ones here and there that were either too short to reliably "decipher" in the converter, or ones I wanted to listen to more extensively. (The only playback options it has are PLAY and PAUSE, so if the track doesn't loop, the only way to hear it "some more" is to save it or reload the file, and reloading resets all instruments to piano and bass drum. Boooo.). I haven't done any cleanup on them yet. I may still tweak the instrument choice a bit after I listen to them some more, and might have to re-do a few where I messed up the song length (generally trying to get in 2 loops of the song plus a 10 second fade, where appropriate).
A few things I've noticed as I've been going through all the tracks:
- Some games, mostly older/first gen SNES games have music that was most likely composed using MIDI and then converted to run in that particular game. I can tell by listening to the percussion, where certain very MIDI-specific things are present, such as a Low and High (Short and Long) Whistle. Super Mario Kart is a good example of this.
- Some games clearly were not scored using MIDI. There are a few metallic melodic percussion sounds that I am having difficulty replicating, particularly melodic snares and melodic cymbals. Currently many of the melodic snare sounds are being semi-replicated using Gunshot, usually pitch-shifted.
- Many newer games use memory swapping to replace sounds between tracks, so a memory address that might point to a trumpet in one track could be an organ in another.
- Some games use sound effects as part of the music, and those are generally difficult to replicate. One that I am actually quite surprised I was able to have sound fairly accurate is some of the higher pitched monkey sounds in the music for Donkey Kong Country. I believe it is the shamisen, along with whatever they did with note manipulation for the monkey sound. Donkey Kong Country, by the way, the most complicated one I have done, with 103 total memory addresses for instruments, and some of those having 2, 3, 4, 5, 6, 7, or 10 variations.
- I was hoping I'd be able to choose an instrument set and then just be able to use it for each track from a game, but usually at least one or more tracks (but not all) will require changing the pitch of at least one instrument for it to be correct. A lot of times it's a single octave (+12 or -12) but there are also a lot of -19,-24, -66, and random numbers of semi-tones, like -7, +5, and... err... -106. That's nearly 9 octaves lower than the pitch it auto-assigns, and lemme tell you, a tune being played 9 octaves too high ain't great!
- A few games I will definitely need to use some GS sounds, as the standard GM banks just can't give me satisfactory results. I may also need to use a few different soundfonts, as Timbres Of Heaven has a few sounds that are too harsh for certain soundtracks. My biggest "missing sound" for now if the EuroHit, which is a GS sound. OrchstralHit just can't cut it, a LOT. At least my MIDI instrument switcher (quattj's MIDIMIS) can change the GM reset that spc2midi puts in the file in to a GS reset.
spc2midi has opened every file I have tried. Super Turrican is giving me some weird volume issues because of how they made the instrument sounds in the game. If I enable the attack and decay, the main instruments all lose about 60% of their volume. However, VGMTrans will not open any file from this game so I can not compare the two version of midi files created from it.
I've decided that I will be creating mp3s of the final MIDI files and uploading soundtracks on YouTube. I also plan on having the MIDI file collections available for download for anyone who wants them. (I seem to have already mentioned this in the original post :p) I'll probably have a GM version of all of them, plus a better sounding GS version for those games that need it. I'm going to call it "The Great SNES MIDI Project", or similar.
I do already have Seikaiju, though I have thus far only used it for event viewing. I'll check out Domino. Looks promising. Anvil Studio has some cool features, but never seems to really be able to do what I actually need. I may request help from people in the future once I've got all of my MIDI files created.
I'm attaching those three aforementioned complete MIDI file sets, in case you're interested, plus other random files I have saved recently.
You can listen to some of the "challenges" towards the end of the Credits tracks in Axelay. It is a non-looping track, with only a single instance of one of the instruments, which is a "spaceship blasting off sound". The only way I could test variations was to save it and hope for the best. The first it just comes across as a "click". The second the pitch is wayyyyy too high. The third is as close as I can get it using GM.
I have one GM/GS test in the Super R-Type folder. The GM version uses Orchestral Hit. The GS version uses Euro Hit, as that is what it needs to be.
All SNES music makes extensive use of pitch bend, so your playback will need to support pitch bend (portamento) or it will sound awful.
For reference, I am playing back (and making all of my instrument selections) using Coolsoft VirtualMIDISynth with the Timbres Of Heaven SF2 3.94 soundfont.
You can compare to the original SNES music here (track names vary as I am using a different game dump to make most of them):
https://www.zophar.net/music/nintendo-snes-spc/axelay
https://www.zophar.net/music/nintendo-snes-spc/battletoads-double-dragon-the-ultimate-team
https://www.zophar.net/music/nintendo-snes-spc/prince-of-persia
plus the other random ones you can search for the specific game on
https://www.zophar.net/music/nintendo-snes-spc
(Oops, file is too big, splitting!)
Part 2 of split file
- Some games, mostly older/first gen SNES games have music that was most likely composed using MIDI and then converted to run in that particular game. I can tell by listening to the percussion, where certain very MIDI-specific things are present, such as a Low and High (Short and Long) Whistle. Super Mario Kart is a good example of this.
Except... I just found out the whistles are combined in to a single instrument, so I will have to painfully go through and re-assign 50% of the whistles in the MIDI file if I use the normal percussion whistle. *sigh*
I gave Domino a quick whirl to test it out. Still a lot to figure out with it, but it seems to have a fair amount of features that should be very useful for me. I particularly like the event list with corresponding parameter graphs that you can show on the bottom to easily adjust things like velocity, just by drawing a line where the notes are. I also found that you can easily switch an instrument track to percussion and vice versa, which is cool for some of the things I am doing.
[quotePost id=13678]
All SNES music makes extensive use of pitch bend, so your playback will need to support pitch bend (portamento) or it will sound awful.
Jason, just a point I noticed in the above post, pitch bend is a controller that changes the pitch of the note up or down, whilst portamento is a feature that blends one note into the next.
Singers often use portamento to blend one note into another.
In MIDI implementation the effect is often used to cut off the attack portion of the waveform from the notes following the first.
Make sense?
JohnG.
Yes, perfect sense. I swear I knew that :p Now I'll have to check if it's actually using both, or just pitch bend. I was looking at a bunch of sound engine specs the other day that showed exactly what parameters they allowed, at least in the particular games that used said engines.
EDIT:
Looking at the specs didn't help :p
In the particular engine N-SPC, they have three commands for variable pitch.
Pitch envelope to
"do pitch slide for all of the following notes" starting on normal and ending on new pitch
Pitch envelope from
"do pitch slide for all of the following notes" starting on new pitch and ending on normal
Pitch Slide
"do pitch slide for current note only" starting on normal and ending on new pitch
So it's probably up to the converter to decide how to handle it, which means I am now loading up one of the MIDI files to see what's in there.
I see a lot of pitch bend, no CC 5, 65, or 84, so no portamento in the resulting MIDI files.
[quotePost id=13784] the effect is often used to cut off the attack portion of the waveform from the notes following the first.
[/quotePost]
Hmmmmm, does it work on percussion channels as well? In my post about playing a partial sound, that would actually be the exact effect I need to get rid of the sharp hit on subsequent repeated snare hits... maybe.
[quotePost id=13786]
Hmmmmm, does it work on percussion channels as well? In my post about playing a partial sound, that would actually be the exact effect I need to get rid of the sharp hit on subsequent repeated snare hits... maybe. [/quotePost]
It's not the channel that's the key, portamento can be applied on any channel whether it's being used for percussion or not.
It's the sample player that responds to the portamento on command and starts playback after the attack phase (or so I'm led to believe).
Aye aye aye. So the converter leaves all percussion expression on the original channel it was on in the game (somewhere on 1 to 8). I haven't noticed too many issues with that (but maybe I just didn't pay enough attention? Could fix some issues I am having with the game I just finished mapping out instruments for). I'm testing out converting a game now where there's lots of volume control on the percussion, and without it it just sound like a big pile of noise. Luckily with this particular game (this testing music track anyway) the percussion is all on channels 2 and 5. I tried several different editors to see if I could successfully move the percussion events on to the percussion channel.
Domino broke other things unrelated to it (seems to have lost pitch bending, maybe some other stuff, but the percussion sounds good).
Anvil Studio couldn't do it (except for repairing the track to prepare for use in the other editors).
Sekaiju... did it! After a lot of trial and error, I figured out:
- first I have to start with a file repaired by Anvil Studio, otherwise some invalid data that the converter creates causes Sekaiju to not save the percussion channel after the edits are made, even though it will play everything fine up until you save
- click on a MIDI track that contains the percussion expression events
- go to the edit menu
- Edit Event's Track
- Enter the actual percussion channel's track number
- Viola! And not the musical kind!
Here are 3 tests so you may listen to the difference. I must say, I'm not looking forward to this part of the process, and it may be next to impossible for some games where percussion is mixed together with melodic instruments rather than having it on its own fixed channels, but I'm glad I found a way to do it that will work at least sometimes.
The 3 files are the original conversion, the Domino effect, and the Sekaiju version.
Original SPC version can be heard at
https://www.zophar.net/music/nintendo-snes-spc/super-mario-world-2-yoshis-island
107 Flower Garden
My current process will be as follows:
0 - Figure out all of the instrumentation
1 - Export MIDI file using spc2midi
2 - Repair MIDI file using Anvil Studio's repair option
This removes some unnecessary note off events, fixes some out of range values, and a few other things
3 - Delete all extra unused channels from the MIDI file in Anvil Studio
Figured I'd give an update as I am progressing. As usual, the more I know, the harder things get.
The converter does a pretty bad job with percussion. Running a converted file through Sema's test-midi-files shows invalid (255) values whenever there should be a Note Off for percussion. While investigating the cause, I discovered that the ONLY thing it moves to the percussion channel is the Note On event. All other items (note off, velocity, etc) are all left on their original channels. I have been trying to find workarounds for it for a while now. I added custom code in to my MIDI editor to try to automatically re-assign the Note Off events, but it is not really accurate. I can only set the Note Off to the same value as the most recent Note On, so some things get multiple offs while others get none.
I just found a very slow, painful, but ultimately quality solution!
All standard melodic instruments are converted correctly, including the full ADSR envelopes. So I thought "What if I mute all percussion, and export just a MIDI file of the melodic portion?" So far, so good. "Okay, what if I mute all of the melodic portion, and export just the percussion? But! ... Set each percussion sound to a unique melodic sound instead, and change it later?" So I just tried it out. It's very time consuming. But here's the new "current process" (with bonus of no longer needing to run the Anvil Studio repair because there are no longer any invalid percussion values, because as far as the converter is concerned, there is no percussion!)
0 - Figure out all of the instrumentation/pitch/etc
1 - Export melodic only MIDI file using spc2midi
2 - Export percussion (*set as melodic) only MIDI file using spc2midi
3 - Modify percussion MIDI file using Sekaiju (so far, unless I find a better option) to re-map all sounds on to percussion channel
4 - Adjust key of each "instrument" (using Sekaiju) to be on its correct percussion sound
5 - Load melodic only MIDI file in to Anvil Studio
6 - Merge the modified now-fully-percussion MIDI file in to the melodic file
7 - Delete all extra unused channels from the MIDI file in Anvil Studio
I've done one track so far, and it came out loads better than any attempt prior to now. I just need to come up with a more streamlined way of keeping track of which percussion instruments are which, as things start to get messy when you move them over from a melodic channel, and I think some of the ones in the track are incorrect (proof of concept, but I can easily change them later with my editor). An added bonus to this method is that for things with "combined" percussion (for example, the high/low whistles in Mario Kart) I can now see both high and low in the note display and reassign the high to high and low to low, where previously I would have had to manually figure out what was what if I didn't just give up and make them all the same.
I'm attaching my first "successful" percussion transplant, plus my previous version of the same track (with the caveat that some percussion sounds may be incorrectly chosen, I just fixed the cowbell :p)