fbpx
Skip to main content

MIDI Forum

Do MIDI files store...
 
Notifications
Clear all

Do MIDI files store GM/GS/XG indicator info? Do banks have common bank #s?

10 Posts
3 Users
0 Reactions
8,694 Views
Jason
Posts: 424
Honorable Member
Topic starter
 

Do MIDI files store any standard indicator to tell if a file should be played or edited using GM, GS, or XG compatible settings? I'm trying to decide if I should make an automated function in my software to check, or if I should leave manual selection up to the user. Also, is there a general consensus on what instruments are mapped to certain banks in each format?

I did notice in my most complex test file (dmed.mid) that it sends a GS reset at the beginning of the file, but I don't want to assume that this will always be the case.

More specifically, I am trying to provide the most accurate instrument name display I can. My midi files have assorted bank selects, and I am not sure of how to go about making sure they have names that match.

For example, most of what I have read says that the majority of midi devices use only the MSB of bank select. However, my files seem to have both MSB and LSB in use.
One file has #25 Acoustic Guitar (nylon), but bank selects of 127/123 and 127/126
Another (the dmed.mid mentioned earlier) has tons of bank selects, with MSBs of 1, 2, 3, 8, 9, 11, 16, 24, and LSBs of mostly 2 and some 1.

I have several .ins instrument name files from openmidiproject that I'd like to incorporate (definition files from Sekaiju midi editor, says compatible with Cakewalk and Sonar instrument definition files) which seem to be fairly complete, but I don't know how to map each of the sections in these files to specific banks.

 
Posted : 27/04/2021 6:38 am
JohnG
Posts: 225
Estimable Member
 

There is no "standard" indicator that I'm aware of but, just as you have noticed, a reset may be included to switch the file player/sound module to the correct mode of operation.
There are the following resets: GM, GM2, GS and XG.
In professional quality XG files, those I'm most familiar with, there's usually a GM reset followed a liitle further down the file with an XG reset, followed by any SysEx commands, then the MSB, LSB, Program voice allocations per channel.
Any help?
JohnG.

 
Posted : 27/04/2021 7:39 am
Jason
Posts: 424
Honorable Member
Topic starter
 

Any help?

A little 😉

I suppose I can see if there is a particular reset, and if not just default to GM naming. Unless there's a semi-reliable way to deduce by which banks are actually used.

 
Posted : 27/04/2021 8:57 am
Jason
Posts: 424
Honorable Member
Topic starter
 

It appears that the Recommended Practices for GM 2 does specify which banks are which for the assorted sound sets.

https://www.midi.org/specifications-old/item/general-midi-2

The pdf for GM Level 2 Specifications lists them all in the appendices. Looks like I'll start with those, then add GS/XG specifics if I can find more info.

It looks like GM2 is recommended on Bank 79H (121)?

And percussion drum sets are selected using a program change on the percussion channel. That explains what I've been wondering about, whether setting a different instrument besides piano on channel 10 would ever have any effect. The answer is, under certain circumstances, yes!

 
Posted : 27/04/2021 10:14 am
Jason
Posts: 424
Honorable Member
Topic starter
 

I am now checking the sysex messages for a GM, GS, or XG reset to choose a mode. If no resets are found, default to GM.
Here are (hopefully) accurate Sysex messages if anyone is interested. Don't 100% quote me on these, this is second hand knowledge culled from ye olde internet. I already had to make a modification to my original check for GS resets because one of the bytes refers to the Roland model ID, so if that is stored in the file, it would not be read correctly using the default GS reset.

[code type=markup]
// GM reset
// F0 7E 7F 09 01 F7 (GM1 hardware)
// F0 05 7E 7F 09 01 F7 (GM1 software)

// F0 7E 7F 09 03 F7 (GM2 hardware)
// F0 05 7E 7F 09 03 F7 (GM2 software)
[/code]

I am assuming (haven't found info yet) that the "10" byte in the XG reset refers to a model ID, similar to Roland, so I am ignoring it. I downloaded a few "official" Yamaha XG midis, and they are properly indicated as being XG files in my program.

[code type=markup]
// Yamaha XG reset
// F0 43 (10) 4C 00 00 7E 00 F7 (hardware)
// F0 08 43 (10) 4C 00 00 7E 00 F7 (software)
[/code]

As above, the "10" here represents the Roland model ID, so I ignore it. My test file in question has a "16" in place of the 10. I have not found a reliable list of Roland model IDs. My files that I know are GS now show up as GS in my program.

[code type=markup]
// Roland GS reset
// F0 41 (10) 42 12 40 00 7F 00 41 F7 (hardware)
// F0 0A 41 (10) 42 12 40 00 7F 00 41 F7 (software)
[/code]

 
Posted : 27/04/2021 1:02 pm
Bavi_H
Posts: 266
Reputable Member
 

SYSEX STORAGE IN A MIDI FILE HAS A LENGTH

Be aware what you are calling "hardware" messages are the bytes that would appear in an actual MIDI transmission on a MIDI cable. What you are calling "software" messages include an extra length indicator after the initial F0, but this only exists as part of the MIDI file specification. In a MIDI file it is important to know in advance how many bytes are part of the System Exclusive message, so a MIDI file includes a "variable-length quantity" value to indicate how many bytes follow that are part of the System Exclusive message. Be aware that a variable-length quantity can be 1 to 4 bytes. The high bit of the byte indicates if there are more bytes in the variable length quantity. The lower seven bits of each of the variable-length quantity bytes are concatenated to form the actual length value. This variable-length quantity only appears in the MIDI file, the actual System Exclusive message that is transmitted on the MIDI output port does not contain the variable-length quantity. In the Standard MIDI Files spec, see PDF page 4 (printed page 2) for how variable-length quantities work, and see PDF page 8 (printed page 6) for how System Exclusive messages are stored in a MIDI file.

SYSEX MANUFACTURER ID

Be aware that System Exclusive messages always contain a Manufacturer ID, it is right after the F0. In your examples, 41 is for Roland, 43 is for Yamaha, and 7E is for a Non-Real Time Universal System Exclusive messages. (Universal System Exclusive messages are defined by MIDI standards instead of a manufacturer.) Be aware that Manufacturer IDs can be 1 byte or 3 bytes. If the first byte is 00 then two more bytes follow in the Manufacturer ID. In the MIDI 1.0 specification, see PDF pages 39, 40, and 83 to 85 (printed pages 34, 35, and T-11 to T-13). Also see the Manufacturer ID list on this website.

SYSEX DEVICE ID

In all of your examples, the next byte after the Manufacturer ID (what you are calling a "model ID" ), is actually a "Device ID" or "Device Number". Be aware that each manufacturer can put whatever they want in their System Exclusive messages after their Manufacturer ID, and the next thing after the Manufacturer ID might not always be a Device ID. But for GM, GS, and XG messages a Device ID does appear after the Manufacturer ID.

The purpose of a Device ID is in case you need to chain together multiple devices that are the exact same manufacturer and model, you can set them to different Device IDs via their control panels, then you can send System Exclusive messages to them independently by putting their corresponding Device ID in the sysex message. There's no "list" of Device IDs, because the user can change the Device ID of their device to whatever they want. You are correct that if you are just trying to tell if the System Exclusive message is a GM, GS, or GX reset message, you don't really care what the device ID is, you should just confirm the rest of the message matches the expected reset message. (Although if you really want to check if the Device ID is in a valid range, it looks like the GM, GS, and XG formats have different allowed ranges for Device IDs.)

Roland calls them "Device IDs" and uses values from hex 00 to 1F. I think hex 10 is the default for Roland GS devices, at least I know it is the default in the Roland Sound Canvas SC-55, the first device to use Roland GS. In the Roland SC-55 manual (from the Roland Manual Archive), page 53 describes Device IDs displayed on the control panel can be from 1 to 32, with a default of 17. Page 72 describes the format of Roland System Exclusive messages, and describes the hex format of the Device ID is 00 to 1F, which is the Device ID shown on the control panel minus 1. (The control panel Device ID 1 is hex 00 in sysex, and so on.) This description says that devices that only respond to one MIDI channel would usually use the same number for their Device ID: A device that responds to a single channel from channel 1 to 16 would use the matching control panel Device ID 1 to 16 or hex 00 to 0F in sysex. Devices that can respond to multiple MIDI channels can use a control panel Device ID from 17 to 32 or hex 10 to 1F in sysex.

Yamaha calls them "Device Numbers", and they can be from hex 10 to 1F, which presumably correspond to Device Number 1 to 16 on the control panel of the device. There is no mention of trying to match them to MIDI channel numbers, so I assume Yamaha's Device Numbers are never related to MIDI channels. Take a look at the manual for the Yamaha MU80 (from the Yamaha Manual Library), the first device to use Yamaha XG. The control panel Device Number setting is on the bottom of page 41 and top of page 42. It is 1 to 16, or "all" to indicate it doesn't care what the incoming device number is. The XG System On System Exclusive message is mentioned on page 124. The device number is listed as hex 1n. It doesn't specifically mention how this maps to the control panel Device Number, but I assume control panel Device Numbers 1 to 16 match sysex hex 10 to 1F.

For System Exclusive messages defined by MIDI standards, they're called Device IDs, and can presumably be any value from hex 00 to 7F, but 7F is the "all device" ID, meaning a device should always respond no matter what its Device ID is. In the MIDI 1.0 standard, Device IDs are mentioned on PDF page 40 (printed page 35). It mentions the Device ID was previously referred to as a "channel byte", but says the Device ID should usually refer to a physical device and not a MIDI channel or virtual device inside a physical device, but there can be exceptions if this makes sense. It only mentions the value hex 7F as being the special "all devices" value, so I presume devices are allowed to use any other values from 00 to 7E as their Device ID for MIDI standard Universal System Exclusive messages if they want to.

SYSEX MODEL IDS

Also, just for your information, be aware that sysex messages usually have some kind of bytes to identify the model or version of the device, but there's not a standard position they will be in for every sysex message. A particular device will look for matching Manufacturer ID byte(s) and Model ID bytes, and possibly Device ID byte to confirm the sysex message is one it should listen to. However, be aware that a device can respond to more than one kind of Model ID, if it can understand the meaning of another model's messages. Yamaha and Roland devices often have a specific model ID but also respond to a generic "GX" or "GS" model ID for the GX or GS features they support. This lets all GX and GS devices use the same types of sysex messages for their GX and GS features.

In fact, the MIDI spec (MIDI 1.0 spec, PDF page 39 / printed page 34) says a device can respond to another manufacturer's messages if that would be useful, but a manufacturer can't define modified or new message types under another manufacturer's ID. Also interesting is that spec apparently requires manufacturers to publish the format of their System Exclusive messages so that anyone can understand and use them if they want to. Older MIDI devices often have detailed descriptions in the manual of all of their MIDI messages including all the manufacturer System Exclusive messages they use. Unfortunately, manuals for modern MIDI devices often do not describe the format of their proprietary System Exclusive messages which saddens me.

 
Posted : 28/04/2021 12:27 am
Jason
Posts: 424
Honorable Member
Topic starter
 

The Wikipedia entry for GM2 has useful bank info for me.
https://en.wikipedia.org/wiki/General_MIDI_Level_2

General MIDI 2 compatible synthesizers access all of the 256 instruments by setting cc#0 (Bank Select MSB) to 121 and using cc#32 (Bank Select LSB) to select the variation bank before a Program Change. Variation bank 0 contains full GM sound set.

Sadly, the GS and XG entries are much less complete.

However, the XG entry links
https://web.archive.org/web/20060926124939/http://www.yamaha.co.uk/xg/reading/pdf/xg_spec.pdf
which is the official XG spec, and lists the bank select MSB and LSB settings. Score! It also includes a complete XG voice/bank table towards the end of the file.

Notably:

Bank Select MSB values are as follows.
00H: Melody voice
01H to 3FH: not used
40H: SFX voice
41H to 7DH: not used
7EH: SFX kit (SFX voices arranged over keyboard)
7FH: Rhythm kit (Rhythm voices arranged over keyboard)

The Bank Select LSB selects from the extended melody voice set. (SFX kit and rhythm kit voices do not
currently support Bank Select LSB extension sets.) Each bank is set for a specific type of variation,
simplifying retrieval of the desired voice.

Also, for XG devices:

XG System On
F0H,43H,1nH,4CH,00H,00H,7EH,00H,F7H
11110000 F0 Exclusive status
01000011 43 YAMAHA ID
0001nnnn 1n Device Number
01001100 4C Model ID
00000000 00 Address High
00000000 00 Address Mid
01111110 7E Address Low
00000000 00 Data
11110111 F7 End of Exclusive

GM System On
F0H,7EH,7FH,09H,01H,F7H
11110000 F0 Exclusive status
01111110 7E Universal Non-realtime ID
01111111 7F Device ID
00001001 09 Sub ID1
00000001 01 Sub ID2
11110111 F7 End of Exclusive

For GS banks/instruments, I'm using this site, which I hope is fairly accurate
https://www.voidaudio.net/gsinstrument.html

 
Posted : 29/04/2021 8:48 am
Jason
Posts: 424
Honorable Member
Topic starter
 

SYSEX STORAGE IN A MIDI FILE HAS A LENGTH
[...]
SYSEX MANUFACTURER ID
[...]
SYSEX DEVICE ID
[...]
SYSEX MODEL IDS
[...]

All very useful info. I've only ever dealt with midi files, and have not used any actual hardware. It's good to know where/how all of the data/info originated.

 
Posted : 29/04/2021 9:01 am
Jason
Posts: 424
Honorable Member
Topic starter
 

I added an icon display for when a file contains sysex resets. As JohnG mentioned above, I am seeing that official XG files do have both a GM and XG reset. A few files have just a GM reset. I think all GS files so far have had a GS reset, except for one that I'm not sure of from a game I used to play. The file I have was "repaired" using Anvil Studio, so it may not be the original. It used to not play properly for me, so it may in fact be a proper GS file that was modified to make it GM compatible.

Not all midi files have a reset, so some inferral will have to be made (and I will allow the user to change the selected naming scheme if it is not correct) about files that don't have any reset. Basically, anything with bank selects outside of normal GM 0 or 121 will get examined, and a secondary guess made from those.

Of note in terms of bank selects, in some files that have a GS reset, I can now tell by looking at the bank selects that they are actually MT-32 midi files (bank select 127 on most of the melody channels). I will be adding those in to my lists, but I don't believe any of my software will play them back properly for me to test.

Bank 127 is also the percussion bank select for XG files, so if either type lacks the proper reset, it will be more difficult to tell which is which.

I may also add the option to insert a reset if one does not exist.

I'm still trying to decipher exactly what the bank select LSB is doing in GS files. I've seen mention that it's the Roland "family" that the tones belong to, but have not found a complete list, only "i.e. 1 - SC-55, 2 - SC-88 etc.". That is the most complete list I've found. Most don't even include the description of option 2 :p Chances are it won't matter for what I'm doing. But I would like to know for completeness sake.

Here's what it looks like right now. The greyed out items are displaying the default (bank 0) name, because it's either an invalid bank select MSB (unlikely) or I just haven't gotten that far in updating the names list (almost 100% likely!). I've made it through program/instrument 49. Anything after that is in progress.

 
Posted : 30/04/2021 9:22 am
Jason
Posts: 424
Honorable Member
Topic starter
 

More info!

https://cdn.roland.com/assets/media/pdf/SC-8850_OM.pdf

This is the manual for the Roland SC-8850, which has an even more complete list of SC programs/instruments than the list I was using that I mentioned a few replies above this one. I "finished" entering all of the GS instrument names in to my program. I recently downloaded a full GS soundtrack from an old Sierra game. It's nearly an hour long, and has all of the music from the game in a single file. I noticed one particular GS instrument was missing it's listing, so I had to search around for a while online to try to locate a list. This manual has that list. It looks like the list I used up to this point stops at the SC-88 Pro. The SC-8850 has quite a few more programs. So it looks like I have yet more work to do!

Also,

Of note in terms of bank selects, in some files that have a GS reset, I can now tell by looking at the bank selects that they are actually MT-32 midi files (bank select 127 on most of the melody channels). I will be adding those in to my lists, but I don't believe any of my software will play them back properly for me to test.

More to this than I thought as well. Several modes in Roland GS use banks 126 and 127, not just MT-32. Even more to still do!

 
Posted : 30/04/2021 4:17 pm
Share: