MusiKraken is an experimental MIDI controller construction kit.
It runs on iOS and Android and is a completely modular system that allows you to combine the values of various sensors built into mobile devices with many MIDI effects. The MIDI events can be sent to other devices via WiFi, Bluetooth and USB cable, or to other apps, and it can host Audio Unit instruments on iOS.
The editor allows you to combine all modules by connecting the available ports. Orange MIDI ports send and receive MIDI events like Note-On and Note-Off, while green value ports send the values of the sensors.
Using the Value-to-MIDI module, you can convert any values to MIDI control change, note, pitch bend and channel pressure events. This way you are completely free in what your MIDI controller setup should be able to do. MusiKraken currently supports the following input sensors:
Touch: The Keyboard module is a typical musical keyboard, but it also allows you to control additional values while playing, like sliding on the keys and using touch pressure or touch radius (if supported). It supports MPE, has two different layouts, and lets you highlight one of 95 scales.
The Chords Pad has a button for each chord in a scale, and vertically has one for each inversion of these chords. While playing these chords, you can also control additional parameters by sliding or changing the touch pressure or radius. And the Touch Pad allows you to control up to five parameters simultaneously by moving your fingers on the screen.
Camera: Use the camera to track face-, hand- and body movements. Face tracking has parameters like head rotation and mouth opening amount. The hand tracking tracks the position of both hands and various hand gestures. And the body pose tracking detects the position of your head, hands and feet on the camera image. And on iOS devices with a TrueDepth sensor, the app can even use the distance to the hand (or other objects) as an input.
Motion Sensors: MusiKraken has a separate Accelerometer module to measure accelerations and a Motion Sensor module which combines the values of the Accelerometer, Gyroscope and Magnetometer to get the current device rotation.
Microphone: This module uses the microphone of the device to detect pitch and amplitude.
The effects in the app allow you do the following:
- The Channel Switcher lets you route MIDI events to specific MIDI channels.
- The Chord Splitter can split chords into separate notes and send each to a different channel to control multiple instruments simultaneously.
- One effect let you transpose the notes and the Arpeggiator can arpeggiate through them.
- The Beat effect lets you simulate percussion instruments with all input sensors, and the Threshold effect plays notes if a value goes over a specific threshold.
- The LFO generates an LFO output value that can be controlled by other input values.
- And the Envelope module generates an ADSR output value based on note input. As all these modules are independent and can be combined, this allows you to build your MIDI controller in any way you want.