MusiKraken

MusiKraken is an experimental MIDI controller construction kit.

It runs on iOS and Android and is a completely modular system that allows you to combine the values of various sensors built into mobile devices with many MIDI effects. The MIDI events can be sent to other devices via WiFi, Bluetooth and USB cable, or to other apps, and it can host Audio Unit instruments on iOS.

The editor allows you to combine all modules by connecting the available ports. Orange MIDI ports send and receive MIDI events like Note-On and Note-Off, while green value ports send the values of the sensors.

Using the Value-to-MIDI module, you can convert any values to MIDI control change, note, pitch bend and channel pressure events. This way you are completely free in what your MIDI controller setup should be able to do. MusiKraken currently supports the following input sensors:

Touch: The Keyboard module is a typical musical keyboard, but it also allows you to control additional values while playing, like sliding on the keys and using touch pressure or touch radius (if supported). It supports MPE, has two different layouts, and lets you highlight one of 95 scales.

The Chords Pad has a button for each chord in a scale, and vertically has one for each inversion of these chords. While playing these chords, you can also control additional parameters by sliding or changing the touch pressure or radius. And the Touch Pad allows you to control up to five parameters simultaneously by moving your fingers on the screen.

Camera: Use the camera to track face-, hand- and body movements. Face tracking has parameters like head rotation and mouth opening amount. The hand tracking tracks the position of both hands and various hand gestures. And the body pose tracking detects the position of your head, hands and feet on the camera image. And on iOS devices with a TrueDepth sensor, the app can even use the distance to the hand (or other objects) as an input.

Motion Sensors: MusiKraken has a separate Accelerometer module to measure accelerations and a Motion Sensor module which combines the values of the Accelerometer, Gyroscope and Magnetometer to get the current device rotation.

Microphone: This module uses the microphone of the device to detect pitch and amplitude.

The effects in the app allow you do the following:

• The Channel Switcher lets you route MIDI events to specific MIDI channels.
• The Chord Splitter can split chords into separate notes and send each to a different channel to control multiple instruments simultaneously.
• One effect let you transpose the notes and the Arpeggiator can arpeggiate through them.
• The Beat effect lets you simulate percussion instruments with all input sensors, and the Threshold effect plays notes if a value goes over a specific threshold.
• The LFO generates an LFO output value that can be controlled by other input values.
• And the Envelope module generates an ADSR output value based on note input. As all these modules are independent and can be combined, this allows you to build your MIDI controller in any way you want.

You can adapt MusiKraken to whatever you are controlling with it. You do not need to adapt your music making style to the MIDI controller, you can instead change the MIDI controller until it fits you and the instrument that you are playing with it.

Because the setup in the app is completely modular, you can make combinations that were never before possible in a MIDI controller. MusiKraken also provides some input methods that do not exist in any other MIDI controller and makes it possible to combine multiple input methods to create new ones.

With MusiKraken, you can be very creative when combining the different modules. One way to use it is to ask yourself how your ideal MIDI controller would look like for controlling a specific virtual instrument: Would it make sense to play the notes with a touch interface, or would it feel more natural when using the motion sensors? Would it make sense to change the expression of the instrument by tracking my mouth or head movements?

The answer to these questions is usually very different if you are controlling a percussion instrument, a virtual string instrument, brass or woodwinds, or a synth. Or you can be completely open to new combinations and invent your own experimental MIDI controller. All modules that are currently included in MusiKraken can be used simultaneously, so the number of possible combinations are nearly unlimited.