Categories: Hardware Prototypes/Non-Commercial products

Submitted by:Irmandy Wicaksono

24 May 2023

Elevator Pitch


Soft, stretchable, and deformable MIDI-KnittedKeyboard for intimate, organic, and expressive musical composition and performance

Product Description


Spanning from the early Musical Telegraph and Electronic Sackbut, to the late EMS Synthi AKS and Moog Synthesizers, electronic music and musical controllers, particularly the keyboards, have enabled people from all walks of life and all around the world to produce and manipulate sound as a mean for creativity, expressions, and shared experience. Note that most of the expressive keyboard interfaces to date rest on a rigid and heavy structure. On the other hand, textiles are ubiquitous in our daily life. They are highly formable and palpable materials with a broad spectrum of patterns, structures, and textures, making them a great candidate for physical interfaces. Inspired by theremin's expressive controls and the soft and deformable tactile properties of knitted textiles, we have invented an interactive textile-based musical interface with a familiar layout of piano keys. A 5-octave keyboard made out of fabric, besides providing new interactions and tactile experiences for musical expressions, can be easily folded, rolled up, and packed in our luggage like a pair of socks or a scarf. It can also be wearable, which extends the functionality of such fabric-based musical controllers, similar to the keyboard tie formerly demonstrated by Laurie Anderson in the "Home of the Brave." In our demo performance, we presented the "Fabric of Time and Space", a contemporary musical piece exclusively written for KnittedKeyboard to demonstrate and illustrate the multi-dimensional expressiveness of the instrument. The piece is a metaphor for the expanding and contracting nature of the universe and this is represented musically by the glissandi of the melody as well as the interplay between major and minor chords. The metaphorical perturbations of space-time were expressed by the interaction between the performer and the fabric. The musical translation of these expressions were used to shape the envelope of the sound.

How It's Innovative

The sensing mechanism of our instrument is based on capacitive and piezoresistive sensing. Every key act as an electrode and is sequentially charged and discharged. This creates an electromagnetic field that can be disrupted by hand’s approach and touch, enabling us to detect non-contact proxemic gesture such as hovering or waving on the air, contact touch, as well as to calculate velocity. The piezo-resistive layer underneath can measure the force and stretched exerted on the keyboard. We map these pressure and stretch values as aftertouch/modulation. The KnittedKeyboard demonstrates a new fabrication process of fabric-based interactive surfaces. It enables performers to experience fabric-based multimodal embodied interaction and unique, intimate and organic tactile experience as they explore the seamless texture and materiality of the electronic textile. The underlining technology would enable further exploration of soft and malleable gestural interfaces that leverage the unique mechanical structures of the materials, as well as the intrinsic electrical properties of the knitted sensors.

See MIDI Innovation In Action

Most Inspiring Use Cases

Our prototype utilizes digital knitting technology and explores intarsia, interlock patterning, and a collection of functional (electrically-conductive and thermoplastic) and non-functional (polyester) fibers to develop a seamless and customized, 5-octave piano-patterned textile for expressive and virtuosic sonic interaction. The individual and combinations of keys could simultaneously sense touch, as well as continuous proximity, stretch, and pressure. The KnittedKeyboard combines both discrete controls from the conventional keystrokes and expressive continuous controls from the non-contact theremin-inspired proximity sensors by waving and hovering on the air, as well as unique physical interactions enabled by the integrated fabric sensors (e.g. squeezing, pulling, stretching, and twisting). The fact that the KnittedKeyboard is squishy and elastic, while also intrinsically integrated with fabric sensors that are responsive to pressure and stretch, enables an intimate relationship between the physical gestures and the sound. Some comments we have received from musicians are that it feels that the fabric interface is alive, as it has a direct embodiment of the sound and the expressions. It makes the tactile-auditory-sensory experience organic.

Expansion Plans

Our expansion plans include choosing a higher-performance central processor with in-built MIDI for a much lower-latency, developing sound packs/libraries exclusively synthesized for KnittedKeyboard/other fabric-based gestural musical controllers, and implementing the new MIDI 2.0 protocol in our system.


Since the KnittedKeyboard was fabricated using machine knitting, a manufacturing process that has been pervasive in the garment and technical fabrics industry, the product is certainly viable for commercialization. Machine knitting is an additive manufacturing process. Therefore, besides being customizable, it is more sustainable and cost-effective, as it creates little to no waste. Currently, it only takes us 40 minutes to make a 5-octave KnittedKeyboard. We plan to transform our rigid sensing PCBs into flexible polyiamide-based ones to make them more robust and flexible, as well as to accommodate the soft and sleek nature of the KnittedKeyboard. We will also develop more robust textile-hardware connection, and think about the integration of an on- board audio generation or haptic feedback system.