Researchers at Massachusetts Institute of Technology’s (MIT) Media Lab have developed a fully functioning knitted keyboard. KnittedKeyboard II, developed by Responsive Environments PhD student Irmandy Wicaksono and demonstrated by Viral Communications PhD student Mike Hao Jiang, is a multi-modal, soft, and stretchable electronic textile musical interface, customized and fabricated with digital knitting and functional fibres.

Inspired by the theremin (an electronic musical instrument controlled without physical contact) and the unique tactile properties of knitted textiles, the keyboard is responsive to touch, pressure, stretch, and proximity, allowing performers to play discrete notes, as well as trigger continuous effects and modulations with virtuosity. The flat knitted prototype helps broaden the scope of fabric-based interactive surfaces, from novel musical controllers, wearables, to smart objects.

The sensing mechanism is based on capacitive and piezo-resistive sensing. Every key acts as an electrode and is sequentially charged and discharged

A keyboard made from fabric, besides providing new interactions and tactile experiences for musical expressions, can be easily folded, rolled up, and packed in luggage like a pair of socks or a scarf, the researchers say.  It can also be wearable, which extends the functionality of such fabric-based musical controllers, like the keyboard tie formerly demonstrated by Laurie Anderson in the ‘Home of the Brave.’

The prototype keyboard utilizes digital knitting technology and explores intarsia, interlock patterning, and a collection of functional (electrically conductive and thermoplastic) and non-functional (polyester) fibres to develop a seamless and customized, 5-octave piano-patterned textile for expressive and virtuosic sonic interaction.

The individual and combinations of keys can simultaneously sense touch, as well as continuous proximity, stretch, and pressure. The KnittedKeyboard II combines both discrete controls from the conventional keystrokes and expressive continuous controls from the non-contact theremin-inspired proximity sensors by waving and hovering on the air, as well as unique physical interactions enabled by the integrated fabric sensors (e.g., squeezing, pulling, stretching, and twisting). It enables performers to experience fabric-based multimodal embodied interaction and unique, intimate, and organic tactile experience as they explore the seamless texture and materiality of the electronic textile, the researchers explain.

“The sensing mechanism is based on capacitive and piezo-resistive sensing. Every key acts as an electrode and is sequentially charged and discharged. This creates an electromagnetic field that can be disrupted by the hand’s approach, enabling us to detect not only contact touch, but also non-contact proxemic gesture such as hovering or waving on the air, contact touch, as well as to calculate strike velocity,” Irmandy Wicaksono says.

“The piezo-resistive layers underneath can measure pressure and stretch exerted on the knitted keyboard. All of the sensor data is converted to musical instrument digital interface (MIDI) messages by a central microprocessor, which will correspond to certain timbral, dynamic, and temporal variations (filter resonance, frequency, glide, reverb, amp, distortion, et cetera), as well as pitch-bend. Audio sequencing and generation software such as Ableton Live and Max/MSP map these MIDI messages to their corresponding channels, controls, notes, and effects.”

‘Fabric of Time and Space’ is a contemporary musical piece exclusively written for KnittedKeyboard II to demonstrate and illustrate the multi-dimensional expressiveness of the instrument. The piece is said to be “a metaphor for the expanding and contracting nature of the universe, and this is represented musically by the glissandi of the melody as well as the interplay between major and minor chords. The metaphorical perturbations of space-time were expressed by the interaction between the performer and the fabric. The musical translation of these expressions were used to shape the envelope of the sound.”

“This work brings us one step closer to the vision of seamless fabric-based interactive surfaces, that is not only applicable in novel musical controllers, but also in wearables, smart objects, and responsive environments. The underlining technology would enable further exploration of soft and malleable gestural interfaces that leverage the unique mechanical structures of the materials, as well as the intrinsic electrical properties of the knitted sensors,” the researchers conclude.

Watch and listen…

KnittedKeyboard II from MIT Media Lab on Vimeo

Research team:

Irmandy Wicaksono, Research Assistant

Mike Hao Jiang, Research Assistant

Joseph A. Paradiso, Alexander W Dreyfoos (1954) Professor

www.media.mit.edu