fbpx

HITar

Categories: Hardware Prototypes/Non-Commercial products

Submitted by:Andrea Martelloni

24 May 2023

Elevator Pitch

HITar

An augmented guitar for percussive fingerstyle, using AI to transform the body into an expressive MIDI controller for rich multi-dimensional control of drum synthesisers.

Product Description

HITar

The HITar augmented guitar was created by Andrea Martelloni, Andrew McPherson, and Mathieu Barthet at Queen Mary University of London. It takes inspiration from the practice of percussive fingerstyle acoustic guitar, which involves layered arrangements, altered tunings, and percussive techniques on the guitar's body to enable musicians to act as a "one-man band". The HITar builds upon percussive fingerstyle's tradition and aesthetics, and reinvents the way the player can interact with the guitar's body by expanding its timbral control.
The HITar technology works with a regular acoustic guitar with five piezo sensors added underneath the areas of the body most used in percussive technique. An Artificial Intelligence engine processes the signal of the piezo sensors in real time, and recognises which part of the hand is used for each hit. This enables to describe the technique used by the player, for example, heel, or fingers. Our AI engine can also "morph" organically from one category to another as the musician gradually moves from one part of the hand to another.
We built a HITar prototype around the MetaSynth physics-based synthesis model by the CNRS-AMU PRISM laboratory (France), by driving the model from the piezo sensor's audio and controlling its parameters with our AI engine. In our demo video we present another HITar prototype where the AI engine outputs MIDI messages enabling to control sample-based virtual instruments, such as Spitfire Audio plugins, and other MIDI devices.

How It's Innovative

The HITar was designed to bridge the gap between the physical and the digital using AI. We wanted to describe each individual percussive hit through a rich and multi-dimensional representation, capturing more than just hit category (e.g. type of sample to be reproduced) and velocity (how loud it's supposed to be). Therefore, we tasked the AI with finding out what makes each body hit different from all the other hits of its class.
This innovation opens up a totally new range of possibilities to percussive guitarists, who can control a digital percussion instrument with timbres not limited to those of a wooden body. Still, the richness of the control yielded by the HITar offers an intimacy and an immediacy not possible with buttons or conventional MIDI control surfaces. Since the strings are sensed by our AI engine too, string hits and slaps also become an organic part of the new generated sounds.
The AI technology behind the HITar is not limited to guitar drumming. If deployed into electronic percussion, the algorithm can offer more richness than conventional sample-triggering electronic percussion, while still providing the flexibility of MIDI compatibility that is lacking in other digital percussions with proprietary synthesis engines. This technology has the potential to develop into the solution for expressive digital percussion similarly to expressive keyboard controllers like the Continuum or the Seaboard.

See MIDI Innovation In Action

Most Inspiring Use Cases

The most immediate use of the HITar is to expand the vocabulary of sounds of percussive guitar, especially to support techniques and sounds that are idiomatic of other percussions but cannot be done on the guitar. For example, with the HITar, the heel of the hand can be used to mute a digital drum, whereas finger hits can be used for resonant hits, introducing the same muting techniques of skin membrane drums.
HITar players can appreciate the fact that their guitar body can sound nothing like a guitar's body. With the MetaSynth engine, we demonstrated that different hand positions can be associated to nuanced synthesis parameters, e.g. the tension of a virtual string or the material of any other virtual surface. Sample libraries can also be used to expand the sound palette to drums, cymbals, world percussion etc., all the way to cinematic soundscapes. The guitar then becomes a compositional tool even beyond the "one man band", opening up uses such as film scoring. We invite you to watch the video included in this submission showcasing all these options.

Expansion Plans

The HITar is still at the prototype stage. As a project within the Artificial Intelligence and Music programme at Queen Mary University of London, we actively conduct research to further refine its workings with iterative development and user studies. Currently, the AI algorithm runs on a laptop, but it was designed and coded so that migration to embedded hardware will be fast. 

Work is under way to support and explore even richer ways to represent hand hits: by using more than two hand parts, therefore supporting more than two MIDI notes, and by involving localisation (an indication of where the guitar body was hit) in the description parameters. 

At its heart, our technology is quite versatile: it is a method for low-dimensional description of impact sounds with very low latency. We intend to explore its application to many other percussion techniques beyond the gestures that our models were trained on, as well as its use with sticks and mallets rather than hands. Its uses beyond percussion, or indeed beyond musical applications, should also be explored too.

Commercialization

Our vision is to bring the technology behind the HITar to market and create a family of musical instruments based on it. We will shortly present the technology driving the HITar as a set of sensors, a single-board computer and a MIDI & audio interface that can augment the rigid body of any stringed or percussion instrument.
We are open to partnerships with new and established players in the musical instrument business to co-design a new family of rich AI-based percussion instruments. We will also seek investment to develop and bring to market our own musical interfaces, consisting in future revisions of the HITar as well as completely new instruments. We want to respond to the growing demand from expert players, bedroom producers and film/media composer to have a digital percussion interface that has the immediacy, intimacy and richness of a traditional acoustic instrument, yet offers the control of a digital interface connected to Digital Audio Workstations and current synthesis engines.