PhotoSynth > InterFACE
An extended-reality musical synthesizer - completely controlled by the face! Sometimes referred to as the Smile Powered Synthesizer =D
By piggybacking onto a person's own natural movements using motion-tracking in real-time, we reduce the complexity required to learn a new instrument to mere moments whilst still retaining full expression and precise control.
Simply open your mouth to begin - it is a lot like singing. The head becomes the primary interface and the mouth the main control - we can convert smiles into musical notes and winks into audio controls. By observing the features on the face over time, we know at any point the angle and orientation of the face as well as basic facial emotions and moods, and we can use these in order to trigger and modify sounds.
This virtual interface has no moving parts, requires no extra hardware, and can run on most modern computers and mobile phones. We can capitalise on a person's own expressions and movements to create unique personalised synthesized music but thanks to MIDI we can also control and manipulate almost any electronic musical instrument too.
Simply connect your MIDI instrument to instantly be able to play it with your head! Without any effort, the face is now an accessible monophonic MIDI controller, whilst with a friend in duet mode you can have polyphony too!
This is a zero-touch interface that piggy backs onto your natural movements to imbue them with new properties and skills. By recognising universally familiar movements, such as smiling and nodding, along with reading simple human emotions, we can form the basis of a new type of smart interface that can reduce the complexity of operation whilst still retaining full expression and control. Without any extra equipment or gear, your computer and studio can now be completely and simply operated by your face. The brief was simple but the end product requires much complexity in order to offer that simplicity :
● Requires no touch to be played
● Be suitable for all ages
● Have little to no learning curve
● Be accessible, inclusive and collaborative
● Be engaging, fun and a little bit magical
The scope of this project’s ability falls far beyond a musical instrument. This is as much a research project into perfecting natural-instinct interfaces for the future of human-machine interaction. Current state of the art motion capture devices are complicated, expensive and bespoke pieces of equipment - we intend to remove the need for any specialised equipment entirely and remove the cost so as to completely democratise accessible interface technology.