AudioPint

The AudioPint vision is to build a portable, durable, and inexpensive sound synthesis and effects system that is based completely on free open-source software, is flexibly reconfigurable and easy to use, and that is capable of accepting a diverse range of input devices for control purposes. AudioPint supports expressivity and experimentation with new forms of sound control.We expect tomorrow's musicians to be using tomorrow's instruments, and to an increasing extent, these instruments can be tools of their own design.

Invisible Media

With Invisible Media we can augment objects around us to make them sensitive to, and able to inform, the focus of our attention in order to provide relevant content. The system is built to minimize bulky wearable gear, and allows the user to navigate this situated information with speech commands, keeping their hands available for manual manipulation of the objects themselves. Information is presented to people auditorily, resulting in a user-system dialog that attempts to mimic a domain expert or recommender who knows what objects are in view of the user and can suggest relevant content. We have created Engine-Info, a training application that teaches the components of an internal combustion engine, as well as a personalized shopping scenario that can suggest appropriate foods in a supermarket based on a person's preferences and health needs.

click the video to play

PureJoy

PureJoy is a flexible vocal expression interface. Think of it as an audio sketchpad for prototyping harmonies, beatboxing, and other short impromptu compositions. PureJoy uses a game joystick as a physical interface to capture and manipulate vocal sounds and other "found" sound, making it a great tool for improvised musique concrète. The use of vocal sound as primary source material takes advantage our most expressive and automatic soundmaking capability, but enables a performer to amplify the number of voices they can produce, enhance the complexity of their rythms, and change the timbre of their sound. Additionally, the joystick is a familiar, playful interface that is commonly used as a manipulator for digital information.

Siftables

Imagine overturning a container of nuts and bolts, then looking through the resulting pile for a particular item. Or spreading photographs out on a tabletop and then sorting them into piles. During these activities we interact with large numbers of small objects at the same time, using all of our fingers and both hands together. These tasks take advantage of our existing skills, and we can effortlessly sift and sort - focusing on our higher level goals rather than the items themselves.

Siftables aims to enable people to interact with information and media in a physical, natural manner that approaches interactions with physical objects in our everyday lives. As an interaction platform, Siftables applies technology and methodology from wireless sensor networks to tangible user interfaces. Siftables are independent, compact devices with sensing, graphical display, and wireless communication capabilities. They can be physically manipulated as a group to interact with digital information and media. Siftables can be used to implement any number of gestural interaction languages and HCI applications.

The Siftables interaction platform is a collaboration with Jeevan Kalanithi. .

the Sound of Touch

The Sound of Touch is a new instrument for real-time capture and sensitive physical stimulation of sound samples using digital convolution. The hand-held wand can be used to (1) record sound, then (2) brush, scrape, strike or otherwise physically manipulate this sound against physical objects. These actions produce sound in a manner that leverages peoples existing intuitions about sonic properties of physical materials. The Sound of Touch permits real-time exploitation of the sonic properties of a physical environment, to achieve a rich and expressive control of digital sound that is not typically possible in electronic sound synthesis and control systems.