Audiocubes

Last updated

Detailed view of AudioCubes made by Percussa Detailed view of Audiocubes.jpg
Detailed view of AudioCubes made by Percussa

AudioCubes are a collection of wireless intelligent light-emitting objects, capable of detecting each other's location, orientation, and user gestures. They were created by Bert Schiettecatte as electronic musical instruments for use by musicians in live performance, sound design, musical composition, and for creating interactive applications in max/msp, pd and C++.

Contents

The concept of AudioCubes was first presented by Schiettecatte in April 2004 at the CHI2004 conference in Vienna. [1] A first prototype of AudioCubes was shown at the Museum for Contemporary Art, MUHKA in Antwerp in December 2004. AudioCubes were used in an art installation created in collaboration with Peter Swinnen during the Champ D’Action Time Canvas festival. [2]

In January 2007, AudioCubes were launched on the market [3] and offered for sale on the website of Percussa, a company Schiettecatte founded in October 2004 to promote AudioCubes.

Hardware architecture

Each AudioCube is identical, and has a small built-in computer which is able to detect the position and location of the other cubes in a network and measure the distances between them. AudioCubes also work without drivers and communicate using high speed HID.

An AudioCube has four onboard infrared sensors (one on each face) to communicate and measure distances to objects nearby, digital signal processors (DSP), a USB-rechargeable battery, and a translucent housing.

MIDI and OSC compatible

AudioCubes work wirelessly with MIDI-compatible software and hardware (e.g. FL Studio, Logic Pro, Reason, drum machines and Monome). A connection must be established using middleware such as MIDIBridge.

AudioCubes also come with an Open Sound Control (OSC) server to send and receive OSC data. [4]

Software Applications

Several applications have been created for use with AudioCubes, each with a different focus, such as sound design, music composition, or live performance, or for creating applications in max/msp, pd and C++.

In addition, a number of Max/MSP patches were created to work with AudioCubes.

Software Uses

AudioCubes for Live Performance

The AudioCubes can be used to send MIDI notes to MIDI compatible software/hardware using MIDIBridge. When two AudioCubes are put next to each other, they detect each other, and triggers are sent as MIDI notes. These triggers can then be used to control on/off type of signals, such as start and stop audio clips in a Digital Audio Workstation (DAW), such as Ableton Live. At each face of the cube, a different audio clip can be assigned in a DAW.

The AudioCubes can also measure distances to nearby objects or your hands when configured as a sensor cube in MIDIBridge. [5] In the same way, this sensor data is sent to the computer as a continuous controller (CC) which can be used to control parameters in the DAW. Since each cube has 4 sensors, up to 4 parameters can be controlled per AudioCube.

In addition, you can also control the RGB colors of the AudioCubes and use this information as feedback during a live performance. [5]

AudioCubes for Sound Design

The sensors of the AudioCubes can also be used to shape sounds. By moving hands and fingers closer or further away from the 4 sensors, it generates 4 different MIDI CCs which can be sent to MIDI compatible instruments. When using an AudioCube in this way, it can be compared to a 4D optical theremin.

The AudioCubes can also be linked to LFOs by using the software application Evolvor. The LFO waveforms are designed in the graphical editors of Evolvor. Each AudioCube is then automatically linked to an LFO, because of the topology detection. LFO signals can be added and removed, by adding and removing AudioCubes. The signals can also be mixed and matched, by mixing and matching AudioCubes.

AudioCubes for Music Composition

When using the Improvisor application, velocity as well as semitone patterns are automatically linked to every AudioCube. Every AudioCube plays the melody created by both patterns. When cubes are placed next to each other they can follow the same melody. In this way, music can be composed by mixing and rearranging AudioCubes.

AudioCubes for Creating Interactive Applications

Several tools have been created to make applications for the AudioCubes in max/msp, pure data, and C++.

Artists

AudioCubes have been used by some performers such as Mark Mosher, [6] Pearls for Swines, Richard Devine, Steve Baltes, Bostich from Nortec, Ilan Kriger, Arecio Smith, Julien Pauty, and the European Bridges Ensemble. [7]

Example of Tangible User Interface

AudioCubes are an example of a Tangible User Interface. In the past few years[ when? ] a lot of research has been done in the field of Tangible User Interfaces. The Reactable is another example of such an interface. It is an installation on which people can move around objects which are followed by a camera and projector on a surface.

Awards

For the creation of the AudioCubes, Bert Schiettecatte received in 2009 the prestigious Qwartz Electronic Music Awards in Paris. He was also invited to give a talk at TEDx Mediterranean in Cannes, September 2010. [8]

Related Research Articles

<span class="mw-page-title-main">MIDI</span> Electronic musical instrument connection standard

MIDI is a technical standard that describes a communication protocol, digital interface, and electrical connectors that connect a wide variety of electronic musical instruments, computers, and related audio devices for playing, editing, and recording music.

<span class="mw-page-title-main">Sound card</span> Expansion card that provides input and output of audio signals

A sound card is an internal expansion card that provides input and output of audio signals to and from a computer under the control of computer programs. The term sound card is also applied to external audio interfaces used for professional audio applications.

<span class="mw-page-title-main">CV/gate</span> Analogue method of electronic sound production

CV/gate is an analog method of controlling synthesizers, drum machines, and similar equipment with external sequencers. The control voltage typically controls pitch and the gate signal controls note on-off.

Open Sound Control (OSC) is a protocol for networking sound synthesizers, computers, and other multimedia devices for purposes such as musical performance or show control. OSC's advantages include interoperability, accuracy, flexibility and enhanced organization and documentation. Its disadvantages include inefficient coding of information, increased load on embedded processors, and lack of standardized messages/interoperability. The first specification was released in March 2002.

Steinberg Media Technologies GmbH is a German musical software and hardware company based in Hamburg. It develops music writing, recording, arranging, and editing software, most notably Cubase, Nuendo, and Dorico. It also designs audio and MIDI hardware interfaces, controllers, and iOS/Android music apps including Cubasis. Steinberg created several industry standard music technologies including the Virtual Studio Technology (VST) format for plug-ins and the ASIO protocol. Steinberg has been a wholly owned subsidiary of Yamaha since 2005.

<span class="mw-page-title-main">Virtual Studio Technology</span> Audio plug-in software interface

Virtual Studio Technology (VST) is an audio plug-in software interface that integrates software synthesizers and effects units into digital audio workstations. VST and similar technologies use digital signal processing to simulate traditional recording studio hardware in software. Thousands of plugins exist, both commercial and freeware, and many audio applications support VST under license from its creator, Steinberg.

<span class="mw-page-title-main">Digital audio workstation</span> Electronic device or application software used for recording, editing and producing audio files

A digital audio workstation is an electronic device or application software used for recording, editing and producing audio files. DAWs come in a wide variety of configurations from a single software program on a laptop, to an integrated stand-alone unit, all the way to a highly complex configuration of numerous components controlled by a central computer. Regardless of configuration, modern DAWs have a central interface that allows the user to alter and mix multiple recordings and tracks into a final produced piece.

<span class="mw-page-title-main">Audio editing software</span> Computer system for manipulating audio

Audio editing software is any software or computer program which allows editing and generating audio data. Audio editing software can be implemented completely or partly as a library, as a computer application, as a web application, or as a loadable kernel module. Wave editors are digital audio editors. There are many sources of software available to perform this function. Most can edit music, apply effects and filters, and adjust stereo channels.

<span class="mw-page-title-main">Max (software)</span> Visual programming language

Max, also known as Max/MSP/Jitter, is a visual programming language for music and multimedia developed and maintained by San Francisco-based software company Cycling '74. Over its more than thirty-year history, it has been used by composers, performers, software designers, researchers, and artists to create recordings, performances, and installations.

<span class="mw-page-title-main">Novation Digital Music Systems</span> British musical equipment manufacturer

Novation Digital Music Systems Ltd. is a British musical equipment manufacturer, founded in 1992 by Ian Jannaway and Mark Thompson as Novation Electronic Music Systems. Today the company specializes in MIDI controllers with and without keyboards, both analog and virtual analog performance synthesizers, grid-based performance controllers, and audio interfaces. At present, Novation products are primarily manufactured in China.

<span class="mw-page-title-main">MIDI controller</span> Device that produces MIDI data

A MIDI controller is any hardware or software that generates and transmits Musical Instrument Digital Interface (MIDI) data to MIDI-enabled devices, typically to trigger sounds and control parameters of an electronic music performance. They most often use a musical keyboard to send data about the pitch of notes to play, although a MIDI controller may trigger lighting and other effects. A wind controller has a sensor that converts breath pressure to volume information and lip pressure to control pitch. Controllers for percussion and stringed instruments exist, as well as specialized and experimental devices. Some MIDI controllers are used in association with specific digital audio workstation software. The original MIDI specification has been extended to include a greater range of control features.

<span class="mw-page-title-main">MIDI keyboard</span> Piano-style keyboard that sends MIDI inputs to a computer or device

A MIDI keyboard or controller keyboard is typically a piano-style electronic musical keyboard, often with other buttons, wheels and sliders, used for sending Musical Instrument Digital Interface (MIDI) signals or commands over a USB or MIDI 5-pin cable to other musical devices or computers. MIDI keyboards lacking an onboard sound module cannot produce sounds themselves, however some models of MIDI keyboards contain both a MIDI controller and sound module, allowing them to operate independently. When used as a MIDI controller, MIDI information on keys or buttons the performer has pressed is sent to a receiving device capable of creating sound through modeling synthesis, sample playback, or an analog hardware instrument. The receiving device could be:

<span class="mw-page-title-main">Lemur (input device)</span>

The Lemur was a highly customizable multi-touch device from French company JazzMutant founded by Yoann Gantch, Pascal Joguet, Guillaume Largillier and Julien Olivier in 2002, which served as a controller for musical devices such as synthesizers and mixing consoles, as well as for other media applications such as video performances. As an audio tool, the Lemur's role was equivalent to that of a MIDI controller in a MIDI studio setup, except that the Lemur used the Open Sound Control (OSC) protocol, a high-speed networking replacement for MIDI. The controller was especially well-suited for use with Reaktor and Max/MSP, tools for building custom software synthesizers.

Human User Interface Protocol is a proprietary MIDI communications protocol for interfacing between a hardware audio control surface and digital audio workstation (DAW) software. It was first created by Mackie and Digidesign in 1997 for use with Pro Tools, and is now part of the Mackie Control Universal (MCU) protocol.

FluidSynth, formerly named iiwusynth, is a free open source software synthesizer which converts MIDI note data into an audio signal using SoundFont technology without need for a SoundFont-compatible soundcard. FluidSynth can act as a virtual MIDI device, able to receive MIDI data from any program and transform it into audio on-the-fly. It can also read in SMF (.mid) files directly. On the output side, it can send audio data directly to an audio device for playback, or to a Raw or Wave file. It can also convert a SMF file directly to an audio file in faster-than-real-time. The combination of these features gives FluidSynth the following major use cases:

<span class="mw-page-title-main">Bidule</span> Music software application

Bidule is a commercial software application for the creation of interactive computer music and multimedia produced by the Canadian company Plogue Arts and Technology. It runs on both Windows and Mac computers.

I-CubeX comprises a system of sensors, actuators and interfaces that are configured by a personal computer. Using MIDI, Bluetooth or the Universal Serial Bus (USB) as the basis for all communication, the complexity is managed behind a variety of software tools, including an end-user configuration editor, Max (software) plugins, and a C++ Application Programming Interface (API), which allows applications to be developed in Mac OS X, Linux and Windows operating systems.

<span class="mw-page-title-main">Cycling '74</span> American software development company

Cycling '74 is an American software development company founded in 1997 by David Zicarelli, headquartered in San Francisco, California and owned by Ableton. The company employs the digital signal processing software tool, Max.

<span class="mw-page-title-main">Making Waves (software)</span>

Making Waves (MW) is computer software designed to produce professional quality audio from basic Windows multimedia PCs. This application was among the first of the 16-bit digital sequencers that evolved from the MS-DOS WAV trackers of the Eighties to become the digital audio workstation software available today including Steinberg Cubase, Pro Tools and ACID Pro. Making Waves enabled a small community of independent artists to use existing hardware to record, sample, mix and render their own original work creating professional-quality audio with a modest investment of less than $100. This same dynamic user community played a significant role in the application's development, suggesting program revisions and performing extensive beta testing. These users were all organized and mentored by Stephen John Steele, the original programmer and developer of Making Waves as well as a founding director of Perceptive Solutions, Spacehead Systems and Making Waves Software Limited.

<span class="mw-page-title-main">Bedroom production</span>

A bedroom producer is an amateur musician who creates, performs, and records their music independently using a home studio, often considered a hobbyist opposed to a professional record producer in the recording industry that works in a traditional studio with clients. Typically bedroom producers use accessible digital technology that costs less than the equipment in a professional studio, such as MIDI controller-based instruments and virtual studio technology, to create music for release to the world. While a professional record producer oversees and guides the recording process, often working alongside multiple people such as studio musicians, singers, engineers, mixers, songwriters, arrangers, and orchestrators, a bedroom producer does everything independently: creating the ideas, recording them and processing them for release. Bedroom producers are often self-taught, learning sound design, mixing and music theory by reading music production blogs and watching tutorials on the internet. As bedroom producers depend on the accessibility of music technology, bedroom production has been made easier with advances in home computing power and digital audio workstations (DAW).

References

  1. Schiettecatte, Bert (24 April 2004). Written at Vienna, Austria. Interaction Design for Electronic Musical Interfaces . CHI 2004 Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery. p. 1549. doi:10.1145/985921.986125. ISBN   978-1-58113-703-3 . Retrieved 15 May 2011.
  2. "AudioCubes at MUHKA, Antwerp". www.percussa.com. 4 December 2004. Archived from the original on 17 May 2011. Retrieved 15 May 2011.
  3. "AudioCubes at Namm 2007". www.percussa.com. 30 January 2007. Archived from the original on 3 October 2011. Retrieved 16 May 2011.
  4. "Guide to Audiocubes | Sound Genetics". soundgenetics.com. 5 October 2020. Retrieved 20 September 2023.
  5. 1 2 Bentall, Robert (25 February 2019). "Composing". British Art Studies (11). doi: 10.17658/issn.2058-5462/issue-11/composing . ISSN   2058-5462.
  6. "Mark Mosher website". markmoshermusic.com. Archived from the original on 9 July 2023. Retrieved 15 May 2011.
  7. "AudioCubes artists". www.percussa.com. Archived from the original on 29 April 2011. Retrieved 15 May 2011.
  8. "AudioCubes designer @ TEDx Mediterranean". www.percussa.com. 2 November 2010. Archived from the original on 10 September 2015. Retrieved 21 January 2022.