Haile (robot)

Last updated September 27, 2023

Haile (pronounced Hi-lee) is a robot percussionist developed by the Georgia Institute of Technology that listens to music in real time and creates an accompanying beat. The robot was designed in 2006 by Georgia Tech's professor of musical technology, Gil Weinberg. He and one of his graduate students, Scott Driscoll, created the robot to be able to "listen like a human, [and] improvise like a robot" (Weinberg).^[1] Haile "listens" through a microphone mounted on the drum and analyzes the sound, separating it into beats, rhythms, pitches and several other qualities. Detecting changes in these qualities helps Haile to assume either a leading or following style of play, roles that define the robot's collaborative abilities. Haile was also the first robot to create an acoustic percussion experience rather than play music through speakers. Its anthropomorphic design, which gives it movable arms that can move in any direction, allow it to create this acoustic music.

Goals and purpose

Driscoll's initial goal for making Haile was to combine the use of auditory input and robotics to create a musical experience that would lead to further human–robot interaction. The final goal was a robot that could translate live music into an acoustic performance that could implement and transcend human capabilities. Haile wasn't designed to replace human musicians, but rather to accompany them with expressive playing.^[2]

These goals led to Weinberg wanting to create an acoustic musical experience. His earlier experiments failed to incorporate the visual or auditory aspects associated with acoustic music. Haile's functional drumming arms add musical cues (visually stimulating bouncing drum sticks and live, acoustic sounds) that other robot performances lack.^[1] Additionally, other attempts at percussion playing robots, Weinberg saw, were limited in the variety of beats that they could produce. Haile is not only preloaded with individual beats, but is also programmed to identify pitch, rhythm, and patterns, allowing it to improvise and play different beats every time, rather than simply mimic.^[3]

Design

Haile's anthropomorphic or human-like design mimics human movements which supports interactive play with other musicians. Its two robotic arms are responsible for creating different sounds; the right arm plays faster notes, while the left arm has larger motions for louder and slower beats. While other robotic drummers at the time were limited to playing only a few locations on the drum, Haile is versatile in its ability to play along a straight line from the rim to the center of the drum.^[1]

Form

Haile's wooden designed was modeled to match the natural feel of a Native American pow wow (Native American gathering), so it was made out of wood rather than metal. The wooden parts were made at the Advanced Wood Product Laboratory Archived 2014-11-24 at the Wayback Machine at Georgia Tech's College of Architecture using a CNC wood router. It was originally designed to play a pow wow drum, a multiplayer drum that supports Haile's collaborative purpose. However, it also was made with metal joints that give it adjustable height so it can play other drums. These joints are the keys to the robot's arm movement up and down, left and right, and front and back. If needed, they detach, allowing for full disassembly.^[4]

Perception

Haile uses a microphone on the drum which first detected rhythms played by a human in real time. The robot identifies tempo and beats allowing it to play along with the other player. It can also adjust to the human's changes in volume, tempo or beat, allowing it to switch between accompanying and lead playing.^[5] Weinberg and his team first developed the robot's low-level perception ability, which includes detecting hit onsets, pitch, amplitude, and density. In terms of sound, a hit refers to a distinct change in both volume and sound quality. Once the outside music is captured, the sound is analyzed through a number of instruments, called perception modules, which each detect a certain aspect of the sound:

Pitch – detects hits and changes in frequency and translates them to find pitches^[3]
Beat – processes onsets and determines rhythms and tempo^[3]
Amplitude – recognizes changes in volume to determine when to assume leading or following roles^[3]
Density – detects changes in rhythm complexity at tempo to also help Haile assume leading or following roles^[6]

Arm mechanics

Haile's arms are driven by two separate means. The left arm uses a linear motor which is responsible for larger movements that correspond to louder sounds. The linear motor, along with a linear encoder is used to control the robot's arm height. The larger motions are louder and more visible, but limit the arm to swinging at a top speed of 11 Hz. The right arm, which plays softer and faster notes, is driven by a solenoid, which uses an aluminum stick and return spring capable of hitting the drum at up to 15 Hz. Both arms are capable of hitting anywhere (from the rim to the center) on the drum through the use of a linear slide, which allows each of them completely independent movement in a single direction.

Playing

Haile's system adopts a leader-follower model, using tempo and beat changes to determine who the current leader is. Haile understands when a new leader emerges based on musical changes (tempo, volume, beat, etc.).^[5] The robot has two modes of play:

As a follower, Haile first analyzes the external music. It then matches and maintains the tempo, allowing the human player to play more complicated rhythms. Haile can also tell when the other player begins to play louder or more quickly, forcing it into the submissive role. When the human beings to play basic rhythms at a steady tempo, the robot takes the lead.^[5]
As a leader, Haile uses rhythms produced earlier by the human and improvises a rhythm with its right arm. The left arm detects and maintains the other player's tempo.^[5]

Challenges

Some of the challenges that Weinberg faced with Haile's programming involved being able to distinguish between different, simultaneous sounds. Initially, analysis algorithms were unable to pick out softer and more subtle notes amidst louder sounds. Also, the inability to filter out ambient noise prevented Haile from working properly. After a considerable amount of adjusting, the filters and input hardware were tuned to differentiate between various volumes of music while ignoring interfering noise.^[2]

As Haile was designed to play in either leading or following roles, early detection algorithms limited the human's ability to lead. The robot was designed to be able to detect changes in the music it heard, but would only respond to changes in tempo. This flaw only allowed the human to lead as long as he or she kept speeding up or slowing down. Weinberg, trying to model human musical interaction, implemented volume and noise density sensors to aid in the robot's ability to define leadership. These additions gave the human longer periods of leadership, giving Haile more opportunity to build upon what it heard.^[5]

Related Research Articles

A drum kit is a collection of drums, cymbals, and sometimes other auxiliary percussion instruments set up to be played by one person. The drummer typically holds a pair of matching drumsticks, and uses their feet to operate hi-hat and bass drum pedals.

<span class="mw-page-title-main">Percussion instrument</span> Type of musical instrument that produces a sound by being hit

A percussion instrument is a musical instrument that is sounded by being struck or scraped by a beater including attached or enclosed beaters or rattles struck, scraped or rubbed by hand or struck against another similar instrument. Excluding zoomusicological instruments and the human voice, the percussion family is believed to include the oldest musical instruments. In spite of being a very common term to designate instruments, and to relate them to their players, the percussionists, percussion is not a systematic classificatory category of instruments, as described by the scientific field of organology. It is shown below that percussion instruments may belong to the organological classes of idiophone, membranophone, aerophone and cordophone.

Rhythm generally means a "movement marked by the regulated succession of strong and weak elements, or of opposite or different conditions". This general meaning of regular recurrence or pattern in time can apply to a wide variety of cyclical natural phenomena having a periodicity or frequency of anything from microseconds to several seconds ; to several minutes or hours, or, at the most extreme, even over many years.

<span class="mw-page-title-main">Electronic keyboard</span> Musical instrument

An electronic keyboard, portable keyboard, or digital keyboard is an electronic musical instrument based on keyboard instruments. Electronic keyboards include synthesizers, digital pianos, stage pianos, electronic organs and digital audio workstations. In technical terms, an electronic keyboard is a synthesizer with a low-wattage power amplifier and small loudspeakers.

A variety of musical terms are encountered in printed scores, music reviews, and program notes. Most of the terms are Italian, in accordance with the Italian origins of many European musical conventions. Sometimes, the special musical meanings of these phrases differ from the original or current Italian meanings. Most of the other terms are taken from French and German, indicated by Fr. and Ger., respectively.

<span class="mw-page-title-main">Cajón</span> Box-shaped percussion instrument

A cajón is a box-shaped percussion instrument originally from Peru, played by slapping the front or rear faces with the hands, fingers, or sometimes implements such as brushes, mallets, or sticks. Cajones are primarily played in Afro-Peruvian music, but have made their way into flamenco as well. The term cajón is also applied to other box drums used in Latin American music, such as the Cuban cajón de rumba and the Mexican cajón de tapeo.

A rhythm section is a group of musicians within a music ensemble or band that provides the underlying rhythm, harmony and pulse of the accompaniment, providing a rhythmic and harmonic reference and "beat" for the rest of the band. The rhythm section is often contrasted with the roles of other musicians in the band, such as the lead guitarist or lead vocals whose primary job is to carry the melody.

Sextet is a composition by American composer Steve Reich. The piece was written and first performed in 1984, and slightly revised in 1985.

In music, transcription is the practice of notating a piece or a sound which was previously unnotated and/or unpopular as a written music, for example, a jazz improvisation or a video game soundtrack. When a musician is tasked with creating sheet music from a recording and they write down the notes that make up the piece in music notation, it is said that they created a musical transcription of that recording. Transcription may also mean rewriting a piece of music, either solo or ensemble, for another instrument or other instruments than which it was originally intended. The Beethoven Symphonies transcribed for solo piano by Franz Liszt are an example. Transcription in this sense is sometimes called arrangement, although strictly speaking transcriptions are faithful adaptations, whereas arrangements change significant aspects of the original piece.

The pakhavaj is a barrel-shaped, two-headed drum, originating from the Indian subcontinent, the oldest version of double sided drums and its descendants are mridangam of Southern India and kendang of Maritime Southeast Asia and other South Asian double-headed drums. Its older forms were made with clay.

Ewe drumming refers to the drumming ensembles of the Ewe people of Ghana, Togo, and Benin. The Ewe are known for their experience in drumming throughout West Africa. The sophisticated cross rhythms and polyrhythms in Ewe drumming are similar to those in Afro-Caribbean music and late jazz. The original purpose of Ewe drumming were sung or performed by warriors. Now the songs and performed to celebrate or for recreational use. For example, Agbadza was originally used as a warrior dance but is now used to celebrate events.

Orchestral enhancement is the technique of using orchestration techniques, architectural modifications, or electronic technologies to modify the sound, complexity, or color of a musical theatre, ballet or opera pit orchestra. Orchestral enhancements are used both to create new sounds and to add capabilities to existing orchestral ensembles.

Samba-reggae is a music genre from Bahia, Brazil. Samba reggae, as its name suggests, was originally derived as a blend of Brazilian samba with Jamaican reggae as typified by Bob Marley.

Gil Weinberg is an Israeli-born American musician and inventor of experimental musical instruments and musical robots. Weinberg is a professor of musical technology at Georgia Tech and founding director of the Georgia Tech Center for Music Technology.

<span class="mw-page-title-main">Sound</span> Vibration that travels via pressure waves in matter

In physics, sound is a vibration that propagates as an acoustic wave, through a transmission medium such as a gas, liquid or solid. In human physiology and psychology, sound is the reception of such waves and their perception by the brain. Only acoustic waves that have frequencies lying between about 20 Hz and 20 kHz, the audio frequency range, elicit an auditory percept in humans. In air at atmospheric pressure, these represent sound waves with wavelengths of 17 meters (56 ft) to 1.7 centimeters (0.67 in). Sound waves above 20 kHz are known as ultrasound and are not audible to humans. Sound waves below 20 Hz are known as infrasound. Different animal species have varying hearing ranges.

<span class="mw-page-title-main">Ghost note</span> Musical note with a rhythmic value, but no discernible pitch

In music, notably in jazz, a ghost note is a musical note with a rhythmic value, but no discernible pitch when played. In musical notation, this is represented by an "X" for a note head instead of an oval, or parentheses around the note head. It should not be confused with the X-shaped notation that raises a note to a double sharp.

Psychoacoustics is the branch of psychophysics involving the scientific study of sound perception and audiology—how human auditory system perceives various sounds. More specifically, it is the branch of science studying the psychological responses associated with sound. Psychoacoustics is an interdisciplinary field of many areas, including psychology, acoustics, electronic engineering, physics, biology, physiology, and computer science.

This is a glossary of jazz and popular music terms that are likely to be encountered in printed popular music songbooks, fake books and vocal scores, big band scores, jazz, and rock concert reviews, and album liner notes. This glossary includes terms for musical instruments, playing or singing techniques, amplifiers, effects units, sound reinforcement equipment, and recording gear and techniques which are widely used in jazz and popular music. Most of the terms are in English, but in some cases, terms from other languages are encountered.

Heavy metal drumming is a style of rock music drum kit playing that developed in the late 1960s and early 1970s, largely in the United States and the United Kingdom. With roots in blues rock and psychedelic/acid rock drum playing, heavy metal drummers play with emphatic beats, and overall loudness using an aggressive performing style. Heavy metal drumming is traditionally characterized by emphatic rhythms and dense bass guitar-and-drum sound. The essence of metal drumming is creating a loud, constant beat for the band using the "trifecta of speed, power, and precision".

Okho is a composition for three percussionists by Iannis Xenakis completed in 1989.

References

1 2 3 Weinberg, Gil; Driscoll, Scott (2007). "The design of a robotic marimba player". Proceedings of the 7th international conference on New interfaces for musical expression – NIME '07. p. 228. doi:10.1145/1279740.1279786. ISBN 978-1-4503-7837-6. S2CID 6727904.
1 2 Weinberg, Gil; Driscoll, Scott; Parry, Mitchell (2005). "Haile-An Interactive Robotic Percussionist" (PDF). International Computer Music Conference Proceedings. 2005. hdl:2027/spo.bbp2372.2005.069. Archived from the original (PDF) on 2016-03-05. Retrieved November 8, 2014.
1 2 3 4 Abshire, Matthew. "Musical robot composes, performs, and teaches". CNN.com. Retrieved October 26, 2014.
↑ Weinberg, Gil; Driscoll, Scott (22 April 2006). "Robot-human interaction with an anthropomorphic percussionist". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1229–1232. doi:10.1145/1124772.1124957. ISBN 978-1-59593-372-0. S2CID 10301135.
1 2 3 4 5 Weinberg, Gil; Blosser, Brian (9 March 2009). "A leader-follower turn-taking model incorporating beat detection in musical human-robot interaction". Proceedings of the 4th ACM/IEEE international conference on Human robot interaction. pp. 227–228. doi:10.1145/1514095.1514149. ISBN 978-1-60558-404-1. S2CID 1200705.
↑ Weinberg, Gil; Driscoll, Scott (10 March 2007). "The interactive robotic percussionist: New developments in form, mechanics, perception and interaction design". Proceedings of the ACM/IEEE international conference on Human-robot interaction. pp. 97–104. doi:10.1145/1228716.1228730. ISBN 978-1-59593-617-2. S2CID 130547.

External links

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[four-1] 1 2 3 Weinberg, Gil; Driscoll, Scott (2007). "The design of a robotic marimba player". Proceedings of the 7th international conference on New interfaces for musical expression – NIME '07. p. 228. doi:10.1145/1279740.1279786. ISBN 978-1-4503-7837-6. S2CID 6727904.

[one-2] 1 2 Weinberg, Gil; Driscoll, Scott; Parry, Mitchell (2005). "Haile-An Interactive Robotic Percussionist" (PDF). International Computer Music Conference Proceedings. 2005. hdl:2027/spo.bbp2372.2005.069. Archived from the original (PDF) on 2016-03-05. Retrieved November 8, 2014.

[two-3] 1 2 3 4 Abshire, Matthew. "Musical robot composes, performs, and teaches". CNN.com. Retrieved October 26, 2014.

[three-4] Weinberg, Gil; Driscoll, Scott (22 April 2006). "Robot-human interaction with an anthropomorphic percussionist". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1229–1232. doi:10.1145/1124772.1124957. ISBN 978-1-59593-372-0. S2CID 10301135.

[five-5] 1 2 3 4 5 Weinberg, Gil; Blosser, Brian (9 March 2009). "A leader-follower turn-taking model incorporating beat detection in musical human-robot interaction". Proceedings of the 4th ACM/IEEE international conference on Human robot interaction. pp. 227–228. doi:10.1145/1514095.1514149. ISBN 978-1-60558-404-1. S2CID 1200705.

[six-6] Weinberg, Gil; Driscoll, Scott (10 March 2007). "The interactive robotic percussionist: New developments in form, mechanics, perception and interaction design". Proceedings of the ACM/IEEE international conference on Human-robot interaction. pp. 97–104. doi:10.1145/1228716.1228730. ISBN 978-1-59593-617-2. S2CID 130547.

[1]

[2]

[3]

[4]

[5]

[6]