Sound and music computing (SMC) is a research field that studies the whole sound and music communication chain from a multidisciplinary point of view. By combining scientific, technological and artistic methodologies it aims at understanding, modeling and generating sound and music through computational approaches.
The Sound and Music Computing research field can be traced back to the 1950s, when a few experimental composers, together with some engineers and scientists, independently and in different parts of the world, began exploring the use of the new digital technologies for music applications. Since then the SMC research field has had a fruitful history and different terms have been used to identify it. Computer Music and Music Technology might be the terms that have been used the most, "Sound and Music Computing" being a more recent term. In 1974, the research community established the International Computer Music Association and the International Computer Music Conference. In 1977 the Computer Music Journal was founded. The Center for Computer Research in Music and Acoustics (CCRMA) at Stanford University was created in the early 1970s and the Institute for Research and Coordination Acoustic/Music (IRCAM) in Paris in the late 1970s.
The Sound and Music Computing term was first proposed in the mid 1990s [1] and it was included in the ACM Computing Classification System. Using this name, in 2004 the Sound and Music Computing Conference was started and also in 2004 a roadmapping initiative was funded by the European Commission that resulted in the SMC Roadmap [2] and in the Sound and Music Computing Summer School.
With increasing research specialization within the SMC field, a number of focused conferences have been created. Particularly relevant are the International Conference on Digital Audio Effects, established in 1998, the International Conference on Music Information Retrieval (ISMIR), established in 2000, and the International Conference on New Interfaces for Musical Expression (NIME), established in 2001.
The current SMC research field can be grouped into a number of subfields that focus on specific aspects of the sound and music communication chain.
SMC research is a field driven by applications. Examples of applications are:
Audio signal processing is a subfield of signal processing that is concerned with the electronic manipulation of audio signals. Audio signals are electronic representations of sound waves—longitudinal waves which travel through air, consisting of compressions and rarefactions. The energy contained in audio signals or sound power level is typically measured in decibels. As audio signals may be represented in either digital or analog format, processing may occur in either domain. Analog processors operate directly on the electrical signal, while digital processors operate mathematically on its digital representation.
IRCAM is a French institute dedicated to the research of music and sound, especially in the fields of avant garde and electro-acoustical art music. It is situated next to, and is organisationally linked with, the Centre Pompidou in Paris. The extension of the building was designed by Renzo Piano and Richard Rogers. Much of the institute is located underground, beneath the fountain to the east of the buildings.
Music information retrieval (MIR) is the interdisciplinary science of retrieving information from music. Those involved in MIR may have a background in academic musicology, psychoacoustics, psychology, signal processing, informatics, machine learning, optical music recognition, computational intelligence, or some combination of these.
Sonification is the use of non-speech audio to convey information or perceptualize data. Auditory perception has advantages in temporal, spatial, amplitude, and frequency resolution that open possibilities as an alternative or complement to visualization techniques.
The following outline is provided as an overview of and topical guide to human–computer interaction:
Human-centered computing (HCC) studies the design, development, and deployment of mixed-initiative human-computer systems. It is emerged from the convergence of multiple disciplines that are concerned both with understanding human beings and with the design of computational artifacts. Human-centered computing is closely related to human-computer interaction and information science. Human-centered computing is usually concerned with systems and practices of technology use while human-computer interaction is more focused on ergonomics and the usability of computing artifacts and information science is focused on practices surrounding the collection, manipulation, and use of information.
New Interfaces for Musical Expression, also known as NIME, is an international conference dedicated to scientific research on the development of new technologies and their role in musical expression and artistic performance.
Eduardo Reck Miranda is a Brazilian composer of chamber and electroacoustic pieces but is most notable in the United Kingdom for his scientific research into computer music, particularly in the field of human-machine interfaces where brain waves will replace keyboards and voice commands to permit the disabled to express themselves musically.
Computational auditory scene analysis (CASA) is the study of auditory scene analysis by computational means. In essence, CASA systems are "machine listening" systems that aim to separate mixtures of sound sources in the same way that human listeners do. CASA differs from the field of blind signal separation in that it is based on the mechanisms of the human auditory system, and thus uses no more than two microphone recordings of an acoustic environment. It is related to the cocktail party problem.
Computer audition (CA) or machine listening is the general field of study of algorithms and systems for audio interpretation by machines. Since the notion of what it means for a machine to "hear" is very broad and somewhat vague, computer audition attempts to bring together several disciplines that originally dealt with specific problems or had a concrete application in mind. The engineer Paris Smaragdis, interviewed in Technology Review, talks about these systems — "software that uses sound to locate people moving through rooms, monitor machinery for impending breakdowns, or activate traffic cameras to record accidents."
Elizabeth D. "Beth" Mynatt is the Dean of the Khoury College of Computer Sciences at Northeastern University. She is former executive director of the Institute for People and Technology, director of the GVU Center at Georgia Tech, and Regents' and Distinguished Professor in the School of Interactive Computing, all at the Georgia Institute of Technology. In 2024, she was elected into the prestigious American Academy of Arts and Sciences.
An audio engineer helps to produce a recording or a live performance, balancing and adjusting sound sources using equalization, dynamics processing and audio effects, mixing, reproduction, and reinforcement of sound. Audio engineers work on the "technical aspect of recording—the placing of microphones, pre-amp knobs, the setting of levels. The physical recording of any project is done by an engineer…"
Stanford University has many centers and institutes dedicated to the study of various specific topics. These centers and institutes may be within a department, within a school but across departments, an independent laboratory, institute or center reporting directly to the dean of research and outside any school, or semi-independent of the university itself.
Human–computer interaction (HCI) is research in the design and the use of computer technology, which focuses on the interfaces between people (users) and computers. HCI researchers observe the ways humans interact with computers and design technologies that allow humans to interact with computers in novel ways. A device that allows interaction between human being and a computer is known as a "Human-computer Interface".
Computational musicology is an interdisciplinary research area between musicology and computer science. Computational musicology includes any disciplines that use computation in order to study music. It includes sub-disciplines such as mathematical music theory, computer music, systematic musicology, music information retrieval, digital musicology, sound and music computing, and music informatics. As this area of research is defined by the tools that it uses and its subject matter, research in computational musicology intersects with both the humanities and the sciences. The use of computers in order to study and analyze music generally began in the 1960s, although musicians have been using computers to assist them in the composition of music beginning in the 1950s. Today, computational musicology encompasses a wide range of research topics dealing with the multiple ways music can be represented.
Sonic interaction design is the study and exploitation of sound as one of the principal channels conveying information, meaning, and aesthetic/emotional qualities in interactive contexts. Sonic interaction design is at the intersection of interaction design and sound and music computing. If interaction design is about designing objects people interact with, and such interactions are facilitated by computational means, in sonic interaction design, sound is mediating interaction either as a display of processes or as an input medium.
The International Society for Music Information Retrieval (ISMIR) is an international forum for research on the organization of music-related data. It started as an informal group steered by an ad hoc committee in 2000 which established a yearly symposium - whence "ISMIR", which meant International Symposium on Music Information Retrieval. It was turned into a conference in 2002 while retaining the acronym. ISMIR was incorporated in Canada on July 4, 2008.
The Sound and Music Computing (SMC) Conference is the forum for international exchanges around the core interdisciplinary topics of Sound and Music Computing. The conference is held annually to facilitate the exchange of ideas in this field.
Xavier Serra is a researcher in the field of Sound and Music Computing and professor at the Pompeu Fabra University (UPF) in Barcelona. He is the founder and director of the Music Technology Group at the UPF.
Stefania Serafin is a professor at the Department of Architecture, Design and Media technology at Aalborg University in Copenhagen.