Computational musicology

Last updated

Computational musicology is an interdisciplinary research area between musicology and computer science. [1] Computational musicology includes any disciplines that use computation in order to study music. It includes sub-disciplines such as mathematical music theory, computer music, systematic musicology, music information retrieval, digital musicology, sound and music computing, and music informatics. [2] As this area of research is defined by the tools that it uses and its subject matter, research in computational musicology intersects with both the humanities and the sciences. The use of computers in order to study and analyze music generally began in the 1960s, [3] although musicians have been using computers to assist them in the composition of music beginning in the 1950s. Today, computational musicology encompasses a wide range of research topics dealing with the multiple ways music can be represented. [4]

Contents

History

This history of computational musicology generally began in the middle of the 20th century. Generally, the field is considered to be an extension of a much longer history of intellectual inquiry in music that overlaps with science, mathematics, technology, [5] and archiving.

1960s

Early approaches to computational musicology began in the early 1960s and were being fully developed by 1966. [6] [3] At this point in time data entry was done primarily with paper tape or punch cards [3] and was computationally limited. Due to the high cost of this research, in order to be funded projects often tended to ask global questions and look for global solutions. [3] One of the earliest symbolic representation schemes was the Digital Alternate Representations of Music or DARMS. The project was supported by Columbia University and the Ford Foundation between 1964 and 1976. [7] The project was one of the initial large scale projects to develop an encoding scheme that incorporated completeness, objectivity, and encoder-directedness. [7] Other work at this time at Princeton University chiefly driven by Arthur Mendel, and implemented by Michael Kassler [8] and Eric Regener helped push forward the Intermediary Musical Language (IML) and Music Information Retrieval (MIR) languages that later fell out of popularity in the late 1970s. The 1960s also marked a time of documenting bibliographic initiatives such as the Repertoire International de Literature Musicale (RILM) created by Barry Brook in 1967.

1970s

Unlike the global research interests of the 1960s, goals in computational musicology in the 1970s were driven by accomplishing certain tasks. [3] This task driven motivation lead to the development of MUSTRAN for music analysis by led by Jerome Wenker and Dorothy Gross at Indiana University. Similar projects like SCORE (SCORE-MS) at Stanford University was developed primarily for printing purposes.

1980s

The 1980s were the first decade to move away from centralized computing and move towards that of personalized computing. This transference of resources led to growth in the field as a whole. John Walter Hill began developing a commercial program called Savy PC that was meant to help musicologists analyze lyrical content in music. Findings from Hill's music were able to find patterns in the conversions of sacred and secular texts where only first lines of texts were changed. [3] In keeping with the global questions that dominated the 1960s, Helmuth Schaffrath began his Essen Folk Collection encoded in Essen Associative Code (ESAC) which has since been converted to humdrum notation. [9] Using software developed at the time, Sandra Pinegar examined 13th century music theory manuscripts in her doctoral work at Columbia University in order to gain evidence on the dating and authoring of texts. [10] The 1980s also introduced MIDI notation.

Methods

Computational musicology can be generally divided into the three main branches relating to the three ways music can be represented by a computer: sheet music data, symbolic data, and audio data. Sheet music data refers to the human-readable, graphical representation of music via symbols. Examples of this branch of research would include digitizing scores ranging from 15th Century neumenal notation to contemporary Western music notation. Like sheet music data, symbolic data refers to musical notation in a digital format, but symbolic data is not human readable and is encoded in order to be parsed by a computer. Examples of this type of encoding include piano roll, kern, [11] and MIDI representations. Lastly, audio data refers to recording of the representations of the acoustic wave or sound that results from changes in the oscillations of air pressure. [12] Examples of this type of encoding include MP3 or WAV files.

Sheet Music Data

Sheet music is meant to be read by the musician or performer. Generally, the term refers to the standardized nomenclature used by a culture to document their musical notation. In addition to music literacy, musical notation also demands choices from the performer. For example, the notation of Hindustani ragas will begin with an alap that does not demand a strict adherence to a beat or pulse, but is left up to the discretion of the performer. [13] The sheet music notation captures the sequence of gestures the performer is encouraged to make within a musical culture, but is by no means fixed to those performance choices.

Symbolic Data

Symbolic data refers to musical encoding that is able to be parsed by a computer. Unlike sheet music data, Any type of digital data format may be regarded as symbolic due to the fact that the system that is representing it is generated from a finite series of symbols. Symbolic data typically does not have any sort of performative choices required on the part of the performer. [4] Two of the most common software choices for analyzing symbolic data are David Huron's Humdrum Toolkit [14] and Michael Scott Cuthbert's music21. [15]

Audio Data

Audio data is generally conceptualized as existing on a continuum of features ranging from lower to higher level audio features. Low-level audio features refer to loudness, spectral flux, and cepstrum. Mid-level audio features refer to pitch, onsets, and beats. Examples of high-level audio features include style, artist, mood, and key. [16]

Applications

Music databases

One of the earliest applications in computational musicology was the creation and use of musical databases. Input, usage and analysis of large amounts of data can be very troublesome using manual methods while usage of computers can make such tasks considerably easier.

Analysis of music

Different computer programs have been developed to analyze musical data. Data formats vary from standard notation to raw audio. Analysis of formats that are based on storing all properties of each note, for example MIDI, were used originally and are still among the most common methods. Significant advances in analysis of raw audio data have been made only recently.

Artificial production of music

Different algorithms can be used to both create complete compositions and improvise music. One of the methods by which a program can learn improvisation is analysis of choices a human player makes while improvising. Artificial neural networks are used extensively in such applications.

Historical change and music

One developing sociomusicological theory in computational musicology is the "Discursive Hypothesis" proposed by Kristoffer Jensen and David G. Hebert, which suggests that "because both music and language are cultural discourses (which may reflect social reality in similarly limited ways), a relationship may be identifiable between the trajectories of significant features of musical sound and linguistic discourse regarding social data." [17] According to this perspective, analyses of "big data" may improve our understandings of how particular features of music and society are interrelated and change similarly across time, as significant correlations are increasingly identified within the musico-linguistic spectrum of human auditory communication. [18]

Non-western music

Strategies from computational musicology are recently being applied for analysis of music in various parts of the world. For example, professors affiliated with the Birla Institute of Technology in India have produced studies of harmonic and melodic tendencies (in the raga structure) of Hindustani classical music. [19]

Research

RISM's (Répertoire International des Sources Musicales) database is one of the world's largest music databases, containing over 700,000 references to musical manuscripts. Anyone can use its search engine to find compositions. [20]

The Centre for History and Analysis of Recorded Music (CHARM) has developed the Mazurka Project, [21] which offers "downloadable recordings . . . analytical software and training materials, and a variety of resources relating to the history of recording."

Research from computational musicology occasionally is the focus of popular culture and major news outlets. Examples of this include reporting in The New Yorker musicologists Nicholas Cook and Craig Sapp while working on the Centre for the History and Analysis of Recorded Music (CHARM), at the University of London discovered the fraudulent recording of pianist Joyce Hatto. [22] On the 334th birthday of Johann Sebastian Bach, Google celebrated the occasion with a Google Doodle that allowed individuals to enter their own score into the interface, then have a machine learning model called Coconet [23] harmonize the melody. [24]

See also

Related Research Articles

Musicology is the scholarly study of music. Musicology research combines and intersects with many fields, including psychology, sociology, acoustics, neurology, natural sciences, formal sciences and computer science.

Music history, sometimes called historical musicology, is a highly diverse subfield of the broader discipline of musicology that studies music from a historical point of view. In theory, "music history" could refer to the study of the history of any type or genre of music. In practice, these research topics are often categorized as part of ethnomusicology or cultural studies, whether or not they are ethnographically based. The terms "music history" and "historical musicology" usually refer to the history of the notated music of Western elites, sometimes called "art music".

<span class="mw-page-title-main">LilyPond</span> Free software scorewriter

LilyPond is a computer program and file format for music engraving. One of LilyPond's major goals is to produce scores that are engraved with traditional layout rules, reflecting the era when scores were engraved by hand.

<i>Raga</i> Melodic mode in Indian music

A raga is a melodic framework for improvisation in Indian classical music akin to a melodic mode. Rāga is a unique feature and central to the tradition of classical Indian music: no equivalent concept exists in classical European music. Each rāga is an array of melodic structures with musical motifs, considered in the Indian tradition to have the ability to "colour the mind" and affect the emotions of the audience.

<span class="mw-page-title-main">Indian classical music</span> Classical music from the Indian subcontinent

Indian classical music is the classical music of the Indian subcontinent. It is generally described using terms like Marg Sangeet and Shastriya Sangeet. It has two major traditions: the North Indian classical music known as Hindustani and the South Indian expression known as Carnatic. These traditions were not distinct until about the 15th century. During the period of Mughal rule of the Indian subcontinent, the traditions separated and evolved into distinct forms. Hindustani music emphasizes improvisation and exploration of all aspects of a raga, while Carnatic performances tend to be short composition-based. However, the two systems continue to have more common features than differences. Another unique classical music tradition from Eastern part of India, i. e. Odissi music has evolved since two thousand years ago.

Music information retrieval (MIR) is the interdisciplinary science of retrieving information from music. Those involved in MIR may have a background in academic musicology, psychoacoustics, psychology, signal processing, informatics, machine learning, optical music recognition, computational intelligence or some combination of these.

Optical music recognition (OMR) is a field of research that investigates how to computationally read musical notation in documents. The goal of OMR is to teach the computer to read and interpret sheet music and produce a machine-readable version of the written music score. Once captured digitally, the music can be saved in commonly used file formats, e.g. MIDI and MusicXML . In the past it has, misleadingly, also been called "music optical character recognition". Due to significant differences, this term should no longer be used.

Computer audition (CA) or machine listening is the general field of study of algorithms and systems for audio interpretation by machines. Since the notion of what it means for a machine to "hear" is very broad and somewhat vague, computer audition attempts to bring together several disciplines that originally dealt with specific problems or had a concrete application in mind. The engineer Paris Smaragdis, interviewed in Technology Review, talks about these systems — "software that uses sound to locate people moving through rooms, monitor machinery for impending breakdowns, or activate traffic cameras to record accidents."

Music informatics is a study of music processing, in particular music representations, fourier analysis of music, music synchronization, music structure analysis and chord recognition. Other music informatics research topics include computational music modeling, computational music analysis, optical music recognition, digital audio editors, online music search engines, music information retrieval and cognitive issues in music. Because music informatics is an emerging discipline, it is a very dynamic area of research with many diverse viewpoints, whose future is yet to be determined.

<span class="mw-page-title-main">Shivaranjani</span> A Janya raga of Carnatic music, also used in Hindustani classical music

Shivaranjani or Sivaranjani is a musical scale used in Indian classical music. There are two scales, one in Hindustani music and one in Carnatic music. The Hindustani rāga is a pentatonic scale, as is the Carnatic scale categorized as Audava-Audava resulting in 5 notes in the Arohanam and 5 in the Avarohanam.

The Music Encoding Initiative (MEI) is an open-source effort to create a system for representation of musical documents in a machine-readable structure. MEI closely mirrors work done by text scholars in the Text Encoding Initiative (TEI) and while the two encoding initiatives are not formally related, they share many common characteristics and development practices. The term "MEI", like "TEI", describes the governing organization and the markup language. The MEI community solicits input and development directions from specialists in various music research communities, including technologists, librarians, historians, and theorists in a common effort to discuss and define best practices for representing a broad range of musical documents and structures. The results of these discussions are then formalized into the MEI schema, a core set of rules for recording physical and intellectual characteristics of music notation documents. This schema is expressed in an XML schema Language, with RelaxNG being the preferred format. The MEI schema is developed using the One-Document-Does-it-all (ODD) format, a literate programming XML format developed by the Text Encoding Initiative.

Cognitive musicology is a branch of cognitive science concerned with computationally modeling musical knowledge with the goal of understanding both music and cognition.

GUIDO Music Notation is a computer music notation format designed to logically represent all aspects of music in a manner that is both computer-readable and easily readable by human beings. It was named after Guido of Arezzo, who pioneered today's conventional musical notation 1,000 years ago.

Harmonic pitch class profiles (HPCP) is a group of features that a computer program extracts from an audio signal, based on a pitch class profile—a descriptor proposed in the context of a chord recognition system. HPCP are an enhanced pitch distribution feature that are sequences of feature vectors that, to a certain extent, describe tonality, measuring the relative intensity of each of the 12 pitch classes of the equal-tempered scale within an analysis frame. Often, the twelve pitch spelling attributes are also referred to as chroma and the HPCP features are closely related to what is called chroma features or chromagrams.

Sound and music computing (SMC) is a research field that studies the whole sound and music communication chain from a multidisciplinary point of view. By combining scientific, technological and artistic methodologies it aims at understanding, modeling and generating sound and music through computational approaches.

<span class="mw-page-title-main">International Society for Music Information Retrieval</span>

The International Society for Music Information Retrieval (ISMIR) is an international forum for research on the organization of music-related data. It started as an informal group steered by an ad hoc committee in 2000 which established a yearly symposium - whence "ISMIR", which meant International Symposium on Music Information Retrieval. It was turned into a conference in 2002 while retaining the acronym. ISMIR was incorporated in Canada on July 4, 2008.

<span class="mw-page-title-main">Xavier Serra</span> American music computing researcher

Xavier Serra is a researcher in the field of Sound and Music Computing and professor at the Pompeu Fabra University (UPF) in Barcelona. He is the founder and director of the Music Technology Group at the UPF.

<span class="mw-page-title-main">David Huron</span>

David Huron is a Canadian Arts and Humanities Distinguished Professor at the Ohio State University, in both the School of Music and the Center for Cognitive and Brain Sciences. His teaching and publications focus on the psychology of music and music cognition. In 2017, Huron was awarded the Society for Music Perception and Cognition Achievement Award.

<span class="mw-page-title-main">Music alignment</span>

Music can be described and represented in many different ways including sheet music, symbolic representations, and audio recordings. For each of these representations, there may exist different versions that correspond to the same musical work. The general goal of music alignment is to automatically link the various data streams, thus interrelating the multiple information sets related to a given musical work. More precisely, music alignment is taken to mean a procedure which, for a given position in one representation of a piece of music, determines the corresponding position within another representation. In the figure on the right, such an alignment is visualized by the red bidirectional arrows. Such synchronization results form the basis for novel interfaces that allow users to access, search, and browse musical content in a convenient way.

<span class="mw-page-title-main">Chroma feature</span>

In Western music, the term chroma feature or chromagram closely relates to twelve different pitch classes. Chroma-based features, which are also referred to as "pitch class profiles", are a powerful tool for analyzing music whose pitches can be meaningfully categorized and whose tuning approximates to the equal-tempered scale. One main property of chroma features is that they capture harmonic and melodic characteristics of music, while being robust to changes in timbre and instrumentation.

References

  1. ↑ "Unfolding the Potential of Computational Musicology" (PDF). Proceedings of the Thirteenth International Conference on Informatics and Semiotics in Organisations: Problems and Possibilities of Computational Humanities.
  2. ↑ Meredith, David (2016). "Preface". Computational Music Analysis. New York: Springer. p. v. ISBN   978-3319259291.
  3. 1 2 3 4 5 6 Hewlett, Walter B.; Selfridge-Field, Eleanor (1991). "Computing in Musicology, 1966-91". Computers and the Humanities. 25 (6): 381–392. doi:10.1007/BF00141188. JSTOR   30208121. S2CID   30204949.
  4. 1 2 Meinard, Müller (2015-07-21). Fundamentals of music processing : audio, analysis, algorithms, applications. Switzerland. ISBN   9783319219455. OCLC   918555094.{{cite book}}: CS1 maint: location missing publisher (link)
  5. ↑ Forte, Allen (1967). "Music and computing: the present situation". Computers and the Humanities. 2 (1): 32–35. doi:10.1007/BF02402463. JSTOR   30203948. S2CID   33681168.
  6. ↑ Berlind, Gary; Brook, Barry S.; Hiller, Lejaren A.; Larue, Jan P.; Logemann, George W. (Fall 1966). "Writings on the Use of Computers in Music". College Music Symposium. 6: 143–157. JSTOR   40373186.
  7. 1 2 Erickson, Raymond F. (1975). ""The Darms Project": A Status Report". Computers and the Humanities. 9 (6): 291–298. doi:10.1007/BF02396292. JSTOR   30204239. S2CID   62220033.
  8. ↑ "Papers of Jamie and Michael Kassler". Trove. Retrieved 2023-02-23.
  9. ↑ "ESAC Data Homepage". www.esac-data.org. Retrieved 2019-02-11.
  10. ↑ Pinegar, Sandra (1991). Textual and conceptual relationships among theoretical writings on measurable music of the thirteenth and early fourteenth centuries (PhD thesis). Columbia University. ProQuest   303944932.
  11. ↑ Huron, David (2002). "Music information processing using the Humdrum Toolkit: Concepts, examples, and lessons". Computer Music Journal. 26 (2): 11–26. doi:10.1162/014892602760137158. S2CID   25996361.
  12. ↑ Müller, Meinard (2015), "Music Representations", in Müller, Meinard (ed.), Fundamentals of Music Processing, Springer International Publishing, pp. 1–37, doi:10.1007/978-3-319-21945-5_1, ISBN   9783319219455
  13. ↑ The raga guide : a survey of 74 Hindustani ragas, Bor, Joep., Rao, Suvarnalata, 1954-, Meer, Wim van der., Harvey, Jane, 1949-, Chaurasia, Hariprasad., Das Gupta, Buddhadev, 1933-, Nimbus Records, 2002, ISBN   978-0954397609, OCLC   80291538 {{citation}}: CS1 maint: others (link)
  14. ↑ "The Humdrum Toolkit: Software for Music Research — humdrum-tools 1 documentation". www.humdrum.org. Retrieved 2019-03-20.
  15. ↑ Cuthbert, Michael Scott; Ariza, Christopher (August 2010). "music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data". In J. Stephen Downie; Remco C. Veltkamp (eds.). 11th International Society for Music Information Retrieval Conference (ISMIR 2010), August 9-13, 2010, Utrecht, Netherlands. International Society for Music Information Retrieval. pp. 637–642. hdl:1721.1/84963. ISBN   9789039353813.
  16. ↑ Pablo Bello, Juan. "Low-level features and timbre" (PDF). nyu.edu. Retrieved 2019-02-11.
  17. ↑ McCollum, Jonathan and Hebert, David (2014) Theory and Method in Historical Ethnomusicology Lanham, MD: Lexington Books / Rowman & Littlefield ISBN   0739168266; p.62. Some of Jensen and Hebert's pioneering findings from 2013 on tendencies in US Billboard Hot 100 songs have since been replicated and expanded upon by other scholars (e.g. Mauch M, MacCallum RM, Levy M, Leroi AM. 2015 The evolution of popular music: USA 1960–2010. R. Soc. Open sci. 2: 150081. https://dx.doi.org/10.1098/rsos.150081).
  18. ↑ Kristoffer Jensen and David G. Hebert (2016). Evaluation and Prediction of Harmonic Complexity Across 76 Years of Billboard 100 Hits. In R. Kronland-Martinet, M. Aramaki, and S. Ystad, (Eds.), Music, Mind, and Embodiment. Switzerland: Springer Press, pp.283-296. ISBN   978-3-319-46281-3.
  19. ↑ Chakraborty, S., Mazzola, G., Tewari, S., Patra, M. (2014) "Computational Musicology in Hindustani Music" New York: Springer.
  20. ↑ RISM database, <http://www.rism.info/>
  21. ↑ Mazurka Project, <http://mazurka.org.uk/>
  22. ↑ Singer, Mark (2007-09-10). "Fantasia for Piano". The New Yorker. ISSN   0028-792X . Retrieved 2019-03-23.
  23. ↑ Huang, Cheng-Zhi Anna; Cooijmans, Tim; Roberts, Adam; Courville, Aaron; Eck, Douglas (2019-03-17). "Counterpoint by Convolution". arXiv: 1903.07227 [cs.LG].
  24. ↑ "Coconet: The ML model behind today's Bach Doodle". magenta.tensorflow.org. 20 March 2019. Retrieved 2019-03-23.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.