Internet of Musical Things

Last updated

The Internet of Musical Things (also known as IoMusT) is a research area that aims to bring Internet of Things connectivity [1] [2] [3] to musical and artistic practices. Moreover, it encompasses concepts coming from music computing, ubiquitous music, [4] human-computer interaction, [5] [6] artificial intelligence, [7] augmented reality, virtual reality, gaming, participative art, [8] and new interfaces for musical expression. [9] From a computational perspective, IoMusT refers to local or remote networks embedded with devices capable of generating and/or playing musical content. [10] [11]

Contents

Introduction

The term "Internet of Things" (IoT) is extensible to any everyday object connected to the internet, having its capabilities increased by exchanging information with other elements present in the network to achieve a common goal. [1] [2] [3] Thanks to the technological advances that have occurred in the last decades, its use has spread to several areas of performance, assisting in medical analysis, traffic control and home security. When its concepts meet music, the Internet of Music Things (IoMusT) emerges.

The term "Internet of Musical Things" also receives numerous classifications, according to the use of certain authors. Hazzard et al., [12] for example, uses it in the context of musical instruments that have QR code that directs the user to a page with information about this instrument, such as manufacturing date and history. Keller and Lazzarini, [13] use this term in ubiquitous music (ubimus) research, while Turchet et al. [14] define IoMusT as a subfield of the Internet of Things, where interoperable devices can connect to each other, aiding the interaction between musicians and the audience.

Like the IoT, the Internet of Music Things can encompass a variety of ecosystems. But generally, it is marked by being employed in musical activities (rehearsals, concerts, recordings, and music teaching) and relying on service and information providers.

In addition to the technological and artistic advantages that this field offers, new opportunities are still arising for the music industry, providing the emergence of new services and applications capable of exploiting the interconnection between logical and physical devices, always keeping the artistic purpose in mind. [10]

Musical things

A musical thing is formally defined as a "computational device capable of acquiring, processing, acting, or exchanging data that serves a musical purpose." [10] In short, these objects are entities that can be used for musical practice, can be connected in local and/or remote networks, and act as senders or receivers of messages. They can be, for example, a smart instrument (instruments that use sensors, actuators and wireless connection for audio processing), [15] [16] wearable devices or any other capable of controlling, generating or executing musical content over the network.

Unlike traditional audio devices, such as microphones and speakers, musical things are not useful by themselves, thus the need arises to insert them into a chain of equipment. [17] Thus, the need arises to think about standards, protocols and means of communication between them. These challenges will be analyzed below.

The challenges of creating musical things

The first challenge concerns the hardware used in musical things. [17] First, one should keep in mind that these devices are not analog. Because of this, they can be reprogrammed and must have internet connectivity and/or another possibility to communicate with other equipment. Secondly, they are not traditional computing devices. This means that they are programmed for a general purpose, not only to perform certain tasks, as is the case with smartphones and personal computers. Finally, it is important to note that they will be employed in an artistic and musical context. In this way, the aesthetic characteristics are as important as the computational ones. [17]

That said, the hardware challenges are clear, and these include the processing capacity, as well as the storage and power consumption of musical things, which must be good enough to withstand artistic performances, while not making these objects expensive or unergonomic and unwieldy. In addition, they should be able to take on different roles in different scenarios. Thus, they should allow users to add or remove components (such as sensors and actuators) aiming to be adaptable, expressive and versatile. [18] [19]

The second challenge deals with the behavior of musical things. They must primarily exchange sound data, but it is desirable that they also exchange control data and processing parameters. [10] In this sense, they must adapt their operating mode in order to be able to cooperate with the other elements present in the network, and also have their software and operating logics updated remotely.

The third adversity is possibly the most sensitive and most difficult topic to deal with. You have to think about what data is possible to share and how to do it. For audio formats, one can think of Pulse Code Moduation (PCM) formats, like WAV, because it is the most common in real-time audio processing systems. However, issues such as latency and quality are not guaranteed. Files in MP3, FLAC or OGG format, on the other hand, require more processing and the latency arising from this can make the environment impractical. [17]

Possible solutions for creating musical things

Possible solutions to these problems include the use of common IoT elements in music practice or the assignment of networking capabilities on behalf of traditional audio objects. Effects units (such as guitar pedals) should be built so that the user is free to remove or insert buttons and sensors, while in logic units the software is modifiable. Audio equipment should send and receive data over the network, and also be remotely controlled. This can be useful for adapting these elements to the different types of data circulating on the network. [17]

The musical instruments, on the other hand, will function similarly to smart musical instruments, where they will be equipped with sensors and actuators capable of capturing stimuli from the environment and from the musicians themselves. Musical aids such as metronomes and tuners can be transposed to digital media, while performance aids such as light and smoke cannons can be controlled and synchronized over the network. [17]

However, IoMusT is not only about making adaptations of what already exists, but also by creating devices, capable of generating new perspectives for musical practices.

This section reviews some of the various application domains that aid an IoMusT environment. The review is not intended to be exhaustive, aiming to describe the main features and functionalities of each area.

Network musical performance

A networked performance is a real-time interaction via machine, which allows artists dispersed across the globe to interact with each other as if they were in the same environment. While not intended to replace the traditional model, it contributes to music creation and its social interactions, promoting creativity and the exchange of cultures. [20] [21]

Among its main characteristics are: low latency, where the sounds produced should be heard almost instantaneously; synchronization, to prevent long delays from hindering interaction in the environment; interoperability through standardization, which allows different devices to communicate over the network; scalability, which makes the system comprehensive and allows distributed participation among users; and easy integration and participation, aspects that ensure that users have no difficulty in finding devices on the network, and can connect or disconnect from it whenever they want. [22] As for the challenges in this area, they can be illustrated by the requirement for high bandwidth and ordering in the transmitted stream, and sensitivity to delay in the delivery of data packets. [23]

Interactive art

Art has always had its interaction marked by the relationship between the artist and the medium he uses to materialize the work, while the audience had only the role of passively observing everything. This began to change when artistic movements led by Allan Kaprow and the Fluxus and Gutai groups began to allow for more active audience participation. [24] In this context, interactive art emerged, characterized by allowing the viewer a degree of active involvement in the show, either by walking among the installations and sculptures, or by producing sounds, images, and movements. [25] [26]

The architecture of these environments is designed to handle different types of signals, ranging from audio and video to those produced by the human body, such as heartbeats. As such, they also require functionality that ensures interoperability and handles data delivery delays.

Ubiquitous music

Ubiquitous music, [4] usually abbreviated to ubimus, is a research field that combines music, technology, and creative processes with strong social and community engagement. Although its original proposal is focused on music production, current technological developments have opened new social and cognitive dimensions to this field, leading it to become increasingly interested in educational and artistic topics. Thus, current perspectives encompass a wide diversity of subjects and actors, ranging from casual participants to highly trained musicians. [27]

The ubimus ecosystem supports the integration of audio tools and audience interaction, and can be reconfigured to meet the needs of users. Consequently, the desired concepts are not dependent on specific implementations. Other important features are conceptual approaches and reliance on empirical methods. These aspects encourage the development of technologies for music creation, especially those that make use of common objects and spaces in the daily lives of those involved in the process.

Web Audio, cloud computing and edge computing

Web Audio is a JavaScript API for audio processing and synthesizing in web applications, representing a technological evolution in this segment. It presents some features common to DAWs, such as audio signal routing, low latency, and effects application. It also allows participative networked performances and expands the capabilities of using smartphones in these media.

Its environment uses audio nodes for manipulating sound in a musical context. They are connected by their inputs and outputs to create paths for routing audio, which is modified by effect nodes along the way. In this way, it is able to support numerous sources with different layouts, as well as being flexible and creating complex functions with dynamic effects. [28]

Web Audio paves the way for using web browsers for musical purposes. Among the advantages observed from this are easy distribution (no installation required) and maintenance, platform and architecture independence, security (the browser can prevent plugins with incorrect behavior from affecting the system), and emergence of new types of collaboration. [28]

Cloud computing, on the other hand, is a structure composed of distributed servers that simulate a centralized network, allowing load balancing and resource replication, minimizing the amount of network consumption and improving its scalability and reliability. It aims to provide numerous services, ranging from file storage to intercommunication between music applications, offering an unprecedented level of participation and performance. [29]

Its main feature is to allow users to access the services without the need for knowledge about the technology used. Thus, they can access them on demand and regardless of location. Other points worth highlighting in this network are: broad access, elasticity, and resource management. [30]

Cloud computing infrastructure is mostly composed of numerous physical machines connected together in a network. Each machine has the same software configurations, but can differ in the central processing unit, memory usage, and disk storage capacity. This model was developed with three main objectives in mind: i) reduce the cost in the acquisition and composition of the elements that form the network infrastructure, allowing it to be heterogeneous and adaptable to the resources required; ii) flexibility in adding or replacing computing resources; iii) ease of access to the services provided by it, where users only need their machines to have an operating system, browser and Internet access to access the resources available in the cloud. [31]

Despite all the advantages listed above about using cloud computing, its centralized mode of operation creates a lot of service load on the network, in particular on costs and bandwidth resources for data transmission. In addition, network performance worsens as the amount of data increases. To address this problem, edge computing has emerged, which is a paradigm that combines cloud computing properties with real-time communication. The term "edge" refers to all the computational and network resources between the data sources and the cloud servers. In this way, objects present in the environment not only consume data and services, but also perform computational processing, decreasing stress on the network and significantly reducing latency in message exchange. [32] [33]

The key attributes of this computing model revolve around geographic distribution, mobility support, location recognition, computing resources and services close to the end user, low latency, context sensitivity, and heterogeneity. [34]

Wearable technologies

Wearable computing is a new approach that has been redefining the way human-machine interaction happens, where electronic devices become directly connected to the user's body. They are called wearable devices and are built in such a way that the technologies and structures they contain are imperceptible, acting as an extension of the human being. Among the most popular models today are smartwatches and smartbands. [35]

Although they are small, they are capable of continuously detecting, collecting, and uploading numerous physiological and sensory data, which aim to improve typical everyday activities such as making payments, assisting in location tracking, monitoring physical and mental health, providing analysis on certain physical activity, and aiding in artistic practice.

They must be able to fulfill three main goals: assign mobility to the user, that is, allow them to use the device in various locations; augment reality, such as generating images or sounds that are not part of the real world; and provide context sensitivity, which is the ability of the equipment to adapt to different environments and stimuli. [36]

It is important to note that although they have connectivity and handle a large amount of data, not all wearable devices are IoT elements, and consequently, IoMusT elements. To be considered as such, they must have access to the internet.

Following a slightly different line of thought, but still using concepts from wearable computing, are the e-textiles. These consist of clothing enhanced with sensors and present some advantages over wearable devices, such as more comfort, more natural interfaces for human interaction and less intrusiveness. From this, electronic devices that are worn next to the human body can be classified according to the location in which they are inserted (wrist, head, feet and so on) and whether they already exist or are still in the prototyping phase. [37]

IoMusT Challenges

In addition to facing the problems inherent to the use of the technology [38] and also those present in IoT, [39] the Internet of Music Things faces specific problems, ranging from technological issues to those artistic and environmental. The main ones are highlighted below.

Technology challenges

The possibility of IoMusT occurrence is dependent on network aspects such as bandwidth, latency and jitter. From this, it is necessary that these networks expand their operation beyond the current state-of-the-art, in order to provide better connection conditions and deal with the three aspects mentioned, in addition to ensuring synchronization and good quality of the representation of multimodal audio content.

With regard to latency, reliability and synchronization, they emerge as one of the main demands in the transmission of audio over a network and in real time, whether local or remote, wired or wireless. This occurs because of the random character of this type of communication, which can cause losses in the transmitted data and the desynchronization between them, even in small networks. [10]

Still about synchronization, it is difficult to occur on devices that do not share the same global clock. Even in cases where this occurs, but with objects on different networks, resynchronization is required from time to time. Existing protocols are insufficient to meet this demand. [40]

The importance of discussing interoperability and standardization of the devices present in this environment is that these concepts are essential pillars for its implementation. This is due to the fact that the devices do not know each other previously and do not have information about the elements in which they will connect. But given the heterogeneity of these objects, in many cases they do not operate under the same protocols nor are they able to interpret the data coming from their neighbors. [11]

Artistic challenges

The main difference between IoMusT and IoT is the concern of the first field with artistic issues. [10] Despite providing advantages, such as the possibility of creation among musicians arranged in different locations around the globe, massive connectivity and new forms of participation by the audience, some problems stand out. Among them, the rupture with the traditional model of artistic interactions, as observed in bands and orchestras; lack of visual feedback; choice of which elements will be displayed and/or controlled by the audience; absence of backup systems for remote concerts; expensive, inaccessible and unergonomic devices and lack of investment to elaborate the necessary infrastructure. [10] [11] [41]

With the enormous amount of data generated in these environments, legal concerns about personal data arise, since the devices are able to collect information from users involved in the process. Issues also appear involving infringement on protected material, copyright infringement, and intellectual property infringement. [10]

Security issues are also worth mentioning. Because it is a system that communicates over the network, IoMusT is subject to attempts to steal sensitive data, denial of service attacks and trojans. Possible solutions involve encryption algorithms, but this can lead to high energy and memory usage of the devices. [42]

Social challenges

One of the first thinkers to analyze the impact of technology on society was Herbert Marcuse. [38] Among the problems cited by the author are: abundance of technology for one part of the population and scarcity for another; establishment of standards and demands by the ruling class; submission of workers to large corporations; retention of economic power and loss of individuality of thought. All these problems are present in IoMusT as well. [11]

Allied to this, other problems can be accentuated, such as non-heterogeneous access to technologies, since people living in suburban or rural areas do not have the same possibilities of access as people living in denser areas; lack of infrastructure, which increases the socio-cultural difference between people and classes; excessive consumption, constant need for innovation, and social apartheid. [11] [43]

Economic challenges

While IoMusT can revolutionize the music industry by providing artificial intelligence algorithms capable of mixing and altering sound, reducing production costs, it can also negatively impact the creative part of this field by replacing human tasks with machine-based solutions, as well as causing reduced employment opportunities in the field. [11]

Environmental challenges

With the growth of electro-electronic devices generated by this area, there is also a concern about environmental issues, especially those concerning waste generation, pollution in the making and use of these materials, use of chemical materials that can be toxic, use of non-renewable resources, and possible occurrences of ecological disturbances. [44] [45]

Possible usage scenarios

IoMusT allows rethinking some musical activities, such as live performances and rehearsals, multiplying the possibilities of interaction between the actors involved in these scenarios (musicians, audience, sound engineers, teachers, students, etc.). [10] Given this brief elucidation, it is possible to think of some usage scenarios that are detailed below.

Scenario 1 - Augmented and immersive experiences

Imagine that when people arrive at a concert of their favorite band, they can choose different interfaces that will accompany them throughout the performance. One person might choose smartglasses (a computing device that adds information according to what the wearer sees), another chooses a wristband that responds to musical stimuli, and a third selects a set of sensors and speakers. All these objects can track the user's movements and send this information to the band. The band, in turn, can tailor its performance according to the audience's emotions, as well as send them stimuli that will be interpreted by the objects they are wearing. [10]

Scenario 2 - Co-located hearing and remote hearing

Again, imagine users with wearable equipment capable of capturing their physiological data. From recording their wearer's movements and emotions (such as heart rate), musicians can decide what song to perform next, choreographers can create steps that best suit the recorded feelings, and the audience itself can make use of this data to control elements that aid the show, such as light and smoke cannons. [10] [46]

Meanwhile, people who were unable to physically attend the performance venue can experience the concert using virtual reality glasses or 360° video systems, allowing them to see behind the scenes of the stage and the details behind the musicians. IoMusT also predicts the possibility of an application that allows remote control. In this way, aspects present at the concert can be modified by the audience that is around the globe. [10] [46]

Scenario 3 - Remote rehearsals

Another possible scenario is a studio that uses IoMusT concepts to record solo artists, duos and small groups as well as orchestras with a variety of instruments. For this, the recording interface can adapt its size according to the amount of equipment connected to it. Musicians can record even if they are not in the same physical location, and audio files can be recorded for later mixing and mastering. [46] Other possibilities include capturing audio from an instrument that is not in the same physical location, remote mixing and configuration of audio systems, obtaining performance data from musicians, and many others. [10]

Scenario 4 - Music learning

Music learning is enriched by IoMusT by allowing the use of applications that display the scores to be played, capture audio in real time, and suggest improvements. Also, smart glasses can be used that indicate the correct position of the fingers on the instrument and share data in the cloud that can be viewed by teachers, who will indicate improvements and the next steps to be taken. [10]

Scenario 5 - Improvisation session with electroacoustic instruments and musical things

This model is about a jam session that combines traditional instruments and electronic devices that exchange information over a network. These instruments can be plugged into speakers or connected to patches, while the users/musicians manipulate them from computer systems. Graphic elements such as videos, animations, and musical information can be displayed to assist the process; some users can participate only by controlling parameters of the instruments, such as volume, recording, instrument effects (delay and reverb, for example), as well as changing colors and resolutions in the graphics. It is also capable of having a sound technician who manages the connections, removing those with low network connection capacity or connecting those who wish to exchange information. [46]

See also

Related Research Articles

Ubiquitous computing is a concept in software engineering, hardware engineering and computer science where computing is made to appear anytime and everywhere. In contrast to desktop computing, ubiquitous computing can occur using any device, in any location, and in any format. A user interacts with the computer, which can exist in many different forms, including laptop computers, tablets, smart phones and terminals in everyday objects such as a refrigerator or a pair of glasses. The underlying technologies to support ubiquitous computing include Internet, advanced middleware, operating system, mobile code, sensors, microprocessors, new I/O and user interfaces, computer networks, mobile protocols, location and positioning, and new materials.

<span class="mw-page-title-main">Wearable computer</span> Small computing device worn on the body

A wearable computer, also known as a body-borne computer, is a computing device worn on the body. The definition of 'wearable computer' may be narrow or broad, extending to smartphones or even ordinary wristwatches.

<span class="mw-page-title-main">Home automation</span> Building automation for a home

Home automation or domotics is building automation for a home. A home automation system will monitor and/or control home attributes such as lighting, climate, entertainment systems, and appliances. It may also include home security such as access control and alarm systems.

Context awareness refers, in information and communication technologies, to a capability to take into account the situation of entities, which may be users or devices, but are not limited to those. Location is only the most obvious element of this situation. Narrowly defined for mobile devices, context awareness does thus generalize location awareness. Whereas location may determine how certain processes around a contributing device operate, context may be applied more flexibly with mobile users, especially with users of smart phones. Context awareness originated as a term from ubiquitous computing or as so-called pervasive computing which sought to deal with linking changes in the environment with computer systems, which are otherwise static. The term has also been applied to business theory in relation to contextual application design and business process management issues.

Mutual authentication or two-way authentication refers to two parties authenticating each other at the same time in an authentication protocol. It is a default mode of authentication in some protocols and optional in others (TLS).

<span class="mw-page-title-main">Smart environment</span> Computing environment involving multiple devices

Smart environments link computers and other smart devices to everyday settings and tasks. Smart environments include smart homes, smart cities, and smart manufacturing.

<span class="mw-page-title-main">Edge computing</span> Distributed computing paradigm

Edge computing is a distributed computing model that brings computation and data storage closer to the sources of data, so that a user is likely to be physically closer to a server than if all servers were in one place. This often makes applications faster. More broadly, it refers to any design that pushes computation physically closer to a user, so as to reduce the latency compared to when an application runs on a single data centre.

The Internet of things (IoT) describes devices with sensors, processing ability, software and other technologies that connect and exchange data with other devices and systems over the Internet or other communications networks. The Internet of things encompasses electronics, communication, and computer science engineering. "Internet of things" has been considered a misnomer because devices do not need to be connected to the public internet; they only need to be connected to a network and be individually addressable.

A smart object is an object that enhances the interaction with not only people but also with other smart objects. Also known as smart connected products or smart connected things (SCoT), they are products, assets and other things embedded with processors, sensors, software and connectivity that allow data to be exchanged between the product and its environment, manufacturer, operator/user, and other products and systems. Connectivity also enables some capabilities of the product to exist outside the physical device, in what is known as the product cloud. The data collected from these products can be then analyzed to inform decision-making, enable operational efficiencies and continuously improve the performance of the product.

<span class="mw-page-title-main">Human–computer interaction</span> Academic discipline studying the relationship between computer systems and their users

Human–computer interaction (HCI) is research in the design and the use of computer technology, which focuses on the interfaces between people (users) and computers. HCI researchers observe the ways humans interact with computers and design technologies that allow humans to interact with computers in novel ways. A device that allows interaction between human being and a computer is known as a "Human-computer Interface (HCI)".

A networked music performance or network musical performance is a real-time interaction over a computer network that enables musicians in different locations to perform as if they were in the same room. These interactions can include performances, rehearsals, improvisation or jamming sessions, and situations for learning such as master classes. Participants may be connected by "high fidelity multichannel audio and video links" as well as MIDI data connections and specialized collaborative software tools. While not intended to be a replacement for traditional live stage performance, networked music performance supports musical interaction when co-presence is not possible and allows for novel forms of music expression. Remote audience members and possibly a conductor may also participate.

Sonic interaction design is the study and exploitation of sound as one of the principal channels conveying information, meaning, and aesthetic/emotional qualities in interactive contexts. Sonic interaction design is at the intersection of interaction design and sound and music computing. If interaction design is about designing objects people interact with, and such interactions are facilitated by computational means, in sonic interaction design, sound is mediating interaction either as a display of processes or as an input medium.

Computation offloading is the transfer of resource intensive computational tasks to a separate processor, such as a hardware accelerator, or an external platform, such as a cluster, grid, or a cloud. Offloading to a coprocessor can be used to accelerate applications including: image rendering and mathematical calculations. Offloading computing to an external platform over a network can provide computing power and overcome hardware limitations of a device, such as limited computational power, storage, and energy.

Fog computing or fog networking, also known as fogging, is an architecture that uses edge devices to carry out a substantial amount of computation, storage, and communication locally and routed over the Internet backbone.

Crowdsensing, sometimes referred to as mobile crowdsensing, is a technique where a large group of individuals having mobile devices capable of sensing and computing collectively share data and extract information to measure, map, analyze, estimate or infer (predict) any processes of common interest. In short, this means crowdsourcing of sensor data from mobile devices.

The industrial internet of things (IIoT) refers to interconnected sensors, instruments, and other devices networked together with computers' industrial applications, including manufacturing and energy management. This connectivity allows for data collection, exchange, and analysis, potentially facilitating improvements in productivity and efficiency as well as other economic benefits. The IIoT is an evolution of a distributed control system (DCS) that allows for a higher degree of automation by using cloud computing to refine and optimize the process controls.

<span class="mw-page-title-main">Subsea Internet of Things</span>

Subsea Internet of Things (SIoT) is a network of smart, wireless sensors and smart devices configured to provide actionable operational intelligence such as performance, condition and diagnostic information. It is coined from the term The Internet of Things (IoT). Unlike IoT, SIoT focuses on subsea communication through the water and the water-air boundary. SIoT systems are based around smart, wireless devices incorporating Seatooth radio and Seatooth Hybrid technologies. SIoT systems incorporate standard sensors including temperature, pressure, flow, vibration, corrosion and video. Processed information is shared among nearby wireless sensor nodes. SIoT systems are used for environmental monitoring, oil & gas production control and optimisation and subsea asset integrity management. Some features of IoT's share similar characteristics to cloud computing. There is also a recent increase of interest looking at the integration of IoT and cloud computing. Subsea cloud computing is an architecture design to provide an efficient means of SIoT systems to manage large data sets. It is an adaption of cloud computing frameworks to meet the needs of the underwater environment. Similarly to fog computing or edge computing, critical focus remains at the edge. Algorithms are used to interrogate the data set for information which is used to optimise production.

The Internet of Military Things (IoMT) is a class of Internet of things for combat operations and warfare. It is a complex network of interconnected entities, or "things", in the military domain that continually communicate with each other to coordinate, learn, and interact with the physical environment to accomplish a broad range of activities in a more efficient and informed manner. The concept of IoMT is largely driven by the idea that future military battles will be dominated by machine intelligence and cyber warfare and will likely take place in urban environments. By creating a miniature ecosystem of smart technology capable of distilling sensory information and autonomously governing multiple tasks at once, the IoMT is conceptually designed to offload much of the physical and mental burden that warfighters encounter in a combat setting.

<span class="mw-page-title-main">Bruno Zamborlin</span> Italian researcher, entrepreneur and artist

Bruno Zamborlin is an AI researcher, entrepreneur and artist based in London, working in the field of human-computer interaction. His work focuses on converting physical objects into touch-sensitive, interactive surfaces using vibration sensors and artificial intelligence. In 2013 he founded Mogees Limited a start-up to transform everyday objects into musical instruments and games using a vibration sensor and a mobile phone. With HyperSurfaces, he converts physical surfaces of any material, shape and form into data-enabled-interactive surfaces using a vibration sensor and a coin-sized chipset. As an artist, he has created art installations around the world, with his most recent work comprising a unique series of "sound furnitures" that was showcased at the Italian Pavilion of the Venice Biennale 2023. He regularly performed with UK-based electronic music duo Plaid. He is also honorary visiting research fellow at Goldsmiths, University of London.

<span class="mw-page-title-main">IoT forensics</span> Branch of digital forensics

IoT forensics is a branch of digital forensics that has the goal of identifying and extracting digital information from devices belonging to the Internet of things field, using a forensically sound and legally acceptable process.

References

  1. 1 2 Atzori, Luigi; Iera, Antonio; Morabito, Giacomo (October 2010). "The Internet of Things: A survey". Computer Networks. 54 (15): 2787–2805. doi:10.1016/j.comnet.2010.05.010. S2CID   2630520.
  2. 1 2 Borgia, Eleonora (December 2014). "The Internet of Things vision: Key features, applications and open issues". Computer Communications. 54: 1–31. doi:10.1016/j.comcom.2014.09.008.
  3. 1 2 Miorandi, Daniele; Sicari, Sabrina; De Pellegrini, Francesco; Chlamtac, Imrich (September 2012). "Internet of things: Vision, applications and research challenges". Ad Hoc Networks. 10 (7): 1497–1516. doi:10.1016/j.adhoc.2012.02.016. hdl: 11383/1762288 . S2CID   205287824.
  4. 1 2 Keller, Damián; Lazzarini, Victor; Pimenta, Marcelo S., eds. (2014). Ubiquitous Music. Computational Music Science. doi:10.1007/978-3-319-11152-0. ISBN   978-3-319-11151-3.[ page needed ]
  5. ↑ Preece, Jenny; Rogers, Yvonne; Sharp, Helen (2015). Interaction design: beyond human-computer interaction (4th ed.). Chichester: John Wiley & Sons. ISBN   978-1-119-02075-2. OCLC   904425795.[ page needed ]
  6. ↑ Rowland, Claire; Goodman, Elizabeth; Charlier, Martin; Light, Ann; Lui, Alfred (2015). Designing Connected Products: UX for the Consumer Internet of Things (1st ed.). Sebastopol: O'Reilly Media. ISBN   978-1-4493-7272-9. OCLC   909772560.[ page needed ]
  7. ↑ Burgoyne, John Ashley; Fujinaga, Ichiro; Downie, J. Stephen (2015). "Music Information Retrieval". A New Companion to Digital Humanities. pp. 213–228. doi:10.1002/9781118680605.ch15. ISBN   978-1-118-68059-9.
  8. ↑ Wu, Yongmeng; Zhang, Leshao; Bryan-Kinns, Nick; Barthet, Mathieu (January 2017). "Open Symphony: Creative Participation for Audiences of Live Music Performances". IEEE MultiMedia. 24 (1): 48–62. doi:10.1109/mmul.2017.19. S2CID   216906649.
  9. ↑ Jensenius, Alexander Refsum; Lyons, Michael (2017). A NIME Reader Fifteen Years of New Interfaces for Musical Expression. Springer. ISBN   978-3-319-47214-0. OCLC   981923724.{{cite book}}: |work= ignored (help)[ page needed ]
  10. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Turchet, Luca; Fischione, Carlo; Essl, Georg; Keller, Damian; Barthet, Mathieu (2018). "Internet of Musical Things: Vision and Challenges". IEEE Access. 6: 61994–62017. Bibcode:2018IEEEA...661994T. doi: 10.1109/ACCESS.2018.2872625 . hdl: 11572/226764 . ISSN   2169-3536. S2CID   53280116.
  11. 1 2 3 4 5 6 Barthet, Mathieu; Schiavoni, Flávio Luiz (5 November 2020). Everyday Use of the Internet of Musical Things: Intersections with Ubiquitous Music (Preprint). doi:10.5281/zenodo.4247759.
  12. ↑ Hazzard, Adrian; Benford, Steve; Chamberlain, Alan; Greenhalgh, Chris; Kwon, Hyosun (2014). Musical intersections across the digital and physical. DMRN+9: Digital Music Research Network (EPSRC).
  13. ↑ Keller, Damián; Lazzarini, Victor (April 2017). "Ecologically Grounded Creative Practices in Ubiquitous Music" (PDF). Organised Sound. 22 (1): 61–72. doi:10.1017/S1355771816000340. S2CID   64353906.
  14. ↑ Turchet, Luca; Fischione, Carlo; Barthet, Mathieu (2017). Towards the Internet of Musical Things (PDF). 14th Sound and Music Computing Conference. pp. 13–20.
  15. ↑ Turchet, Luca; McPherson, Andrew; Fischione, Carlo (2016). Smart Instruments: Towards an Ecosystem of Interoperable Devices Connecting Performers and Audiences. Proceedings of Sound and Music Computing Conference. pp. 498–505. hdl:11572/226231.
  16. ↑ Aramaki, Mitsuko; Davies, Matthew E. P.; Kronland-Martinet, Richard; Ystad, Sølvi, eds. (2018). Music Technology with Swing. Lecture Notes in Computer Science. Vol. 11265. doi:10.1007/978-3-030-01692-0. ISBN   978-3-030-01691-3.[ page needed ]
  17. 1 2 3 4 5 6 Vieira, Romulo; Goncalves, Luan; Schiavoni, Flavio (2020). "The things of the Internet of Musical Things: Defining the difficulties to standardize the behavior of these devices". 2020 X Brazilian Symposium on Computing Systems Engineering (SBESC). pp. 1–7. doi:10.1109/SBESC51047.2020.9277862. ISBN   978-1-7281-8286-5.
  18. ↑ Polianytsia, Andrii; Starkova, Olena; Herasymenko, Kostiantyn (2016). "Survey of hardware IoT platforms". 2016 Third International Scientific-Practical Conference Problems of Infocommunications Science and Technology (PIC S&T). pp. 152–153. doi:10.1109/INFOCOMMST.2016.7905364. ISBN   978-1-5090-5715-3.
  19. ↑ Maksimovic, Mirjana; Vujovic, Vladimir; Davidovic, Nikola; Milosevic, Vladimir; Perisic, Branko (2014). Raspberry Pi as Internet of Things hardware: Performances and Constraints. IcETRAN.
  20. ↑ Gresham-Lancaster, Scot (1998). "The Aesthetics and History of the Hub: The Effects of Changing Technology on Network Computer Music". Leonardo Music Journal. 8: 39–44. doi:10.2307/1513398. JSTOR   1513398. S2CID   60594845.
  21. ↑ Lazzaro, John; Wawrzynek, John (2001). A case for network musical performance. 11th International Workshop on NOSSDAV. pp. 157–166. doi:10.1145/378344.378367. ISBN   978-1-58113-370-7.
  22. ↑ Alexandraki, Chrisoula; Koutlemanis, P.; Gasteratos, Petros; Valsamakis, Nikolas; Milolidakis, Giannis (2008). Towards the Implementation of a Generic Platform for Networked Music Performance: the DIAMOUSES Approach. International Conference on Mathematics and Computing. S2CID   42314531.
  23. ↑ Xiaoyuan Gu, Matthias Dick (2004). "NMP - a new networked music performance system". IEEE Global Telecommunications Conference Workshops, 2004. GlobeCom Workshops 2004. pp. 176–185. doi:10.1109/GLOCOMW.2004.1417570. ISBN   978-0-7803-8798-0.
  24. ↑ Luciane Nardim, Thaise (31 August 2009). Allan Kaprow, performance e colaboração: estrategias para abraçar a vida como potencia criativa (Thesis) (in Portuguese). doi:10.47749/T/UNICAMP.2009.470906.[ page needed ]
  25. ↑ Forny, Leonardo (2006). "Arte e Interação: Nos Caminhos da Arte Interativa?". Razón y Palabra (in Spanish). 53. ISSN   1605-4806 – via Redalyc.
  26. ↑ Vieira, Rômulo; Oliveira Júnior, Igino de; Almeida, Marcela; Schiavoni, Flávio Luiz (2020). Uma aproximação entre participação do público em espetáculo artístico e jogos: racionalidade e improviso. SBGames. pp. 116–124 – via ResearchGate.
  27. ↑ Keller, Damián; Schiavoni, Flávio; Lazzarini, Victor (8 August 2019). "Ubiquitous music: Perspectives and challenges" (PDF). Journal of New Music Research. 48 (4): 309–315. doi:10.1080/09298215.2019.1659828. ISSN   0929-8215. S2CID   202768440.
  28. 1 2 Smus, Boris (2013). Web audio API. Sebastopol: O'Reilly Media. ISBN   978-1-4493-3268-6. OCLC   848884051.[ page needed ]
  29. ↑ Carvalho Junior, Antônio Deusany de; Queiroz, Marcelo; Essl, Georg (2015). Computer Music through the Cloud: Evaluating a Cloud Service for Collaborative Computer Music Applications. International Computer Music Conference – via ResearchGate.
  30. ↑ Dillon, Tharam; Wu, Chen; Chang, Elizabeth (2010). "Cloud Computing: Issues and Challenges". 2010 24th IEEE International Conference on Advanced Information Networking and Applications. pp. 27–33. doi:10.1109/AINA.2010.187. ISBN   978-1-4244-6695-5.
  31. ↑ Sousa, Flávio; Moreira, Leonardo; Machado, Javam (2021). "Computação em Nuvem: Conceitos, Tecnologias, Aplicações e Desafios" – via ResearchGate.{{cite journal}}: Cite journal requires |journal= (help)
  32. ↑ Khan, Wazir Zada; Ahmed, Ejaz; Hakak, Saqib; Yaqoob, Ibrar; Ahmed, Arif (August 2019). "Edge computing: A survey". Future Generation Computer Systems. 97: 219–235. doi:10.1016/j.future.2019.02.050. S2CID   86399908.
  33. ↑ Shi, Weisong; Cao, Jie; Zhang, Quan; Li, Youhuizi; Xu, Lanyu (October 2016). "Edge Computing: Vision and Challenges". IEEE Internet of Things Journal. 3 (5): 637–646. doi:10.1109/JIOT.2016.2579198. S2CID   4237186.
  34. ↑ Yu, Wei; Liang, Fan; He, Xiaofei; Hatcher, William Grant; Lu, Chao; Lin, Jie; Yang, Xinyu (2018). "A Survey on the Edge Computing for the Internet of Things". IEEE Access. 6: 6900–6919. Bibcode:2018IEEEA...6.6900Y. doi: 10.1109/ACCESS.2017.2778504 . ISSN   2169-3536. S2CID   3818445.
  35. ↑ Mann, S. (February 1997). "Wearable computing: a first step toward personal imaging". Computer. 30 (2): 25–32. doi:10.1109/2.566147. S2CID   28001657.
  36. ↑ Billinghurst, M.; Starner, T. (January 1999). "Wearable devices: new ways to manage information". Computer. 32 (1): 57–64. doi:10.1109/2.738305.
  37. ↑ Seneviratne, Suranga; Hu, Yining; Nguyen, Tham; Lan, Guohao; Khalifa, Sara; Thilakarathna, Kanchana; Hassan, Mahbub; Seneviratne, Aruna (2017). "A Survey of Wearable Devices and Challenges". IEEE Communications Surveys & Tutorials. 19 (4): 2573–2620. doi:10.1109/COMST.2017.2731979. S2CID   27686190.
  38. 1 2 Oliveira, Fernanda Tatiani (2017). "Apontamentos sobre a tecnologia em Herbert Marcuse". Revista Primordium (in Portuguese). 1 (2).
  39. ↑ van Kranenburg, Rob; Bassi, Alex (December 2012). "IoT Challenges". Communications in Mobile Computing. 1 (1). doi: 10.1186/2192-1121-1-9 . S2CID   14089146.
  40. ↑ Rottondi, Cristina; Chafe, Chris; Allocchio, Claudio; Sarti, Augusto (2016). "An Overview on Networked Music Performance Technologies". IEEE Access. 4: 8823–8843. Bibcode:2016IEEEA...4.8823R. doi: 10.1109/ACCESS.2016.2628440 . hdl: 11311/1158199 . ISSN   2169-3536. S2CID   32894276.
  41. ↑ Avila, Juan Pablo Martinez; Greenhalgh, Chris; Hazzard, Adrian; Benford, Steve; Chamberlain, Alan (2019). Encumbered Interaction: a Study of Musicians Preparing to Perform. Conference on Human Factors in Computing Systems. pp. 1–13. doi: 10.1145/3290605.3300706 . ISBN   9781450359702.
  42. ↑ Weber, Rolf H. (October 2015). "Internet of things: Privacy issues revisited". Computer Law & Security Review. 31 (5): 618–627. doi:10.1016/j.clsr.2015.07.002.
  43. ↑ Silva Junior, Igino; Schiavoni, Flávio Luiz (2019). "Sustainable Interfaces for Music Expression". Anais do Simpósio Brasileiro de Computação Musical (SBCM 2019). pp. 63–68. doi:10.5753/sbcm.2019.10424.
  44. ↑ Cubitt, Sean (2017). Finite media: environmental implications of digital technologies. Durham. ISBN   978-0-8223-6281-4. OCLC   952139233.{{cite book}}: CS1 maint: location missing publisher (link)[ page needed ]
  45. ↑ Eren, Halit (1999). "Environmental Impacts of Technology". Wiley Encyclopedia of Electrical and Electronics Engineering. doi:10.1002/047134608X.W7303. ISBN   978-0-471-34608-1.
  46. 1 2 3 4 Vieira, Rômulo; Schiavoni, Flávio (2021). "Sunflower: An environment for standardized communication of IoMusT". Audio Mostly 2021. pp. 175–181. doi:10.1145/3478384.3478414. ISBN   978-1-4503-8569-5.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.