Improvise+=Chain: Listening to the Ensemble Improvisation of an Autoregressive Generative Model

https://www.nime.org/proc/nime2023_94/index.html

Atsuya Kobayashi, Ryo Nishikado, and Nao Tokui. 2023. Improvise+=Chain: Listening to the Ensemble Improvisation of an Autoregressive Generative Model. Proceedings of the International Conference on New Interfaces for Musical Expression.

Abstract:

This paper describes Improvise+=Chain, an audio-visual installation artwork of autonomous musical performance using artificial intelligence technology. The work is designed to provide the audience with an experience exploring the differences between human and AI-based virtual musicians. Using a transformer decoder, we developed a four-track (melody, bass, chords and accompaniment, and drums) symbolic music generation model. The model generates each track in real time to create an endless chain of phrases, and 3D visuals and LED lights represent the attention information between four tracks, i.e., four virtual musicians, calculated within the model. This work aims to highlight the differences for viewers to consider between humans and artificial intelligence in music jams by visualizing the only information virtual musicians can communicate with while humans interact in multiple modals during the performance.