September 2023

This DEMO presents Marine Theunissen’s [student member, UQAM] research-creation, GENERATIVE CHORUS, which is conceived as a laboratory-work carried out between 2018 and 2023 with some forty participants. The work lies at the crossroads of the performing arts (movements) and artificial intelligence technologies (system design and algorithm training) around the sensation of “connection” between humans and other-than-humans. Her research-creation was presented at the Agora Hydro Québec du cœur des sciences de l’UQAM from April 12 to 14, 2023, with the support of Hexagram and AEDEPAUQAM.

In this DEMO, Theunissen returns to the 3 aspects of her work: investigating the sensation of connection, designing a technological device (in) partnership and creating strategies for disseminating the laboratory form in the performing arts.

---

An artistic investigation of choral dynamics

What are the relational dynamics that allow the sensation of “connection” to emerge between improvising individuals? What are the criteria that enable us to identify that we are experiencing a moment of connection with others, how important is this in the context of improvisation, and are these connections observable to external witnesses? Finally, what is the extent of our sense of connection to other-than-humans, and can we collectively stage it with artificial intelligence technologies? It is with these questions in mind that Marine Theunissen has launched the GENERATIVE CHORUS project to investigate the sensation of connection experienced when improvising collective movement in a technological context. Through a series of laboratories carried out since 2018, she will explore its conditions of emergence, what hinders it, and all the collective strategies, states and phenomena that revolve around the sensation of connection.

Theunissen’s investigation takes as its basic framework a choreographic proposal structured around “rules of movement” inspired by the murmurations of birds (such as starlings) that form an “improvised collective movement regime”:

1. Move only in reaction to others;
2. Stay in constant motion;
3. Play with (im)balances and boundaries (macro-micro, inside-out, balanced-chaotic, etc.);
4. Enter, exit and stop when desired or necessary.

These rules are the most recent version of the improvised collective movement regime. The proposals for movement that these rules formulate leave room for interpretation by the performers and the collective, who seize them, try them out, refute them and adapt them in constant negotiation.

One of the findings of this investigation into choral dynamics lies in the identification of the role of constant evaluation of appropriateness (of the performers’ choices, of the device’s responses, of the action at large) in the sense of connection for the performers (and potentially the witnesses).

---

Designing a partnering technological device

With the aim of experimenting with the issues, limits and relational possibilities between human performers and a device including artificial intelligence technologies, Theunissen is attempting an approach to system design as a partner. Far from wishing for a generic model, she and Raphaël Dely (interactive designer) develop a reflection on what would make a “good” partner for the project’s group of performers. They draw on their experiences with humans in the experimental laboratory to design their technological device. Their aim is to create a device that performs, not a tool or instrument.

This device works in 3 steps and used the training principle advocated by Fiebrink and Sonami on training RNAs from small databases:

1. A camera captures the stage on which the humans move, and the video stream is routed to the MAX  program (IRCAM – Cycling74);
2. In MAX, the device processes video images to extract motion data via several machine learning algorithms (optical float, blobs) derived from modules created by Jean-Marc Pelletier and layers coded by Raphaël Dely.
3. The extracted data can be used to deduce (by cross-referencing) information such as dynamic engagement of bodies, number of bodies, direction and intensity of movement, for example. These data can be used to train artificial neural networks to identify certain collective improvisation configurations via Wekinator (and the  Fiebrink Input Helper plugin).
4. Artificial Intelligence algorithms (RNA) determine musical repertoires (musical notes generated via Markov chains created by Benjamin Day Smith) and playing effects (via Reaper and Amplitube). This information enables the device to “play” a virtual instrument in Reaper, using Spitfire Audio‘s Guitar Harmonics to generate a modified Electric Guitar sound.
5. Finally, a system of gauges that fill and empty as the performance progresses adds elements of surprise to the interpretation, by varying the sensitivity of the device and digitally transposing human notions of desire and boredom.

Theunissen and Dely placed the expressiveness of the device and its coherence within the context of GENERATIVE CHORUS‘s work at the heart of their design. Throughout its iterations, the design process aimed to experiment with processes that would be accessible to performing artists unaccustomed to technology in general and artificial intelligence in particular. Their design choices were therefore not based on the complexity and performance of individual technologies, but on the assembly of simple technologies and the collective nature of the device’s agentivity. Finally, following the principle of training on small volumes of data, they went so far as to train their algorithms on stage.

---

Processes for disseminating the laboratory form in the performing arts

Defending and structuring the laboratory-as-work approach (laboratory-work), Theunissen also reflects on the dissemination strategies surrounding her practice. Between 2018 and 2023, she developed visual, media and performance artworks to bring the public into contact with the process of making. Her first initiative in this direction developed in collaboration with Hexagram as part of the Taking Care exhibition curated by Anna Kerekes at the Ars Electronica Festival in 2018..

Theunissen has thus begun to develop what she calls “document-works” which, based on the findings and processes of the laboratory-work, enable singular aspects of research-creation to be shared. At Ars Electronica, she presents a triptych of works:

1. A participatory performance that invites volunteers from the audience to move around according to the same rules as the performers in the laboratory;
2. A micro-documentary that explains the general approach of the project and shares initial analyses against a backdrop of practice documentation (video recordings of laboratory improvisations – LAVI 2018);
3. A multiplayer video game (up to 14 players) that attempts to transpose some of the performers’ improvisational experience into improvisation so that players can experience it through this medium. This experimental video game was co-created by Raphaël Dely and Robert Edilber.

In 2021, curator Emilia Paunescu gave Marine Theunissen carte blanche to create the document-work Labelling for the NOVA festival in Bucharest. In this installation, Theunissen explores the labeling process with the aim of generating a hand-crafted database for her project. Testifying to the questions raised by the practical and ethical dimensions of labeling, she takes a transparent approach to human actions in the design of AI databases. The sharing of her work through her own thoughts and the testimonies of the performers with the public in turn generates a form of opacity through overabundance. This installation of suspended paper and mirrors also incorporates digital tablets that stream singular phenomena from the improvising collective via a montage of images from laboratory documentation. The installation also contains an interactive experiment that allows the public to generate their own labels based on their own observations of movement.

During the doctoral presentation in April 2023, Theunissen and Dely co-created an installation on the model of the mind-map, often used in the context of research-creation in an academic setting. In a performative approach and by hijacking this methodological tool (mind-map), they return day after day for a week to the whole project (2018-2023) and specifically to what nourished the design choices of the technological device. This work-document and the multiplayer video game were exhibited alongside the public experiments offered between April 12 and 14, 2023 at UQAM’s Agora du Cœur des Sciences, thanks to Hexagram.

Drawing on their experience and multiple collaborations in creating interactive experiences for the stage and digital arts, Dely and Theunissen decided to form the LABORARE collective in 2022.

From November 2 to 5, 2023, they will present the Effet d’Entraînement project at Tangente, some aspects of which are inherited from the GENERATIVE CHORUS research-creation project. They are also embarking on two new projects: Rising Tides (2024-2025), a technological performance for a performer and water; and Veilleuses (2024-2026), a science-fiction play in the form of 3 solos.

Cette publication est également disponible en : Français (French)