Résonance Latente is a series of interactive audiovisual (A/V) improvisational stage performances between David Drouin—electronic musician and student member—and an artificial intelligence agent responsible for the visual counterpart. The performances took place in the context of a research‑creation project exploring the potential of machine‑learning techniques to augment A/V improvisational capacities. This project was conducted as part of a master’s degree in Communication (research‑creation in experimental media profile) at Université du Québec à Montréal.
The project examines how engaging with these techniques can benefit from their capacity for emergent novelty while maintaining modalities of control that foster sensemaking. In this DEMO, David Drouin outlines the trajectory of this research.
Improvisational Fictionalism and the Aesthetics of Behavior
Philosopher Eric Lewis offers an interesting perspective for addressing the question of sense in musical improvisation with a machine through his theory of improvisational fictionalism (Lewis, 2019). He argues that there is no loss of authenticity in improvising with a machine if it possesses the capacity to produce convincing results. According to Lewis, one can therefore “treat them as imaginary partners in our improvisations, playing a game of make-believe.”
To determine what might constitute convincing results in the context of a visual agent, I draw on the aesthetics of behavior (Penny, 2000). This framework emphasizes that designing behaviors involves creating a contingent model of what might happen in the world and how a system can respond to it in order to guide the user’s aesthetic attention in a direction coherent with the work itself. This leads to an aesthetic negotiation between interaction dynamics and authorial intentions. This perspective makes it possible to analyze the degrees and morphology of the deployment of an agent’s behaviors over time (Audry, 2021). This grounding guided my creative choices toward setting up conditions that might encourage the agent’s emergence of behaviors capable of adopting recognizable forms and metamorphosing over time. Ultimately, it is through my observation of the machine’s behaviors within the sociocultural context of performance that I am able to evaluate its situated production of novelty (Bown, 2021) and judge whether these form convincing results “that open the game of make‑believe.”
The visual agent
The agent consists of an assemblage of software using artificial neural networks that I programmed and trained to produce figurative video collages based on the analysis of human musical gestures. These collages draw on a corpus of video archives from Beauce (DeLessard, n.d.), my region of origin. This material was selected for its transbiographical qualities—embedding the biographical within the fictional—as well as for its strong potential to generate poetic meaning during improvisation.
Operating diagram of the visual agent. Photo : ⒸDavid Drouin.
The training process involves associating musical analysis—such as sound spectrum or rhythmic analysis of an instrument—with visual‑effect parameters, such as the repetition rate of a video loop or the selection of an image to display. These associations are generated through shared descriptors that I manually assigned, or by allowing neural‑network algorithms to classify data based on patterns they determine statistically. These trainings were conducted using small datasets (Abuzuraiq & Pasquier, 2024) that I created or assembled myself. This approach facilitates navigation and manipulation of the neural networks throughout the training process and allows me to better grasp the influence of creative decisions on their behavior.
Overview of the creation process for training data through qualitative associations. Photos : ⒸDavid Drouin
To generate a variety of behaviors, a central algorithm orchestrates the musical signals that trigger visual events. It also determines how these signals can be analyzed through trained neural networks and properties such as cadence, delaying, or smoothing of each analysis. In the event of silence from a signal it has selected, the algorithm falls back on active musical signals in order to create a convincing machine listening effect of the ongoing musical performance.
Visualization of the orchestration of musical signals and neural networks.
In Residency: Scenography, Performative Language, and Transformative Potential
In January 2026, a residency at Hexagram‑UQAM made it possible to finalize a scenographic concept that allows me to see the image, thereby strengthening the mutual influence between my human performance and that of the machine and thus fostering proper interaction dynamics. This scenography is centered on the use of a translucent tulle screen placed between the audience and myself, enabling video projection on both sides. I combined this with lighting effects to make myself appear within the image and to expose the interactive feedback loop by highlighting key performative moments, such as the transition from total silence to sound and the first sound‑image interactions that follow.
Scenography with tulle screen and lighting counterpoint. Photos : ⒸOlivier Rodrigue
I also explored and refined the project’s performative and musical language through daily practices recorded in the space. The tension I experienced between focusing on my musical performance and observing the image made me realize that I was experiencing an interaction similar to improvisation with another human: I can trust my partner‑agent’s contributions and listening in order to focus on the attentive execution of my own gestures, just as I can let myself be carried by its visual contributions and by the perceived totality of our emerging performance. By taking stock of the immersion I experienced in the image, and of the effect of presence created by my body illuminated by stage lights, I developed an improvisational framework with marked pauses in order to deliberately amplify moments of gazes and interactions.
Exploration of performative language. Photo : ⒸOlivier Rodrigue.
These practices also served as a laboratory for providing the artificial agent with a neural network acting as a memory of the ongoing performance, enabling it to associate musical motifs with its behaviors and to return to them should similar motifs reappear. This rudimentary experiment led to perceptible transformations in the agent’s behavior. This was particularly notable during a public performance on January 29, 2026, where the agent’s visual gestures were highly consistent and markedly different from all my previous practices. In the game of make‑believe, one can amuse oneself by believing that the presence of an audience pushed the agent to perform differently.
Public performance marking the end of the residency (January 2026).
Performing an Algorithmically Mediated Regionalist Nostalgia
The performative manipulation of a collection of regional archives through algorithms that generate an interactive remix raises additional questions. This type of reflective nostalgia, as sociologist Dariusz Brzeziński reminds us, creatively transforms the past and does not seek to “bring to light the truth of reality but to uncover the ambivalences of that reality” (Brzeziński, 2024, p. 97). At a time when on‑demand nostalgia becomes accessible to everyone through social networks and AI techniques, Résonance Latentes invites reflection on the meanings produced by a nostalgia now mediated by algorithms.
Algorithmically performed Beauceron regionalist nostalgia. Photos : ⒸDavid Drouin.
Audry, S. (2021). Art in the Age of Machine Learning. MIT Press.
Bown, O. (2021). Beyond the Creative Species. MIT Press.
Brzezińsk, D. (2024). Algorithmic Nostalgia. Longing for the Past in the Age of Artificial Intelligence.
Lewis, E. (2019). Intents and Purposes. University of Michigan Press
Penny, S. (2000). Agents as Artworks and Agent Design as Artistic Practice.
Abuzuraiq, A. M. et Pasquier, P. (2024). Towards Personalizing Generative AI with Small Data for Co Creation in the Visual Arts.
Cover image : David Drouin
David Drouin is an electronic musician and media artist based in Tiohtià:ke/Mooniyang/Montréal. His practice focuses on live performance, improvisation, and audiovisual interactions. He is a master’s candidate in Communication (research creation profile in experimental media) at the Université du Québec à Montréal and a student member of mXlab, the research group on media creation beyond the human.
His collaborative work on immersive works and interactive installations has been presented at venues and events including the Planétarium de la Cité des sciences de Paris (France), Nobel Week Lights (Sweden), MUTEK Digital Village (Montréal), Société des arts technologiques (Montréal), Ignite Art & Light Festival (United States), and Patchlab Digital Art Festival (Poland).
Acknowledgements
Thanks to Olivier Rodrigue, Julien Dajez, Emma Cloutier‑Nadon, and Isaak Paul‑Rivest for their support during the residency, and to Mr. Normand DeLessard for the use of his archival collection.
