How can machine learning technologies transform artistic practices in real-time? To address this question, the artist-researchers explored the ability of machine learning technologies to create a symbiosis between three artistic forms: robotic installation, audiovisual (AV) performance, and virtual reality (VR).
This experience culminated in a restitution-exhibition, Symbiosis: Useful Fictions 3, presented in June 2023 at the Espace Pierre Cardin of the Théâtre de la Ville, in Paris. Explore this residency journal where, day by day, variables shift to uncover the potential of artificial agents by means of reinforcement learning (RL).
Tuesday: AV Performance + VR + RL
The artist-researchers set up an artificial agent navigating through a latent space. The data generated by the agent is assigned to control parameters that produce a visual performance. This visual performance is then integrated as a texture within the virtual reality environment.
The result is a series of floating cubes that rise and reflect the movements of the artificial agent. The user interacting with the virtual reality setup can manipulate these cubes, thereby influencing and disrupting the agent’s aesthetic propositions.
The artist-researchers employ robots to create a dynamic, fragmented, and rotating screen. A projection is distributed across the fragments of this robotized canvas. The artificial agent generates behavioural patterns, taking the form of lines, streaks, and geometries.
The movements of the robotic projection surface reshape the composition of both the canvas and the projection. The performer, operating the console, guides the agent by providing directives to enhance the aesthetic of the work or to explore alternative arrangements.
The artist-researchers integrate 3D-printed elements with robots, whose movements are controlled by agents operating in virtual reality. The agents’ behaviour is trained using reinforcement learning.
In the virtual reality environment, the yellow-legged agent is linked to the movements of the physical robots, while the gray-legged agents mirror the movements of the user interacting with the setup. As a result, the robots in the physical space move in synchrony with their virtual counterparts.
Multifaceted, this project extends beyond the artistic realm. It incorporates ethnographic observations of the collaboration between the virtual reality (VR), audiovisual (AV), and robotics teams. Through detailed note-taking, along with photos and videos, this external observation aimed to capture the dynamics between individuals and teams.
The observer also conducted self-confrontation sessions, revisiting the most significant moments of each day. These moments could be the most challenging, the most joyful, or those laden with intense emotions. The goal was to understand how each participant felt about the day’s events.
This approach provided each artist-researcher with an opportunity to reflect on and share their experiences and emotions. It also enabled an analysis of the challenges and opportunities inherent in working symbiotically across disciplines.
Within mXlab, the Research Group on Beyond-Human Media Creation focuses on media-based research-creation that includes non-human entities such as algorithms, robots, images, sounds, light, matter, microorganisms, plants, and more.
The group’s work adopts a transdisciplinary approach to media research-creation and is structured around three key axes:
Creative practices in artificial intelligence and artificial life — Sofian Audry
Media creation within extended collectives (humans, machines, materials, etc.) — Danny Perreault
Media memories and imaginaries — Katharina Niemeyer
One of the group’s objectives is to develop inclusive and adaptive approaches, tools, and media works that offer alternatives to the often homogenized and standardized creations of industrial production. From a reflective and critical stance, both theoretical and practical, the group explores the creation of artworks and the development of open-source hardware and software tools, while advancing the conceptual, material, and historical dimensions of media creation.
Le mXlab — Research Group on Beyond-Human | Media Creation extends its gratitude to its partners:
Université du Québec à Montréal (UQAM) École nationale des arts décoratifs de Paris (ENSAD) Arizona State University (ASU) Conseil de recherche en sciences humaines (CRSH) Réseau Hexagram
Supervisors Sofian Audry* (École des Médias de l’UQAM) Samuel Bianchini* (EnsadLab, École des Arts décoratifs de Paris)
Saclay Team Natalia Balska*, Alex M.Lee*, Corentin Loubet, Etienne Montenegro*, Danny Perreault* et Ola Siebert*
Montreal Team Victor Drouin-Trempe* et Gabriel.le Tran*
Coordination Josée Brouillard*
Methodology Florence Millerand et Louis-Claude Paquin*
