Xenolalia is an artscience project created by Sofian Audry [co-investigator member, UQAM] and TeZ [collaborator member] that investigates computational and biological sentience through a live experiment. The work brings together an artificial neural network and a colony of living microorganisms in a feedback loop where they collaborate to generate a speculative living alphabet. This process is rendered through a series of displays at micro, meso and macro scales.
The alphabet is created using an iterative morphogenetic process involving a generative artificial neural network and a colony of light-responsive microorganisms. The neural network generates a glyph which is projected on the petri dish. Attracted by light, the microorganisms move towards the shape, acting as a living photographic substrate. The image they form is captured and sent back to the neural network, which generates another image in response. By repeating this interactive conversation between the neural network and the Euglena gracilis micro-algae colony, a hybrid computational-biological symbol slowly comes into being.
Visitors of this bio-artificial hybrid observatory witness an unusual form of communication and the unfolding of a generative alphabet of a speculative language, left to be interpreted by humans. The system suggests prospective forms of interactions between computational, carbon-based, and even xenobiological entities, which could potentially emerge in the future.
The concept
The renaissance of artificial intelligence through its commercialization since the mid-2000s has triggered a mix of fear and excitement. In particular, the recent successes of AI are due to the advent of deep learning, a machine learning approach based on artificial neural networks that are able to learn from information. This disruptive technology is redefining our conceptions of cognition, creativity, life, and language.
What is the relationship between life, symbols, language, cognition, intelligence, and consciousness? How do symbols emerge within different scopes: in cells, brains, and societies? In particular, are symbols and languages “alive” and if so, in what way? In Xenolalia, the artists approach these questions by involving three kinds of entities in the creation of a new set of symbols: (1) a deep learning system known as an auto-encoder neural network; (2) a microscopic colony of photosynthetic organisms known as Euglena gracilis who hypothetically possess a form of proto-consciousness; and (3) the macroscopic presence of human observers who project their own meaning upon the work.
The research-creation process of the work is guided by the questions and provides the method of inquiry, whereas the work itself acts as a response. These different forms of life interact with one another symbiotically and come to “imagine” new glyphs, thus generating a “living alphabet” in real time. These glyphs conceptually form a new vocabulary for the spectators, who thus become witnesses to an uncanny form of communication which they have to decode and interpret.
The process
While Sofian and TeZ’s work involves a lot of technological considerations, all of their decisions are made in the service of creating an experience for the visitor. What do we see, hear, touch? How do we circulate through the work? What do we feel and imagine?
One of their overarching goals was to find an equilibrium between the different agencies in place. They sought to preserve as much as possible the autonomy of the generative systems and balance the contribution between the biological and the computational agents, while creating an experience that could be perceived and interpreted by human agents.
These considerations are at the core of their collaboration which began several years ago.
They both met while working with previous Hexagram co-director Chris Salter on research-creation projects in the mid-2010s. Sofian was then a student member and TeZ, a collaborator. They started working as a duo in 2017, while Sofian was doing their postdoc at MIT. From 2017 to 2020, they worked together remotely and through a series of residencies at TeZ’s Optofonica space in Amsterdam, as well as Sofian’s Art & Artificial Agents (A3) lab at the University of Maine and at Clarkson University. While in Maine they collaborated with biologist Nishad Jayasundara, an expert in biological adaptation related to anthropogenic environmental change, and with whom they refined their understanding of the growing conditions and light reactions of the Euglena gracilis colonies.
Due to the coronavirus pandemic, they had to work remotely since February 2020, which dramatically complicated and slowed down their creative process. In 2021 they presented a video documentation of they process as part of the Hexagram MUTEK Forum in 2021.
The last steps of their research-creation process were accomplished at Hexagram-UQAM in the Experimentation Room through a first residency in October 2022 and a second and last residency in March 2023. During the October 2022 residency, Sofian worked on fully automating the experiment, integrating microfluidic control to refill the petri dish with the computational generative processes. An important improvement involved adaptively adjusting the exposure on the petri dish so as to give enough time for the microalgaes to move towards the light in order to contribute to the generative process.
The March 2023 residency at Hexagram was their first opportunity since 2020 to meet in person. Their main objective was to fine-tune the apparatus and create a presentation format that could be experienced directly by the public: an observatory allowing visitors to experience the work at multiple scales. The first week was dedicated to making the technical aspects of the installation more robust. In the second week, they focused on running the device, generating glyphs, and preparing the presentation.
Open Lab
On the last day of our residency on March 9th 2023, they invited the public to experiment the work during an open lab at Hexagram-UQAM’s Experimentation Room. As part of the event, Hexagram members Filip Dukanic, Alice Jarry, and Ola Siebert were invited to respond to glyphs generated by the apparatus in an inverted form of prompt-based generative AI experience. Living drinks and hors d’oeuvres including homemade kimchi devilled-eggs, kombucha, and spirulina were served. More than thirty visitors came to discover their fascinating work and encounter the bio-computational agencies performing within it.
Credits
Xenolalia is a project by Sofian Audry & TeZ.
Biologist Collaborator: Nishad Jayasundara
Assistants: Etienne Montenegro (coding, electronics, fabrication), Guillaume Cramoisan (fabrication), Matthew Loewen (coding), Ian Donnelly (biology), and Brynn Yarbrough (biology), Federico Murgia (video), Kyan Daigneault (web), and Alexandra Lamoureux (web).
The video documentation was produced by Léa Martin through the support of Hexagram’s DEMO program.
A special thanks to the Hexagram team: Isabelle Boucher, Max Boutin, Victor Brayant, Manuelle Freire, Evelyn Leblanc, Anatole Michaud, and Jason Pomrenski.
Xenolalia is supported by Hexagram, the Canada Council for the Arts, the Conseil des Arts et des Lettres du Québec, the Mondriaan Fund, the University of Maine and the Université du Québec à Montréal.
Biographies
Sofian Audry is an artist, scholar, Professor of Interactive Media within the School of Media at the University of Quebec in Montreal (UQAM) and Co-Director of the Hexagram Network for Research-Creation in Art, Culture and Technology. Their work explores the behavior of hybrid agents at the frontier of art, artificial intelligence, and artificial life, through artworks and writings. Audry’s book Art in the Age of Machine Learning examines machine learning art and its practice in art and music (MIT Press, 2021). Their artistic practice branches through multiple forms including robotics, installations, bio-art, and electronic literature.
Maurizio Martinucci (aka TeZ) is an Italian interdisciplinary artist and independent researcher, living and working in Amsterdam. He uses technology as a means to explore perceptual effects and the relationship between sound, light and space. He focuses primarily on generative compositions with spatialized sound for live performances and installations. In his works he adopts custom developed software and hardware, featuring original techniques of sonification and visualization to investigate and magnify subtle vibrational phenomena.
