The research-creation “Scalability”, presented in this DEMO by Gaëlle Scali [student member, UQAM], is an audio conceptual exploration of fractal mathematical landscapes.
The project is a music-video series in 9 episodes that envisions fractal universes as cosmological metaphors. These universes are seen through the computational eyes of software (Mandelbulb 3D, Unity) and musical machines (Max MSP, Ableton Live, Spat Revolution). Scali, with the collaboration of Gabrielle Couillard [student member, UQAM] produces speculative universes on which the ear can project sonic fictions. In their research, the fractal becomes a vector of speculative fiction turned towards a post-psychedelic and post-digital science fiction imaginary, openly glitch in the sense that Kim Cascone defines it in his article “The Aesthetics of Failure: “Post-Digital” Tendencies in Contemporary Computer Music” (2000).
The universes created are inspired by the metanarrative concept of sonic fiction (Kodwo Eshun) and scientific myths in the context of contemporary and electronic visual music. The audio-visual language created disengages from the gravitational pull of narrativity and flirts with poetry, mutant textures, mixillogics and its shifts towards more obscure metanarratives. Pierre Schaeffer’s reduced listening, Hyde’s visual suspension, Michel Chion’s writings on audiovision, or Don Ihde’s phenomenology are important notions when it comes to studying sound in images.
The music-videos “Scalability” are research stations of sonic and speculative writing, sonification of visual experiences from improvisation, programming, musical composition and electronic live music performance.
Scalability at ARS ELECTRONICA (2021)
Video presented as part of the Hexagram Garden “Emergence/y” at ARS ELECTRONICA 2021 festival.
Artistic, audio and visual design : Gaëlle Scali Sound design : Gabrielle Couillard Mathematics and Development : ATYPIX Research and Development : Ludovic Amaru
Scalability, binaural version (2021)
Binaural audio version. We recommend that you wear headphones for a better listening experience.
Credits :
Artistic, audio and visual design : Gaëlle Scali Sound design : Gabrielle Couillard Mathematics and Development : ATYPIX Research and Development : Ludovic Amaru
In this clip, two “fractal Mandelbrot” game object components move on the screen. The components are mixed together by a difference smooth calculation. Each of the “fractal Mandelbrot” game objects have a Julia animation, with an additional modifier process. Unity receives as input midi notes information sent by Ableton Live. The audio sequence that plays on the image is the mathematical sequence A137560 extracted from the online encyclopedia of integer sequences oeis.org. Unity interprets the incoming note sequence and performs programmed actions in real time. For all incoming notes C C# D# F# G, a 45 degree rotation on the Y axis is executed, for all incoming notes D F A, a 45 degree rotation on the X axis is executed, for all incoming notes E G# A# B, Unity activates a glow effect (color change) via the midi note script. The midi control change 12 of the synthesizer plays on the powering of the game objects ”fractal Mandelbrot” and the midi control change 13 modifies the scale (the scale of the fractal) via the midi control script.
Generation and exploration of a game object component “fractal Mandelbrot” in Unity in VR mode
Currently, this research-creation is evolving into real-time musical and visual technologies. Scali and Couillard are looking to create an immersive experience where we could dive into the heart of a fractal from a virtual reality experience. The idea is to be able to interact in real time with the object and activate a sonic fractal score. They also seek to develop this sonification in a spatialized sound path in real time, from a human, machine, software interface.
The graphic, spatial, and sonic interaction is between Ableton Live, Max MXP, and Unity software from midi change control, midi note, audio input, user input. They envision this work as a sonic, graphic, visual score and seek to write an improvisational framework for the user to experiment with a system.
From this encounter, between the system and its user, a framework for improvisation and performative writing to the image opens up. The project focuses on interactive musical writing based on programming from theoretical tools inspired by cybernetics and post-digital, audiovision and sonic fiction.
Biographies
Gabrielle COUILLARD : After completing a bachelor’s degree in theatre studies at UQAM, Gabrielle Couillard explored the world of live performance in both sound and production, particularly interested in the design of soundscapes that evoke significant places and feelings. Currently completing a master’s degree in experimental media, she intertwines different creative experiences and audio techniques, including poetry, field recording and sound chain transformations, to create small abstract sound worlds. She is particularly interested in issues of sound spatialization and immersivity, both from a technical and creative point of view. She explores the relationship between body and space through sound, through her own experience of the body in a situation of disability. Since 2019, she has been mixing sound improvisation through telematics with her practice, with collaborators from Colombia (Festival international de la imagen, 2020 and 2021), Toronto (Exit Points series, produced by Michael Palumbo) and Montreal (including the live drone series with Kasey Pocius, weekly webcasts that lasted a year in the heart of the pandemic), among others.
Gaëlle SCALI : French-born visual artist and musician Gaëlle Scali lives and works in Montreal, Quebec. She completed a post-graduate degree in electroacoustic practice “Arts and sound creations” at the École Nationale d’Art (Ensa) de Bourges between 2018/2019 and a master’s degree in visual arts at the École Supérieure des Beaux-Arts de Montpellier (Esba-Moco) between 2003/2009. She is interested in improvised music, computer assisted music composition/programming practices, live electronic performance and the history of electronic music. Her work explores the physical and immersive dimension of sound through performance and improvised music. Also involved in a visual art approach, she is interested in the media territory of techno music, its technical, cultural, social and aesthetic dimensions, and in sound mediation in the visual arts. Gaëlle is currently doing a master’s degree at UQAM (Université du Québec à Montréal) in media-experimental research-creation.
Ludovic AMARU : Passionate about computers since the age of 12, Ludovic Amaru began coding in basic on a Casio calculator, Texas Instrument, in assembler on Hewlett Packard and on an Amiga computer. His passion for computer science led him to train in electronics, computer management and development and his affinities lead him to be interested in computer music (Amiga Pro Tracker, Cubase, Akai S2000…), video games – he was responsible for the communication pole of the Parisian association of gamers Baloku (2008-2010). Ludovic masters many development languages such as Turbo Pascal (Delphi), Java Script, C#… He is currently a developer at Apo’G (Paris) and collaborates on various artistic and utility projects in research and development. He lives and works in Draguignan in the south of France.
Atypix : Anarchist activist, passionate about fractal and mathematics since he was a child, Atypix obtained a DEA in mathematical logic and the foundations of computer science at Paris 7 Diderot (2004). He continues his research on the links between mathematics, logic and computer science by studying Christol’s theorem, a bridge between algebraicity, automaticity and logical definability. This inquisitive person likes to feed on probable or improbable bridges between scientific practices, philosophy, art and history. He is currently a computer engineer in CAD/CAM (Computer Aided Design and Manufacturing) in the Montpellier area in France.
