How do you capture the sound of the stars?
That was the challenge placed before composer Tina Tallon — for the score of a Marvel movie, no less. Tallon, an AI and the Arts professor at the University of Florida, had a solution at the ready: a computer model she coded that turns spectral data from stars into sound.
“When you burn different elements, they give off certain colors of light or different types of electromagnetic radiation. That’s one of the parameters astronomers use to determine different properties of stars,” she explains. “I had already started playing with those absorption and emission wavelengths and turning them into sound in graduate school.”
Her turn as a Hollywood composer is just the latest in a series of creative reinventions for Tallon, who studied biomedical engineering as an undergrad at the Massachusetts Institute of Technology before realizing, in her senior year, that she wanted to make music.
“My parents were obviously intensely concerned about my ability to pay my rent and put food on the table,” she recalls.
But Tallon faced a more immediate challenge.
“I had taken a musical analysis class at MIT and I was just really, really intrigued by it. I remember coming into my professor’s office, and saying, ‘I think I might want to go to music school. He said, ‘if you’re going to apply to composition programs, you have to have actually composed things.’ I had to write a ton of music and get it performed and recorded in a month and a half.”
She pulled it off, but after graduate school, found that teaching at conservatories left her engineering expertise on the shelf. That changed when she came to UF in 2021.
“I do a lot of collaborations with people in medicine and engineering,” she says. “At a conservatory, they’re not interested in that. You teach the students how to write a string quartet, and that’s it. It’s really wonderful that I’ve found a position that allows me to bring all of myself and all of the things I do to the table.”
Her contributions to the score for “The Marvels” are a prime example. Using the sonification model she created, Tallon turned spectral data she pulled from astronomy research into low-frequency warbles and otherworldly glissandos that underpin composer Laura Karpman’s compositions. Central to her approach is ensuring her sonifications are more than just pretty sounds.
“I want to make sure that I’m actually communicating something immutable about the data set,” she says. “You could just take an image, translate it into a spectrum and map it to different parameters, but if you turn that same picture 90 degrees, you’re going to get completely different sounds. What is it actually telling you about the data?”
In a forthcoming study, Tallon and an undergraduate student proposed the sonification model as a way to make star data more accessible to blind and low-vision astronomers.
“This isn’t just a way to make cool sounds, it’s something that could be really helpful,” she said.
Throughout her work, Tallon wants to show that AI can empower artists and fuel their creativity, not just mine their creations to build its models.
“There are a lot of fantastic things that we can use AI for, particularly building these collaborations between artists and scientists or people across all different disciplines,” she says. “That isn’t about extraction or exploitation, it’s about how we can work together to be more inclusive and facilitate more open dialogue — more nuanced, sophisticated conversations about what it is to be human.”
Source:
Tina Tallon, PhD
Assistant Professor of AI and the Arts, Music Composition, College of the Arts
tallonc@ufl.edu