Art vs. Science, Part Two: You want raw data? You can't handle raw data!

To follow up on my earlier post about Semiconductor's short film "Magnetic Movie," I want to share my favorite Semiconductor film: "Brilliant Noise." It gives me goosebumps every time I watch it.

In daily life, we avoid looking at the sun, but I challenge you to rip your eyes away from this film. In Semiconductor's hands, the sun is dynamic, unpredictable, even ominous. Quite the makeover for an object we take for granted!

More. . .

Brilliant Noise from Semiconductor on Vimeo.
Ruth Jarman and Joe Gerhardt
NASA Space Sciences Laboratory, UC Berkeley, California, USA. 2006
(click the four arrows on the bottom bar of the viewer to expand the film to screen size)

"Brilliant Noise" is similar to "Magnetic Movie": it blurs the line between data and art, while celebrating the mystery and wonder of a hidden world we rarely contemplate. But unlike "Magnetic Movie", which mixes fanciful illustrations of magnetic field lines with real backdrops, the visual and auditory raw material for "Brilliant Noise" is all actual data. The artistry lies in arranging, editing and filtering that data:

The visual noise in the images is caused by natural and man made interferences. The white noise is cosmic rays impacting the CCD of the satellite camera, we also see frame dropouts and one frame taken from a ground based observatory which shows the silhouette of a plane as it crosses the path of the observatory. We wanted to leave these flaws in as they reveal something about the tools man uses to capture these images and makes them more tangible. These disturbances are routinely removed by NASA to create a cleaner image, they then go on to colourise them.

The sound is derived from solar natural radio and controlled via digitally sampling the intensity of the brightness of the image. The sound is intrinsically born from the image, creating a symphony by the Sun. By doing this we wanted to enhance the sun as natural phenomena. Working with a documentary approach, we wanted to indulge in the raw material that is our Sun, using the image to control the fluctuation of the sound would emphasize the transitions and processes taking place.

One could argue that in this film, the artists do less manipulation of the raw data than NASA does when its people clean, edit, stitch together, and colorize astronomical images for public consumption. If so, who's representing the data more accurately - artists or scientists?

Don't get me wrong: I'm a huge fan of APoD. (My laptop wallpaper is a stunning full-color panorama of the cosmos.) But those images are not raw. I wouldn't say they're "not authentic," or "not real," because they do represent actual events and objects; but most of these representations are so thoroughly processed that they no longer resemble anything a human eye or telescope has ever actually seen.

But I don't want to pick on astronomers. The same could be said of most scientific data published today. Consider neuroimaging, in which measures of blood flow or brain metabolism are translated into a colorful spectrum which is then overlaid on a virtual "slice" of the brain. Voila, your brain looks just like a Lite-Brite!

People often assert that regions of the brain "light up" when we think or talk or move. But of course our brain isn't "lighting up" in any literal sense - we've just grown so used to viewing fMRI/PET/BOLD that we talk as if figure were physiology. Personally, I'm not as upset as many of my peers by imprecise language like "the brain lights up". We have to talk about brain activity somehow, and marrying it to a visual representation is effective for many audiences. But we also have to be careful to keep the original methodology in perspective. Don't forget how the data were gathered in the first place, and you won't make the mistake of thinking an fMRI represents "thought".

What I like about "Brilliant Noise" is that it emphasizes the imperfections and glitches in even the most beautiful data. It reminds me of the early microscopes I used as a student, crude images of neurons captured with jury-rigged lab equipment, an era before confocal microscopy and Photoshop. All that audiovisual noise makes the film feel intimate and authentic.

Yes, raw data need to be cleaned up. But when representations of that data become too smooth and polished, when software packages can massage data into art, it's easy to forget how hard experimentation really is. It's not enough to collect a data set; it must be filtered for noise, analyzed, interpreted, and ultimately converted to a representation that viewers can understand. At every step, choices are made. At every step, the technology used by the scientist leaves its imprint. The final product of this process is idiosyncratic, not inevitable.

This is what Semiconductor tells us. Those stunning NASA images of our cosmos have a B-side, and "Brilliant Noise" is it: an eerie, grungy, steampunky data remix. When the airplane's outline blinks in and out of existence, interrupting the flow of light, the viewer is forced to disengage from the abstract forms unfolding on the screen and remember that this film is constructed of still images taken from specific locations, at specific times, using imperfect technology. Sometimes planes get in the way. Sometimes the sun is obscured by clouds or static. You could make a very different film out of the same data if you chose different stills, if you left out the plane artifact, if you colored the sun a happy shade of orange, if you used "Thus Spake Zarathustra" as the soundtrack, if you added a melodramatic Hollywood voiceover. How often do people think critically about the images they see in the New York Times' Science Section or the films they see on NOVA? And yet each of these images has been processed, packaged and framed in a myriad of ways!

Scientific figures, like art, are interpretations of reality. Artists may take more license than scientists - much of the time, at least - but even something as dispassionate as an X-ray or photograph distorts reality in certain ways. Our eyes and ears are far from objective; our brains and memories even less so. Pushing past our biases, biological limitations, and technological limitations so we can study what is actually there (whatever that means!) can be the hardest part of science. And if I were to show "Magnetic Movie" and "Brilliant Noise" to my classes, that's the topic I'd like to discuss afterward.

Thoughts?