Art vs. Science, Part Two: You want raw data? You can't handle raw data!


To follow up on my earlier post about Semiconductor's short film "Magnetic Movie," I want to share my favorite Semiconductor film: "Brilliant Noise." It gives me goosebumps every time I watch it.

In daily life, we avoid looking at the sun, but I challenge you to rip your eyes away from this film. In Semiconductor's hands, the sun is dynamic, unpredictable, even ominous. Quite the makeover for an object we take for granted!

More. . .

Brilliant Noise from Semiconductor on Vimeo.
Ruth Jarman and Joe Gerhardt
NASA Space Sciences Laboratory, UC Berkeley, California, USA. 2006
(click the four arrows on the bottom bar of the viewer to expand the film to screen size)

"Brilliant Noise" is similar to "Magnetic Movie": it blurs the line between data and art, while celebrating the mystery and wonder of a hidden world we rarely contemplate. But unlike "Magnetic Movie", which mixes fanciful illustrations of magnetic field lines with real backdrops, the visual and auditory raw material for "Brilliant Noise" is all actual data. The artistry lies in arranging, editing and filtering that data:

The visual noise in the images is caused by natural and man made interferences. The white noise is cosmic rays impacting the CCD of the satellite camera, we also see frame dropouts and one frame taken from a ground based observatory which shows the silhouette of a plane as it crosses the path of the observatory. We wanted to leave these flaws in as they reveal something about the tools man uses to capture these images and makes them more tangible. These disturbances are routinely removed by NASA to create a cleaner image, they then go on to colourise them.

The sound is derived from solar natural radio and controlled via digitally sampling the intensity of the brightness of the image. The sound is intrinsically born from the image, creating a symphony by the Sun. By doing this we wanted to enhance the sun as natural phenomena. Working with a documentary approach, we wanted to indulge in the raw material that is our Sun, using the image to control the fluctuation of the sound would emphasize the transitions and processes taking place.

One could argue that in this film, the artists do less manipulation of the raw data than NASA does when its people clean, edit, stitch together, and colorize astronomical images for public consumption. If so, who's representing the data more accurately - artists or scientists?

Don't get me wrong: I'm a huge fan of APoD. (My laptop wallpaper is a stunning full-color panorama of the cosmos.) But those images are not raw. I wouldn't say they're "not authentic," or "not real," because they do represent actual events and objects; but most of these representations are so thoroughly processed that they no longer resemble anything a human eye or telescope has ever actually seen.

But I don't want to pick on astronomers. The same could be said of most scientific data published today. Consider neuroimaging, in which measures of blood flow or brain metabolism are translated into a colorful spectrum which is then overlaid on a virtual "slice" of the brain. Voila, your brain looks just like a Lite-Brite!

People often assert that regions of the brain "light up" when we think or talk or move. But of course our brain isn't "lighting up" in any literal sense - we've just grown so used to viewing fMRI/PET/BOLD that we talk as if figure were physiology. Personally, I'm not as upset as many of my peers by imprecise language like "the brain lights up". We have to talk about brain activity somehow, and marrying it to a visual representation is effective for many audiences. But we also have to be careful to keep the original methodology in perspective. Don't forget how the data were gathered in the first place, and you won't make the mistake of thinking an fMRI represents "thought".

What I like about "Brilliant Noise" is that it emphasizes the imperfections and glitches in even the most beautiful data. It reminds me of the early microscopes I used as a student, crude images of neurons captured with jury-rigged lab equipment, an era before confocal microscopy and Photoshop. All that audiovisual noise makes the film feel intimate and authentic.

Yes, raw data need to be cleaned up. But when representations of that data become too smooth and polished, when software packages can massage data into art, it's easy to forget how hard experimentation really is. It's not enough to collect a data set; it must be filtered for noise, analyzed, interpreted, and ultimately converted to a representation that viewers can understand. At every step, choices are made. At every step, the technology used by the scientist leaves its imprint. The final product of this process is idiosyncratic, not inevitable.

This is what Semiconductor tells us. Those stunning NASA images of our cosmos have a B-side, and "Brilliant Noise" is it: an eerie, grungy, steampunky data remix. When the airplane's outline blinks in and out of existence, interrupting the flow of light, the viewer is forced to disengage from the abstract forms unfolding on the screen and remember that this film is constructed of still images taken from specific locations, at specific times, using imperfect technology. Sometimes planes get in the way. Sometimes the sun is obscured by clouds or static. You could make a very different film out of the same data if you chose different stills, if you left out the plane artifact, if you colored the sun a happy shade of orange, if you used "Thus Spake Zarathustra" as the soundtrack, if you added a melodramatic Hollywood voiceover. How often do people think critically about the images they see in the New York Times' Science Section or the films they see on NOVA? And yet each of these images has been processed, packaged and framed in a myriad of ways!

Scientific figures, like art, are interpretations of reality. Artists may take more license than scientists - much of the time, at least - but even something as dispassionate as an X-ray or photograph distorts reality in certain ways. Our eyes and ears are far from objective; our brains and memories even less so. Pushing past our biases, biological limitations, and technological limitations so we can study what is actually there (whatever that means!) can be the hardest part of science. And if I were to show "Magnetic Movie" and "Brilliant Noise" to my classes, that's the topic I'd like to discuss afterward.


More like this

Magnetic Movie Semiconductor (Ruth Jarman and Joseph Gerhardt) Last fall I stopped by the Hirshhorn Museum's Black Box theatre to watch a short film by Semiconductor (the artistic team of Ruth Jarman and Joseph Gerhardt). Magnetic Movie is a color-drenched, imaginative tour of Berkeley's Space…
Recent advances in functional neuroimaging have enabled researchers to predict perceptual experiences with a high degree of accuracy. For example, it is possible to determine whether a subject is looking at a face or some other category of visual stimulus, such as a house. This is possible because…
I've covered the work of Semiconductor before on these pages: Ruth Jarman and Joe Gerhardt are responsible for a series of visual and acoustic sculptures that intersect art and science. Previously they brought magnetic fields to life for Channel 4, and recaptured minerals squeezed into existence…
Cajal's Revenge, 2007 Katherine Sherwood It's been a while since I've been able to focus on my Art vs. Science series of posts, so I understand if you've forgotten Part One and Part Two. A quick recap: in Part One, I asked, I think that lay audiences approach scientific art differently - perhaps…

I think that NASA should be considered artists as well. After all, they're taking the pictures! They CARE about the data, they want to portray it for what they believe it is - beautiful.

People are fed a lot of false information from many different sources, how they chose to analyze it is a personal decision I think. I agree that scientific organizations should endeavor to produce 'true' scientific data, but I don't think they should be limited to producing only raw data. The bottom line is NASA and others are doing a great job bringing the universe to us, and we should be thankful that so many talented dedicated people are concerned with our viewing pleasure.

Brilliant. Thanks for sharing this. As a non-scientist, data is not something I'm accustomed to.

I enjoyed the "steampunky" video, but I enjoyed even more your discussion of the unavoidable arbitrariness of scientific data interpretation. Our senses are limited and the data has to be compressed and altered in order to be comprehensible. Choices have to be made, and thus enters a certain artfulness into interpretive presentations.

I also like the APOD images while knowing well that someone has massaged those images for maximum visual impact.

Excellent point. The interpretation becomes a metaphor to make understanding clearer, not depict a clear reality.

A gentleman at the conference mentioned this about images from space as well. I've been pondering the point made on Ideonexus about computer platforms being artistic visual metaphors. A lot like your "brain lights up" example.

Brilliant noise falls into that uncomfortable space, I think, that most (non-sciencey) people don't like to consider. Or else they may make the mistake of writing off science as only one kind of subjective truth.

There's still value in playing with it: what it most fascinating is how the science and subjective artistry mix.

Best blog post title ever!

I wish more people would understand that the scientific enterprise is merely the most difficult and constrained artistic enterprise ever embarked upon by humankind.

When I hear the words "art versus science", it feels like a nonsensical statement - like saying "art versus the most creatively draining art humanly possible".

Very interesting blog entry. Fascinating art as well.

For those interested, the plane silhouette can be seen at approx. the 3:47 mark.

The style of this film actually reminded me a great deal of the film in "the ring": discordant audio, abrupt cuts, menacing visuals. If the sun comes out of a television in a week and kills me I'll be only slightly less perturbed than impressed.

foop, I don't know if you meant to, but you made me laugh. ;) Non-scientists have a lot more insight than they often realize! In fact, my next post in this series is going to be about how sometimes scientists are less qualified to judge science-inspired art. . .

What a spectacular film! I'm tempted to call it extraordinary, but for the fact that it is ordinary, at least for the sun. ;)

As I've written elsewhere, "at their respective best, both science and religion (re)awaken or invigorate our capacity for wonder...Where science seeks to demystify, and to build on each subsequent revelation to learn more, religion aims to make sacred that which is taken for granted, to make the ordinary again extraordinary." Semiconductor's short films are more art, perhaps, than science, but I think the line is blurred, and to excellent effect.

Spectacular. Solar flares remind me of lightning in an entirely different atmosphere.

This origami site is neat, see the folding creases on a single sheet (plane), they remind me of the symmetry of a pebble dropped in a calm pond.

the scientific enterprise is merely the most difficult and constrained artistic enterprise ever embarked upon by humankind.

Arvind, I beg to differ.

Try writing an novel without the letter 'e', like Perec did. Not easy. No, maybe even... somewhat constrained? Yes?

What irks me, I guess, is the pedantry: science is like art, but of course it is a far, far superior form. Or rather, it is far more difficult. Yes scientists are so much more admireable than artists, aren't they?

Science and art are alike in many respects, but they use different modes of meaning. Is all.

By Tyrone Slothrop (not verified) on 24 Sep 2009 #permalink