Part of this past weekend’s meeting of the Committee on Informing the Public was to evaluate 100+ proposals for “mini-grants” of up to $10,000 for new outreach activities. It wouldn’t be appropriate to go into detail about any of the proposals or what we decided (the PI’s of the proposals we decided to fund will be notified soon), but there was one issue that came up again and again that I think is appropriate for the blog, which is what should be considered as a successful effort, particularly in the online world.
A large number of the proposals we were considering had “new media” components to them– videos to be placed on youTube and the like, web sites with informative descriptions of physics topics, blogs about physics, and so on. One of the things we argued about at length is how to measure the impact of these sorts of items. The obvious metric would be something like the number of times the video got played on YouTube, or the number of hits a web site gets, but then it’s damnably difficult to agree on a number that would constitute success.
Let’s say you put together a video about physics, such as this one of me talking to my dog about the Many-Worlds Interpretation of quantum mechanics:
How many people have to watch this for me to call it a successful example of physics outreach to the general public?
At the time of this writing, that video has been viewed 380 times. Now, on the one hand, you could compare this to the total viewership for YouTube videos that “go viral,” like the LHC Rap, which has drawn more than 6,000,000 views to date. In which case, you would consider my little video a dismal failure.
But then, is that really an appropriate standard to set? Trying to get a video to go viral is a little like trying to find the Higgs Boson by sitting in your room and waiting for a passing cosmic ray to create one. It might happen– there are lots of cosmic rays, and many of them have enough energy to make a Higgs– but it wouldn’t be a great basis for a research program. Viral videos are, more or less by definition, random and unpredictable, so it seems a little unfair to consider something a failure because it didn’t manage a wildly unlikely achievement.
A more realistic standard might be to ask how many people are being reached by this that wouldn’t've been reached some other way. For example, you could note that 380 people viewing this video would be close to 70% of an entering class at Union, which is at least three times as many people as I could reasonably expect to teach in the course of my day job. I use this sort of standard a lot when I talk about traffic to the blog– as I note in that post, the average daily number of pageviews for this blog last year was larger than the enrollment of Union, so it’s reaching out to a much bigger audience than I have via my teaching. And that audience is global– I get hits from all around the world.
This does lead to some local variation in standards. For example, in another discussion I noted that Sean Carroll’s talk here drew 100+ people, and one of the other members replied “A hundred people is nothing!” Which might be true, at a state university with a student population of 10,000+, but it’s very good for a science talk at a small liberal arts college. I don’t think there’s any other metric that’s dramatically better, though.
the only other standard that really comes to mind is also a relative measure, namely “how well did this do relative to similar outreach efforts by the same people?” In which case, the video embedded above is a disappointment– my other two book-related videos have been viewed thousands of times, and the Bohr-Einstein puppet video has been viewed around 10,000 times, if you add up all the different versions. But that’s also subject to the viral effect– a large fraction of those 10,000 views came because John Scalzi linked to it from Whatever, which gets 50,000 readers a day. And it’s not remotely reasonable to expect John to promote anything and everything I put on YouTube.
So, in the end, I don’t have a solid idea for how to quantify the effect of outreach programs on the Internet. On the off chance that somebody else does have an idea, though, I’ll throw it out to my wild and worldly readers: What would you say is a reasonable standard for a successful outreach program?