Casual Fridays: Turns out, we’re not so good at influencing your thoughts

Last week’s study generated plenty of interest: it was the fastest we’ve ever gotten 400 responses. The study was based on a claim by this web site that they could influence your thoughts with 98 percent accuracy using a simple math quiz. If you haven’t tried it yet, unfortunately I’m going to spoil it for you right now: you’re supposed to think of a red hammer.

We wanted to answer a few questions about the claim. First, 98 percent? Really? Having done a survey or two, I’d be impressed with a survey that could predict 98 percent of the responses to “what is 2 + 2?”

Second, to the extent that the “thought control” works, how does it work?

To try to find out, we created two different versions of the test. The first version attempted to duplicate the original test exactly (our graphics aren’t quite as fancy, but we suspected that fancy graphics don’t really influence the results).

The second version changed a few things. We used different math problems, and we also changed the text of the instructions. We changed the claim to suggest that participants would think of a “yellow screwdriver” instead of a red hammer.

You can see the first version here

You can see the second version here

So, were we able to influence people’s thoughts? Take a look at the results:

Regardless of whether we claimed we were trying to cause people to think of a red hammer or a yellow screwdriver, only around 20 percent of respondents actually thought of a red hammer. This is clearly well short of the claimed 98 percent! Our respondents were just as likely to think of a blue hammer. Could the makers of the initial test simply have been lying?

Clearly they were at least guilty of stretching the truth. And, as many as 68 percent of respondents did think of either red or hammer. Perhaps they were relying on respondents to be impressed that half of their claim appeared to be true.

And what of our attempts to influence the results by changing the math problems and instructions? In our “yellow screwdriver” version we tried to come up with problems whose answers sounded a little like the words we were trying to evoke: the final problem, for example, resulted in an answer of 178, which, when you think about, sounds a little like “screwdriver.” The following two graphs show the pattern of responses to the two versions of the tests:

Note that the results are nearly identical, no matter which version of the test was given: it couldn’t have been the specific math problems that were motivating the responses — since we used different math problems in each version, the problems couldn’t be priming a response. Similarly, it doesn’t appear that the specific instructions had anything to do with the response.

So why do so many people think of “red” and “hammer”? I think part of the answer might come from this study from the Cognitive Daily archives. Adults tend to associate the color red with anger. Being asked to do a series of pointless arithmetic problems in your head may also make you angry, so this test might lead to a disproportionate “red” response. Otherwise, we might expect to see more “blue” responses. Indeed, our group did respond blue quite often — nearly as often as they said red. Perhaps Cognitive Daily readers have more patience for arithmetic than the general population, so a more typical response might involve even more “red” answers.

What about “hammer”? My guess is that hammer is just the most commonly thought of tool — it’s almost the prototypical tool.

Anyone have any other ideas? Let us know in the comments.

1. #1 Scott Spiegelberg
April 28, 2006

Politics? The symbols for Communism, ingrained in culture as The Bad Guy for the impressionable part of the participants’ lives. And no one thinks of sickles as tools.

And hammer can have a negative/violent connotation on its own. I admit that I had thought of red hammer, as part of the yellow screwdriver group. And I was a little ticked at the question. It was because I was doing something that required some concentration, and then suddenly thrown into a touchy-feely question with no apparent connection or purpose. It is the same irritation as in reading a complex murder mystery, and then finding out that the solution relies on a cheap trick that was never hinted at in the story.

2. #2 Dave Munger
April 28, 2006

A couple of points I neglected to mention in the main post:

First, we’d expect mostly blue because, as the article I linked in the main post indicates, blue is cross-culturally the favorite color among adults.

Second, an interesting artifact of the odd/even birthdays qualifier: We got consistently more responses to the odd birthdays test. Why? There are 8 months with an odd number of days. Those months have more odd birthdays than even birthdays (for example, January has 16 odd birthdays and only 15 even birthdays). This means there are 186 odd birthdays but only 179 even birthdays. This means that by the time we had 200 yellow screwdriver responses, we should have had only about 192 red hammer responses. How many did we actually have at that point? 179.

3. #3 Ben
April 28, 2006

…the final problem, for example, resulted in an answer of 178, which, when you think about, sounds a little like “screwdriver.”

How exactly are you pronouncing ‘screwdriver’? ðŸ˜‰

4. #4 Lab Cat
April 28, 2006

Red is my favorite color and I never associate it with anger. Or rarely.

I can’t remember what I thought, I know that I was thrown by the last question and couldn’t think of a tool at all!

5. #5 Dave Munger
April 28, 2006

Ben:

Yeah, it’s a stretch, but if you surf the net a bit looking for explanations for this phenomenon, you’ll find equally disingenuous explanations about numbers that sound like “hammer,” so we were trying to match that.

6. #6 Eric
April 28, 2006

Well I said blue hammer: it occurs to me that as I look at my computer windows xp has quite a bit of blue on it by default, and who knows maybe that primed myself for that color. Possibly those on macs get less blue responses? I can’t explain red, that’s odd.

As for hammer i’m pretty sure that it is the typical tool that comes to mind for anybody. It comes to mind readily for the same reason that a robin comes to mind quicker than a penguin when people are asked to picture a bird.

7. #7 magista
April 28, 2006

Huh. I took the red hammer test (even birthday) and I said silver screwdriver.

Of course, that could be because I’d used one just previously in the day to take apart an old ammeter…

8. #8 Jokermage
April 28, 2006

Thanks for answering my question! I hope you had as much fun as I did with this.

9. #9 Katherine Moore
April 29, 2006

I would think hammer is the most commonly used tool.

And I would think that red or maybe blue is the most commonly favorite or thought of color.

10. #10 Rob S
April 29, 2006

Another possibility is that linguistically, red may be the “first” colour after light/dark.
See http://pages.slc.edu/~ebj/IM_97/Lecture14/L14.html .
The hypothesis is that as language becomes more complex, the light spectrum is divided into an increasing number of named colours. The first distinction is black/white or light/dark, the next colour to be named is supposedly “red”, possibly because this is the colour of blood. If this is true, maybe our responses to the test are influenced by the same perceptual (?) bias. YMMV

11. #11 Alejandro
April 29, 2006

I don’t know if the explanation is what Rob S suggests, but for me “red” seems to be the first colour that comes to mind when asked to name one quickly (even though it’s not my favourite). I don’t think the math problems have any influence besides “clearing your mind” of all colours so the last question can get an uninfluenced response.

12. #12 Peter
April 29, 2006

I was doing the math problems in another language, so I couldn’t get primed. I still thought of a red tool (screwdriver). So I don’t think it has anything to do with the priming of a particular response. Anyhow, can you explain in more detail why onehundREDseventyeight should prime a yellow tool?

13. #13 Dave Munger
April 30, 2006

Peter: Good point — I was actually saying it “one seventy-eight,” so the red thing didn’t come into play. But maybe hundRED does help inspire “red.” Although the large number of blue responses, in either case, suggest that many respondents aren’t getting primed to say “red.”

14. #14 Jeremy
May 1, 2006

I was expecting the two quizzes to be identical and the ansers to be different. This would presumably test for compliance (a concious or non-concious “yeah I guess I was thinking of that”). I reckon this would be a more interesting experiment to carry out.

15. #15 Dave Munger
May 1, 2006

Jeremy–

Given the fact that the two quizzes were different and the answers were the same, I don’t think we need to conduct your experiment; we’ve already answered the question. Surely if we gave the identical quiz and substituted the “yellow screwdriver” prompt at the end of one of them, the results would be the same as what we saw here.

16. #16 Phronk
May 2, 2006

Very interesting…I’ve seen things like this before, and always wondered if there actually was anything to them. I’m glad you actually tested it out and debunked the accuracy claims. Any plans on pointing this study out to the author of the quiz? ðŸ™‚

Regarding Jeremy’s comment above…I think he has a point…it’s possible that the “answer” would have had an effect if it was manipulated independently. There’s a remote possibility that the different math problem and the different “answer” both had an opposite effect, cancelling each other out and leading to the null results. I really doubt that though.

17. #17 Bug
May 3, 2006

This is a standard magic trick (or charlatain mind-reading trick, for the less honest). The particular math problems don’t matter (as you discovered), they’re just to distract from what’s really going on and perhaps to make sure people really do pick the first tool that pops into their head.

Basically it just relies on the fact that each culture has just one or two cannonical examples of a class and it’s likely that people will think of that example first. In the US:

Tool: hammer (2nd place is probably screwdriver)
Color: red (followed by blue?)
Vegetable: carrot (followed by celery)
Number from 1-4: 3

The magic just trick involves having a piece of paper with the common answer on it, possibly with the 2nd-most-common answer on the other side. If you do it with a room full of people it works even better, because then if you’re wrong you can claim you were “picking up the thoughts of that gentleman over there” instead. If you’re a con-man, you then convince the true-believers that they should give you lots of money to invest for them.

Oh, and the claim of 98% success is almost certainly just a bald-faced lie to make the whole effect even more impressive when the site does guess what the person was thinking, and to further distract from the true workings of the trick.

18. #18 Gary
May 12, 2006

This effect can also be pretty well predicted by simple word frequency. If you count how many times various words are used in a database of spoken or written English, then “hammer” and “red” will be the words most frequently used to specify a tool and a color, respectively. All things being equal, when you’re asked to name an exemplar of a category, you’ll be most likely to name the most frequently used word. This is similar to the archetype explanation previously suggested, but is superior in that it can function without the establishment of archetypal representations (btw, category learning literature suggests that archetypes are not actually used).

19. #19 Jill
October 6, 2006

I took this test and came up with red hammer. I chose RED for EVERYTHING since it is my favorite color. Hammer I chose since it was the first tool to come to mind (& probably most popular) tool out there. Although, I must admit that I hate math and have little patience for it………