Some Experimental Mathematics!

In physics, you come up with an idea, formulate it mathematically, find the theory’s predictions about the real world, and test those predictions by experiment. This works because God is subtle, but not malicious (to borrow Einstein’s words). In more concrete language, the laws of physics fit well together and make sense as a coherent, testable experimental whole.

Mathematical truths are not so easily verifiable. Because mathematics in some sense encompasses a much wider spectrum of possibilities, experimental tests can’t cover all the needed ground in most cases. The laws of physics are all mathematical statements, but math is a much broader subject that just that small subset of math that’s directly useful in physics. Much broader. As a result, most mathematical statements have to be proven mathematically. Mathematical “experiments” with computers are usually inadequate. There are exceptions like the four-color theorem which can be checked by brute force experimental calculation but those are not the rule.

A commenter who writes the very interesting blog The Outer Hoard mentioned a post of his from several years ago doing a mathematical experiment on the prime numbers. It goes like this: get some paper with a fine grid drawn on it. Start counting, and for each number go forward one space from your starting point. When you reach a prime number, turn left. Repeat. You get patterns that look like this (bigger pictures at his site):

It certainly doesn’t look like a random glob. There are stretches where the path seems to have a strongly preferred direction. Is this actually what’s happening, or is it just an illusion of randomness?

We have observed that there are regions of numbers in which every fourth interval between one prime and the next tends to be larger than the other intervals. Several intellectuals (though none of them mathematicians) have confidently told me that there’s nothing significant about this, that it boils down to the same clustering tendency you get when dealing with random numbers. And they add, as though it were some sort of reducto ad absurdum, that if my observation were significant, it would imply that there’s something special about the number four.

Unfortunately I’m not a mathematician either. I’m a recreational amateur at best. I like my lab, and the direct connection to physical facts. Theory is great and I use and develop it when needed, but I’m not naturally a theorist by temperament or innate skill. Still, I know a few things and we might be able to make some headway here.

The problem with trying to mathematically determine the expected behavior from first principles is that the properties of the spacing between the primes is not completely understood. The most important open problem in mathematics is the Riemann hypothesis, and it has to do with how “randomly” the primes are spaced. On smaller intervals, the tighest a “turn” can be are when the primes are spaced one apart from each other: (5,7) or (11,13) for instance. But even for something that simple, it’s not even known for sure if there are infinite twin prime pairs, much less how they’re distributed. So we have to use experiments for the moment until a good mathematician shows up.

First, we know the primes are not distributed randomly. They’re in exactly the places where the prime numbers are! But we do know that in a way their spacing has certain overall properties that can be thought of as statistical in nature. For instance and relevant to this exploration, the density of the primes near n is about 1/log(n). So if we expect a random walk, it’s a random walk with a gradually increasing step size as the primes thin out. We can avoid this by starting at high n, and stopping before n gets really tremendously higher. If we start at 1,000,000 and end at 2,000,000 the average space between prime numbers only increases by a factor of 5%.

Suggested experiment number 1: create actual random walks with 1/log(n) probability of turning left at each step, run many of those experiments for (say) 10,000 steps, and measure the distance from the starting point. The do the same thing for actual numbers using the original algorithm between n = 1,000,000 and 1,010,000. Then do it again between 1,010,000 and 1,020,000. Keep going until you have lots of random number data and lots of prime number data and see if there’s a significant difference in the distance from the origin between the random and the primes data.

Suggested experiment number 2: Run control groups. Instead of always turning left, turn in a random direction. And to supplement that, run a test where the turns are made after every other prime number, which should wash out any preferred direction in the original experiment while retaining the overall global average spacing of the primes. How do those distances and directions from the origin statistically differ from the original algorithm?

There is a pretty good physical analogy to this: Brownian motion, which is a physical process which is pretty much a random walk. It’s the random jiggling and moving that happens when a tiny particle like a pollen grain is randomly battered by the atoms surrounding it. From the Wikipedia article, a picture of a 2-d random walk:

Quite similar. My guess is that the apparent directional preferences in the prime numbers are just statistical flukes that you’d expect to see every once in a while, but I’m not at all certain. It looks a lot like Brownian motion. But that’s not a rigorous observation. Experimental rigor will require quantitative measurements such as in the suggested experiments above. And even then since we’re in mathematics, computational experiment isn’t sufficient to prove anything anyway. Nevertheless, it’s a good starting point.

1. #1 Uncle Al
September 30, 2008

Or swallow it as a whole. Run the lump then FT to get its power spectrum. Is there structure?

2. #2 Abby Normal
September 30, 2008

My math skills are pretty rudimentary (for a Science Blog commenter). But it seems to me that 4 may look significant here because it’s the only number being tested. Any blip is going to make 4 look special. I’d like to see the same experiment done turning 120, 72, and 60 degrees to examine 3,5, and 6 respectively. Does that make any sense or did I miss the point?

3. #3 Paul Johnson
September 30, 2008

Strange. number spirals are a lot more promising it seems. I wonder why. Maybe because probability not being strong enough causes the orientation of the path to change too much muddling the image?

September 30, 2008

Thanks for the link and the discussion (I wasn’t expecting that, and certainly wasn’t expecting my humble blog to be described as “very interesting”). You’ve also smashed the record for the number of page views my blog has received in a single day – previously 195; now more than twice that. When I get a moment, I’ll append the original post with a link back.

The experiment was not fast, for example the primes to a million was an overnight job, which was a disincentive for doing further experiments (such as the type Abby suggests). As for the directional tendency, it’s much more striking when watching the experiment progress rather than just seeing the final result; several times I had to restart and adjust the parameters because the line had gone off the edge of the screen, and a few iterations of that makes a strong impression that something weird’s going on!

5. #5 Matt Springer
September 30, 2008

Are you generating the primes as you go? If so, you could probably save orders of magnitude of time by downloading a list of pre-computed primes and just having the program read off the list. Alternatively, there’s purpose-built programs you could use to generate your own list of primes at high speed, then feed that list to the graphics program.

In fact, I’d probably generate the list, and then write up a script to find the distance between each successive prime and make a list of those distances. Feeding that into the program should make it fly, as there’s then almost no computation required other than the drawing itself.

6. #6 Sheldarcol
October 1, 2008

You fascinate me.

P.S. You’ll get me into that robotic car as soon as I’ve shot off that pink gun. In other words, there’s a greater chance of you getting to the moon and back.

7. #7 Ian
October 3, 2008

“It certainly doesn’t look like a random glob”

I find that to be a strange thing to say. Just what would a random glob look like? Evenly spaced throughout? It would only look like that once in a random while….

How would the glob look if you factored in The Prime of Miss Jean Brodie?