Information theory
oracknows | June 27, 2014Yesterday’s post was just too depressing to contemplate and even more depressing to write. It was a total downer after seen the awesomeness that was John Oliver gloriously skewering America’s Quack Dr. Mehmet Oz. That’s why I think it would be good to finish this week on an amusing note. Well, it would be amusing if it weren’t for my knowledge that the woman who wrote the post I’m about to “analyze” has an autistic child and is subjecting that child to quackery. Actually, that’s true of pretty much every woman who blogs at the wretched hive of scum and autism biomed quackery where this post…
oracknows | April 14, 2010With all the anti-vaccine nonsense going on and my feeling the obligation to fire a broadside at "America's doctor," there was a tasty bit of woo that totally escaped my attention from an old "friend" of the blog. Actually, he's an old "friend" of many skeptical blogs, both here on ScienceBlogs and around the blogosphere. In fact, it's a man so steeped in only the finest quantum woo that I once coined a name for it: Choprawoo.
Yes, we're talkin' Deepak "Quantum Consciousness" Chopra! He's back and woo-ier than ever in--where else?--that repository of woo, quackery, and anti-vaccine lunacy,…
goodmath | September 21, 2009 If you regularly follow comments on this blog, you'll know that I've been
having a back-and-forth with a guy who doesn't know much about information
theory, and who's been using his ignorance to try to assemble arguments against the
Kolmogorov-Chaitin information-theoretic measure of information in a string.
In the course of doing that, I came up with what I think are some interesting ways
of explaining bits of it, so I thought I'd promote it to the top-level to share
with more readers.
To be absolutely clear up front: I'm far from an expert on K-C theory. It's something that I find…
goodmath | August 20, 2009 As lots of you have heard, William Dembski and Robert Marks just had
href="http://ieeexplore.ieee.org/xpl/tocresult.jsp?isYear=2009&isnumber=5208652&Submit32=View+Contents">a
paper published in an IEEE journal. In the last couple of days, I've received about 30
copies of the paper in my email with requests to analyze it.
My biggest criticism of the paper is how utterly dull it is. It's obvious
how they got it published - they removed anything that's really interesting from it. It's
a rehash of the stuff they've written before, stripped of any content that directly hints
at the…
goodmath | May 15, 2009 In my Dembski rant, I used a metaphor involving the undescribable numbers. An interesting confusion came up in the comments about just what that meant. Instead of answering it with a comment, I decided that it justified a post of its own. It's a fascinating topic which is incredibly counter-intuitive. To me, it's one of the great examples of how utterly wrong our
intuitions can be.
Numbers are, obviously, very important. And so, over the ages, we've invented lots of notations that allow us to write those numbers down: the familiar arabic notation, roman numerals, fractions, decimals,…
goodmath | March 8, 2009 Like a lot of other bloggers, I often get annoying email from people. This week, I've been dealing with a particularly annoying jerk, who's been bothering me for multiple reasons. First, he wants me to "lay off" the Christians (because if I don't, God's gonna get me). Second, he wants to convince me to become a Christian. And third, he wants to sell me on his brilliant new compression scheme.
See, aside from the religious stuff, he's a technical visionary. He's invented a method where he can take a source document, and repeatedly compress it, making it smaller each time.
This is a stupid…
goodmath | December 19, 2008 Since my post a couple of weeks ago about NASA and the antenna evolution experiment,
I've been meaning to write a followup. In both comments and private emails, I've gotten
a number of interesting questions about the idea of fitness landscapes, and some of the things
I mentioned in brief throwaway comments. I'm finally finding the time to write about
it now.
Fitness Landscapes
First, let me review what a fitness landscape is.
Suppose you're searching for something. There are N parameters that define the thing you're searching for. Then we can say that you're doing a search for a target…
goodmath | August 18, 2008 Someone sent me some questions about information theory; or actually,
about some questions raised by a bozo creationist arguing about information
theory. But the issues raised are interesting. So while I'm
nominally snarking at a clueless creationist, I'm really using it
as an excuse to talk about one of my favorite mathematical subjects.
The creationist is confused by the information theory idea of information and complexity. That is somewhat confusing.
Intuitively, we think of information in terms of semantics and communication. We think that information is related to semantic content.…
goodmath | September 17, 2007 Both in comments, and via email, I've received numerous requests to take a look at
the work of Dembski and Marks, published through Professor Marks's website. The site is
called the "Evolutionary Informatics Laboratory". Before getting to the paper, it's worth
taking just a moment to understand its provenance - there's something deeply fishy about
the "laboratory" that published this work. It's not a lab - it's a website; it was funded
under very peculiar circumstances, and hired Dembski as a "post-doc", despite his being a full-time professor at a different university. Marks claims that his…
goodmath | July 27, 2007
While reading Mandelbrot's text on fractals, I found something that surprised me: a relationship
between Shannon's information theory and fractals. Thinking about it a bit, it's not really that suprising;
in fact, it's more surprising that I've managed to read so much about information theory without
encountering the fractal nature of noise in a more than cursory way. But noise in a communication channel
is fractal - and relates to one of the earliest pathological fractal sets: Cantor's set, which Mandelbrot
elegantly terms "Cantor's dust". Since I find that a wonderfully description,…
goodmath | February 28, 2007 Multiple people have written to me, after seeing yesterday's
href="http://scienceblogs.com/goodmath/2007/02/basics_algorithm_1.php">algorithms basics post, asking me to say more about sorting algorithms. I have to say that it's not my favorite topic - sorting is one of those old bugaboos that you can't avoid, but which gets really dull after a while. But there is a kernel of interest to it - sorting can be used to demonstrate a lot of interesting ideas about
computational complexity.
In that vein, let's start with a little bit of complexity. There's actually a trick based on Kolmogorov/…
goodmath | February 5, 2007 I haven't taken a look at Uncommon Descent in a while; seeing the same nonsense
get endlessly rehashed, seeing anyone who dares to express disagreement with the
moderators get banned, well, it gets old. But then... Last week, DaveScott (which is, incidentally, a psueudonym!) decided to retaliate against my friend and fellow ScienceBlogger Orac, by "outing" him, and publishing his real name and employer.
Why? Because Orac had dared to criticize the way that a potential, untested
cancer treatment has been hyped recently in numerous locations on the web, including UD.
While reading the message…
goodmath | December 26, 2006Being a Nice Jewish BoyTM, Christmas is one of the most boring days of the
entire year. So yesterday, I was sitting with my laptop, looking for something interesting to read. I try to regularly read the [Panda's Thumb][pt], but sometimes when I don't have time, I just drop a bookmark in my "to read" folder; so on a boring Christmas afternoon, my PT backlog seemed like exactly what I needed.
[One of the articles in my backlog caught my interest.][pt-sc] (I turned out to be short enough that I should have just read it instead of dropping it into the backlog, but hey, that's how things go…
goodmath | August 7, 2006Ω is my own personal favorite transcendental number. Ω isn't really a specific number, but rather a family of related numbers with bizzare properties. It's the one real transcendental number that I know of that comes from the theory of computation, that is important, and that expresses meaningful fundamental mathematical properties. It's also deeply non-computable; meaning that not only is it non-computable, but even computing meta-information about it is non-computable. And yet, it's *almost* computable. It's just all around awfully cool.
So. What is it Ω?
It's sometimes called the *halting…
goodmath | August 6, 2006I received an email from someone with some questions about information theory; they relate to some sufficiently common questions/misunderstandings of information theory that I thought it was worth turning the answer into a post.
There are two parts here: my correspondent started with a question; and then after I answered it, he asked a followup.
The original question:
------------------------
>Recently in a discussion group, a member posted a series of symbols, numbers,
>and letters:
>
>`+
>
>The question was what is its meaning and whether this has information or not.
>…
goodmath | July 31, 2006While taking a break from some puzzling debugging, I decided to hit one of my favorite comedy sites, Answers in Genesis. I can pretty much always find something sufficiently stupid to amuse me on their site. Today, I came across a gem called ["Information, science and biology"][gitt], by the all too appropriately named "Werner Gitt". It's yet another attempt by a creationist twit to find some way to use information theory to prove that life must have been created by god.
It looks like the Gitt hasn't actually *read* any real information theory, but has rather just read Dembski's wretched…
goodmath | July 27, 2006I've been taking a look at William Dembski's paper, "[Information as a Measure of Variation][imv]". It was recommended to me as a paper demonstrating Demsbki's skill as a mathematician that isn't aimed at evolution-bashing. I'm not going to go into too much detail about it; it's just not that good. If this is the best work he's done as a mathematician, well, that's pretty sad.
The main problems with the paper are:
1. He either doesn't understand or misrepresents some of the fundamentals of the field he's allegedly discussing;
2. He presents many of the central ideas of the paper (that is,…
goodmath | June 24, 2006Back when I first started this blog on blogspot, one of the first things I did was write an introduction to information theory. It's not super deep, but it was a decent overview - enough, I thought, to give people a good idea of what it's about, and to help understand why the common citations of it are almost always misusing its terms to create bizzare conclusions, like some ["law of conservation of information",][conservation]
[conservation]: http://en.wikipedia.org/wiki/Law_of_conservation_of_information "wikipedia on Dembski's law of CoI"
This is a slight revision of that introduction. For…
goodmath | June 16, 2006I was recently sent a link to yet another of Dembski's wretched writings about specified complexity, titled Specification: The Pattern The Signifies Intelligence.
While reading this, I came across a statement that actually changes my opinion of Dembski. Before reading this, I thought that Dembski was just a liar. I thought that he was a reasonably competent mathematician who was willing to misuse his knowledge in order to prop up his religious beliefs with pseudo-intellectual rigor. I no longer think that. I've now become convinced that he's just an idiot who's able to throw around…
goodmath | June 13, 2006 As I mentioned yesterday, I'm going to repost a few of my critiques of the bad math of the IDists, so that they'll be here at ScienceBlogs. Here's the first: Behe and irreducibly complexity. This isn't quite the original blogger post; I've made a few clarifications and formatting fixes; but the content remains essentially the same. You can find the original post in my blogger information theory index. The original publication date was March 13, 2006.
Today, I thought I'd take on another of the intelligent design sacred cows: irreducible complexity. This is the cornerstone of some of the…