Does online match-making really work?

The NYTimes has an article today about the "science" of online match-making. I put that in quotes because there really isn't any clear evidence about whether it works either way.

You have no doubt seen the ads on TV for the two most popular match-making sites: eHarmony and Chemistry.com. These two differ in approach from the other popular dating site match.com because eHarmony and Chemistry pick matches for you according to a secret algorithm while match.com lets you pick for yourself. (The people at match.com own Chemistry.com, and I assume created it as a competitor for eHarmony.)

Both eHarmony and Chemistry claim to have science behind them. Both have psychologists on staff conducting research about their success rates and what predicts success, and I will certainly grant that those scientists have access to an absurd amount of data about relationships. However, most of this data is not free to the public, nor have the algorithms -- for reasons of intellectual property -- been released. So color me skeptical.

Anyway, John Tierney at the Times brings up many of these issues:

So far, except for a presentation at a psychologists' conference, the company has not produced much scientific evidence that its system works. It has started a longitudinal study comparing eHarmony couples with a control group, and Dr. Buckwalter says it is committed to publishing peer-reviewed research, but not the details of its algorithm. That secrecy may be a smart business move, but it makes eHarmony a target for scientific critics, not to mention its rivals.

In the battle of the matchmakers, Chemistry.com has been running commercials faulting eHarmony for refusing to match gay couples (eHarmony says it can't because its algorithm is based on data from heterosexuals), and eHarmony asked the Better Business Bureau to stop Chemistry.com from claiming its algorithm had been scientifically validated. The bureau concurred that there was not enough evidence, and Chemistry.com agreed to stop advertising that Dr. Fisher's method was based on "the latest science of attraction."

Dr. Fisher now says the ruling against her last year made sense because her algorithm at that time was still a work in progress as she correlated sociological and psychological measures, as well as indicators linked to chemical systems in the brain. But now, she said, she has the evidence from Chemistry.com users to validate the method, and she plans to publish it along with the details of the algorithm.

"I believe in transparency," she said, taking a dig at eHarmony. "I want to share my data so that I will get peer review."

Until outside scientists have a good look at the numbers, no one can know how effective any of these algorithms are, but one thing is already clear. People aren't so good at picking their own mates online. Researchers who studied online dating found that the customers typically ended up going out with fewer than 1 percent of the people whose profiles they studied, and that those dates often ended up being huge letdowns. The people make up impossible shopping lists for what they want in a partner, says Eli Finkel, a psychologist who studies dating at Northwestern University's Relationships Lab.

In a blog post on the subject, Tierney cites this study performed by Steve Carter and Chadwick Snow on eHarmony and outcomes. (It was presented at the American Psychological Society meeting in 2004 meaning it has not yet been peer-reviewed.)

The study claims that individuals matched recently through eHarmony who ended up married were happier than recently matched couples who met through conventional means.

I share Tierney's skepticism about this data for several reasons:

• One, it isn't peer-reviewed and the data hasn't been released, though it looks like they would like to. Still, even if they release it, I doubt we are going to get the whole story.
• Two, it is a cross-sectional study that compares people who got married through eHarmony to people who got married other ways. The control group was recruited off the Internet by a commercial research firm. There are huge problems with selection bias in both of these cases. You can't even begin to talk about success rate for the eHarmony group because they selected out all those people who didn't succeed. Likewise, for the control group, who knows what kind of selection bias was involved in couples who were willing to participate in online surveys. It is possible that they were slanted collectively to being happier with one another, but you don't know this.
• Three, all of the comparisons were done for married couples who hadn't been married longer than five years. Now I have never been married, but I kind of doubt that is long enough to figure out if a marriage works. People grow apart, and it doesn't need to happen in the first half-decade.

All-in-all, while the companies -- particularly eHarmony -- are beating their chests about their scientific cred, until they come out with data I am still very skeptical.

To argue the other side, however, there are several speculative reasons why they might not be wrong. To suggest why, I should describe some of my personal experiences with this subject.

I have participated in dating sites on-and-off for about 5 years now. Mostly this has been through match.com, though I did try eHarmony.

I stopped doing eHarmony because after taking this obscenely long survey probing into my personal life, they give you only 5 matches -- none of whom in my case had any pictures. Having gone on several anonymous dates through match.com at this point, I was really skeptical that someone I knew nothing about -- not even what they looked like -- would be my soul-mate. (Call me fickle about that.) Maybe it is just that I don't trust decisions that I am not making myself, but I decided that their whole system wasn't going to work with me.

This is where it gets weird, though. I went on their site, and they indicated that I couldn't cancel my subscription with them unless I called up the company. Maybe they have changed this now, but I hate forceful selling practices like that. So I call the number, and I ended up having this way-too-long conversation with him that went something like this:

Representative: "But why do you want to leave eHarmony?!"
Jake: "I don't know. It just didn't seem like my thing."
Representative: "But don't you appreciate the wonder of our secret algorithm?! Didn't you get enough matches?"
Jake: "Um not really. It just didn't work out. Sometimes these things don't work out."
Representative: "Was it is something we did? We could try some much harder."
Jake: "No, no. It's not you. It's me."

The whole thing left me with the impression that I had just broken up with a website.

I can compare this experience to my experience with match.com -- an experience that has been at least mixed. Sometimes it has been awesome, and I have met some great people. Sometimes I have met some people that I was clearly not compatible with. A small minority of the people I have met people were clearly certifiable. However, I think the largest group of people I have met on match.com are puzzling because they didn't convince me that they were interested in anything serious or even really dating at all.

This is where I think eHarmony and match.com differ, and why I think that eHarmony might work quite well for some people. The people who join eHarmony in my experience are people who are really, really ready to get married. The people who join match.com are generally more interested in casual dating. This, I think, accounts for at least some of the difference in results. If you go to a site looking for long-term and all that you meet are other people looking for long-term, then you are likely to find it. The entire concept of a match-making site functions on the selection bias of people willing to settle down. Absent this selection bias, I am not convinced that it would be any more successful than meeting someone at your average bar.

I am interested to here other people's online dating experiences. Has anyone been successful with eHarmony? What was your impression of their typical clientele?