Does online match-making really work?

The NYTimes has an article today about the "science" of online match-making. I put that in quotes because there really isn't any clear evidence about whether it works either way.

You have no doubt seen the ads on TV for the two most popular match-making sites: eHarmony and Chemistry.com. These two differ in approach from the other popular dating site match.com because eHarmony and Chemistry pick matches for you according to a secret algorithm while match.com lets you pick for yourself. (The people at match.com own Chemistry.com, and I assume created it as a competitor for eHarmony.)

Both eHarmony and Chemistry claim to have science behind them. Both have psychologists on staff conducting research about their success rates and what predicts success, and I will certainly grant that those scientists have access to an absurd amount of data about relationships. However, most of this data is not free to the public, nor have the algorithms -- for reasons of intellectual property -- been released. So color me skeptical.

Anyway, John Tierney at the Times brings up many of these issues:

So far, except for a presentation at a psychologists' conference, the company has not produced much scientific evidence that its system works. It has started a longitudinal study comparing eHarmony couples with a control group, and Dr. Buckwalter says it is committed to publishing peer-reviewed research, but not the details of its algorithm. That secrecy may be a smart business move, but it makes eHarmony a target for scientific critics, not to mention its rivals.

In the battle of the matchmakers, Chemistry.com has been running commercials faulting eHarmony for refusing to match gay couples (eHarmony says it can't because its algorithm is based on data from heterosexuals), and eHarmony asked the Better Business Bureau to stop Chemistry.com from claiming its algorithm had been scientifically validated. The bureau concurred that there was not enough evidence, and Chemistry.com agreed to stop advertising that Dr. Fisher's method was based on "the latest science of attraction."

Dr. Fisher now says the ruling against her last year made sense because her algorithm at that time was still a work in progress as she correlated sociological and psychological measures, as well as indicators linked to chemical systems in the brain. But now, she said, she has the evidence from Chemistry.com users to validate the method, and she plans to publish it along with the details of the algorithm.

"I believe in transparency," she said, taking a dig at eHarmony. "I want to share my data so that I will get peer review."

Until outside scientists have a good look at the numbers, no one can know how effective any of these algorithms are, but one thing is already clear. People aren't so good at picking their own mates online. Researchers who studied online dating found that the customers typically ended up going out with fewer than 1 percent of the people whose profiles they studied, and that those dates often ended up being huge letdowns. The people make up impossible shopping lists for what they want in a partner, says Eli Finkel, a psychologist who studies dating at Northwestern University's Relationships Lab.

In a blog post on the subject, Tierney cites this study performed by Steve Carter and Chadwick Snow on eHarmony and outcomes. (It was presented at the American Psychological Society meeting in 2004 meaning it has not yet been peer-reviewed.)

The study claims that individuals matched recently through eHarmony who ended up married were happier than recently matched couples who met through conventional means.

I share Tierney's skepticism about this data for several reasons:

  • One, it isn't peer-reviewed and the data hasn't been released, though it looks like they would like to. Still, even if they release it, I doubt we are going to get the whole story.
  • Two, it is a cross-sectional study that compares people who got married through eHarmony to people who got married other ways. The control group was recruited off the Internet by a commercial research firm. There are huge problems with selection bias in both of these cases. You can't even begin to talk about success rate for the eHarmony group because they selected out all those people who didn't succeed. Likewise, for the control group, who knows what kind of selection bias was involved in couples who were willing to participate in online surveys. It is possible that they were slanted collectively to being happier with one another, but you don't know this.
  • Three, all of the comparisons were done for married couples who hadn't been married longer than five years. Now I have never been married, but I kind of doubt that is long enough to figure out if a marriage works. People grow apart, and it doesn't need to happen in the first half-decade.

All-in-all, while the companies -- particularly eHarmony -- are beating their chests about their scientific cred, until they come out with data I am still very skeptical.

To argue the other side, however, there are several speculative reasons why they might not be wrong. To suggest why, I should describe some of my personal experiences with this subject.

I have participated in dating sites on-and-off for about 5 years now. Mostly this has been through match.com, though I did try eHarmony.

I stopped doing eHarmony because after taking this obscenely long survey probing into my personal life, they give you only 5 matches -- none of whom in my case had any pictures. Having gone on several anonymous dates through match.com at this point, I was really skeptical that someone I knew nothing about -- not even what they looked like -- would be my soul-mate. (Call me fickle about that.) Maybe it is just that I don't trust decisions that I am not making myself, but I decided that their whole system wasn't going to work with me.

This is where it gets weird, though. I went on their site, and they indicated that I couldn't cancel my subscription with them unless I called up the company. Maybe they have changed this now, but I hate forceful selling practices like that. So I call the number, and I ended up having this way-too-long conversation with him that went something like this:

Representative: "But why do you want to leave eHarmony?!"
Jake: "I don't know. It just didn't seem like my thing."
Representative: "But don't you appreciate the wonder of our secret algorithm?! Didn't you get enough matches?"
Jake: "Um not really. It just didn't work out. Sometimes these things don't work out."
Representative: "Was it is something we did? We could try some much harder."
Jake: "No, no. It's not you. It's me."

The whole thing left me with the impression that I had just broken up with a website.

I can compare this experience to my experience with match.com -- an experience that has been at least mixed. Sometimes it has been awesome, and I have met some great people. Sometimes I have met some people that I was clearly not compatible with. A small minority of the people I have met people were clearly certifiable. However, I think the largest group of people I have met on match.com are puzzling because they didn't convince me that they were interested in anything serious or even really dating at all.

This is where I think eHarmony and match.com differ, and why I think that eHarmony might work quite well for some people. The people who join eHarmony in my experience are people who are really, really ready to get married. The people who join match.com are generally more interested in casual dating. This, I think, accounts for at least some of the difference in results. If you go to a site looking for long-term and all that you meet are other people looking for long-term, then you are likely to find it. The entire concept of a match-making site functions on the selection bias of people willing to settle down. Absent this selection bias, I am not convinced that it would be any more successful than meeting someone at your average bar.

I am interested to here other people's online dating experiences. Has anyone been successful with eHarmony? What was your impression of their typical clientele?

Tags

More like this

It is a great place to meet people indeed! Especially when the two of you chat on IM and found that there's a connection, even if it's just a little bit. This might lead to real life dating and possibly marriage! I have seen people who met online and got married!

I'm not surprised to hear about your experiences with leaving eHarmony. They have been spamming me regularly for months now, even though I'm fairly certain that I never even joined their site.

If I were interested in applied social psychology, I would compare dating outcomes for paysites such as match.com with free sites like plentyoffish.com. I imagine that part of the reason why people are prepared to pay for this service (unlike pretty much every single other online service) is that you're only going to be exposed to people who were also willing to pay. Those people are likely to differ from the norm in their commitment to dating, social status, and many other variables.

So you might expect that the old adage that you get what you pay for applies to online dating, too.

Online match making could work... IF YOU DON'T PAY! I'm sorry, but match.com has never shown me any one I'm really interested in. If I want to go deeper with a person I'd have to pay at least 30 bucks to get a subscription for a potential chance of meeting someone. That doesn't seem to be right.

Another site, OKcupid.com has shown me a lot of potential matches that are similar, but I chose how I answer the questions and how I want my match to answer, and how much should it matter, the site does the search and all.

JustsayHi.com takes a simplier approach... am I interested in this person. Yes or no or maybe. If we both are maybe or yes (or one of each) the site tells me. Then we take it from there if we really are interested.

I have yet to hear back from anyone on plentyoffish.com, there's minimal matching, and I don't "get it" the site is weak, and it's cold, but It's large, and that could be good.

But here's the kicker all three sites? FREE! 100 percent. From day 1. eHarmony, chemistry, and match don't just require you to pay, they require BOTH participants to pay, and somehow I don't think "I'm desperate enough to pay for a matching service" to be a good indicator of a future partner, or even a date. There's a good amount of free sites.

Don't let someone say "well you don't care enough if you're not willing to pay to find your dream girl". The simple fact is that paying for an unknown quantity is a joke, and most of the pay date sites will do anything to make sure you don't leave the site, personally avoid them and find free dating sites, that at least show who's active (or organize the site based on activity level) then you can at least find people who are around, interested in dating, and talk to them. Somehow I think that's a better solution to dating than expecting someone else to set you up with random people.