If you have been using a dating website recently and assumed the matches it gave you had a data-driven scientific basis, you could be in for a big surprise. In a blog admission that has gained huge publicity, US-based OKCupid.com (owned by Match.com) has revealed a number of experiments it has run to test the quality of its matching algorithms.
In one case, it swapped the results around so people with a 90 per cent match were told it was only 30 per cent and vice-versa. (You can read in more detail here.) Turns out that if you tell them they are compatible, they will get into a conversation - tell them they are not and they won’t.
So far, so placebo. What that outcome reveals is the role of expectations in conditioning how humans behave. This was further proven by another of OK Cupid’s other tests, which removed the text from daters’ profiles half the time they were displayed and then looked at how this impacted on results. Turns out copy is only responsible for 10 per cent of how an individual gets rated by others. Again, this serves to prove that what we see affects how we perceive it - dish up Michelin-star quality food in a burger box and it will not attract the same reverence or be considered worth £30 a course.
Oddly, OK Cupid appears not only to have ignored the role of these expectations within its dating service’s matching processes, it may also not have realised how they would affect customers when they learned they had been the subject of these experiments. To put it mildly, they have not been amused.
For one thing, some daters will undoubtedly have been chatting to and even met others that (according to the data and algorithm) they really had nothing in common with. That involves a high level of personal investment, given the potential for excitement or humiliation in equal parts that any date involves. Users of the website have trusted it to save them from some of that exposure by doing the preliminary filtering on their behalf.
Now they have learned this might not have been the case, it is hard to see what will stop them from abandoning the site in disgust. Worse, some may even respond to the disclosure with legal threats (this was in the United States, after all - a country founded by lawyers). It matters little that the service is free - they have been put in jeopardy, whether measured in a few minutes of fruitless online banter or the bigger risk of a real-world meeting.
Which makes the admission by OK Cupid of its practices a perfect touchstone for any company adopting a test-and-learn strategy. Before embarking on the scientific quest for the perfect business, consider the human costs to your customers. Are they simply being offered variations of the same core product/service, in which case there is no risk to them of any genuine loss (be that time, utility or value)? Or is something of real meaning being tested, in which case some form of consent is the only way to proceed in order to avoid being bitten.