Ian Smith, principal analyst at RAPP, won the Data Science/Big Data Leader at the DataIQ Talent Awards in June. The award, which he was not expecting, came on the back of building a recommendation engine for auction house Christie’s, a significant achievement considering it needed to be built from a cold start. Toni Sekinah reports.
Ian Smith, principal analyst at RAPP, has said that he likes learning and is always looking to take part in projects that will stretch him. He found such a project when he was tasked with creating a recommendation engine for Christie’s auction house, especially considering he had never created one from the ground up before. “I just set about reading a lot of white papers and online research on to establish the best approach,” he said.
Smith had an understanding of how the majority of recommendation engines operate and explained how a movie recommendation engine would normally work. “If you’ve got 100,000 people that have watched one film and 80,000 of those people have watched another film, you say to the other 20,000 people, ‘you should watch this film’ - it’s quite easy to do,” he said.
The principal challenge he encountered with the Christie’s engine was the lack of data volume. Auction houses sell mainly one-off items and it isn’t possible for a 100,000 people to buy a Picasso painting. Therefore, one cannot apply the same kind of logic as with the movies. This is called the "cold-start" problem. As a result, Smith had to “get a bit more creative in the data” that he used. The cold start problem, he said, is typically solved by looking at the attributes of an item.
If he were to create a recommendation engine for film that had the same cold start problem, Smith said he would have looked at other attributes such as the director, actors and the genre. “We see who’s been watching this director, these actors, this genre and say, ‘right, here’s a new film that fits what you’ve watched before. Here’s your recommendation’,” said Smith.
He applied this technique to the items sold through Christie’s, taking the attributes of those items, displaying them to clients and essentially saying, "this is similar to what you’ve bought before". Thankfully, according to Smith, specialists at Christie’s write beautiful brochures and catalogues for the sales which have descriptions and tags. Smith and his team were able to piggy back on them and use that detail to make the new recommendations.
Smith and his team were also able to use information from the preference centre where clients of Christie’s save the types of items they are interested in. Smith said that, with that action, the clients were giving an indication of what was important to them. For example, a client that is seeking a ring could say what material they would prefer it to be made of, the desired age of the ring and that it be from a particular jewellry designer.
The analyst pointed out that, as time goes by and the clients’ preferences change, he would be able to use that change in behaviour to alter the recommendation. As he is always looking to improve on what he’s created, Smith is now working on the third iteration of the recommendation engine, which actually comprises several smaller engines that produce a scores with recommendation lists built from those scores. He said that the first one only had three data sources. The second, current iteration has more data sources and Smith had come up with a way to make the scores that come out of each engine comparable. With the third iteration, which he is still building at the moment, Smith is attempting to take engines that are effectively in different silos and combine them into one.
Despite his achievement, Smith was shocked by his win, as he had seen who he was up against and thought he was out of the running. “I looked at the other short-listed candidates and thought ‘nope, I’m not going to win’,” he remembered. Although he was stunned, he was also pleased and described the accolade as “absolutely lovely.”
That is not to say that he has peaked and will be looking for an easy ride in his career from now on. He said: “What I’ve done with Christie’s has been a big challenge and it’s grown on what I’ve done before,” he commented. “I certainly hope that, in five years’ time, Christie’s will be part of the journey to bigger and better things down the road.”