Only 13% of data scientists say their models are always deployed, according to Rexel Analytics Data Science Survey 2017. That's a realistic figues, according to three veteran data practitioners.
Sanjeevan Bala, head of data science at Channel 4, suggested that the question may not be specific enough. When people speak about deployment, the focus tends to be on technology deployment. However, deployment cannot be a measure of success if the model is not put into operation, which could happen if the end users have not been trained how to or are resistant to using it.
Nevertheless, let us imagine that all deployed data science models are successfully operationalised. What would be an obstacle to a data science model making it that far?
With my perspective as a non-data practitioner, I conjured up two possible scenarios. It could be that data scientists are working on models that they find interesting, but are extraneous to the needs of the business. Perhaps, the rest of the business sees the data scientists as wacky inventors who, after months working in isolation, present outlandish models that the business doesn’t quite know what to do with, which is simply the result of an organisational culture clash.
Bala pointed that the problem could be something as straightforward as the technology used by different parties. Often cloud-based, Hadoop structures and open source technologies will be used by data science teams where the focus is on agile innovation, experimentation and moving very quickly. On the other hand, IT teams are more likely to use more traditional technologies and platforms which may be different thus creating a challenge.
"Organisations are happy to ask for models and the promise of magic."
Business culture clash
It turns out, I wasn't too far from the truth. According to Enda Ridge, head of data science and algorithms at Sainsbury’s, it’s the second scenario - where the business culture isn’t receptive to the breakthrough models presented - that creates the biggest obstacle data science deployment.
He said: “Sometimes organisations are happy to ask for models and the promise of magic. When the answers come out and it turns out to be a little bit disruptive or expose some inefficiencies, they are less comfortable.”
Wade Munsie, director of advanced analytics and BI development at Royal Mail, concurred, saying that a business culture that is inhospitable to data science will be a hindrance.
He said this can often happen if top executives in the business bring in a data science function to keep up with the Joneses of the business world. If they hire a bunch of data scientists - having been inspired by a presentation at a conference - without knowing what they want to achieve, the data science team can end up failing or not getting anything into production.
In a similar vein to the wacky professor analogy, Ridge said that if data science is done in a fancy, removed, windowless lab, the business will not engage or explain its processes, which means the data scientists are less likely to create something that the business needs.
"Only five years ago, companies being ready for self-serve analytics was a big deal."
Munsie said that it is important for the company itself to be ready for data science and that could be very difficult for those that have only recently brought in data scientists to implement a data science strategy. “Only five years ago, companies being ready for self-serve analytics was a big deal,” he said by way of demonstrating how advanced and unfamiliar data science techniques are to some organisations.
Munsie added that some companies or departments are cautious of data science as it has the potential to bring about large-scale change in a particular department, and could lead to scenarios of people saying: “If it wasn’t invented here, how useful can it be?”
Renegade or overenthusiastic data science teams
It also seems that a lack of direction within the data science team can lead to misalignment between what that team does and what the business needs. Ridge, who is also the author of "Guerrilla Analytics: A Practical Approach to Working with Data", said that often organisations make the mistake of hiring a few data scientists before deciding on a data science strategy and putting a data leader in place. He said: “Without a leader like a chief analytics officer, the scientists end up working in isolation, and perhaps directed to whimsical projects and ‘curiosities’.”
Bala said that data scientists, who often come from a maths or science background, are excited by solving problems in a creative way, and while this is intellectually stimulating, they can end up creating algorithms that are too complex to operationalise.
He gave the example of the algorithm that was developed in 2009 for a Netflix competition. At $1 million, the algorithms proved too expensive to operationalise and was never used due to prohibitive engineering costs.
Bala also said that data science teams that are focused on research and development look to spin out lots of proofs of concept in order to showcase to the rest of the business what they are capable of. Very often, the resulting models are not linked to the operational considerations of the business and so this may be a case of quantity being prioritised over quality.
With some obstacles identified, what can organisations do to overcome them and increase the percentage of data science models that are deployed in their organisation?
"They know they are not trying to solve the world’s problems."
It seems that informed leadership, both within the data science team and in the wider business, is imperative for the effective implementation of data science models. Munsie said that the right leadership can reduce any misalignment between what data scientists are expected to do by a business and what they can actually achieve.
He said: “If the right leadership is bringing the data science function in, they know what they are trying to do, and they know they are not trying to solve the world’s problems.”
Ridge also said that for data science to function effectively within a business, the leaders of the organisation must a clear idea of what can be achieved. He said: “[They need] a clear strategy from the top, cascaded to business units, identifying the areas where you can leverage data relatively easy and for a lot of value.”
Ridge also stressed the importance of a leader within the data science team. Although Ridge said that this leader could come from a different business unit, ideally they would have a data science or analytics background and be able to translate business problems into potential data science solutions.
"Go off and do what you need to do - but keep them engaged.”
Communication and collaboration
Bala said that for data science to succeed it is crucial to focus on the business outcome and to do that, there needs to be a business stakeholder who owns that outcome.
To get those stakeholders on board, Munsie said they must fully comprehend what the data science team is trying to do. “Formulating that story with the stakeholder upfront is key. Then go off and do what you need to do - but keep them engaged,” he said.
For Munsie, it is imperative that he work with people whose role it is to tell the story of the data. He said the data storyteller is essential to make sure that the translation isn’t lost between what was intended and what is received. “Just presenting the facts and figures is a waste of time unless you’ve got the story behind it,” he explained.
He calls these people data journalists, although he concedes that in other companies this might be the role might of data analysts. He said that his data journalists tell the story about using data and get the rest of the business on board with what the data science team is aiming to do.
Munsie said: “That’s the only way you get buy-in and once you’ve got the buy-in, the algorithms become easier to adopt and the change management around it becomes easier.”
Ridge added that a CAO can play a similar role in that they can help the business focus on the valuable opportunities and efficiencies that can be realised in accordance with the strategy and the maturity of the business.
"It is far better and more successful if data teams spend a lot of time in the business."
Bala also said that it is important to get the data model creation process in the right order from the start. If the process starts with business units asking the data science team to come up with solutions, it is not the best approach to take.
He said: “It’s far better and more successful if data teams spend a lot of time in the business, understanding the processes and what outcome they are trying to affect and ensuring there is a stakeholder who wants that particular outcome.” After that, the data science team should figure out what models they need and what sort of data they need to support that outcome. Bala called this the “push approach,” whereby the data science team pushes out practical, cost-efficient, uncomplicated models to the rest of the business, having clearly understood a business process.
According to Bala, the opposite to that is the “pull approach” where the business doesn’t know what questions it could or should be asking of the data science team and ends up asking business intelligence or reporting type questions which underutilises the abilities of the data science team.
Marcin Druzkowski, senior data scientists at Ocado explained to attendees at the DataIQ Future conference in October the way in which his team works, which seems to be very much in line with the push approach. An algorithm that improved the way that contact centre staff dealt with emails from customers was devised after Druzowski and members of his team bedded into the contact centre to find out what was the biggest problem that they could solve and how.
Ridge is a big advocate of collaboration across the business units. He said: “The best path to success here is being able to break through silos and mobilise cross-functional teams.” He said this is because so many skills are needed including engineering support, data gathering and extraction as well as the creation of a usable front end that the business can use to access the models.
"Data science should be very focused and targeted ... instead of trying to boil the ocean."
For those companies that recognise they face some of these challenges, Munsie advised that data science should focus on lots of small wins and the business should not bite off more data science than the business can chew, adding: “Data science should be very focused and targeted on solving the things it can solve instead of trying to boil the ocean.”
Ridge also said that, if an organisation wants to "do science", it needs to understand that previous ways of working will be disrupted and so, before embarking on a data science strategy, a business needs to ask itself some serious questions. Is there the flexibility to gather data from loads of different places? Is there the willingness to run trials that might disrupt the business and their customers slightly? Is there readiness for cultural change or to change decision-making processes according to what the model says?
Bala echoed the need for cultural change and gave a tip on how barriers of resistance could be broken down; by working with the business, understanding where the business has come from and then interjecting the data science aspect so that it is complementary.
One hundred percentage deployment of a data scientist’s models is an extremely ambitious goal, according to Munsie and Ridge. Munsie said that in his team, his head of data science belongs to the school of ‘fail fast, fail often.’ As a result, his data scientists are not expecting to have wins all the way along. Ridge concurred by saying that not all projects will be successful because science explores the unknown.
Some more questions may have to be added in the Rexel Analytics Data Science Survey 2018 so that we can find out who are these infallible data scientists who produce perfect models and what their secret is.