It’s the chicken and the egg question of data analysis. Should you devise the question first and find the data to answer it or analyse the data first and later find a problem that data can solve? During a presentation on collecting data for strategic insights at Digital Transformation Expo, a data professional from Transport for London and a cloud specialist from Microsoft shared their perspectives on their ways of working.
Anthony Bartolo, senior cloud advocate at Microsoft said it is important to know the question first. This is because the question determines the type of pattern that you look for in the data. He said: “Is it a classification? Is it an anomaly? This pertains to what question you are trying to answer, what opportunity are you trying to get to. When you know that, then you know the algorithm and the pattern you are searching for.”
Dale Campbell, principal data scientist at Transport for London agreed and said that the question should always originate from a business problem. He said: “If you identify a significant insight and you have no mechanism by which to then bring that into the business process, or the decision-making process, it’s worth nothing.”
He said that all questions that data scientists and analysts seek the answers to should come from the position of ‘If I knew x then I could decide on y’. Campbell stated that once the question has been decided, there are a variety of methods to look for the answers.
He said that the tools at the disposal of data scientists mean that many patterns can come to light. He said: “With the ability of neural nets, unsupervised modelling and deep learning you can come to patterns that you might not have recognised that still facilitate and solve that business problem. So we try not to have a constrained view as to which is the best approach.”
In keeping with the analogy of paths and journeys, Bartolo said that business achievements are stops along a journey rather than destinations because solutions and methods of pattern-finding will evolve. He said: “The data you capture is different because what may be important one day may not be important the next.”
Campbell added that there is now more of a focus drawing many insights from data, whereas in the past, one decision would be made from data which would then be locked away. He said businesses that are mature enough to take on those insights are excited by this and will be able to move quickly, but unfortunately, those that aren’t will end up in a difficult position of losing money and trade.