If you are reading this blog then you are likely to work with data in your daily life, whether through a marketing or IT based function. Is the title a bit of hype dreamed up to coincide with ‘Big Data’ or is there a real need for a ‘Scientist’, when working with data in the world of the internet of things. So, when this question was posed at the DataIQ conference recently I felt compelled to sit and listen to the views of the panel.
It soon became clear that there were differing views on the role of the Data Scientist, and of course organisational structure and business needs drove this in different ways. When polled as to whether their business had a ‘Data Scientist’ around 30% of the data focused audience said that their organisation did, and as the panel discussion got deeper into the subject it became clear that the complexity and types of data available within the bucket that is ‘Big Data’ needs a more scientific approach to interpret and drive insight across an organisation.
Consequently, ‘Big Data’ needs Data Scientists. Looking at business problems and more importantly innovation, ‘Big Data’ is the raw ingredient, APIs are often the glue and the Data Scientist is used to drive a team to apply scientific methodology to validate the hypotheses. This is most definitely a different approach to the standard business intelligence team. The landscape has changed and although statisticians are still needed, technologies demand that they are also developers too.
It seems that the true value of a Data Scientist is to support business decisions which a Business Intelligence team has always done but ‘Big Data’ has changed the game. In the Data Scientist’s team there is the need for a data engineer, to consolidate and process data, then the analysts are able to review and find patterns in the data for the scientists to review; validate and interpret for business use.
However, part of the paradigm shift is that the Data Scientist needs to have broader skills than an academic qualification provides, it has been said that a true Data Scientist is like a unicorn (rather hard to come by). This is a classic problem in how to make sense of ‘Big Data’ for the wider business, scientists need to translate their findings into a compelling story, and define what success looks like. Equally, business leaders need to be more than numerate, if they are to innovate in this fast moving ‘Big Data’ landscape.