Four years ago, Harvard Business Review proclaimed that the role of the data scientist would be the sexiest job of the 21st Century. Clearly, there are still 83 years for this to happen. But as we enter 2017, what is life really like for data scientists and the organisations that employ them?
The truth is that, much of the time, newly-appointed data scientists inherit a mess of disorganised data. They find themselves managing it reactively, rather than proactively using it to deliver insights. Organising data may be one part of the job, but doing so at the expense of strategic analysis that adds value to the business is frustrating for everyone. Someone has to spend time ensuring data accuracy, consistency and accessibility. But should it be the data scientist? Probably not.
A recent Aberdeen research report shows that companies employing data scientists see a 15% year-on-year increase in organic revenue and other associated benefits. But this can only happen when a data scientist’s skills are deployed effectively.
Before creating a data scientist position, organisations need to ensure their data is fit-for-purpose. And before taking on new roles, data scientists should grill their potential employers about the data systems, processes and technologies in place.
Don’t put the cart before the horse
Data scientist employers need to define exactly what the position entails and set realistic expectations. For many organisations, the roles of data organiser and data analyst are not mutually exclusive. This can result in organisational tasks being lumped together with analytical tasks. If you want to maximise the potential of your data scientist, this attitude needs to change.
When data is not ready to work with, employing a data scientist is a waste of time and money. Processes and people need to be in place to ensure that data is clear and cohesive, accurate and accessible. Otherwise, data scientists get bogged down in governance and management when they should be unlocking meaningful insights that can deliver tangible business value.
The data scientist check-list
Data scientists themselves need to acknowledge that the work environment has not necessarily kept pace with data needs in the digital age. They need to take responsibility for their own destiny. That means having the courage to turn the tables in an interview situation to discover the true state of the data environment they could be about to enter.
Ask about how data is centralised, organised and managed. Find out whether the company has invested in technologies such as master data management. Ask if it’s possible to meet and speak with the people who are responsible for organising and maintaining data on an ongoing basis.
Any employer worth their salt will see this interrogation as an indication of the shrewd and intelligent approach you will bring to data analysis. If you reach the final stage of the selection process and find they are unwilling or unable to divulge the information you need, alarm bells should be ringing. Could you be about to find yourself wading through a quagmire of disorganised data, unable to deliver on your employer’s expectations? Your exciting new role could quickly transpire to be a poisoned chalice.
Someone’s got to do the dirty work, just not the data scientist
Many organisations are still grappling with the inherent messiness of big data. It’s not just the sheer volume, but the number of sources and the mixture of structured and unstructured information that is challenging.
Getting big data into a workable state takes a concerted effort and investment in appropriate people and technologies. You need professionals who are skilled at the gathering, processing and storage of data. People who can develop cohesive standards, guidelines and procedures as well as data management technologies and customised software tools. I recently heard someone put it very succinctly: “data scientists draw the plans, but data engineers are needed to pour the foundations”.
The scope of the data scientist role will continue to evolve. But the bottom line is that there is no magic bullet. If you want to leverage insights from your data, you first need to manage it. The data scientist role may well be the sexiest of the century, but the data engineering role could be the mightiest. Organisations with the foresight to employ both are set to flourish.