I think a lot of us were counting down on New Year’s Eve with a mixture of exhaustion and hope. To call 2020 a challenging year would be an understatement. While a new lockdown in the UK may have tempered optimism, the reality is that with vaccines on the way, 2021 can still be a year of opportunity.
This is particularly true for data science and there’s no need to wait for lockdown to lift in order to seize it. The outsized role data science played last year has positioned the industry to thrive. We are likely to see some big changes over the next 12 months that will dictate how data science is used by organisations over the next decade.
At a macro level, data science has played an intrinsic part in helping governments across the world and, by extension, the general public to understand and respond to the pandemic. While not all of this was positive publicity (see the exam results fiasco), it has led to a much wider realisation of the potential of data analysis and tools such as artificial intelligence and the need for their proper implementation.
I believe this visibility will translate into more people, both casually and in a professional sense, educating and up-skilling themselves in data science. A by-product will hopefully be a public that is better equipped to give informed consent on how their data is used. In the long run, this may raise the bar on transparency and data ethics as more people understand the full implications of sharing their data.
The flip to online-only was a wake-up call.
Drilling down into specific trends, two important issues emerged during and after the first lockdown. First, for many businesses digital transformation moved from a buzzword thrown around by consultants to a fundamental issue critical to their survival. For those that had only paid lip service to it, or worse, invested poorly, the flip overnight to an online-only world was a profound wake-up call.
We saw many companies failing to provide the most basic services or information to their customers. Remote workers were left with inadequate systems, contact centres and websites were overwhelmed, and huge gaps in company knowledge and processes were exposed. A lot of businesses realised that they knew a lot less about their customers and their own organisation than they thought.
The root of many of these problems was a lack of proper data management - information was incomplete, inaccurate, irrelevant, on the wrong systems or inaccessible to all but a few. Companies that had spent wisely in data tools and integrated them into a coherent digital transformation project while also training their staff performed exceptionally well and outdid their competitors by a considerable margin.
The immediate response to this issue was a huge and rapid investment in infrastructure. Not all of this money will have been spent well. Similarly, throwing money at the problem will not have solved intrinsic problems such as a lack of skills or data education within workforces or properly integrated processes.
I expect investment to continue apace this year, but a greater emphasis placed by business leaders on making their entire organisations truly digital and data-led. This will inevitably lead to greater scrutiny of existing data teams to determine ROI.
A knock-on effect will also be more demand for data skills in general. However, with the skills gap in the UK already wide and Brexit likely to reduce the talent pool further, the cost of data science will rise. The most progressive organisations will take this opportunity to up-skill their existing staff and focus on educating their workforce to become more data literate. This will help to safeguard return on their data investment.
CEOs were frustrated despite having invested in data.
The second big issue was how businesses responded to radically altered customer behaviour. With continually changing Government guidance, B2C businesses were faced with a huge number of unknowns in a dynamic environment. A lot of organisations quickly realised that data provided the only tool to help them navigate these choppy waters.
The organisations that struggled badly had a history of under-investment or had seen data as relevant only to their marketing departments and siloed it there accordingly. Customer experience is affected by every part of your operations as a business, so organisations paid a heavy price for not using data to inform and automate in these areas.
The power of data science and its crucial role in providing insights was plain for everyone to see. I am guessing there are a lot of CEOs who were surprised by this, frustrated by not having what they needed at their fingertips despite having made an investment in data, and perhaps struggling to know exactly what they needed to demand of their data teams to offer them the business returns they need.
Many will now spend 2021 up-skilling themselves so that they are better able to develop their company’s data capability and hold their existing teams to account. I would expect plenty to find the perhaps less sexy field of data engineering is where they need to place their focus. They are missing the building blocks they need for data science to take effect.
In short, perception of data science has moved from a “nice to have” to a realisation that it is a fundamental business tool. This brings challenges and opportunities for the data science industry. On the one hand, there will be more scrutiny and demands placed on data science teams. Simply producing nicely visualised data reports will not cut it for data analysts and scientists within organisations that have seen their competitors steal a march.
There will be an expectation that data scientists can and should do more. The scientists that adapt to this world by bringing greater commercial awareness to their role, partnering with and educating the business, will find that their influence grows exponentially.
On the other hand, I expect 2021 will bring a greater willingness from organisations to experiment with data beyond the traditional domain of data science within areas such as communications to functions such as HR. With remote working likely to become an enduring trend, there is a lot of scope for data science to play a key role in aiding the monitoring, assessment and wellbeing of workers.
A critical area brought to the fore in 2020 was the continued need for rapid progress in diversity and inclusion. In HR, data has the potential to provide a more thorough and objective assessment ahead of decisions, which will in turn should offer a greater opportunity for inclusion. For example, by identifying top performing talent which, for numerous cultural or institutional reasons, may have previously not been visible to the leadership.
2021 is the year when leadership sets up for data science.
I believe we will look back at 2021 as a year when a lot of knowledge was gained which laid the groundwork for more wholesale change towards truly data-led organisations, rather than those that simply have data teams.
If 2020 was the year that data science was pushed into the limelight, 2021 is going to be the year when organisations really see what it can do and leaders have the opportunity to set themselves up for the future. The conditions are there - commercial incentives, high-profile proven case studies and wide-scale awareness.
I’ve only touched on a few of the areas that I think will see big changes. I sincerely hope that one of the major trends will be organisation-wide data literacy programmes. Otherwise, business leaders will find themselves continually disappointed by their data work failing to fulfil its potential.