Data professionals are spending so long searching for and preparing data that they have only limited time to glean the vital insights that are key to their roles.
That is according to a new report from IDC and Alteryx, entitled The State of Data Science and Analytics, which quizzed over 800 individuals performing data functions across territories, industries, company sizes and departments.
According to the report, approximately 54 million data workers around the world face common challenges associated with the complexity, diversity and scale of their organisations’ data.
In an increasingly data-driven world, the term ‘data worker’ spans beyond the 54 million identified in this study, but the findings are indicative of the challenges specific to those engaging in significant data work in their jobs.
The report claims that data workers spend more than 40% of their time collating data, from an average of more than six data sources. They also have to study 40 million rows of data and seven different outputs along their analytic journey.
IDC director of data integration and integrity software research Stewart Bond said: “Data is at the core of digital transformation, but until organisation leaders address these inefficiencies to improve effectiveness, their digital transformation initiatives can only get so far.
“Consolidating platforms and looking for tools that address the needs of any data worker, whether a trained data scientist or an analyst in the line of business, can help reduce the friction that many organisations experience on their path to becoming data-driven.”
A lack of collaboration, knowledge gaps and resistance to change were commonly quoted issues in the study around data science and analytics. Respondents also blamed a lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the biggest barriers to overcome.
Alteryx chief data and analytics officer Alan Jacobson added: “Collecting data alone won’t digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies.
"The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyse data effectively to drive real, business-changing results. As the data landscape becomes more complex, this survey exposes the tip of the iceberg when it comes to the sheer volume of workers needing to conduct analysis on a daily basis and the untapped potential for them to drive meaningful business impact.”