Big data analytics is the science and art of bringing information and context together. There seem to be discussions of what big data can offer organisations everywhere and the majority are investing fortunes in the technology required to store and process the terabytes or petabytes of data. Investments here are necessary - compliance regulations, business continuity, recovery storage and business intelligence (BI) tools are increasingly part of the technology fabric in all industries.
However, there is growing confusion and frustration with the current output from funding a BI strategy. CIOs and CFOs spend millions on producing key charts and data to validate their corporate strategy, yet the gap between the plan and outcome is leaving many scratching their heads over the “return on big data”.
But why? Many aspects could be the problem. However, a regularly-occurring oversight is the presumption that the continuous presence of human input on data analysis is unimportant. The general practice within organisations is that only a select few have access to big data and the tools required to make sense of it. Even then, once business rules are implemented, they are often left unchecked by both business and technology leaders. Due to a lack of attention, this static implementation begins negatively impacting how the business achieves its strategy.
This is not to advocate that the “golden keys” to information be handed to every single employee of an organisation. Instead, it is about employees being given the training and resources to access data. In the not-so-distant future, tiered storage and a virtual data processing cluster for every worker could be the norm. This is not very different from when manufacturers adopted Kanban and WCM (World Class Manufacturing) by which employees are empowered to find optimal ways of working with tools and incentives.
Heads of technology and CFOs are likely concerned at the thought of every employee being given access to such tools - but recognition of how individuals and groups make decisions is paramount for businesses to succeed. Organisational designers and business specialists agree on a common ethos: diverse teams make better decisions with better results. The same reigns true for a data analytics strategy: if properly supported with technology and an organisational set of performance indicators, it can drive a win-win relationship between employees and corporate outcomes.
Logic versus intuition
New research has demonstrated how critical thinking and emotions are tightly linked. Whereas in the past there was much support for the separation of logical and emotional thinking, there is now a growing recognition that emotional investment in decisions drives more positive outcomes as it generates commitment to act. This is because human beings have an innate ability to create and recognise patterns, not just based on logic, but on the emotive responses of people around us. Intellectual capacity at birth enables all of us to assimilate the world into discernable emotional, linguistic and “if-this-then-that” patterns, thereby unlocking the ability, later in life, to make interpretations based on those patterns. In the day-to-day world this can be facial expressions or language, but the same skills can be applied to big data when analytics and data visualisations come together.
While exciting, it is also complex as every individual views patterns in their own way, based on the unique way each individual builds his or her intellectual capacities over a lifetime. As a person’s ability to recognise patterns is distinct, so must be the ability to look at information in ways that make sense. For example, an HR resource could provide tremendous value to the CFO by predicting labour cost impacts months, if not years in advance by aggregating applicant salary requests, recruitment site details and inflationary data. This, when put into the context of big data analytics, means that by enabling all employees to have access to collected data and the tools to analyse it in their own way, new ideas and insights can emerge that may not have been thought of if restricted only within the scope of the traditional leadership and IT teams.
The “democracy of data analytics”
By advocating a “data analytics democracy”, a company can unlock new insights and new ways of looking at data to extract more intelligence. Not all the data should be made accessible to every single employee in the business - this would be a mistake. However, employees should have access to relevant as well as affiliate (adjacent, if you will) data and tools that will allow them to explore and use information in potentially new ways. This won’t counter master data management programmes or investments in analytical tools if the right security policies are put in place. Instead, imagine a data analytics playground where employees are able to explore, change apparatus and move data to create new ways of working. And, like most playgrounds, a fence and the watchful eye of the technology team are there to supervise employees at “play with data” – on elastic storage platforms designed for experimentation while capturing the potential contributions of the employees.
Big data and the immense capabilities of analytical thinking need to come out of the back office and be given a human perspective. Only when the context of entrepreneurial employees is injected into corporate information will businesses be able to move to the next level of BI, where the voice of data democracy is heard. More storage and more access to tools will produce more innovation, greater productivity and a greater personal and emotional investment by employees into the business. Only then will a stronger and more unified company, from top to bottom, emerge.