Truth and reconciliation: harder in data even than in politics?
Countries that have undergone internal conflicts and civil war often find that a process of truth and reconciliation is valuable. By acknowledging the faults and failings of the past in a fully-transparent way, society and government can be restored to a more trusted relationship. Often, this requires a neutral party to step in a mediate between those conflicting positions. From South Africa to Northern Ireland, the impact of this approach is clearly visible.
Data may not suffer from the same degree of bloodshed and fighting. But entrenched positions, blame and finger-pointing are no less evident when it comes to establishing the underlying truth of any given data point. How often have you wasted time in meetings trying to reconcile different sets of numbers, only for the discussion to break down into highly-charged infighting?
It is no surprise, therefore, that the datification of many companies undergoing a digital transformation is often led by a focus on business intelligence. The creation of a “ministry of truth” ensures that business-critical numbers are consistent and aligned is a key first step. Investing in data to remove disruptive discrepancies is essential for any organisation that wants to unlock to value of its data asset.
The way in which this gets done is evolving, not least with the emergence of sophisticated new tools and a diminishing of the “keep everything now, figure it out later” approach to big data. If you have heard mocking references to “data swamps” recently, it is because of a growing understanding that the integrity and credibility of data can not be assumed unless there has been a strong governance and quality check first.