Preparing to deliver an enterprise-wide data quality and data governance programme is challenging. Nigel Turner, vice president of information management strategy at Trillium Software, offers a roadmap to avoid some of the bumps in the road.
The Grand Canyon in Arizona is not the world’s longest, widest or deepest chasm, but is certainly the best known. At over 277 miles long, up to 18 miles wide and in parts over 6,000 feet deep - that’s over 1.1 miles - it is a formidable barrier. It also provides a good metaphor for data quality (DQ).
In many organisations, DQ improvement efforts remain isolated and fragmented. The predominant focus is application-specific, such as validating, cleansing and enriching customer addresses in a CRM system. There are good reasons for this. A specific data quality problem is identified, a project team assembled to address it, the team analyses the cause of the problem, selects methods and tools to address it, and invariably improves the data. The team may then be dissolved, or go on to address another DQ problem, usually in the same area of the business.
This traditional approach has delivered real benefits to organisations. But it is becoming increasingly clear that it will never deliver improved DQ across the enterprise as a whole. In this application-centric paradigm, DQ improvement is narrowly focused, with each DQ problem addressed separately, making re-use of solutions difficult. Moreover, as multiple partial solutions are delivered, the costs of delivering DQ improvement rise and the real root causes of many of the problems (which may emanate within another part of the organisation) are not tackled. To achieve true enterprise-wide DQ improvement requires a paradigm shift - a chasm must be crossed.
The greater scope and scale of addressing DQ across the enterprise is self-evident. But there are other differences. First, as the scope and scale of the problems tackled increases, complexity rises markedly. DQ problems almost always arise from shortcomings in process design and execution, people not doing the right things, and failures in IT.
And it’s almost always a combination of these things that generate poor quality data. For instance, when a process is not engineered to create and sustain good data, data gets corrupted, people develop workarounds and IT struggles to cope with the fallout. When looking at problems end-to-end across an enterprise, these problems become increasingly intertwined. Many processes are involved, many people are affected, and large swathes of the IT estate are impacted.
In these circumstances, it is usually hard to untangle the complexity, so that it is difficult to establish where any particular DQ problem starts and ends, and who is involved in creating it. Worst of all, no-one appears to own the problem end-to-end.
In these circumstances, the main challenge facing the DQ practitioner is to find a way through the forest of confusion and obfuscation. Only by doing this can an effective strategy be developed and delivered. This strategy is essential if the chasm between application-specific and enterprise DQ is to be crossed.
Two ancient historical figures fortunately provide guidance on how to make this happen. The first was Lao Tze, who lived around 600 BC, whose day job was curator of the Imperial Archives of the King of Zhou. When asked about the best way to tackle a lengthy and arduous journey, his advice was that, “every journey of 1,000 miles starts with a single step”. Lesson one from history is therefore that when faced with what appears to be a massive task, make a start - acting is better than endless analysis.
The second great inspiration is Alexander the Great who lived some three centuries after Lao Tzu. When he entered the Kingdom of Phrygia on his journey of conquest, Alexander was confronted by the puzzle of the Gordian knot. The knot was a tangled and seemingly intractable puzzle - whoever could untie it would become the rightful king of Phrygia.
Legend says that Alexander examined the knot, took out his sword, cut through it in one bold stroke and claimed the kingship. The key lesson from this is to seek ways to cut through and overcome complexity. When trying to tackle DQ at the organisational level, it’s vital to think about what the ends of the enterprise DQ journey look like, but plan a strategy to get there in realistic and manageable steps.
So how do you achieve this in practice? A great place to start is to understand the key strategic goals of your organisation. When I meet DQ professionals, I am often surprised that many do not know what their organisation’s goals are, yet then go on to complain that their business colleagues do not understand the value better DQ undoubtedly brings.
To sell DQ at the enterprise level and to win the support, resource and funding to start to make it happen, DQ people must be able to relate their mission to what the organisation is trying to achieve, whether it’s increased revenue, lower costs, higher productivity, closer compliance with regulation and legislation, one-to-one customised marketing communications, and so on.
Then go out and talk to the business and IT people who are tasked with making these things happen. Ask them questions like: What key data do you rely on? Where is this data sourced? Who creates it? Who uses it? How accurate, reliable and complete is it? What happens when the data is found not to be accurate, reliable or complete? Why do problems occur? What is the impact on the business and IT? Who, if anyone, is responsible for the data? What costs are incurred to fix it or work around the problem?
From these discussions start to build a log of DQ issues, focusing particularly on common data themes that emerge, for example, incomplete address data, poor quality sales data, missing stock records, etc. Use the above analysis to create a DQ vision, a statement of intent with a set of long term aspirations, for enterprise DQ improvement. This should lay out how data supports achievement of key business goals now and into the future.
This vision should be concise and expressed in business language, readily understood by all in the organisation. It should also contain a statement of what high level steps (for example, developing standard rules for address management to support all key business functions) will be taken to make this vision a reality. All in the DQ team should be able to explain it to anyone in the business whom they talk to.
The next vital activity is to build a cross- enterprise team to start to address and tackle the issues, with a core focus on those that can be directly related to the delivery of key corporate goals. Addressing DQ end-to-end requires a new emphasis on horizontal governance structures as often the places where DQ problems are felt are not where the problems originate. For instance, sales may generate inaccurate addresses, but the real pain is experienced in despatch and invoicing, when the goods ordered can not be delivered to a valid address or the invoice gets lost in the ether.
Trying to use the usual vertical/departmental management chains to resolve this DQ issue rarely works. The much more effective alternative is to assemble and sustain a cross-enterprise Steering Group from all affected parts of the organisation. This should be led by the business and supported by IT. Together they can understand the total impact of problems on the organisation as a whole and have at least a fighting chance of alleviating them.
They can also prioritise the enterprise DQ issues logged above and endorse the business cases for tackling them. This Steering Group can start by addressing a single DQ issue, but become a permanent and evolving structure as further issues are tackled. It can also start to assign data ownership and lay the foundations of enterprise-wide data governance.
In parallel with this, also consider how IT can best be organised to deliver the technical components of DQ improvement projects. Many companies have successfully done this by creating an IT Centre of Excellence (CoE) for DQ. The CoE becomes the sole point of delivery for DQ improvement projects, selects and manages enterprise-wide DQ tools, develops and implements common approaches and methodologies and, most important, delivers the IT components of DQ improvement initiatives. The CoE Head becomes a critical member of the Steering Group and provides a single point of contact into IT.
So, with a clear vision of how DQ supports corporate success, a single log of cross-organisational problems, a Steering Group aligned to support and deliver them, and IT organised and tooled to support technical deliveries, the enterprise DQ chasm can start to be traversed. It may - and probably will - be a long journey, but the first steps have been taken.
And remember there is one more potential chasm to cross along the journey - your own skills as a DQ practitioner. Ensure you have the will, motivation and leadership to persuade others to come with you on your journey to the enterprise DQ destination. Do not underestimate the importance of nurturing your own soft skills. Your powers of communication, influence and negotiation will be critical in convincing others of the need for a new approach to DQ improvement. Enterprise-wide DQ initiatives require strong leadership, so be prepared to be a leader.
Some key lessons learned from companies who have successfully delivered real enterprise-wide DQ improvement and started to bridge the chasm are:
•The critical success factor is to foster and nurture cross-organisational and cross-functional collaborative working. Steering Groups and CoEs are ways of making this happen, but also important is to communicate often with stakeholders, including senior managers and all creators and consumers of data. This ensures sustained support for your initiative and reinforces its value to your organisation.
•Any enterprise DQ initiative must be led by the business with IT closely aligned behind it. Starting an enterprise DQ programme within IT can work, but getting business leadership and continued, active business involvement is imperative.
•To ensure that the work has visibility and influence and to break down organisational barriers, it is essential to secure a senior business champion. Ideally, he or she should be at CxO level.
•Ensure that you identify all the key stakeholders of the data domains you are addressing and create and implement a stakeholder engagement plan to underpin your communications activities. Don’t just focus on senior and middle managers, also get representatives of data creators and consumers involved. They have a more in-depth understanding of the impact of the improvements you are introducing and their effect on business processes and how people act.
•Do not kick off any constituent DQ improvement project without a business case. This will help to ensure it will deliver provable business value and help to get business support.
•Ensure you start to realise benefits as early as possible by regular deliveries of value-enhancing projects. Aim to deliver something of real benefit to the business at least every 90 days, even if this is a proof of concept, a trial or a pilot. Then communicate its results to all affected stakeholders. Select initial projects with great care. Failure of early projects can jeopardise the whole initiative and damage your credibility.
•Finally, developing and enforcing standardised approaches, methodologies and DQ improvement toolsets across the enterprise is vital to encourage reuse and the extension of solutions to new data areas and problems. Ensure the tools you acquire and deploy are scalable to enterprise level.
As the great Welsh politician David Lloyd George once said, “don't be afraid to take a big step if one is indicated. You can't cross a chasm in two small jumps.” If you are bold and lead from the front, chasms can be conquered.
(This article is based on a webinar presentation delivered by the author to the International Association for Information and Data Quality (IAIDQ) in February 2012.)