Data and analytics (D&A) leaders are hearing about DataOps and are eager to explore how it can help them deliver insights faster, with higher quality and resiliency in the face of constant change. Early conversations, however, reveal confusion about what DataOps really is, how it can be applied and what a successful strategy looks like. This confusion should be expected - it’s a new concept and what it looks like for each company differs depending on what they expect to do with their data.
What doesn’t differ is that D&A leaders need to make better-informed, faster decisions about their company’s data with a focus on automation, real-time risk assessment, and continuous value delivery to support the business’s growth in 2021.
Defining and building DataOps
Gartner defines DataOps as a collaborative data management practice focused on improving the communication, integration and automation of data flows between data managers and data consumers across an organisation. Rather than simply throwing data over the virtual wall, where it becomes someone else’s problem, the development of data pipelines and products becomes a collaborative exercise with a shared understanding of the value proposition.
To implement DataOps successfully, D&A leaders must align it with how data is consumed, rather than how it is created. By doing so, you inherently remove the roadblocks to data access and thus collaboration, innovation, and continuous value creation.
There are three value propositions you can adopt for an effective strategy:
The utility value proposition treats data as a utility, similar to any other utility you use regularly. Like when you turn on a tap at home then use the water for cooking, drinking and cleaning, data in a utility-driven value proposition is built on delivery reliability and adaptability for any downstream purpose.
The delivery model for this data as a utility is product-based and will be led by traditional IT operations with a focus on adding new data sources and ways to access them quickly. Implemented architectures may be deployed on-premises or take advantage of IaaS or PaaS in the cloud. Whichever architecture you choose, leveraging APIs will be the key to making it accessible.
Implementation of DataOps for the utility value proposition looks similar to implementation of its DevOps cousin, but has unique challenges. Since data is provided as a generic utility, the team creating the data product will likely be disconnected from the various consumers, thus shifting the focus of collaboration from “what data is required?” to “how can various potential uses of the data be supported?”. Following the water analogy, you’ll need to be able to build taps, shower heads and hoses to support the various uses of your utility.
The enabler value proposition is about how data and analytics supports specific use cases. These may be use cases for targeted analytics, like fraud detection, analysis of customer churn or supply chain optimisation. The delivery model is commonly project-based, but it could also use product or program delivery styles if the data product will continue to be used after the project.
For this value proposition, DataOps must focus on early and frequent collaboration with the business unit stakeholders who are the consumers for the specific data product serving their use case. If the required data assets are not already available, the team may have to locate and get permission to access data owned by other parts of the business, or capture external data sources, which will likely require collaboration with senior leadership.
The driver value proposition is about using data and analytics to innovate, creating new commercial products and services, generating new revenue streams and entering new markets. It’s an eco-system designed around quick, ad-hoc access to data for discovery purposes, but not limited to analytics.
Based on anecdotal evidence from users of Gartner’s inquiry service and discussions at the D&A Summit, the driver value proposition is where most new investment is occurring in data and analytics programmes. It is also the proposition that causes intractable challenges relating to data governance and the promotion of new discoveries into production.
An idea may emerge from your lab or data lake that needs to evolve into a production-quality data product for use across the organisation or by specific parties. Essentially, DataOps in this value proposition must provide the bridge from “can we do this?” to “how do we provide an optimised, governed data product to the data consumers that need it?”.
After exploring the utility, enabler and driver value propositions, the likely question is, “which one do I deliver?” There is no single right answer. The value propositions should not be viewed as a maturity model - every business will have all three, either in a centralised or decentralised deployment model. Framing each value proposition in the context of delivering DataOps enables you to foster collaboration between stakeholders and implementers, and thus ensure you deliver the right value proposition with the right data at the right time.
Ted Friedman is research vice president at Gartner