Data is the lifeblood of business and its supply has never been greater. Personal data, big data, open data are all expanding the opportunities which can be identified and developed by smart organisations. At the same time, the challenges around capturing, managing and exploiting data have probably never been greater. When it comes to the entirely new data sets, such as open data, a skills gap could limit the extent to which those market opportunities get realised.
That is why the Institute of Direct Marketing has partnered with the Open Data Institute to launch a brand new training course, Open Data for Marketers. Due to be run in January and March next year, it is aimed at practitioners who want to develop their basic literacy in this emerging field. To help launch the course, the IDM and ODI ran a joint evening event in October at which the the potential of Open Data for marketing was discussed through a series of presentations and debate.
For many, the first question is what open data actually looks like and how it can be deployed. Three flash talks from start-ups supported by ODI helped to explain. The first was by Francis Irving, CEO of ScraperWiki, a self-service platform which pulls open data from web sites and converts it into reports. One live application has been for a major social media platform which wanted to understand who among its users was a prospect for its paid-for tools by identifying followers of existing customers.
The second presentation came from Jessi Baker, co-founder of Provenance. The web platform draws on open data to identify products and bring together information on how it has been sourced. Its initial focus is on fashion and homeware, not least in the wake of the Bangladesh clothing factory disaster, since a lot of retailers and their customers are now looking to be more ethical.
In the third flash talk, founder and director of CarbonCulture, Luke Nicholson, explained that the business is aiming at climate change by making it easier for companies to build-in sustainability using high-tech metering to monitor carbon use in the workplace. This is then published in real-time where employees and partners can see it, creating a stimulus for change. When piloted at the Department for Energy and Climate Change, it achieved 40 per cent take-up among staff and drove a 10 per cent reduction in gas usage. Seven more departments have now adopted it.
Applications of this sort are an essential part of the open data movement as they show how data sets can actually be converted into usable resources. ODI has recognised this as a critical step, not least given the sheer diversity of the information on offer.
Already, 15,000 different public sector data sets have been opened up via www.data.gov.uk. “What can you do with it? We are looking to identify the potential through projects such as OpenHealthLab, which took anonymised data on prescriptions for every primary care trust and GP, then used Hadoop to mine it. That told us in some areas GPs are over-prescribing brand-name statins for cardio-vascular conditions, even though they cost twenty times the generic equivalent. NHS guidelines are to prescribe generics - if that happened it would save £200 million a year,” he said.
In a second example, Coleman explained how a peer-to-peer lending organisation opened up data on its 14 million records covering £378 million of loans. This identified that while the lending was distributed evenly to borrowers right across the UK, it was largely originating in the South East. “That sort of insight could be used by the banking sector to identify patterns of lending ahead of new regulations in 2014,” he said, especially with non-traditional lending having grown 80 per cent year-on-year during the downturn.
As an example of how open and big data cross over, the NHS prescribing data source comprises 100 million rows every month. Even that is dwarfed by the data which flows into O2 every day, generating 1.1 billion events. Telefónica UK became a partner of the ODI in July 2013 as part of its open data strategy which will eventually see customers able to access their transactional data in full (or get a trusted third party to do it for them).
In the meantime, Telefónica Dynamic Insights is making use of anonymised, aggregated call record and cell data to help marketers gain a better understanding of their target audience. “We can compare 6pm versus 2am in Oxford Street, for example, or look at footfall at Christmas or when the street is only open to pedestrians. That has implications for retailers, the public sector, travel and more,” said Hugo Pinto, sector communications manager of Telefónica Dynamic Insights.
This data can be used in marketing planning, for example to identify outdoor advertising spaces that will be seen by a target audience on their journey to work or when out shopping. Eventually, this level of profiling will be possible in real time. “The data can be applied to a customer database at postcode level to identify customers versus non-customers and look at their behaviour, for example, who is in Chelsea on a match day compared to a non-match day. That changes the questions you can ask,” he explained.
It is in this type of application that the real value of both open and big data will emerge - and also where a critical dimension of compliance needs to be observed. As Peter Galdies, director of DQM Group, which sponsored the ODI/IDM event, said during the panel debate, “if organisations want to compete, they need data. To get it, they will have to have greater transparency with customers.”
Based on questions raised by delegates on the night, it was clear that the distinction between open data - which Coleman stressed is always anonymised - and personal data is one that concerns many practitoners. Galdies stressed the need to tread this line carefully: “If brands want to be acceptable, they will have to take a lead. Things are already closing up in the form of the European Union’s Data Protection Regulation which will impose tighter controls over the ways we collect and use data.” He added that terms and conditions might need to become more explicit and also more transparent about data practices.
To achieve that, marketers will need the right skills to understand open data and when it is being used in a compliant way. That is not always easy, as Max Kelly, former managing director of Virgin Insight, explained: “We spent £100,000 and used a lawyer to answer that question - it was very painful and was not black and white. It depends. Even if you have anonymised it, hashed the identities and used it in aggregate, if somebody has the key for that, it is not anonymous.”
All of which underlines the importance of developing knowledge of this area of data practice, as stressed by Mike Cornwell, CEO of the IDM: “With the arrival of open data, both students and executives need to gain the necessary skills to support projects That is why we are launching the training scheme in quarter one of 2014.”
In the spirit of the open data movement, the event invited participants to crowd-source the syllabus by providing comments on what was good about its existing outline and what might need to be added. Open data is now firmly on the business agenda - what has to happen next is to train up the necessary human resources required to exploit it.