The music industry needs to find new revenue streams and is hoping open data might help to find them. But what does it mean for businesses to let their information be accessed by third parties while they create their own mash-ups? David Reed went to Music 4.5 to find out.
What does the data value exchange look like? A good example would be Last.fm, the music discovery service. Sign up and upload your personal playlists (or “scrobbles” as the company calls them) and in return get personalised recommendations and free access to them via its internet radio.
The business model is based on selling that information to music labels and artists who want to promote their events, gigs and recordings to the right audience. Not much different from hundreds of other social networks with the same approach. Except that Last.fm has operated an open data policy from the outset.
That makes it profoundly different since it allows third parties to develop services, just as it creates its own using the data freely provided by consumers. Remarkably, the service launched before the concept of big data had been dreamt up. “We launched before even Facebook, Google+ or FourSquare. We were one of the first companies to take the Hadoop tool and build a database on top,” recalled technical team leader Adrian Woodhouse at a Music 4.5 open data event in March, hosted by Lewis Silkin.
Choosing to create in this new environment, rather than using a conventional relational database, was a bold step. But as Woodhouse noted, “Last.fm was not the result of lots of strategic thin king and focus groups. We just made it up.”
The principle of open data was also important. “We had to make a strong case to users as to why they should give us their data. Our first principle was that users own their own data. There is no moderator. It is a self-perpetuating feedback loop,” he said.
With 75 billion scrobbles and 636 million tracks, it has unarguably been a success. But those founding principles have caused their own problems, not least the decision to be open data. For one thing, the company did not have the right controls in place to manage runaway demand for data from external developers.
“As pioneers, we don’t always get things right,” said Chris Wilson, commercial director. “We have yet to set out the rules, but we are about to apply the learnings about how others have taken advantage of us. For example, we had no API keys, so it was a free-for-all that led to over-usage. The data we had collected was being used in an uncontrolled way.”
As an open data source in a very popular sector, Last.fm was receiving thousands of calls on its servers by developers wanting to create new, related services. Meeting that demand required resources, both human and technical, which all come at a cost. So the business is working out what rules it should apply to ensure it can maintain its open data principles without having to keep an open chequebook to pay for them.
It is a dilemma which open data proponents need to be able to resolve to maintain momentum. The upside is to have service enhancements being developed which the business would never be able to afford to create in its own right. The downside is that these benefits are not always obvious at the outset.
“It is difficult to explain to the business why they should open their data. It is easy to say why they should consume open data in their products and services,” said William Lovegrove, founder and CEO of Release Mobile, which recently launched Datownia - a cloud-based data sharing platform aimed at helping SMEs connect their data to mobile apps and business systems.
He recognises that there is a “Jekyll and Hyde” opinion about open data in many organisations, not least because it is a disruptive force. “It is challenging to understand from a business perspective. But there are two drivers for business - it enables complementary products and services to grow and it supports reputation management,” said Lovegrove.
The music industry is struggling with the concept of open data as a value creator, having shown itself to be slow at adapting to rapid market changes. Historically, music labels based their business model on restricting supply and controlling distribution. Open data as a principle runs entirely counter to this view.
“It took a lot of disruption to the system to get to the point where you can pay for a download,” pointed out Gavin Starks, CEO of the Open Data Institute. Labels have also not had strong data management skills at their heart. “Meta-data was so terrible in the early days of iTunes that the best-selling track was by ‘Unknown Artist’,” he said.
Responding to the challenge of open data will prove to be no easier for the music industry than digital distribution. “Be under no illusion - creating a robust business model out of something we don’t yet understand is very hard work. The music business is grappling with that,” said Starks.
He recalled that in the early 1990s, labels feared that allowing online access to their music would lead to lost sales. Instead, restricting digital access did lose them sales, but chiefly as a result of piracy and peer-to-peer file sharing. Once access portals like iTunes became embedded in the same mobile devices used to play content, however, digital sales began to take off.
Changing a mindset based on closed systems to one accepting of openness will be no small task. Starks said: “Everyone wants to open up data as long as they can keep their own data closed.”
Paul Gathercole has been looking at this issue as VP, digital tools, global, for Universal Music Group International. The guiding data principles within the business are to acquire, organise, use and learn. “It is still a work in progress,” he admitted. It is also one where the learnings have emerged from an unforeseen quarter.
“It took an unexpected turn for us. Interns were using data on behalf of the business and they started to build tools for very narrow applications based on open data, for example an iTunes daily content list. That escalated as one issue led to another,” he said.
Different parts of the business were looking for specific answers by drawing on such open data sources, building single-use APIs in order to generate a standalone report. Organising that into a coherent strategy was one of the challenges faced by the business, as was improving the quality of its own meta-data.
“We started to develop and classify and create a structure so we could build APIs for everything,” recalled Gathercole. “But we had a lack of standards and rules on how to apply them, for example our artist identifiers are not standardised. APIs also often change and stop being compatible.”
Being able to deliver dashboards to business users is one way in which data becomes visible and valuable in a company. For a music label, being able to correlate radio airplay against iTunes downloads is a useful marketing insight. But this also creates risks if those users are simply able to build their own reports. “It is easy to understand the wrong question. So we have been moving away from dashboards towards dialogue,” he said.
When a query is raised by a business user, the data analysts no longer simply build an API and dashboard to answer it. Instead, they will discuss why that individual wants a specific piece of information and whether there is another way to tackle that question. “That drives cultural change, although that is a slow game,” he admitted.
The way in which data insights are presented is also important to how they are adopted. “We’re a very visual industry. People tend to slice and dice by the impact findings make on screen. You have to remember that your data ends up on a big screen being discussed,” he said.
Gathercole has adopted a rapid prototype and testing process to allow data projects to remain achievable and to get to some value early on. This means that, “everything is a beta and it is not about having all the answers. It is just about having a better degree of data fluency. Too much is as bad as not enough”
While the answer to the open data issue has yet to be discovered, it does mean Universal Music is undergoing a cultural shift in readiness for the new environment. That includes changing its recruitment process to bring in data scientists who are comfortable with exploring data.
Somewhere in all of those data sources - both open and proprietary - could reside the next breakthrough idea for the music industry. They certainly hold a lot of potential value in terms of content which music lovers may pay a premium for. Last.fm has proven that aggregating data into a sum that is greater than the parts delivers a real reward. Others in the industry will start to explore their own options. Open data will be an unavoidable part of how that value gets realised, even if how that happens is not yet clear.