Society is collecting, storing and analysing more data than ever. This is the data age and it has the potential to deliver enormous value for governments, organisations and business. Key to delivering value is using advanced analytics and data science to obtain insights. So what needs to be done for data to deliver on its promise?
For data really to make a difference, three things need to happen:
Open data in the UK
The Government aims to promote innovation through the use of data, working towards a society that benefits from open data to improve its citizens’ lives. Open data initiatives are gaining ground. Indeed, the UK ranked number one in the world for its progressive approach to open data according to the Open Data Barometer (2013). The implementation of data.gov.uk, the open data portal, is testament to this. In addition, the launch of the Open Data Institute and incentives for public organisations to release more open data shows its commitment. While the open data commitment is clear and progress is being made, there is still a lot more that can be done.
The right tools for open data to make a tangible difference
As well as needing open data, the right tools are required to uncover speedy and important insights. With the growth in data comes a need to be able to analyse it effectively. This requires cost-effective and advanced analytic technology, from the warehouse to the database to visualisation.
NHS Digital dataset – Putting open data to use
One example of an open data set is the Health & Social Care Information Centre (HSCIC) data, the body that was recently renamed NHS Digital. NHS Digital’s datasets comprise a great deal of anonymised data, including prescription data. According to NHS Digital’s website, its national datasets “define a standard set of information that is generated from care records…which can then be used to monitor and improve services.”
Using the right tools, asking the right questions and analysing datasets like this provide huge potential to help policy making and service improvements. By unleashing important insights, open data can truly help lives for the better.
Indeed, we at EXASOL have worked with the HSCIC dataset to find valuable information that is being used to inform policy and help combat serious health issues. We’ve found that by working with these vast datasets - and by using the right technology for both fast analysis and visualisations - organisations can uncover important information about the issues they are wishing to address. This was previously not possible.
Using open data and advanced analytics to combat serious health issues
We worked with the HSCIC dataset to uncover national hotspots and seasonal trends of antibiotic prescriptions. We took the same dataset and looked at allergies in the UK, identifying some very interesting insights into where the most allergy prescriptions are being dispensed. Our analysis was visualised using Tableau.
The 652 million rows of data cover every prescription given out in England over the last five years. Using an instance of EXASOL’s in-memory database, this data was analysed and trends uncovered, both nationally and by county, district, month and prescription type. By joining the data to geographic areas, a high resolution map comprising 32,000 colour-coded polygons covering the UK could be generated, allowing country-wide prescribing for both antibiotics and allergy medicines to be visualised in the form of a heat map for the first time.
Data reveals the middle-England allergy crisis
The allergy research found that allergies requiring medication are increasing year-on-year across the country with a 24 per cent increase in the past five years alone.
The research additionally looked at the prescribing of epinephrine auto-injectors (EpiPens), the adrenaline shots that can save the lives of people suffering extreme allergic reactions, known as anaphylaxis, to antigens like bee stings, peanuts and seafood. The most surprising result of this research is the geographical difference across the country, with the Home Counties prescribing three times the number of EpiPens per person compared to the North West. It was found that Surrey, Buckinghamshire and Hertfordshire had the highest prescribing counties compared to Manchester, Lancashire and South Yorkshire and has shown that severe allergies are a crisis of the affluent middle-class.
Revealling trends around antibiotic prescriptions
Analysis of the same NHS Digital dataset revealed that the most deprived coastal towns are prescribing the most antibiotics in the country, with Clacton-on-Sea - the UK’s most deprived area - almost twice the national average. It also found that doctors prescribe 59 per cent more antibiotics in December than they do in August, despite the fact that illnesses treated by antibiotics are not seasonal.
The visual heatmap of prescribing is so detailed that it is possible to single out individual surgeries overprescribing. For instance, there was a surprising hotspot in London’s wealthy Kensington & Chelsea borough and, when the area was analysed, it was found that the hotspot was due to one practice used by the Chelsea pensioners and war veterans living at Royal Hospital Chelsea.
The power of data
Awareness of these issues is critical and insights can help policymakers find solutions. By analysing the readily-available data with the right tools, results can be drawn that can have a significant impact on our knowledge of an issue. The insights can also provide insights for decision-making that would otherwise go unnoticed, such as new links between antibiotic prescribing and deprivation and uncovering extreme allergies in the home counties.
In fact, the results of our research into antibiotic prescriptions have been discussed during a parliamentary debate and acknowledged personally by Dame Sally Davis, the Chief Medical Officer, showing the importance of the findings. Only by using the right tools and data science, can data on this scale be analysed.
Right technology, right tool and right questions
With the right data and the right technology, you can turn any problem into a data problem and uncover information to help address it. Society is only at the start of the open data journey, but it is evident that the more open data there is, the more it can help humanity.
Advanced data analytics and data science can lead to innovative approaches to problem solving, based on hard facts. Data, combined with the right tools in an in-memory world, makes the possibilities endless and can only be held back by human endeavour and ingenuity.