The Centre for Data Ethics & Innovation (CDEI), the government advisory body set up to address the challenges posed by the adoption of AI and data-driven technologies, is to investigate the potential for bias in the use of algorithms in crime and justice, financial services, recruitment and local government.
While it has been recognised that algorithms have huge potential for preventing crime, protecting the public and improving the way services are delivered, there are growing concerns that decisions made in these areas are having a significant impact on people’s lives.
Professionals in these fields are increasingly using algorithms built from data to help them make decisions. But there is a risk that any human bias in that data will be reflected in recommendations made by the algorithm
The CDEI wants to ensure those using such technology can understand the potential for bias and have measures in place to address these concerns. It also aims to help guarantee fairer decisions and, where possible, improve processes.
For example, some police forces have already started to use algorithms to feed into their decision-making - such as the Harm Assessment Risk Tool in Durham which is being used to assist officers in deciding whether an individual is eligible for deferred prosecution based on the future risk of offending.
The use of algorithms in policing is controversial, with human rights organisation Liberty claiming the programs could reinforce bias and stereotypes.
In financial services, data analysis has long been used to inform decisions about whether people can be granted loans. But the rise of data and AI machine-learning presents increased issues about the transparency and fairness of such decisions.
In recruitment, computer algorithms can be used to screen CVs and shortlist candidates. This could help potentially limit the impact of unconscious bias, where people discriminate against candidates because of their background. But there have also been reports of such technology inadvertently exacerbating gender bias.
The CDEI will also investigate how data is used to shape online experiences through personalisation and micro-targeting - for example where consumers search for a product and then ads for similar products appear later in their browser.
This review will explore where, how and why online targeting approaches are used, and their impact on members of the public. The CDEI is launching a series of nationwide workshops to investigate public views on the acceptability of micro-targeting.
Digital Secretary Jeremy Wright said: "Technology is a force for good and continues to improve people’s lives but we must make sure it is developed in a safe and secure way.
"Our Centre for Data Ethics & Innovation has been set up to help us achieve this aim and keep Britain at the forefront of technological development.
"I’m pleased its team of experts is undertaking an investigation into the potential for bias in algorithmic decision-making in areas including crime, justice and financial services. I look forward to seeing the centre’s future recommendations to help make sure we maximise the benefits of these powerful technologies for society."