Biases in AI: How neutral is Technology?

Artificial Intelligence (AI) is a key technology of the future. But it also harbors risks – especially regarding gender equality. Biases in datasets and models applications can increase the underrepresentation of women in general but particularly in the Global South. The re:publica panel “Beating the gender bias in AI” discussed, how feminist digital policy could meet these challenges. In the following article Felix Dengg illustrates a more general perspective on this topic by highlighting various inequality features of AI and identifying causes and possible solutions.

As ever more decisions in all aspects of life are being delegated to technology and software – the societal consequences are much broader than just a gain of efficiency and convenience. Algorithms increasingly shape crucial decisions in life. Recommendation algorithms are employed to recommend specific products to buy or routes to take to get to a place. Technological platforms such as Facebook, or Youtube apply algorithms that filter and sort content and hence decide what content we see and what media we consume. And we see classification and profiling algorithms that evaluate creditworthiness or support judges in their decisions, influencing crucial decisions on people’s lives.

The rationale for applying algorithms is their power to handle big data sets, find and derive unknown patterns and insights about real life phenomena, make predictions and automate decisions. Once developed, an algorithm can run with very little cost, compared to a worker. Algorithms have made incredible progress in handling natural language or recognize images, tasks that could traditionally only be done by humans. But while we have this intuitive idea of technologies as neutral tools, their development is always based on existing conditions and conceptions of the world.

Ethical issues arise in this context when human biases and prejudices make their way into automated decision-making systems. Biases describe misjudgment in human decision making. For example the confirmation bias – humans’ tendency to seek out, favor and use information that confirms their existing beliefs. Such biases can rely on racial stereotypes  for example when people tend to have misconceptions of People of Color. These biases end up influencing automated decision-making systems, potentially manifesting or exacerbating inequality. In the United States, algorithms deployed in the criminal system have been shown to disproportionally target the Black population. Here algorithms are used for criminal risk assessment, to compute the probability that an individual will commit another crime in order to inform judges’ sentences. Hence these biases literally shape choices about people’s lives – but not in a neutral way. Biases typically exacerbate existing inequalities – the most important areas, where biases emerge are:

Gender

Especially in the workplace, where a significant gender pay gap exists in most sectors, constructing and using algorithms from existing data can be problematic. As algorithms have been used for making hiring and promotion decisions, some algorithms such as Amazon’s has been shown to discriminate against women. But the problem extends far beyond the workplace, in areas such as banking and loans or speech recognition.

Race

In the book ‘Algorithms of Oppression’ – How search engines reinforce racism’ the author Safiya Umoja Noble examines Google’s search engine algorithms self-declared neutrality. She argues that search engines are racist as long as they mirror negative biases in society and the people that create them. She explains how algorithms privilege whiteness by showing how search engines deliver positively connotated images when searching for keywords like ‘white’ and negatively connotated when searching for terms like ‘Black’. This can reinforce stereotypes and biases.

Inequality

In her book ‘Automating Inequality’ Author Virgina Eubanks examines the life-and-death impacts that the deployment of automated decision making has on public services in the USA.  She makes the case that the automation of the provision of welfare in the US failed to lead to better outcomes for the poor. The author discusses predictive police algorithms, algorithms that decide on welfare payments, or algorithms that predict probabilities of child abuse in families. She makes the argument that the deployment of technology is a continuation of the United States history of attempts to profile, police and punish poor people. The promise of improved fairness and efficiency of the outsourcing of decisions to technology can come as a justification for the continuation of existing inequalities. So why are technological solutions not as neutral as we might think they are?

Biased Training Data

Decision making systems use statistics to find patters in existing data to make predictions about probabilities of future events. For example, an algorithm analyses historical crime statistic that include zip codes, age, income, school and other socio-demographic factors, in order to predict how likely an individual is to commit another crime. One problem with the approach is the existing data set. If in the past a certain population has been disproportionally targeted by the police such as minorities or low-income communities in the U.S., the statistic will show them to be more likely to commit a crime which in turn affects the predictions of the model. In this way, the initial racist profiling will be perpetuated by higher incarceration rates. And while it is unproblematic to erase race or gender related data from data sets, there are almost certainly factors that might correlate with these aspects that are difficult to uncover and might lead to the same biased results.

At the same time, statistics about a whole population are used to inform decisions about an individual’s live. For the individual it becomes harder to escape its statistical fate, the prediction manifests itself, leading to longer sentences and possibly less chances for reintegration in society. Marbre Stahly-Butts of Law for Black Lives has called this kind of data-driven risk assessment ‘a way to sanitize and legitimize oppressive systems’ that ultimately distracts from the actual problems affecting these communities.

Lack of data

As training data is the basis for AI applications, the availability of such data is crucial for their development. Applications such as translation apps or voice recognition systems for example need huge data sets in the specific language in order to deliver results. This data however might not be as widely available in certain sectors or areas. This is the case when there is less economic incentive for technology companies to collect, share and collaborate on data. Countries where this is the case risk falling behind in the digital economy and becoming dependent on technologies from the global north. Many African countries have less developed data ecosystems in which training data is not available which prevents the emergence of AI companies and initiatives. In turn, when reliance on foreign technologies grows, the risks for adverse effects such as biases or untransparent decisions rises (more information here).

Lack of Diversity in Artificial Intelligence Research

Another reason why automated decision making fails to accomplish neutrality is the lack of diversity in artificial intelligence research. Especially the big American tech companies that are responsible for a large part of AI research and applications, are particularly non-diverse [1]. Black workers only represent 2.5% of Google’s workforce and only 4% of Facebook’s and Microsoft’s. At the same time only 10% of AI researchers at Google are female. Globally, only about 22% of AI professionals are female. The rationale is that the less diverse the research team, the less likely it is to create automated decision-making technology that works fairly and well to all populations. A non-diverse population might be blind to the experiences of other parts of the population and unfamiliar with their struggles. Besides race and gender, there is also a lack of geographic and cultural diversity. The fact that such technology is developed in the global north can lead to complications for the deployment in the global south.

• Conducting external audits & accountability
• Providing algorithms publicly accessible for scrutiny
• Making data sets available to the public: Making data sets available would enable independent parties to audit the data sets and check for biases. However there is considerable hurdles with regards to anonymity and privacy
• Setting up research teams more diverse

• The Algorithmic Justice League: mission is to raise awareness about the impacts of AI, equip advocates with empirical research, build the voice and choice of the most impacted communities, and galvanize researchers, policy makers, and industry practitioners to mitigate AI harms and biases. It is a movement to shift the AI ecosystem towards equitable and accountable AI.
• The Center for Equity, Gender & Leadership of the University of Berkeley in California has launched the ‘Mitigating Bias in Artificial Intelligence (AI): An Equity Fluent Leadership Playbook‘, which gives a framework to address bias in AI.