In a relatively short time, artificial intelligence (AI) has been integrated into our daily lives. Now nearly half (45%) of the US population uses generative AI tools, while millions of people around the world use services like ChatGPT to compose emails or Midjourney to generate new images. AI is fueling the advent of a new digital era, increasing our speed and efficiency in tackling creative or professional challenges, while helping to drive new innovations.
The use of AI does not stop there. It has become an essential part of essential services that ensure the smooth functioning of our society – from loan approval and admission to higher education, to access to mobility platforms and soon to access to medical care. Online identity verification has evolved from opening a bank account to a wide range of applications on the Internet.
Nevertheless, AI systems have the ability to behave biased towards end users. In recent months, Uber Eats and Google have discovered how much the use of AI can threaten the legitimacy and reputation of their online services. However, people are also vulnerable to prejudice. These can be systemic, as evidenced by facial recognition bias – the tendency to better recognize members of one’s own ethnic group (OGB, or Own Group Bias) – a phenomenon that is now well documented.
The challenge lies here. Online services have become the backbone of the economy; 81% of people say they access services online every day. With lower costs and faster execution, AI is an attractive choice for companies that manage high customer volumes. However, despite all the benefits, it is crucial to recognize the biases this brings, as well as the responsibility of companies to implement safeguards to protect their reputations and the wider economy.
A bias prevention strategy should focus on four key pillars: identifying and measuring bias, being aware of hidden variables and jumping to conclusions, designing rigorous training methods, and adapting the solution to the use case.
VP Applied AI, Onfido.
Pillar 1: Detect and assess biases
The fight against bias starts with implementing robust processes for measuring it. AI biases often lurk in extensive data sets and only become apparent after several correlated variables are disentangled.
It is therefore critical for companies using AI to establish good practices such as confidence interval-based measurement, using data sets of appropriate size and variety, and using appropriate statistical tools manipulated by competent individuals.
These companies should also strive to be as transparent as possible about these biases, for example by publishing public reports such as the ‘Bias Whitepaper’ that Onfido published in 2022. These reports should be based on real production data and not synthetic or test data. .
Public benchmarking tools such as the NIST FRVT (Face Recognition Vendor Test) also produce bias analyzes that can be exploited by these companies to communicate and reduce bias in their systems.
Based on these observations, companies can understand where bias is most likely to occur in the customer journey and work to find a solution – often by training the algorithms with more complete data sets to produce fairer results. This lays the foundation for rigorous treatment of biases and increases the value of the algorithm and its user journey.
Pillar 2: Beware of hidden variables and hasty conclusions
The bias of an AI system is often hidden in multiple correlated variables. Let’s take the example of facial recognition between biometrics and identity documents (“face matching”). This step is essential in user identity verification.
An initial analysis shows that the performance of this recognition is less good for people with a dark skin color than for an average person. Under these circumstances, it is tempting to conclude that the system deliberately punishes people with dark skin.
However, expanding the analysis further, we see that the proportion of people with dark skin in African countries is higher than in the rest of the world. Furthermore, these African countries use, on average, lower quality identity documents than those in the rest of the world.
This decline in document quality explains most of facial recognition’s relatively poor performance. When we measure the performance of facial recognition for dark-skinned people, limiting ourselves to European countries that use higher quality documents, we see that the biases virtually disappear.
In statistical terms, we say that the variables ‘document quality’ and ‘country of origin’ are confusing compared to the variable ‘skin color’.
We provide this example not to convince that algorithms are not biased (they are), but to emphasize that measuring bias is complex and prone to hasty but incorrect conclusions.
It is therefore crucial to conduct a comprehensive bias analysis and study all hidden variables that could influence the bias.
Pillar 3: Develop rigorous training methodologies
The training phase of an AI model provides the best opportunity to reduce its biases. Indeed, it is difficult to compensate for this ex post bias without resorting to ad hoc methods that are not robust.
The datasets used for learning are the most important lever with which we can influence learning. By correcting the imbalances in the datasets, we can significantly influence the behavior of the model.
Let’s take an example. Some online services may be used more often by people of a certain gender. If we train a model on a uniform sample of the production data, this model will likely behave more robustly on the majority gender, at the expense of the minority gender, causing the model to behave more randomly.
We can correct this bias by sampling the data from each gender equally. This will likely result in a relative reduction in performance for the majority gender, but to the benefit of the minority gender. For a critical service (such as an application acceptance service for higher education), this consideration of the data is completely logical and easy to implement.
Online identity verification is often associated with critical services. This verification, which often involves biometrics, requires designing robust training methods that reduce bias as much as possible regarding the variables exposed to biometrics, namely: age, gender, ethnicity and country of origin.
Finally, working with regulators, such as the Information Commissioner’s Office (ICO), allows us to take a step back and think strategically about reducing bias in models. In 2019, Onfido worked with the ICO to reduce bias in its facial recognition software, which led to Onfido dramatically reducing the performance gaps between age and geographic groups of its biometric system.
Pillar 4: Tailor the solution to the specific use case
There is no single measure of bias. In its glossary of model fairness, Google identifies at least three different definitions of fairness, each valid in its own way but leading to very different model behavior.
For example, how can you choose between ‘forced’ demographic equality and equal opportunity, taking into account the variables specific to each group?
There is no clear answer to this question. Each use case requires its own reflection on the area of application. For example, in the case of identity verification, Onfido uses the “normalized rejection rate”, where the rejection rate is measured by the system for each group and compared to the total population. A percentage greater than 1 corresponds to an over-rejection of the group, while a percentage lower than 1 corresponds to an under-rejection of the group.
In an ideal world, this normalized rejection rate would be 1 for all groups. In practice, this is not the case for at least two reasons: first, because the datasets needed to achieve this goal are not necessarily available; and secondly, because certain confounding variables are beyond our control (this is the case, for example, with the quality of identity documents mentioned in the example above).
The pursuit of perfection hinders progress
It is not possible to completely rule out bias. Therefore, it is essential to measure the bias, continuously reduce it and communicate openly about the limitations of the system.
Research on bias is widely accessible and there are numerous publications available on the subject. Major companies like Google and Meta continue to contribute to this knowledge by publishing in-depth technical articles, as well as accessible articles and training materials, as well as special datasets for bias analysis. For example, last year Meta released the Conversational Dataset, aimed at bias analysis in models.
As AI developers continue to innovate and applications evolve, biases will emerge. Nevertheless, this should not deter organizations from implementing these advanced technologies as they have the potential to significantly improve digital services.
By implementing effective measures to combat bias, companies can ensure continuous improvements in customers’ digital experiences. Customers will benefit from access to the right services, the ability to adapt to new technologies and receive the necessary support from the companies they interact with.
We have the best customer feedback tools.
This article was produced as part of Ny BreakingPro’s Expert Insights channel, where we profile the best and brightest minds in today’s technology industry. The views expressed here are those of the author and are not necessarily those of Ny BreakingPro or Future plc. If you are interested in contributing, you can read more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro