The Colour of Data

 Up until recently, when you heard about Artificial Intelligence (AI), it was usually a new breakthrough out of Silicon Valley or a ground-breaking discovery from the labs at MIT. Maybe you read an article or two on how AI would change this industry or that. For example, when unleashed on massive data troves that companies now collect as part of doing business, AI can provide business insight, help with cybersecurity, and improve the customer experience in countless ways.  

AI is everywhere and in fact, you have encountered it multiple times already today. The minute you reach for your smartphone and unlock is using facial recognition, you’ve begun your daily journey with AI. Check your social media and there it is again, serving you personalized feeds, suggesting friends, and showing targeted ads. Oh, did you get a push notification on your phone, alerting you to possible fraudulent charge on your credit card? That’s AI again. It makes life easier and, as 21^st-century consumers, we have come to expect high levels of user experience and convenience made possible only by the efficiencies of AI. Many find it creepy and even more take issue with the apparent lack of privacy involved in third-party smartphone apps or lax security practices in financial institutions. But we’re working on that.  

Another pitfall of the widespread use of AI is one that is just beginning to make its mark on the horizon – and which has the potential to devalue today’s AI products so drastically that some scientists are calling this a scientific emergency. The problem? Algorithms are racist.  

Algorithmic bias is everywhere 

The algorithms used in an AI system can be biased because they are based on datasets that are biased. Take, for example, the use of AI in city police forces. Some are using AI to predict criminality, using deep neural networks to create a facial recognition system that is supposedly capable of identifying people who are going to commit crimes.  

The algorithm used in such cases is based on data collected by police departments which, clearly, can be subject to racist tendencies themselves. If police are more apt to police minority neighborhoods and more likely to make arrests with minorities, then the datasets are going to lean toward equating criminal activity with being a minority.  

“If you overpolice certain communities, and only detect crime within those communities, and then try to provide a heat map of predictions, any AI will predict that crimes will occur in the places that they’ve happened before.¹” 

Perpetuating racial bias in the areas where it does the most harm 

Algorithmic bias also exists in the financial industry. You could argue that racial bias in healthcare is, literally, life-threatening to minority populations and is therefore more harmful. However, over the long term, being locked out of lower-interest mortgage rates is extremely dangerous to the African-American community, too. The opportunity to afford better homes in better neighborhoods can be uplifting to the point of becoming life-saving for a minority community, when you take the long view of things.  

Machine Learning (ML), a type of AI, works with historical data. It looks for patterns and then uses what it finds to make predictions about future behavior. If lending has always taken place in the context of prejudice on the part of lenders, then the data that’s going to be fed into the ML is, essentially, garbage.  

Garbage in, garbage out 

Historically, African-American and Latino borrowers have been charged higher interest rates than their White counterparts with similar credit scores. Lenders make decisions based partly on how likely a potential borrower is to shop around. The algorithm gives the borrower a framework for creating strategic pricing and as a result, Black and Latino borrowers are offered loans with higher interest rates and with more basis points on refinance loans: garbage in, garbage out.  

There are examples of algorithmic bias everywhere – what’s the solution? We can start by recognizing the problem so we don’t end up inherently trusting an algorithm because it’s AI. We can also start re-examining the algorithms that are in use. If you are in the position of purchasing an AI system for your job, you’ll want to make sure that you’re not purchasing a flawed (biased) system.  

AI consumers should do their due diligence when researching systems 

Consider the case of a man who was wrongfully arrested because of facial recognition AI. The algorithm identified him as the man who shoplifted $3,800 worth of watches from an upscale boutique in downtown Detroit. Blurry photos from shop video surveillance, combined with poor policework, can result in wrongful arrests like this one but the problem is exacerbated exponentially when algorithmic bias enters the picture.  

AI systems exist along a very wide spectrum, with some better than others, insofar as accuracy and bias. A 2019 study of more than 100 facial recognition systems revealed bias against African-American and Asian faces. Such algorithms are used not only to match surveillance videos with mug shots and predict criminality but also to predict recidivism. They factor into prison sentencing and bond amounts. Such tools often return higher risk profiles for African-Americans, compounding the systemic racism in the criminal justice system in an incredibly insidious and subtle manner. Subtle because it’s difficult to fathom that a seemingly innocent mathematical formula based on cold, hard data and set loose on such massive troves of data could result in so much error. Insidious because the bad algorithms are increasing racial inequality in system that already has a terrible track record in this area and because a machine isn’t supposed to be racist.   

A call for regulation? 

Clearly, police stations should purchase high-quality algorithms that are tested for accuracy and all kinds of bias, including the racial type. There are third-party organizations that perform this service but it’s an industry still in the nascent phases of development. There is no precedent for auditing AI, nor are there any regulations or standards in place for AI. At the very least, companies who use AI, including those in financial services, should understand how algorithms work. They should examine the data that fed the AI to create the algorithms, and they should monitor the results consistently for racial and other types of bias.