AI Failures

In prior articles, we talked about what AI can do, now let’s consider what it can’t. In this post we’ll explore the limits of AI technology, at least as it stands today. There are three major ways that AI fails.

Some tasks are simply beyond the current capabilities of AI

The first kind of failure is when AI is just not ready for the task at hand. AI is unable to do many tasks that people find quite easy. In some cases, this is due to lack of good training data or a well defined objective. But many times AI fails because it requires operating conditions to be too perfect and consistent compared to reality. In some cases, it requires too much fundamental knowledge that only a lifetime of being human can provide. Some tasks simply require too much dexterity and agility for a robot to complete at anywhere near human speed.

One area that AI is just not ready yet is the task of grasping random objects. For many basic tasks, speed is a critical limitation for AI-powered robots. Without thinking about it, our brains, eyes and muscles do an incredible amount of data collection and calculations to move and manipulate objects precisely and quickly. Yet the simple task of picking up unpredictable and unusually shaped objects at human speeds still eludes AI. Many robots are only capable of picking up objects at a rate of less than 100 mean-picks-per-hour (a good measure for this task). UC Berkeley’s Dex-Net robot was able to achieve a pretty impressive 200 to 300 mean-picks-per-hour. But this is still quite slow compared to humans, which are twice as fast as that.

There is hope around the corner, but AI is just not there yet. Google created an Arm Farm of 14 robots to continuously learn how to pick up objects successfully and quickly. While they have improved after millions of grasps, at their current learning pace, these robots may need decades more practice in order to be better than a human.

A very common AI failure is when it works
most of the time, but fails completely on occasion

Another kind of AI failure is when it generally works but it cannot deal with corner cases. Many AI applications work under controlled conditions, but fail as soon as those conditions change. Because these failures are unexpected, they can lead to comic results, and in some cases tragic ones. It’s when we start relying on AI, that humans can become vulnerable to these corner case failures.

A very tragic example of this is when an Uber self-driving car killed a woman walking her bike across the street at night in Arizona in March 2018. It is the first known autonomous vehicle related pedestrian death on a public road. The vehicle was in autonomous mode with a human safety driver at the wheel. Sadly, the driver did not react fast enough to avoid the accident.

The investigation is ongoing, but it appears that the car’s sensors did detect the pedestrian six seconds before impact. However the system was confused about what it was seeing, perhaps because it was unusual (a person walking their bike), unexpected (in the street, not in a crosswalk) or poor lighting (at night, under street lights). It classified the pedestrian first, as an unrecognized object, then as another vehicle and finally, as a bicycle. Uber stopped it’s self-driving road tests after the accident, until resuming nine months later.

Each of these corner case failures must be addressed and investigated, and the learnings fed back into the system to prevent a recurrence. Much like how air travel became more safe over many years of experience and accident investigations, it may take a long time before we don’t have to think about such failures in autonomous vehicles.

A more insidious form of AI failure is due
to a poor deployment of the technology

The most common version of this is when uncorrected bias in the training data yields inaccurate results. Another type of poor deployment is when humans inaccurately interpret or use the results of AI, sometimes attributing more intelligence or meaning than is justified.

AI is only a mirror to humanity, and sometimes the image isn’t pretty. Bias is everywhere, and data sets about humans are particularly susceptible to bias. Left uncorrected, AI programs will simply reflect the example bias that already exists in the data. The subject of bias is big enough for a separate article.

This is especially troubling in the apparent racial bias of AI used in the justice system. There are several AI-powered risk-assessment algorithms being used around the country to predict the outbreak of violent crime, determine the level of supervision for inmates, or even provide information for sentencing. These AI systems have been used by police and courts to predict the likelihood that convicts will reoffend. Research by ProPublica showed that the system was biased against black prisoners, disproportionately flagging them as being more likely to reoffend than white prisoners (45% to 24% respectively). These predictions may have led to longer sentences than justified.

Suffice to say, the blind application of AI in this context is dangerous. In order to use these tools, society needs to correct the bias in its historical data, otherwise we risk tainting our AI with mistakes and failures of the past.

Even in a less serious context, not controlling the quality of training data can lead to a serious deployment failure. In 2016, Microsoft released an AI chatter bot called Tay. Within 16 hours of interacting with people on Twitter, Tay went from a friendly “hello world” to a barrage of racist and inflammatory comments!

Microsoft was forced to shut down Tay due to her offensive tweets. She was designed to converse with people across Twitter, and develop a personality shaped by these conversations. Microsoft claimed that Tay was targeted by trolls that attacked the service by making racist and extreme comments. Unfortunately, Tay couldn’t choose to ignore these offensive comments, and incorporated them into her personality.

This kind of failure is hard to prevent since the training data, in this case tweets from users, is real-time and unpredictable. Also, unless there is some intelligent filter placed on the training data, bad actors may taint the outcome.

In short, AI is not an exact or predictable tool. It will make mistakes. It is irresponsible to simply hope for the best or blindly trust in algorithms. We must only use this powerful technology with sufficient safeguards to prevent bad outcomes, especially severe ones.