Can AI Be Biased and Make Mistakes Like Humans?

Artificial intelligence can stand at the forefront of the Fourth Industrial Revolution. It was announced by HIT Robot Group Vice President Chen Quliang at the GMIS 2019 summit. It means that AI will be more often used in manufacturing and business operations, and the fate of hundreds of millions of people will depend on the success and decisions of machines. However, AI can not only solve problems, but also create them, taking human weaknesses and mistakes as the standard of behavior.

Like father like son

Let us recall how artificial intelligence works. Any artificial intelligence is a specialized solution, the effectiveness of which depends on the quality of training. Simply put, if it is necessary for AI to qualitatively recognize diseases or people in photos, it needs to be trained. And this is the main issue that can literally turn into a problem for businesses. It can happen that during training AI doesn’t always make right conclusions.

Today, when AI-based projects are growing rapidly, ethical issues become even more urgent. One of them is racism. AI can have different biases. After all, it learns from data that reflects the current biased decisions that people make. For example, Microsoft’s AI chat bot Tay, launched on March 23, 2016, turned from a peace-loving interlocutor into a real Nazi in less than a day. He was taught racist remarks and insults by Twitter users.

It is not the only example. Another well-known case is when with the help of AI, Northpointe created an assistant judge in the United States that could help make fair decisions. The program was supposed to identify a “risk assessment score” based on available data. This algorithm was much more likely to predict repeated offence and left African-Americans behind bars, especially those who lived in areas with a predominant share of African-Americans. Only 20% of suspects for whom the software identified a high risk of committing crimes did indeed commit them within a few years.

The software was trained on specific court decisions. However, as we know, people who work as judges are guided not only by objective data from the body of crime and the defendant’s story. Decisions are often affected by fatigue, and the time of day  – whether the case is being considered before or after lunch.

Therefore, AI can inherit biases from the data on which it is trained, but at the same time, at least we have control over factors it takes into account. On the other hand, people can uncontrollably transfer their personal relationships and emotions to the subject of their analysis.

Can AI be objective?

Dmitry Parshin, Head of Artezio Software Development Center, believes that it can be so. However, there is one thing to bear in mind.  “For successful operation of artificial intelligence, machines must use the already accumulated data so that they can learn to make decisions based on previous experience. It means that machine learning algorithms rely on data and practices that exist in our society. Thus, artificial intelligence is based on all prior experience that includes certain prejudices dominating in the public conscience. Therefore, AI learns from human prejudices as well,” he believes.

For example, due to learning difficulties, Tay chat bot was rejected. Based on AI, the chat bot was meant to start and engage in dialogues politely and cheerfully, seeking to attract people through playful conversations. But it soon became popular among trolls, who, using machine learning, taught him to support racist ideas and anti-Semitic statements, call for genocide, admire Hitler, and praise Trump. For these reasons, Tay was very soon criticized by users.

Errors in training resulted in failure of several more projects. For example, Amazon created an AI algorithm to automate an employee search and recruitment system. However, everything went not as it had been expected, the system began to refuse female applicants. “The company wanted to get a tool that would select the top five resumes out of a hundred, and then hire these people,” said one of the interlocutors to Reuters. The initial idea was that the system itself would evaluate resumes and cover letters of job applicants and then assign them a rating of one to five stars.

“The management literally wanted us to provide the algorithm with 100 resumes, then it will spit out the top five, and the company can hire those people, – said one of the software developers to Reuters. – However, it became clear that the created system did not adhere to the principles of gender neutrality, since previously most resumes had been submitted by men. Thus, the algorithm began to reject applications that included the word “women.”

Later, the program was edited so that artificial intelligence did not mark this word and its derivatives as something negative, but it did not help much. Amazon recognized the program as unreliable, and the project was closed, however, according to Reuters, for some time the HR Department took into account recommendations of the algorithm, although their final decision was not based on them.

Why is this happening?

In machine learning, a programmer provides AI with an initial dataset. For example, the face recognition AI receives many images, some of which are labeled as “faces,” and some as “not faces.” Over time, AI comes up with its own templates and algorithms that allow it to highlight a face in the image.

But what happens if the initial faces that AI receives are mostly white males? Eventually, AI can decide that to be white is a necessary criterion and will reject faces with a black skin color. Today facial recognition systems are used in the subway, at railway stations, and airports. What will be the cost of their error?

Today, when developing complex AI training schemes, you can get far from the results that were originally expected. The problem is that the methodological basis for teaching AI is not always a high quality one. All automated research of content generated by web users leads to “wrong” conclusions and results. It means that for quality AI training, it is impossible to quickly collect an array of data from open sources. Even in order to teach the warehouse robot to recognize a wrinkled box[AU2] , you must manually create a high-quality image database, which will take several months.

Dangerous knowledge from the Internet

Artezio experts note a recent study that was published at Cornell University in the United States. The researchers created AI to detect and analyze expressions containing “hate speech”. “It turned out that a disproportionate percentage of online posts containing racist, sexist, and simply aggressive language is written in African American English.” They decided not to conduct further software development.

In the same study, a similar Google experiment was mentioned, during which it was found that the vast majority of sexist statements on the internet are humiliating or offensive to men, while the number of abusive messages about women is relatively small. It was decided not to conduct further software development.

Finally, a 2016 Demos AI study showed that more than half of the misogynistic (offensive to women) online messages are posted by women.

“As for the inclusion of various racial and gender characteristics in AI in contextual analysis, this task is currently unsolvable. Therefore, by definition, any AI will be a politically incorrect racist and sexist tool until it learns to deliver results needed by fighters for social justice,” believes the expert.

Of course, you can reduce the risk of transferring prejudices and biases in AI.

However, Dmitry Parshin says that this is difficult to perform, because all such prejudices are closely related to our lives, and the data is only a statement of facts that does not always take into account the context. “You can remove obvious inconsistencies from the training database, but this will only be a small part of them. In real life, we ourselves cannot fully recognize any actions and deeds as obviously sexist or racist even when formally it is so. As you see, everything is confusing and complicated: we sometimes make mistakes and cannot clearly explain what will be ethically correct,” he says.

No freedom from prejudice?

As business increasingly relies on AI, it is important to bear in mind that AI may possess human weaknesses of its creators. Therefore, technology companies should strive to become more diverse in the selection of information for the system and be prepared to critically analyze data provided by AI. All services using artificial intelligence that are designed to categorize personality traits will encounter racist, sexist, and other human biases when training.