"France is to Paris what Japan is to Tokyo."
"Man is to king as woman is to queen."
One of the most commonly used approaches in artificial intelligence constructs this type of analogies. When asked by such an AI system to complete the proposition "Man is to king as woman is to X," it responds with "queen." The AI has learned these analogies from a corpus of texts.
[* : A fairly clear introduction can be found at this link: A Simple Introduction to Word Embeddings]
In 2016, researchers from the University of Boston and the Microsoft Research Center in Cambridge trained such a system using texts retrieved from Google news. Its deductions are quite striking:
- "Man is to developer as woman is to homemaker"
- "Man is to surgeon as woman is to nurse"
- "Man is to merchant as woman is to housewife"
The researchers proposed a method to compensate for this bias from the data. However, their method requires explicit correction and thus requires these biases to be identified.
[* : all the details are in their article https://arxiv.org/pdf/1607.06520.pdf]
Statistically, there are currently more surgeons than there are female surgeons and more nurses than male nurses. It is natural, therefore, that these biases are reflected in Google news articles. The algorithm relies on these data and simply generalizes these biases.
AI is also widely used to describe images. An example application would be to detect the presence of a person and recognize what they are doing. And the results can be impressive: in the following photo, an algorithm is capable of identifying that a woman is cutting fruits with a knife in a kitchen and is not using cooking utensils.
Here again, the algorithm has learned from the provided data. Here again, it has learned biases from this data. It has even amplified them: when the scene depicts a person cooking, half of the men are recognized as women. In kitchen photos containing 67% women and 33% men, after its training, the algorithm saw 84% women and 16% men. The researchers at the University of Virginia who highlighted this amplification propose a method* to reduce it. It leads the algorithm to recognize 80% women and 20% men. They have indeed reduced the bias but have not eliminated it. Perhaps they could achieve this by combining their approach with the one mentioned at the beginning of this article.
[* : all the details are in their article https://homes.cs.
This article presents two examples of cases where biases included in the training data are reflected and generalized in the responses of two algorithms. The question that arises for us is how to build unbiased datasets. Or how to identify all the biases contained in this data. This is a particularly difficult exercise given that data scientists who analyze this data are precisely asked to "make it speak," to find weak signals.
Distinguishing weak signals from biases poses a real challenge. To achieve this, it is necessary to understand the reality behind the data. It is necessary to know that the profession of surgeon is not reserved for men. Therefore, data scientists must work hand in hand with domain experts who know the reality of things.
