Posts

How to better track cyber hate: AI to the rescue

/2 Comments/in , /by

The widescale use of social media, sometimes under cover of anonymity, has liberated speech and led to a proliferation of ideas, discussions and opinions on the internet. It has also led to a flood of hateful, sexist, racist and abusive speech. Confronted with this phenomenon, more and more platforms today are using automated solutions to combat cyber hate. These solutions are based on algorithms that can also introduce biases, sometimes discriminating against certain communities, and are still largely perfectible. In this context, French researchers are developing ever more efficient new models to detect hate speech and reduce the bias.

On September 16 this year, internet users launched a movement calling for a one-day boycott of Instagram. Supported by many American celebrities, the “Stop Hate for Profit” day aimed to challenge Facebook, the mother company of the photo and video sharing app, on the proliferation of hate, propaganda and misinformation on its platforms. Back in May 2019, during its bi-annual report on the state of moderation on its network, Facebook announced significant progress in the automated detection of hate content. According to the company, between January and April 2019, more than 65% of these messages were detected and moderated before users even reported them, compared with 38% during the same period in 2018.

Strongly encouraged to combat online hate content, in particular by the “Avia law” (named after the member of parliament for Paris, Lætitia Avia), platforms use various techniques such as detection by keywords, reporting by users and solutions based on artificial intelligence (AI). Machine learning allows predictive models to be developed from corpora of data. This is where biases can be damaging. “We realized that the automated tools themselves had biases against gender or the user’s identity and, most importantly, had a disproportionately negative impact on certain minority groups such as Afro-Americans,” explains Marzieh Mozafari, PhD student at Télécom SudParis. On Twitter, for example, it is difficult for AI-based programs to take into account the social context of tweets, the identity and dialect of the speaker and the immediate context of the tweet all at once. Some content is thus removed despite being neither hateful nor offensive.

So how can we minimize these biases and erroneous detections without creating a form of censorship? Researchers at Télécom SudParis have been using a public dataset collected on Twitter, distinguishing between tweets written in Afro-American English (AAE) and Standard American English (SAE), as well as two reference databases that have been annotated (sexist, racist, hateful and offensive) by experts and through crowdsourcing. “In this study, due to the lack of data, we mainly relied on cutting-edge language processing techniques such as transfer learning and the BERT language model, a pre-trained, unsupervised model”, explain the researchers.

Developed by Google, the BERT (Bidirectional Encoder Representations from Transformers) model uses a vast corpus of textual content, containing, among other things, the entire content of the English version of Wikipedia. “We were able to “customize” BERT [1] to make it do a specific task, to adjust it for our hateful and offensive corpus”, explains Reza Farahbakhsh, a researcher in data science at Télécom SudParis. To begin with, they tried to identify word sequences in their datasets that were strongly correlated with a hateful or offensive category. Their results showed that tweets written in AAE were almost 10 times more likely to be classed as racist, sexist, hateful or offensive compared to tweets written in SAE. “We therefore used a reweighting mechanism to mitigate biases based on data and algorithms,” says Marzieh Mozafari. For example, the number of tweets containing “n*gga” and “b*tch” is 35 times higher among tweeters in AAE than in SAE and these tweets will often be wrongly identified as racist or sexist. However, this type of word is common in AAE dialects and is used in everyday conversation. It is therefore likely that they will be considered hateful or offensive when they are written in SAE by an associated group.

“In fact, these biases are also cultural: certain expressions considered hateful or offensive are not so within a certain community or in a certain context. In French, too, we use certain bird names to address our loved ones! Platforms are faced with a sort of dilemma: if the aim is to perfectly identify all hateful content, too great a number of false detections could have an impact on users’ “natural” ways of expressing themselves,” explains Noël Crespi, a researcher at Télécom SudParis. After reducing the effect of the most frequently used words in the training data through the reweighting mechanism, this probability of false positives was greatly reduced. “Finally, we transmitted these results to the pre-trained BERT model to refine it even further using new datasets,” says the researcher.

Can automatic detection be scaled up?

Despite these promising results, many problems still need to be solved in order to better detect hate speech. These include the possibility of deploying these automated tools for all languages spoken on social networks. This issue is the subject of a data science challenge launched for the second consecutive year: the HASOC (Hate Speech and Offensive Content Identification in Indo-European Languages), in which a team from IMT Mines d’Alès is participating. “The challenge aims to accomplish three tasks: determine whether or not content is hateful or offensive, classify this content into one of three categories: hateful, offensive or obscene, and identify whether the insult is directed towards an individual or a specific group,” explains Sébastien Harispe, a researcher at IMT Mines Alès.

“We are mainly focusing on the first three tasks. Using our expertise in natural language processing, we have proposed a method of analysis based on supervised machine learning techniques that take advantage of examples and counter-examples of classes to be distinguished.” In this case, the researchers’ work focuses on small datasets in English, German and Hindi. In particular, the team is studying the role of emojis, some of which can have direct connotations with hate expressions. The researchers have also studied the adaptation of various standard approaches in automatic language processing in order to obtain classifiers able to efficiently exploit such markers.

They have also measured their classifiers’ ability to capture these markers, in particular through their performance. “In English, for example, our model was able to correctly classify content in 78% of cases, whereas only 77% of human annotators initially agreed on the annotation to be given to the content of the data set used,” explains Sébastien Harispe. Indeed, in 23% of cases, the annotators expressed divergent opinions when confronted with dubious content that probably needed to have been studied with account taken of the contextual elements.

What can we expect from AI? The researcher believes we are faced with a complex question: what are we willing to accept in the use of this type of technology? “Although remarkable progress has been made in almost a decade of data science, we have to admit that we are addressing a young discipline in which much remains to be developed from a theoretical point of view and, especially, for which we must accompany the applications in order to allow ethical and informed uses. Nevertheless, I believe that in terms of the detection of hate speech, there is a sort of glass roof created by the difficulty of the task as it is translated in our current datasets. With regard to this particular aspect, there can be no perfect or flawless system if we ourselves cannot be perfect.”

Besides the multilingual challenge, the researchers are facing other obstacles such as the availability of data for model training and the evaluation of results, or the difficulty in assessing the ambiguity of certain content, due for example to variations in writing style. Finally, the very characterization of hate speech, subjective as it is, is also a challenge. “Our work can provide material for the humanities and social sciences, which are beginning to address these questions: why, when, who, what content? What role does culture play in this phenomenon? The spread of cyber hate is, at the end of the day, less of a technical problem than a societal one” says Reza Farahbakhsh.

[1] M. Mozafari, R. Farahbakhsh, N. Crespi, “Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model”, PLoS ONE 15(8): e0237861. https://doi.org/10.1371/journal.pone.0237861

Anne-Sophie Boutaud

Also read on I’MTech

Algorithmic bias, discrimination and fairness
AI

AI for interoperable and autonomous industrial systems

/0 Comments/in , /by

At Mines Saint-Étienne, researchers Olivier Boissier, Maxime Lefrançois and Antoine Zimmermann are using AI to tackle the issue of interoperability, which is essential to the industry of the future. The standardization of information in the form of knowledge graphs has allowed them to enable communication between machines that speak different languages. They then operate this system via a network of autonomous distributed agents on each machine to automate a production line.

Taking a train from France to Spain without interoperability means having to get off at the border since the rails are not the same in both countries. A train that hopes to cross over from one rail line to another is sure to derail. The same problem is posed on factory floors – which is why the interoperability of production lines is a key issue for the industry of the future. In an interoperable system, machines can communicate with one another in order to work together automatically, even if they don’t speak the same language. But this is not easy to implement. Factory floors are marked by a kind of cacophony of computer languages. And every machine has its own properties: a multitude of manufacturers, different applications, diverse ways of sending, measuring and collecting information etc. Such heterogeneity reduces the flexibility of production lines. During the Covid-19 crisis, for example, many companies had to reconfigure all of their machines by hand to set up new production operations, such as manufacturing masks. “As of now, on factory floors everything is coded according to an ideal world. Systems are incapable of adapting to change,” says Maxime Lefrançois, a specialist in web semantics. Interoperability also goes hand in hand with competition. Without it, ensuring that a factory runs smoothly would require investing in a single brand of equipment to be certain the various parts are compatible.  

There is no single method for making a system interoperable. Along with his colleagues at Mines Saint-Étienne, the researcher is addressing the issue of interoperability using an approach based on representing data about the machines (manufacturer, connection method, application, physical environment etc.) in a standardized way, meaning independent of the language inherent to a machine. This knowledge is then used by what is known as a multi-agent software system. The goal is to automate a production process based on the description of each machine.

Describing machines to automate decision-making

What does the automation of an industrial system imply? Service delegation, primarily. For example, allowing a machine to place an order for raw materials when it detects a low stock level, instead of going through a human operator. For this, the researchers are developing mechanisms for accessing and exchanging information between machines using the web of things. “On the web, we can set up a communication interface between the various devices via standardized protocols. These methods of interaction therefore reduce the heterogeneity of the language of connected devices,” explains Antoine Zimmermann, an expert in knowledge representation at Mines Saint-Étienne. All of the modeled data from the factory floor is therefore accessible to and understood by all the machines involved.

More importantly, these resources may then be used to allow the machines to cooperate with one another. To this end, the Mines Saint-Étienne team has opted for a flexible approach with local decision-making. In other words, an information system called an autonomous agent is deployed on each device and is able to interact with the agents on other machines. This results in a 4.0 word-of mouth system without loss of information. “An autonomous agent decides what to do based on what the machines upstream and downstream of its position are doing. This reasoning software layer allows the connected device to adjust its behavior according to current status of the system,” says Olivier Boissier, who specializes in autonomous agent systems at Mines Saint-Étienne. For example, a machine can stop a potentially dangerous process when it detects information indicating that a device’s temperature is too high. Likewise, it would no longer be necessary to redesign the entire system to add a component, since it is automatically detected by the other machines.

Read more on I’MTech: A dictionary for connected devices

Depending on the circumstances of the factory floor, a machine may also connect to different production lines to perform other tasks. “We no longer code a machine’s specific action, but the objective it must achieve. The actions are deduced by each agent using the data it collects. It therefore contributes to fulfilling a general mission,” adds the researcher. In this approach, no single agent can achieve this objective alone as each one has a range of action limited to its machine and possesses only part of the knowledge about the overall line. The key to success it therefore cooperation. This makes it possible to transition from producing cups to bottles, simply by changing the objective of the line, without reprogramming it from A to Z.

Towards industrial experiments

Last summer, the IT’m Factory technological platform, a simulated industrial space at Mines Saint-Étienne, hosted a case study for an interoperable and cooperative distributed system. This production line starts out with a first machine responsible for retrieving a cup in a storage area and placing it on a conveyor. A filling system then fills the cup with a liquid. When this second machine has run out of product to pour, it places a remote order with a supplier. At every step, several methods of cooperation are possible. The first is to send a message from one agent to another in order to notify it of the task it has just performed. A second method uses machine perception to detect the action performed by the previous machine. A certain method may be preferable depending on the objectives (production speed etc.).

The researchers have also shown that a robot in the middle of the line may be replaced by another. Interoperability made it possible for the line to adapt to hardware changes without impacting its production. This issue of flexibility is extremely important with a view towards integrating a new generation of nomadic robots. “In September 2020, we start the SIRAM industry of the future project, which should make it possible to deploy interoperable, adaptable information systems to control mobile robotic assistants,” says Maxime Lefrançois. In the future, these devices could be positioned at strategic locations in companies to assist humans or retrieve components at different parts of the production line. But to do so, they must be able to interact with the other machines on the factory floor.  

Anaïs Culot