Posts

AI-4-Child “Chaire” research consortium: innovative tools to fight against childhood cerebral palsy

/0 Comments/in , , /by

In conjunction with the GIS BeAChild, the AI-4-Child team is using artificial intelligence to analyze images related to cerebral palsy in children. This could lead to better diagnoses, innovative therapies and progress in patient rehabilitation. But also a real breakthrough in medical imaging.

The original version of this article was published on the IMT Atlantique website, in the News section.

Cerebral palsy is the leading cause of motor disability in children, affecting nearly two out of every 1,000 newborns. And it is irreversible. The AI-4-Child chaire (French research consortium), managed by IMT Atlantique and the Brest University Hospital, is dedicated to fighting this dreaded disease, using artificial intelligence and deep learning, which could eventually revolutionize the field of medical imaging.

“Cerebral palsy is the result of a brain lesion that occurs around birth,” explains François Rousseau, head of the consortium, professor at IMT Atlantique and a researcher at the Medical Information Processing Laboratory (LaTIM, INSERM unit). “There are many possible causes – prematurity or a stroke in utero, for example. This lesion, of variable importance, is not progressive. The resulting disability can be more or less severe: some children have to use a wheelchair, while others can retain a certain degree of independence.”

Created in 2020, AI-4-Child brings together engineers and physicians. The result of a call for ‘artificial intelligence’ projects from the French National Research Agency (ANR), it operates in partnership with the company Philips and the Ildys Foundation for the Disabled, and benefits from various forms of support (Brittany Region, Brest Metropolis, etc.). In total, the research program has a budget of around €1 million for a period of five years.

Chaire AI-4-Child, François Rousseau
François Rousseau, professor at IMT Atlantique and head of the AI-4-Child chaire (research consortium)

Hundreds of children being studied in Brest

AI-4-Child works closely with BeAChild*, the first French Scientific Interest Group (GIS) dedicated to pediatric rehabilitation, headed by Sylvain Brochard, professor of physical medicine and rehabilitation (MPR). Both structures are linked to the LaTIM lab (INSERM UMR 1101), housed within the Brest CHRU teaching hospital. The BeAChild team is also highly interdisciplinary, bringing together engineers, doctors, pediatricians and physiotherapists, as well as psychologists.

Hundreds of children from all over France and even from several European countries are being followed at the CHRU and at Ty Yann (Ildys Foundation). By bringing together all the ‘stakeholders’ – patients and families, health professionals and imaging specialists – on the same site, Brest offers a highly innovative approach, which has made it a reference center for the evaluation and treatment of cerebral palsy. This has enabled the development of new therapies to improve children’s autonomy and made it possible to design specific applications dedicated to their rehabilitation.

“In this context, the mission of the chair consists of analyzing, via artificial intelligence, the imagery and signals obtained by MRI, movement analysis or electroencephalograms,” says Rousseau. These observations can be made from the fetal stage or during the first years of a child’s life. The research team is working on images of the brain (location of the lesion, possible compensation by the other hemisphere, link with the disability observed, etc.), but also on images of the neuro-musculo-skeletal system, obtained using dynamic MRI, which help to understand what is happening inside the joints.

‘Reconstructing’ faulty images with AI

But this imaging work is complex. The main pitfall is the poor quality of the images collected, due to camera shake or artifacts during the shooting. So AI-4-Child is trying to ‘reconstruct’ them, using artificial intelligence and deep learning. “We are relying in particular on good quality views from other databases to achieve satisfactory resolution,” explains the researcher. Eventually, these methods should be able to be applied to routine images.

Significant progress has already been made. A doctoral student is studying images of the ankle obtained in dynamic MRI and ‘enriched’ by other images using AI – static images, but in very high resolution. “Despite a rather poor initial quality, we can obtain decent pictures,” notes Rousseau.  Significant differences between the shapes of the ankle bone structure were observed between patients and are being interpreted with the clinicians. The aim will then be to better understand the origin of these deformations and to propose adjustments to the treatments under consideration (surgery, toxin, etc.).

The second area of work for AI-4-Child is rehabilitation. Here again, imaging plays an important role: during rehabilitation courses, patients’ gait is filmed using infrared cameras and a system of sensors and force plates in the movement laboratory at the Brest University Hospital. The ‘walking signals’ collected in this way are then analyzed using AI. For the moment, the team is in the data acquisition phase.

Several areas of progress

The problem, however, is that a patient often does not walk in the same way during the course and when they leave the hospital. “This creates a very strong bias in the analysis,” notes Rousseau. “We must therefore check the relevance of the data collected in the hospital environment… and focus on improving the quality of life of patients, rather than the shape of their bones.”

Another difficulty is that the data sets available to the researchers are limited to a few dozen images – whereas some AI applications require several million, not to mention the fact that this data is not homogeneous, and that there are also losses. “We have therefore become accustomed to working with little data,” says Rousseau. “We have to make sure that the quality of the data is as good as possible.” Nevertheless, significant progress has already been made in rehabilitation. Some children are able to ride a bike, tie their shoes, or eat independently.

In the future, the AI-4-Child team plans to make progress in three directions: improving images of the brain, observing bones and joints, and analyzing movement itself. The team also hopes to have access to more data, thanks to a European data collection project. Rousseau is optimistic: “Thanks to data processing, we may be able to better characterize the pathology, improve diagnosis and even identify predictive factors for the disease.”

* BeAChild brings together the Brest University Hospital Centre, IMT Atlantique, the Ildys Foundation and the University of Western Brittany (UBO). Created in 2020 and formalized in 2022 (see the French press release), the GIS is the culmination of a collaboration that began some fifteen years ago on the theme of childhood disability.

Planning for the worst with artificial intelligence

/0 Comments/in , /by

Given that catastrophic events are rare by nature, it is difficult to prepare for them. However, artificial intelligence offers high-performing tools for modeling and simulation, and is therefore an excellent tool to design, test and optimize the response to such events. At IMT Mines Alès, Satya Lancel and Cyril Orengo are both undertaking research on emergency evacuations, in case of events like a dam breaking or a terrorist attack in a supermarket.

“Supermarkets are highly complex environments in which individuals are saturated with information,” explains Satya Lancel, PhD student in Risk Science at Université Montpellier III and IMT Mines Alès. Her thesis, which she started over two years ago, is on the subject of affordance, a psychological concept that states that an object or element in the environment is able to suggest its own use. With this research, she wishes to study the link between the cognitive processes involved in decision-making and the functionality of objects in their environment.

In her thesis, Lancel specifically focuses on affordance in the case of an armed attack within a supermarket. She investigates, for example, how to optimize instructions to encourage customers to head towards emergency evacuation exits. “The results of my research could act as a foundation for future research and be used by supermarket brands to improve signage or staff training, in order to improve evacuation procedures”, she explains.

Lancel and her team obtained funding from the brand U to perform their experiments. This agreement allowed them to study the situational and cognitive factors involved in customer decision-making in several U stores. “One thing we did in the first part of my research plan was to observe customer behavior when we added or removed flashing lights at the emergency exits,” she describes. “We remarked that when there was active signage, customers are more likely to decide to head towards the emergency exits than when there was not,” says the scientist. This result suggests that signage has a certain level of importance in guiding people’s decision-making, even if they do not know the evacuation procedure in advance.

 “Given that it is forbidden to perform simulations of armed attacks with real people, we opted for a multi-agent digital simulation”, explains Lancel. What is unique about this kind of simulation is that each agent involved is conceptualized as an autonomous entity with its own characteristics and behavioral model. In these simulations, the agents interact and influence each other with their behavior. “These models are now used more and more in risk science, as they are proving to be extremely useful for analyzing group behavior,” she declares.

To develop this simulation, Lancel collaborated with Vincent Chapurlat, digital systems modeling researcher at IMT Mines Alès. “The model we designed is a three-dimensional representation of the supermarket we are working on,” she indicates. In the simulation, aisles are represented by parallepipeds, while customers and staff are represented by agents defined by points. By observing how agents gather and how the clusters they form move around, interact and organize themselves, it is possible to determine which group behaviors are most common in the event of an armed attack, no matter the characteristics of the individuals.

Representing the complexity of reality

Outside of supermarkets, Cyril Orengo, PhD student in Crisis Management at the Risk Science Laboratory at IMT Mines Alès, is studying population evacuation in the event of dam failure. The case study chosen by Orengo is the Sainte-Cécile-d’Andorge dam, near the city of Alès. Based on digital modeling of the Alès urban area and individuals, he plans to compare evacuation time for a range of scenarios and perform cartography of various city roads that are likely to be blocked. “One of the aims of this work is to build a knowledge base that could be used in the future by researchers working on preventive evacuations,” indicates the doctoral student.

He, too, has chosen to use a multi-agent system to simulate evacuations, as this method makes it possible to combine individual parameters with agents to produce situations that tend to be close to a credible reality. “Among the variables selected in my model are certain socio-economic characteristics of the simulated population,” he specifies. “In a real-life situation, an elderly person may take longer to go somewhere than a young person: the multi-agent system makes it possible to reproduce this,” explains the researcher.

To generate a credible simulation, “you first need to understand the preventive evacuation process,” underlines Orengo, specifying the need “to identify the actors involved, such as citizens and groups, as well as the infrastructure, such as buildings and traffic routes, in order to produce a model to act as a foundation to develop the digital simulation”. As part of his work, the PhD student analyzed INSEE databases to try and reproduce the socioeconomic characteristics of the Alès population. Orengo used a specialized platform for building agent simulations to create his own. “This platform allows researchers without computer programming training to create models, controlling various parameters that they define themselves,” explains the doctoral student. One of the limitations of this kind of simulation is computing power, which means only a certain number of variables can be taken into account. According to Orengo, his model still needs many improvements. These include “integrating individual vehicles, public transport, decision-making processes relating to risk management and more detailed reproduction of human behaviors”, he specifies. For Lancel, virtual reality could be an important addition, increasing participants’ immersion in the study, “By placing a participant in a virtual crowd, researchers could observe how they react to certain agents and their environment, which would allow them to refine their research,” she concludes.

Rémy Fauvel

Projet MAESTRIA AVC

A European consortium for early detection of stroke and atrial fibrillation

/0 Comments/in , , , /by

The European project MAESTRIA, launched in March 2021 and set to run 5 years, will take on the major challenges of data integration and personalized medicine with the aim of preventing heart rhythm problems and stroke. How? By using artificial intelligence approaches to create multi-parametric digital tools. Led by Sorbonne University and funded by the European Union to the tune of €14 million, the project brings together European, English, Canadian and American partners. An interview with Anne-Sophie Taillandier, Director of Teralab, IMT’s Big Data and AI platform, which is a member of the consortium.   

In what health context was the MAESTRIA developed?

Anne-Sophie Taillandier – Atrial fibrillation (AF), heart rhythm disorder and stroke are major health problems in Europe. Most often, they are the clinical expression of atrial cardiomyopathy, which is under-recognized due to a lack of specific diagnostic tools.

What is the aim of MAESTRIA?

AST – MAESTRIA (for Machine Learning Artificial Intelligence for Early Detection of Stroke and Atrial Fibrillation) aims to prevent the risks associated with atrial fibrillation in order to ensure healthy ageing in the European population. Multidisciplinary research and stratified approaches (involving adapting  a patient’s treatment depending on his/her biological characteristics) are needed to diagnose and treat AF and stroke.

What technologies will be deployed?

AST – “Digital twin” technologies, a powerful data integrator combining biophysics and AI, will be used to generate virtual twins of human heart atria using patient-specific data.

MAESTRIA will create digital multi-parametric digital tools based on a new generation of biomarkers that integrate artificial intelligence (AI) and big data from cutting-edge imaging, electrocardiography and omics technologies (including physiological responses modulated by individual susceptibility and lifestyle factors). Diagnostic tools and personalized therapies for atrial cardiomyopathy will be developed.

Unique experimental large-animal models, ongoing patient cohorts and a prospective cohort of MAESTRIA patients will provide rigorous validation of the new biomarkers and tools developed. A dedicated central laboratory will collect and harmonize clinical data. MAESTRIA will be organized as a user-centered platform that is easily accessible via clinical parameters commonly used in European hospitals.

What is the role of Teralab, IMT’s Big Data and AI platform?

AST – The TeraLab team, led by Natalie Cernecka and Luis Pineda, is playing a central role in this project, in three ways. First of all, TeraLab will be involved in making heterogeneous, sensitive health data available for the consortium, while ensuring legal compatibility and security.

Second, TeraLab will build and manage the data hub for the project data, and make this data available to the team of researchers so that they can aggregate and analyze it, and then build a results demonstrator for doctors and patients.

And last but not least, TeraLab will oversee the data management plan or DMP, an essential part of the management of any European project. It is a living document that sets out a plan for managing the data used and generated within the framework of the project. Initiated at the start of the project, this plan is updated periodically to make sure that it still appropriate in light of how the project is progressing. It is even more necessary when it’s a matter of health data management.

Who are the partners for MAESTRIA ?

AST – MAESTRIA is a European consortium of 18 clinicians, scientists and pharmaceutical industry representatives, at the cutting edge of research and medical care for AF and stroke patients. A scientific advisory board including potential clinician users will help MAESTRIA respond to clinical and market needs.

It’s an international project, focused on the EU countries, but certain partners come from England, Canada and the United States. Oxford University, for example, has developed interesting solutions for the processing and aggregation of cardiological data. It is a member of the consortium and we will, of course, be working with its researchers.

We have important French partners such as AP-HP (Assistance Publique-Hôpitaux de Paris, Paris Hospital Authority) involved in data routing and management. The project is coordinated by Sorbonne University.

What are the next big steps for the project?

AST – The MAESTRIA has just been launched, the first big step is making the data available and establishing the legal framework.

Because the data used in this project is heterogeneous – hence the importance of aggregating it – we must understand the specific characteristics of each kind of data (human data, animal data, images, medical files etc.) and adapt our workspaces to users. Since this data is sensitive, security and confidentially challenges are paramount.

Learn more about MAESTRIA

Interview by Véronique Charlet

Digital innovations in health

Innovation in health: towards responsibility

/0 Comments/in /by

Digital innovations are paving the way for more accurate predictive medicine and a more resilient healthcare system. In order to establish themselves on the market and reduce their potential negative effects, these technologies must be responsible. Christine Balagué, a researcher in digital ethics at Institut Mines-Télécom Business School, presents the risks associated with innovations in the health sector and ways to avoid them.

“Until now, the company has approached technology development without looking at the environmental and social impacts of the digital innovations produced. The time has come to do something about this, especially when it comes to human lives in the health sector”, says Christine Balagué, a researcher at Institut Mines-Telecom Business School and co-holder of the Good in Tech Chair [1]. From databases and artificial intelligence for detecting and treating rare diseases, to connected objects for monitoring patients; the rapid emergence of tools for prediction, diagnosis and also business organization is making major changes in the healthcare sector. Similarly, the goal of a smarter hospital of the future is set to radically change the healthcare systems we know today. The focus is on building on medical knowledge, advancing medical research, and improving care.

However, for Christine Balagué, a distinction must be made between the notion of “tech for good” – which consists of developing systems for the benefit of society – and “good in tech”. She says “an innovation, however benevolent it may be, is not necessarily devoid of bias and negative effects. It’s important not to stop at the positive impacts but to also measure the potential negative effects in order to eliminate them.” The time has come for responsible innovation. In this sense, the Good in Tech chair, dedicated to responsibility and ethics in digital innovations and artificial intelligence, aims to measure the still underestimated environmental and societal impacts of technologies on various sectors, including health.

Digital innovations: what are the risks for healthcare systems?

In healthcare, it is clear: an algorithm that cannot be explained is unlikely to be commercialized, even if it is efficient. Indeed, the potential risks are too critical when human lives are at stake. However, a study published in 2019 in the journal Science on the use of commercial algorithms in the U.S. health care system demonstrated the presence of racial bias in the results of these tools. This discrimination between patients, or between different geographical areas, therefore gives rise to an initial risk of unequal access to care. “The more automated data processing becomes, the more inequalities are created,” says Christine Balagué. However, machine learning is increasingly being used in the solutions offered to healthcare professionals.

For example, French start-ups such as Aiintense, incubated at IMT Starter, and BrainTale use it for diagnostic purposes. Aiintense is developing decision support tools for all pathologies encountered in intensive care units. BrainTale is looking at the quantification of brain lesions. These two examples raise the question of possible discrimination by algorithms. “These cases are interesting because they are based on work carried out by researchers and have been recognized internationally by the scientific peer community, but they use deep learning models whose results are not entirely explainable. This therefore hinders their application by intensive care units, which need to understand how these algorithms work before making major decisions about patients,” says the researcher.

Furthermore, genome sequencing algorithms raise questions about the relationship between doctors and their patients. Indeed, the limitations of the algorithm, the presence of false positives or false negatives are rarely presented to patients. In some cases, this may lead to the implementation of unsuitable treatments or operations. It is also possible that an algorithm may be biased by the opinion of its designer. Finally, unconscious biases associated with the processing of data by humans can also lead to inequalities. Artificial intelligence in particular thus raises many ethical questions about its use in the healthcare setting.

What do we mean by a “responsible innovation”? It is not just a question of complying with data processing laws and improving the health care professional’s way of working. “We must go further. This is why we want to measure two criteria in new technologies: their environmental impact and their societal impact, distinguishing between the potential positive and negative effects for each. Innovations should then be developed according to predefined criteria aimed at limiting their negative effects,” says Christine Balagué.

Changing the way innovations are designed

Liability is not simply a layer of processing that can be added to an existing technology. Thinking about responsible innovation implies, on the contrary, changing the very manner in which innovations are designed. So how do we ensure they are responsible? Scientists are looking for precise indicators that could result in a “to do list” of criteria to be verified. This starts with the analysis of the data used for learning, but also by studying the interface developed for the users, through the architecture of the neural network that can potentially generate bias. On the other hand, existing environmental criteria must be refined by taking into account the design chain of a connected object and the energy consumption of the algorithms. “The criteria identified could be integrated into corporate social responsibility in order to measure changes over time,” says Christine Balagué.

In the framework of the Good In Tech chair, several research projects, including a thesis, are being carried out on our capacity to explain algorithms. Among them, Christine Balagué and Nesma Houmani (a researcher at Télécom SudParis) are interested in algorithms for electroencephalography (EEG) analysis. Their objective is to ensure that the tools use interfaces that can be explained to health care professionals, the future users of the system. “Our interviews show that explaining how an algorithm works to users is often something that designers aren’t interested in, and that making it explicit would be a source of change in the decision-making process,” says the researcher. The ability to explain and interpret results are therefore two key words guiding responsible innovation.

Ultimately, the researchers have identified four principles that an innovation in healthcare must follow. The first is anticipation in order to measure the potential benefits and risks upstream of the development phase. Then, a reflexive approach allows the designer to limit the negative effects and to integrate into the system itself an interface to explain how the technological innovation works to physicians. It must also be inclusive, i.e. reaching all patients throughout the territory. Finally, responsive innovation facilitates rapid adaptation to the changing context of healthcare systems. Christine Balagué concludes: “Our work shows that taking into account ethical criteria does not reduce the performance of algorithms. On the contrary, taking into account issues of responsibility helps to promote the acceptance of an innovation on the market”.

[1] The Chair is supported by the Institut Mines-Télécom Business School, the School of Management and Innovation at Sciences Po, and the Fondation du Risque, in partnership with Télécom Paris and Télécom SudParis.

Anaïs Culot

Also read on I’MTech :

Healthcare: what makes some connected objects a success and others a flop?
Eclairer boites noires, algorithms

Shedding some light on black box algorithms

/0 Comments/in , /by

In recent decades, algorithms have become increasingly complex, particularly through the introduction of deep learning architectures. This has gone hand in hand with increasing difficulty in explaining their internal functioning, which has become an important issue, both legally and socially. Winston Maxwell, legal researcher, and Florence d’Alché-Buc, researcher in machine learning, who both work for Télécom Paris, describe the current challenges involved in the explainability of algorithms.

What skills are required to tackle the problem of algorithm explainability?

Winston Maxwell: In order to know how to explain algorithms, we must draw on different disciplines. Our multi-disciplinary team, AI Operational Ethics, focuses not only on mathematical, statistical and computational aspects, but also on sociological, economic and legal aspects. For example, we are working on an explainability system for image recognition algorithms used, among other things, for facial recognition in airports. Our work therefore encompasses these different disciplines.

Why are algorithms often difficult to understand?

Florence d’Alché-Buc: Initially, artificial intelligence used mainly symbolic approaches, i.e., it simulated the logic of human reasoning. Logical rules, called expert systems, allowed artificial intelligence to make a decision by exploiting observed facts. This symbolic framework made AI more easily explainable. Since the early 1990s, AI has increasingly relied on statistical learning, such as decision trees or neural networks, as these structures allow for better performance, learning flexibility and robustness.

This type of learning is based on statistical regularities and it is the machine that establishes the rules which allow their exploitation. The human provides input functions and an expected output, and the rest is determined by the machine. A neural network is a composition of functions. Even if we can understand the functions that compose it, their accumulation quickly becomes complex. So a black box is then created, in which it is difficult to know what the machine is calculating.

How can artificial intelligence be made more explainable?

FAB: Current research focuses on two main approaches. There is explainability by design where, for any new constitution of an algorithm, explanatory output functions are implemented which make it possible to progressively describe the steps carried out by the neural network. However, this is costly and impacts the performance of the algorithm, which is why it is not yet very widespread. In general, and this is the other approach, when an existing algorithm needs to be explained, it is an a posteriori approach that is taken, i.e., after an AI has established its calculation functions, we will try to dissect the different stages of its reasoning. For this there are several methods, which generally seek to break the entire complex model down into a set of local models that are less complicated to deal with individually.

Why do algorithms need to be explained?

WM: There are two main reasons why the law stipulates that there is a need for the explainability of algorithms. Firstly, individuals have the right to understand and to challenge an algorithmic decision. Secondly, it must be guaranteed that a supervisory institution such as the  French Data Protection Authority (CNIL), or a court, can understand the operation of the algorithm, both as a whole and in a particular case, for example to make sure that there is no racial discrimination. There is therefore an individual aspect and an institutional aspect.

Does the format of the explanations need to be adapted to each case?

WM: The formats depend on the entity to which it needs to be explained: for example, some formats will be adapted to regulators such as the CNIL, others to experts and yet others to citizens. In 2015, an experimental service to deploy algorithms that detect possible terrorist activities in case of serious threats was introduced. For this to be properly regulated, an external control of the results must be easy to carry out, and therefore the algorithm must be sufficiently transparent and explainable.

Are there any particular difficulties in providing appropriate explanations?

WM: There are several things to bear in mind. For example, information fatigue: when the same explanation is provided systematically, humans will tend to ignore it. It is therefore important to use varying formats when presenting information. Studies have also shown that humans tend to follow a decision given by an algorithm without questioning it. This can be explained in particular by the fact that humans will consider from the outset that the algorithm is statistically wrong less often than themselves. This is what we call automation bias. This is why we want to provide explanations that allow the human agent to understand and take into consideration the context and the limits of algorithms. It is a real challenge to use algorithms to make humans more informed in their decisions, and not the other way around. Algorithms should be a decision aid, not a substitute for human beings.

What are the obstacles associated with the explainability of AI?

FAB: One aspect to be considered when we want to explain an algorithm is cyber security. We must be wary of the potential exploitation of explanations by hackers. There is therefore a triple balance to be found in the development of algorithms: performance, explainability and security.

Is this also an issue of industrial property protection?

WM: Yes, there is also the aspect of protecting business secrets: some developers may be reluctant to discuss their algorithms for fear of being copied. Another counterpart to this is the manipulation of scores: if individuals understand how a ranking algorithm, such as Google’s, works, then it would be possible for them to manipulate their position in the ranking. Manipulation is an important issue not only for search engines, but also for fraud or cyber-attack detection algorithms.

How do you think AI should evolve?

FAB: There are many issues associated with AI. In the coming decades, we will have to move away from the single objective of algorithm performance to multiple additional objectives such as explainability, but also equitability and reliability. All of these objectives will redefine machine learning. Algorithms have spread rapidly and have enormous effects on the evolution of society, but they are very rarely accompanied by instructions for their use. A set of adapted explanations must go hand in hand with their implementation in order to be able to control their place in society.

By Antonin Counillon

Also read on I’MTech: Restricting algorithms to limit their powers of discrimination

 

How to better track cyber hate: AI to the rescue

/2 Comments/in , /by

The widescale use of social media, sometimes under cover of anonymity, has liberated speech and led to a proliferation of ideas, discussions and opinions on the internet. It has also led to a flood of hateful, sexist, racist and abusive speech. Confronted with this phenomenon, more and more platforms today are using automated solutions to combat cyber hate. These solutions are based on algorithms that can also introduce biases, sometimes discriminating against certain communities, and are still largely perfectible. In this context, French researchers are developing ever more efficient new models to detect hate speech and reduce the bias.

On September 16 this year, internet users launched a movement calling for a one-day boycott of Instagram. Supported by many American celebrities, the “Stop Hate for Profit” day aimed to challenge Facebook, the mother company of the photo and video sharing app, on the proliferation of hate, propaganda and misinformation on its platforms. Back in May 2019, during its bi-annual report on the state of moderation on its network, Facebook announced significant progress in the automated detection of hate content. According to the company, between January and April 2019, more than 65% of these messages were detected and moderated before users even reported them, compared with 38% during the same period in 2018.

Strongly encouraged to combat online hate content, in particular by the “Avia law” (named after the member of parliament for Paris, Lætitia Avia), platforms use various techniques such as detection by keywords, reporting by users and solutions based on artificial intelligence (AI). Machine learning allows predictive models to be developed from corpora of data. This is where biases can be damaging. “We realized that the automated tools themselves had biases against gender or the user’s identity and, most importantly, had a disproportionately negative impact on certain minority groups such as Afro-Americans,” explains Marzieh Mozafari, PhD student at Télécom SudParis. On Twitter, for example, it is difficult for AI-based programs to take into account the social context of tweets, the identity and dialect of the speaker and the immediate context of the tweet all at once. Some content is thus removed despite being neither hateful nor offensive.

So how can we minimize these biases and erroneous detections without creating a form of censorship? Researchers at Télécom SudParis have been using a public dataset collected on Twitter, distinguishing between tweets written in Afro-American English (AAE) and Standard American English (SAE), as well as two reference databases that have been annotated (sexist, racist, hateful and offensive) by experts and through crowdsourcing. “In this study, due to the lack of data, we mainly relied on cutting-edge language processing techniques such as transfer learning and the BERT language model, a pre-trained, unsupervised model”, explain the researchers.

Developed by Google, the BERT (Bidirectional Encoder Representations from Transformers) model uses a vast corpus of textual content, containing, among other things, the entire content of the English version of Wikipedia. “We were able to “customize” BERT [1] to make it do a specific task, to adjust it for our hateful and offensive corpus”, explains Reza Farahbakhsh, a researcher in data science at Télécom SudParis. To begin with, they tried to identify word sequences in their datasets that were strongly correlated with a hateful or offensive category. Their results showed that tweets written in AAE were almost 10 times more likely to be classed as racist, sexist, hateful or offensive compared to tweets written in SAE. “We therefore used a reweighting mechanism to mitigate biases based on data and algorithms,” says Marzieh Mozafari. For example, the number of tweets containing “n*gga” and “b*tch” is 35 times higher among tweeters in AAE than in SAE and these tweets will often be wrongly identified as racist or sexist. However, this type of word is common in AAE dialects and is used in everyday conversation. It is therefore likely that they will be considered hateful or offensive when they are written in SAE by an associated group.

“In fact, these biases are also cultural: certain expressions considered hateful or offensive are not so within a certain community or in a certain context. In French, too, we use certain bird names to address our loved ones! Platforms are faced with a sort of dilemma: if the aim is to perfectly identify all hateful content, too great a number of false detections could have an impact on users’ “natural” ways of expressing themselves,” explains Noël Crespi, a researcher at Télécom SudParis. After reducing the effect of the most frequently used words in the training data through the reweighting mechanism, this probability of false positives was greatly reduced. “Finally, we transmitted these results to the pre-trained BERT model to refine it even further using new datasets,” says the researcher.

Can automatic detection be scaled up?

Despite these promising results, many problems still need to be solved in order to better detect hate speech. These include the possibility of deploying these automated tools for all languages spoken on social networks. This issue is the subject of a data science challenge launched for the second consecutive year: the HASOC (Hate Speech and Offensive Content Identification in Indo-European Languages), in which a team from IMT Mines d’Alès is participating. “The challenge aims to accomplish three tasks: determine whether or not content is hateful or offensive, classify this content into one of three categories: hateful, offensive or obscene, and identify whether the insult is directed towards an individual or a specific group,” explains Sébastien Harispe, a researcher at IMT Mines Alès.

“We are mainly focusing on the first three tasks. Using our expertise in natural language processing, we have proposed a method of analysis based on supervised machine learning techniques that take advantage of examples and counter-examples of classes to be distinguished.” In this case, the researchers’ work focuses on small datasets in English, German and Hindi. In particular, the team is studying the role of emojis, some of which can have direct connotations with hate expressions. The researchers have also studied the adaptation of various standard approaches in automatic language processing in order to obtain classifiers able to efficiently exploit such markers.

They have also measured their classifiers’ ability to capture these markers, in particular through their performance. “In English, for example, our model was able to correctly classify content in 78% of cases, whereas only 77% of human annotators initially agreed on the annotation to be given to the content of the data set used,” explains Sébastien Harispe. Indeed, in 23% of cases, the annotators expressed divergent opinions when confronted with dubious content that probably needed to have been studied with account taken of the contextual elements.

What can we expect from AI? The researcher believes we are faced with a complex question: what are we willing to accept in the use of this type of technology? “Although remarkable progress has been made in almost a decade of data science, we have to admit that we are addressing a young discipline in which much remains to be developed from a theoretical point of view and, especially, for which we must accompany the applications in order to allow ethical and informed uses. Nevertheless, I believe that in terms of the detection of hate speech, there is a sort of glass roof created by the difficulty of the task as it is translated in our current datasets. With regard to this particular aspect, there can be no perfect or flawless system if we ourselves cannot be perfect.”

Besides the multilingual challenge, the researchers are facing other obstacles such as the availability of data for model training and the evaluation of results, or the difficulty in assessing the ambiguity of certain content, due for example to variations in writing style. Finally, the very characterization of hate speech, subjective as it is, is also a challenge. “Our work can provide material for the humanities and social sciences, which are beginning to address these questions: why, when, who, what content? What role does culture play in this phenomenon? The spread of cyber hate is, at the end of the day, less of a technical problem than a societal one” says Reza Farahbakhsh.

[1] M. Mozafari, R. Farahbakhsh, N. Crespi, “Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model”, PLoS ONE 15(8): e0237861. https://doi.org/10.1371/journal.pone.0237861

Anne-Sophie Boutaud

Also read on I’MTech

Algorithmic bias, discrimination and fairness
AI

AI for interoperable and autonomous industrial systems

/0 Comments/in , /by

At Mines Saint-Étienne, researchers Olivier Boissier, Maxime Lefrançois and Antoine Zimmermann are using AI to tackle the issue of interoperability, which is essential to the industry of the future. The standardization of information in the form of knowledge graphs has allowed them to enable communication between machines that speak different languages. They then operate this system via a network of autonomous distributed agents on each machine to automate a production line.

Taking a train from France to Spain without interoperability means having to get off at the border since the rails are not the same in both countries. A train that hopes to cross over from one rail line to another is sure to derail. The same problem is posed on factory floors – which is why the interoperability of production lines is a key issue for the industry of the future. In an interoperable system, machines can communicate with one another in order to work together automatically, even if they don’t speak the same language. But this is not easy to implement. Factory floors are marked by a kind of cacophony of computer languages. And every machine has its own properties: a multitude of manufacturers, different applications, diverse ways of sending, measuring and collecting information etc. Such heterogeneity reduces the flexibility of production lines. During the Covid-19 crisis, for example, many companies had to reconfigure all of their machines by hand to set up new production operations, such as manufacturing masks. “As of now, on factory floors everything is coded according to an ideal world. Systems are incapable of adapting to change,” says Maxime Lefrançois, a specialist in web semantics. Interoperability also goes hand in hand with competition. Without it, ensuring that a factory runs smoothly would require investing in a single brand of equipment to be certain the various parts are compatible.  

There is no single method for making a system interoperable. Along with his colleagues at Mines Saint-Étienne, the researcher is addressing the issue of interoperability using an approach based on representing data about the machines (manufacturer, connection method, application, physical environment etc.) in a standardized way, meaning independent of the language inherent to a machine. This knowledge is then used by what is known as a multi-agent software system. The goal is to automate a production process based on the description of each machine.

Describing machines to automate decision-making

What does the automation of an industrial system imply? Service delegation, primarily. For example, allowing a machine to place an order for raw materials when it detects a low stock level, instead of going through a human operator. For this, the researchers are developing mechanisms for accessing and exchanging information between machines using the web of things. “On the web, we can set up a communication interface between the various devices via standardized protocols. These methods of interaction therefore reduce the heterogeneity of the language of connected devices,” explains Antoine Zimmermann, an expert in knowledge representation at Mines Saint-Étienne. All of the modeled data from the factory floor is therefore accessible to and understood by all the machines involved.

More importantly, these resources may then be used to allow the machines to cooperate with one another. To this end, the Mines Saint-Étienne team has opted for a flexible approach with local decision-making. In other words, an information system called an autonomous agent is deployed on each device and is able to interact with the agents on other machines. This results in a 4.0 word-of mouth system without loss of information. “An autonomous agent decides what to do based on what the machines upstream and downstream of its position are doing. This reasoning software layer allows the connected device to adjust its behavior according to current status of the system,” says Olivier Boissier, who specializes in autonomous agent systems at Mines Saint-Étienne. For example, a machine can stop a potentially dangerous process when it detects information indicating that a device’s temperature is too high. Likewise, it would no longer be necessary to redesign the entire system to add a component, since it is automatically detected by the other machines.

Read more on I’MTech: A dictionary for connected devices

Depending on the circumstances of the factory floor, a machine may also connect to different production lines to perform other tasks. “We no longer code a machine’s specific action, but the objective it must achieve. The actions are deduced by each agent using the data it collects. It therefore contributes to fulfilling a general mission,” adds the researcher. In this approach, no single agent can achieve this objective alone as each one has a range of action limited to its machine and possesses only part of the knowledge about the overall line. The key to success it therefore cooperation. This makes it possible to transition from producing cups to bottles, simply by changing the objective of the line, without reprogramming it from A to Z.

Towards industrial experiments

Last summer, the IT’m Factory technological platform, a simulated industrial space at Mines Saint-Étienne, hosted a case study for an interoperable and cooperative distributed system. This production line starts out with a first machine responsible for retrieving a cup in a storage area and placing it on a conveyor. A filling system then fills the cup with a liquid. When this second machine has run out of product to pour, it places a remote order with a supplier. At every step, several methods of cooperation are possible. The first is to send a message from one agent to another in order to notify it of the task it has just performed. A second method uses machine perception to detect the action performed by the previous machine. A certain method may be preferable depending on the objectives (production speed etc.).

The researchers have also shown that a robot in the middle of the line may be replaced by another. Interoperability made it possible for the line to adapt to hardware changes without impacting its production. This issue of flexibility is extremely important with a view towards integrating a new generation of nomadic robots. “In September 2020, we start the SIRAM industry of the future project, which should make it possible to deploy interoperable, adaptable information systems to control mobile robotic assistants,” says Maxime Lefrançois. In the future, these devices could be positioned at strategic locations in companies to assist humans or retrieve components at different parts of the production line. But to do so, they must be able to interact with the other machines on the factory floor.  

Anaïs Culot