Posts

digital simulation

Digital simulation: applications, from medicine to energy

/0 Comments/in , /by

At Mines Saint-Étienne, Yann Gavet uses image simulation to study the characteristics of an object. This method is more economical in terms of time and cost, and eliminates the need for experimental measurements. This field, at the intersection of mathematics, computer science and algorithms, is used for a variety of applications ranging from the medical sector to the study of materials.

What do a human cornea and a fuel cell electrode have in common? Yann Gavet, a researcher in applied mathematics at Mines Saint-Étienne1 is able to model these two objects as 2D or 3D images in order to study their characteristics. To do this, he uses a method based on random fields. “This approach consists in generating a synthetic image representing a surface or a random volume, i.e. whose properties will vary from one point to another across the plane or space,” explains the researcher. In the case of a cornea, for example, this means visualizing an assembly of cells whose density differs according to whether we look at the center or the edge. The researcher’s objective? To create simulations with properties as close as possible to the reality.

Synthetic models and detecting corneal disorders

The density of cells that make up our cornea –the transparent part at the front of the eye– and its endothelium, provides information about its health. To perform these analyses, automatic cell detection and counting algorithms have been developed using deep neural networks. Training them thus requires access to large databases of corneas. The problem is that these do not exist in sufficient quantity. “However, we have shown that it is possible to perform the training process using synthetic images, i.e. simulated by models,” says Yann Gavet.

How does it work? Using deep learning, the researcher creates graphical simulations based on key criteria: size, shape, cell density or the number of neighboring cells. He is able to simulate cell arrangements, as well as complete and realistic images of corneas. However, he wants to combine the two. Indeed, this step is essential for the creation of image databases that will allow us to train the algorithms. He focuses in particular on the realism of the simulation results in terms of cell geometry, gray levels and the “natural” variability of the observations.

Although he demonstrated that training using synthetic corneal data does not require perfectly realistic representations to perform well, improving accuracy will be useful for other applications. “As a matter of fact, we transpose this method to the simulation of material arrangements that compose fuel cell electrodes, which requires more precision,” explains the researcher.

Simulating the impact of microstructures on the performance of a fuel cell

The microstructure of fuel cell electrodes impacts the performance and durability of solid oxide cells. In order to improve these parameters, researchers want to identify the ideal arrangement of the materials that make up the electrodes, i.e., how they should be distributed and organized. To do this, they play with the “basic” geometry of an electrode: its porosity and its material particle size distribution. This therefore targets the morphological parameters on which the manufacturers intervene when designing the electrodes.

To identify the best performing structures, one method would be to build and test a multitude of configurations. This is an expensive and time-consuming practice. The other approach is based on the simulation and optimization of a large number of configurations. Subsequently, a second group of models simulating the physics of a battery can in turn identify which structures best impact the battery’s performance.

The advantage of the simulations is that they target specific areas within the electrodes to better understand their operation and their overall impact on the battery. For example: exchange zones such as “triple phase” points where ionic, electronic and gaseous phases meet, or exchanges between material surfaces. “Our model allows us to evaluate the best configuration, but also to identify the associated manufacturing process that offers the best energy efficiency for the battery,” says Yann Gavet.

In the medium term, the researcher wishes to continue his work on a model whose dimensions are similar to the observations made in X-ray tomography. An algorithmic challenge that will require more computing time, but will also lead to results that are closer to the reality of the field.

1 Yann Gavet is a researcher at the Georges Friedel laboratory, UMR CNRS/Mines Saint-Étienne

Anaïs Culot

Digital innovations in health

Innovation in health: towards responsibility

/0 Comments/in /by

Digital innovations are paving the way for more accurate predictive medicine and a more resilient healthcare system. In order to establish themselves on the market and reduce their potential negative effects, these technologies must be responsible. Christine Balagué, a researcher in digital ethics at Institut Mines-Télécom Business School, presents the risks associated with innovations in the health sector and ways to avoid them.

“Until now, the company has approached technology development without looking at the environmental and social impacts of the digital innovations produced. The time has come to do something about this, especially when it comes to human lives in the health sector”, says Christine Balagué, a researcher at Institut Mines-Telecom Business School and co-holder of the Good in Tech Chair [1]. From databases and artificial intelligence for detecting and treating rare diseases, to connected objects for monitoring patients; the rapid emergence of tools for prediction, diagnosis and also business organization is making major changes in the healthcare sector. Similarly, the goal of a smarter hospital of the future is set to radically change the healthcare systems we know today. The focus is on building on medical knowledge, advancing medical research, and improving care.

However, for Christine Balagué, a distinction must be made between the notion of “tech for good” – which consists of developing systems for the benefit of society – and “good in tech”. She says “an innovation, however benevolent it may be, is not necessarily devoid of bias and negative effects. It’s important not to stop at the positive impacts but to also measure the potential negative effects in order to eliminate them.” The time has come for responsible innovation. In this sense, the Good in Tech chair, dedicated to responsibility and ethics in digital innovations and artificial intelligence, aims to measure the still underestimated environmental and societal impacts of technologies on various sectors, including health.

Digital innovations: what are the risks for healthcare systems?

In healthcare, it is clear: an algorithm that cannot be explained is unlikely to be commercialized, even if it is efficient. Indeed, the potential risks are too critical when human lives are at stake. However, a study published in 2019 in the journal Science on the use of commercial algorithms in the U.S. health care system demonstrated the presence of racial bias in the results of these tools. This discrimination between patients, or between different geographical areas, therefore gives rise to an initial risk of unequal access to care. “The more automated data processing becomes, the more inequalities are created,” says Christine Balagué. However, machine learning is increasingly being used in the solutions offered to healthcare professionals.

For example, French start-ups such as Aiintense, incubated at IMT Starter, and BrainTale use it for diagnostic purposes. Aiintense is developing decision support tools for all pathologies encountered in intensive care units. BrainTale is looking at the quantification of brain lesions. These two examples raise the question of possible discrimination by algorithms. “These cases are interesting because they are based on work carried out by researchers and have been recognized internationally by the scientific peer community, but they use deep learning models whose results are not entirely explainable. This therefore hinders their application by intensive care units, which need to understand how these algorithms work before making major decisions about patients,” says the researcher.

Furthermore, genome sequencing algorithms raise questions about the relationship between doctors and their patients. Indeed, the limitations of the algorithm, the presence of false positives or false negatives are rarely presented to patients. In some cases, this may lead to the implementation of unsuitable treatments or operations. It is also possible that an algorithm may be biased by the opinion of its designer. Finally, unconscious biases associated with the processing of data by humans can also lead to inequalities. Artificial intelligence in particular thus raises many ethical questions about its use in the healthcare setting.

What do we mean by a “responsible innovation”? It is not just a question of complying with data processing laws and improving the health care professional’s way of working. “We must go further. This is why we want to measure two criteria in new technologies: their environmental impact and their societal impact, distinguishing between the potential positive and negative effects for each. Innovations should then be developed according to predefined criteria aimed at limiting their negative effects,” says Christine Balagué.

Changing the way innovations are designed

Liability is not simply a layer of processing that can be added to an existing technology. Thinking about responsible innovation implies, on the contrary, changing the very manner in which innovations are designed. So how do we ensure they are responsible? Scientists are looking for precise indicators that could result in a “to do list” of criteria to be verified. This starts with the analysis of the data used for learning, but also by studying the interface developed for the users, through the architecture of the neural network that can potentially generate bias. On the other hand, existing environmental criteria must be refined by taking into account the design chain of a connected object and the energy consumption of the algorithms. “The criteria identified could be integrated into corporate social responsibility in order to measure changes over time,” says Christine Balagué.

In the framework of the Good In Tech chair, several research projects, including a thesis, are being carried out on our capacity to explain algorithms. Among them, Christine Balagué and Nesma Houmani (a researcher at Télécom SudParis) are interested in algorithms for electroencephalography (EEG) analysis. Their objective is to ensure that the tools use interfaces that can be explained to health care professionals, the future users of the system. “Our interviews show that explaining how an algorithm works to users is often something that designers aren’t interested in, and that making it explicit would be a source of change in the decision-making process,” says the researcher. The ability to explain and interpret results are therefore two key words guiding responsible innovation.

Ultimately, the researchers have identified four principles that an innovation in healthcare must follow. The first is anticipation in order to measure the potential benefits and risks upstream of the development phase. Then, a reflexive approach allows the designer to limit the negative effects and to integrate into the system itself an interface to explain how the technological innovation works to physicians. It must also be inclusive, i.e. reaching all patients throughout the territory. Finally, responsive innovation facilitates rapid adaptation to the changing context of healthcare systems. Christine Balagué concludes: “Our work shows that taking into account ethical criteria does not reduce the performance of algorithms. On the contrary, taking into account issues of responsibility helps to promote the acceptance of an innovation on the market”.

[1] The Chair is supported by the Institut Mines-Télécom Business School, the School of Management and Innovation at Sciences Po, and the Fondation du Risque, in partnership with Télécom Paris and Télécom SudParis.

Anaïs Culot

Also read on I’MTech :

Healthcare: what makes some connected objects a success and others a flop?
Eclairer boites noires, algorithms

Shedding some light on black box algorithms

/0 Comments/in , /by

In recent decades, algorithms have become increasingly complex, particularly through the introduction of deep learning architectures. This has gone hand in hand with increasing difficulty in explaining their internal functioning, which has become an important issue, both legally and socially. Winston Maxwell, legal researcher, and Florence d’Alché-Buc, researcher in machine learning, who both work for Télécom Paris, describe the current challenges involved in the explainability of algorithms.

What skills are required to tackle the problem of algorithm explainability?

Winston Maxwell: In order to know how to explain algorithms, we must draw on different disciplines. Our multi-disciplinary team, AI Operational Ethics, focuses not only on mathematical, statistical and computational aspects, but also on sociological, economic and legal aspects. For example, we are working on an explainability system for image recognition algorithms used, among other things, for facial recognition in airports. Our work therefore encompasses these different disciplines.

Why are algorithms often difficult to understand?

Florence d’Alché-Buc: Initially, artificial intelligence used mainly symbolic approaches, i.e., it simulated the logic of human reasoning. Logical rules, called expert systems, allowed artificial intelligence to make a decision by exploiting observed facts. This symbolic framework made AI more easily explainable. Since the early 1990s, AI has increasingly relied on statistical learning, such as decision trees or neural networks, as these structures allow for better performance, learning flexibility and robustness.

This type of learning is based on statistical regularities and it is the machine that establishes the rules which allow their exploitation. The human provides input functions and an expected output, and the rest is determined by the machine. A neural network is a composition of functions. Even if we can understand the functions that compose it, their accumulation quickly becomes complex. So a black box is then created, in which it is difficult to know what the machine is calculating.

How can artificial intelligence be made more explainable?

FAB: Current research focuses on two main approaches. There is explainability by design where, for any new constitution of an algorithm, explanatory output functions are implemented which make it possible to progressively describe the steps carried out by the neural network. However, this is costly and impacts the performance of the algorithm, which is why it is not yet very widespread. In general, and this is the other approach, when an existing algorithm needs to be explained, it is an a posteriori approach that is taken, i.e., after an AI has established its calculation functions, we will try to dissect the different stages of its reasoning. For this there are several methods, which generally seek to break the entire complex model down into a set of local models that are less complicated to deal with individually.

Why do algorithms need to be explained?

WM: There are two main reasons why the law stipulates that there is a need for the explainability of algorithms. Firstly, individuals have the right to understand and to challenge an algorithmic decision. Secondly, it must be guaranteed that a supervisory institution such as the  French Data Protection Authority (CNIL), or a court, can understand the operation of the algorithm, both as a whole and in a particular case, for example to make sure that there is no racial discrimination. There is therefore an individual aspect and an institutional aspect.

Does the format of the explanations need to be adapted to each case?

WM: The formats depend on the entity to which it needs to be explained: for example, some formats will be adapted to regulators such as the CNIL, others to experts and yet others to citizens. In 2015, an experimental service to deploy algorithms that detect possible terrorist activities in case of serious threats was introduced. For this to be properly regulated, an external control of the results must be easy to carry out, and therefore the algorithm must be sufficiently transparent and explainable.

Are there any particular difficulties in providing appropriate explanations?

WM: There are several things to bear in mind. For example, information fatigue: when the same explanation is provided systematically, humans will tend to ignore it. It is therefore important to use varying formats when presenting information. Studies have also shown that humans tend to follow a decision given by an algorithm without questioning it. This can be explained in particular by the fact that humans will consider from the outset that the algorithm is statistically wrong less often than themselves. This is what we call automation bias. This is why we want to provide explanations that allow the human agent to understand and take into consideration the context and the limits of algorithms. It is a real challenge to use algorithms to make humans more informed in their decisions, and not the other way around. Algorithms should be a decision aid, not a substitute for human beings.

What are the obstacles associated with the explainability of AI?

FAB: One aspect to be considered when we want to explain an algorithm is cyber security. We must be wary of the potential exploitation of explanations by hackers. There is therefore a triple balance to be found in the development of algorithms: performance, explainability and security.

Is this also an issue of industrial property protection?

WM: Yes, there is also the aspect of protecting business secrets: some developers may be reluctant to discuss their algorithms for fear of being copied. Another counterpart to this is the manipulation of scores: if individuals understand how a ranking algorithm, such as Google’s, works, then it would be possible for them to manipulate their position in the ranking. Manipulation is an important issue not only for search engines, but also for fraud or cyber-attack detection algorithms.

How do you think AI should evolve?

FAB: There are many issues associated with AI. In the coming decades, we will have to move away from the single objective of algorithm performance to multiple additional objectives such as explainability, but also equitability and reliability. All of these objectives will redefine machine learning. Algorithms have spread rapidly and have enormous effects on the evolution of society, but they are very rarely accompanied by instructions for their use. A set of adapted explanations must go hand in hand with their implementation in order to be able to control their place in society.

By Antonin Counillon

Also read on I’MTech: Restricting algorithms to limit their powers of discrimination