Posts

Infographie représentant l'interconnexion entre différents systèmes

Machines surfing the web

/1 Comment/in , /by

There has been constant development in the area of object interconnection via the internet. And this trend is set to continue in years to come. One of the solutions for machines to communicate with each other is the Semantic Web. Here are some explanations of this concept.

 “The Semantic Web gives machines similar web access to that of humans,” indicates Maxime Lefrançois, Artificial Intelligence researcher at Mines Saint-Etienne. This area of the web is currently being used by companies to gather and share information, in particular for users. It makes it possible to adapt product offers to consumer profiles, for example. At present, the Semantic Web occupies an important position in research undertaken around the Internet of Things, i.e. the interconnection between machines and objects connected via the internet.

By making machines work together, the Internet of Things can be a means of developing new applications. This would serve both individuals and professional sectors, such as intelligent buildings or digital agriculture. The last two examples are also the subject of the CoSWoT1 project, funded by the French National Research Agency (ANR). This initiative, in which Maxime Lefrançois is participating, aims to provide new knowledge around the use of the Semantic Web by devices.

To do so, the projects’ researchers installed sensors and actuators in the INSA Lyon buildings on the LyonTech-la Doua campus, the Espace Fauriel building of Mines Saint-Etienne, and the INRAE experimental farm in Montoldre. These sensors record information, like the opening of a window or the temperature and CO2 levels in a room. Thanks to a digital representation of a building or block, scientists can construct applications that use the information provided by sensors, enrich it and make decisions that are transmitted to actuators.

Such applications can measure the CO2 concentration in a room, and according to a pre-set threshold, open the windows automatically for fresh air. This could be useful in the current pandemic context, to reduce the viral load in the air and thereby reduce the risk of infection. Beyond the pandemic, the same sensors and actuators can be used in other cases for other purposes, such as to prevent the build-up of pollutants in indoor air.

A dialog with cards

The main characteristic of the Semantic Web is that it registers information in knowledge graphs: kinds of maps made up of nodes representing objects, machines or concepts, and arcs that connect them to one another, representing their relationships. Each hub and arc is registered with an Internationalized Resource Identifier (IRI): a code that makes it possible for machines to recognize each other and identify and control objects such as a window, or concepts such as temperature.

Depending on the number of knowledge graphs built up and the amount of information contained, a device will be able to identify objects and items of interest with varying degrees of precision. A graph that recognizes a temperature identifier will indicate, depending on its accuracy, the unit used to measure it. “By combining multiple knowledge graphs, you obtain a graph that is more complete, but also more complex,” declares Lefrançois. “The more complex the graph, the longer it will take for the machine to decrypt,” adds the researcher.

Means to optimize communication

The objective of the CoSWoT project is to simplify dialog between autonomous devices. It is a question of ‘integrating’ the complex processing linked with the Semantic Web into objects with low calculating capabilities and limiting the amount of data exchanged in wireless communication to preserve their batteries. This represents a challenge for Semantic Web research.  “It needs to be possible to integrate and send a small knowledge graph in a tiny amount of data,” explains Lefrançois. This optimization makes it possible to improve the speed of data exchanges and related decision-making, as well as to contribute greater energy efficiency.

With this in mind, the researcher is interested in what he calls ‘semantic interoperability’, with the aim of “ensuring that all kinds of machines understand the content of messages that they exchange,” he states. Typically, a connected window produced by one company must be able to be understood by a CO2 sensor developed by another company, which itself must be understood by the connected window. There are two approaches to achieve this objective. “The first is that machines use the same dictionary to understand their messages,” specifies Lefrançois, “The second involves ensuring that machines solve a sort of treasure hunt to find how to understand the messages that they receive,” he continues. In this way, devices are not limited by language.

IRIs in service of language

Furthermore, solving these treasure hunts is allowed by IRIs and the use of the web. “When a machine receives an IRI, it does not need to automatically know how to use it,” declares Lefrancois. “If it receives an IRI that it does not know how to use, it can find information on the Semantic Web to learn how,” he adds. This is analogous to how humans may search for expressions that they do not understand online, or learn how to say a word in a foreign language that they do not know.

However, for now, there are compatibility problems between various devices, due precisely to the fact that they are designed by different manufacturers. “In the medium term, the CoSWoT project could influence the standardization of device communication protocols, in order to ensure compatibility between machines produced by different manufacturers,” the researcher considers. It will be a necessary stage in the widespread roll-out of connected objects in our everyday lives and in companies.

While research firms are fighting to best estimate the position that the Internet of Things will hold in the future, all agree that the world market for this sector will represent hundreds of billions of dollars in five years’ time. As for the number of objects connected to the internet, there could be as many as 20 to 30 billion by 2030, i.e. far more than the number of humans. And with the objects likely to use the internet more than us, optimizing their traffic is clearly a key challenge.

[1] The CoSWoT project is a collaboration between the LIMOS laboratory (UMR CNRS 6158 which includes Mines Saint-Étienne), LIRIS (UMR CNRS 5205), Hubert Curien laboratory (UMR CNRS 5516) INRAE, and the company Mondeca.

Rémy Fauvel

Read on I’MTech

A dictionary for connected devices
AI

AI for interoperable and autonomous industrial systems

/0 Comments/in , /by

At Mines Saint-Étienne, researchers Olivier Boissier, Maxime Lefrançois and Antoine Zimmermann are using AI to tackle the issue of interoperability, which is essential to the industry of the future. The standardization of information in the form of knowledge graphs has allowed them to enable communication between machines that speak different languages. They then operate this system via a network of autonomous distributed agents on each machine to automate a production line.

Taking a train from France to Spain without interoperability means having to get off at the border since the rails are not the same in both countries. A train that hopes to cross over from one rail line to another is sure to derail. The same problem is posed on factory floors – which is why the interoperability of production lines is a key issue for the industry of the future. In an interoperable system, machines can communicate with one another in order to work together automatically, even if they don’t speak the same language. But this is not easy to implement. Factory floors are marked by a kind of cacophony of computer languages. And every machine has its own properties: a multitude of manufacturers, different applications, diverse ways of sending, measuring and collecting information etc. Such heterogeneity reduces the flexibility of production lines. During the Covid-19 crisis, for example, many companies had to reconfigure all of their machines by hand to set up new production operations, such as manufacturing masks. “As of now, on factory floors everything is coded according to an ideal world. Systems are incapable of adapting to change,” says Maxime Lefrançois, a specialist in web semantics. Interoperability also goes hand in hand with competition. Without it, ensuring that a factory runs smoothly would require investing in a single brand of equipment to be certain the various parts are compatible.  

There is no single method for making a system interoperable. Along with his colleagues at Mines Saint-Étienne, the researcher is addressing the issue of interoperability using an approach based on representing data about the machines (manufacturer, connection method, application, physical environment etc.) in a standardized way, meaning independent of the language inherent to a machine. This knowledge is then used by what is known as a multi-agent software system. The goal is to automate a production process based on the description of each machine.

Describing machines to automate decision-making

What does the automation of an industrial system imply? Service delegation, primarily. For example, allowing a machine to place an order for raw materials when it detects a low stock level, instead of going through a human operator. For this, the researchers are developing mechanisms for accessing and exchanging information between machines using the web of things. “On the web, we can set up a communication interface between the various devices via standardized protocols. These methods of interaction therefore reduce the heterogeneity of the language of connected devices,” explains Antoine Zimmermann, an expert in knowledge representation at Mines Saint-Étienne. All of the modeled data from the factory floor is therefore accessible to and understood by all the machines involved.

More importantly, these resources may then be used to allow the machines to cooperate with one another. To this end, the Mines Saint-Étienne team has opted for a flexible approach with local decision-making. In other words, an information system called an autonomous agent is deployed on each device and is able to interact with the agents on other machines. This results in a 4.0 word-of mouth system without loss of information. “An autonomous agent decides what to do based on what the machines upstream and downstream of its position are doing. This reasoning software layer allows the connected device to adjust its behavior according to current status of the system,” says Olivier Boissier, who specializes in autonomous agent systems at Mines Saint-Étienne. For example, a machine can stop a potentially dangerous process when it detects information indicating that a device’s temperature is too high. Likewise, it would no longer be necessary to redesign the entire system to add a component, since it is automatically detected by the other machines.

Read more on I’MTech: A dictionary for connected devices

Depending on the circumstances of the factory floor, a machine may also connect to different production lines to perform other tasks. “We no longer code a machine’s specific action, but the objective it must achieve. The actions are deduced by each agent using the data it collects. It therefore contributes to fulfilling a general mission,” adds the researcher. In this approach, no single agent can achieve this objective alone as each one has a range of action limited to its machine and possesses only part of the knowledge about the overall line. The key to success it therefore cooperation. This makes it possible to transition from producing cups to bottles, simply by changing the objective of the line, without reprogramming it from A to Z.

Towards industrial experiments

Last summer, the IT’m Factory technological platform, a simulated industrial space at Mines Saint-Étienne, hosted a case study for an interoperable and cooperative distributed system. This production line starts out with a first machine responsible for retrieving a cup in a storage area and placing it on a conveyor. A filling system then fills the cup with a liquid. When this second machine has run out of product to pour, it places a remote order with a supplier. At every step, several methods of cooperation are possible. The first is to send a message from one agent to another in order to notify it of the task it has just performed. A second method uses machine perception to detect the action performed by the previous machine. A certain method may be preferable depending on the objectives (production speed etc.).

The researchers have also shown that a robot in the middle of the line may be replaced by another. Interoperability made it possible for the line to adapt to hardware changes without impacting its production. This issue of flexibility is extremely important with a view towards integrating a new generation of nomadic robots. “In September 2020, we start the SIRAM industry of the future project, which should make it possible to deploy interoperable, adaptable information systems to control mobile robotic assistants,” says Maxime Lefrançois. In the future, these devices could be positioned at strategic locations in companies to assist humans or retrieve components at different parts of the production line. But to do so, they must be able to interact with the other machines on the factory floor.  

Anaïs Culot