The need for explicability is not new! Charles Peirce, an American philosopher and logician, introduced abductive reasoning, i.e. the search for explanations.
How to explain the decisions made by AI algorithms?
Many methods used in symbolic AI, which are based on the modeling of knowledge with approaches such as logic, symbolic learning, etc., are said to be “explicable in essence”, because the sequence of reasoning, which leads to a decision, is identified. But this is only partially true, because if the problem posed becomes too large, with a large number of logical formulas, very complex decision trees, very numerous association rules, the explanation becomes difficult.
The question of explicability is all the more relevant today that the second paradigm of AI, the statistical approaches to AI, has come back to the forefront in recent years. While symbolic AI is based on rules and reproduces human reasoning, statistical approaches to AI rely on statistical learning methods, in particular artificial neural networks that are trained on large volumes of data. These approaches are part of machine learning including deep learning (DL) — although not the only one. It is very difficult to extract and express the rules of what neural networks do, which start from data.
Deep learning is a sub-branch of machine learning (ML) that mimics the functioning of the human brain during data processing. It allows machines to learn without human supervision. It gives the ability to understand spoken words and what’s behind them, translate, identify objects and make informed decisions.
Despite being a branch of machine learning, DL systems do not have limited learning capabilities like traditional ML algorithms. Instead, DL systems can continually improve their abilities beyond imagination as they are fed larger and more consistent data.
How to explain an AI decision?
First of all, it is necessary to define what to explain, for whom, how and why… The choice of explainability tools or methods depends on the answer given to these questions.
For neural networks, it is possible to answer them at the level of the data used, at the level of the functioning of the network itself or at the level of the result produced.
For the operation, we can ask ourselves if it is necessary to explain. Let’s take the example of aspirin, it was used for a long time without anyone knowing how it worked. And when its functioning was understood, it was used to develop new things, without changing the use that was made of it. In the same way, one can drive a car without understanding the engine, but with a level of knowledge that is sufficient to use it well.
At the level of the final result, the explanation may need to go through intermediate steps to explain it better.
We expect an algorithm to be neutral, but nothing is ever neutral! The doctor triggers an imaging test for his patient because he is looking for something he could identify in that image, he has an intention. This introduces biases, which are not statistical, but cognitive, of framing, confirmation, complacency, etc. We find these same biases when faced with results produced by an algorithm.
Moreover, we should not forget that we trust the algorithm all the more when it shows what we are looking for. Another factor is the cost of the error, because it is very different depending on whether it is detected wrongly or rightly.
Explainability depends on the user and the use of an algorithm
Explanation is a process of conversation, of communication. We adapt the level of explanation according to the person we are talking to. Let’s take an image showing a tumor. The doctor will explain this image and the tumor differently depending on whether he is talking to his staff, to students, to an audience in a conference or to his patient.
We must also ask ourselves why we want to explain. Is it to justify, to control how an algorithm works, to discover scientific knowledge, a phenomenon? The objectives vary and this will require different tools. The stakes also differ, there are issues of trust, ethics, responsibility, and possibly economic issues.
The need for explicability is stronger at the moment
It is essentially due to deep neural networks, which are increasingly used, which have millions of parameters and which are extremely complex. There is a lot of reliance on data with the expectation that increasing the volumes used will help improve the results. That said, there is a lot of domain knowledge that could be used.
This is what hybrid AI proposes to do, which combines several approaches to AI. It combines knowledge and data, symbolic AI and neural networks, logic and learning.
Whatever the approach, the role of the human being remains paramount and it will always be necessary to justify the decisions made by an algorithm.
