Explainable AI

Control

To move from proof of concept to fully-fledged implementation, you need to be confident that your systems satisfies certain intended requirements, and that they do not have any unwanted behaviours. If the system makes a mistake, organisations need to be able to identify that something is going wrong in order to take corrective action or even to shut down the AI system. XAI can help your organisation retain control over AI by monitoring performance, flagging errors and providing a mechanism to turn the system off. From a data privacy point of view, XAI can help to ensure only permitted data is being used, for an agreed purpose, and make it possible to delete data if required.

Safety

There have been several concerns around safety and security of AI systems, especially as they become more powerful and widespread. This can be traced back from to range of factors including deliberately unethical design, engineering oversights, hacking and the effect of the environment the AI operates in. XAI can help to identify these kinds of faults. It’s also important to work closely with cyber detection and protection teams to guard against hacking and deliberate manipulation of learning and reward systems.

Trust

Building trust in artificial intelligence means providing proof to a wide array of stakeholders that the algorithms are making the correct decisions for the right reasons. Explainable algorithms can provide this up to a point, but even with state of the art machine learning evaluation methods and highly interpretable model architectures, the context problem persists: AI is trained on historical datasets which reflect certain implicit assumptions about the way the world works. By gaining an intuitive understanding of a model’s behaviour, the individuals responsible for the model can spot when the model is likely to fail and take the appropriate action. XAI also helps to build trust by strengthening the stability, predictability and repeatability of interpretable models. When stakeholders see a stable set of results, this helps to strengthen confidence over time.

Ethics

It’s important that a moral compass is built into the AI training from the outset and AI behaviour is closely monitored thereafter through XAI evaluation and where appropriate, a formal mechanism that aligns a company’s technology design and development with its ethical values and principles and risk appetite may be necessary.

Accountability

It’s important to be clear who is accountable for an AI system’s decisions. This in turn demands a clear XAI-enabled understanding of how the system operates, how it makes decisions or recommendations, and how it learns and evolves over time and how to ensure it functions as intended. To assign responsibility for an adverse event caused by AI, a chain of causality from the AI agent back to the person or organisation needs to be established that can be reasonably held responsible for its actions. Depending on the nature of the adverse event, responsibility will sit with different actors within the causal chain that lead to the problem.

Regulation

While AI is lightly regulated at present, this is likely to change as its impact on everyday lives becomes more pervasive. Regulatory bodies and standard institutions are focusing on a number of AI-related areas, with the establishment of standards for governance, accuracy, transparency and explainability being high on the agenda. Further regulatory priorities include safeguarding potentially vulnerable consumers.