RESPONSIBLE AI: THE LEGAL PERSPECTIVE
New technologies, however impressive, will never eliminate the fog, friction, and chaos of war.
Dr. Ibin Yede
Is responsible AI an oxymoron, or whether responsible use of AI is an oxymoron? Technology cannot be held responsible for the follies of humans. It is the use of that technology because any technology can be both good and bad, depending on how the humans using it behave and what their intentions are.
Old Wine in New Bottle?
There are many legal concerns related to responsible uses of AI in the military domain, with many issues resembling the concerns that occur in the commercial use of AI. However, working for the military and being in the business ofkilling people and blowing things up, the risks are different. Also, the applicable legal framework is different. We are primarily guided by international humanitarian law (IHL) that regulates the conduct of facilities, namely the rules for how you can use force, when you can do it, against what targets and under which precautions.
We have moved beyond discussing whether these rules apply at all. That was the starting point of the discussions because we were dealing with very old rules that do not explicitly address AI. Now, everyone agrees that it does, like in the cyber domain; it is all about figuring out how we apply those rules to this new technology. This means that when we create AI-enabled weapon systems, we have to make sure that they can be used in accordance with the key rules of distinction, proportionality, and precautions in attack.
New laws must ensure that both commanders' intent can be followed by the system, which should be capable of correctly identifying lawful targets in accordance with principles of proportionality and precautions. All AI-enabled systems must undergo a legal review before they are fielded, which can get tricky because of the nature of AI-enabled systems and their complexity.
The third effect of IHL being applicable is that a military decision-maker who decides to use an AI-enabled system for a particular attack is responsible for that decision, just like he or she is responsible for deciding to use a hellfire missile fired by a drone or a sniper. So, there is no responsibility gap. The fact that IHL does not specifically mention AI-enabled weapon systems but has rules that are human-centred does not mean that they are per se prohibited or not prohibited; there is no explicit prohibition. The rules that we have currently do not stand in the way of using AI-enabled systems, at least not in all circumstances. Considering the object and purpose of international humanitarian law, which is to lower the suffering of those subjected to armed conflicts, we are perhaps obliged to use technology whenever that can be used to make us more precise and lower collateral damage.
Challenges
There might be a positive obligation to ensure better solutions using AI-enabled technology. But there are three overall challenges.
The first one is the control problem. How do we ensure that systems can be used in accordance with the commander's intent and international humanitarian law?When a human being is not necessarily pulling the trigger, this problem becomes even more difficult considering the combination of AI systems' brittleness and the dynamic nature of modern battlefields. You might train a system to perform well under certain circumstances, but as soon as circumstances on the ground change, the system might have difficulties acting correctly.So, how do commanders and legal advisors ensure we can control the system?
The second overall problem is the black box problem. The complexity of these systems and the fact that machine learning and neural networks per se are not explainable and transparent makes it very difficult for us to understand how the system translates the input into output. From an IHL perspective, that is challenging because we like predictability and reliability, and how can we establish that when we don't understand the parameters for the algorithm's decision-making?
And the third challenge is the accountability challenge. The further away from actually pulling the trigger a person is, the harder it can be to attribute any unlawful acts committed by using an AI-enabled system to an individual. There are several competing regimes. You can have the commander with responsibility, but there might also be production flaws and design flaws that could lead back to product liability. You could have state responsibility. States are always responsible for war crimes committed by them, regardless of the means they're using. There are competing regimes, and identifying the correct one might prove difficult.
New technologies, however impressive, will never eliminate the fog, friction, and chaos of war. Despite all rules and regulations, a specific chain of events can lead to terrible consequences, beginning with system design and development all the way through updating or not updating the model in the field. Different actions at every step along the way can prevent tragedy. The data used should be sufficiently representative of the operational environment.Second, rushing the system into fielding without sufficient testing evaluation. Humans can and do catch a lot of problems, but it becomes much harder, perhaps even impossible in some cases, to do so once a weapon system is automated fully. For AI models, this is an important point. Testing never ends.
The Way Ahead
Probably 98 per cent of commanders in the field right nowdo not fully understand how AI models work. They do not comprehend the AI model's limitations and possible failure modes. They also did not grasp the implications of removing humans from the final targeting decisions. Better training would help in this regard, both for military commanders and, equally important, for civilian policymakers who should be armed with questions to raise so they could press the military for better answers. A lack of technical understanding is not an excuse when bad things happen on the battlefield.
The U.S. military and, indeed, the militaries of every nation want to deploy AI to help them to assure and deter and, should deterrence fail, to prevail in conflict. That is what national defence and security are all about. Everybody wants to gain a technological edge. Still, some aspects of transparency are compatible with national security, such as consensus on principles of AI assurance, a risk framework for AI-enabled military systems and some non–proliferation agreement.
We are entering a new, somewhat uncharted world with AI-enabled systems. If an AI-enabled military system fails, there will be serious consequences not only for the two sides involved but globally as well. It is possible to maintain operational security and not reveal sensitive capabilities while taking steps internationally to ensure AI's safe, lawful, and ethical use.
