The nexus of robotics and autonomous systems (RAS) and artificial intelligence (AI) has the potential to change the nature of warfare. RAS offers the possibility of a wide range of platforms—not just weapon systems—that can perform "dull, dangerous, and dirty" tasks—potentially reducing the risks to soldiers and Marines and possibly resulting in a generation of less expensive ground systems. Other nations, notably peer competitors Russia and China, are aggressively pursuing RAS and AI for a variety of military uses, raising considerations about the U.S. military's response—to include lethal autonomous weapons systems (LAWS)—that could be used against U.S. forces.
The adoption of RAS and AI by U.S. ground forces carries with it a number of possible implications, including potentially improved performance and reduced risk to soldiers and Marines; potential new force designs; better institutional support to combat forces; potential new operational concepts; and possible new models for recruiting and retaining soldiers and Marines. The Army and Marines have developed and are executing RAS and AI strategies that articulate near-, mid-, and long-term priorities. Both services have a number of RAS and AI efforts underway and are cooperating in a number of areas.
A fully manned, capable, and well-trained workforce is a key component of military readiness. The integration of RAS and AI into military units raises a number of personnel-related issues that may be of interest to Congress, including unit manning changes, recruiting and retention of those with advanced technical skills, training, and career paths.
RAS and AI are anticipated to be incorporated into a variety of military applications, ranging from logistics and maintenance, personnel management, intelligence, and planning to name but a few. In this regard, most consider it unlikely that appreciable legal and ethical objections to their use by the military will be raised. The most provocative question concerning the military application of RAS and AI being actively debated by academics, legal scholars, policymakers, and military officials is that of "killer robots" (i.e., should autonomous robotic weapon systems be permitted to take human life?).
Potential issues for Congress include the following:
The nexus of robotics and autonomous systems (RAS) and artificial intelligence (AI) has the potential to change the nature of warfare. RAS offers the possibility of a wide range of platforms—not just weapon systems—that can perform "dull, dangerous, and dirty" tasks—potentially reducing the risks to soldiers and Marines. Regarding AI, one report suggests
One of the promises of AI in the military that seems to guarantee its adoption is its broad applicability. AI can be used to increase effectiveness and efficiency for more than just combat operations. AI can improve supply lines, enhance the training of new soldiers, and increase the effectiveness and efficiency of intelligence gathering and processing. But their effectiveness in combat operations seems especially promising. AI is not a wholly revolutionary idea to be applied to the military domain, and it is merely the next logical step in the digitization and mechanization of the modern battlefield.1
As a stated imperative in the National Defense Strategy, the Department of Defense (DOD) and the Services are pursuing RAS and AI for a wide variety of applications.2 Aside from the programmatic and budgetary considerations for Congress, another key aspect of these technologies that may merit consideration by Congress is articulated in the following passage from a U.S. Air Force document:
Authorizing a machine to make lethal combat decisions is contingent on political and military leaders resolving legal and ethical questions. These include the appropriateness of machines having this ability, under what circumstances should it be employed, where responsibility for mistakes lies, and what limitations should be placed on the autonomy of such systems.... Ethical discussions and policy decisions must take place in the near term in order to guide the development of future [unmanned aircraft systems] capabilities, rather than allowing the development to take its own path apart from this critical guidance.3
Apart from the U.S. military's pursuit of RAS and AI, there has been a proliferation of RAS and AI internationally, ranging from foreign militaries, to violent nonstate groups to criminal organizations. These technologies have already been used to a limited degree against U.S. military forces. How the United States will address further foreign advances in these realms may be of interest to Congress.
There are a variety of definitions for the following terms and, for the purposes of this report, the following definitions will be used:
The level of independence that humans grant a system to execute a given task. It is the condition or quality of being self-governing to achieve an assigned task based on the system's own situational awareness (integrated sensing, perceiving, analyzing), planning, and decisionmaking. Autonomy refers to a spectrum of automation in which independent decisionmaking can be tailored for a specific mission, level of risk, and degree of human-machine teaming.4
A powered machine capable of executing a set of actions by direct human control, computer control, or both. It is composed minimally of a platform, software, and a power source.5
RAS is an accepted term in academia and the science and technology (S&T) community and highlights the physical (robotic) and cognitive (autonomous) aspects of these systems. For the purposes of this concept, RAS is a framework to describe systems with a robotic element, an autonomous element, or more commonly, both.6
The capability of a computer system to perform tasks that normally require human intelligence such as visual perception, speech recognition, and decisionmaking.7
In the 115th Congress, multiple bills included definitions for AI and incorporated an often-cited classification scheme that categorizes AI systems as designed to think rationally, act rationally, think like humans, or act like humans.8 These classifications were broadly incorporated into the first definition of AI in statute, included in the John S. McCain National Defense Authorization Act for Fiscal Year 2019 (P.L. 115-232), which states that the term AI includes
(1) Any artificial system that performs tasks under varying and unpredictable circumstances without significant human oversight, or that can learn from experience and improve performance when exposed to data sets.
(2) An artificial system developed in computer software, physical hardware, or other context that solves tasks requiring human-like perception, cognition, planning, learning, communication, or physical action.
(3) An artificial system designed to think or act like a human, including cognitive architectures and neural networks.
(4) A set of techniques, including machine learning that is designed to approximate a cognitive task.
(5) An artificial system designed to act rationally, including an intelligent software agent or embodied robot that achieves goals using perception, planning, reasoning, learning, communicating, decision-making, and acting.
While not specified in these definitions, a distinction between narrow and general AI is important when discussing the current and future abilities of AI systems. The term "narrow AI" describes technologies tailored to particular, narrowly defined tasks; the AI systems in use today fall within this category. While narrow AI systems can exceed human capabilities in their specific task set, they cannot understand context or apply what the systems have learned to related tasks. In contrast, "general AI" refers to systems that demonstrate intelligent behavior across a range of cognitive tasks, which is unlikely to occur for decades or longer, according to most analysts.9
Machine learning is an application of AI that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. It focuses on the development of computer programs that can access data and use it to learn for themselves. The process of learning begins with observations or data, such as examples, direct experience, or instruction, in order to look for patterns in data and make better decisions in the future based on the examples that are provided by humans. The primary aim is to allow the computers to learn automatically without human intervention or assistance and adjust actions accordingly.10
A weapon system that, once activated, can select and engage targets without further intervention by a human operator. This includes human-supervised autonomous weapon systems that are designed to allow human operators to override operation of the weapon system, but can select and engage targets without further human input after activation.11
Much has been written about how RAS and AI have affected society and, in particular, the workplace. A comprehensive 2017 study by the International Bar Association's (IBA) Global Employment Institute offers some interesting insights on both society and the workplace:
Modern information technologies and the advent of machines powered by artificial intelligence (AI) have already strongly influenced the world of work in the 21st century. Computers, algorithms and software simplify everyday tasks, and it is impossible to imagine how most of our life could be managed without them. However, is it also impossible to imagine how most process steps could be managed without human force? The information economy characterized by exponential growth replaces the mass production industry based on economy of scales.
When we transfer the experience of the past to the future, disturbing questions arise: what will the future world of work look like and how long will it take to get there? Will the future world of work be a world where humans spend less time earning their livelihood? Alternatively, are mass unemployment, mass poverty and social distortions also a possible scenario for the new world, a world where robots, intelligent systems and algorithms play an increasingly central role? What is the future role of a legal framework that is mainly based on a 20th century industry setting? What is already clear and certain is that new technical developments will have a fundamental impact on the global labor market within the next few years, not just on industrial jobs but on the core of human tasks in the service sector that are considered 'untouchable.' Economic structures, working relationships, job profiles and well-established working time and remuneration models will undergo major changes.
In addition to companies, employees and societies, education systems and legislators are also facing the task of meeting the new challenges resulting from constantly advancing technology. Legislators are already lagging behind and the gap between reality and legal framework is growing.13
The study further suggests that because of RAS and AI, society has entered a "Fourth Industrial Revolution,"14 described as
[t]he technical integration of cyber physical systems (CPS) into production and logistics and the use of the 'internet of things'(connection between everyday objects) and services in (industrial) processes—including the consequences for a new creation of value, business models as well as downstream services and work organization. CPS refers to the network connections between humans, machines, products, objects and ICT (information and communication technology) systems. Within the next five years, it is expected that over 50 billion connected machines will exist throughout the world. The introduction of AI in the service sector distinguishes the fourth industrial revolution from the third.15
The analysis also provides examples of Fourth Industrial Revolution robotics and artificial intelligence:
Well-known examples from the field of robotics and AI are the so-called 'smart factories', driverless cars, delivery drones or 3D printers, which, based on an individual template, can produce highly complex things without changes in the production process or human action in any form being necessary.
Well-known service models are, for example, networking platforms like Facebook or Amazon Mechanical Turk, the economy-on-demand providers Uber and Airbnb, or sharing services, such as car sharing, Spotify and Netflix. Studies show that merely due to sharing services the turnover of the sector will grow twentyfold within the next ten years.16
If, as some suggest, society is in a "Fourth Industrial Revolution" what are the implications for the U.S. military as a whole and, in particular, U.S. ground forces, namely the Army and Marine Corps?
The U.S. 2018 National Defense Strategy of the United States of America, in describing DOD's strategic approach,17 states
The Department will invest broadly in military application of autonomy, artificial intelligence, and machine learning, including rapid application of commercial breakthroughs, to gain competitive military advantages.18
In this regard, the Army and Marines are directed to pursue RAS and AI in support of the National Defense Strategy, but there are also more practical reasons why the Army and Marines might emphasize the development of RAS and AI. Some of these reasons include the following:
Since 2001, the U.S. military—the Army and Marine Corps in particular—has focused on counterinsurgency and counterterrorism operations, with modernization for traditional ground combat receiving less emphasis. The 2018 National Defense Strategy of the United States of America changed the military's focus from counterinsurgency and counterterrorism, noting
The central challenge to U.S. prosperity and security is the reemergence of long-term, strategic competition by what the National Security Strategy classifies as revisionist powers. It is increasingly clear that China and Russia want to shape a world consistent with their authoritarian model—gaining veto authority over other nations' economic, diplomatic, and security decisions.19
This change of strategic focus toward great power competition has prompted a renewed emphasis on preparing for conventional ground combat both in training and modernization, which may be contributing to a greater focus by the Army on RAS and AI. The Army's June 2018 Vision statement notes
This modernization includes experimenting with and developing autonomous systems, artificial intelligence, and robotics to make our Soldiers more effective and our units less logistically dependent.20
Revisionist powers and smaller states are also modernizing and seeking these technologies as well. One defense expert suggests
The robotics revolution isn't American-made. It isn't even American-led. Countries around the world are pushing the envelope in autonomy, many further and faster than the United States. Conversations in U.S. research labs and the Pentagon's E-ring are only one factor influencing the future of autonomous weapons. Other nations get a vote too. What they do will influence how the technology develops, proliferates, and how other nations—including the United States—react.21
As Secretary of the Army Mark Esper reportedly noted, "Whoever gets to robotics and AI first, it'll be a game changer on the battlefield."22 In this regard, the stage appears set for nations to aggressively pursue RAS and AI over the near- and long-term to achieve a battlefield advantage.
RAS and AI have been described as changing the very nature of work and workforce design, with some experts predicting an acceleration of this trend over the next two decades.23 RAS and AI advances in the private sector in areas such as transportation, logistics, manufacturing, health care, and engineering could be readily adapted by the military and ground forces. A potential added incentive is the adoption of these technologies would likely face little international opposition, as these types of technologies do not readily fall into the category of autonomous weapons.
Regarding the civilian labor market, researchers from industry, government, and academia have conducted numerous studies and surveys attempting to predict the impact of AI and automation on the U.S. and international workforce.24 While the reports vary in the many ways—including the populations studied, the timeframe for projected impacts, predicted numbers of jobs lost or gained, and whether the study looks at whole jobs or skills/tasks—there are some overarching takeaways. First, impacts are very difficult to predict, even for experts working in AI and automation. For example, in a 2014 survey of expert "technology builders and analysts" by Pew Research Center, 48% of respondents predicted that AI and robots would displace more jobs than they created by 2025, while the remaining 52% predicted that more jobs would be created than displaced.25 Second, the range of methodologies used in such workforce reports makes comparing studies challenging, thereby adding to the difficulty in projecting workforce impacts. Third, the studies raise additional questions that may have implications for both civilian and military workers. These include, but are not limited to, the following:
These civilian workforce issues could have implications for military organizations. As RAS and AI have changed the global civilian labor market, some believe they will eventually affect military personnel management models.28 They contend that
new technologies will permit the automation of many tasks currently performed by soldiers. As automation and AI allow civilian business leaders to place humans in different kinds of work, so too will military personnel planners be forced to think anew about the recruiting and employment opportunities of a new global workforce approach. It is likely to drive the creation of new military personnel models and in turn the designing of new ground force structures. This, along with the disruptive technologies of robotics, AI, and human augmentation could enable new operating concepts.29
Fewer soldiers and Marines could have a direct impact on the size and allocation of the defense budget, not just in military compensation, but also in military logistics, construction, and health care, for example.
Advances in technology, sensors, computers, and networked communications have served to cut through a large portion of the "fog of war" that military planners and commanders have to contend with by providing a vast array and amount of data, including real-time data. One study observes:
The number of images and signal intercepts are well beyond the capacity of the existing analyst community, so there are huge backlogs for translators and image interpreters, and much of the collected data are never reviewed.30
The dilemma facing human analysts is further characterized:
Today's analysts also face a wide variety of data streaming in from different platforms and sensors—data they must integrate (or fuse) to ensure accurate, comprehensive situational awareness. Their workstations comprise multiple screens, each showing different streams of data and each loaded with different suites of tools. In many cases, the applications, databases, and operating systems underlying these tools are produced by different vendors and are not interoperable. Sailors told us they are overwhelmed as they struggle to master the functions provided by each tool in the suite at their workstations. Another challenge is the existence of multiple and often mutually exclusive security domains (different classification levels).31
Automated systems and AI can be of significant value in assisting military analysts, planners, and commanders in processing and synthesizing large and diverse data sets.
The availability of significant quantities and types of data and the required speed of action often required to address time-sensitive military threats presents a challenge for military decisionmakers:
Processing speed and communication capabilities also increasingly tilt the equation against human decision makers, both generally and especially in military situations. Humans now process at about one-millionth the speed of machines. Machines are becoming faster. Humans aren't. When instant response is imperative, even our Defense Department's proponents of humans in the loop concede that their desired human control cannot be achieved. It can be anticipated that this exception will allow the rule as the range of tasks that can be accomplished by machines grows, machine speeds increase (both in calculation and in kinetic operations), and autonomous operations proliferate. As two observers conclude, "military superpowers in the next century will have superior autonomous capabilities, or they will not be superpowers."32
Automated systems and AI can also be of great value in dealing with military situations where a "manual" or human-in-the-loop response is insufficient, such as dealing with "swarms" of unmanned ground or aerial vehicles or an inbound hypersonic weapon.
Although the future is unpredictable, it is a reasonable assumption that selected RAS and AI advances in the private sector such as logistics, data analysis, and education and training will be adopted by militaries to enhance their institutional and operational effectiveness. Some of the potential implications of RAS and AI for U.S. ground forces include the following:
RAS and AI have the potential to improve both the individual performance of troops as well as the performance of virtually every type of unit. RAS has applicability in lightening soldiers' and Marines' individual combat loads, improving situational awareness at the squad and platoon levels, and serving as "teammates" rather than simply tools.34 AI can be employed as a planning and decision support tool and could be a central component in automated weapon systems (AWS), which can provide protection from incoming aircraft, missiles, rockets, artillery and mortar shells, and other threats.
Some believe RAS and AI also show great promise in reducing physical risks to soldiers and Marines. RAS and AI can be used in such missions as explosive ordnance disposal (EOD), route clearance, obstacle breaching, and chemical, biological, radiological, and nuclear (CBRN) reconnaissance—all considered extremely high-risk operations—in a manner that significantly limits troop exposure to these hazards. As previously noted, RAS and AI are expected to play a greater role in force protection, particularly from the risk of aircraft, missile, rocket, and artillery and mortar attack.
One report suggests that
a highly capable and sustainable land combat battlegroup in 2030 may consist of as few as 250–300 human soldiers and several thousand robotic systems of various sizes and functions. By the same token, many functions of artillery and combat engineer units, currently undertaken by humans, might be better done by robots in human-robot teams. This has the potential to reduce the size of these types of units by hundreds of combat arms personnel. This approach could free up personnel for redeployment into areas where the art of war demands leadership and creativity-enabling intelligence functions; training and education; planning; and, most importantly, command and leadership.35
In some cases, RAS- and AI-inspired force redesigns could not only be revolutionary but controversial as well. The Army, for example, plans to replace the current M-2 Bradley-series infantry fighting vehicle with an Optionally-Manned Fighting Vehicle (OMFV), which can also be operated remotely instead of by a crew.36 Remotely transporting an infantry unit in an OMFV could give rise to concerns that should the remote control capability fail or be disrupted, the vehicle's occupants would be unduly vulnerable to enemy fire.
From an institutional perspective, RAS and AI have a wide range of applicability in training and educating troops and leaders which, in addition to improved efficiency, could result in both cost savings as well freeing up personnel previously dedicated to these tasks for other assignments. From an institutional support perspective, RAS and AI developed for private sector use can most likely be readily adapted for military use. Military warehouse and depot functions are likely prime candidates for RAS and AI applications, potentially increasing efficiency, reducing costs, and freeing up personnel for other endeavors. Efforts are also underway for the partial automation of both ground and air logistics so that unmanned systems can be used to deliver supplies and evacuate casualties. Another potential application being explored by the Army is using AI to predict when vehicle parts might breakdown and prevent equipment failures before they happen.37 RAS and AI also have potential applications in treatment of wounded soldiers in combat zones and in rear areas as well.
RAS and AI offer the possibility of new operational concepts for ground forces. One potential concept would be to "saturate an operational area with small autonomous systems that force an adversary to move, be detected, and be targeted by friendly forces."38 Another possible operational concept to mitigate the effects of enemy anti-access/area denial (A2/AD)39 capabilities during forced entry operations (such as an airborne assault or amphibious landing) could be to employ autonomous air, ground, and naval systems to attack A2/AD systems prior to the introduction of U.S. ground forces. As a corollary to such potential new RAS/AI-enhanced offensive operational concepts, U.S. policymakers and defense officials may also explore what sort of defensive countermeasures and systems might be required should potential U.S. adversaries employ RAS and AI in a similar manner against U.S. and allied forces. As one study suggests, "some of the major platforms and strategies upon which current military forces rely might be rendered obsolete, or at least highly vulnerable" if RAS and AI are employed in combat.40
How soldiers and Marines are recruited, trained and educated, and retained is likely to change as RAS and AI become a more prevalent part of the military. One study notes that
as robots replace humans in many "dirty, dull, and dangerous" functions, it is possible that many lower ranking soldiers may be displaced. This will necessitate a change to the traditional career pyramids, where the mass of the Army is found in the lowest ranks.41
Such a fundamentally different force could have profound impacts on the institutional Army and Marine Corps and could put both services in even greater competition with the private sector for highly skilled and educated recruits. It could also result in fewer opportunities in the U.S. military for those with limited education or those lacking technical skills. The Army cautions, however, that the displacement of lower-ranking soldiers is a "hypothesis" and that many of these soldiers would not be replaced but instead fill new positions to support RAS and AI.42
While it remains to be seen to what extent the Army and Marines adopt RAS and AI for battlefield use, it is reasonable to assume potential adversaries might seek to pursue these capabilities, possibly even to a greater extent than the United States. For example, while the United States may choose not to pursue autonomous weapons based on moral, ethical, and legal considerations, other nations might not feel so obligated and aggressively develop these capabilities for possible use against U.S. forces. In this and other cases, the Army and Marines could be required to develop new systems, tactics, operational concepts, and possibly even units to counter the threat posed by enemy RAS and AI.
Other nations have military RAS and AI aspirations. Recognizing the importance of these technologies, in 2017 the Chinese government reportedly stated its goal of being the world's premier artificial intelligence innovation center by 2030,43 with Russian President Vladimir Putin stating, "Whoever becomes the leader in this sphere [AI] will become ruler of the world."44 One analyst notes
Armed robots are also proliferating on the ground and at sea. South Korea has deployed a robot sentry gun to its border with North Korea. Israel has sent an armed robotic ground vehicle, the Guardium, on patrol near the Gaza border. Russia is building an array of ground combat robots and has plans for a robot tank. Even the Shiite militias in Iraq have gotten in on the game, fielding an armed ground robot in 2015.45
These technologies are not the exclusive purview of nations or paramilitary groups. One report notes in 2017 a criminal group used a swarm of small unmanned aerial vehicles against a FBI hostage rescue team's observation post in an attempt to force them from their hidden position.46
The following sections provide a brief illustrative description of selected non-U.S. RAS and AI efforts. For the selected countries, the efforts discussed are only examples and might not constitute that nation's entire military RAS/AI program.
One study notes, "The Russian Military Industrial Committee has approved a plan that would have 30 percent of Russian combat power consist of entirely remotely controlled and autonomous robotic platforms by 2030."47 At a 2016 military technology forum, Russia reportedly unveiled the Vikhr (Whirlwind) unmanned ground combat vehicle (UCGV) (Figure 1) based on its BMP-3 infantry fighting vehicle (IFV).48
The Vikhr is reportedly armed with a stabilized 30mm Shipunov 2A72 automatic cannon, a coaxial 7.62mm Kalashnikov PKT/PKTM machine gun, and six ready-to-launch 9M133M Kornet-M (AT-14 Spriggan) anti-tank guided missiles (ATGMs).49 The Vikhr can be reconfigured to accommodate a variety of weapons. The 2A72 main gun can be replaced by a single or twin-barrel 23mm 2A14 anti-aircraft cannon, 12.7mm NSVT or Kord heavy machine gun, or a 30mm Gsh-6-30K six-barrel naval automatic cannon. The Vikhr can also accommodate surface-to-air missiles of Igla (SA-18 Grouse) or 9K333 Verba man-portable air defense systems, as well as Shmel-M reactive flame throwers. Foreign artillery systems can also be integrated onto the Vikhr. The Vikhr can also be equipped with four mini unmanned aerial vehicles (UAVs) to provide a surveillance capability. The Vikhr is said to be remotely controlled up to a distance of 10 kilometers.
Russia has also developed the Uran-9 a smaller robotic tank (Figure 2) with a 30mm Shipunov 2A72 automatic cannon, four ready-to-launch 9M120-1 Ataka (Spiral-2) ATGMs, four Igla-V surface-to-air missiles, and a 7.62mm Kalashnikov PKT/PKTM machine gun. The Uran-9 can also mount a Shmel-M reactive flame thrower. The Uran-9 can be remotely controlled up to a distance of 3 kilometers. Russia reported in 2018 that it had tested the Uran-9 in Syria in "near combat conditions" to conduct mine clearing and force protection operations.50
Source: Nikolai Novichkov, "New Russian Combat UGV Breaks Cover, Uran-9 Readies for Service," IHS Jane's International Defence Review, October 2016, p. 30.
Reportedly, the Russian Ministry of Defense is urgently pursuing AI along with the Ministry of Education and Science.51 While it may not currently possess the relevant high-technology culture and funds, Russia is undertaking efforts to organize its academic, scientific, and commercial communities to develop Russian AI and to compete globally.52
One report discusses China's AI aspirations:
People's Liberation Army PLA thinkers expect AI to reshape the character of war itself, from today's "informatized" ways of warfare into "intelligentized" warfare, in which AI is critical. According to Lt. Gen. Liu Guozhi, who leads the Central Military Commission's (CMC) Science and Technology Commission, AI could accelerate military transformation, reshaping military units' programming, operational styles, equipment systems, and models of combat power generation, ultimately leading to a profound military revolution.
He warns, "facing disruptive technology, [we] must ... seize the opportunity to change paradigms. If you don't disrupt, you'll be disrupted!" So the PLA is pursuing intelligent and autonomous unmanned systems; AI-enabled data fusion, information processing, and intelligence analysis; war-gaming, simulation, and training; defense, offense, and command in information warfare; and intelligent support to command decision-making, among other applications. In particular, the CMC Joint Staff Department has called for the PLA to leverage the "tremendous potential" of AI in planning, decision support, and operational command.53
Another report suggests China has already developed a range of unmanned aerial, underwater, surface, and ground platforms and is working on cutting-edge unmanned systems, including those with stealth, swarming, and hypersonic capabilities.54 China believes these modern unmanned systems could be used to introduce a persistent presence in disputed waters or territories.55 These reports suggest China has wide-ranging military applications for RAS and AI in mind, which could present a multifaceted challenge to the U.S. military.
Pentagon officials have reportedly noted that China has made it a national goal to acquire foreign technology—through both licit and illicit means—to advance its military technologies, including AI and unmanned aerial vehicle (UAV) technology.56 Army officials note this presents a number of challenges and limitations when working with U.S. academic institutions to develop RAS and AI, as many of these institutions have significant numbers of foreign nationals enrolled in science and engineering programs.57
The South Koreans have developed the Samsung SGR-A1 robot sentry to defend South Korea against North Korean border intrusion.58 In 2007, it was revealed the SGR-A1 had a fully autonomous mode, and a number of press sources began referring to it as a fully autonomous weapons system, which resulted in a great deal of negative press, although Samsung and South Korean officials noted that a human was required to engage targets. Reportedly, the SGR-A1s, which cost $200,000 apiece, are remotely operated sentries that mount either a 5.5mm machine gun or a 40mm automatic grenade launcher, which work in conjunction with cameras and radar systems that can detect intruders with heat and motion sensors and can challenge them through audio or video communications.59 The SGR-A1 is deployed throughout the 160-mile Korean demilitarized zone (DMZ).
The Brimstone missile developed for the Royal Air Force is an aircraft-launched, fire-and-forget missile designed to destroy ground vehicles or small boats (Figure 3).60 Brimstone has two modes of operation, one that involves a human "painting" a target with a laser and the other "fire and forget" mode where the missile's software seeks a predesignated target type within an established kill box. Brimstone has reportedly been used against Islamic State targets in Syria. Saudi Arabia has acquired the Brimstone missile as well.61 While the United States has a similar system—the Long Range Anti-Ship Missile (LRASM)—Brimstone is an example of an exportable, semi-autonomous weapon system (which could be converted to an autonomous weapon system by adding a loiter capability and switching it to a single fire-and-forget mode) against which U.S. ground forces will likely need to develop countermeasures.
Army and Marine RAS and AI efforts are governed by various policies and managed by a number of different organizations. The following sections provide an overview of selected authorities and managing organizations.
Office of the Secretary of Defense (OSD) Unmanned Systems Integrated Roadmap 2017-2042, June 2018
OSD's Unmanned Systems Integrated Roadmap is to
provide overarching strategic guidance that will align the Services' unmanned systems goals and efforts with the DOD strategic vision. This strategic guidance will focus on reducing duplicative efforts, enabling collaboration, identifying challenges, and outlining major areas where DOD and industry may collaborate to further expand the potential of unmanned systems. As DOD has embraced the use of unmanned systems across nearly every operating environment, this strategy will allow DOD to capitalize on the technology advancements and paradigm shift that unmanned systems provide.62
The overarching themes of OSD's Unmanned Systems Integrated Roadmap include
In addition to autonomous systems, it also addresses cyber operations, information assurance, the electromagnetic spectrum, and electronic warfare.
Joint Concept for Robotic and Autonomous Systems (JCRAS), October 16, 2016
DOD's 2016 JCRAS stipulates that by 2035, the Joint Force will employ integrated human-RAS teams in diverse combinations to expand the Joint Force commander's options.64 Noting that "war will remain a human endeavor with humans retaining responsibility and accountability for military actions" the JCRAS establishes the following future precepts:
Department of Defense (DOD) Directive 3000.09, Change 1, May 8, 2017, Autonomy in Weapons Systems establishes
DOD policy assigns responsibilities for the development and use of autonomous and semi-autonomous functions in weapon systems, including manned and unmanned platforms and establishes guidelines designed to minimize the probability and consequences of failures in autonomous and semi-autonomous weapon systems that could lead to unintended engagements.66
The directive also stipulates that "autonomous and semi-autonomous weapon systems shall be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force," precluding the development of fully autonomous weapons systems.67 This reluctance to pursue fully autonomous weapons systems was further emphasized during 2017 testimony to the Senate Armed Services Committee, when then Vice Chairman of the Joint Chiefs of Staff General Paul Selva stated, "I am an advocate for keeping the restriction, because we take our values to war.... I do not think it is reasonable for us to put robots in charge of whether or not we take a human life."68
In a similar manner, the 2018 Department of Defense Artificial Intelligence Strategy lays out DOD's strategic approach to effectively integrate AI into the Department and maintain military advantage.69
In June 2018, DOD established the Joint Artificial Intelligence Center (JAIC) to accelerate the delivery of AI-enabled capabilities to the Joint Force and synchronize DOD AI activities.70 Reportedly, DOD hopes to attract "world-class" AI talent to the new organization. A decision where to locate the JAIC is pending.71
The Army has a variety of organizations involved in its RAS and AI efforts. In managing these efforts, there are three major organizations involved. At the Army level, the Assistant Secretary of the Army for Acquisitions, Logistics, and Technology (ASA [ALT]) Program Director for Robotics is responsible for RAS. In terms of major command-level management, the Training and Doctrine Command (TRADOC) Capability Manager for RAS exercises management of Army ground and air RAS efforts, as well as overall RAS integration. In July 2018, the Army stood up Army Futures Command (AFC),72 intended to establish unity of command and effort that consolidates the Army's modernization process under one roof. It is not yet established how AFC will manage the Army's RAS and AI efforts, but it is expected that AFC will have a major role in managing these efforts.
On October 2, 2018, the Army announced the creation of the Army Artificial Intelligence Task Force in Support of the Department of Defense Joint Artificial Intelligence Center (A-AI TF). The Army's intent is to "establish a scalable A-AI TF under U.S. Army Futures Command (AFC) consisting of hand-selected Army personnel with specific skill sets to lead Army AI efforts and support DOD projects, principally based at Carnegie Mellon University."74 The A-AI TF will work with Carnegie Mellon's National Robotics Engineering Center in Pittsburg, PA, and the Army expects to achieve an initial operating capability at Carnegie Mellon University in early November 2018.
In a similar manner, a number of organizations are involved in Marine RAS and AI efforts. The Marine Corps Combat Development and Integration Command (CDIC) is responsible for developing RAS and AI concepts of operation and employment. Next, the Marine Corps Warfighting Laboratory is responsible for scientific and technological (S&T) development of RAS and AI. Finally, the Marine Corps Systems Command (MCSC) is responsible for the acquisition of RAS and AI technologies/systems.
In March 2017, the Army published its Robotics and Autonomous Systems Strategy.76 The Army describes its RAS objectives as follows:
1. Increase situational awareness. Complex terrain and enemy countermeasures limit soldiers' abilities to see and fight at the battalion level and below. Advancements in RAS allow for persistent surveillance and reconnaissance over wide areas, often going where manned systems cannot, thereby increasing standoff distances, survivability and reaction time for commanders.
2. Lighten the soldiers' physical and cognitive workloads. Excessive equipment requirements reduce stamina and endurance. Autonomous systems lighten equipment loads and increase soldier speed, mobility, stamina and effectiveness. Vast amounts of information overload leaders' ability to make decisions. RAS facilitate mission command by collecting, organizing, and prioritizing data to facilitate decision-making as well as improving tactical mobility while reducing cyber, electronic, and physical signatures.
3. Sustain the force with increased distribution, throughput, and efficiency. Logistics distribution is resource intensive. Soldiers and teams become vulnerable at the end of extended supply lines. Air and ground unmanned systems and autonomy-based capabilities enhance logistics at every stage of supply movement to the most forward tactical resupply points. RAS move materiel to the most urgent points of need and provide options for Army logistics distribution to the warfighter.
4. Facilitate movement and maneuver. Joint combined arms maneuver in the 21st century requires ready ground combat forces capable of outmaneuvering adversaries physically and cognitively in all domains. Through credible forward presence and resilient battle formations, future ground forces integrate and synchronize joint, interorganizational, and multinational capabilities to create temporary windows of superiority across multiple domains; seize, retain, and exploit the initiative; and achieve military objectives. Investments in Anti-Access/Area Denial (A2/AD) capabilities allows future enemies to engage Army forces earlier and at greater distances. In addition, adversaries will look to emplace obstacles to threaten movement and maneuver across extended avenues of advance. As a counter, Army forces employ RAS to extend the depth of the area of operations and to provide responses to enemy action. RAS expand the time and space at which Army forces can operate and improve the ability to overcome obstacles.
5. Protect the force. The congested and contested future operational environment (OE) increases soldiers' exposure to hazardous situations. RAS technologies will enhance soldiers' survivability by providing greater standoff distance from enemy formations, rockets, artillery, and mortars as well as placing less soldiers at risk during convoy operations.77
While the Army does not have specific strategic objectives for AI like it does RAS, the Army's RAS Strategy does suggest a future role for AI:
Artificial intelligence (AI) is the capability of computer systems to perform tasks that normally require human intelligence such as perception, conversation, and decision-making. Advances in AI are making it possible to cede to machines many tasks long regarded as impossible for machines to perform. AI will play a key role in RAS development as reasoning and learning in computers evolves. AI will improve the ability for RAS to operate independently in tasks such as off-road driving and analyzing and managing mass amounts of data for simplified human decision-making. Increasingly, AI will account for operational factors such as mission parameters, rules of engagement, and detailed terrain analysis. As human-machine collaboration matures, AI will contribute to faster and improved decision-making in five areas: identifying strategic indications and warnings; advancing narratives and countering adversarial propaganda; supporting operational/campaign-level decision-making; enabling leaders to employ "mixed" manned/unmanned formations; and enhancing the conduct of specific defensive missions in which functions of speed, amount of information, and synchronization might overwhelm human decision making.78
Within the context of the Army's RAS Strategy, Army leadership has established near-, mid-, and far-term RAS priorities.
Near-term priorities are partially funded in current budgets and consist of the following:
Mid-term priorities have research and procurement funding lines submitted for the budget under consideration and consist of the following:
Far-term priorities have limited research and development funding programmed in the budget and consist of the following:
The Marines' RAS and AI Strategy is articulated in the Marine Corps Robotic and Autonomy Strategy (MCRAS).82 The MCRAS's stated objectives are to
As part of the Marines' AI strategy, they hope to speed up and improve decisionmaking in the following areas:
In a manner similar to the Army's, the Marines have established near-, mid-, and far-term priorities.
Developing autonomous robotic ground systems that can successfully navigate tactically cross-country is a significant challenge that will need to be overcome before these systems can be employed effectively on the battlefield. One researcher describes the challenge of autonomous ground navigation as well as a brief history of driverless car development as follows:
Autonomous unmanned aerial vehicle (UAV) navigation, for example, is relatively straightforward, since the world model according to which it operates consists simply of maps that indicate preferred routes, height obstacles and no-fly zones. Radars augment this model in real time by indicating which altitudes are clear of obstacles. Global Positioning System (GPS) coordinates convey to the UAV where it needs to go, with the overarching goal of the GPS coordinate plan being not to take the aircraft into a no-fly zone or cause it to collide with an obstacle.
In comparison, navigation for driverless cars is much more difficult. Cars not only need similar mapping abilities, but they must also understand where all nearby vehicles, pedestrians and cyclists are, and where all these are going in the next few seconds. Driverless cars (and some drones) do this through a combination of sensors like LIDAR (Light Detection and Ranging), traditional radars, and stereoscopic computer vision. Thus the world model of a driverless car is much more advanced than that of a typical UAV, reflecting the complexity of the operating environment. A driverless car computer is required to track all the dynamics of all nearby vehicles and obstacles, constantly compute all possible points of intersection, and then estimate how it thinks traffic is going to behave in order to make a decision to act.
Driverless car development originated with a Defense Advanced Research Projects Agency (DARPA) program in 2004. When the program ended in 2007, driverless cars could move only slowly through closed courses, and not without accidents. A decade later, industry is on the verge of commercializing driverless cars around the world. This rapid progress is a result of the significant industry-sponsored Research and Development (R&D) investment, as well as competition for the multi-billion-dollar automotive consumer market. Meanwhile—and paradoxically, given the origins of the technology—there has been very little progress in military autonomous vehicle development.86
Overcoming the challenges of tactical cross-country autonomous navigation (e.g., avoiding obstacles such as barbed wire, minefields, and antitank ditches) and using terrain to shield military vehicles from detection and engagement by direct-fire weapons remains, for the foreseeable future, a crucial developmental challenge for both U.S. and foreign ground forces.
The following sections provide a brief description of selected unclassified Army and Marine Corps RAS and AI efforts.
Source: Todd South, "These Two Army BCTs Will Be the First to Put Robotic Vehicles in Their Formations," Army Times, April 10, 2018.
The SMET is an unmanned robotic vehicle intended to provide logistical support to squads in Army Infantry Brigade Combat Teams (IBCTs) and Marine Infantry Battalions.87 The SMET will be designed to operate in unmanned and optionally manned modes and will be required to carry up to 1,000 lb., operate over 60 miles in 72 hours, and generate 3 kW stationary and 1 kW moving to enable equipment and charge batteries. The target cost for the SMET is no more than $100,000 per system. The SMET is largely a commercial off-the-shelf effort, as a number of vendors had previously developed prototypes for past Army robotic initiatives.
Conceptually, the SMET is intended to carry troops, food and water, ammunition, supplies, and other weapons such as mortars and anti-armor weapons. The SMET is also to be expandable and could conduct route clearance and breaching operations by the addition of special mission modules. Theoretically, the SMET could also be configured to conduct reconnaissance and serve as a semi- or fully autonomous weapon system when armed.
In December 2017, the Army selected four vendors—Team Polaris (Applied Research Associates, Polaris Defense, and Neya Systems), General Dynamics Land Systems, HDT Global, and Howe and Howe Technologies—to conduct a six-month operational technology demonstration in FY2019.88 Sixty-four SMETs (16 from each vendor) are to be made available to the 1st Brigade Combat Team (BCT), 10th Mountain Division, Ft. Drum, NY, and the 2nd BCT, 101st Airborne Division, Ft. Campbell, KY, for evaluation in November 2018. Army plans call for SMET to transition to a Program of Record by first quarter FY2020, after which the Army would transition into Low Rate Initial Production (LRIP) of the SMET. By the second or third quarter of FY2021, the first Army units will begin to receive the SMETs. Depending on budgets, the Army could eventually procure as many as 5,723 SMETs.
Source: Kevin Lilley, "Driverless Army Convoys: 6 Takeaways from the Latest Test," Army Times, July 1, 2016.
The Army's Leader-Follower Technology for Tactical Wheeled Vehicles (TWVs) effort revolves around a suite of sensors and vehicle upgrades intended to provide TWVs the capability of linking three unmanned vehicles to a single manned vehicle during the conduct of logistics road convoy operations.89 This effort is intended to reduce the number of soldiers required to operate a convoy, thereby reducing the number of exposed soldiers to risk of injury from attack.
The Army's Tank Automotive Research, Development and Engineering Center (TARDEC) of Warren, MI, is reportedly working with several industry partners on the effort.90 Robotic Research LLC, of Gaithersburg, MD, is providing the autonomy kit for the vehicles, and Oshkosh Defense of Oshkosh, WI, is building the kits that allow the trucks to be remotely operated. Lockheed Martin of Bethesda, MD, is the integrated systems developer, and DCS Corporation of Alexandria, VA, is creating the graphic user interface allowing the soldier in the lead vehicle to control the follower trucks.
Plans call for a year-long operational technical demonstration in late FY2019 involving deploying 60 systems to two Palletized Load System (PLS) Truck Companies.91 If successful, the Army plans to enter Low-Rate Initial Production in FY2021 and Full-Rate Production in FY2023, with production completed by FY2027.92
Source: Army Briefing, Army Robotic and Autonomous Systems (RAS) Portfolio, April 11, 2018.
The Next Generation Combat Vehicle (NGCV) is intended to replace the M-2 Bradley Infantry Fighting Vehicle, which has been in service since 1981. As part of this effort, the Army plans for the NGCV to have the capability to be "optionally manned"—meaning that soldiers will be onboard the NGCV most of the time, but the vehicle will have the ability to conduct limited operations without them.93 As part of this effort, the Army also plans to develop Robotic Combat Vehicles to serve as "wingmen" for the NGCV. The Army's long-term vision is for RCVs to act as "scouts and escorts" for manned NGCVs, with soldiers controlling the RCVs from stations in the NGCV.94 At present, it takes one soldier to direct a single ground robotic vehicle by remotely controlling every movement and action. The Army hopes to improve AI to the extent that a single soldier can control "a squadron of robots."95
The Army plans to develop an initial set of six experimental NGCV prototypes—two manned NGCVs and four RCVs—for delivery by the end of FY2019.96 In FY2020, the Army plans for hands-on soldier testing with the final effort—testing a company-sized element (14 RCVs)—from FY2023 to FY2024.97
Source: Maddy Longwell, "The Army Hopes AI Can Help Prevent Vehicle Failure," C4ISRNet, June 26, 2018.
In June 2018, the Army reportedly awarded a $1 million yearlong contract to Uptake Technologies, an industrial AI company based in Chicago, IL, to build AI software for the M-2 Bradley intended to "predict component failures, decrease the frequency of unscheduled maintenance, and improve the productivity of repair operations."98 The basic concept is to install sensors inside the Bradley's engine and other components to record information such as temperature and revolutions per minute (RPM) and transmit it to the Uptake software, where machine learning would look for patterns in data that match known engine failures in similar vehicles.99 Reportedly, Uptake will install its software on 32 M-2 Bradleys at Ft. Hood, TX, not only to predict when future repairs might be needed but also to optimize the timing of general maintenance.100 In addition, if the software performs as envisioned, it could also "prevent the Army from doing unnecessary preventative maintenance work, saving time and money."101 If successful, Army officials reportedly could expand this effort to the entire Bradley fleet as well as other combat vehicle fleets. Officials are cautious, noting that these industrial machine-learning technologies have not yet been fully tested on military vehicles.102
Source: Marine Corps Robotic and Autonomy Strategy (MCRAS), June 2018.
The Marine RV (M) is a custom-built, multimission-tracked platform that incorporates AI but also fulfils the need for man-in-the-loop capabilities. It is intended to accommodate a wide variety of mission modules, ranging from route clearance and breaching; logistics; reconnaissance, surveillance, and target acquisition (RSTA); casualty evacuation; and direct- and indirect-fire weapons.
As part of efforts to develop innovative concepts of operation, the Marines are exploring a Fully Autonomous First Wave Concept whereby robotic and autonomous aerial, amphibious, and ground platforms would be employed as the first wave of an amphibious assault to address enemy anti-access/area denial (A2/AD) capabilities. As part of the Marines' Expeditionary Advance Base Operations (EABO), implementation of Fully Autonomous First Wave could result in the need for fewer Marines to participate in high-risk amphibious assault operations, as well as a reduction in the logistical footprint needed to support Marine amphibious operations.
A fully manned, capable and well-trained workforce is a key component of military readiness. The integration of RAS and AI into military units raises a number of personnel-related issues that may be of interest to Congress.
The introduction of RAS and AI will almost certainly lead to significant changes in how units are organized, equipped, and manned. Sometimes this is conceptualized as a need for reduced manpower, as RAS and AI are used to replace personnel and reduce unit manning. For example, the use of leader-follower technology could lead to the ability to provide logistical support to the deployed units with fewer truck drivers. However, from another perspective, manpower savings in one area may be offset by manpower increases in another. Some observers note that the increased use of unmanned aircraft have increased manning requirements rather than reducing them:
Yet the military's growing body of experience shows that autonomous systems don't actually solve any given problem, but merely change its nature. It's called the autonomy paradox: The very systems designed to reduce the need for human operators require more manpower to support them.
Consider unmanned aircraft—which carry cameras aloft without a crew, yet require multiple shifts of operators, maintainers and intelligence analysts on the ground to extract useful data—and it becomes clear that many researchers and policymakers have been asking the wrong question. It is not "what can robots do without us?" but "what can robots do with us?"104
Another consideration revolves around assessments of risks associated with potential failure of RAS and AI systems. National security leaders may want to retain manpower to provide certain capabilities in the event RAS and AI systems are degraded or inadequate for a given mission or requirement.
The introduction of RAS and AI brings with it a greater need for military personnel with advanced technical knowledge. The military has extensive experience bringing new members into the armed forces (recruiting) and convincing those members to stay in the armed forces after their initial term of service has ended (retention). However, for the great majority of individuals currently brought into the armed forces, the highest degree they have is a high school diploma, in the case of enlisted personnel, or a bachelor's degree, in the case of officers.105
As the military integrates RAS and AI into its formations, the need to recruit and retain those with advanced technical training will increase. At least some military personnel will need to have a sophisticated understanding of RAS and AI to assist with the design, acquisition, programming, testing, and quality control of such systems. A more sizable population will need to have fairly extensive knowledge of particular systems to effectively use and maintain them. Recruiting and retaining the types of individuals needed to perform these roles may be challenging. They will likely need to have high cognitive ability and years of advanced technical training, in addition to meeting the physical and behavioral standards for joining the armed forces. In addition, the military will likely face intense competition from the private sector for the individuals who do meet all of these requirements.
As RAS and AI become integrated into military formations, the need to train servicemembers on how to use and maintain such systems will increase. While the scope of training required will vary depending on the types of systems introduced and the work roles of individuals, the time and expenses required to develop appropriate training curricula and to train individuals will likely be significant. Such training will not be a one-time event, but will need to be ongoing to accommodate the fielding of new systems and upgrades to older systems. At present, the armed forces provide most military skill training through an extensive network of military schools and training sites. However, this might not be the optimal way to develop and maintain proficiency in RAS and AI systems, particularly for those whose work roles require current, high-level knowledge.
Most individuals who join the armed forces come in at the lowest enlisted or officer grade (rank) and are gradually promoted to higher grades over the course of their military career. These career paths are well defined, and a close connection is typically maintained between one's grade and technical skill, professional military knowledge, leadership experience, authority, status, and compensation. For most military personnel, the career path takes an individual from a focused specialist in a given skill set to a leader of increasingly larger organizations, which themselves are increasingly complex in terms of personnel, systems, and capabilities. Various levels of professional military education, as well as broadening assignments, are designed to facilitate this development.
However, maintaining the desired supply of individuals with expertise in RAS and AI may require a different career path model—for example, higher entry level grades or a technical career track that requires fewer assignments to leadership positions. In addition, in light of rapid technological change, maintaining expertise in RAS and AI might be enhanced by periodic assignments outside of the military—for example, in the private sector or academia. One career path model that has been cited as potentially applicable to those with advanced technical training is the one which exists in the military medical community.106 In this model, officers are routinely brought in at a higher grade based primarily on their medical skills, with less emphasis placed on developing professional military skills and large scale organizational leadership. However, it is not yet clear that this model would be ideal for RAS and AI experts.
The legal and ethical debate over the development and deployment of autonomous weapons systems largely concerns technology that does not currently exist—mobile robotic weapons systems capable of selecting and attacking targets without human intervention.107 While some argue that such weapons can and should be developed and fielded if they meet the requirements of the international law of armed conflict (LOAC, also known as international humanitarian law, or IHL) for lawful weapons108—described more fully below—others view the lack of human participation in the decision to take a human life to make such systems inherently unlawful or unethical.109
Opponents of fully autonomous weapons systems urge the adoption of an international treaty banning such weapons110 or at least providing a legal framework to ensure that such weapons are subject to meaningful human control.111 Others advocate a moratorium on developing such weapons until advances in technology enable the weapons to comply with LOAC.112 Proponents of autonomous weapons systems argue that the possibility that such systems may enhance compliance with LOAC could make their use more ethical than fighting wars without them.113 They recommend rigorous testing and review of proposed weapons under the existing Article 36 legal review process.114 "Article 36 Review" refers to the obligation of states to evaluate new or modified weapons to ensure they do not violate LOAC. The obligation stems from Article 36 of Additional Protocol I (API).115
Two well-established general principles governing armed attacks during armed conflict are distinction and proportionality. Distinction refers to distinguishing between combatants and military targets on the one hand, and civilians and civilian objects on the other.116 Proportionality refers to the balance between the military advantages expected to be gained from an attack versus the probable extent of collateral civilian harm.117 Some observers note the difficulty posed by reducing these principles to a digital format such that autonomous weapons systems will be capable of obeying them.118
A corollary to these basic principles is individual and group accountability for failure to adhere to them. Some commentators have raised the concern that robots are not amenable to deterrence or punishment, and wonder who will be held responsible for any noncompliant behavior.119
As far as objects are concerned, lawful targeting requires the ability to distinguish between military objectives and civilian objects. The API defines military objectives as objects that by "nature, location, purpose, or use make an effective contribution to military action and whose total or partial destruction, capture, or neutralization, in the circumstances ruling at the time, offers a definite military advantage."120 Objects not meeting those criteria are civilian objects.121 The nature of an object may change, making constant reassessment necessary.122 An object ordinarily used for civilian purposes may not be targeted unless the adversary is using it for military purposes such that it meets the criteria above, with any doubt to be resolved in favor of its treatment as a civilian object.123 Moreover, the API prohibits targeting facilities or other objects that are necessary for the survival of the civilian population or that would be likely to cause environmental harm if subjected to an armed attack.124
There are also rules applicable to targeting individuals. Under the API and customary international law, only combatants are lawful targets, and even they are protected if they become wounded or surrender.125 The API bars targeting civilians unless, and for so long as, they participate directly in hostilities.126 The API also protects certain military personnel, such as medics and clergy.127
Like the principle of distinction, the principle of proportionality has as its central aim the protection of civilians during armed conflict.128 It requires balancing the expected military advantage that an armed attack may obtain against the potential harm to civilians and civilian objects.129 Whether a given attack is proportionate is a judgment made at the initiation of the attack, which is subject to fluid conditions on the battlefield.130 There is disagreement among practitioners and academics as to how these dissimilar components—concrete military advantage versus incidental deaths of civilians and damage to property—might effectively be evaluated to determine what civilian harm is excessive.131 No clear formula exists.132 Some doubt that it will ever be possible to create an algorithm capable of making an adequate proportionality assessment.133
A central premise of the law of armed conflict is that the permissible means and methods of warfare are "not unlimited."134 Specifically, the API prohibits employing weapons and methods of warfare that are of a nature to cause superfluous injury or unnecessary suffering, or that are intended or may be expected to cause widespread long-term and severe damage to the environment.135 Specific types of weapons that these treaties ban include chemical and biological weapons.136 These treaties also forbid weapons that are indiscriminate in their effect, that is, those that are not able to be used in accordance with the principle of distinction.137
Consequently, API Article 36 requires that parties test new weapons to ensure compliance with the law of war:
In the study, development, acquisition or adoption of a new weapon, means or method of warfare, a High Contracting Party is under an obligation to determine whether its employment would, in some or all circumstances, be prohibited by this Protocol or by any other rule of international law applicable to the High Contracting Party.138
The proper test, then, is whether, in some or all circumstances, an autonomous weapons system is indiscriminate by nature or is of such a nature as to cause superfluous injury or unnecessary suffering, or is likely to cause widespread long-term and severe damage to the environment. In other words, an autonomous weapons system, like all other weapon systems, must be able to distinguish targets appropriately and to ensure that the attack is proportionate to the expected military advantage. Some argue that the fact that the weapon system operates autonomously does not per se govern whether it can meet these criteria.139 As the argument goes, as long as the autonomous system can be programmed with sufficiently reliable targeting information to ensure that it can be fired at a lawful target, the autonomous weapon system can survive an Article 36 review.140 On the other hand, some scientists have warned that robotic systems capable of distinguishing between civilians and combatants or making proportionality judgments are not likely in the short term.141 The ban on weapons that cause superfluous or unnecessary damage or suffering likewise arguably can be met by, among other things, ensuring the payload is not a prohibited type, such as biological or chemical in nature.
As mentioned above, the Department of Defense has a directive on the development of autonomous weapons systems.142 It states that "[a]utonomous and semi-autonomous weapons systems shall be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force."143 The directive requires the evaluation and testing process to ensure that autonomous weapons systems
(a) Function as anticipated in realistic operational environments against adaptive adversaries.
(b) Complete engagements in a timeframe consistent with commander and operator intentions and, if unable to do so, terminate engagements or seek additional human operator input before continuing the engagement.
(c) Are sufficiently robust to minimize failures that could lead to unintended engagements or to loss of control of the system to unauthorized parties.144
The directive emphasizes the need for an effective human-machine interface, requiring that autonomous weapons systems
(a) Be readily understandable to trained operators.
(b) Provide traceable feedback on system status.
(c) Provide clear procedures for trained operators to activate and deactivate system functions.145
Whether these efforts will assuage critics may turn on the "appropriate levels of human judgment over the use of force" that the Department of Defense adopts.146
Some opponents of developing fully autonomous weapons systems have raised a variety of concerns that do not necessarily implicate humanitarian law, but rather, concern whether the availability of such systems could make armed conflict more prevalent.147 According to this view, national leaders may be more likely to involve their nations in hostilities when the risk to human soldiers is minimized.148 A counter argument is that most new weapons are developed in order, at least in part, to minimize risk to troops and have been subject to the same criticism.149 Another argument for such systems is that robots are unlikely to replace soldiers on the battlefield entirely,
Opponents also argue that human compassion and other emotions are necessary to ethical war-fighting.152 Human empathy, some argue, helps soldiers to evaluate the intent of potential human targets to determine whether they actually pose a threat; machines, they argue, may possibly never be programmable to emulate compassion or empathy effectively.153 On the other side, proponents of such systems argue that human emotions—fear, anger, and the instinct for self-preservation—may lead to adverse consequences on the battlefield.154 Robots, they posit, may not be subject to human errors or unlawful behavior induced by human emotions.155
One other argument skeptics of automated weapon systems make is that the phenomenon of "automation bias," in which humans trust computer interpretations rather than their own senses and instincts regarding their environment, may nullify meaningful human control over such systems.156 A variation of this concern is that relying on robots will make killing too remote for soldiers, dehumanizing the endeavor by making it seem more like participating in a video game.157
Notable futurists, such as Elon Musk and the late Steven Hawking, have warned that AI has the potential to be far more dangerous than nuclear weapons158 and could be "the worst event in the history of our civilization."159 While of note perhaps more important is a growing level of organized groups concerned with LAWS. The following are examples of organized groups concerned with lethal autonomous weapon systems (LAWS).
In April 2017, DOD established the Algorithmic Warfare Cross-Functional Team to oversee Project Maven, a project to use AI to autonomously extract objects of interest from moving or still UAV imagery. Project Maven would develop computer vision algorithms, trained using AI techniques such as machine learning, to better identify targets. Tools built through Project Maven would be used to complement the labor-intensive process of humans analyzing drone video, a practice that delays providing results to warfighters for targeting purposes. It was suggested that such AI-enhanced tools could allow human analysts to process up to two to three times as much data within the same time period, providing more time-sensitive targeting data and, according to DOD, a reduction of collateral damage and civilian casualties.
Google is one of a number of companies involved in Project Maven. In March 2018, Google's role in Project Maven came to light and a number of Google employees reportedly expressed concern "that the company would offer resources to the military for surveillance technology involved in drone operations" while "others argued that the project raised important ethical questions about the development and use of machine learning."161 Reportedly about a dozen Google employees resigned in protest and about 4,000 employees signed a petition demanding "a clear policy stating that neither Google nor its contractors will ever build warfare technology."162 Reportedly Google will not renew its contract with DOD for Project Maven when the current contract expires in 2019.
The Campaign to Stop Killer Robots163 is a nonprofit umbrella organization that advocates for a ban on LAWS. Their belief is autonomous weapon systems are immoral and lack the decisionmaking capability of humans and, therefore, should be banned. They advocate for the preemptive ban on LAWS, arguing that "the introduction of such weapons would violate international humanitarian law and risk devastating consequences to civilian populations."164 The campaign to Stop Killer Robots is said to be emulating the past success of a nongovernment organization (NGO) movement in the late 1990s—the International Campaign to Ban Landmine (ICBL).165 The ICBL successfully lobbied for the negotiation of an international ban on landmines—the Ottawa Convention—and won the Nobel Peace Prize in 1997. Some note the similarities between autonomous weapon systems, landmines, and cluster munitions movements and suggest the Campaign to Stop Killer Robots and its proven NGO strategy raises some interesting possibilities.166
The parties to the United Nation's Convention on Conventional Weapons (CCW) has met on a variety of occasions—most recently from August 27 to August 31, 2018—to discuss lethal autonomous weapons. In November 2017 in Geneva, Switzerland, the U.N. Group of CCW Government Experts (GCE) reportedly affirmed that "international humanitarian law continues to apply fully to all weapon systems, including the potential development and use of lethal autonomous weapon systems."167 More happened at the GCE meeting, however. Their goal was to set the stage for negotiating an annex to the CCW, but they were not successful. The Russians, for example, strongly opposed any restrictions. There was, however, general agreement that meaningful human control over autonomous weapons systems is required.168 Accordingly, meaningful human control over LAWS can be exercised in five instances:
On September 12, 2018, Members of the European Parliament overwhelming passed a nonbinding resolution urging Member States and the European Council to work toward an international ban on weapon systems that lack meaningful human control over the critical function of selecting and engaging targets. It was also noted that France and Germany are not working toward and do not support the strict and total prohibition of autonomous weapons.
Regarding opposition to LAWS, DOD notes the current challenge:
A lack of trust by the Warfighters, and the wider public, is a major roadblock in DOD's continued development and use of autonomous unmanned systems. This lack of trust is highlighted by international efforts at the United Nations (U.N.) to consider policies that would prohibit the deployment of autonomous systems with lethal capabilities. Additionally, there are technological shortcomings in the current abilities of AI regarding ethical thinking that may limit the public's trust of autonomous unmanned systems developed for military capabilities.
Situational ethical reasoning is currently not coherently implementable by AI in the range of scenarios that military forces may encounter. Given this limitation, it is paramount to ensure that human authority, accountability, and the ability to maintain command and control (C2) are preserved as increasing numbers of unmanned systems become more widely deployed.171
As part of the discussion of the ethics and legality of LAWS that will have a direct impact on whether or not the Army and Marines develop LAWS for battlefield use, DOD notes
In considering the specific use of weaponized systems, Department of Defense Directive (DoDD) 3000.09, Autonomy in Weapon Systems, signed in November 2012, established policies and assigned responsibilities to shape the development and use of autonomous functions in weapon systems, including manned and unmanned platforms. It mandates that autonomous and semi-autonomous weapon systems be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force. DoDD 3000.09 also requires that persons who authorize the use of, direct the use of, or operate autonomous and semi-autonomous weapon systems must do so with appropriate care and in accordance with the law of war, applicable treaties, weapon system safety rules, and applicable ROE. DoDD 3000.09 underwent a mandatory periodical update with administrative changes (Change 1, May 8, 2017), but a more substantive update was expected to be completed in late 2017. That substantive update, when released, is expected to involve clarifications of definitions and processes rather than a shift in the overall thrust of the policy.
DOD does not currently have an autonomous weapon system that can search for, identify, track, select, and engage targets independent of a human operator's input. These tasks currently rely heavily on a human operator using remote operation, also referred to as "tele-operation." In the future, weaponization will be a crucial capability in mission sets where the unmanned system is directly supporting forces engaging in hazardous tasks.
In the realms of public and international diplomacy, concerned states, non-governmental organizations (NGOs), and experts in AI have urged an immediate and intensive effort to formulate and secure an international treaty restricting the development, deployment, and use of weapon systems that can autonomously locate, select, and attack human targets. In response to similar expressions of concern from some High Contracting Parties to the Geneva Conventions on Certain Conventional Weapons (CCW), the UN Office in Geneva hosted informal experts' meetings on lethal autonomous weapon systems (LAWS) in 2014, 2015, and 2016. The CCW established a Group of Governmental Experts (GGE) which met to discuss LAWS in a more formalized setting in 2017. A second meeting is foreseen for 27 to 31 August 2018. If such restrictions on autonomous weapon systems were to come into existence, and if the U.S. were to follow it, the ban would severely limit the ability to develop and use lethal autonomous weapon systems.172
In view of this policy position and, in consideration to the challenges posed by a possible treaty regulating AI, this suggests future DOD development of LAWS is possible, given favorable advances in AI technology and an absence of international restrictions.
Even under the current DOD directive to have humans in the loop for LAWS, some analysts have called for caution, as people might give too much deference to technology. For example, one analyst points to the 1988 case of the USS Vincennes guided missile cruiser mistakenly shooting down an Iranian passenger jet. The analyst notes that while there were indicators that it was a civilian aircraft, and final authorization for firing lay with the crew, no one was willing to challenge the computer.173
Congress has an active and ongoing interest in RAS and AI. In its September 2018 report titled "Rise of the Machines: Artificial Intelligence and its Growing Impact on U.S. Policy," the House Subcommittee on Information Technology notes
The loss of American leadership in AI could also pose a risk to ensuring any potential use of AI in weapons systems by nation-states comports with international humanitarian laws. In general, authoritarian regimes like Russia and China have not been focused on the ethical implications of AI in warfare, and will likely not have guidelines against more bellicose uses of AI, such as in autonomous weapons systems.174
As Congress continues policy debates on RAS and AI, potential considerations could include, but are not limited to, the following:
As the United States pursues military RAS and AI applications and systems, an examination of foreign military RAS and AI efforts and their potential impact on U.S. ground forces could be of benefit to policymakers. One report notes
The emphasis on armed robots underscores the difference between U.S. concepts of operations, in which unmanned ground systems largely support intelligence, surveillance, and reconnaissance (ISR) and augment warfighters' capabilities, while the Russian military contemplates small to large unmanned ground vehicles (UGVs) doing the actual fighting in the near future alongside or ahead of the human fighting force.175
It has been suggested that military RAS and AI have the potential to change the very nature of war.176 At the strategic level, this could affect how the U.S. organizes ground forces, how it fights, and what types of major weapon systems it will need. At the operational and tactical levels, enemy RAS and AI capabilities could dictate specific weapon systems design, the development of new types of units to address enemy RAS and AI, and how brigade to squad level units conduct tactical operations.
While the DOD and the Services emphasizes the development of RAS and AI for the U.S. military, it might be considered prudent to have a full understanding of foreign military RAS and AI efforts to help guide U.S. efforts and possibly preclude strategic or tactical RAS and AI "surprises" posed by foreign militaries or nonstate groups.
Regarding fully autonomous weapon systems, one report notes
Some of the most prominent leaders in these fields are publicly warning about the dangers in an unconstrained environment. Military operations enabled by these technologies, and especially by artificial intelligence, may unfold so quickly that effective responses require taking humans out of the decision cycle. Letting intelligent machines make traditionally human decisions about killing other humans is fraught with moral peril, but may become necessary to survive on the future battlefield, let alone to win. Adversaries will race to employ these capabilities and the powerful operational advantages they may confer.177
While DOD continues to eschew fully automated ground combat systems, there is a possibility that other nations could aggressively pursue fully autonomous weapons systems to achieve a battlefield advantage. This raises a question: are other nation's efforts to develop LAWS sufficient justification for the United States to do the same? The evolution of other weapons, such as hypersonic weapons, could also result in U.S. development of fully autonomous weapon systems from a purely defensive perspective. Despite DOD's insistence that a "man in the loop" capability will always be part of RAS systems, it is possible if not likely, that the U.S. military could feel compelled to develop ground fully autonomous weapon systems in response to comparable enemy ground systems or other advanced threat systems that make any sort of "man in the loop" role impractical.
While U.S. development of RAS and AI for use by ground forces is articulated in a variety of strategies and directives, it can be argued that equally and, possibly, more important is the development of technologies, systems, formations, and tactics, techniques, and procedures designed to counter foreign use of RAS and AI against U.S. ground forces. Such a "defensive" approach to RAS and AI could prove to be particularly prudent, as it is possible other nations may introduce RAS and AI systems into ground combat before the United States, thereby necessitating an effective means of countering their use. Furthermore, foreign RAS and AI advances not directly related to weapons such as AI-enhanced military decisionmaking and intelligence or RAS-enabled logistics could have an indirect impact on U.S. ground forces and may be taken into consideration when developing U.S. counter RAS and AI capabilities.
Many cutting-edge innovations in AI and RAS are occurring in the private sector, and these innovations could be adapted for use by U.S. ground forces. However, engaging with the private sector raises policy considerations for the Army and Marine Corps, as well as DOD more broadly. For example, if policymakers seek to assist the military in making use of emerging technologies such as AI and RAS, they might consider expanding support for entities such as the Defense Innovative Unit. If the Administration or Congress seeks to have more in-house expertise in AI and RAS, they might also consider supporting the development of service incentives or recruitment models to attract additional experts, either on a permanent or temporary basis.
There are other potential considerations which might merit further examination. As previously noted, there are a number of security challenges associated with DOD and the Services engaging with academic institutions and the private sector in regards to foreign national students and employees who might pose a security risk. Another consideration is how academia and private industry view the morality of military RAS and AI use. As technologies in these areas advance, some academic and private sector institutions might choose to "opt out" of working with DOD on technologies which they might view as counter to their values—much as Google reportedly is doing with DOD's Project Maven. If this becomes a pervasive practice, DOD efforts to develop RAS and AI technologies could be significantly hindered. Some experts have argued that a productive relationship could be encouraged by DOD approaching AI development and use with a focus on ethics and safety.178 The memorandum establishing the JAIC includes these principles, which must be considered within the Center's overarching goal of accelerating the delivery of AI-enabled capabilities.179
Aside from equipping the Services, personnel-related issues may be a major congressional concern as the Services incorporate various levels of RAS and AI into the force both in the near and far term. As such, Congress might choose to examine a number of topics in the following areas:
Questions Congress may consider include the following:
Questions Congress may consider include the following:
Some considerations for training servicemembers on RAS and AI include the following:
Some considerations for developing a career path model for RAS and AI experts include the following:
Legal and ethical considerations associated with the possible development and adoption of fully autonomous lethal weapon systems are both considerable and complicated. As RAS and AI technology development advances in both the commercial and military sectors, a point will likely be reached where U.S. policymakers will be faced with the decision—can the United States (legally) and should the United States (ethically) develop LAWS? The Administration will likely have its own views on the legality and morality of LAWS, as will Congress. As previously noted
Authorizing a machine to make lethal combat decisions is contingent on political and military leaders resolving legal and ethical questions. These include the appropriateness of machines having this ability, under what circumstances should it be employed, where responsibility for mistakes lies, and what limitations should be placed on the autonomy of such systems.... Ethical discussions and policy decisions must take place in the near term in order to guide the development of future [unmanned aircraft systems] capabilities, rather than allowing the development to take its own path apart from this critical guidance.180
In view of this, there appears to be an opportunity for Congress to examine these issues and perhaps define its role in the overall U.S. governmental debate on LAWS. These debates and resulting policy decisions and guidance could prove beneficial to DOD as a whole—and the Army and Marines in particular—as they develop RAS and AI systems and capabilities for use on the battlefield.
Given current deliberations on LAWS and ongoing discussions in the U.N., it is conceivable that at some point in the future, formal international efforts could be undertaken to regulate LAWS development and use. As DOD has previously noted
If such restrictions on autonomous weapon systems were to come into existence, and if the U.S. were to follow it, the ban would severely limit the ability to develop and use lethal autonomous weapon systems.181
Such restrictions could limit other nations as well. Should deliberate formal efforts to regulate LAWS emerge in the coming years, the United States has the option of playing an active role, a passive role, or no formal role whatsoever. Because of the potential national security implications of a potential LAWS ban, as well as its impact on Army and Marine RAS and AI developmental efforts, policymakers both in the Administration and Congress might examine and decide what the appropriate level of U.S. involvement should be in such an international regulatory process.
Author Contact Information
Adam Wunische, "AI Weapons Are Here to Stay," The National Interest," August 7, 2018.
Summary of the 2018 National Defense Strategy of the United States of America, Sharpening the American Military's Competitive Edge, 2018, p. 5.
Headquarters, United States Air Force, United States Air Force Unmanned Aircraft Systems Flight Plan 2009-2047,
Washington, DC, May 18, 2009, p. 41.
Taken directly from Joint Chiefs of Staff, Joint Concept for Robotic and Autonomous Systems (JCRAS), October 16, 2016, p. 2.
Taken directly from M.L. Cummings, Artificial Intelligence and the Future of Warfare, Chatham House, The Royal Institute of International Affairs, January 2017, p. 2.
See the FUTURE of Artificial Intelligence Act (S. 2217 and H.R. 4625), the AI JOBS Act of 2018 (H.R. 4829), and the John S. McCain National Defense Authorization Act for Fiscal Year 2019 (P.L. 115-232). The AI classification scheme is presented in Peter Norvig and Stuart J. Russell, Artificial Intelligence: A Modern Approach, 3rd ed. (New Jersey: Prentice Hall, 2009).
For an overview of AI and selected, overarching issues and policy considerations, see CRS In Focus IF10608, Overview of Artificial Intelligence, by Laurie A. Harris.
Taken directly from Expert System, https://www.expertsystem.com/machine-learning-definition/, accessed August 30, 2018.
Taken directly from Department of Defense (DOD) Directive 3000.09, Change 1, May 8, 2017, Autonomy in Weapons Systems, pp. 13-14.
For additional information, see CRS Recorded Event WRE00207, Artificial Intelligence: Innovation, Impacts, & Policy Considerations for the 115th Congress, by Laurie A. Harris.
Wisskirchen et al., "Artificial Intelligence and Robotics and Their Impact on the Workplace," International Bar Association Global Employment Institute, April 2017, p. 9.
The report's authors consider the First Industrial Revolution as the beginning of the Industrial Age around 1800, the Second Industrial Revolution as the beginning of electrification at the end of the 19th century, and the Third Industrial Revolution as the beginning of digitization in the 1970s.
Wisskirchen, p. 12.
Ibid., p. 13.
For a detailed discussion of AI and national security, see CRS Report R45178, Artificial Intelligence and National Security, by Daniel S. Hoadley and Nathan J. Lucas.
Summary of the 2018 National Defense Strategy of the United States of America: Sharpening the American Military's Competitive Edge, p. 7.
Ibid., p. 2.
The Army Vision (2018-2028), https://www.army.mil/article/206270/the_army_vision_2018_2028, June 6, 2018.
Paul Scharre, Army of None: Autonomous Weapons and the Future of War, 2018, p. 102.
Sydney J. Freedberg Jr., "Evolutionary Revolution: Esper's Army Modernization Philosophy," BreakingDefense.com, May 16, 2018.
Mick Ryan, Human-Machine Teaming for Future Ground Forces, Center for Strategic and Budgetary Assessments, 2018, p. 9.
For a partial listing of these studies, see, for example, Erin Winik, "Every study we could find on what automation will do to jobs, in one chart," MIT Technology Review, January 25, 2018.
Pew Research Center, AI, Robotics and the Future of Jobs, August 2014, p. 5.
David H. Autor, "Why Are There Still So Many Jobs? The History and Future of Workplace Automation," Journal of Economic Perspectives, Vol. 29, No. 3, Summer 2015, pp. 3-30.
Jaime Condliffe, "Could the AI Talent Shortage Be Eased If Tech Giants Learn to Share?" MIT Technology Review, November 10, 2017, and Jacques Bughin, Eric Hazan, Sree Ramaswamy, et al., How Artificial Intelligence Can Deliver Real Value to Companies, McKinsey Global Institute, June 2017.
Mick Ryan, Human-Machine Teaming for Future Ground Forces, Center for Strategic and Budgetary Assessments, 2018, p. 9.
Isaac R. Porche III, Bradley Wilson, Erin-Elizabeth Johnson, "How do We Deal with a Flood of Data," RAND, June 23, 2014.
Richard Danzig, "Technology Roulette: Managing Loss of Control as Many Militaries Pursue Technological Superiority," Center for New American Security, June 2018, p. 15.
Information in this section is taken from Mick Ryan, Human-Machine Teaming for Future Ground Forces, Center for Strategic and Budgetary Assessments, 2018, pp. 11-14.
Dr. Bob Sadowski, "Shaping the Future: Army Robotics and Autonomous Systems," U.S. Army Tank Automotive Research, Development and Engineering Center, March 2016, p. 6.
Mick Ryan, p. 20.
Jason Sherman, "Army Previews Desired Capabilities for Next Generation Combat Vehicle," Inside the Army, August 31, 2018.
Victoria Leoni, "Here's How Artificial Intelligence Could Predict When Your Army Vehicle Will Break Down," Military Times, June 27, 2018.
Mick Ryan, p. 12.
RAND's 2013 report: The Army's Role in Overcoming Anti-Access and Area Denial Challenges," p. ix characterizes A2/AD as "Anti-access (A2) challenges prevent or degrade the ability to enter an operational area. These challenges can be geographic, military, or diplomatic. Area denial (AD) refers to threats to forces within the operational area. As they relate to U.S. ground forces (the Army and Marine Corps), AD threats are characterized by the opponent's ability to obstruct the actions of U.S. forces once they have deployed."
Mick Ryan, p. 13.
Ibid., p. 39.
Information provided to CRS from Army Staff, October 5, 2018.
Paul Mozor, "Beijing Wants A.I. to Be Made in China by 2030," New York Times, July 20, 2017.
"The Race for AI," Defense One, March 2018, Forward.
Scharre, p. 102.
Patrick Turner, "A Criminal Gang Used a Drone Swarm to Obstruct an FBI Hostage Raid," Defense One, May 3, 2018.
Mick Ryan, p. 13.
Nikolai Novichkov, "New Russian Combat UGV Breaks Cover, Uran-9 Readies for Service," IHS Jane's International Defence Review, October 2016, p. 30.
Kelsey Atherton, "Russia Confirms its Armed Robot Tank was in Syria," C4ISRNet.com, May 7, 2018.
Samuel Bendett, "Here's How the Russian Military is Organizing to Develop AI," Defense One, July 20, 2018.
"The Race for AI," Defense One, March 2018, p. 14.
Gabriel Dominguez, "China Seeking to Surpass U.S. in Military Technologies, Says Canadian Report," Jane's Defence Weekly, June 6, 2018, p. 6.
Gabriel Dominguez, "China Threatening U.S. DOD's Technical, Industrial Base, Say Pentagon Officials," Jane's Defence Weekly, June 27, 2018, p. 8.
CRS meeting with Army Staff, July 24, 2018.
Scharre, pp. 104-105.
Tim Hornyak, "Korean Machine Gun Robots Start DMZ Duty," C/NET.com, July 14, 2010.
Scharre, pp. 105-107.
"Britain Uses Brimstone Missiles for the First Time in Syria," Indian Defense News, January 12, 2016.
Office of the Secretary of Defense, Unmanned Systems Integrated Roadmap 2017-2042, June 2018, p. 1.
Ibid., p. 4.
Joint Concept for Robotic and Autonomous Systems (JCRAS), October 16, 2016, p. v.
Department of Defense (DOD) Directive 3000.09, Change 1, May 8, 2017, Autonomy in Weapons Systems, p. 1.
Ibid., p. 2.
Senate Committee on Armed Services, Hearing to Consider the Nomination of General Paul J. Selva, USAF, for Reappointment to the Grade of General and Reappointment to be Vice Chairman of the Joint Chiefs of Staff, 115th Congress, 1st Session, July 18, 2017.
2018 Department of Defense, Artificial Intelligence Strategy, June 2018.
Deputy Secretary of Defense Memorandum, Subject: Establishment of the Joint Artificial Intelligence Center, June 27, 2018.
Justin Doubleday, "Military's New AI Center Aims to Accelerate Fielding, Attract "World-Class" Talent," InsideDefense.com, July 12, 2018.
For additional information on Army Futures Command, see CRS Insight IN10889, Army Futures Command (AFC), by Andrew Feickert.
Army Directive 2018-18, Army Artificial Intelligence Task Force in Support of the Department of Joint Artificial Intelligence Center, October 2, 2018.
Information in this section is from a CRS meeting with Marine Corps Staff, September 4, 2018.
U.S. Army Training and Doctrine Command, The U.S. Army Robotic and Autonomous Systems Strategy, March 2017, http://www.arcic.army.mil/App_Documents/RAS_Strategy.pdf.
Ibid., pp. 1-2.
Ibid., p. 3.
Ibid., p. 2.
Ibid., p. 5.
Ibid., p. 10.
Marine Corps Robotic and Autonomy Strategy (MCRAS), June 2018.
Ibid., p. 8.
Ibid., p. 12.
Ibid., p. 16.
M.L. Cummings, Artificial Intelligence and the Future of Warfare, Chatham House, The Royal Institute of International Affairs, January 2017, pp. 4 and 9-10.
Information in this section is taken from U.S. Army Briefing, Army Ground Robotics Update, November 14, 2017, and Melanie Rovery, "SMET Advances: U.S. Army Selects Its Robot Mules for the Next Phase," Jane's International Defence Review, March 2018.
Information in this section was provided to CRS by the Army Staff, October 5, 2018.
Army Briefing, PM Force Projection Overview, February 28, 2017.
Information in this section is from Vivienne Machi, "Army Rolling Ahead with Manned-Unmanned Convoys," National Defense, April 4, 2018.
Army Briefing, Army Robotic and Autonomous Systems (RAS) Portfolio, April 11, 2018.
Army Briefing, Army Ground Robotics Overview: OSD Joint Technology Exchange Group, April 24, 2018.
Sydney J. Freedberg Jr., "Army Pushes Bradley Replacement; Cautious on Armed Robots," Breaking Defense, June 27, 2018.
Sean Kimmons, "Initial Prototypes for Next-Generation Combat Vehicle to Focus on Manned-Unmanned Teaming," Army News Service, March 26, 2018.
Ibid. and Army Briefing, Army Robotic and Autonomous Systems (RAS) Portfolio, April 11, 2018.
Ashley Tressel, "DIUx Awards Contract to Use AI Software for Bradley," Insidedefense.com, June 26, 2018.
James Vincent, "The U.S. Army is Using Machine Learning to Predict When Combat Vehicles Need Repair," The Verge, June 26, 2018.
Jack Corrigan, "How Artificial Intelligence Could Keep U.S. Army Vehicles Ready for Action," Defense News, August 11, 2018.
Aaron Gregg, "Army to Use Artificial Intelligence to Predict Which Vehicles Will Break Down," The Washington Post, June 26, 2018.
Information in this section is from a CRS meeting with Marine Corps Staff, September 4, 2018.
"The Autonomy Paradox: Why 'Unmanned Systems' Don't Shrink Manpower Needs," Armed Forces Journal, (October, 2011), http://armedforcesjournal.com/the-autonomy-paradox/.
The largest exception to this is the military medical community, many of who join the armed forces after completing extensive civilian schooling and training. Lawyers and chaplains are also typically recruited after completion of additional schooling or training.
Secretary of Defense Memorandum, The Next Two Links to the Force of the Future, June 9, 2019.
See International Committee of the Red Cross, Autonomous weapon systems: Technical, military, legal and humanitarian aspects, Expert Meeting, Geneva, Switzerland, 26-28 March 2014, at 61 ("In theory, it would only be one step further [from current unmanned but remotely controlled weapons systems] to allow a mobile unmanned system to fire a weapon without human intervention, but one that would entail a technological leap in capability while simultaneously raising significant legal and ethical questions."). With respect to unmanned aerial vehicles, DOD has predicted that "the level of autonomy should continue to progress from today's fairly high level of human control/intervention to a high level of autonomous tactical behavior that enables more timely and informed human oversight." Markus Wagner, The Dehumanization of International Humanitarian Law: Legal, Ethical, and Political Implications of Autonomous Weapon Systems, 47 Vand. J. Transnat'l L. 1371, 1374 (2014) (quoting FY 2009-2034 Unmanned Systems Integrated Roadmap 27 (2009)).
See. e.g., Kenneth Anderson and Matthew Waxman, Law and Ethics for Autonomous Weapon Systems Why a Ban Won't Work and How the Laws of War Can 4 (Hoover Institution and Stanford University 2013) (recommending high-level weapons reviews to determine propriety of autonomous weapons systems on a case-by-case basis).
See Christof Heyns, Autonomous Weapons Systems: Living a Dignified Life and Dying a Dignified Death, in Autonomous Weapons Systems 3, 10 (Nehal Bhuta, et al., eds. 2016) (arguing that delegating to a machine the decision to use deadly force against a human being could be an inherently arbitrary deprivation of the right to life). For an alternate view, see Dieter Birnbacher, Are Autonomous Weapons a Threat to Human Dignity, in Autonomous Weapons Systems 105, 120 (Nehal Bhuta, et al., eds. 2016) (arguing that "for victims whose dignity is at stake, it is a matter of indifference whether the threat they are exposed to comes from manned or unmanned weapons…").
See Mary Ellen O'Connell, Banning Autonomous Killing: The Legal and Ethical Requirement that Humans Make Near-Time Lethal Decisions, in American Way of Bombing: Changing Ethical and Legal Norms, from Flying fortresses to Drones 224, 236 (Matthew Evangelista and Henry Shue eds., 2014) (arguing that it is imperative that human beings retain sovereignty over kill decisions and advocating a treaty banning fully autonomous killing).
See Heyns, supra note 109, at 13 (describing emerging view that the central question as to the legality of autonomous weapons systems will turn on whether such systems allow "meaningful human control" over the decision to use force against a human being).
See Guglielmo Tambunrrini, On Banning Autonomous Weapons Systems from Deontological to Wide Consequentialist Reasons, in Autonomous Weapons Systems, supra note 109, at 122, 131 (arguing that current state-of-the-art and foreseeable developments in robotics and AI "overwhelmingly support the moral obligation of suspending autonomous weapons systems development" until compliance with LOAC principles can be convincingly demonstrated).
See Anderson and Waxman, supra note 108, at 21 (arguing there may be "serious humanitarian risks to prohibition [of autonomous weapons systems], given the possibility that autonomous weapons systems could in the long run be more discriminating and ethically preferable to alternatives").
Id. at 4.
Protocol Additional to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflict, adopted June 8, 1977, 1125 U.N.T.S. 3, entered into force December 7, 1978, art. 36 (hereinafter API). The United States is not a party to API but may regard parts of it as representing customary international law. See Anderson and Waxman, supra note 104, at 9; Wagner, supra note 103, at 1385 (noting that the United States has incorporated many provisions of API into its military manuals).
Jean-Marie Henckaerts and Louise Doswald-Beck, I Customary International Humanitarian Law 3 (2005) (hereinafter "ICRC Study") Rule 1 states that "The parties to the conflict must at all times distinguish between civilians and combatants…." Rule 7 states that "The parties to the conflict must at all times distinguish between civil objects and military objectives…." Id. at 25.
Id. at 46 (Rule 14 states "Launching an attack which may be expected to cause incidental loss of civilian life, injury to civilians, damage to civilian objects, or a combination thereof, which would be excessive in relation to the concrete and direct military advantage anticipated is prohibited).
Wagner, supra note 107, at 1388 (positing that computers, which may be capable of computing quantitative assessments, are currently less capable of making qualitative assessments, and questioning whether "the principle of distinction and the principle of proportionality can be encoded into digital format").
Id. at 1403 (noting that culpability requires moral agency, which machines can never possess, and that a traditional deterrence rationale does not apply).
API, supra note 115, art. 52(2).
Id. art 52(1).
Wagner, supra note 107, at 1390.
API, supra note 115, art. 52(3).
Id. arts. 54-56.
Id. art. 41.
Id. art. 51(3).
Id. art. 41.
Wagner, supra note 107, at 1396.
API, supra note 115, art. 57 (requiring, among other things, that those responsible for military operations "refrain from deciding to launch any attack which may be expected to cause incidental loss of civilian life, injury to civilians, damage to civilian objects, or a combination thereof, which would be excessive in relation to the concrete and direct military advantage anticipated").
Wagner, supra note 107, at 1396.
Id. at 1396-98.
Id. at 1399.
Regulations concerning the Laws and Customs on Land, Annex to Convention IV Respecting the Laws and Customs of War on Land art. 22 (1907) (hereinafter Hague Regulation); API, supra note 115, art. 35(1).
API, supra note 115, art. 35(2)-(3).
See, e.g., Convention on the Prohibition of the Development, Production, Stockpiling, and Use of Chemical Weapons and on their Destruction, Apr. 29, 1997, 1974 U.N.T.S. 45 (hereinafter CWC); Convention on the Prohibition of the Development, Production and Stockpiling of bacteriological (Biological) and Toxin Weapons and on their Destruction, Mar. 26, 1975, 1975 UST 583, 1015 UNTS 163 (hereinafter BWC). For a list of other types of weapons that may be banned under customary international law, see ICRC Study, supra note 116, part IV.
ICRC Study, supra note 116, at 40 (Rule 12 describes indiscriminate attacks as those "(a) which are not directed at a specific military objective; (b) which employ a method or means of combat which cannot be directed at a specific military objective; or (c) which employ a method or means of combat the effects of which cannot be limited as required by international humanitarian law; and consequently, in each such case, are of such a nature to strike military objectives and civilians or civilian objects without distinction"). See also id. at 244 (Rule 71 states that "the use of weapons which are by nature indiscriminate is prohibited").
API, supra note 115, art. 36.
Anderson and Waxman, supra note 108, at 10.
See Tambunrrini, supra note 112, at 130 (citing profound technological challenges to be resolved, such as effective in situ target discrimination and recognition of the status of those otherwise hors de combat") citing Ronald Arkin, Governing Lethal Behavior in Autonomous Robots (2009)).
Department of Defense (DOD) Directive 3000.09, Change 1, May 8, 2017, Autonomy in Weapons Systems.
Dan Saxon, A Human Touch: Autonomous Weapons, DoD Directive 3000.09 and the Interpretation of 'Appropriate Levels of Human Judgment over the Use of Force', 165, 165-66 in Autonomous Weapons Systems (Nehal Bhuta, et al., eds. 2016).
See, e.g., Human Rights Watch, et al., Losing Humanity: The Case Against Killer Robots 37 (2012) (hereinafter HRW) (arguing that "assigning combat functions to robots minimizes military casualties but risks making it easier to engage in armed conflict and shifts the burden of war onto the civilian population"); Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions.)
Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, A/HRC/23/47, at 11 (2013) (Th[e] ease [of deploying lethal autonomous robotic systems [LARS] rather than human soldiers] could potentially affect political decisions. Due to the low or lowered human costs of armed conflict to States with LARs in their arsenals, the national public may over time become increasingly disengaged and leave the decision to use force as a largely financial or diplomatic question for the State, leading to the "normalization" of armed conflict.").
Anderson and Waxman, supra note 108, at 18 (arguing that "this concern is not special to autonomous weapons. The same objection has already been made with respect to remotely-piloted UAVs and high-altitude bombing before that. Generally, it can be made with respect to any technological development that either reduces risk to one's own forces or reduces risk to civilians, or both").
Id. at 7 (noting that "no one seriously expects remotely-controlled or autonomous systems to completely replace humans on the battlefield. Many military missions will always require humans on the ground, even if in some contexts they will operate alongside and in conjunction with increasingly automated, sometimes autonomous, systems").
Id. at 15 ("It may well be, for instance, that weapons systems with greater and greater levels of automation can—in some battlefield contexts, and perhaps more and more over time—reduce misidentification of military targets, better detect or calculate possible collateral damage, or allow for using smaller quanta of force compared to human decision-making.")
HRW, supra note 147, at 38 (Robots cannot identify with humans, which means that they are unable to show compassion, a powerful check on the willingness to kill. For example, a robot in a combat zone might shoot a child pointing a gun at it, which might be a lawful response but not necessarily the most ethical one.").
See Tambunrrini, supra note 112, at 129 (noting technical challenge associated with programming a robot to recognize persons who are hors de combat, which may require "viewpoint independent classification of bodily postures and gestures in variable illumination conditions, in addition to an understanding of emotional expressions and real-time reasoning about deceptive intentions….").
Anderson and Waxman, supra note 108, at 15 ("We should not rule out in advance possibilities of positive technological outcomes—including the development of technologies of war that might reduce risks to civilians by making targeting more precise and firing decisions more controlled (especially compared to human-soldier failings that are so often exacerbated by fear, panic, vengeance, or other emotions—not to mention the limits of human senses and cognition).").
Noel Sharkey, Staying in the Loop: Human Supervisory Control of Weapons, in Autonomous Weapons Systems 23, 35 (Nehal Bhuta, et al., eds. 2016) ("The operation of automatic reasoning has been shown to favor the uncritical acceptance of suggestions and maintains a strong bias.").
P.W. Singer, Wired for War 395 (2005) (arguing that "technology might lessen the likelihood of anger-fueled rage, but also make some soldiers too calm, too unaffected by killing").
Catherine Clifford, "Elon Musk: Mark my Words—AI is Far More Dangerous than Nukes," CNBC, March 13, 2018.
Arjun Kharpal, "Steven Hawking Says AI Could be Worst Event in the History of Our Situation," CNBC, November 6, 2017.
Information in this section is taken from Cheryl Pellerin, "Project Maven to Deploy Computer Algorithms to War Zone by Year's End," DOD News, June 21, 2017; Kelsey Atherton, "Targeting the Future of DOD's Controversial Project Maven Initiative, C4ISRnet, July 27, 2018; Nick Statt, "Google Reportedly Leaving Project Maven Military AI Program After 2019, The Verge, June 1, 2018; and Daisuke Wakabayashi and Scott Shane, "Google Will Not Renew Pentagon Contract That Upset Employees," New York Times, June 1, 2018.
Wakabayashi and Shane.
https://www.stopkillerrobots.org/, accessed August 31, 2018.
Michael C. Horowitz and Julia Macdonald, "Will Killer Robots be Banned? Lessons from Past Civil Society Campaigns," Lawfare, November 5, 2017.
Sean Welsh, "Kill Switch," Jane's Intelligence Review, March 2018, p. 51.
Ibid., pp. 51-52.
European Parliament, 2014-2019, Motion for a Resolution on Autonomous Weapon Systems, September 5, 2018 and Brooks Tigner, "MEPs Lobby for Total International Ban on Lethal Autonomous Weapons," Jane's Defence Weekly, September 19, 2018, p. 4.
Office of the Secretary of Defense, Unmanned Systems Integrated Roadmap 2017-2042, June 2018, pp. 21-22.
Ibid., pp. 22-23.
William A. Galston, "Why the Government Must Help Shape the Future of AI," Brookings Institute, September 13, 2018.
Subcommittee on Information Technology, Committee on Oversight and Government Reform, U.S. House of Representatives, "Rise of the Machines: Artificial Intelligence and its Growing Impact on U.S. Policy," September 2018, p. 6.
The Race for AI, Defense One, March 2018, p. 10.
See CRS Report R45178, Artificial Intelligence and National Security, by Daniel S. Hoadley and Nathan J. Lucas and Peter Singer; "Getting to Grips with Military Robots," The Economist, January 25, 2018; and Aaron Mehta, "AI Makes Mattis Question Fundamental Beliefs About War," C4ISRnet.com, February 17, 2018.
David Barno and Nora Bensahel, "War in the Fourth Industrial Revolution," War on the Rocks, June 19, 2018.
Jade Leung and Sophie-Charlotte Fischer, "JAIC: Pentagon Debuts Artificial Intelligence Hub," Bulletin of the Atomic Scientists," August 8, 2018, at https://thebulletin.org/2018/08/jaic-pentagon-debuts-artificial-intelligence-hub/.
Deputy Secretary of Defense Memorandum, Subject: Establishment of the Joint Artificial Intelligence Center, June 27, 2018.
Headquarters, United States Air Force, United States Air Force Unmanned Aircraft Systems Flight Plan 2009-2047,
Washington, DC, May 18, 2009, p. 41.
Office of the Secretary of Defense, Unmanned Systems Integrated Roadmap 2017-2042, June 2018, pp. 22-23.