Military Applications of Extended Reality
https://crsreports.congress.gov
Updated February 5, 2025
Military Applications of Extended Reality
Although commercial and consumer industries have been investing in extended reality (XR) for decades, recent advances have expanded the number of potential applications for the U.S. military. As such, the Office of the Under Secretary of Defense for Research and Engineering has identified human-machine interfaces for XR as 1 of 14 critical technology areas for the Department of Defense (DOD). As DOD continues to develop XR and related applications, Congress may consider implications for defense authorizations and appropriations, military force structure, and cybersecurity.
Overview
XR encompasses three main categories of physical and digital environments (Figure 1):
• Virtual reality (VR), a fully immersive digital environment (e.g., video games that place the user within the virtual world of the game).
• Augmented reality (AR), an overlay of digital objects on physical environments (e.g., Instagram filters that overlay preset digital effects on a user’s videos or photographs).
• Mixed reality (MR), a hybrid of physical and digital environments in which physical and digital objects can interact. Unlike AR, MR could enable a user to manipulate physical or digital objects and share their view of those objects with other users within the same mixed reality environment (e.g., collaboratively marking adversary troop locations on a projected digital map).
Figure 1. Main Categories of Extended Reality
Source: Tutorials Link, “Difference Between AR, VR, MR,” at https://tutorialslink.com/Articles/Difference-Between-AR-VR-MR/973. A number of advanced enabling capabilities, such as 5G and edge computing—a type of computing that is done “at or near the source of data”—are likely to expand XR applications in the future. These capabilities could improve data rates, increase user capacity, and reduce latency (i.e., time delay), all of which could support large-scale, networked applications. DOD has tested 5G-enabled applications of XR at Joint Base Lewis–McChord (WA) and Joint Base San Antonio (TX).
Military Applications of Extended Reality
The U.S. military is exploring a range of applications for XR, with research and development programs in each of the services. These applications include tactical, flight, maintenance, medical, and other training, as well as warfighting.
Training According to former Under Secretary of Defense for Research and Engineering Heidi Shyu, DOD intends to leverage “AR/VR and live training ... [that is being matured] by the gaming industry” as a basis for developing its own tailored XR programs. Doing so could enable the military to conduct training exercises that are too costly or dangerous to conduct in physical environments, as well as enable servicemembers in distant locations to train together.
For example, the Army’s Synthetic Training Environment (STE)—an XR training environment intended to complement or integrate with live training—seeks to enable soldiers “to train where they will fight, with the partners
Military Applications of Extended Reality
https://crsreports.congress.gov
they will fight with, and in complex operational environments to include dense urban, woodland, jungle, desert, and sub-terrain, before the first fight begins.” STE is to be designed to enable soldiers to more efficiently “increase proficiency through repetition.” These factors could, in turn, increase both readiness and lethality.
The Air Force uses XR for flight training—with the intent of reducing cost, training time, and wear on aircraft. It is also exploring XR for maintenance training and is in the process of building virtual training hangars “to enable training anywhere and anytime” on a variety of airframes. Similarly, the Navy seeks to use XR to connect engineers and maintainers, who could work together to address maintenance issues across the globe in real-time.
DOD is also examining applications of XR for medical training. According to the Air Force, XR could “[increase] the availability of training, without a need to increase manpower availability for training setup.” This application could allow for distributed learning and create greater efficiencies for understaffed medical training courses.
Warfighting The military is continuing to explore applications of XR for warfighting (Figure 2). It has long incorporated XR into the heads-up and helmet-mounted displays (HUD and HMD, respectively) used by pilots and aircrew. These displays can provide dynamic flight and sensor information intended to increase the users’ situational awareness and improve weapons’ targeting. In the case of the F-35 fighter aircraft’s HMD, inputs from the F-35’s external cameras provide pilots with a 360-degree view of their surroundings; it also displays night vision and thermal imagery—all of which can be overlaid with the technical details (e.g., altitude, speed) of any detected objects.
Likewise, the Army is developing the Integrated Visual Augmentation System (IVAS), a ruggedized (i.e., strengthened) MR headset that is currently based on Microsoft’s commercially available HoloLens. Army documents state that IVAS is to “[integrate] next generation 24/7 situational awareness tools and high-resolution digital sensors to deliver a single platform that improves soldier sensing, decision making, target acquisition, and target engagement.” The service concluded technology maturation initiatives to integrate IVAS technologies into both ground and air vehicle platforms in FY2024.
Figure 2. Illustrative Battlefield Use of XR
Source: https://jasoren.com/augmented-reality-military/.
Potential Issues for Congress
Congress may consider a number of issues as it continues to evaluate DOD investments in current and emerging military applications of XR.
Affordability Military applications of XR vary considerably in terms of up-front development costs, with one of the U.S. military’s largest XR programs, IVAS, costing at least $22 billion to field over 10 years. Once fielded, however, XR systems may reduce training costs by removing the need to centralize personnel, use live ammunition, or operate platforms during training scenarios. To assess these issues, Congress could direct an independent analysis of the potential benefits and drawbacks (e.g., cognitive overload) of XR training and warfighting applications against both their costs and their anticipated savings. This analysis may determine whether there are less costly, alternative means of achieving any identified benefits. Congress may also seek to obtain information about the projected lifecycle costs—including maintenance requirements—for XR systems.
Technological Maturity While some applications of XR are relatively mature— particularly those incorporating standalone AR—others are at a more nascent stage of development, require greater levels of technology integration, or have otherwise experienced delays in fielding or testing. Congress may continue to seek information about the technological maturity of XR systems and subsequently determine whether those systems warrant the requested funding levels. Congress may also assess the technological maturity of any necessary enabling capabilities to determine whether they are sufficiently mature and funded.
Personnel XR applications may have a number of implications for military personnel and force structure. If the U.S. military is able to achieve efficiencies in training or warfighting, for example, it may be able to shift personnel away from training units or reduce overall manpower requirements— with a smaller number of troops retained at higher levels of readiness. Conversely, XR applications may produce greater demand for maintainers or IT and cybersecurity
Military Applications of Extended Reality
https://crsreports.congress.gov | IF12010 · VERSION 4 · UPDATED
personnel. This demand could offset reductions occurring elsewhere in the force or even increase overall manpower requirements.
Cybersecurity Some analysts have raised concerns about the potential cybersecurity vulnerabilities of XR systems, particularly those that rely upon high-value-target databases for weapons maintenance, image classification, or other functions. If such systems are infiltrated, they could provide an adversary with critical information about U.S. weapons systems, as well as information about how the U.S. military trains, and thus how it intends to fight in the event of a conflict. XR systems used for warfighting could additionally enable an adversary to distort the common operational picture used to coordinate military actions or cause the system to misidentify people and platforms— potentially resulting in fratricide or unintended civilian casualties. Congress may request briefings on the findings
of DOD cybersecurity tests of XR systems or withhold funds from systems found to have significant vulnerabilities.
Related CRS Products
CRS In Focus IF11251, National Security Implications of Fifth Generation (5G) Mobile Technologies, by John R. Hoehn and Kelley M. Sayler. CRS In Focus IF10159, Cybersecurity, by Eric A. Fischer and Catherine A. Theohary.
Kelley M. Sayler, Analyst in Advanced Technology and Global Security
IF12010
Disclaimer
This document was prepared by the Congressional Research Service (CRS). CRS serves as nonpartisan shared staff to congressional committees and Members of Congress. It operates solely at the behest of and under the direction of Congress. Information in a CRS Report should not be relied upon for purposes other than public understanding of information that has been provided by CRS to Members of Congress in connection with CRS’s institutional role. CRS Reports, as a work of the United States Government, are not subject to copyright protection in the United States. Any CRS Report may be reproduced and distributed in its entirety without permission from CRS. However, as a CRS Report may include copyrighted images or material from a third party, you may need to obtain the permission of the copyright holder if you wish to copy or otherwise use copyrighted material.