link to page 1 
Updated May 26, 2022
Military Applications of Extended Reality
Although commercial and consumer industries have been
Figure 1. Main Categories of Extended Reality
investing in extended reality (XR) for decades, recent
advances have expanded the number of potential
applications for the U.S. military. Indeed, in February 2022,
the Office of the Under Secretary of Defense for Research
and Engineering identified human-machine interfaces for
XR as 1 of 14 critical technology areas for the Department
of Defense (DOD). As DOD increases spending on XR and
related applications, Congress may consider the
implications for defense authorizations and appropriations,
military force structure, and cybersecurity.
Overview
XR encompasses three main categories of physical and
digital environments (Figure 1):
Virtual reality (VR), a fully immersive digital
environment (e.g., video games that place the user
within the virtual world of the game).
Augmented reality (AR), an overlay of digital
Source: Tutorials Link, “Difference Between AR, VR, MR,” at
objects on physical environments (e.g., Instagram
https://tutorialslink.com/Articles/Difference-Between-AR-VR-MR/973.
filters that overlay preset digital effects on a user’s
videos or photographs).
A number of advanced enabling capabilities, such as 5G
Mixed reality (MR), a hybrid of physical and
and edge computing—a type of computing that is done “at
digital environments in which physical and digital
or near the source of data”—are likely to expand XR
objects can interact. Unlike AR, MR could enable
applications in the future. These capabilities could improve
a user to manipulate physical or digital objects and
data rates, increase user capacity, and reduce latency (i.e.,
share their view of those objects with other users
time delay), all of which could support large-scale,
within the same mixed reality environment (e.g.,
networked applications. DOD is currently testing 5G-
collaboratively marking adversary troop locations
enabled applications of XR at Joint Base Lewis–McChord
on a projected digital map).
(WA) and Joint Base San Antonio (TX).
Military Applications of Extended Reality
The U.S. military is exploring a range of applications for
XR, with research and development programs in each of the
services. These applications include tactical, flight,
maintenance, medical, and other training, as well as
warfighting.
Training
According to Under Secretary of Defense for Research and
Engineering Heidi Shyu, DOD intends to leverage “AR/VR
and live training ... [that is being matured] by the gaming
industry” as a basis for developing its own tailored XR
programs. Doing so could enable the military to conduct
training exercises that are too costly or dangerous to
conduct in physical environments, as well as enable
servicemembers in distant locations to train together.
For example, the Army’s Synthetic Training Environment
(STE)—an XR training environment intended to
complement or integrate with live training—seeks to enable
soldiers “to train where they will fight, with the partners
https://crsreports.congress.gov
link to page 2 
Military Applications of Extended Reality
they will fight with, and in complex operational
Figure 2. Illustrative Battlefield Use of XR
environments to include dense urban, woodland, jungle,
desert, and sub-terrain, before the first fight begins.” STE is
to be designed to enable soldiers to more efficiently
“increase proficiency through repetition.” These factors
could, in turn, increase both readiness and lethality.
The Air Force uses XR for flight training—with the intent
of reducing cost, training time, and wear on aircraft. It is
also exploring XR for maintenance training and is in the
process of building virtual training hangars “to enable
training anywhere and anytime” on a variety of airframes.
Similarly, the Navy seeks to use XR to connect engineers
and maintainers, who could work together to address
maintenance issues across the globe in real-time.
DOD is also examining applications of XR for medical
Source: https://jasoren.com/augmented-reality-military/.
training. According to the Air Force, XR could “[increase]
Potential Issues for Congress
the availability of training, without a need to increase
Congress may consider a number of issues as it continues to
manpower availability for training setup.” This application
evaluate DOD investments in current and emerging military
could allow for distributed learning and create greater
applications of XR.
efficiencies for understaffed medical training courses.
Affordability
Warfighting
Military applications of XR vary considerably in terms of
The military is continuing to explore applications of XR for
up-front development costs, with one of the U.S. military’s
warfighting (Figure 2). It has long incorporated XR into the
largest XR programs, IVAS, costing up to $22 billion to
heads-up and helmet-mounted displays (HUD and HMD,
field over 10 years. Once fielded, however, XR systems
respectively) used by pilots and aircrew. These displays can
may reduce training costs by removing the need to
provide dynamic flight and sensor information intended to
increase the users’ situational awareness and improve
centralize personnel, use live ammunition, or operate
weapons’ targeting.
platforms. To assess these issues, Congress could direct an
In the case of the F-35 fighter aircraft’s
independent analysis of the potential benefits and
HMD, inputs from the F-35’s external cameras provide
drawbacks (e.g., cognitive overload) of XR training and
pilots with a 360-degree view of their surroundings; it also
warfighting applications against both their costs and their
displays night vision and thermal imagery—all of which
savings. This analysis may determine whether there are less
can be overlaid with the technical details (e.g., altitude,
costly, alternative means of achieving any identified
speed) of any detected objects.
benefits. Congress may also seek to obtain information
about the projected lifecycle costs—including maintenance
Likewise, the Army is developing the Integrated Visual
requirements—for XR systems.
Augmentation System (IVAS), a ruggedized (i.e.,
strengthened) MR headset based on Microsoft’s
Technological Maturity
commercially available HoloLens. Army documents state
While some applications of XR are relatively mature—
that IVAS “integrates next generation 24/7 situational
particularly those incorporating standalone AR—others are
awareness tools and high resolution digital sensors to
at a more nascent stage of development, require greater
deliver a single platform that improves soldier sensing,
levels of technology integration, or have otherwise
decision making, target acquisition, and target
experienced delays in fielding or testing. Congress may
engagement.” Army documents indicate that the system is
continue to seek information about the technological
to be eventually incorporated into both ground and air
maturity of XR systems and subsequently determine
vehicle platforms.
whether those systems warrant the requested funding levels.
Congress may also assess the technological maturity of any
necessary enabling capabilities to determine whether they
are sufficiently mature and funded.
Personnel
XR applications may have a number of implications for
military personnel and force structure. If the U.S. military is
able to achieve efficiencies in training or warfighting, for
example, it may be able to shift personnel away from
training units or reduce overall manpower requirements—
with a smaller number of troops retained at higher levels of
readiness. Conversely, XR applications may produce
greater demand for maintainers or IT and cybersecurity
personnel. This demand could offset reductions occurring
https://crsreports.congress.gov
Military Applications of Extended Reality
elsewhere in the force or even increase overall manpower
of DOD cybersecurity tests of XR systems or withhold
requirements.
funds from systems found to have significant
vulnerabilities.
Cybersecurity
Some analysts have raised concerns about the potential
Related CRS Products
cybersecurity vulnerabilities of XR systems, particularly
those that rely upon high-value-target databases for
CRS In Focus IF11251, National Security Implications of Fifth
weapons maintenance, image classification, or other
Generation (5G) Mobile Technologies, by John R. Hoehn and
functions. If such systems are infiltrated, they could provide
Kelley M. Sayler.
an adversary with critical information about U.S. weapons
CRS In Focus IF10159, Cybersecurity, by Eric A. Fischer and
systems, as well as information about how the U.S. military
Catherine A. Theohary.
trains, and thus how it intends to fight in the event of a
conflict. XR systems used for warfighting could
additionally enable an adversary to distort the common
Kelley M. Sayler, Analyst in Advanced Technology and
operational picture used to coordinate military actions or
Global Security
cause the system to misidentify people and platforms—
potentially resulting in fratricide or unintended civilian
IF12010
casualties. Congress may request briefings on the findings
Disclaimer
This document was prepared by the Congressional Research Service (CRS). CRS serves as nonpartisan shared staff to
congressional committees and Members of Congress. It operates solely at the behest of and under the direction of Congress.
Information in a CRS Report should not be relied upon for purposes other than public understanding of information that has
been provided by CRS to Members of Congress in connection with CRS’s institutional role. CRS Reports, as a work of the
United States Government, are not subject to copyright protection in the United States. Any CRS Report may be
reproduced and distributed in its entirety without permission from CRS. However, as a CRS Report may include
copyrighted images or material from a third party, you may need to obtain the permission of the copyright holder if you
wish to copy or otherwise use copyrighted material.
https://crsreports.congress.gov | IF12010 · VERSION 3 · UPDATED