Semiconductors and Artificial Intelligence
September 18, 2023
Semiconductors and Artificial Intelligence
The increasing popularity of artificial intelligence (AI) has
Early AI models used commercial, off-the-shelf logic chips
drawn congressional attention, and many Members are
called central processing units (CPUs) for training and
considering proposals to regulate the quickly evolving
inference. Although CPUs are still sufficient for inference,
landscape. Technical progress in AI has been enabled in
leading AI models now primarily train using graphics
large part by advances in the underlying computational
processor units (GPUs) originally designed for video
hardware—also known as semiconductors, integrated
rendering. GPUs enable parallel processing of information;
circuits, microelectronics, or simply chips—that offer
by contrast, CPUs process information serially. Parallel
increased processing power to improve the development of
processing allows the AI model to train faster using large
AI systems. This In Focus describes the types of
amounts of data by dividing tasks and executing them
semiconductors used in AI, concerns related to their supply
simultaneously. Additionally, many chip design firms are
chains, and challenges for the regulation of semiconductors
increasingly offering custom logic chips designed for
to promote U.S. competitiveness in AI.
particular applications, including AI training, called
application-specific integrated circuits (ASIC) or
Artificial Intelligence Models
accelerators. Logic chips used for AI applications are also
AI refers broadly to computational systems that can learn
referred to generally as AI chips.
from data and make decisions such as predictions,
recommendations, or classifications. AI systems can be
To train the largest AI models, many logic chips are
implemented for diverse applications, including
connected together in large clusters with other
speech/visual recognition, autonomous driving, robotic
semiconductor hardware (e.g., memory and networking
process automation, and as virtual assistants.
chips) in data centers or supercomputing facilities. For
example, Meta is building a supercomputer for AI research
A popular class of AI systems is deep neural networks,
that is anticipated to contain 16,000 GPUs, and a startup
which use algorithms, or models, to mimic neurons in the
called Inflection AI is building a cluster of 22,000 GPUs for
brain to identify complex patterns. This AI model typically
its AI model. Some supercomputers built by private firms
involves two stages: training and inference. In the training
such as Meta, Tesla, and NVIDIA are larger than many
phase, the model is fed data that can be labeled (e.g.,
nationally owned supercomputers around the world.
thousands of pictures of dogs to learn all the variations of
dogs) or unlabeled to identify patterns. In the inference
Companies that train AI models may purchase and maintain
phase, the trained model is used to enable predictions and
their own chip hardware infrastructure or may train their
guide decisions, such as autonomous driving systems
models remotely using the cloud by paying fees to access
recognizing dogs as obstacles to avoid. The training phase
the hardware they need. According to the Federal Trade
typically requires the most computational power.
Commission, “cloud services can be expensive and are
currently provided by only a handful of firms, raising the
Generally, the accuracy of AI models increases with
risk of anticompetitive practices.” Top cloud service
training on larger amounts of data, which in turn requires
providers in the United States for AI applications include
more computational power. Popular large language models,
Amazon Web AI Services, Microsoft Azure AI, and Google
such as GPT-3, are trained on billions or trillions of text
Cloud AI.
data to process and generate text. Given the large data sets
associated with AI models, some of the largest AI models
AI training typically benefits from improving two technical
can take weeks or months to train, using thousands of chips
parameters for AI chips: higher processing power and faster
and costing millions of dollars. These high costs are due in
chip-to-chip transfer speeds. A common metric used to
large part to the electricity required to operate the hardware. market the processing power of different AI chips is a
measurement of the number of mathematical operations a
Semiconductor Use in AI Models
chip can do in one second, calculated in trillions of
Semiconductors are tiny electronic devices designed to
operations per second (TOPS). Chip-to-chip transfer speeds
enable functions such as processing, storing, sensing, and
are generally reported by measuring how fast a chip can
moving data or signals. AI models employ different types
send information, or bytes, into and out of the chip in
of chips, including memory chips to store large amounts of
gigabytes per second (1 gigabyte is 1 billion bytes).
data and logic chips to process the data. According to
forecasts from Gartner, revenues from semiconductors used
Many large AI models, such as GPT models from OpenAI,
in AI may increase rapidly from around $44 billion in 2022
and leading AI research papers do not explicitly report the
to $120 billion in 2027.
amount of computational power used to train the AI model.
Additionally, there are no standard methods or tools to
measure the amount of computational power used to train
https://crsreports.congress.gov
Semiconductors and Artificial Intelligence
AI models, as TOPS may be calculated differently by
to its annual report. Similarly, in March 2023, NVIDIA
different companies and may not be the most optimal
marketed an H800 chip that does not require a license as an
metric to evaluate and compare AI models. Transparency in
alternative to the newest H100 products, which fall under
computing usage for AI training and standard methods for
the controls.
measuring computing power globally may support
regulatory efforts for AI.
Additionally, the October 2022 export controls restrict chip
manufacturing facilities globally from manufacturing
AI Chip Design and Manufacturing
certain advanced chips for Chinese-headquartered chip
U.S.-headquartered companies, both established firms and
design firms without a license if the manufacturer uses
start-ups, lead globally in the design of specialized logic
U.S.-origin technology or software (i.e., Advanced
chips for AI applications. However, the large majority of
Computing Foreign Direct Product Rule). As the United
these chip-design firms rely wholly on contract
States is a global leader in the production of semiconductor
manufacturing services to produce and package their
manufacturing equipment, this rule would apply to most
designs. As the highest performance AI chips require the
foreign chip manufacturing firms, including TSMC, which
most advanced manufacturing processes in the world, the
previously produced advanced chips for Chinese AI chip
majority of AI chip designers rely on the two logic chip
design companies such as Biren. The rules require licenses
manufacturing firms currently capable of producing their
to export certain advanced manufacturing equipment to
designs: Taiwan Semiconductor Manufacturing Company
chip manufacturers in China and Macau to impede the
(TSMC) and Samsung.
manufacturers’ ability to produce advanced chips.
The top AI chip designer by revenue and usage in AI
The export controls are designed to limit the ability of
research is U.S.-headquartered NVIDIA, one of the first
China and Macau to buy or produce certain advanced chips
companies to mass market GPUs in the early 2000s.
that can be used for AI applications. However, there are no
Leading GPU products from NVIDIA used in AI
controls on Chinese AI firms to use cloud service providers
applications, in order of increasing computational power,
inside or outside of the country to train AI models.
are marketed by the names V100 (2017), A100 (2020), and
H100 (2022). Each successor chip can transfer information
Selected Federal Actions and
into and out of the chip faster than its predecessor, enabling
Considerations for Congress
higher-speed communications between large clusters of
In January 2021, Congress enacted the National Artificial
chips and faster AI training. These higher performance
Intelligence Initiative Act of 2020 (Division E of P.L. 116-
metrics may enable an AI model to train faster than it
283), which seeks to advance U.S. leadership in AI research
would using other commercially available GPUs and, in
and development. Part of the act seeks to establish a
turn, may lead to relatively lower costs.
roadmap for a National AI Research Resource, a shared
research infrastructure for AI researchers and students.
Top U.S.-headquartered AI chip design start-ups include,
Directed by Executive Order 13859, the National Institute
by company valuation, SambaNova, Cerebras, and
of Standards and Technology conducted a study that
Graphcore. Smaller entities such as start-ups often face
recommended the federal government “commit to deeper,
challenges to prototyping and producing their designs due
consistent, long-term engagement in AI standards
to the high cost of and limited access to contract
development activities,” including the development of
manufacturing services from companies such as TSMC. As
“metrics to quantifiably measure and characterize AI
competitiveness in AI benefits from advancements in chip
technologies, including but not limited to aspects of
hardware, promoting access to prototyping and
hardware and its performance.” As Congress considers
manufacturing services for U.S.-based firms may boost
legislation to regulate the AI landscape, standard methods
long-term innovation.
and tools to measure, for example, how much
computational hardware AI models used for training may
Export Controls on AI Chips
help govern these technologies.
In October 2022, the Department of Commerce
implemented controls that require licenses for exports to
In August 2022, President Biden signed P.L. 117-167,
China and Macau of certain advanced logic and other chips
known as the CHIPS and Science Act. The act appropriated
that can be used for applications such as AI training and for
$39 billion to expand domestic semiconductor
building supercomputers. Controls apply to those logic
manufacturing capacity and $11 billion for research and
chips with chip-to-chip transfer speeds of 600 gigabytes per
development of next-generation semiconductor
second or more and computational power over 600 TOPS.
technologies. Congress may exercise its oversight authority
with respect to the effectiveness of expanding domestic
Under this definition, exports of leading AI chips, including
manufacturing capacity for advanced logic chips and
NVIDIA’s A100 and H100, to China and Macau are
improving manufacturing accessibility for smaller entities.
restricted. In recent years, China accounted for about a
quarter of total annual revenues for NVIDIA. In November
Additionally, as many AI models are trained using cloud
2022, NVIDIA began marketing an A800 chip, which had a
services, Congress may consider export control reforms that
lower chip-to-chip transfer speed of 400 gigabytes per
enable the Department of Commerce to exercise regulatory
second (compared with 600 gigabytes per second in the
authority over providers of cloud services that sell access to
A100), to “provide alternative products not subject to the
large amounts of computational power.
new license requirements” to customers in China, according
https://crsreports.congress.gov
Semiconductors and Artificial Intelligence
IF12497
Manpreet Singh, Analyst in Industrial Organization and
Business
Disclaimer
This document was prepared by the Congressional Research Service (CRS). CRS serves as nonpartisan shared staff to
congressional committees and Members of Congress. It operates solely at the behest of and under the direction of Congress.
Information in a CRS Report should not be relied upon for purposes other than public understanding of information that has
been provided by CRS to Members of Congress in connection with CRS’s institutional role. CRS Reports, as a work of the
United States Government, are not subject to copyright protection in the United States. Any CRS Report may be
reproduced and distributed in its entirety without permission from CRS. However, as a CRS Report may include
copyrighted images or material from a third party, you may need to obtain the permission of the copyright holder if you
wish to copy or otherwise use copyrighted material.
https://crsreports.congress.gov | IF12497 · VERSION 1 · NEW