Risk Assessment in the President's National Strategy for Homeland Security

Risk Assessment in the President's National Strategy for Homeland Security

Order Code RS21348
October 31, 2002
CRS Report for Congress
Received through the CRS Web
Risk Assessment in the President’s National
Strategy for Homeland Security
Rob Buschmann
Analyst in American National Government
Government and Finance Division
Summary
The President’s National Strategy for Homeland Security suggests that agencies
responsible for homeland security may use risk assessment techniques as tools to
accomplish their missions. Agencies may perform risk assessments on a national scale
in some areas, such as critical infrastructure. National homeland security risk
assessments may be beneficial, or even essential, but a number of limitations could
reduce or eliminate their usefulness. In addition, awareness of risk assessment’s basic
methods and potential shortcomings can help Congress to effectively oversee agencies
responsible for homeland security. This report is intended to provide background on the
use of risk assessment in the homeland security context and will not be updated.
Risk Assessment and Its Components: Defining the Terms
Risk assessment is a policy tool designed to help decision makers understand risks
to human welfare, safety, and property. Risk experts debate the exact meaning of the term
“risk,” but most accept two concepts as central: chance and consequence. Chance refers
to the probability that an adverse event will happen; consequence, to the loss associated
with that event. Risk assessment assists decision makers by providing quantitative and
qualitative estimates of the chance of, and possible consequences from, an adverse event.
Many existing laws, new legislation, and newspaper articles, however, refer to “threat”
and “vulnerability” assessments. How are these processes related to risk assessment?

Threat assessment usually refers to the process of identifying possible “initiators”
of adverse events, although the term, along with others in the risk assessment process, is
sometimes used loosely.1 A threat assessment is typically the earliest step of a risk
assessment – the threat assessment defines or identifies the dangerous actors, their intents
1 For more precise definitions of threat, vulnerability, and risk assessments as used here, see CRS
terrorism briefing book entry, “Assessing Risks,” at
[http://www.congress.gov/brbk/html/ebter225.html].
Congressional Research Service ˜ The Library of Congress

CRS-2
and capabilities, and what events they might initiate. The succeeding portions of the risk
assessment attempt to quantify the chance of specific consequences of those events.
While threat assessments focus on the initiators of events, vulnerability assessments
tend to focus on who or what is threatened and the nature and extent of the damage
adverse events might cause. A vulnerability assessment typically names the things,
persons, or populations that could be affected by an event. In addition, some vulnerability
assessments attempt to measure the “toughness” of targets — their susceptibility to certain
types of attack. For terrorists, the actual or perceived toughness of a target can be a factor
in determining which targets to attack. Therefore, in addition to consequences, a
homeland security vulnerability assessment might inform the assessor about the
probability of an attack as well as the probability of its success.
In short, a risk assessment usually contains threat and vulnerability components
which contribute to an overall picture of probability, and possible consequences, of an
adverse event. In this report, risk assessment means a process that includes both threat
and vulnerability assessments. Recent press coverage suggests that risk assessment and
its components may be significant tools in homeland security; the Bush Administration
has started to address this in its National Strategy for Homeland Security (the President’s
Strategy).2
Risk Assessment in the President’s Strategy
The President’s Strategy begins with a definition of homeland security that reads like
a prescription for risk assessment and risk assessment-based management: “Homeland
security is a concerted national effort to prevent terrorist attacks within the United States,
reduce America’s vulnerability to terrorism, and minimize the damage and recover from
attacks that do occur.”3 White House official Richard Falkenrath, director of policy and
plans for the Office of Homeland Security, has added that comparing threats to
vulnerabilities will, in the long run, be the most useful exercise for determining how to
allocate homeland security resources. However, it is unclear exactly how risk assessment
will be used, since the threat from terrorism seems to differ considerably from the threats
that risk assessment methods normally evaluate, such as cancer risks from environmental
pollutants or the chance of a bridge collapsing. Falkenrath has stated that the ability to
perform risk assessments for terrorist attacks does not exist adequately in the government
today and needs to be established.4
2 See Sydney J. Freedberg, Jr. and Siobhan Gorman, “Are We Safer?” National Journal, Aug.
10, 2002.
3 Executive Office of the President, Office of Homeland Security, National Strategy for
Homeland Security, p. 2. Available at
[http://www.whitehouse.gov/homeland/book/nat_strat_hls.pdf], visited 9/30/2002. The homeland
security bills, H.R. 5005 and its Senate counterparts, do not contain such a definition.
4 See The Brookings Institution, Homeland Security: The White House Plan Explained and
Examined, Sept. 4, 2002. Available at
[http://www.brookings.org/comm/events/20020904homeland.htm], visited 9/30/2002.

CRS-3
The President’s Strategy includes the following broad plans for dealing with
homeland security issues, plans that include what some experts indicate are good risk
assessment practices:
! Threats will be matched with vulnerabilities to reach a measure of risk;5
! Vulnerability and threat assessments will be maintained and updated to
keep up with a constantly changing environment;6
! Assessments of all national infrastructure risks will be done so “risk-
shifting” between sectors does not occur when management decisions are
made.7
The President’s Strategy focuses on six “critical mission areas”: intelligence and
warning, border and transportation security, domestic counterterrorism, protecting critical
infrastructure, defending against catastrophic terrorism, and emergency preparedness and
response. Two of these areas, intelligence and warning and protecting critical
infrastructure, appear to give risk assessment a central role. The other four areas in the
President’s Strategy also refer to risk assessment or its threat and vulnerability
components, though risk assessment does not seem to play as large a part in their
missions.
In intelligence and warning, the President’s Strategy envisions analysts and
operatives from various existing agencies combining their efforts to create a complete
threat assessment for domestic terrorism. The proposed Department of Homeland
Security (DHS) would have the lead in domestic vulnerability assessments, which DHS
would then match against the threats provided by the intelligence community to produce
a complete picture of the risks facing the United States.
In protecting critical infrastructure, the DHS would undertake comprehensive
national assessments of threats and vulnerabilities across all critical infrastructure and key
asset areas. The goal is to “build and maintain a complete, current, and accurate
assessment of vulnerabilities and preparedness of critical targets” and “to ensure we
reduce the overall risk to our country, instead of inadvertently shifting risk from one
potential set of targets to another.”8 In addition, through this initiative, the DHS would
be responsible for maintaining and updating this national assessment and using it to create
a current image of the threats and vulnerabilities facing the United States.
5 See the U.S. General Accounting Office, Combating Terrorism: Threat and Risk Assessments
Can Help Prioritize and Target Program Investments, GAO/NSIAD-98-74, April 1998.
6 See The National Research Council, Science and Judgement in Risk Assessment (Washington,
D.C.: Taylor and Francis, 1994); also relevant is the Council’s recommendation of an “iterative
approach,” which demands among other things that risk assessments increase in detail as the
consequences from the events they analyze increase in probability and severity.
7 See John D. Graham and Jonathan Baert Wiener, eds., Risk vs. Risk: Tradeoffs in Protecting
Health and the Environment (Cambridge: Harvard University Press, 1995) for an explanation of
how risks are changed and shifted between populations.
8 National Strategy, p. 31-33.

CRS-4
As a final note, the President’s Strategy’s use of risk assessment could provide a
framework for budgeting. Agencies involved in homeland security seem likely to be
instructed by the President to use risk assessment techniques to help set priorities and
assign resources. Many already do, such as the Coast Guard and the Customs Service.
Effective oversight of these agencies may require an understanding of how they use risk
assessment techniques in budget allocation.
Issues in Risk Assessment
To its advocates, risk assessment is a rational, systematic way to deal with society’s
risks, from the discovery of those risks to the formulation of policy options to reduce
them. Some experts, however, note that improper or incomplete use of risk assessment
can result in misinformed decision making and a loss of public faith in government.
While the President’s Strategy does not clearly explain how agencies focused on
homeland security are to employ risk assessment or its components, several generic issues
seem to arise whenever those techniques are used, especially on a national level.9 The
policy implications of these issues likely will be of interest to Congress as it reviews the
President’s Strategy and contemplates the enactment of legislation.
Uncertainty.
Every risk assessment is an estimate. A common criticism of risk assessment is that
few assessments are accompanied by statements of the confidence that risk assessors have
in their conclusions. Some critics of risk assessment observe that risk numbers are useless
without a clear presentation of the data upon which those numbers are based. Many risk
assessments, they argue, draw conclusions that are not the only ones that can realistically
be drawn from the data. Critics are concerned that once a conclusion is stated in a risk
assessment, further discussion about alternative conclusions tends to disappear along with
the uncertainty risk assessors have about the judgment they have made. On the other
hand, some advocates of risk assessment argue that uncertainty is hard to measure and that
constantly reporting uncertainty and ranges of possibility can unnecessarily confuse the
issues. A national-level risk assessment may lose meaning as a public policy tool and
public credibility if vulnerability and threat assessments are not accompanied by
uncertainty estimates. However, advocates are concerned that too much information may
cause a risk assessment to be disregarded as overly technical.
Risk Interactions.
Risk assessments generally analyze specific events or categories of events, producing
probabilities and consequences for each. Such a process can miss risk interactions, when
two or more risks influence each other. For example, should three separate events occur
in a single community — an attack on a mobile communications system, another on a
power grid, and the release of biological weapons in a crowded shopping center — the
calculations, and the results, change. When considered separately, each event’s chance
and consequences may be able to be estimated. If all of the attacks happen
9 A further discussion on some of these issues can be found in Bernard D. Goldstein’s chapter
titled “Risk Assessment as an Indicator for Decision Making” in Robert W. Hahn, ed., Risks,
Costs, and Lives Saved (Washington, D.C.: The AEI Press, 1996), p. 67-84.

CRS-5
simultaneously, however, one cannot simply add the consequences from each to find the
result; the whole could be greater than the sum of the parts. A power grid shutdown
would likely cause more confusion when combined with a biological attack, as the lights
go out in the shopping center; a communications outage could prevent emergency workers
from responding quickly to the attack. Some risk assessment methods are being
developed which will attempt to handle such complicated scenarios.10
Data Quality.
Some risk assessors attempt to find the chance and the consequences of an adverse
event to the most precise degree possible. The more numbers are used to create a risk
assessment, the more likely a risk assessor will look at events for which probability and
consequences can be easily quantified. Most often, easily quantified chance and
consequences come only after analysis of years of relatively common events when the
circumstances of those events and their effects can be clearly measured. Critics claim that
what risk assessment cannot measure, it implicitly disregards as unimportant, and with
rare events (such as terrorist attacks) this problem becomes significant. A national
valuation of key assets and critical infrastructure may take into account only the “most
measurable” damage, possibly de-emphasizing less easily quantified things, such as
environmental or psychological damage from terrorist attacks. In a related issue, the
Office of Management and Budget (OMB) recently released its Information Quality
Guidelines for federal agencies.11 The guidelines attempt to ensure that data used by
executive agencies in their decision making is accurate and produced by a rigorous
application of the scientific method. Some groups have expressed concern that these
guidelines may lead to agencies being limited by OMB to making decisions based only
on completely quantified data — perhaps limiting the effect of important, but less
measurable, issues. Congressional oversight of agencies using risk assessment in
homeland security decisions may need to include a review of the scope, quantity, and
validity of data used in those decisions.

Disregarding Very Rare Catastrophes.
Risk assessors sometimes, through choice or inadvertence, do not consider very
unlikely and catastrophic adverse events, especially if a large amount of time passes
without any such events occurring. Some scholars have noted that this tendency is
“particularly noteworthy for relatively complex systems and for estimates of extremely
low probabilities.... (I)t has even earned a name: the ‘disqualification heuristic’, or the
10 These methods are beginning to be discussed and used by the insurance industry. Models are
being developed that play out terrorist scenarios in the same manner as military war games. For
insurance models, see Joseph B. Treaster, “The Race to Predict Terror’s Costs,” New York Times,
Sept. 1, 2002, Money and Business/Financial Desk. For war game scenarios, see Karen Kaplan,
“The Sims Take On Al Qaeda,” Los Angeles Times, Nov. 2, 2001, Financial Desk.
11 Office of Management and Budget, “Guidelines for Ensuring and Maximizing the Quality,
Objectivity, Utility, and Integrity of Information Disseminated by the Office of Management and
Budget,” Federal Register, vol. 67, no. 84, May 1, 2002, p. 21779.

CRS-6
temptation to conclude that certain highly unpalatable outcomes simply couldn’t occur.”12
It seems that a terrorist attack with catastrophic consequences in the United States could
be the result of just such a complex system failure of law enforcement, security forces,
and intelligence. In other words, there is a possibility that as time goes by and very rare
and catastrophic events do not occur, an agency’s risk assessors may begin to disregard
the risk of those types of events entirely.13
Differing Agency Risk Assessment Methods.
Many of the agencies that will manage homeland security functions have existing
methods for risk assessment, many if not all of which have been tailored for the specific
problems those agencies face. A national risk assessment may alter or appear inconsistent
with these risk assessment methods if imposed as a new model. On the other hand, if the
national risk assessment relies upon each of these agencies for information about risks
without specifying a consistent method, the results could be difficult to integrate into a
meaningful whole, as each agency bases its assessments on its own criteria.
Distribution of Consequences.
A national risk assessment may not take regional considerations into account.
Consequences tend to be aggregated and are averaged over time and space in large-scale
risk analyses. Such aggregation may hide areas of extremely high and low vulnerability
and threat within a single “average” area or industry. This may be particularly significant
in homeland security, where symbolism may be as important to terrorists as large-scale
destruction. An attack on a target of minor national importance as defined by a national
risk assessment – perhaps a remote chemical or nuclear plant – may incapacitate a small
community and give terrorists a symbolic victory, widely reported by the media.
Conclusion
The use of risk assessment in homeland security could aid decision makers by more
clearly delineating the probability of, and consequences from, terrorist attacks. Risk
assessment is used in many forms throughout government and private industry to help set
priorities, anticipate problems, and allocate resources. However, Congress may want to
consider the many uncertainties and policy choices that are present in almost any risk
assessment, regardless of the area in which it is performed. The issues discussed here, and
others related to risk assessment, may be raised in congressional consideration of
homeland security.14
12 William R. Freudenburg, “Heuristics, Biases, and the Not-So-General Public: Expertise and
Error in the Assessment of Risks,” in Social Theories of Risk, edited by Sheldon Krimsky and
Dominic Golding (Westport, CT: Praeger Publishers, 1992), p. 232.
13 Ibid., p. 242.
14 For additional reading on the use of risk assessment in a specific issue area, see CRS Issue
Brief IB94036, The Role of Risk Analysis and Risk Management in Environmental
Protection, by Linda-Jo Schierow.