Customer Experience Research, Usability Testing, and the Paperwork Reduction Act

Agencies may conduct customer research to help understand how people experience the delivery of federal services and to improve such delivery and experiences. The Paperwork Reduction Act (PRA, codified at 44 U.S.C. §§3501-3521) generally imposes procedural and approval requirements on agencies’ efforts to collect information from the public, establishing procedures intended to minimize the burden associated with federal information collections, including from agency research. In 2024, the Government Accountability Office (GAO) reported that some viewed the PRA as a barrier to improving customer experience (sometimes abbreviated as CX), in part because it creates a months-long process for agencies to undertake before soliciting information from federal customers through surveys, questionnaires, feedback forms, and focus groups, among other methods.

Issued in December 2021, Executive Order 14058, “Transforming Federal Customer Experience to Rebuild Trust in Government,” discussed the need for agencies to understand how people experience the delivery of federal government services, defining customer experience as “the public’s perceptions of and overall satisfaction with interactions with an agency, product, or service.” Among its directives, the executive order instructed the Office of Management and Budget (OMB) to update and clarify its existing guidance to agencies on administrative flexibilities under the PRA to facilitate CX research.

On November 21, 2024, the Office of Information and Regulatory Affairs (OIRA), the OMB component that reviews and approves proposed information collections (44 U.S.C. §3504(c)(1)), issued a memorandum to agencies to clarify when a category of CX research known as usability testing is subject to the PRA’s information collection requirements (44 U.S.C. §3507). This In Focus puts into context OIRA’s memorandum, discusses the PRA’s information collection requirements, and concludes with some considerations for Congress.

Usability Testing

In its memorandum, OIRA defines usability testing as “a type of user research that consists of asking users to navigate or perform tasks with a paper or digital form or a federal website while observing users in person or remotely, and engaging with them by asking semi- structured or open-ended questions.” For example, users might be asked where on a website they would click to find or do something or be asked what was frustrating or confusing when trying to execute a web-based task.

According to GAO, usability testing is often used in software development that uses agile practices, where an

iterative and incremental approach “emphasizes early and continuous software delivery, fast feedback cycles, rhythmic delivery cadence, the use of collaborative teams, and measuring progress in terms of working software.” Agile development is often compared to waterfall methods, where software requirements are set at the beginning; software is designed, developed, and tested in linear and sequential phases; and work from a previous phase is generally not revisited or reconsidered, which may create uncertainty until testing. In this context of software development, usability is described as the effectiveness, efficiency, and overall satisfaction of a product or service to its end user.

Legislative Context

Congress has taken steps to improve the usability of public- facing agency content, particularly in efforts to modernize the federal government’s technology and internet use. The 21^st Century Integrated Digital Experience Act (P.L. 115- 336) directed agencies to develop and design public-facing websites, web-based forms and applications, and digital services around users’ needs and to continually test such products to ensure these needs are being addressed. In its implementation guidance (M-23-22), OMB advised agencies to employ usability testing and noted that well- designed digital forms can improve the accuracy and usability of information collected from the public, reduce business inefficiencies, enhance security, and reduce the costs and labor associated with using paper-based forms. For instance, research conducted by the General Services Administration (GSA) Office of Evaluation Sciences (OES) found that changing the placement of instructions on a form affected the likelihood that the form would be completed and submitted.

Federal Information Collections

Generally, if an agency wants to collect the same information from 10 or more members of the public (e.g., asking 10 people the same question), including when that information might be provided voluntarily, then the collection is likely subject to the PRA. This means that the agency has to follow certain steps and obtain OIRA’s approval before collecting the information. These steps— referred to as the “clearance process”—include internal agency review processes and publication of a Federal Register notice with a 60-day public comment period. The agency must then submit an information collection request (ICR) to OIRA, which is accompanied by a second notice in the Federal Register with a 30-day comment period. OIRA then has 60 days to approve or disapprove the agency’s request. Thus, an agency is to spend no less than 90 days going through the clearance process in addition to the time spent internally.

Customer Experience Research, Usability Testing, and the Paperwork Reduction Act

https://crsreports.congress.gov

Figure 1. Clearance Process for Information Collection Requests

Source: 44 U.S.C. §3506(c)(2), 44 U.S.C. §3507(b), and 44 U.S.C. §3507(g).

Applicability to Usability Testing In 2016, OIRA issued guidance on the applicability of the PRA to observations of users interacting with digital products and services. OIRA said that under its regulations implementing the PRA, facts or opinions obtained through direct observations or non-standardized oral communications during such observations are not considered to be information for the purposes of information collections and thus not necessarily subject to the PRA (5 C.F.R. §1320.3(h)(9)).

In its 2024 memorandum, OIRA reiterated that usability testing is often not subject to the PRA but was aware of confusion regarding the law’s applicability, which created a barrier to usability testing. Questions concerning applicability may also result in agencies going through the clearance process when seeking to conduct usability testing, resulting in a potentially avoidable delay in gathering information that could improve CX. OIRA stated that the PRA’s information collection requirements do not apply when agency staff or contractors (1) directly observe any number of individuals interacting with a form, website, or service or (2) ask questions to any number of individuals during usability testing.

OIRA also specified that the PRA would apply if the agency conducts usability testing to make causal inferences in a way that implies statistical rigor, including if the agency is planning to publicly disseminate descriptive statistics derived from the information collected from usability testing. For example, the GSA OES research discussed above would likely be subject to the PRA because it employs practices akin to a randomized controlled trial to determine whether the placement of instructions on a form has an effect on form submission.

Using the Results of Usability Testing

If usability testing is conducted on an underlying information collection activity that has been approved by OIRA (e.g., a web-based application form), then that underlying information collection is still subject to the PRA’s procedural and approval requirements. Certain changes can be made based on what is learned in usability testing without triggering the PRA’s clearance process (i.e.,

new public notice and comment periods). Substantial or material changes that might derive from the results of usability testing would trigger the PRA’s clearance process, as statute requires (44 U.S.C. §3507(h)(3)).

De Minimis and Non-Substantive Changes In previous guidance, OIRA has advised agencies of their ability to implement de minimis changes to information collections. These types of changes do not require further approval from OIRA before implementation. De minimis changes are “those that affect the look and feel of a collection, but do not change the nature or type of information collected” and do not increase the burden of a collection (e.g., the time it takes to complete a form). Examples of de minimis changes include changes to visual presentation (e.g., color choices), the size of a form field, and other cosmetic and design-related revisions. In its 2024 memorandum, OIRA said that “minor plain language edits” (e.g., reordering and simplifying instructions) can also be implemented at any time by an agency.

Non-substantive changes, however, require OIRA’s approval before an agency can proceed. OIRA is less specific in its 2024 memorandum and other PRA-related guidance on what non-substantive changes might include, except that these changes do not introduce new concepts and do not create new uses for the collected information beyond what was originally approved by OIRA. OIRA recommended that an agency submit a usability testing plan as part of an ICR that goes through the normal clearance process, describe the “categories of changes” that might be implemented based on the results of usability testing over the information collection’s approval period—generally, three years (44 U.S.C. §3507(g))—and discuss such plans and possible changes in its Federal Register notices. If such plans are submitted, OIRA states that it is more likely to approve “most changes stemming from usability testing … as though they are non-substantive changes under the PRA.”

Considerations for Congress

The PRA influences the timeline on which agencies collect information that could inform their efforts to improve federal CX. Congress could consider exempting from the PRA voluntary information collections that are conducted for the purposes of developing or improving digital services or in conjunction with an agency’s use of agile software development practices, thereby eliminating the clearance process for these information collections in these specific contexts. Congress has created exemptions for some voluntary information collections before—for example, in the context of public health emergencies (see 42 U.S.C. §247d(f))—and has considered doing so in the context of CX in the past (see from the 117^th Congress, S. 671 §5(d), H.R. 4688 §5(d), S. 1934 §12, and H.R. 3609 §12). Alternatively, Congress could amend the specifics of the clearance process. For example, Congress might reconsider the requirement of two comment periods. According to GAO, agencies receive “few, if any, comments” during a single 60-day comment period.

Natalie R. Ortiz, Analyst in Government Organization and Management

Customer Experience Research, Usability Testing, and the Paperwork Reduction Act

https://crsreports.congress.gov | IF12878 · VERSION 1 · NEW

IF12878

Disclaimer

This document was prepared by the Congressional Research Service (CRS). CRS serves as nonpartisan shared staff to congressional committees and Members of Congress. It operates solely at the behest of and under the direction of Congress. Information in a CRS Report should not be relied upon for purposes other than public understanding of information that has been provided by CRS to Members of Congress in connection with CRS’s institutional role. CRS Reports, as a work of the United States Government, are not subject to copyright protection in the United States. Any CRS Report may be reproduced and distributed in its entirety without permission from CRS. However, as a CRS Report may include copyrighted images or material from a third party, you may need to obtain the permission of the copyright holder if you wish to copy or otherwise use copyrighted material.