Case Studies of How Generative Artificial Intelligence is Changing Work

Case Studies of How Generative Artificial Intelligence is Changing Work

Case Studies of How Generative Artificial Intelligence is Changing Work

Updated May 16, 2025 (IN12537)

Generative Artificial Intelligence (GenAI) is a new technology that can generate new code, text, images, and other content based on machine learning systems and can be trained using large quantities of previous content. During the 118th Congress, there were multiple congressional hearings on how GenAI technology may affect work and workers.

GenAI became widely available only recently, and there are few case studies documenting how it is already changing the experience of work. This Insight summarizes two such case studies identified by CRS. In each of these studies, a new GenAI tool was intentionally rolled out to some workers before others so that researchers could compare workers who had access to the tool with similar workers who did not yet have access.

In these studies, GenAI tools boosted job performance more for less experienced software developers and customer service agents. These findings highlight how much is still unknown about which workers will be most affected by GenAI technology.

AI-Based Coding Assistance for Software Developers

In The Effects of Generative AI on High Skilled Work: Evidence from Three Field Experiments with Software Developers, Kevin Zheyuan Cui et al. describe the rollout of GitHub CoPilot to thousands of software developers at Microsoft in 2022, and to Accenture and an anonymous large electronics manufacturing company in 2023. GitHub CoPilot is a GenAI tool that makes suggestions of code and code documentation. Not every software developer given early access to the tool used it—between 60% and 70% of people who had early access tried it during the study period. Overall, those who had early access to GitHub CoPilot completed more coding tasks and completed them more successfully than their peers without early access. Among Microsoft software developers, the likelihood of trying the tool, the likelihood of continuing to use the tool after trying it, and the gains from tool access on productivity were all larger for more recent hires.

AI-Based Conversation Assistance for Customer Support Agents

In Generative AI at Work, Danielle Li, Erik Brynjolfsson, and Lindsey Raymond describe the rollout of an AI chat assistant (built on GPT-3 technology) to thousands of customer support agents in a large software company during 2020-2021. Customer support agents at this company—most based in the Philippines—helped U.S. clients deal with problems during calls lasting an average of 40 minutes. Customer support agents needed both technical knowledge and skill dealing with frustrated customers to do the job well. Their performance was measured by how quickly they successfully resolved customers' issues, as well as by customer satisfaction surveys.

The GenAI tool used at this company was trained on a dataset of previous customer-agent conversations from the best-performing agents. This tool listened to agents' discussions with customers and offered real-time suggestions for what agents should say, which agents could either use or ignore.

The agents who had access to the tool and were newer or less fluent in English used it to learn how do their job more like the best-performing customer service agents whose conversations were used to train the tool. The GenAI tool was most helpful for agents helping customers with "moderately rare" problems, in which new agents might have little experience but there was enough data to train the tool. Customer service agents with access to the GenAI tool who had two months of experience handled calls as well as agents without the tool who had six months of experience. Workers who used the tool to improve their performance continued to do better when the tool was unavailable. Agents based in the Philippines were able to interact with customers more like agents based in the United States.

Using the GenAI tool appeared to make the customer service job less stressful for agents who used it: agents who had access to the tool saw a reduction in negative customer speech (such as yelling or swearing at agents) and in customer requests to speak with a manager. Agents who used the tool were also less likely to leave the company.

A previous version of this Insight also summarized the paper Artificial Intelligence, Scientific Discovery, and Product Innovation. On May 16, 2025, the MIT Department of Economics released a statement saying, "the findings reported in this paper should not be relied on in academic or public discussions of these topics."