Augment Before You Automate: What the Field Evidence Says About AI in the Support Organization

Few functions hold as much promise from AI as customer service, and few carry as much at stake if that promise is delivered unevenly. The financial case is straightforward to describe. The brand case deserves equal attention, because every conversation a customer has with a company is, in their mind, a conversation about the company itself, and a growing share of those conversations now involve a machine.

For leadership teams, the central question is less about which technology to select and more about where AI belongs in relation to the people who serve customers. It can work alongside agents, supporting them in the moment. It can stand in front of them, handling conversations on its own. These approaches differ in their economics, their risks and their long-term effect on the talent a company will rely on. Each has capable advocates. The evidence points toward a more deliberate order of operations.

The principle, in a sentence: use AI first to make people better at the work, and then to take over the work that does not need people.

This perspective draws on the strongest field evidence currently available: a study of 5,179 support agents, two randomized experiments at Alibaba published in 2026, payroll and government data on the workforce, and the forecasts of the industry's leading analyst firm. We gave greater weight to randomized evidence than to forecasts, and we note where the evidence is limited.

Five Findings That Should Shape Your Rollout

1. Assistance delivers the most value to newer team members. A study of 5,179 agents found a 14% average productivity gain, and 34% for novice and low-skilled workers, with minimal impact on experienced ones.
2. Autonomy performs well on simple issues and less well on emotional ones. In a randomized experiment, AI handling standardized chats cut handling time sharply, but chats that reached a human because the customer was upset ran 40.8% longer and were rated far lower.
3. The same tool can strengthen some performers and unsettle others. A second experiment found lower service quality among top performers using an AI drafting tool.
4. The workforce is changing, beginning at the entry level. Payroll data shows a 13% relative employment decline for early-career workers in the most AI-exposed occupations, and US government data projects a 5% decline in customer service representative jobs by 2035.
5. Analysts and early adopters are signaling a rebalance. The leading analyst firm expects humans to remain in the loop, and one of the most visible early adopters has publicly adjusted its approach.

Each is explored below, along with what it may mean for the business and how we would sequence the work.

Finding 1: Assistance Lifts the Newest Agents the Most

The most cited study in this field is Erik Brynjolfsson, Danielle Li and Lindsey Raymond's Generative AI at Work, published in The Quarterly Journal of Economics in 2025. It followed 5,179 customer support agents who were given access to a generative AI assistant that suggested responses. Measured by issues resolved per hour, access to the tool increased productivity by 14% on average. The gain was 34% for novice and low-skilled workers, and the authors report minimal impact on experienced and highly skilled workers.

The authors offer suggestive evidence that the model works by spreading the practices of more able workers. They also report that AI assistance improved customer sentiment, increased employee retention and may lead to worker learning.

What it may mean for the P&L. Here is our own arithmetic, not a result from the paper. If agents resolve 14% more issues per hour and volume stays constant, the same workload needs about 12% fewer agent-hours. For the novice cohort, a 34% gain implies roughly 25% fewer hours for the same work. These figures set aside training, quality and the cost of the tool itself, so they are best read as an order of magnitude rather than a forecast. The strategic point is where the value concentrates: in ramp time, and in the cohort that turns over most. Higher retention, which the authors also report, adds to that value, because every agent who stays is an agent who does not have to be recruited and trained again.

Limits. This was one company, it tested assistance and not autonomy, and it was a chat operation, not a phone line.

Finding 2: One Tool, Different Effects Across the Team

A 2026 randomized field experiment at Alibaba, Generative AI in Action (Ni, Wang, Feng, Lu, Wang and Zhou), tested an assistant that drafts a diagnosis and a proposed solution at the opening of an after-sales chat. Agents could adopt the draft, change it or set it aside.

On average the tool improved speed and customer ratings, with no significant effect on customer retrials, a repeat contact used here as a proxy for whether the problem was actually solved. Lower-performing agents benefited most, which suggests the tool can narrow performance gaps. Top performers saw declines in both subjective and objective service quality. The authors link this to more time spent away from the chat, slower responses and immediate customer retrials, which they read as weakened service continuity.

What it may mean for governance. An average can describe a successful rollout while masking uneven effects underneath it. It may be worth asking every AI deployment to report results by experience band, and inviting the most experienced agents to decide when suggestions are useful to them. The paper is a preprint on arXiv, last revised in July 2026.

Finding 3: When AI Takes the First Conversation

A second Alibaba experiment, Agentic AI and Human-in-the-Loop Interventions (Wang, Zhu, Feng, Lu and Jia, May 2026), is closer to the choice many companies are now weighing. It randomly assigned 647 remote customer service workers on the Taobao platform (345 in the control group, 302 treated) and analyzed 680,676 chats. For the treated workers, a language-model agent handled standardized issues, which made up under 10% of chat volume. The workers supervised it and stepped in when an algorithm flagged a chat or when they judged it necessary. They also continued to handle the rest of their chats themselves.

Overall, average chat duration fell 3.2%, and 16.8% for the chats the AI was eligible to handle. Retrial effects were not statistically significant. Customer ratings on the AI-eligible chats fell by 0.412 points, while ratings on the workers' other chats rose by 0.091 points. The aggregate conceals what matters most, which is what happened inside each AI-eligible chat. Compared with similar chats handled entirely by humans:

Three observations stand out. The large time savings came from the 35% of chats the AI completed alone, though those chats were rated lower than comparable human chats. Technical handovers preserved quality even though they took longer, which suggests the human added real value there. And emotional handovers were the most difficult: slower, with more repeat contacts and noticeably lower ratings.

The authors explored why. In emotional escalations the workers sent fewer messages, took a smaller share of the conversation turns, and did less information seeking and solution providing. Their empathy was similar across types. The authors attribute the pattern to lower perceived recoverability, meaning the human inherits a conversation that already feels lost.

What it may mean for brand risk. The efficiency appears on the handle-time dashboard. The cost of a difficult handover appears elsewhere: in ratings, in repeat contacts and, potentially, in customer loyalty. The 8.6% of chats that escalated on emotion are where the brand is most exposed, and they merit a named owner, a dedicated measure and the company's most experienced people.

Limits. The paper's own introduction says it found no material change in overall service quality, which sits awkwardly beside the significant rating drops on AI-eligible chats, so the table is best read as reported. The rating scale is not described in the parts we reviewed, retrial is an internal proxy for resolution, and this is one platform, one AI system and a workforce of gig workers paid per chat. It is a 2026 preprint. We read it as a map of where to look, not as a forecast for any particular operation.

Finding 4: The Workforce Is Changing, Beginning at the Entry Level

Brynjolfsson, Chandar and Chen's Canaries in the Coal Mine? (August 26, 2025) used payroll records from ADP. It found that early-career workers aged 22 to 25 in the most AI-exposed occupations experienced a 13 percent relative decline in employment since generative AI became widely adopted, and it names customer service agents among the exposed occupations. The authors also distinguish between uses of AI. The declines appear where AI is used to automate tasks, not where it augments people. They caution that factors other than generative AI may be involved.

The US Bureau of Labor Statistics Occupational Outlook Handbook counts 2,666,000 customer service representatives in 2025 and projects a 5% decline (141,800 jobs) by 2035. It attributes the expected drop in demand, especially in retail trade, to tasks that continue to be automated, and notes that companies will keep in-house service centers for complex inquiries.

What it may mean for talent strategy. This is our inference, not a finding. Entry-level support roles are where many organizations develop the experienced agents who later handle the most complex and emotional conversations. If automation reduces that first rung, it is worth asking where tomorrow's senior agents will come from. A plan that narrows the entry level without another way to build judgment may deliver savings in the near term and a capability gap later.

Finding 5: Analysts and Early Adopters Signal a Rebalance

In a June 10, 2025 release, Gartner predicted that by 2027, half of the organizations that expected to significantly reduce their customer service workforce will abandon those plans. It cited a March 2025 poll of 163 service leaders in which 95% said they plan to keep human agents to help define AI's role. In a September 10, 2025 release, it predicted that none of the Fortune 500 will have fully eliminated human customer service by 2028. Analyst Kathy Ross put it this way: "We expect fewer human agents, but not completely agentless organizations." These are forecasts, and the September release does not describe its methodology.

The clearest real-world example is Klarna. After announcing that its chatbot did the work of 700 agents, its CEO told Bloomberg in May 2025 that cost had been too predominant a factor and had led to lower quality, and that it was critical to make clear to customers that there will always be a human if they want one. A spokesperson added that the company was "very much still AI-first" (Fortune). The lesson is not that AI fell short. It is that the balance between cost and quality moved, publicly, and in one direction.

Where Regulators Are Looking

Two recent developments show where oversight is heading, and both bear on how the human path is designed.

In California, AB 1609, the Right to Human Customer Service Act, was signed on September 28, 2026. According to Covington's summary and the bill record, it applies to large private businesses, bars representing a chatbot as a human, requires disclosure when a customer might be misled, and requires a good-faith effort to connect a customer to a human agent within 15 minutes of a request or to offer an appointment within one business day. The sources we reviewed do not state when the obligations begin, so that date is worth confirming with counsel.

In the United Kingdom, Ofcom's September 2026 review of AI in telecoms found limited evidence of systemic harm and said its existing rules are sufficient for now. It expects firms to be very clear when customers are dealing with generative AI, and said it would be concerned if AI acted as a gatekeeper to support channels. It also said it will act quickly if it sees harm.

The direction is consistent: be transparent that it is AI, and keep a person within reach. This perspective is not legal advice, but a plan that makes human access slow or hard to find now carries regulatory as well as reputational considerations.

Decisions Worth Making Early

Before approving a scope, a leadership team may find it useful to settle six questions.

1. Which issue types are suited to autonomy? Naming them explicitly helps. Standardized, low-emotion, reversible issues are a natural starting point, with everything else remaining with people by default.
2. What is the standard for reaching a human? A target time and a single, visible path, measured from the moment the customer asks, give the organization something concrete to design to. Regulation is moving in this direction as well.
3. Who owns the handover? One executive owner for the experience of customers who move from AI to a person, with authority over staffing, tooling and the measure, tends to prevent the gap from falling between teams.
4. How will talent be developed? If the simplest work is automated, newer agents will need another route to building judgment, and experienced agents will benefit from a role in design and review.
5. What are the stop conditions? Agreeing in advance which results would pause an expansion, such as lower ratings or more repeat contacts on AI-handled work than on comparable human work, keeps the decision objective.
6. What will the board see? Defining the scorecard below before launch means the story is not written after the fact.

A Scorecard for the Board

Handle time is the most readily available measure. In the Alibaba experiments it improved even where customer ratings did not. We would add the measures below, each viewed by issue type and by agent experience:

A Sequence That Follows the Evidence

Phase 1: assist. Introduce suggestions and summaries to agents, newest first. The gate: customer rating and repeat contact are no worse, and handling time improves, for each experience band.

Phase 2: automate the narrow. Let AI handle a short list of standardized issues, with a visible, fast path to a person. The gate: AI-handled conversations score at least as well as comparable human-handled ones on rating and repeat contact, not only on time.

Phase 3: widen with care. Add issue types one at a time, review emotional and complex handovers weekly, and pause expansion in any category where outcomes slip.

Ninety days, as we would plan it. In the first 30 days, establish baselines on every measure above, by issue type and experience band, and choose the pilot scope. In days 31 to 60, run the assistance pilot with the newest cohort alongside a comparison group. In days 61 to 90, review results against the gates, and only then decide whether to open a narrow autonomous pilot. This timetable is our recommendation, not a finding from the research.

What We Do Not Yet Know

1. Most of this evidence comes from chat, not phone calls. Voice adds timing, accents and background noise, and one 2026 benchmark, τ-Voice, reports that voice agents kept only 30% to 45% of their text performance.
2. The studies come from particular firms and platforms. Every organization's mix of issues, customers and agents will differ.
3. The two Alibaba papers and the Stanford paper are preprints or working papers.
4. Effects on skills, retention and morale over several years are still mostly suggestive.
5. The Gartner statements are predictions, not measurements.

A Closing Thought

The organizations that look best in three years may not be the ones that moved fastest. They are more likely to be those that used AI to raise the floor of every conversation, automated what was genuinely routine, and kept the human path quick and well resourced for the moments that matter most. That is ultimately a choice about sequence, ownership and measurement, and it tends to be easier to make before the dashboards begin to look favorable.

See It Working

At GoZupees, we build voice, chat and email agents for telecom, ISP, insurance and housing teams on our VersaTalk and Helix-CX platforms, connected to your CRM, billing and diagnostic systems, with human handover designed into the flow. Visit www.gozupees.com to schedule a live demo with our operations team.