Customer support: spread expertise without flattening judgment

The customer-support evidence supports using AI to help less experienced people draw on established practice, while measuring quality and outcomes separately for the specialists whose work already performs well.

Which study and which version should managers use?

The 2025 Quarterly Journal of Economics article studies a staggered introduction of an AI assistant among 5,172 customer-support agents. It reports a 15% average increase in issues resolved per hour, with substantial differences by experience and skill. Less experienced workers improved speed and quality; the most experienced workers had small speed gains and small quality declines. QJE25

The organization is not named in the paper’s public description, so this article does not invent a company identity. The published sample and result differ from the earlier NBER working paper’s 5,179 agents and 14% estimate. We use the journal version. The design concerns assisted human support in one setting, not a randomized test of fully autonomous agents in every service business.

The management choice is who needs what kind of support

A single average hides an important allocation decision. An inexperienced agent needs help identifying the problem and a credible next response. An experienced specialist can be slowed or misdirected by a suggestion that adds little to their existing judgment. The manager should therefore define the job of the assistant for each group rather than mandate the same interaction for everyone.

Our Decision Loop reconstruction is a proposed way to apply that lesson. It is not the study company’s documented process. A service-operations manager can use it to compare where the assistant improves accepted service, where it adds review burden and which questions still require expert escalation.

Use the Loop to make expertise transfer testable

Signal: establish the current issue mix, resolution standard and escalation pattern, separating newer agents from experienced specialists. Structure: identify the situations in which a newer agent lacks access to established practice, as distinct from situations where the policy itself is unclear. AI cannot solve a policy conflict by presenting an answer more fluently.

Options: compare assistance for selected issue classes, optional assistance for specialists and the current human-led route. Pilot: the service owner selects a bounded group and comparable issue types, with approved knowledge and explicit escalation. Measure: examine resolved issues alongside quality, repeat contact, escalation and review effort. Verify that an apparent resolution has not merely pushed work to the next interaction.

What should the service manager’s scorecard contain?

A useful scorecard shows the agent experience band, issue category, accepted resolution, unresolved or repeated contact, escalation reason and the effort required to review suggestions. Keep the knowledge version with the record. A policy correction should change what the assistant suggests, not leave the old guidance as a familiar default.

Treat proficiency as a learning question rather than an employment verdict. Ask whether people can explain why an answer applies and recognize an exception without blindly copying the suggestion. Use the evidence to improve coaching, source quality and escalation; do not turn a short pilot into automated decisions about individuals’ employment.

Why is 15% not the number to put in your business case?

The reported estimate belongs to the study’s organization, tool, rollout and work mix. Your baseline, language, issue complexity, customer expectations and source coverage differ. Even an improvement in issues resolved per hour does not directly establish a cash saving if staffing, demand and service obligations remain unchanged.

The strongest objection to broad rollout is the heterogeneous effect: the people who need the least assistance can face a different trade-off. Preserve an expert path and compare experience groups. If policy exceptions or specialist judgment dominate the work, improving the underlying knowledge and escalation route can be a better first intervention than increasing assistant usage.

What is the first useful local test?

Next week, a service-operations owner and quality lead can agree a narrow issue class, a current knowledge source and the definition of an accepted resolution. Review recent authorized examples with newer agents and specialists before introducing suggestions. Run a limited assisted route against comparable work and inspect follow-up outcomes, not only the first answer.

Before launch, the service owner and quality lead should name the first scheduled quality review, choose an observation window covering a normal service cycle, and agree local quality floors and extension criteria. Compare issue category, complexity, channel and experience bands with the baseline; retain unresolved and repeated cases in the count. Pause the affected group or issue path immediately when its quality floor is breached or acceptance cannot be verified. If the sample is too small to interpret, continue observation without expanding access.

Include the ongoing work needed to keep that route useful: knowledge maintenance, expert escalation, quality sampling and the relevant system costs. Assign an owner to policy changes and their propagation into suggestions. A short improvement that depends on uncounted specialist support does not establish the economics of a lasting service workflow.

Reconsider after the first comparable review period if specialist quality falls, repeat contacts rise or acceptance cannot be verified. Extend only for the groups and issue classes whose evidence supports it. The first positive value event is a person resolving an appropriate issue correctly with less avoidable friction; the average study result is context, not a quota or a promise.

Questions managers ask

What productivity improvement did the published study report?

The 2025 QJE article reports 15% more issues resolved per hour on average in its 5,172-agent sample. It is not a universal expected uplift.

Why should results be split by experience?

The study found different effects for less experienced workers and the most experienced workers, including small quality declines for the latter. One average conceals that management trade-off.

Does the study prove an autonomous chatbot can replace support staff?

No. It studies AI assistance for human agents in a particular organization. Autonomous service, staffing changes and your business case require separate evidence.

Sources and scope

QJE25: Generative AI at Work — Quarterly Journal of Economics; 2025-02-04. Primary peer-reviewed study; public full text.

Research checked 2026-10-27. Next review 2026-11-27. Suggested pilots are editorial proposals, not observed Upskill Pro customer results.