BCG: test the task before you delegate the decision

The BCG consultant experiment makes a practical case for testing the task boundary before expanding AI delegation, because strong drafting performance did not guarantee correct analysis on a different task.

What is the real evidence behind the jagged frontier?

Harvard Business School’s institutional summary describes an experiment involving 758 BCG consultants. Participants benefited on tasks within the tested model’s capability frontier, but AI users were less likely to reach the correct answer on a task outside it. The summary reports a 19-percentage-point correctness disadvantage on that outside-frontier task. The study used a 2023 GPT-4 setting. HBS23

This article relies on the accessible HBS research summary; the working-paper PDF could not be fetched during research. It does not present that PDF as independently read. The important boundary is between task types under those experimental conditions, not between “good users” and “bad users,” and not a current accuracy benchmark for every AI system.

Frame the decision as an allocation of work

A strategy-team manager should ask which part of a management problem can be prepared by AI and which needs an independent evidence route. Producing candidate options, identifying inconsistencies in a source pack and deciding whether the investment case is correct are separate tasks. A successful option-generation exercise does not authorize autonomous validation of the investment.

The proposed reconstruction applies the Upskill Decision Loop to that allocation problem. It does not claim BCG used this framework. The reader’s output is a delegation map: what AI prepares, how the result is checked, who owns the consequential conclusion and what stops the workflow.

Map Signal, Structure and Options to observable checks

Signal: gather representative tasks from a recurring strategy or operating review, including a case where the available evidence points away from the attractive recommendation. Keep the sources and the expected checks visible. Structure: distinguish language transformation, evidence extraction, numerical calculation and business judgment rather than treating the entire brief as a single unit.

Options: compare manual preparation, AI preparation with an independent check, and a more narrowly scoped automated task. Choose the route against the error that matters, the cost of checking it and the consequences of an undetected mistake. The relevant choice is not whether the organization is “pro-AI”; it is where the current workflow can support justified delegation.

Use Pilot and Measure to test the whole review route

Pilot: the strategy lead and a reviewer who can access the original evidence compare the routes on a fixed task set. Record both the errors in the first output and the errors that survive the proposed review process. Measure: evaluate accepted correctness and total preparation-plus-review effort. A long review process that still accepts the wrong answer is not a successful safety mechanism.

Build the working brief around explicit statements: task, source, expected result or check, AI role, independent reviewer, acceptance decision and next re-evaluation. Mark unresolved cases. A model update or new data source changes the tested configuration and requires the owner to decide what must be tested again.

The strongest objection is that the frontier keeps moving

A 2023 experiment cannot tell a manager exactly where a 2026 model succeeds. Treating the historical task boundary as permanent would be the wrong lesson. A model can improve at a previous failure and still introduce a different failure through new tools, sources or a more complex sequence of actions.

The response is a living local evaluation, not permanent distrust or permanent permission. A small, relevant task set can support a bounded decision. It cannot certify every future use case. The manager should also protect the reviewer’s independence: repeating the same model’s reasoning in a second chat is not the same as checking the original evidence.

What can a strategy lead do next week?

Select a recurring analytical brief and ask its decision owner which error would reverse the recommendation. Assemble a small representative task set from authorized material and define the check before comparing outputs. Run AI preparation in shadow mode, with no external commitment, while the existing process provides a comparison.

Keep a separate held-out task set that is not used to refine prompts or show model answers. Repeated improvement on familiar evaluation questions can reflect tuning to the test rather than a safer delegation boundary. The independent reviewer owns the held-out results and checks new configurations before those cases become routine training examples.

In this pilot, a cycle comprises preparation of the selected analytical brief, independent evidence review and a recorded acceptance decision. The strategy lead schedules the checkpoint after two complete instances, retaining failed and unfinished work in the record. An immediate failure is handled when it occurs, not postponed until that checkpoint.

Suspend the affected delegation immediately when a consequential error survives review or a required check fails. Use the two-cycle checkpoint to assess the remaining, still-authorized work; investigate whether a different source or deterministic calculation can make the task reliable. Extend only when the current configuration meets the agreed standard on relevant and exception cases. The usable first outcome is a clearer allocation of judgment, not a claim that a model has become generally trustworthy.

Questions managers ask

What is a jagged AI frontier?

An uneven boundary of capability: a model can perform well on some tasks and poorly on tasks that appear similar. The cited experiment demonstrates this within a specific model and task setting.

Is the 19-point result a current accuracy estimate?

No. It is an experimental comparison described in HBS’s summary of the 2023 study, not a benchmark for the model or workflow you use today.

What should managers measure in a delegation pilot?

The errors that survive the actual review process, accepted correctness, and total preparation-plus-review effort on representative tasks, including exceptions.

Sources and scope

HBS23: Humans vs. Machines: Untangling the Tasks AI Can (and Cannot) Handle — Harvard Business School Working Knowledge; 2023-11-22. Institutional primary research summary, read in full public HTML.

Research checked 2026-10-27. Next review 2026-11-27. Suggested pilots are editorial proposals, not observed Upskill Pro customer results.