Is Your Agent Spending the Same Compute on Every Task? Build Adaptive Agents That Escalate Only When Needed

A production agent should not spend the same amount of compute on every task.

Some requests are easy:

• parse a known structure,
• retrieve a known record,
• make a deterministic transformation,
• call one reliable tool,
• run one test,
• answer from a high-confidence source.

Others are not:

• the task is ambiguous,
• multiple reasoning paths disagree,
• the cheap model fails verification,
• the search space is large,
• the action has expensive consequences,
• the evidence is incomplete,
• the current plan is not making progress.

If every task goes through the full advanced-agent stack, the system becomes slow, expensive and fragile.

If every task stays on the cheapest path, difficult cases fail unnecessarily.

The useful architecture is therefore not:

cheap agent
or
expensive agent

It is:

start cheap
   ↓
measure uncertainty / failure
   ↓
choose the smallest escalation
   ↓
verify again
   ↓
stop when evidence is sufficient

This is adaptive compute allocation.

The core rule is:

Do not spend more inference compute because a task feels difficult. Spend more when a measurable signal says the current path is insufficient.

That distinction turns a pile of advanced techniques into a runtime policy.

The problem: fixed compute budgets are usually wrong

Suppose every coding request always does this:

frontier model
  ↓
5 reasoning samples
  ↓
Tree of Thoughts
  ↓
MCTS
  ↓
3 critics
  ↓
final verifier

You may get a sophisticated system.

You may also get a very expensive way to rename a variable.

The opposite design is also bad:

one cheap local model
       ↓
      done

That path may be excellent for repetitive repository tasks and inadequate for a subtle concurrency bug.

The right question is not:

Which architecture is best?

It is:

Which architecture should this task receive at this point in the trajectory?

That is a routing problem over compute policies.

Adaptive agents are control systems

An adaptive agent has at least five pieces:

current state
    ↓
cheap policy
    ↓
result + evidence
    ↓
uncertainty / failure signals
    ↓
escalation controller
    ↓
next compute policy

The controller may choose among:

no escalation
more tool evidence
more reasoning steps
more samples
Best-of-N
specialist expert
stronger model
Tree of Thoughts
MCTS
adversarial review
human escalation

The important point is that these are actions available to the runtime.

They are not a fixed ladder that every request must climb.

Start with the cheapest path that could plausibly succeed

A good default is:

exact deterministic path
        ↓ if unavailable
cheap model / local model
        ↓ if insufficient
stronger reasoning or more evidence
        ↓ if insufficient
search / specialists / frontier model

For example, a coding assistant might start with:

git status
  ↓
static inspection
  ↓
local model patch
  ↓
run tests

If the tests pass and the required behavior is verified, stop.

There is no reason to invoke MCTS merely because MCTS exists.

If the tests fail, the runtime now has a concrete signal that escalation may be justified.

What should trigger escalation?

The strongest escalation signals come from the environment.

Examples:

• required test failed,
• schema validation failed,
• browser state did not change as expected,
• deployment health degraded,
• retrieved evidence conflicts,
• required source is missing,
• plan made no measurable progress,
• critic found a verified defect,
• verifier returned UNKNOWN,
• route confidence is low,
• multiple samples disagree materially,
• a tool action repeatedly fails,
• the current expert has low historical success on this task class.

Weak signals include:

• the model says the task is hard,
• the model says it is uncertain,
• the model requests more thinking,
• the response is long,
• the prompt contains many words.

Model self-confidence can be useful telemetry.

It should not be the only trigger.

External failure is a better trigger than internal uncertainty

Imagine two tasks.

Task A

The model says:

I am only 60% confident.

But the deterministic test suite passes every required acceptance test.

There may be no reason to escalate.

Task B

The model says:

I am highly confident this is correct.

But the integration test fails.

The runtime should escalate.

Therefore:

model confidence
      <
external verification evidence

whenever external evidence exists.

A simple escalation contract

Represent escalation decisions explicitly.

from dataclasses import dataclass, field
from enum import Enum


class Escalation(str, Enum):
    NONE = "none"
    MORE_EVIDENCE = "more_evidence"
    MORE_SAMPLES = "more_samples"
    STRONGER_MODEL = "stronger_model"
    SPECIALIST = "specialist"
    SEARCH = "search"
    ADVERSARIAL_REVIEW = "adversarial_review"
    HUMAN = "human"


@dataclass
class RuntimeSignals:
    verified: bool = False
    verification_unknown: bool = False
    failed_checks: int = 0
    disagreement: float = 0.0
    progress_delta: float = 0.0
    route_confidence: float = 1.0
    repeated_failures: int = 0
    risk: float = 0.0


@dataclass
class EscalationDecision:
    action: Escalation
    reason: str
    estimated_cost: float = 0.0
    evidence: list[str] = field(default_factory=list)

The runtime should be able to answer:

Why did this task receive additional compute?

If the answer is only “the model wanted to think more,” the policy is not very inspectable.

Deterministic escalation rules are a strong baseline

Do not jump immediately to an LLM deciding which LLM to call next.

A simple controller may outperform a learned router.

def choose_escalation(s: RuntimeSignals) -> EscalationDecision:
    if s.verified:
        return EscalationDecision(
            action=Escalation.NONE,
            reason="required verification passed",
        )

    if s.failed_checks > 0:
        return EscalationDecision(
            action=Escalation.MORE_EVIDENCE,
            reason="external verification failed",
            evidence=[f"failed_checks={s.failed_checks}"],
        )

    if s.disagreement > 0.35:
        return EscalationDecision(
            action=Escalation.MORE_EVIDENCE,
            reason="candidate disagreement is high",
            evidence=[f"disagreement={s.disagreement:.2f}"],
        )

    if s.route_confidence < 0.6:
        return EscalationDecision(
            action=Escalation.SPECIALIST,
            reason="expert routing confidence is low",
        )

    if s.repeated_failures >= 2:
        return EscalationDecision(
            action=Escalation.STRONGER_MODEL,
            reason="cheap path failed repeatedly",
        )

    if s.verification_unknown and s.risk > 0.7:
        return EscalationDecision(
            action=Escalation.HUMAN,
            reason="high-risk task lacks sufficient verification evidence",
        )

    return EscalationDecision(
        action=Escalation.NONE,
        reason="no escalation trigger fired",
    )

This controller is simple.

That is a feature.

You now have a baseline against which a learned escalation policy must prove itself.

Escalation should be specific

A common mistake is:

failure
  ↓
use a bigger model

But not every failure is a model-capacity failure.

Different failures imply different escalations.

This is why adaptive compute is not just model routing.

It is mechanism routing.

Local model → frontier model cascades

One of the most useful real-world patterns is:

local model
    ↓
external verification
    ├── PASS → stop
    └── FAIL / UNKNOWN
             ↓
      frontier model
             ↓
         verify again

This gives local models the easy workload while reserving expensive models for cases where they can add value.

A more advanced cascade can be:

deterministic tool
      ↓
local model
      ↓
local specialist
      ↓
frontier generalist
      ↓
frontier specialist / search

But each level should have a measurable reason to exist.

Adaptive reasoning depth

Reasoning itself can be budgeted dynamically.

Instead of:

always generate 20 reasoning steps

use:

reason
  ↓
check whether uncertainty decreased
  ↓
continue only if another step could change the decision

One simple metric is decision change.

Ask:

Did the additional reasoning produce a new hypothesis, new evidence request, new constraint, or different action?

If not, continued reasoning may just be token expansion.

Track:

reasoning steps
useful state changes
new evidence requests
verified-success delta

A long reasoning trace is not evidence that the compute was useful.

Adaptive sampling

Self-consistency does not require a fixed sample count.

Start with a small number:

N = 2 or 3

Measure agreement.

If the answers converge and external evidence supports the result, stop.

If they disagree materially, sample more or gather stronger evidence.

def next_sample_count(current_n: int, agreement: float) -> int:
    if agreement >= 0.9:
        return current_n
    if agreement >= 0.7:
        return min(current_n + 2, 7)
    return min(current_n * 2, 10)

Again, the important signal is not disagreement alone.

It is whether additional samples improve the final verified decision.

Adaptive search width

Tree search can also change width dynamically.

Use a narrow frontier when one branch is clearly superior under strong evidence.

Use a wider frontier when scores are close or uncertain.

clear evidence
    ↓
beam width 1–2

uncertain evidence
    ↓
beam width 4–8

This is preferable to fixing a large beam width for every task.

You can define:

def beam_width(score_margin: float, evaluator_variance: float) -> int:
    if score_margin > 0.4 and evaluator_variance < 0.1:
        return 2
    if score_margin > 0.2:
        return 4
    return 8

The exact numbers are application-specific.

The principle is general:

Spend search compute where the frontier is ambiguous.

Adaptive MCTS budgets

MCTS is especially expensive if treated as a default.

A better strategy is:

beam search first
     ↓
measure shallow-score reliability
     ↓
if promising branches have unstable / delayed value
     ↓
allocate MCTS budget

MCTS should be an escalation triggered by evidence that greedy or beam-like pruning is unreliable.

Possible triggers:

• low rank stability across evaluator samples,
• high pruning regret in similar historical tasks,
• delayed objective signals,
• long-horizon action consequences,
• weak correlation between shallow score and final verified result.

This is much stronger than:

Complex task → run MCTS.

Verification-driven escalation

The cleanest adaptive architecture is often organized around the verifier.

attempt
  ↓
verify
  ├── PASS → return
  ├── FAIL → diagnose → targeted escalation
  └── UNKNOWN → gather more evidence / escalate

The verifier becomes a control signal.

That means verification is no longer merely the final gate.

It also helps decide how much more work is warranted.

Failure type should control the next action

Suppose a coding agent fails.

The naive policy is:

retry the model

The adaptive policy asks what failed.

Syntax failure

Use parser/compiler evidence.

Test failure

Inspect the failing test and affected code path.

Missing repository context

Retrieve relevant symbols/files.

Wrong architecture assumption

Escalate to broader repository reasoning or specialist review.

Repeated patch failure

Try alternative strategies or search.

High-risk migration

Require stronger verification or human review.

The recovery mechanism should match the failure.

Avoid escalation loops

Adaptive agents introduce a new failure mode:

fail
 ↓
escalate
 ↓
fail
 ↓
escalate
 ↓
fail
 ↓
escalate forever

Every adaptive system therefore needs hard budgets.

Track at least:

max escalation levels
max model calls
max tool calls
max wall time
max tokens
max monetary cost
max search nodes

And return an explicit termination reason.

class Termination(str, Enum):
    VERIFIED_SUCCESS = "verified_success"
    VERIFIED_FAILURE = "verified_failure"
    BUDGET_EXHAUSTED = "budget_exhausted"
    NO_PROGRESS = "no_progress"
    HUMAN_REQUIRED = "human_required"
    EVIDENCE_UNAVAILABLE = "evidence_unavailable"

A budget exhaustion is not a success.

It is a named outcome.

Budget as first-class state

Keep compute budget in runtime state.

@dataclass
class ComputeBudget:
    model_calls_left: int
    tool_calls_left: int
    token_budget: int
    cost_budget: float
    search_nodes_left: int

Every escalation consumes budget.

The controller can then ask:

Is the expected value of another escalation worth the remaining budget?

This is much more useful than an unbounded while not done: loop.

Expected value of escalation

A production runtime can estimate whether escalation is worthwhile.

Suppose historical telemetry says:

local model verified success:      82%
frontier escalation success:       +9 percentage points
average extra cost:                $0.11
average extra latency:             4.5 s

For low-value tasks, escalation may not be worth it.

For a deployment-changing action, it may be trivial compared with the cost of failure.

Conceptually:

expected value
=
probability escalation changes outcome
× value of correct outcome
− escalation cost
− latency penalty
− operational risk

You do not need perfect monetary estimates.

Even rough task classes are better than pretending every request has identical value.

Risk-aware compute allocation

Risk should influence escalation.

Compare:

"rename this local variable"

with:

"apply this production database migration"

Even if model confidence is identical, the second task deserves stronger verification.

A useful policy separates:

difficulty
risk
uncertainty

They are not the same thing.

A simple task can be high risk.

A difficult task can be low risk.

Compute policies should consider both.

Confidence is useful only if calibrated

If an escalation controller uses confidence, measure it.

Suppose the router reports:

confidence = 0.9

Does that correspond to roughly 90% verified success?

If not, the number is decorative.

Track calibration curves:

predicted confidence bucket
vs
actual verified success

Useful metrics include:

Brier score
expected calibration error
false-confidence rate
false-uncertainty rate

But even a calibrated model confidence should remain secondary to direct environment evidence when available.

Adaptive routing between experts

The previous post on agent-level Mixture of Experts introduced expert routing.

Adaptive agents add another dimension:

route
 ↓
execute
 ↓
observe outcome
 ↓
reroute if evidence says current expert is insufficient

For example:

code formatter
   ↓ FAIL
static analyzer
   ↓ evidence insufficient
local coding model
   ↓ tests fail
frontier coding specialist

The route changes because the state changed.

This is more powerful than classifying the task once at the beginning.

Learning from historical success

Adaptive policies can start deterministic.

Later, historical data may improve them.

Record:

task features
initial route
verification result
escalation sequence
models used
experts used
search budget
cost
latency
final verified result

Then ask:

For tasks like this, which escalation usually changes failure into verified success?

That can eventually support learned routing.

But this introduces an important distinction:

Memory records what happened. Learning changes future policy because of what happened.

The next posts in this series will go deeper into that boundary.

A complete adaptive-agent skeleton

Here is a compact provider-agnostic runtime.

from dataclasses import dataclass, field
from enum import Enum
from typing import Any, Callable


class Verdict(str, Enum):
    PASS = "pass"
    FAIL = "fail"
    UNKNOWN = "unknown"


class ComputeLevel(str, Enum):
    CHEAP = "cheap"
    DEEP_REASONING = "deep_reasoning"
    MULTI_SAMPLE = "multi_sample"
    SPECIALIST = "specialist"
    SEARCH = "search"
    FRONTIER = "frontier"


@dataclass
class Verification:
    verdict: Verdict
    evidence: list[str] = field(default_factory=list)
    failed_checks: list[str] = field(default_factory=list)


@dataclass
class Attempt:
    level: ComputeLevel
    output: Any
    verification: Verification
    cost: float = 0.0
    latency_ms: int = 0


@dataclass
class Budget:
    max_attempts: int = 5
    max_cost: float = 1.00


class AdaptiveAgent:
    def __init__(
        self,
        policies: dict[ComputeLevel, Callable[[dict], Any]],
        verifier: Callable[[dict, Any], Verification],
    ):
        self.policies = policies
        self.verifier = verifier

    def next_level(self, attempts: list[Attempt]) -> ComputeLevel | None:
        if not attempts:
            return ComputeLevel.CHEAP

        last = attempts[-1]

        if last.verification.verdict == Verdict.PASS:
            return None

        if last.level == ComputeLevel.CHEAP:
            if last.verification.verdict == Verdict.UNKNOWN:
                return ComputeLevel.MULTI_SAMPLE
            return ComputeLevel.DEEP_REASONING

        if last.level == ComputeLevel.DEEP_REASONING:
            return ComputeLevel.SPECIALIST

        if last.level == ComputeLevel.MULTI_SAMPLE:
            return ComputeLevel.SPECIALIST

        if last.level == ComputeLevel.SPECIALIST:
            return ComputeLevel.SEARCH

        if last.level == ComputeLevel.SEARCH:
            return ComputeLevel.FRONTIER

        return None

    def run(self, task: dict, budget: Budget) -> tuple[Any, list[Attempt]]:
        attempts: list[Attempt] = []
        total_cost = 0.0

        while len(attempts) < budget.max_attempts:
            level = self.next_level(attempts)

            if level is None:
                break

            output = self.policies[level](task)
            verification = self.verifier(task, output)

            attempt = Attempt(
                level=level,
                output=output,
                verification=verification,
            )
            attempts.append(attempt)
            total_cost += attempt.cost

            if verification.verdict == Verdict.PASS:
                return output, attempts

            if total_cost >= budget.max_cost:
                break

        best = attempts[-1].output if attempts else None
        return best, attempts

This skeleton is intentionally simple.

A production system would make next_level() depend on richer evidence.

But the architecture is visible:

attempt
 ↓
verify
 ↓
choose next compute policy
 ↓
repeat within budget

Do not treat the escalation ladder as mandatory order

The sample implementation above uses a simple ordered ladder for clarity.

Real systems should not always do:

cheap
→ reasoning
→ specialist
→ search
→ frontier

Sometimes the correct response to failure is immediately:

run a deterministic tool

or:

retrieve missing evidence

or:

ask for human authorization

The runtime should select the smallest mechanism that addresses the diagnosed failure.

Coding-agent application

A coding agent can use this cascade:

repository state
   ↓
deterministic inspection
   ↓
local coding model
   ↓
tests / lint / type check
   ├── PASS → stop
   └── FAIL
        ↓
retrieve failing symbols + history
        ↓
local specialist / deeper reasoning
        ↓
verify
        ↓
frontier model only if needed
        ↓
search / multi-agent review for hard cases

Useful escalation signals:

• failing tests,
• repeated patches to same lines,
• no reduction in failing test count,
• conflicting static-analysis results,
• broad architectural changes,
• high-risk migration paths.

Do not use vector-memory uncertainty when git diff, the compiler or test runner can tell you the exact state.

Research-agent application

A research agent can begin cheaply:

query
 ↓
retrieve primary sources
 ↓
extract claims
 ↓
claim/source verification

Escalate when:

• sources disagree,
• key claims lack primary evidence,
• retrieved sources are stale,
• multiple interpretations remain plausible,
• the claim is high consequence.

Possible escalation:

broader retrieval
→ specialist source search
→ independent synthesis samples
→ adversarial claim review

More prose is not necessarily more research.

More evidence coverage is.

Customer-support application

Support tasks vary dramatically.

Cheap path:

identify customer
 ↓
retrieve ticket/order state
 ↓
deterministic policy lookup
 ↓
local response generation

Escalate when:

• policy conflict exists,
• refund authority is required,
• customer state is inconsistent,
• issue repeats after remediation,
• legal/safety constraints are implicated.

The adaptive agent may route to:

billing specialist
technical specialist
fraud system
human supervisor

The escalation should follow the failure or risk signal.

Data and analytics application

For a data agent:

schema inspection
 ↓
SQL generation
 ↓
query validation
 ↓
run against safe environment
 ↓
result checks

Escalate when:

• schema ambiguity exists,
• query plan is unexpectedly expensive,
• result invariants fail,
• data quality checks disagree,
• destructive transformations are requested.

Possible advanced mechanisms:

specialist SQL expert
query-plan analyzer
alternative query search
frontier model for semantic ambiguity
human approval for destructive writes

Browser-agent application

A browser agent should use direct page state as its primary signal.

observe DOM
 ↓
choose action
 ↓
execute
 ↓
verify page-state transition

Escalate if:

• element is missing,
• expected URL/state did not change,
• modal/permission state is unexpected,
• repeated clicks do not progress,
• irreversible submission is imminent.

Do not escalate merely because the page HTML is large.

Escalate because the current policy cannot reliably determine the next safe action.

DevOps and incident-response application

Adaptive compute is especially useful in incident systems.

Start with deterministic telemetry:

alerts
metrics
logs
health checks
recent deploys

Then use models to synthesize hypotheses.

Escalate when:

• hypotheses remain tied,
• the blast radius is increasing,
• rollback safety is uncertain,
• evidence contradicts the current diagnosis,
• the next action is high risk.

Possible escalation:

more diagnostics
specialist service model
Tree of Thoughts
adversarial incident review
human incident commander

Again:

Evidence determines compute allocation.

Application matrix

The adaptive mechanism is the same.

The trigger is domain-specific.

Metrics for adaptive agents

You cannot optimize adaptive compute if you only record final success.

Track:

verified success rate
first-pass success rate
escalation rate
escalation success delta
unnecessary escalation rate
missed escalation rate
average compute level
model calls per task
tool calls per task
latency per task
cost per task
cost per verified success
budget-exhaustion rate
human-escalation rate

Two particularly useful metrics are:

Escalation precision

Of tasks that were escalated, how many genuinely benefited?

beneficial escalations
----------------------
all escalations

Escalation recall

Of tasks where a more expensive mechanism would have fixed the failure, how many did the controller escalate?

These reveal different problems.

Low precision means you are wasting compute.

Low recall means you are leaving capability unused.

Oracle escalation

Offline, run the expensive path even when production would not.

Then ask:

Did the expensive mechanism produce a verified improvement?

This creates an oracle escalation label.

Now you can measure:

controller decision
vs
whether escalation would actually have helped

That is a much stronger basis for training or tuning an adaptive router than subjective labels such as “hard task.”

Escalation regret

Define escalation regret as cases where:

cheap path failed
expensive path would have succeeded
controller did not escalate

Also measure the opposite:

cheap path already succeeded
controller escalated anyway

Call that wasted escalation.

Together they characterize the controller.

Failure attribution

When an adaptive system fails, ask where.

Capability failure

No available policy could solve the task.

Trigger failure

A useful escalation existed, but the runtime did not detect the need.

Routing failure

Escalation occurred, but to the wrong mechanism.

Budget failure

The correct mechanism existed but the budget stopped too early.

Verification failure

The runtime had the correct answer but failed to recognize it—or accepted a wrong one.

Control-loop failure

The system repeatedly escalated without gaining evidence or progress.

This decomposition is essential.

Otherwise every failure gets blamed on “the model.”

A controlled experiment

Do not deploy adaptive compute because it sounds efficient.

Benchmark it.

Compare:

A. cheap model only
B. frontier model only
C. fixed advanced stack
D. deterministic local → frontier cascade
E. verification-driven adaptive controller
F. learned controller, if justified

Measure:

verified success
median latency
p95 latency
average cost
cost per verified success
escalation rate
wasted escalation
escalation regret
budget exhaustion

The adaptive system should earn its complexity.

A useful target curve

You want something like:

most tasks
   ↓
cheap path succeeds

some tasks
   ↓
one targeted escalation

few tasks
   ↓
expensive search / frontier / multi-agent

very few tasks
   ↓
human intervention

If every task reaches the expensive tier, the adaptive controller is not doing much.

If no task reaches it, you may be leaving capability on the table.

The deeper principle: intelligence is partly compute allocation

The advanced techniques in this series can now be viewed together.

Chain of Thought
→ allocate compute across intermediate reasoning

Self-Consistency
→ allocate compute across independent samples

Tree of Thoughts
→ allocate compute across partial branches

MCTS
→ adaptively reallocate compute using accumulated branch value

Mixture of Experts
→ allocate work across specialized capabilities

Planner–Executor–Critic
→ allocate responsibility across roles

Adversarial Review
→ allocate review compute toward unresolved claims

Adaptive Agents
→ decide when any of those mechanisms should run

That is the unifying architecture.

Advanced agents are increasingly systems for deciding:

where should the next unit of computation go?

But adaptive does not mean self-modifying

An adaptive runtime changes what it does within or between tasks based on current evidence.

That is different from changing its underlying policy permanently.

adaptive execution
      ≠
learning from previous runs

A runtime that chooses a frontier model after a local failure is adaptive.

A runtime that notices over thousands of tasks that local model X performs poorly on migration work and permanently updates its routing policy is learning.

That distinction matters.

Because once the system begins changing future policy from historical outcomes, new questions appear:

• Which outcomes are trustworthy training signals?
• How do you avoid learning from verifier bugs?
• How do you prevent bad episodes from poisoning policy?
• How do you evaluate policy changes before deployment?
• How do you roll back a learned routing change?

That is where the series goes next.

Final rule

A sophisticated agent does not run every sophisticated mechanism on every task.

It begins with the cheapest plausible path.

It watches the environment.

It measures failure, uncertainty, risk and progress.

Then it allocates additional compute only where that compute has a reason to change the outcome.

The rule is simple:

Start cheap. Escalate on evidence. Verify every escalation. Stop when the evidence is sufficient.

That is adaptive agency without uncontrolled complexity.

Next

Advanced Agents From First Principles 09: Can Your Agent Actually Learn From Previous Runs? Turn Verified Trajectories Into Better Future Policies Without Poisoning the System.