What Happens When Too Many Agents Compete for the Same Resources?

A single agent can look healthy in isolation.

It gets a request.

It calls a model.

It launches a few search branches.

It opens a browser.

It runs tests.

It asks a verifier to check the result.

Everything works.

Then production traffic arrives.

Ten agents start at once.

Then fifty.

Then five hundred.

Now every agent still has a perfectly reasonable local plan.

Collectively, the platform can become unusable.

The same GPU queue is saturated.

The browser pool is exhausted.

A shared database connection pool reaches its limit.

An external API starts returning 429s.

Long-running research jobs occupy every worker.

High-priority incident-response agents sit behind low-priority batch work.

Speculative search consumes the model capacity that verification needed.

Retries amplify the overload.

Queues grow faster than they drain.

Latency explodes.

And eventually the system reaches a state where every individual component is “trying harder” while the platform as a whole is getting worse.

This is the next layer of agent engineering.

The central rule is:

A production agent platform must schedule demand globally, not merely optimize each run locally.

The previous posts gave us increasingly sophisticated per-run control:

agent run
   ↓
route
   ↓
search
   ↓
observe uncertainty
   ↓
spend compute dynamically
   ↓
verify

Then we distributed the work:

run
 ↓
coordinator
 ↓
workers
 ↓
leases / fencing / idempotency

But none of that answers this question:

Which runs should be allowed to consume shared resources right now?

That is a platform scheduling problem.

Local Optimization Can Destroy Global Performance

Suppose each agent has a dynamic budget scheduler.

Each scheduler sees uncertainty and decides that another model call would probably help.

Individually, that can be correct.

Across 1,000 concurrent runs, it can be disastrous.

run A -> another model call
run B -> another model call
run C -> another model call
...
run N -> another model call

The local policy knows about its own budget.

It may not know that the shared inference cluster is already at 98% capacity.

This gives us two distinct control layers:

per-run control plane
    decides how a run spends its allowance

platform control plane
    decides how much allowance the run receives now

Do not merge them conceptually.

A run may want more compute.

The platform may correctly refuse.

The Resource Model Is Multi-Dimensional

An agent platform rarely has one bottleneck.

It has several.

For example:

resource envelope
├── frontier model requests
├── local model GPU slots
├── tokens per minute
├── browser sessions
├── sandbox workers
├── database connections
├── repository worktrees
├── CPU execution slots
├── memory
├── outbound HTTP concurrency
├── external API rate limits
├── verification capacity
└── money

A run might be cheap in one dimension and expensive in another.

A research agent may consume little CPU but enormous model and retrieval capacity.

A coding agent may consume moderate model capacity but many sandbox/test workers.

A browser agent may be bottlenecked almost entirely by browser sessions and third-party rate limits.

A data agent may be constrained by database queries and memory rather than inference.

So the scheduler must reason about a resource vector, not one generic concurrency number.

A compact representation might look like this:

from dataclasses import dataclass

@dataclass(frozen=True)
class ResourceRequest:
    model_calls: int = 0
    gpu_slots: int = 0
    browser_slots: int = 0
    sandbox_slots: int = 0
    db_connections: int = 0
    outbound_requests: int = 0
    verifier_slots: int = 0
    expected_cost_usd: float = 0.0

The exact fields will differ by system.

The architectural point is the same:

Resource demand should be explicit enough that the platform can reason about contention before work starts.

Admission Control Comes Before Queueing

A common mistake is to accept every task and put it into a queue.

That sounds safe because a queue appears to provide buffering.

But an unbounded queue is often just delayed failure.

If requests arrive faster than the system can complete them, then:

arrival rate > service rate

means:

queue length -> infinity

in the idealized case.

In production it usually means:

• latency becomes unacceptable,
• jobs become stale before they start,
• memory or storage fills,
• deadlines are missed,
• retries create more demand,
• operators lose visibility into what is actually runnable.

Admission control asks a more basic question:

Should this work enter the active system at all right now?

Possible outcomes are not merely accept or reject.

They may include:

ADMIT_NOW
QUEUE
DEFER
DOWNGRADE
SHED
REJECT

For example:

• an interactive coding request may be admitted immediately;
• a nightly repository scan may be queued;
• a speculative benchmark may be deferred;
• a research task may be downgraded to a cheaper model;
• low-value background work may be shed entirely during an incident.

That is not failure.

That is load management.

A Simple Admission Controller

Start simple.

Do not begin with an LLM deciding whether the platform has capacity.

Ordinary software is usually enough.

from dataclasses import dataclass
from enum import Enum

class AdmissionDecision(str, Enum):
    ADMIT = "admit"
    QUEUE = "queue"
    DEFER = "defer"
    REJECT = "reject"

@dataclass(frozen=True)
class Capacity:
    model_slots_free: int
    browser_slots_free: int
    verifier_slots_free: int

@dataclass(frozen=True)
class RunRequest:
    priority: int
    needs_model: bool
    needs_browser: bool
    needs_verifier: bool
    deadline_ms: int | None = None


def admit(req: RunRequest, capacity: Capacity) -> AdmissionDecision:
    if req.needs_verifier and capacity.verifier_slots_free <= 0:
        return AdmissionDecision.QUEUE

    if req.needs_browser and capacity.browser_slots_free <= 0:
        return AdmissionDecision.QUEUE

    if req.needs_model and capacity.model_slots_free <= 0:
        if req.priority >= 100:
            return AdmissionDecision.QUEUE
        return AdmissionDecision.DEFER

    return AdmissionDecision.ADMIT

This is deliberately boring.

That is good.

You can test it.

You can inspect it.

You can reason about overload behavior.

You can evolve it later when evidence demonstrates that the simple policy is insufficient.

Quotas Prevent Noisy Neighbors

Suppose one tenant launches 1,000 deep-research jobs.

Without quotas, that tenant can consume all available inference capacity.

Every other user suffers.

This is the classic noisy-neighbor problem.

Agent systems make it worse because one user request can fan out into many internal operations.

For example:

1 user task
 ↓
4 candidate generators
 ↓
8 search branches
 ↓
3 specialist agents per branch
 ↓
2 verifiers

The external request count may be one.

The internal demand may be dozens or hundreds of resource-consuming operations.

Quota accounting must therefore happen at the correct layer.

Useful quota dimensions include:

per tenant
per user
per project
per repository
per workload class
per model tier
per external integration

And possible quota units include:

concurrent runs
model calls per minute
tokens per hour
browser-minutes
sandbox CPU-seconds
verification calls
daily spend

Quota Is Not the Same as Rate Limiting

These ideas overlap but solve different problems.

A rate limit usually constrains velocity:

requests / second

A quota may constrain accumulated or concurrent consumption:

100 concurrent tasks
$50/day
1M tokens/hour

You often need both.

For example:

rate limit:
    max 20 model calls / second

quota:
    max 100,000 model calls / day

The first protects immediate capacity.

The second protects longer-term allocation and cost.

Fairness Is a Scheduling Policy, Not a Feeling

If multiple groups compete for scarce capacity, what does fair mean?

First-come-first-served is one answer.

It is not always a good one.

Suppose the queue contains:

A: 3-hour background repository analysis
B: 2-second interactive code completion
C: 10-minute research task
D: 5-second incident diagnostic

Pure FIFO can make D wait behind A.

That may be operationally unacceptable.

So we need an explicit fairness model.

Common approaches include:

• FIFO within a class,
• strict priority classes,
• weighted fair queuing,
• deficit round robin,
• tenant shares,
• deadline-aware scheduling,
• shortest-remaining-processing-time approximations,
• combinations of these.

There is no universal best scheduler.

The important rule is:

Make the trade-off explicit and measurable.

Weighted Fairness

Imagine three workload classes:

interactive: weight 5
production automation: weight 3
background analysis: weight 1

Under contention, the scheduler can distribute service roughly according to those weights.

This does not mean interactive work gets unlimited resources.

It means it receives a larger share while other classes still make progress.

A simplified allocation could look like:

weights = {
    "interactive": 5,
    "automation": 3,
    "background": 1,
}

def shares(total_slots: int) -> dict[str, int]:
    total_weight = sum(weights.values())
    result = {}

    for name, weight in weights.items():
        result[name] = max(1, total_slots * weight // total_weight)

    return result

Real schedulers need to deal with unused shares, bursts, minimum guarantees and heterogeneous resource types.

But the core idea is useful:

Fairness should survive load, not disappear exactly when capacity becomes scarce.

Priority Without Starvation

Strict priority queues are tempting.

P0 before P1
P1 before P2
P2 before P3

But a constant stream of P0 work can starve P2 forever.

That can be disastrous for maintenance, indexing, memory compaction, benchmark evaluation and other background functions.

Aging is one solution.

The longer a job waits, the more effective priority it gains.

For example:

effective_priority = base_priority + waiting_time_bonus

This provides urgency without permanent starvation.

Another approach is reserved capacity.

For example:

80% dynamic shared pool
10% background maintenance reserve
10% verification reserve

Reserved capacity can look inefficient during light load.

Under stress it may be what keeps the platform correct.

Verification Needs Platform-Level Protection

We have repeatedly protected verification inside a single agent run.

The same principle applies globally.

Suppose every agent reserves local verification budget.

That still does not help if the global verifier pool has been consumed by speculative work from other runs.

So the platform may need a protected verifier lane.

shared capacity
├── generation/search pool
├── tool-execution pool
└── protected verification pool

Why?

Because the worst overload failure is not merely slow responses.

It is a system that can still generate actions but no longer has enough capacity to establish whether those actions were correct.

That creates false completion pressure.

The platform rule should be:

Speculative work may wait. Required verification must retain a path to completion.

This does not imply infinite verification capacity.

It means verification competes under a different policy than optional exploration.

Backpressure Means Saying “Not Yet”

Consider a pipeline:

request
  ↓
planner
  ↓
model inference
  ↓
browser/tool execution
  ↓
verification
  ↓
commit

Suppose verification can process 100 tasks per minute.

Upstream generation produces 500 tasks per minute.

If generation continues unrestricted, the verification queue grows endlessly.

Backpressure propagates the bottleneck upstream.

verifier saturated
      ↑
reduce tool execution
      ↑
reduce branch expansion
      ↑
reduce admission

This is essential.

Without backpressure, every subsystem optimizes its own throughput and pushes the problem downstream.

With backpressure, the platform recognizes that downstream capacity limits upstream usefulness.

Backpressure Is Different From Failure

A saturated downstream resource may be healthy.

It is simply busy.

The correct response may be:

WAIT

rather than:

RETRY IMMEDIATELY

Immediate retries during overload make the problem worse.

This is retry amplification again, now at platform scale.

A backpressure-aware client may receive something like:

@dataclass(frozen=True)
class CapacitySignal:
    available: bool
    retry_after_ms: int | None
    queue_depth: int
    saturation: float

The calling layer can then reduce or postpone work instead of blindly retrying.

Queue Depth Is Not Enough

A queue of 1,000 tiny jobs may be healthy.

A queue of 20 jobs that each require 30 minutes of GPU time may be disastrous.

So queue depth alone is weak.

Better signals include:

estimated queued work
oldest job age
p50 wait time
p95 wait time
service rate
arrival rate
resource utilization
deadline miss probability
per-class backlog

One useful abstraction is work seconds.

Instead of:

queue length = 50

estimate:

queued GPU work = 3,200 GPU-seconds

That gives the scheduler a more realistic view of backlog.

Of course estimates will be imperfect.

That is still often better than pretending every job is equal.

Workload Classes Matter

Not all agent work has the same operational semantics.

A useful platform may classify work into categories such as:

interactive
production automation
incident response
verification
background indexing
offline benchmark
speculative research
maintenance

Each class can have different:

• admission thresholds,
• queue limits,
• resource shares,
• deadlines,
• retry behavior,
• model tiers,
• preemption rules,
• verification requirements.

For example:

interactive coding
    low latency target
    moderate compute ceiling
    immediate admission preference

background benchmark
    high latency tolerance
    large total compute allowance
    defer under contention

incident response
    high priority
    protected diagnostic capacity
    aggressive deadline

verification
    protected reserve
    cannot be silently skipped

This is much better than treating all jobs as generic “agent requests”.

Should Jobs Be Preempted?

Sometimes a high-priority task arrives while all workers are busy.

One option is preemption.

pause / cancel low-priority work
              ↓
free capacity
              ↓
run urgent task

This can be useful.

It can also be dangerous.

Preemption is easiest when work is:

• checkpointable,
• idempotent,
• side-effect free,
• resumable,
• cheap to restart.

It is much harder when a worker is halfway through an irreversible external operation.

So a job should expose preemption safety explicitly.

@dataclass(frozen=True)
class JobMetadata:
    workload_class: str
    priority: int
    checkpointable: bool
    side_effecting: bool
    safe_to_preempt: bool

The scheduler should not infer this from a model’s confidence.

It should be part of the runtime contract.

Reservation Versus Opportunistic Capacity

Some work needs predictable capacity.

Other work can consume leftovers.

A useful architecture separates:

reserved capacity
    guaranteed for critical classes

shared capacity
    allocated fairly

opportunistic capacity
    consumed only when idle

For example:

GPU cluster
├── 20% verification / critical reserve
├── 60% shared production pool
└── 20% opportunistic benchmark/search pool

If the shared production pool is quiet, the opportunistic pool may borrow capacity.

When production demand rises, that borrowed capacity is reclaimed.

This lets expensive experiments use idle hardware without compromising production guarantees.

Bursting

Hard quotas can waste capacity.

Suppose tenant A has a 20% share but no current work.

Tenant B has a burst of requests.

It can be reasonable to let B temporarily exceed its nominal share.

This is bursting.

But bursting should be reclaimable.

Otherwise temporary borrowing becomes permanent domination.

A common policy is:

guaranteed share + burst credits

or token-bucket style allocation.

For example:

class TokenBucket:
    def __init__(self, capacity: float, refill_per_second: float):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_per_second = refill_per_second

    def allow(self, cost: float) -> bool:
        if self.tokens < cost:
            return False
        self.tokens -= cost
        return True

A real implementation also needs time-based refill and concurrency safety.

The point is conceptual:

Allow bursts without surrendering long-term fairness.

Deadline-Aware Scheduling

Some tasks have explicit deadlines.

For example:

• respond to a user within 10 seconds,
• diagnose an incident before a failover window closes,
• verify a deployment before the rollout continues,
• finish a nightly job before business hours.

A scheduler can use deadlines as part of priority.

But be careful.

If a task cannot possibly finish before its deadline, aggressively prioritizing it may just waste scarce resources.

A better scheduler can estimate feasibility.

estimated remaining work
        vs
remaining deadline

If success is no longer plausible, possible actions include:

downgrade scope
return partial result
switch to cheaper/faster policy
cancel
mark deadline miss

That is more useful than consuming the entire cluster on an impossible deadline.

Cost-Aware Scheduling

Capacity is not only physical.

External model APIs may scale almost infinitely from your perspective but have financial cost.

So the platform may have a global spend envelope.

hourly spend ceiling
daily spend ceiling
per-team budget
per-project budget

This can interact with priority.

For example:

high-value production task
    allowed expensive frontier model

low-priority background task
    local model only during budget pressure

The scheduler may therefore choose not only when work executes, but which resource tier it receives.

request
  ↓
platform scheduler
  ├── local model
  ├── small hosted model
  ├── frontier model
  └── defer

This is related to routing, but the decision is driven by shared platform economics rather than merely task capability.

Rate Limits Are External Capacity Constraints

Your platform may be healthy while a dependency is saturated.

Examples:

• GitHub API rate limit,
• search API quota,
• browser target throttling,
• database QPS limit,
• model-provider RPM/TPM limits.

Treat these as first-class resources.

Do not hide them inside tool errors.

capacity state
├── internal GPU available: yes
├── browser slots available: yes
├── GitHub API budget: low
└── verifier capacity: healthy

Then scheduling can adapt before failures occur.

For example:

GitHub budget low
    ↓
prefer cached repository state
    ↓
coalesce duplicate reads
    ↓
defer background scans
    ↓
reserve remaining calls for interactive/verification work

This is much better than letting every worker independently discover the rate limit by receiving errors.

Coalescing Duplicate Work

At platform scale, many agents may request the same information at the same time.

For example:

20 agents -> fetch same repository tree
30 agents -> retrieve same documentation page
15 agents -> query same model metadata

Instead of executing all calls independently, the platform can coalesce them.

same cache key / operation identity
          ↓
one in-flight request
          ↓
multiple subscribers

This is sometimes called single-flight behavior.

It reduces:

• external API pressure,
• cost,
• latency variance,
• duplicate work.

But only when the operation is safe to share.

Do not coalesce requests that depend on tenant-private state or mutable transactional context unless the cache key captures that scope correctly.

Head-of-Line Blocking

A shared queue can create another problem.

Suppose the first queued job needs:

2 GPU slots + 4 browser slots

but only one browser slot is free.

Behind it are ten jobs that only need a model slot and could run immediately.

If the scheduler blindly preserves queue order, everything waits.

That is head-of-line blocking.

A resource-aware scheduler can scan for runnable jobs while preserving fairness constraints.

queue
├── A needs unavailable browser capacity
├── B runnable now
├── C runnable now
└── D runnable now

Run B/C/D if policy allows, while ensuring A does not starve forever.

Again, scheduling is a multi-resource problem.

The Scheduler Needs Backpressure From Real Utilization

Agent schedulers should not rely only on static configuration.

They need runtime signals.

Useful signals include:

GPU utilization
GPU memory pressure
model queue depth
model p95 latency
browser pool occupancy
sandbox startup latency
database pool saturation
external 429 rate
verification backlog
worker lease churn
retry rate
queue age

But avoid reacting too aggressively to noisy measurements.

If the scheduler changes policy every few seconds, you can create oscillation.

high load -> throttle
load falls -> release throttle
load rises -> throttle
...

Hysteresis helps.

For example:

enter overload mode at 90%
leave overload mode below 70%

This prevents constant flipping around one threshold.

Overload Modes

A production agent platform benefits from explicit overload states.

For example:

NORMAL
CONSTRAINED
SEVERE
EMERGENCY

Each state can activate deterministic policy changes.

NORMAL
    full routing options
    background work allowed

CONSTRAINED
    reduce speculative width
    defer new background work

SEVERE
    cheap models first
    suspend benchmarks
    preserve interactive + verification

EMERGENCY
    critical workloads only
    strict admission
    no optional search

This is much easier to reason about than hundreds of independent thresholds changing behavior invisibly.

And again:

Known operational safety policy should remain deterministic unless strong evidence justifies something more complex.

Backpressure Should Reach the Agent Policy

The global scheduler should not merely queue the run while the run’s internal policy behaves as if capacity were infinite.

A capacity signal can be exposed to the per-run scheduler.

For example:

@dataclass(frozen=True)
class PlatformPressure:
    model_pressure: float
    browser_pressure: float
    verifier_pressure: float
    external_api_pressure: float
    overload_mode: str

Then the run can adapt within allowed boundaries.

Example:

model pressure high
    ↓
reduce Best-of-N from 8 to 3
    ↓
use deterministic diagnostics first
    ↓
escalate only when evidence supports it

But there is an important boundary.

The local agent should not be allowed to override global quotas simply because it believes its task is important.

run requests resources
platform grants or denies

Authority remains with the platform scheduler.

Fairness Metrics

If you claim your scheduler is fair, measure it.

Useful metrics include:

per-class throughput
per-tenant throughput
per-class wait time
p95 queue age
starvation incidents
quota rejection rate
share utilization
borrowed capacity
preemption rate
deadline miss rate

You can also track whether actual resource shares match configured weights over a suitable window.

For example:

configured share: 30%
actual 1-hour share: 58%

That may be fine during bursting.

It may indicate broken fairness if sustained during contention.

Backpressure Metrics

Measure whether overload is propagating correctly.

For example:

queue growth rate
admission rejection/defer rate
retry-after compliance
upstream fan-out reduction
external 429 rate
verification backlog
resource saturation duration

A useful question is:

When one downstream resource saturates, does upstream demand fall?

If not, your backpressure mechanism is probably decorative.

Scheduling Failure Taxonomy

Trajectory observability should include platform-level failure labels.

For example:

ADMISSION_OVERLOAD
QUOTA_EXCEEDED
STARVATION
PRIORITY_INVERSION
HEAD_OF_LINE_BLOCKING
VERIFICATION_STARVATION
RATE_LIMIT_EXHAUSTION
RETRY_AMPLIFICATION
RESOURCE_FRAGMENTATION
PREEMPTION_FAILURE
BACKPRESSURE_FAILURE
SCHEDULER_OSCILLATION
DEADLINE_MISS
BUDGET_EXHAUSTION

This makes incidents diagnosable.

Instead of:

agent timed out

we might get:

FAIL
└── DEADLINE_MISS
    ├── queue_wait_ms = 14200
    ├── model_wait_ms = 3100
    ├── verifier_wait_ms = 8200
    └── root cause = verification pool saturation

That is a platform problem, not a prompt problem.

Distributed Scheduling Needs Idempotency Too

Step 20 introduced leases and fencing for distributed task execution.

The platform scheduler itself must respect those semantics.

A queued run might be:

• admitted,
• assigned,
• worker lost,
• requeued,
• assigned again.

That must not create duplicate side effects.

So scheduling integrates with:

operation identity
attempt identity
lease epoch
checkpoint identity
fencing token

The scheduler decides where work runs.

The ownership layer decides whether that worker still owns the attempt.

The commit boundary decides whether the attempt may mutate shared state.

Do not collapse these responsibilities into one giant coordinator object.

Platform Scheduler Events

Extend the event taxonomy.

Useful events include:

RUN_SUBMITTED
RUN_ADMITTED
RUN_QUEUED
RUN_DEFERRED
RUN_SHED
RUN_STARTED
RUN_PREEMPTED
RUN_RESUMED
QUOTA_CHECKED
RESOURCE_RESERVED
RESOURCE_RELEASED
BACKPRESSURE_APPLIED
OVERLOAD_MODE_CHANGED
DEADLINE_AT_RISK
RATE_LIMIT_PRESSURE

Each event should carry enough context for later analysis.

For example:

{
  "event": "RUN_QUEUED",
  "run_id": "r-1024",
  "workload_class": "interactive",
  "priority": 80,
  "reason": "model_pool_saturated",
  "queue_depth": 34,
  "estimated_wait_ms": 1800,
  "scheduler_version": "sched-v7"
}

Do not log hidden model reasoning.

Log operational state and explicit scheduler decisions.

Scheduling Policies Should Be Versioned

The scheduler is part of the control plane.

Treat it like code.

For example:

scheduler_version: sched-v7

classes:
  incident:
    weight: 10
    max_queue: 50
    reserved_model_slots: 4

  interactive:
    weight: 5
    max_queue: 1000

  automation:
    weight: 3
    max_queue: 5000

  background:
    weight: 1
    max_queue: 10000
    defer_in_overload: true

verification:
  reserved_fraction: 0.15

overload:
  constrained_at: 0.80
  severe_at: 0.90
  emergency_at: 0.97

Then policy changes are:

• diffable,
• reviewable,
• replayable,
• testable,
• reversible.

That is much safer than embedding scheduling behavior across dozens of agents.

Test the Scheduler With Synthetic Load

Do not wait for a real overload incident.

Create load tests.

Examples:

Test 1: Noisy neighbor

tenant A submits 10,000 background jobs
tenant B submits 10 interactive jobs

Expected:

• tenant B retains service,
• tenant A is quota-limited or queued,
• verification remains available.

Test 2: Verification saturation

verifier latency increases 10x

Expected:

• upstream speculative fan-out reduces,
• verification reserve remains protected,
• admission becomes stricter,
• false completion does not increase.

Test 3: External API rate limit

GitHub allowance drops to 5% remaining

Expected:

• background repository scans defer,
• duplicate reads coalesce,
• interactive/verification work retains access.

Test 4: Priority flood

continuous high-priority traffic

Expected:

• lower classes slow down,
• but aging/reservations prevent permanent starvation where required.

Test 5: Retry storm

model provider returns 503 for 30 seconds

Expected:

• retries back off,
• admission reduces,
• queues remain bounded,
• service recovers without synchronized retry spikes.

Compare Scheduling Policies Fairly

Like every advanced mechanism in this series, the scheduler must earn its complexity.

Compare policies under the same workload traces.

For example:

A: FIFO
B: strict priority
C: weighted fairness
D: weighted fairness + admission control
E: D + dynamic backpressure

Measure:

verified task success
p50 latency
p95 latency
p99 latency
cost
throughput
starvation
queue age
deadline misses
external 429s
verification backlog

Do not declare the sophisticated scheduler better merely because it has more mechanisms.

It must improve the outcomes that matter.

Coding-Agent Example

Imagine a shared coding platform serving:

interactive bug fixes
PR reviews
repository indexing
nightly refactoring experiments
CI repair agents

The expensive resources are:

• frontier-model calls,
• repository worktrees,
• sandbox test workers,
• GitHub API calls.

A reasonable policy may be:

interactive fixes
    high weight
    short queue target

CI repair
    high priority during active failures

PR review
    medium weight

repository indexing
    background
    pause under GitHub pressure

nightly experiments
    opportunistic only

If GitHub API capacity falls, the system can:

1. stop speculative repository scans,
2. reuse cached tree state where valid,
3. reserve remaining calls for active fixes and verification,
4. defer background indexing.

The model prompt has nothing to do with this.

This is runtime engineering.

Research-Agent Example

A research platform may have:

interactive research questions
scheduled reports
large literature reviews
benchmark crawls
source verification

The scheduler may protect:

• primary-source retrieval,
• citation verification,
• user-facing interactive work.

Broad speculative retrieval can be reduced under pressure.

For example:

normal mode:
    query 12 retrieval sources

constrained mode:
    query top 5

severe mode:
    query 3 authoritative sources first

Again, complexity degrades gracefully instead of collapsing abruptly.

Browser-Agent Example

Browser pools are often scarce and stateful.

Suppose you have 100 browser sessions.

A few aggressive agents can consume them all through speculative exploration.

So the scheduler may define:

interactive browser tasks: 40 reserved
production automation: 40 shared
research/discovery: 20 opportunistic

If interactive demand spikes, opportunistic exploration is cancelled or deferred.

Side-effecting browser flows may receive stronger isolation and lower concurrency than read-only discovery.

Data-Agent Example

Data agents may compete for:

• warehouse query slots,
• database connections,
• memory,
• CPU workers.

One complex query can consume far more capacity than many small ones.

So queue depth is especially misleading.

Use estimated query cost when possible.

small metadata query
    cost weight 1

medium aggregate
    cost weight 10

full historical recomputation
    cost weight 500

Then weighted admission can protect transactional or interactive workloads from expensive background analysis.

DevOps-Agent Example

During an incident, dozens of diagnostic agents may wake up at once.

That is exactly when infrastructure may already be degraded.

Bad policy:

incident detected
 ↓
launch every diagnostic everywhere

Better policy:

incident detected
 ↓
enter incident workload class
 ↓
reserve diagnostic capacity
 ↓
run cheapest/highest-EVI checks first
 ↓
fan out only if uncertainty remains
 ↓
protect verifier + authoritative-state reads

This connects directly back to the earlier posts on uncertainty and Expected Value of Information.

Global scheduling should not erase those mechanisms.

It should constrain them according to shared capacity.

Global Scheduling + Dynamic Per-Run Budgets

Now we can combine the layers.

platform scheduler
    ↓ grants resource envelope
run scheduler
    ↓ allocates envelope by expected value
uncertainty model
    ↓ identifies what blocks progress
EVI policy
    ↓ selects useful observation
speculative executor
    ↓ runs bounded parallel work
distributed coordinator
    ↓ owns attempts safely
verifier
    ↓ proves result

This is a much more realistic architecture than one giant autonomous agent.

Each layer solves a different problem.

And importantly:

Every layer can be simplified or removed if evidence shows it does not earn its cost.

A Compact Platform Scheduler Interface

A useful interface might look like this:

from dataclasses import dataclass
from enum import Enum

class ScheduleAction(str, Enum):
    START = "start"
    QUEUE = "queue"
    DEFER = "defer"
    DOWNGRADE = "downgrade"
    SHED = "shed"

@dataclass(frozen=True)
class PlatformRequest:
    run_id: str
    tenant_id: str
    workload_class: str
    priority: int
    deadline_ms: int | None
    resources: ResourceRequest

@dataclass(frozen=True)
class ScheduleDecision:
    action: ScheduleAction
    reason: str
    scheduler_version: str
    granted_resources: ResourceRequest | None = None
    retry_after_ms: int | None = None

Notice what is missing.

No hidden chain-of-thought.

No free-form “agent intuition”.

The scheduler makes an operational decision based on explicit state.

Do You Actually Need a Sophisticated Scheduler?

Maybe not.

If you have:

• low traffic,
• one tenant,
• one model,
• no meaningful external rate limits,
• no latency classes,
• plenty of spare capacity,

then a bounded FIFO queue may be completely adequate.

Start there.

Add quotas when noisy neighbors appear.

Add priorities when workload classes matter.

Add reservations when critical work is starved.

Add weighted fairness when multiple groups need guaranteed progress.

Add dynamic backpressure when bottlenecks move around.

The evidence-first progression is:

simple queue
   ↓
measure failure
   ↓
add smallest scheduling mechanism
   ↓
benchmark
   ↓
keep only if it earns its cost

Do not build Kubernetes for three cron jobs.

Final Architecture

At this point the advanced agent platform has several nested control loops.

                           incoming work
                                ↓
                     PLATFORM SCHEDULER
              admission / quota / fairness / pressure
                                ↓
                        granted envelope
                                ↓
                          RUN CONTROL
               routing / budget / stopping policy
                                ↓
                         UNCERTAINTY
                    what blocks progress?
                                ↓
                     VALUE OF INFORMATION
                  what should we observe next?
                                ↓
                  SPECULATIVE / PARALLEL WORK
                                ↓
                DISTRIBUTED WORKER COORDINATION
                   leases / fencing / retries
                                ↓
                         SIDE EFFECT GATE
                                ↓
                          VERIFICATION
                                ↓
                     PASS / FAIL / UNKNOWN
                                ↓
                    trajectory + evidence

The system is more complex than a single model call.

But the complexity now has explicit responsibilities.

That matters.

The platform scheduler does not reason about code correctness.

The verifier does not decide tenant fairness.

The worker lease does not decide whether MCTS deserves another node.

The model does not decide whether it still owns a distributed mutation.

Those boundaries are what make the architecture understandable.

The Core Rules

If you remember only a few things from this post, remember these:

1. Local agent optimization is not global platform optimization.
2. Admission control is safer than accepting infinite backlog.
3. Resource demand is multi-dimensional.
4. Quotas prevent one workload from consuming the whole platform.
5. Fairness must be explicit and measurable.
6. Priority needs starvation protection.
7. Verification deserves protected platform capacity.
8. Backpressure must propagate upstream.
9. Retries during overload can amplify failure.
10. External rate limits are real capacity constraints.
11. Schedulers should degrade optional complexity before correctness mechanisms.
12. Scheduling policy should be versioned, replayable and reversible.
13. Simple FIFO is often the right starting point.
14. A more sophisticated scheduler must beat the simpler baseline under realistic load.

And the most important rule is still the same rule that has guided the entire series:

Add a mechanism because you can identify the failure it is supposed to fix and measure whether it fixes it.

At platform scale, that rule becomes even more important.

Because once hundreds or thousands of agents compete for shared resources, complexity can fail globally even when every individual agent looks locally reasonable.

The next step is to deal with another consequence of operating at that scale:

What happens when dependencies fail partially?

A model provider slows down.

A browser service starts timing out.

A retrieval API becomes flaky.

A verifier cluster is degraded.

A database remains reachable but returns stale replicas.

The next post will look at failure containment, circuit breakers, bulkheads and graceful degradation—how an advanced agent platform keeps one broken dependency from cascading through every run.