Skill Orchestration for Long-Horizon Agents

Chain-of-thought did something subtle. It did not teach a model any new fact. It changed where the intermediate work happened — from inside the forward pass, where it was cramped and error-prone, to the output stream, where it could be laid out, checked, and extended. Nye et al. (2021) called it a scratchpad; Wei et al. (2022) showed that a few worked examples of this externalized reasoning unlock arithmetic, commonsense, and symbolic tasks that the same model fails at when asked for the answer directly. Kojima et al. (2022) then showed the trigger can be as thin as “Let’s think step by step,” and Wang et al. (2022) showed that sampling several such traces and voting beats any single one.

The lesson I take from that lineage is not “reasoning helps.” It is that there is latent competence a model cannot express until you give it the right external structure to express it in.

For a single question, that missing structure was reasoning. For a long-horizon agent — one that runs for dozens or hundreds of steps, touches tools, files, and services, and has to recover when something goes wrong — I think the missing structure is not reasoning. It is procedure: knowing when to enter a workflow, which expert routine to follow, when to switch, when to validate, and when to stop. And the object that is supposed to carry procedure is the skill.

This post is an argument about what skills are actually for, and where the real research problem lives. The short version:

Chain-of-thought externalizes reasoning; skills externalize procedure. The unsolved part is not how to write a skill — it is orchestration: deciding when a procedure is needed, which one, and how to invoke it.

I’ll build up to that, connect it to the classical options framework from hierarchical RL (which has been asking “when do I invoke a sub-behavior, and when do I terminate it?” since 1999), and end with the experiments I think would settle it — plus one preliminary result of our own.

From reasoning to procedure

The reasoning story has a clean shape. A model has the knowledge but can’t serialize the computation in one shot, so we let it write the computation down. ReAct extended this from pure thought to thought interleaved with action — the model reasons, calls a tool, reads the result, reasons again. Tree of Thoughts let it branch and backtrack; Reflexion let it write a verbal post-mortem after a failed episode and carry it into the next try. All of these are variations on the same move: take something the model was doing implicitly and badly, and give it an explicit place to do it.

Now push the horizon out. The task is no longer “answer this question.” It is: onboard this customer, reproduce this bug and open a PR, run this multi-stage data pipeline and validate the output, book this trip under these constraints. The model still needs to reason — but reasoning is no longer the bottleneck. The bottleneck is sequencing: which phase am I in, is this the moment to validate or to keep building, did I skip a step that this class of task always requires, should I switch from “implement” mode to “recover” mode because the last three actions failed the same way.

That knowledge — how this kind of work is normally done — is real, and it is mostly tacit. A senior engineer doesn’t rederive the deploy checklist every time; they have a procedure. The claim of this post is that a skill is the externalized form of exactly that tacit procedural knowledge, and that we have been studying the easy half of the problem (how to represent and store skills) while mostly ignoring the hard half (how an agent decides, mid-trajectory, to reach for one).

Where long-horizon agents actually break

It helps to be concrete about the failure. Long-horizon agent benchmarks are brutal in a specific way. On AppWorld, a controllable world of interacting apps and APIs, the strongest agent at release completed about 49% of “normal” tasks and only 30% of the harder split — not because any single API call is hard, but because the tasks chain many calls and one procedural slip poisons the rest. On GAIA, questions that humans answer 92% of the time were answered ~15% of the time by a strongly-equipped GPT-4, and the gap is almost entirely in stringing tools together over a long horizon, not in any individual lookup.

The most useful recent evidence is diagnostic rather than aggregate. Wang et al. (2026), building the HORIZON diagnostic over 3,100+ trajectories across four agentic domains, argue that long-horizon failures “remain poorly characterized” and show horizon-dependent degradation: the same agent that is reliable at short horizons decays as the trajectory lengthens, in structured ways that don’t reduce to “the model got a fact wrong.”

I want to name the distinction that this evidence points to, because the whole argument rests on it:

• Factual / tool failures — the agent doesn’t know something, calls the wrong API, or malforms an argument. More knowledge or a better tool fixes these.
• Procedural failures — the agent knows the facts and has the tools, but does the wrong routine at the wrong time: validates too late, never enters a recovery routine after repeated failures, re-plans when it should commit, or barrels ahead when it should stop and ask.

Chain-of-thought, tools, and retrieval attack the first bucket. The second bucket is what skills are supposed to attack — and it is the bucket that gets worse as the horizon grows.

What a skill is (and isn’t)

The word “skill” is overloaded, so let me pin it. In current agent practice, a skill is often introduced as a format: Anthropic’s Agent Skills are folders of instructions, scripts, and resources that an agent discovers by name and loads on demand, using progressive disclosure — the name and one-line description sit in context always, the body is pulled in only when relevant, so the bundled context is effectively unbounded. That is a genuinely good engineering pattern. But the format is not the essence.

The lineage that captures the essence better is Voyager, whose agent grows an “ever-growing skill library of executable code” for Minecraft — skills that are temporally extended, interpretable, and compositional, stored when they work and retrieved to solve novel tasks. CodeAct makes the same point about the medium: expressing an agent’s actions as executable code, rather than fixed JSON tool-calls, gives it a far more expressive and composable action space. What both are really trading in is not files — it is reusable procedure.

So the definition I’ll use:

A skill is a reusable procedural prior — an externalized expert routine for a class of tasks, shaped by ordering constraints, failure handling, and operational judgment. It is how a kind of work is done, not merely what tool exists.

Here’s the part that surprised me. The most recent survey of the area, Zhou et al. (2026), defines skills almost exactly this way — “reusable procedural artifacts that coordinate tools, memory, and runtime context” — and then organizes the entire field along four lifecycle stages: representation, acquisition, retrieval, and evolution. Notice what is not a first-class axis there. When the agent should decide to invoke a procedure at all, which of several it should choose given where it is in the trajectory, when it should abandon one mid-way — the orchestration — is folded into “retrieval” and largely left implicit. The field has a rich account of how skills come to exist and get stored, and a thin account of how they get used in time.

That gap is the subject of the rest of this post.

The orchestration problem: when, which, how

Reframe the central question away from skill definition and toward skill orchestration. Concretely it splits three ways.

When should an agent consult a procedure? Candidate moments: at task start, at a phase transition, after a spike in uncertainty, before an irreversible action, after a failure, or — the interesting case — only when the agent detects that its current trajectory lacks the right procedure. The last one is a capability I’ll call procedural self-awareness, and I think it is the real target.

Which procedure? A single unified stage-level skill? One of many fine-grained sub-skills? A dedicated validator routine? A recovery routine? Or none — because for this task, from-scratch reasoning is genuinely fine and a skill would only add rigidity?

How should it be invoked? The mechanisms differ enormously in context cost and in how much they constrain the agent:

None of these is dominant; the right choice depends on how certain the agent is that a procedure applies, and how costly it is to be wrong.

Laid out like this, orchestration is obviously not a formatting or library-management problem. It is a dynamic control problem: at each point in a long trajectory, choose whether to run the base policy or to hand control to a temporally-extended routine, and choose when to hand it back. Which is a problem the RL community has a name and a formalism for.

Skills are options in disguise

If “invoke a routine, run it for a while, decide when to stop” sounds familiar, it should. That is precisely an option in the sense of Sutton, Precup & Singh (1999). An option is a temporally extended behavior defined by three things:

$$ o = \langle \mathcal{I},; \pi,; \beta \rangle $$

• an initiation set $\mathcal{I} \subseteq \mathcal{S}$ — the states where the option can be started (the “when/which” of invocation),
• an intra-option policy $\pi$ — how the option behaves while it runs (the procedure itself),
• a termination condition $\beta: \mathcal{S} \to [0,1]$ — the probability of handing control back at each state (the “when to switch/stop”).

The mapping to skill orchestration is one-to-one:

Read that mapping slowly, because it is the cleanest thing I can offer. A skill is an option: the description-match that decides whether a skill is eligible is its initiation set; the routine’s body is its intra-option policy; and “should I keep following this skill or bail out and re-plan” is its termination condition $\beta$. The Option-Critic architecture even showed you can learn $\pi$ and $\beta$ together end-to-end — learning termination is learning when to stop trusting a procedure, which is exactly the failure I described in the HORIZON evidence above.

There are two honest consequences of taking this mapping seriously.

The first is a warning. Much of what reads as novel in agent-skill work is a re-derivation of hierarchical RL with natural language swapped in for learned parameters. If your paper’s contribution is “the agent decides when to invoke a sub-behavior and when to terminate it,” a reviewer who knows the options literature will not be impressed by the framing alone. You have to say what is different.

And things genuinely are different, which is the second consequence — the opportunity. In the LLM setting:

• Options are specified in natural language and code, not learned as opaque policies — so they are inspectable, editable, and shareable across agents. Automated pipelines like ADAS (a meta-agent that programs new agents in code) and AFlow (Monte-Carlo tree search over code-represented workflows) are, in options language, searching the space of initiation sets and intra-option policies — but in a representation a human can read.
• Termination is judged semantically by the model itself (“this isn’t working, re-plan”), not by a scalar $\beta$ tuned on returns.
• Options compose and transfer zero-shot — Voyager reuses its skill library in a fresh world — in a way learned options historically struggled to.

So the right stance is not “skills are new.” It is “skills are options whose components are legible, and that changes what is easy and what is hard.” The hard part moves from learning $\beta$ to deciding, in context and without gradient signal, when the current procedure has stopped applying.

Measuring orchestration: contracts and verifiers

Here is a problem that quietly sabotages this whole research program: success rate is nearly blind to orchestration. An agent can reach the right answer through a sloppy trajectory — right skill, wrong moment, or no skill where one would have helped but luck covered for it — and score exactly the same as an agent that did the procedure cleanly. If the metric can’t see procedural timing, we can’t study procedural timing.

Two threads point at a fix.

The first is process-level verification. Lightman et al. (2023) showed that supervising each reasoning step (process reward) rather than only the final answer (outcome reward) trains substantially more reliable reasoners, and released step-level labels to do it. The generalization to agents is direct: reward and measure the trajectory’s procedure, step by step, not just its endpoint.

The second is contracts as an observational layer. This is where my own work sits, so I’ll flag the bias. In ContractBench we studied whether agents preserve observation contracts — artifacts like presigned URLs or session tokens whose later use is constrained by an external system — and found contract compliance to be an emergent, regression-prone capability that bigger or newer models do not reliably improve. Independently, Fan et al. (2026) reframe skill drift — skills silently breaking as their dependencies evolve — as contract violation, and show that extracting executable contracts from skill docs and checking them lifts repair success from 10% to 78%.

The reframe I want to make explicit: a contract is not the essence of a skill. A contract is a measurement and control layer around skill use. It is the instrument that lets you ask, of a completed trajectory, “was the right procedure followed, at the right time?” — which is exactly the question that success rate refuses to answer. Fan et al. use contracts to check that a skill’s dependencies still hold; the direction I care about is using a verifier to check that the orchestration was right: that a needed skill fired, that it fired at the correct phase, and that it terminated when it stopped applying.

Call the missing metric procedural timing — precision and recall over “the right skill was invoked at the right moment,” scored against a per-task annotation of where a procedure should enter and switch. Right now “procedural self-awareness” is an adjective. A procedural-timing metric is how you turn it into a number.

Evidence: when a procedure faults, terminate or repair?

Everything above is about when and whether to run a procedure. There’s a companion decision I’ve glossed: when the agent detects that a procedure has gone wrong mid-flight, what should it do? In options terms this is the termination question — hand control back, or keep going? There’s the reflexive answer, and a less obvious third one.

I have data on exactly this, from our own work on contract-enforced agents (the measurement layer of ContractBench, extended with a repair engine). Setup: an agent runs a multi-step task; a contract watches a sensitive step — here the recipient of a money transfer — and fires when the value being consumed didn’t come from a trusted source. That firing is a detected procedural fault. Three policies on the fault:

• none — ignore it (the floor: no orchestration).
• block — terminate the offending step ($\beta = 1$, a hard stop).
• mend — repair the step (rebind to the trusted value) and continue.

On AgentDojo’s banking suite under prompt injection (GPT-4o, 144 task×injection pairs), scoring task utility and attack-success:

Two things worth pulling out. First, the policy on a detected fault materially moves the outcome. Block buys safety by throwing away utility (0.75 → 0.63); mend recovers most of it (0.69) at equal safety. “Terminate on violation” — the reflexive choice — leaves task success on the table that “repair and continue” keeps: 16 task instances that block failed, mend rescued, with zero safety regressions. The termination policy is not a detail; it is a first-class orchestration knob.

Second, and closer to this post’s thesis: the success metric is blind to all of it. The 0.69-vs-0.63 gap is the only thing utility sees; underneath, the contract layer logged over a thousand taint events and the handful of repairs that actually separate the policies. The procedural signal exists — but only because a verifier was watching. Success rate alone would have told you almost nothing about how the agent handled the fault.

And the honest boundary. Swap the strong model for a weak one (GPT-4o-mini) and the effect collapses — repair barely fires and mend ≈ block ≈ none, because a weak agent rarely produces a clean, repairable fault to begin with. I saw the same thing rerunning on AppWorld’s benign money tasks with GPT-4o-mini: near-zero repairs, no separation between policies. Orchestration-on-fault only pays when the agent is competent enough to generate faults worth repairing — itself a finding, and a caution against expecting these mechanisms to rescue weak agents.

A preliminary result: unified skills don’t seem to hurt

One empirical note, offered as an observation rather than a claim. A natural fear about orchestration is that it demands fine-grained routing — many small sub-skills and a dispatcher smart enough to pick among them at every phase. In our own ongoing experiments, we do not see the penalty that fear predicts: a single stage-level unified skill — one coherent procedural frame for a whole class of tasks — performs on par with a decomposed set of fine-grained sub-skills, with no degradation we can currently detect.

I want to be careful about what that does and does not mean. It is preliminary, it is on our task distribution, and “no degradation we can detect” is not “no difference.” But if it holds up, the reading is interesting: a unified skill may work because it supplies a coherent procedural frame and removes handoff ambiguity — the cost of choosing among many sub-skills and stitching their boundaries can outweigh the benefit of specialization. In options terms, one option with a broad initiation set and a well-judged termination can beat a committee of narrow options plus a fallible policy-over-options. This is a stage in a larger question, not the answer to it, and I’m flagging it precisely so it can be shot down.

Open problems, and what I’d test next

The research program I’d like to see is not “standardize the skill file format harder.” It is to study skill application as dynamic control, and to measure it.

1. Build tasks where the agent must decide whether a procedure is needed at all — including tasks where reaching for a skill is the wrong move and from-scratch reasoning wins. Orchestration you can’t fail is orchestration you can’t study.
2. Run the four-arm comparison: unified skill vs. many sub-skills vs. dynamic retrieval vs. no-skill, on the same long-horizon distribution — over benchmarks like AppWorld and our own SkillsBench.
3. Score procedural timing, not just success — did the right skill fire at the right moment — using the contract/verifier layer as the instrument.
4. Ask whether good orchestration can become a training signal. If we can label when a skill should have fired, that label is exactly the kind of process-level supervision Lightman et al. (2023) used for reasoning — and recent work on internalizing skills into weights (SIRI, LatentSkill) suggests orchestration competence, once measurable, could be distilled rather than prompted forever.

Underneath all four is one capability. A CoALA-style agent has procedural memory — a store of skills — but a store is not the same as knowing when to draw from it. Generative Agents and MemGPT showed that memory only becomes useful when paired with a policy for retrieval and reflection; procedure is the same. The thing worth building is procedural self-awareness: an agent that notices its current trajectory is missing the right routine, and reaches for it — or notices the routine it’s running has stopped applying, and lets go.

Chain-of-thought made a model’s reasoning visible so it could get better. The bet here is that making a long-horizon agent’s procedure visible — and measurable — is the next version of the same move. Skills are not structured prompts. They are the scaffold; orchestration is the skill that uses them.

References

[1] Nye, M., Andreassen, A., Gur-Ari, G., et al. “Show Your Work: Scratchpads for Intermediate Computation with Language Models.” arXiv:2112.00114, 2021. https://arxiv.org/abs/2112.00114

[2] Wei, J., Wang, X., Schuurmans, D., et al. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” NeurIPS, arXiv:2201.11903, 2022. https://arxiv.org/abs/2201.11903

[3] Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., Iwasawa, Y. “Large Language Models are Zero-Shot Reasoners.” NeurIPS, arXiv:2205.11916, 2022. https://arxiv.org/abs/2205.11916

[4] Wang, X., Wei, J., Schuurmans, D., et al. “Self-Consistency Improves Chain of Thought Reasoning in Language Models.” ICLR, arXiv:2203.11171, 2022. https://arxiv.org/abs/2203.11171

[5] Yao, S., Zhao, J., Yu, D., et al. “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR, arXiv:2210.03629, 2023. https://arxiv.org/abs/2210.03629

[6] Yao, S., Yu, D., Zhao, J., et al. “Tree of Thoughts: Deliberate Problem Solving with Large Language Models.” NeurIPS, arXiv:2305.10601, 2023. https://arxiv.org/abs/2305.10601

[7] Shinn, N., Cassano, F., Berman, E., et al. “Reflexion: Language Agents with Verbal Reinforcement Learning.” NeurIPS, arXiv:2303.11366, 2023. https://arxiv.org/abs/2303.11366

[8] Schick, T., Dwivedi-Yu, J., Dessì, R., et al. “Toolformer: Language Models Can Teach Themselves to Use Tools.” arXiv:2302.04761, 2023. https://arxiv.org/abs/2302.04761

[9] Wang, X., Chen, Y., Yuan, L., et al. “Executable Code Actions Elicit Better LLM Agents (CodeAct).” ICML, arXiv:2402.01030, 2024. https://arxiv.org/abs/2402.01030

[10] Wang, G., Xie, Y., Jiang, Y., et al. “Voyager: An Open-Ended Embodied Agent with Large Language Models.” arXiv:2305.16291, 2023. https://arxiv.org/abs/2305.16291

[11] Hu, S., Lu, C., Clune, J. “Automated Design of Agentic Systems (ADAS).” ICLR, arXiv:2408.08435, 2024. https://arxiv.org/abs/2408.08435

[12] Zhang, J., Xiang, J., Yu, Z., et al. “AFlow: Automating Agentic Workflow Generation.” ICLR, arXiv:2410.10762, 2024. https://arxiv.org/abs/2410.10762

[13] Sutton, R. S., Precup, D., Singh, S. “Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning.” Artificial Intelligence, 112(1–2):181–211, 1999. https://www.sciencedirect.com/science/article/pii/S0004370299000521

[14] Bacon, P.-L., Harb, J., Precup, D. “The Option-Critic Architecture.” AAAI, arXiv:1609.05140, 2017. https://arxiv.org/abs/1609.05140

[15] Sumers, T. R., Yao, S., Narasimhan, K., Griffiths, T. L. “Cognitive Architectures for Language Agents (CoALA).” TMLR, arXiv:2309.02427, 2023. https://arxiv.org/abs/2309.02427

[16] Park, J. S., O’Brien, J. C., Cai, C. J., et al. “Generative Agents: Interactive Simulacra of Human Behavior.” UIST, arXiv:2304.03442, 2023. https://arxiv.org/abs/2304.03442

[17] Packer, C., Wooders, S., Lin, K., et al. “MemGPT: Towards LLMs as Operating Systems.” arXiv:2310.08560, 2023. https://arxiv.org/abs/2310.08560

[18] Lightman, H., Kosaraju, V., Burda, Y., et al. “Let’s Verify Step by Step.” ICLR, arXiv:2305.20050, 2023. https://arxiv.org/abs/2305.20050

[19] Trivedi, H., Khot, T., Hartmann, M., et al. “AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents.” ACL, arXiv:2407.18901, 2024. https://arxiv.org/abs/2407.18901

[20] Mialon, G., Fourrier, C., Swift, C., et al. “GAIA: A Benchmark for General AI Assistants.” arXiv:2311.12983, 2023. https://arxiv.org/abs/2311.12983

[21] Wang, X. J., Bai, H., Sun, Y., et al. “The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break (HORIZON).” arXiv:2604.11978, 2026. https://arxiv.org/abs/2604.11978

[22] Zhou, Y., Shu, W., Su, Y., et al. “A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications.” arXiv:2605.07358, 2026. https://arxiv.org/abs/2605.07358

[23] Anthropic. “Equipping Agents for the Real World with Agent Skills.” Anthropic Engineering, 2025. https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills

[24] Fan, L., Tian, Y., Li, Z., Lu, Z. “Skill Drift Is Contract Violation: Proactive Maintenance for LLM Agent Skill Libraries.” arXiv:2605.10990, 2026. https://arxiv.org/abs/2605.10990

[25] ContractBench. “Can LLM Agents Preserve Observation Contracts?” arXiv:2605.17281, 2026. https://arxiv.org/abs/2605.17281

[26] SkillsBench. “Benchmarking How Well Agent Skills Work Across Diverse Tasks.” arXiv:2602.12670, 2026. https://arxiv.org/abs/2602.12670

[27] “SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Training.” arXiv:2606.02355, 2026. https://arxiv.org/abs/2606.02355

[28] “LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents.” arXiv:2606.06087, 2026. https://arxiv.org/abs/2606.06087

This is a research-in-progress note; the unified-skill result is preliminary and the framing will change as the experiments mature. Corrections and pointers to prior work I’ve missed are welcome — open an issue on GitHub.