Shepherd: Programmable Meta-Agents via Reversible Execution Traces
Important
Shepherd is in early alpha and under active development. APIs may still change between releases. Feedback and issues are very welcome!
Install | Quickstart | Permissions | Examples | Docs | Citation
Shepherd is a runtime substrate for agent work that needs inspection, reversibility, and supervision. It records agent runs as durable, inspectable execution traces, with retained workspace outputs that can be reviewed before they are selected, applied, released, or discarded.
Platforms. Shepherd requires Python 3.11+. OS-level grant enforcement is executed on both macOS (Seatbelt) and Linux (Landlock, in a privileged container). Windows is unsupported (enforcement would be advisory-only at best) — use WSL.
Installation
pip install shepherd-aiWorking on Shepherd itself? Install the local editable closure instead:
python -m venv .venv && . .venv/bin/activate && pip install -r requirements-dev.txt
(see CONTRIBUTING.md).
Quickstart
Shepherd is an agent framework: a task's implementation can be a sandboxed agent, and its work comes back as a reviewable proposal — nothing touches your files until you accept it. Here the whole body of a task is a Claude agent.
Needs the
claudeCLI — signed in (a Claude subscription works) or with anANTHROPIC_API_KEY. Neither? Jump to the Offline Quickstart — it runs anywhere, keyless.On a subscription, a sandboxed run is most reliable with a long-lived token:
export CLAUDE_CODE_OAUTH_TOKEN=$(claude setup-token). A short-lived signed-in session can't be refreshed from inside the sandbox, so it may work interactively yet fail here —shepherd doctor claude(add--probefor a real auth round-trip under Shepherd's config, in the parent — not a jailed run) tells you which credential you have before you run. If Claude returns an org-policy error (HTTP 403), that's an account/organization limit, not a login problem — a different key or your org admin is the fix. And an outrightclaudeCLI hang (e.g. a stale version) surfaces as a budget timeout, not an auth error.
A task is a plain Python function with no body; the signature and docstring
are the contract the agent fulfils at runtime — including its permissions:
repo: sp.GitRepo is the explicit writable workspace-handle grant that lets the
agent write the repository (see
Permissions):
def write_program(
repo: sp.GitRepo,
prompt: str,
output_path: str = "program.py",
) -> None:
"""Write a small, self-contained Python program that does what `prompt` asks.
Save it to output_path. It must run with plain `python3`, read no input,
and finish on its own within about ten seconds.
"""Set up a scratch workspace and check the agent lane is ready:
mkdir /tmp/agent-task && cd /tmp/agent-task
shepherd init # turn this directory into a Shepherd workspace
shepherd doctor claude # confirm claude CLI, sign-in/key, and sandbox are readyFetch the demo and let the agent work (about a minute):
shepherd demo write agent-task > agent_task.py
python agent_task.pyThe agent writes donut.py — but not into your directory. It lands as a
retained output: a proposal held safely to one side, which you can run
without applying anything:
shepherd run changeset --latest --read donut.py | python3 -Ten seconds of spinning ASCII donut, straight out of the retained output. If you like it, keep it; if not, throw it away — the trace remembers either way (the demo prints both commands with the real run id):
shepherd run select <run-ref> # keep it
shepherd run apply <run-ref> # ...or merge it onto a workspace that moved on
shepherd run discard <run-ref> # ...or notEdit PROMPT in agent_task.py and re-run to ask for anything else — the
contract stays the same. For an agent that edits existing files, see
shepherd demo write claude-readme.
Offline Quickstart
No API key required. This runs the same retained-output machinery through Shepherd's deterministic provider — the agent lane above, minus the agent:
mkdir /tmp/shepherd-quickstart && cd /tmp/shepherd-quickstart
shepherd init # turn this directory into a workspace
shepherd demo write quickstart > quickstart_demo.py
python quickstart_demo.py # register + run a task, retaining its result
shepherd run list # the run and its status
shepherd run changeset --latest # what it wrote, kept as a retained outputInspect the full record with shepherd run show --latest (add --json to any
read command for the durable machine payload); see the
docs for backend selection and the complete
run surface.
Permissions: the signature is the permission surface
For the common single-workspace writer, repo: sp.GitRepo is the clean spelling:
an explicit read-write handle grant. Use May[GitRepo, ReadOnly] when a task
must inspect without mutating; an unannotated repo parameter is just an
ordinary value parameter, not a handle.
A task can also declare a read-only or read-write grant per bound repository, in its signature:
from shepherd import task, May, GitRepo, ReadOnly, ReadWrite
@task
def apply_documented_fix(
docs: May[GitRepo, ReadOnly], # read-only: writes refused at the OS
backend: May[GitRepo, ReadWrite], # writable root
issue: str,
) -> None: ...On a jailed device the grant is compiled to that run's writable roots and
enforced at the native syscall jail (macOS Seatbelt; Linux Landlock): a
write to a ReadOnly-granted repository, or to any managed path not covered by
a ReadWrite grant, is refused at the syscall — before the last undo point, not
advised and not caught only at a merge gate. Reading the signature is reading
the permission surface, and shepherd task show renders it expanded. Grants are
whole-profile per binding (a bound repository is entirely writable or entirely
read-only). Bindings are named with ws.bind(root="backend/", name="backend")
and passed to a run with workspace.run(task, bindings={...}); each run's world
output is inspected per binding with run.changeset(name="backend") and settled
once with select / apply / release / discard (apply three-way-merges a
candidate onto a workspace that already moved on, when their changes are
path-disjoint).
Scope (P-030 v0.2). Per-binding whole-profile
ReadOnly/ReadWriteover disjoint named bindings, on a jailed device, filesystem / Git substrate, same-process value-children. Enforcement is executed on both macOS Seatbelt and Linux Landlock (the latter in a privileged container). Sub-root /where(path=…)grants are not part of this cut.
Examples
The demo scripts above are the Python surface in miniature — checked-in copies
live in examples/quickstart/.
The visual-artifact notebooks live in
examples/notebooks/visual_artifact/notebooks/
— launch them with make notebooks.
Development
Useful local gates:
make dev-install
uv run pytest integration-tests/test_quickstart_core.py -q
make baselineDocumentation
Full documentation lives at docs.shepherd-agents.ai. In this repository the docs are authored under docs/shepherd/, starting with the
Quickstart guide and
Concepts — tasks, effects, scopes, permissions, and the trace.
Reproducing Paper Results
The full experiment code — the meta-agent applications and the framework-performance microbenchmarks — lives in a companion repository: shepherd-agents/shepherd-experiments. It bundles the frozen substrate snapshot used for the paper, so the numbers stay reproducible against the exact version that produced them.
Acknowledgments
We thank E2B, Tinker, and Modal for their generous compute support during this project.
Citation
@misc{yu2026shepherdenablingprogrammablemetaagents,
title={Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Traces},
author={Simon Yu and Derek Chong and Ananjan Nandi and Dilara Soylu and Jiuding Sun and Christopher D Manning and Weiyan Shi},
year={2026},
eprint={2605.10913},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2605.10913},
}License
This project is licensed under the MIT License — see the LICENSE file for details.
