antoinezambelli/forgePublic

A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows

AI summary: A Python reliability layer for self-hosted LLM tool-calling and workflow constraint management.

Stars
2.3K
+-2 today
Forks
173
Watchers
14
Open issues
3
Open PRs
0
Contributors
~10
Commits
95
Branches
1
Views (30d)
3

PythonMITCreated Feb 16, 2026Last push 8d agoLatest release v0.10.1+9 this month

Quick answers

What is forge?
A Python reliability layer for self-hosted LLM tool-calling and workflow constraint management.
What does forge do?
Forge is a specialized Python framework designed to make tool-calling highly reliable for self-hosted LLMs. It acts as a reliability layer inside an agentic loop, applying rescue parsing, retry nudges, and response validation to ensure models execute tools correctly. Developers can optionally enforce workflow structures using prerequisites and required steps to constrain the model's behavior. By focusing strictly on tool execution rather than orchestration, it significantly boosts the success rate of smaller local models (like 8B parameters). It is domain-agnostic and built to enhance existing single-agent setups rather than replace multi-agent graphs.
Who is forge for?
AI engineers and developers building robust single-agent systems using local or self-hosted LLMs. It requires proficiency in Python and an understanding of function calling APIs.
How do I get started with forge?
pip install forge
How popular is forge on GitHub?
antoinezambelli/forge has 2,268 stars and 173 forks on GitHub.
What license does forge use?
antoinezambelli/forge is released under the MIT license.

Star history

since Jul 28, 2026
01K2KJul 2026Sep 2026Oct 2026Nov 2026
2.3K stars as of Nov 18, 2026. Measured daily since Jul 28, 2026; GitHub no longer exposes earlier star timestamps.

Update history

2 recorded
  • Nov 7, 2026New release v0.10.1 (previously v0.10.0).
  • Oct 28, 2026New release v0.10.0 (previously v0.9.5).

Contribution activity

commits per day, last 52 weeks
NovDecJanFebMarAprMayJunJulAugSepOctNovMonWedFri2025-11-23: 0 commits2025-11-24: 0 commits2025-11-25: 0 commits2025-11-26: 0 commits2025-11-27: 0 commits2025-11-28: 0 commits2025-11-29: 0 commits2025-11-30: 0 commits2025-12-01: 0 commits2025-12-02: 0 commits2025-12-03: 0 commits2025-12-04: 0 commits2025-12-05: 0 commits2025-12-06: 0 commits2025-12-07: 0 commits2025-12-08: 0 commits2025-12-09: 0 commits2025-12-10: 0 commits2025-12-11: 0 commits2025-12-12: 0 commits2025-12-13: 0 commits2025-12-14: 0 commits2025-12-15: 0 commits2025-12-16: 0 commits2025-12-17: 0 commits2025-12-18: 0 commits2025-12-19: 0 commits2025-12-20: 0 commits2025-12-21: 0 commits2025-12-22: 0 commits2025-12-23: 0 commits2025-12-24: 0 commits2025-12-25: 0 commits2025-12-26: 0 commits2025-12-27: 0 commits2025-12-28: 0 commits2025-12-29: 0 commits2025-12-30: 0 commits2025-12-31: 0 commits2026-01-01: 0 commits2026-01-02: 0 commits2026-01-03: 0 commits2026-01-04: 0 commits2026-01-05: 0 commits2026-01-06: 0 commits2026-01-07: 0 commits2026-01-08: 0 commits2026-01-09: 0 commits2026-01-10: 0 commits2026-01-11: 0 commits2026-01-12: 0 commits2026-01-13: 0 commits2026-01-14: 0 commits2026-01-15: 0 commits2026-01-16: 0 commits2026-01-17: 0 commits2026-01-18: 0 commits2026-01-19: 0 commits2026-01-20: 0 commits2026-01-21: 0 commits2026-01-22: 0 commits2026-01-23: 0 commits2026-01-24: 0 commits2026-01-25: 0 commits2026-01-26: 0 commits2026-01-27: 0 commits2026-01-28: 0 commits2026-01-29: 0 commits2026-01-30: 0 commits2026-01-31: 0 commits2026-02-01: 0 commits2026-02-02: 0 commits2026-02-03: 0 commits2026-02-04: 0 commits2026-02-05: 0 commits2026-02-06: 0 commits2026-02-07: 0 commits2026-02-08: 0 commits2026-02-09: 0 commits2026-02-10: 0 commits2026-02-11: 0 commits2026-02-12: 0 commits2026-02-13: 0 commits2026-02-14: 0 commits2026-02-15: 0 commits2026-02-16: 0 commits2026-02-17: 0 commits2026-02-18: 0 commits2026-02-19: 0 commits2026-02-20: 0 commits2026-02-21: 0 commits2026-02-22: 0 commits2026-02-23: 0 commits2026-02-24: 0 commits2026-02-25: 0 commits2026-02-26: 0 commits2026-02-27: 0 commits2026-02-28: 0 commits2026-03-01: 0 commits2026-03-02: 0 commits2026-03-03: 0 commits2026-03-04: 0 commits2026-03-05: 0 commits2026-03-06: 0 commits2026-03-07: 0 commits2026-03-08: 0 commits2026-03-09: 0 commits2026-03-10: 0 commits2026-03-11: 0 commits2026-03-12: 0 commits2026-03-13: 0 commits2026-03-14: 0 commits2026-03-15: 2 commits2026-03-16: 1 commit2026-03-17: 1 commit2026-03-18: 0 commits2026-03-19: 3 commits2026-03-20: 0 commits2026-03-21: 1 commit2026-03-22: 1 commit2026-03-23: 5 commits2026-03-24: 1 commit2026-03-25: 2 commits2026-03-26: 1 commit2026-03-27: 2 commits2026-03-28: 1 commit2026-03-29: 1 commit2026-03-30: 0 commits2026-03-31: 3 commits2026-04-01: 2 commits2026-04-02: 4 commits2026-04-03: 0 commits2026-04-04: 0 commits2026-04-05: 0 commits2026-04-06: 0 commits2026-04-07: 0 commits2026-04-08: 0 commits2026-04-09: 0 commits2026-04-10: 0 commits2026-04-11: 1 commit2026-04-12: 0 commits2026-04-13: 0 commits2026-04-14: 0 commits2026-04-15: 0 commits2026-04-16: 0 commits2026-04-17: 2 commits2026-04-18: 0 commits2026-04-19: 0 commits2026-04-20: 0 commits2026-04-21: 1 commit2026-04-22: 0 commits2026-04-23: 0 commits2026-04-24: 0 commits2026-04-25: 0 commits2026-04-26: 0 commits2026-04-27: 0 commits2026-04-28: 0 commits2026-04-29: 1 commit2026-04-30: 0 commits2026-05-01: 1 commit2026-05-02: 0 commits2026-05-03: 0 commits2026-05-04: 0 commits2026-05-05: 0 commits2026-05-06: 0 commits2026-05-07: 0 commits2026-05-08: 0 commits2026-05-09: 0 commits2026-05-10: 0 commits2026-05-11: 0 commits2026-05-12: 0 commits2026-05-13: 0 commits2026-05-14: 0 commits2026-05-15: 0 commits2026-05-16: 0 commits2026-05-17: 0 commits2026-05-18: 0 commits2026-05-19: 0 commits2026-05-20: 0 commits2026-05-21: 0 commits2026-05-22: 1 commit2026-05-23: 2 commits2026-05-24: 3 commits2026-05-25: 1 commit2026-05-26: 3 commits2026-05-27: 0 commits2026-05-28: 0 commits2026-05-29: 1 commit2026-05-30: 0 commits2026-05-31: 6 commits2026-06-01: 1 commit2026-06-02: 1 commit2026-06-03: 0 commits2026-06-04: 0 commits2026-06-05: 0 commits2026-06-06: 0 commits2026-06-07: 0 commits2026-06-08: 0 commits2026-06-09: 0 commits2026-06-10: 0 commits2026-06-11: 1 commit2026-06-12: 0 commits2026-06-13: 0 commits2026-06-14: 0 commits2026-06-15: 0 commits2026-06-16: 1 commit2026-06-17: 0 commits2026-06-18: 1 commit2026-06-19: 0 commits2026-06-20: 4 commits2026-06-21: 0 commits2026-06-22: 0 commits2026-06-23: 0 commits2026-06-24: 0 commits2026-06-25: 0 commits2026-06-26: 0 commits2026-06-27: 1 commit2026-06-28: 0 commits2026-06-29: 0 commits2026-06-30: 0 commits2026-07-01: 0 commits2026-07-02: 0 commits2026-07-03: 2 commits2026-07-04: 1 commit2026-07-05: 0 commits2026-07-06: 0 commits2026-07-07: 0 commits2026-07-08: 0 commits2026-07-09: 0 commits2026-07-10: 2 commits2026-07-11: 0 commits2026-07-12: 0 commits2026-07-13: 0 commits2026-07-14: 0 commits2026-07-15: 0 commits2026-07-16: 0 commits2026-07-17: 0 commits2026-07-18: 1 commit2026-07-19: 0 commits2026-07-20: 0 commits2026-07-21: 0 commits2026-07-22: 0 commits2026-07-23: 0 commits2026-07-24: 0 commits2026-07-25: 0 commits2026-07-26: 0 commits2026-07-27: 0 commits2026-07-28: 0 commits2026-07-29: 1 commit2026-07-30: 0 commits2026-07-31: 0 commits2026-08-01: 0 commits2026-08-02: 1 commit2026-08-03: 3 commits2026-08-04: 0 commits2026-08-05: 0 commits2026-08-06: 0 commits2026-08-07: 0 commits2026-08-08: 0 commits2026-08-09: 2 commits2026-08-10: 0 commits2026-08-11: 0 commits2026-08-12: 0 commits2026-08-13: 1 commit2026-08-14: 0 commits2026-08-15: 1 commit2026-08-16: 1 commit2026-08-17: 2 commits2026-08-18: 0 commits2026-08-19: 0 commits2026-08-20: 1 commit2026-08-21: 1 commit2026-08-22: 0 commits2026-08-23: 0 commits2026-08-24: 0 commits2026-08-25: 0 commits2026-08-26: 1 commit2026-08-27: 0 commits2026-08-28: 0 commits2026-08-29: 1 commit2026-08-30: 0 commits2026-08-31: 1 commit2026-09-01: 0 commits2026-09-02: 0 commits2026-09-03: 0 commits2026-09-04: 0 commits2026-09-05: 0 commits2026-09-06: 0 commits2026-09-07: 0 commits2026-09-08: 0 commits2026-09-09: 0 commits2026-09-10: 0 commits2026-09-11: 0 commits2026-09-12: 0 commits2026-09-13: 0 commits2026-09-14: 0 commits2026-09-15: 0 commits2026-09-16: 0 commits2026-09-17: 0 commits2026-09-18: 0 commits2026-09-19: 0 commits2026-09-20: 0 commits2026-09-21: 0 commits2026-09-22: 0 commits2026-09-23: 0 commits2026-09-24: 0 commits2026-09-25: 0 commits2026-09-26: 0 commits2026-09-27: 0 commits2026-09-28: 0 commits2026-09-29: 0 commits2026-09-30: 0 commits2026-10-01: 0 commits2026-10-02: 0 commits2026-10-03: 0 commits2026-10-04: 0 commits2026-10-05: 0 commits2026-10-06: 0 commits2026-10-07: 0 commits2026-10-08: 0 commits2026-10-09: 0 commits2026-10-10: 0 commits2026-10-11: 0 commits2026-10-12: 0 commits2026-10-13: 0 commits2026-10-14: 1 commit2026-10-15: 1 commit2026-10-16: 0 commits2026-10-17: 0 commits2026-10-18: 0 commits2026-10-19: 0 commits2026-10-20: 0 commits2026-10-21: 0 commits2026-10-22: 0 commits2026-10-23: 1 commit2026-10-24: 1 commit2026-10-25: 0 commits2026-10-26: 0 commits2026-10-27: 0 commits2026-10-28: 0 commits2026-10-29: 0 commits2026-10-30: 0 commits2026-10-31: 0 commits2026-11-01: 2 commits2026-11-02: 1 commit2026-11-03: 0 commits2026-11-04: 0 commits2026-11-05: 0 commits2026-11-06: 0 commits2026-11-07: 0 commits2026-11-08: 0 commits2026-11-09: 0 commits2026-11-10: 0 commits2026-11-11: 0 commits2026-11-12: 1 commit2026-11-13: 0 commits2026-11-14: 0 commits2026-11-15: 0 commits2026-11-16: 0 commits2026-11-17: 0 commits2026-11-18: 0 commits2026-11-19: 0 commits2026-11-20: 0 commits2026-11-21: 0 commits
95 commits in the last yearLessMore

Signals and awards

derived from tracked data
  • Permissive license

    MIT

  • Continuous integration

    Automated checks passing

What forge does

Forge is a specialized Python framework designed to make tool-calling highly reliable for self-hosted LLMs. It acts as a reliability layer inside an agentic loop, applying rescue parsing, retry nudges, and response validation to ensure models execute tools correctly. Developers can optionally enforce workflow structures using prerequisites and required steps to constrain the model's behavior. By focusing strictly on tool execution rather than orchestration, it significantly boosts the success rate of smaller local models (like 8B parameters). It is domain-agnostic and built to enhance existing single-agent setups rather than replace multi-agent graphs.

AI engineers and developers building robust single-agent systems using local or self-hosted LLMs. It requires proficiency in Python and an understanding of function calling APIs.

  • Rescue parsing: Enforces reliability for LLM tool-calling through rescue parsing and retry nudges.
  • Workflow constraints: Allows developers to define strict workflow constraints like required steps and prerequisites.
  • Response validation: Validates model responses to ensure they match expected schemas before execution.
  • Local model optimization: Specifically optimized to improve the tool-calling success rates of smaller, self-hosted models.
  • Domain-agnostic integration: Operates as a domain-agnostic layer that fits into existing single-agent loops.

Where teams use it

Local Agent Reliability

Dramatically improves the tool-calling accuracy of 8B parameter local models running on constrained hardware.

Strict Workflow Execution

Forces an LLM to complete a specific sequence of validation steps before returning a final answer.

Error Recovery

Automatically catches malformed JSON or hallucinated tool names and nudges the model to correct them.

Custom Agent Enhancement

Integrates into custom Python agents to handle the brittle parsing layer of LLM responses.

Getting started: pip install forge

README

main branch

forge

PyPI Tests codecov Python 3.12+ License: MIT

A reliability layer for self-hosted LLM tool-calling. You give forge a set of tools; the model calls whichever it wants in whatever order. Workflow structure is opt-in — required_steps, prerequisites, and terminal_tool let you constrain the loop when you need to, but forge's guardrails (rescue parsing, retry nudges, response validation) apply with zero required steps too.

Forge takes an 8B local model from single digits to 84% across forge's 26-scenario v0.7.0 eval suite — and even lifts Sonnet 4.6 from 85% to 98% on the same workload (Anthropic numbers measured in v0.6.0; not re-run in v0.7.0 since the cost is non-trivial).

What forge isn't:

  • Not an agent orchestrator. Forge sits inside one agentic loop and makes its tool calls reliable. Multi-agent graphs, DAG planners, and cross-agent coordination are out of scope.
  • Not a coding harness. Forge is domain-agnostic. If you're building a coding agent (or already using one like opencode, aider, Cline), proxy mode lifts your existing harness with forge's guardrails — no rewrite.

Three ways to use it:

  • Proxy server — Drop-in proxy (forge-proxy from the standalone distribution, or python -m forge.proxy from the Python package) speaking both the OpenAI chat-completions and Anthropic Messages (/v1/messages) APIs, sitting between any client and a local model server. Point OpenAI-compatible tools (opencode, Continue, aider) or Claude Code at it and forge applies guardrails transparently — the client thinks it's talking to a smarter model. Most popular entry point.

  • WorkflowRunner — Define tools, pick a backend, run structured agent loops. Forge manages the full lifecycle: system prompts, tool execution, context compaction, and guardrails. SlotWorker adds priority-queued access to a shared inference slot with auto-preemption — for multi-agent architectures where specialist workflows share a GPU slot. Best when you're building on forge directly.

  • Guardrails middleware — Use forge's reliability stack (composable middleware) inside your own orchestration loop. You control the loop; forge validates responses, rescues malformed tool calls, and enforces required steps.

Supports generic OpenAI-compatible endpoints, Ollama, llama-server (llama.cpp), Llamafile, vLLM, and Anthropic as backends.

Standalone Forge Proxy

Forge Proxy is a self-contained developer sidecar: point an OpenAI- or Anthropic-compatible client at it to add Forge guardrails without rewriting the client or integrating the Python library. The command bundles Forge, its private Python runtime, and HTTP clients, so the host does not need Python or pip. It does not install a backend executable, model, GPU stack, service, credentials, or client configuration.

Install the latest verified standalone Proxy release:

Linux and macOS:

curl -fsSL https://raw.githubusercontent.com/antoinezambelli/forge/main/install.sh | sh

Windows PowerShell:

irm https://raw.githubusercontent.com/antoinezambelli/forge/main/install.ps1 | iex

Open a refreshed terminal, then create and validate a profile:

forge-proxy init
forge-proxy check

See Forge Proxy Installation for supported platforms, exact-version installation, profiles, updates, recovery, and uninstall.

Python Library Install

Use the Python package for WorkflowRunner, guardrails middleware, development, or a Python-managed Proxy. It requires:

  • Python 3.12+
  • A running LLM backend (see below)
pip install forge-guardrails                # core only
pip install "forge-guardrails[anthropic]"   # supported alias; client uses core HTTP dependencies

The Python package intentionally does not install a global forge-proxy command. Run its Proxy implementation with python -m forge.proxy; the standalone installer above is the sole owner of the forge-proxy command and its update/uninstall lifecycle.

For development:

git clone https://github.com/antoinezambelli/forge.git
cd forge
pip install -e ".[dev]"

Backend setup (pick one)

llama-server (recommended — top 10 eval configs all run on llama-server):

# Install from https://github.com/ggml-org/llama.cpp/releases
llama-server -m path/to/Ministral-3-8B-Instruct-2512-Q8_0.gguf --jinja -ngl 999 --port 8080

Ollama (alternative — easier setup, slightly weaker on harder workloads):

# Install from https://ollama.com/download
ollama pull ministral-3:8b-instruct-2512-q4_K_M

Anthropic (API, no local GPU needed):

pip install -e ".[anthropic]"
export ANTHROPIC_API_KEY=sk-...

See Backend Setup for full instructions and Model Guide for which model fits your hardware.

Quick Start

Start llama-server however you normally do (e.g. in a separate shell):

llama-server -m path/to/Ministral-3-8B-Instruct-2512-Q8_0.gguf --jinja -ngl 999 --port 8080

Then the Python you'll run (e.g. from another shell):

import asyncio
from pydantic import BaseModel, Field
from forge import (
    Workflow, ToolDef, ToolSpec,
    WorkflowRunner, LlamafileClient,
    ContextManager, TieredCompact,
)

def get_weather(city: str) -> str:
    return f"72°F and sunny in {city}"

class GetWeatherParams(BaseModel):
    city: str = Field(description="City name")

workflow = Workflow(
    name="weather",
    description="Look up weather for a city.",
    tools={
        "get_weather": ToolDef(
            spec=ToolSpec(
                name="get_weather",
                description="Get current weather",
                parameters=GetWeatherParams,
            ),
            callable=get_weather,
        ),
    },
    required_steps=[],
    terminal_tool="get_weather",
    system_prompt_template="You are a helpful assistant. Use the available tools to answer the user.",
)

async def main():
    client = LlamafileClient(
        gguf_path="path/to/Ministral-3-8B-Instruct-2512-Q8_0.gguf",
        mode="native",
        recommended_sampling=True,
    )
    ctx = ContextManager(strategy=TieredCompact(keep_recent=2), budget_tokens=8192)
    runner = WorkflowRunner(client=client, context_manager=ctx)
    await runner.run(workflow, "What's the weather in Paris?")

asyncio.run(main())

For multi-step workflows, multi-turn conversations, and backend auto-management, see the User Guide. If you're building a long-running session (CLI, chat server, voice assistant), see the long-running session advisory for important guidance on filtering transient messages.

Proxy Server

Upgrading from pre-0.9: 0.9.0 is one intentional breaking Proxy update. See Migrating to Forge 0.9 for the complete old/new/action table.

Drop-in proxy that sits between any client and a local model server, speaking both the OpenAI chat-completions API and the Anthropic Messages API (/v1/messages). Point your client at the proxy (e.g. http://localhost:8081/v1) and forge applies its guardrails transparently — the client thinks it's talking to a smarter model.

This is the path for using forge with an existing harness (opencode, Continue, aider, Cline, anything that speaks the OpenAI chat-completions schema — or Claude Code, which speaks the Anthropic Messages API). No Python rewrite. Reasoning replay defaults to none: Forge still captures reasoning for observability, but keeps it out of backend-facing history on later turns — the most token-efficient policy, and statistically indistinguishable from replay-all on the eval suite (see reasoning-replay results). Use --reasoning-replay keep-last to replay only the latest reasoning block, or --reasoning-replay full for the historical replay-all behavior.

# External llama-server — you manage the backend, forge proxies it
python -m forge.proxy --backend-url http://localhost:8080 --backend llamaserver --port 8081

# External Anthropic-shaped downstream
python -m forge.proxy --backend-url https://gateway.example --backend anthropic --model claude-route --port 8081

# Managed mode — forge spawns or attaches to the backend, then starts the proxy
python -m forge.proxy --backend llamaserver --gguf path/to/model.gguf --port 8081

# Managed vLLM — pass a model directory or HF repo id via --model-path
python -m forge.proxy --backend vllm --model-path /path/to/awq-dir --port 8081

Then configure your client to use http://localhost:8081/v1 as the API base URL.

Claude Code: the proxy also serves the Anthropic Messages API on POST /v1/messages, so you can point Claude Code at a forge-guarded local model — set ANTHROPIC_BASE_URL=http://localhost:8081 and ANTHROPIC_AUTH_TOKEN=anything for the claude process. See Using forge with Claude Code for the full setup (native-vs-prompt FC, Anthropic-shape downstreams, cache_control).

Backend compatibility:

  • Managed mode spawns llama-server, llamafile, or vLLM, or attaches to an existing Ollama daemon. Supported selectors are llamaserver, llamafile, ollama, and vllm (use --gguf for the GGUF-based backends, --model-path for vLLM, or --model for Ollama). Stopping Forge unloads the selected Ollama model but does not stop or own the daemon.
  • External mode uses --backend-url. Omission or --backend openai selects generic OpenAI compatibility; the complete explicit selector set is openai, anthropic, llamaserver, llamafile, ollama, and vllm. The specialized selectors choose the corresponding adapter and metadata behavior. On the generic OpenAI/llama profile, --model is a fallback used only when the inbound request omits model; an inbound value wins. On Anthropic and vLLM profiles, --model pins the wire identity. For unpinned vLLM, the first inference request discovers the served identity. --budget-tokens independently supplies only a reporting denominator and never suppresses required unpinned identity discovery.

What proxy mode fortifies

On tool-bearing inference requests, forge applies (in order):

  1. Response validation — each tool call in the model's response is checked against the tools array in the request. Calls to unknown tool names or with malformed shapes are caught before the response returns to your client.
  2. Rescue parsing — when the model emits tool calls in the wrong format (JSON in a code fence, Mistral's [TOOL_CALLS]name{args}, Qwen's <tool_call>...</tool_call> XML), forge extracts the structured call and re-emits it in the canonical OpenAI tool_calls schema. Biggest practical lift for Mistral-family models.
  3. Retry loop with error tracking — if validation fails, forge retries inference up to --max-retries (default 3) with a corrective tool-result message on the canonical channel, rather than returning a malformed response. From your client's perspective the proxy looks like a single request that just took a few extra ms.
  4. Optional synthetic respond tool injection — --inject-respond-tool opts into a synthetic respond tool when tools are present. It defaults off. When enabled, the call is stripped from the outbound response and becomes normal text. See ADR-013 for the rationale.

Tool-free requests bypass validation, rescue, and retry and go directly to the selected backend adapter.

What proxy mode does not do

Proxy mode is single-shot per request; some forge features need multi-turn workflow state that the OpenAI chat-completions schema doesn't carry:

  • Prerequisite enforcement and step-ordering — these need a workflow definition spanning turns. Available in WorkflowRunner.
  • Context compaction and session memory — proxy mode never compacts or deletes caller history. The preserved llama/OpenAI adapter normalization may merge consecutive visible same-role messages for backend template compatibility; that is distinct from budget-driven compaction. Managing the rolling window is the client's job.
  • Unmanaged backend operation — metadata and --budget-tokens are reporting-only. The operator owns model allocation/swaps, overflow rejection, readiness, and backend failures. Managed Proxy modes retain backend, manual, forge-full, and forge-fast allocation behavior.

Read-only backend metadata is forwarded only on GET /health, /v1/health, /v1/models, /models, and /props. Use /forge/health for Forge liveness. Forwarding is transparent: if the selected backend does not implement a route, Forge returns its status unchanged, including Ollama's 404 for /health. /forge/usage reports one last-completed process-local snapshot or 204; it is not a live meter, ledger, or persistent session API. Forge Proxy is a per-operator sidecar and does not authenticate callers; --backend-api-key authenticates Forge to the backend, not callers to Forge.

For the full guardrail surface, use WorkflowRunner directly. The proxy trades depth for "use forge with your existing setup, no rewrite."

Useful flags

Flag Default Purpose
--max-retries N 3 Retry budget per validation failure
--no-rescue (rescue on) Disable rescue parsing (debugging only)
--budget-mode {backend,manual,forge-full,forge-fast} backend Managed backend allocation/reporting mode; Proxy never compacts caller history
--budget-tokens N — Positive manual allocation with managed --budget-mode manual; reporting denominator only in unmanaged mode
--serialize / --no-serialize auto Force request serialization (single-slot backends)
--extra-flags ... — Terminal argv remainder for Forge-spawned llama-server, llamafile, or vLLM; rejected for Ollama and unmanaged mode

Managed Ollama also rejects process/KV controls it cannot apply: cache_type_k, cache_type_v, n_slots, and kv_unified.

Docker

You can run the forge proxy as a Docker container.

Build the image:

docker build -t forge-proxy .

Run the container:

# Connect to an external backend (e.g. vLLM hosted on the same machine)
docker run -p 8081:8081 forge-proxy --backend-url http://host.docker.internal:8000 --backend vllm --budget-tokens 8192

Note: If your backend is running on localhost of the host machine, use http://host.docker.internal:PORT (on macOS/Windows) or the host's IP address to allow the container to reach it.

Backends

Backend Best for Native FC?
OpenAI-compatible Existing local or hosted OpenAI-shaped endpoints Yes
Ollama Easiest setup, model management built-in Yes
llama-server Best performance, full control Yes (with --jinja)
Llamafile Single binary, zero dependencies Yes, or prompt-injected
vLLM High-throughput serving, AWQ/GPTQ weights Yes (server-side parser)
Anthropic Frontier baseline, hybrid workflows Yes

See Backend Setup for installation and Model Guide for which model to pick.

Running Tests

python -m pytest tests/ -v --tb=short
python -m pytest tests/ --cov=forge --cov-report=term-missing

For proxy changes, also run the deterministic proxy smoke test and the manual real-backend sanity check described in Contributing.

Eval Harness

Scenarios measuring how reliably a model + backend combo navigates multi-step tool-calling workflows, with a baseline tier and an advanced_reasoning tier for top-end separation. See Eval Guide for the current scenario inventory and full CLI reference.

The released run-level outcome corpus is available as the Forge eval dataset on Hugging Face.

# llama-server (start in another terminal first; see Eval Guide)
python -m tests.eval.eval_runner --backend llamafile --llamafile-mode prompt --gguf "path/to/Ministral-3-8B-Instruct-2512-Q8_0.gguf" --runs 10 --stream --verbose

# Batch eval (JSONL output, automatic resume)
python -m tests.eval.batch_eval --config all --runs 50

# Reports — ASCII table by default; --html / --markdown export views
python -m tests.eval.report eval_results.jsonl
python -m tests.eval.report eval_results.jsonl --html docs/results/dashboard.html
python -m tests.eval.report eval_results.jsonl --markdown docs/results/

Project Structure

src/forge/
  __init__.py          # Public API exports
  errors.py            # ForgeError hierarchy
  server.py            # setup_backend(), ServerManager, BudgetMode
  core/
    messages.py        # Message, MessageRole, MessageType, MessageMeta
    workflow.py        # ToolSpec, ToolDef, ToolCall, TextResponse, Workflow
    inference.py       # run_inference() — shared front half (compact, fold, validate, retry)
    runner.py          # WorkflowRunner — the agentic loop
    slot_worker.py     # SlotWorker — priority-queued slot access
    steps.py           # StepTracker
  guardrails/
    guardrails.py      # Guardrails facade — applies the full stack in foreign loops
    nudge.py           # Nudge dataclass
    response_validator.py  # ResponseValidator, ValidationResult
    step_enforcer.py   # StepEnforcer, StepCheck
    error_tracker.py   # ErrorTracker
  clients/
    base.py            # ChunkType, StreamChunk, LLMClient protocol
    ollama.py          # OllamaClient (native FC)
    llamafile.py       # LlamafileClient (native FC or prompt-injected)
    openai_compat.py   # OpenAICompatClient (generic OpenAI-shaped endpoints)
    vllm.py            # VLLMClient (vLLM-specific identity and response handling)
    anthropic.py       # AnthropicClient (frontier baseline)
  context/
    manager.py         # ContextManager, CompactEvent
    strategies.py      # CompactStrategy, NoCompact, TieredCompact, SlidingWindowCompact
    hardware.py        # HardwareProfile, detect_hardware()
  prompts/
    templates.py       # Tool prompt builders (prompt-injected path)
    nudges.py          # Retry and step-enforcement nudge templates
  tools/
    respond.py         # Synthetic respond tool (respond_tool(), respond_spec())
  proxy/
    __main__.py        # CLI entry point: python -m forge.proxy
    proxy.py           # ProxyServer — programmatic start/stop API
    server.py          # Raw asyncio HTTP server, SSE streaming
    handler.py         # Request handler — bridge between HTTP and run_inference
    convert.py         # OpenAI messages ↔ forge Messages conversion
tests/
  unit/                # Deterministic tests — no LLM backend required
  eval/                # Eval harness — model qualification against real backends

Documentation

  • Forge Proxy Installation — Standalone platform installation, profiles, updates, recovery, and uninstall
  • User Guide — Usage patterns, multi-turn, context management, guardrails, slot worker, long-running session advisory
  • Model Guide — Which model and backend for your hardware
  • Backend Setup — Backend installation and server setup
  • Eval Guide — Eval harness CLI reference, batch eval
  • Architecture — Full design document
  • Workflow Internals — Workflow design and runner internals
  • Contributing — How to set up, test, and add new backends or scenarios

Paper

The forge guardrail framework and ablation study are published as:

Zambelli, A. Forge: Closing the Agentic Reliability Gap Between Self-Hosted and Frontier Language Models. https://doi.org/10.1145/3786335.3813193

A pre-publication preprint is also available at docs/forge_ieee_preprint.pdf — kept as a historical artifact. Cite the published version above; the DOI link may not resolve immediately depending on the publisher's release timing.

License

MIT — Copyright (c) 2025-2026 Antoine Zambelli

View on GitHub

Recent activity

commits and pull requests

Releases and announcements

19 total
  1. v0.10.1: Multipart text merge compatibilityv0.10.1Nov 2, 202652 downloads

    A multipart-text compatibility fix for the Python library and standalone Forge Proxy. ### Fixed - **Same-role message merging accepts multipart text content.** The Llamafile/llama-compatible adapter now extracts text from list content before merging, avoiding a `TypeError` in affected Proxy requests, including those using reasoning replay. Plain-string formatting and existing merge rules are unchanged; nonmerged messages retain their original shape. Merged content is text-only and does not preserve non-text blocks or block-level metadata. Fixes #151 via #163; thanks to @xh123637. - Standalone Forge Proxy bundles include the same multipart-text merge fix.

  2. v0.10.0: SDK-independent Anthropic transportv0.10.0Oct 24, 202656 downloads

    An Anthropic transport-compatibility release for the Python library and standalone Forge Proxy. Messages requests no longer depend on the installed Anthropic Python SDK's method signatures, auth defaults, or API-version default. ### Changed - **Anthropic transport no longer depends on the Python SDK.** Proxy and direct clients send Messages requests over HTTP. Native Anthropic requests retain the caller's `anthropic-version` through guardrail retries; requests without an Anthropic version default to `2023-06-01`. Both that contract and `2026-10-21` use the existing body conversion, without silently removing caller fields or upgrading requests. Other caller-header filtering, credential rules, guardrails, and reasoning behavior remain unchanged. - Anthropic transport errors now use Forge-authored summaries, keeping raw backend diagnostics on `BackendError.body` rather than in error messages. - Standalone Proxy bundles use the SDK-free HTTP client. The published `anthropic` install extra remains available as an empty compatibility alias.

  3. v0.9.5: Equivalent dual-auth compatibilityv0.9.5Aug 30, 2026123 downloads

    ## [0.9.5] — 2026-08-29 A credential-interoperability release for Forge's direct clients and standalone Proxy. Equivalent representations of one credential can now pass through without weakening Forge's rejection of genuinely distinct credentials. ### Changed - **Equivalent dual-auth headers represent one credential identity.** Direct clients and Forge Proxy now accept `Authorization` plus `X-Api-Key` when both carry the same normalized token. Same-protocol traffic preserves both headers; cross-protocol traffic emits one canonical target header. Unequal pairs, repeated same-name headers, and static-plus-inbound credentials remain rejected. This is client-agnostic compatibility for callers such as Claude Code's `apiKeyHelper`.

  4. v0.9.4: Anthropic reasoning capturev0.9.4Aug 26, 202636 downloads

    ## [0.9.4] — 2026-08-26 An Anthropic interoperability release for Forge Proxy and direct Anthropic client usage. Native plaintext thinking returned alongside tool calls now participates in Forge's existing reasoning-replay policy. ### Added - **Anthropic tool-call reasoning is captured consistently.** Streamed and non-streamed native `thinking` content is mapped to `ToolCall.reasoning`, preferred over the existing visible-text fallback, and exposed through Forge Proxy's OpenAI-compatible responses according to the configured replay policy. Text-only responses remain visible text, and native signed thinking-block round-tripping remains outside this release's scope. #152

  5. v0.9.3: Proxy command ownership hotfixv0.9.3Aug 22, 2026102 downloads

    ## [0.9.3] — 2026-08-21 A command-ownership hotfix for the standalone Forge Proxy distribution. Proxy request handling, routing, backend behavior, and guardrail policy are unchanged. ### Fixed - **The standalone installer exclusively owns `forge-proxy`.** The `forge-guardrails` Python package no longer creates a competing global command; Python-managed Proxy execution remains available through `python -m forge.proxy`, including in the Docker image. - **Foreign commands are preserved.** Installation and update refuse when an unowned `forge-proxy` is already on PATH or occupies the intended command pathname. Reinstall and uninstall also preserve a command that replaced a previously owned shim, leaving its package manager responsible for removal. - **Command ownership is exercised on every native candidate.** The shared lifecycle validates collision refusal, successful name-based resolution, replaced-command refusal, and ownership-aware uninstall on Windows, Linux, and macOS without release-specific migration logic.

Code frequency

additions and deletions
+56.4K-56.4KWeek of 2026-03-15: +56,362 linesWeek of 2026-03-15: -2,100 linesWeek of 2026-03-22: +2,556 linesWeek of 2026-03-22: -353 linesWeek of 2026-03-29: +2,237 linesWeek of 2026-03-29: -23,030 linesWeek of 2026-04-05: +1,452 linesWeek of 2026-04-05: -381 linesWeek of 2026-04-12: +143 linesWeek of 2026-04-12: -1 linesWeek of 2026-04-19: +2,742 linesWeek of 2026-04-19: -2,350 linesWeek of 2026-04-26: +8,261 linesWeek of 2026-04-26: -2,590 linesWeek of 2026-05-03: +0 linesWeek of 2026-05-03: -0 linesWeek of 2026-05-10: +0 linesWeek of 2026-05-10: -0 linesWeek of 2026-05-17: +1,567 linesWeek of 2026-05-17: -2,907 linesWeek of 2026-05-24: +4,867 linesWeek of 2026-05-24: -581 linesWeek of 2026-05-31: +3,400 linesWeek of 2026-05-31: -705 linesWeek of 2026-06-07: +2,762 linesWeek of 2026-06-07: -765 linesWeek of 2026-06-14: +863 linesWeek of 2026-06-14: -195 linesWeek of 2026-06-21: +3,187 linesWeek of 2026-06-21: -129 linesWeek of 2026-06-28: +1,041 linesWeek of 2026-06-28: -52 linesWeek of 2026-07-05: +840 linesWeek of 2026-07-05: -60 linesWeek of 2026-07-12: +21 linesWeek of 2026-07-12: -1 linesWeek of 2026-07-19: +0 linesWeek of 2026-07-19: -0 linesWeek of 2026-07-26: +982 linesWeek of 2026-07-26: -244 linesWeek of 2026-08-02: +4,145 linesWeek of 2026-08-02: -298 linesWeek of 2026-08-09: +18,575 linesWeek of 2026-08-09: -9,839 linesWeek of 2026-08-16: +9,647 linesWeek of 2026-08-16: -619 linesWeek of 2026-08-23: +797 linesWeek of 2026-08-23: -164 linesWeek of 2026-08-30: +29 linesWeek of 2026-08-30: -29 linesWeek of 2026-09-06: +0 linesWeek of 2026-09-06: -0 linesWeek of 2026-09-13: +0 linesWeek of 2026-09-13: -0 linesWeek of 2026-09-20: +0 linesWeek of 2026-09-20: -0 linesWeek of 2026-09-27: +0 linesWeek of 2026-09-27: -0 linesWeek of 2026-10-04: +0 linesWeek of 2026-10-04: -0 linesWeek of 2026-10-11: +441 linesWeek of 2026-10-11: -645 linesWeek of 2026-10-18: +1,223 linesWeek of 2026-10-18: -886 linesWeek of 2026-10-25: +0 linesWeek of 2026-10-25: -0 linesWeek of 2026-11-01: +125 linesWeek of 2026-11-01: -21 linesWeek of 2026-11-08: +162 linesWeek of 2026-11-08: -55 linesWeek of 2026-11-15: +0 linesWeek of 2026-11-15: -0 linesMar 15, 2026Nov 15, 2026
+128.4K lines added, -49K removed over the last year.

Commits per week

last 52 weeks
130Week of 2025-11-23: 0 commitsWeek of 2025-11-30: 0 commitsWeek of 2025-12-07: 0 commitsWeek of 2025-12-14: 0 commitsWeek of 2025-12-21: 0 commitsWeek of 2025-12-28: 0 commitsWeek of 2026-01-04: 0 commitsWeek of 2026-01-11: 0 commitsWeek of 2026-01-18: 0 commitsWeek of 2026-01-25: 0 commitsWeek of 2026-02-01: 0 commitsWeek of 2026-02-08: 0 commitsWeek of 2026-02-15: 0 commitsWeek of 2026-02-22: 0 commitsWeek of 2026-03-01: 0 commitsWeek of 2026-03-08: 0 commitsWeek of 2026-03-15: 8 commitsWeek of 2026-03-22: 13 commitsWeek of 2026-03-29: 10 commitsWeek of 2026-04-05: 1 commitsWeek of 2026-04-12: 2 commitsWeek of 2026-04-19: 1 commitsWeek of 2026-04-26: 2 commitsWeek of 2026-05-03: 0 commitsWeek of 2026-05-10: 0 commitsWeek of 2026-05-17: 3 commitsWeek of 2026-05-24: 8 commitsWeek of 2026-05-31: 8 commitsWeek of 2026-06-07: 1 commitsWeek of 2026-06-14: 6 commitsWeek of 2026-06-21: 1 commitsWeek of 2026-06-28: 3 commitsWeek of 2026-07-05: 2 commitsWeek of 2026-07-12: 1 commitsWeek of 2026-07-19: 0 commitsWeek of 2026-07-26: 1 commitsWeek of 2026-08-02: 4 commitsWeek of 2026-08-09: 4 commitsWeek of 2026-08-16: 5 commitsWeek of 2026-08-23: 2 commitsWeek of 2026-08-30: 1 commitsWeek of 2026-09-06: 0 commitsWeek of 2026-09-13: 0 commitsWeek of 2026-09-20: 0 commitsWeek of 2026-09-27: 0 commitsWeek of 2026-10-04: 0 commitsWeek of 2026-10-11: 2 commitsWeek of 2026-10-18: 2 commitsWeek of 2026-10-25: 0 commitsWeek of 2026-11-01: 3 commitsWeek of 2026-11-08: 1 commitsWeek of 2026-11-15: 0 commitsNov 23, 2025Nov 15, 2026
95 commits in the last 52 weeks.

When work happens

weekday and hour
SunMonTueWedThuFriSat036912151821Sun 0:00 — 0 commitsSun 1:00 — 0 commitsSun 2:00 — 1 commitsSun 3:00 — 0 commitsSun 4:00 — 0 commitsSun 5:00 — 0 commitsSun 6:00 — 0 commitsSun 7:00 — 0 commitsSun 8:00 — 0 commitsSun 9:00 — 0 commitsSun 10:00 — 1 commitsSun 11:00 — 0 commitsSun 12:00 — 0 commitsSun 13:00 — 2 commitsSun 14:00 — 2 commitsSun 15:00 — 1 commitsSun 16:00 — 0 commitsSun 17:00 — 0 commitsSun 18:00 — 0 commitsSun 19:00 — 3 commitsSun 20:00 — 0 commitsSun 21:00 — 0 commitsSun 22:00 — 5 commitsSun 23:00 — 4 commitsMon 0:00 — 1 commitsMon 1:00 — 0 commitsMon 2:00 — 1 commitsMon 3:00 — 0 commitsMon 4:00 — 0 commitsMon 5:00 — 0 commitsMon 6:00 — 0 commitsMon 7:00 — 0 commitsMon 8:00 — 1 commitsMon 9:00 — 1 commitsMon 10:00 — 0 commitsMon 11:00 — 0 commitsMon 12:00 — 1 commitsMon 13:00 — 0 commitsMon 14:00 — 0 commitsMon 15:00 — 0 commitsMon 16:00 — 1 commitsMon 17:00 — 2 commitsMon 18:00 — 1 commitsMon 19:00 — 1 commitsMon 20:00 — 2 commitsMon 21:00 — 0 commitsMon 22:00 — 2 commitsMon 23:00 — 1 commitsTue 0:00 — 0 commitsTue 1:00 — 1 commitsTue 2:00 — 0 commitsTue 3:00 — 0 commitsTue 4:00 — 0 commitsTue 5:00 — 3 commitsTue 6:00 — 1 commitsTue 7:00 — 0 commitsTue 8:00 — 0 commitsTue 9:00 — 0 commitsTue 10:00 — 0 commitsTue 11:00 — 0 commitsTue 12:00 — 0 commitsTue 13:00 — 0 commitsTue 14:00 — 0 commitsTue 15:00 — 0 commitsTue 16:00 — 1 commitsTue 17:00 — 2 commitsTue 18:00 — 0 commitsTue 19:00 — 0 commitsTue 20:00 — 0 commitsTue 21:00 — 1 commitsTue 22:00 — 1 commitsTue 23:00 — 1 commitsWed 0:00 — 1 commitsWed 1:00 — 0 commitsWed 2:00 — 1 commitsWed 3:00 — 0 commitsWed 4:00 — 0 commitsWed 5:00 — 0 commitsWed 6:00 — 0 commitsWed 7:00 — 0 commitsWed 8:00 — 0 commitsWed 9:00 — 1 commitsWed 10:00 — 0 commitsWed 11:00 — 0 commitsWed 12:00 — 0 commitsWed 13:00 — 0 commitsWed 14:00 — 0 commitsWed 15:00 — 0 commitsWed 16:00 — 0 commitsWed 17:00 — 0 commitsWed 18:00 — 0 commitsWed 19:00 — 3 commitsWed 20:00 — 0 commitsWed 21:00 — 0 commitsWed 22:00 — 1 commitsWed 23:00 — 1 commitsThu 0:00 — 2 commitsThu 1:00 — 0 commitsThu 2:00 — 1 commitsThu 3:00 — 1 commitsThu 4:00 — 1 commitsThu 5:00 — 0 commitsThu 6:00 — 0 commitsThu 7:00 — 0 commitsThu 8:00 — 1 commitsThu 9:00 — 1 commitsThu 10:00 — 0 commitsThu 11:00 — 0 commitsThu 12:00 — 0 commitsThu 13:00 — 0 commitsThu 14:00 — 0 commitsThu 15:00 — 0 commitsThu 16:00 — 1 commitsThu 17:00 — 1 commitsThu 18:00 — 1 commitsThu 19:00 — 0 commitsThu 20:00 — 2 commitsThu 21:00 — 0 commitsThu 22:00 — 1 commitsThu 23:00 — 1 commitsFri 0:00 — 1 commitsFri 1:00 — 1 commitsFri 2:00 — 0 commitsFri 3:00 — 0 commitsFri 4:00 — 0 commitsFri 5:00 — 0 commitsFri 6:00 — 0 commitsFri 7:00 — 2 commitsFri 8:00 — 0 commitsFri 9:00 — 1 commitsFri 10:00 — 1 commitsFri 11:00 — 0 commitsFri 12:00 — 0 commitsFri 13:00 — 0 commitsFri 14:00 — 0 commitsFri 15:00 — 0 commitsFri 16:00 — 0 commitsFri 17:00 — 0 commitsFri 18:00 — 4 commitsFri 19:00 — 1 commitsFri 20:00 — 0 commitsFri 21:00 — 0 commitsFri 22:00 — 2 commitsFri 23:00 — 0 commitsSat 0:00 — 1 commitsSat 1:00 — 0 commitsSat 2:00 — 2 commitsSat 3:00 — 0 commitsSat 4:00 — 1 commitsSat 5:00 — 0 commitsSat 6:00 — 0 commitsSat 7:00 — 0 commitsSat 8:00 — 1 commitsSat 9:00 — 0 commitsSat 10:00 — 0 commitsSat 11:00 — 0 commitsSat 12:00 — 0 commitsSat 13:00 — 1 commitsSat 14:00 — 0 commitsSat 15:00 — 0 commitsSat 16:00 — 1 commitsSat 17:00 — 3 commitsSat 18:00 — 1 commitsSat 19:00 — 1 commitsSat 20:00 — 0 commitsSat 21:00 — 2 commitsSat 22:00 — 0 commitsSat 23:00 — 1 commits
Commit volume by weekday and hour (UTC). Larger dots mean more commits.

Who is committing

last 52 weeks
Maintainer commits82 (86%)
Community commits13 (14%)

95 commits in total over the last year.

DateListRankStars gained
May 20, 2026daily#12+58
  • public-apis/public-apis

    A collective list of free APIs

    500.5K stars · Python

  • morluto/rea

    Reverse engineer anything with agents, from app behavior down to native binaries.

    471.6K stars · TypeScript

  • donnemartin/system-design-primer

    Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

    378.7K stars · Python

  • awesome-selfhosted/awesome-selfhosted

    A list of Free Software network services and web applications which can be hosted on your own servers

    339K stars

  • affaan-m/ECC

    The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

    307.9K stars · JavaScript

  • practical-tutorials/project-based-learning

    Curated list of project-based tutorials

    290.1K stars · Python