Architecture¶

Overview¶

flowchart TB
    agent_eth["<b>pa-container-agent</b><br/><br/>eth0<br/>isolated-net"]

    subgraph net["isolated-net<br/>(--internal, no gateway)"]
        direction TB
    end
    style net fill:none,text-align:left

    subgraph proxy["<b>pa-container-agent-proxy</b>"]
        direction TB
        mitm["mitmproxy<br/>transparent :8080<br/>allowlist + token_replacer"]
        dns["mitmproxy<br/>DNS :5353<br/>resolves 'llama' / provider hosts → eth1 IP"]
    end
    style proxy fill:none,text-align:left

    subgraph host["<b>Host</b><br/>llama-server (host process)"]
        direction TB
    end
    style host fill:none,text-align:left

    eth1["eth1<br/>isolated-net"]
    eth0["eth0<br/>upstream network<br/>internet + MASQUERADE"]

    %% L3 routing: agent → isolated-net → proxy eth1
    agent_eth -.->|L3 routed via proxy eth1 IP| eth1
    agent_eth --- net
    net --- eth1

    %% Proxy internal: L4 interception & forwarding
    eth1 -->|REDIRECT<br/>80/443 → :8080| mitm
    eth1 -->|REDIRECT<br/>53 → :5353| dns
    eth1 -->|DNAT provider:<cp> → llama_net| llama_net
    mitm -->|egress| eth0
    dns -->|egress| eth0
    eth1 -.->|opt-in FORWARD → eth0 → MASQUERADE| eth0
    eth0 -->|internet| Out["Internet"]

    subgraph llama_net["llama-server reachability"]
        direction TB
        podman_net["podman<br/>host.containers.internal<br/>(gvproxy)"]
    end
    style llama_net fill:none,text-align:left

    llama_net --> host

The key components of the system are

1. llama-server (host process): Runs llama.cpp's llama-server natively on the host. It provides OpenAI-compatible API endpoints for one or more local LLM models. Each model is permitted in the workspace via <project>/.pa-container/models.yaml and resolved against host model definitions at ~/.config/pa-container/models/<model-id>.yaml or gateways at ~/.config/pa-container/gateways/<gateway-id>.yaml. The system seeds models.yaml from the coding-agent/default/project/ template and default model definitions from coding-agent/default/global/models/ on first run. Unlike the proxy, a llama-server is a shared host resource. It uses the provider name plus a fingerprint of the model definition configuration (flags and model files) as its key. Projects with an identically configured provider share one process. This avoids loading the model twice. A same-named provider with different parameters gets its own server. This prevents attaching the wrong model.

2. pa-container-agent-proxy (container): A transparent proxy container based on Debian with mitmproxy. It intercepts the pi container's HTTP, HTTPS, and DNS traffic. The system installs a self-signed CA certificate into the pi container image so it can decrypt HTTPS. Each workspace gets its own proxy and its own isolated network. The system names the proxy by a hash of the project path. Its mitmweb web UI runs on an auto-assigned host port that the system logs at startup. Two addons run on the intercepted traffic. The first is an allowlist that blocks non-allowlisted hosts. The second is a token_replacer that redacts API keys, bearer tokens, cookies, and JWTs. The system denies all non-HTTP protocols by default. The agent cannot reach the internet except through the proxy. The agent can only use protocols that the proxy inspects (HTTP, HTTPS, DNS) or that you explicitly opt in to (see Proxy egress policy).

3. pa-container-agent (container): The main agent container. The system builds it from debian:trixie-slim. It includes Node 26, Python 3.14.6, uv for dependency management, and (for nested containers) rootless podman plus podman-compose. The toolchain builder image compiles all of these from source and copies them in. No workspace needs to install them. The agent connects only to an internal isolated-net network. Its default route and DNS point at the proxy so all traffic goes through it. The proxy reaches the host llama-server via host.containers.internal on podman for macOS or directly on Linux. This requires no socat. With nested_containers.enabled, the agent runs its own rootless podman inside itself. Those containers live in the agent's namespaces. Their traffic also goes through the proxy (see Nested containers).

Networking¶

The proxy's iptables rules enforce a default-deny forward policy. HTTP, HTTPS, and DNS from the agent go through mitmproxy. The proxy uses REDIRECT to ports 8080 and 5353. This bypasses the FORWARD chain entirely. The system denies every other protocol by default. Opt-in forwarding uses plain NAT. You configure it per-project in .pa-container/config.yaml under egress.allow. Options include ssh, smtp, git, ntp, and custom TCP or UDP ports. mitmproxy does not inspect these opt-in connections. The system creates isolated-net with --internal. This removes the external gateway. The agent has no route to the internet except the default route and DNS pointed at the proxy.

By default the whole stack is IPv4-only. The isolated network has no IPv6 subnet. Both containers disable IPv6 via sysctl. This prevents agent traffic from escaping the transparent-proxy REDIRECT over v6. Set network.ipv6: true in the project's .pa-container/config.yaml to enable IPv6. IPv6 mode creates the network with an IPv6 subnet (--ipv6). It mirrors the proxy's REDIRECT, NAT, and FORWARD rules in ip6tables. It gives the agent an IPv6 default route through the proxy. The container runtime and the host must both have IPv6 egress. This option targets Linux hosts running podman with working host IPv6. Leave it false on macOS.

Startup orchestration & parallel initialization¶

When pi runs (src/run.py), startup dependencies are orchestrated concurrently using start_dependencies_parallel() in src/containers.py:

flowchart TB
    Sweep["<b>1. Crash Recovery & Sweepers</b><br/>Reap dead servers, stale proxy locks, and dangling containers"]

    Sweep --> S1
    Sweep --> S2
    Sweep --> Proxy

    subgraph Parallel["<b>2. Parallel Dependency Startup</b> (start_dependencies_parallel)"]
        direction TB
        S1["<b>llama-server 1</b><br/>Port allocated<br/>Model weight loading & /health polling"]
        S2["<b>llama-server 2</b><br/>Port allocated<br/>Model weight loading & /health polling"]
        Proxy["<b>pa-container-agent-proxy</b><br/>Create isolated network<br/>Start proxy container & poll mitmweb"]
    end
    style Parallel fill:none,text-align:left

    S1 -.->|"port_ready_event (~15ms)"| Proxy
    S2 -.->|"port_ready_event (~15ms)"| Proxy

    S1 --> Barrier["<b>3. Health Readiness Barrier</b><br/>Synchronizes all server & proxy /health checks"]
    S2 --> Barrier
    Proxy --> Barrier

    Barrier -->|All healthy| Agent["<b>4. Start pa-container-agent</b><br/>Attach to isolated-net<br/>Route default gateway & DNS to proxy"]
    Barrier -.->|Any failure or SIGINT| Rollback["<b>ExitStack Rollback</b><br/>Clean teardown of all started resources"]

Dependency lifecycle stages¶

1. Crash recovery & orphan sweeping:

   • containers.sweep_orphaned_servers() stops host llama-server processes whose client sessions have terminated.
   • containers.sweep_orphaned_proxies() cleans up stale proxy locks, stops orphaned proxy containers, and removes dangling bridge networks.
   • containers.cleanup_orphaned_agent_containers() removes exited or abandoned agent containers from prior crashed runs.

2. Parallel dependency startup (start_dependencies_parallel):

   • Multi-server concurrency: If multiple local models are allowed in models.yaml, all llama-server instances launch in parallel worker threads, downloading models and loading weights into memory/VRAM concurrently.
   • Port readiness signaling: Each Server assigns its host port immediately upon process spawn and signals port_ready_event (~15ms).
   • Concurrent proxy initialization: ContainerNetworkManager builds LLAMA_PORTS port forwarding rules as soon as server ports are allocated, launching the proxy container and polling mitmweb health in parallel with server weight loading.
   • Health synchronization barrier: The coordinator thread waits for all servers and the proxy container to complete health verification.
   • Clean rollback via ExitStack: If any server or the proxy fails to start (e.g. OOM or timeout), ExitStack automatically cancels outstanding startup tasks and cleans up all started containers and processes.

3. Agent container launch:

   • Inspects the running proxy's internal interface IP and provisions shadow volumes and tmpfs mounts.
   • Launches pa-container-agent attached to the isolated bridge network with default route and DNS directed to the proxy.

For in-depth explanations of proxy egress forwarding rules, nested container architecture, and the toolchain builder image, see Design details.

Project Structure¶

.
├── build.sh                          # Build script (delegates to src/build.py)
├── run.sh                            # Run script (delegates to src/run.py)
├── pyproject.toml                    # uv project: dependencies, dependency-groups, ruff/mypy/pytest config
├── uv.lock                           # Pinned dependency lockfile (committed)
├── .gitignore
│
├── src/                              # Python source for build and run utilities
│   ├── build.py                      # Builds proxy and agent container images
│   ├── run.py                        # Orchestration entrypoint: validation + main() lifecycle
│   ├── config.py                     # Shared constants (paths, env, logging) — no side effects
│   ├── config_types.py               # Shared TypedDict declarations (zero circular dependencies)
│   ├── runtimes.py                   # ContainerRuntime abstraction (PodmanRuntime)
│   ├── models.py                     # Model + ServerConfig/ModelConfig (download, checksum)
│   ├── server.py                     # Server: llama-server lifecycle & multi-client refcounting
│   ├── network.py                    # ContainerNetworkManager: proxy + isolated network lifecycle
│   ├── containers.py                 # Dependency orchestration (start_dependencies_parallel, sweeper)
│   ├── images.py                     # Image resolution, hashing, building, and pruning
│   ├── volumes.py                    # Named shadow and nested container volume management
│   ├── project.py                    # Project key derivation, project scoping, and template seeding
│   ├── security.py                   # Security configuration and mount policy validation
│   ├── yaml_strict.py                # Strict duplicate-key YAML parser
│   ├── util.py                       # Shared utilities (env loading, validation, port allocation, signals)
│   └── tests/                        # Pytest test suite
│       ├── conftest.py
│       ├── test_build.py
│       ├── test_config_schema.py
│       ├── test_config_types.py
│       ├── test_containers.py
│       ├── test_flow_export.py
│       ├── test_images.py
│       ├── test_models.py
│       ├── test_network.py
│       ├── test_project.py
│       ├── test_run.py
│       ├── test_runtimes.py
│       ├── test_security.py
│       ├── test_server.py
│       ├── test_util.py
│       ├── test_volumes.py
│       └── test_yaml_strict.py
│
├── coding-agent-builder/          # Toolchain builder image (not shipped; agent + proxy COPY --from it)
│   ├── Containerfile                 # EVERY version/commit/sha pin (as ARGs) + one stage per component
│   ├── common.sh                     # Verified fetch/clone, required-ARG check, memory-capped jobs, manifest
│   ├── build-node.sh                 # Node 26 (official tarball, or from source; replaces the node: image)
│   ├── build-python.sh               # CPython 3.14 (from source, PGO) + uv + podman-compose
│   ├── build-go.sh                   # Go compiler toolchain (verified official tarball, staged)
│   ├── build-rust.sh                 # Rust toolchain (verified official installer, staged)
│   ├── build-podman.sh               # podman 6.x with an explicit BUILDTAGS set (not Debian's 5.4)
│   └── build-network.sh              # netavark + aardvark-dns 2.0 (the pair podman 6 requires)
│
├── coding-agent/                  # Main agent container
│   ├── Containerfile                 # Image definition (apt, toolchain COPY, pi, /etc/containers, CA)
│   ├── entrypoint.sh                 # Container entrypoint (default route via proxy, git config, uv venv)
│   └── default/                      # Seed templates (global and project configurations)
│       ├── global/                   # Seed template for ~/.config/pa-container/
│       │   ├── config.yaml           # Global configuration (runtime, llama, build)
│       │   ├── models/               # Seed model definitions (~/.config/pa-container/models/)
│       │   │   └── Qwen3.6-35B-A3B-UD-Q6_K_XL.yaml
│       │   ├── gateways/             # Seed gateway directory (~/.config/pa-container/gateways/)
│       │   └── chat-templates/       # Bundled Jinja chat templates (~/.config/pa-container/chat-templates/)
│       │       └── Qwen3.6-35B-A3B-UD-Q6_K-XL/
│       │           └── chat_template.jinja
│       └── project/                  # Seed template for <project>/.pa-container
│           ├── agents/               # → .pa-container/agents
│           │   └── pi/               # Pi coding agent template
│           │       ├── AGENTS.md     # Agent instructions
│           │       ├── config.json
│           │       ├── settings.json
│           │       └── .pi_ignore
│           ├── models.yaml           # Allowed models allowlist for the workspace
│           ├── config.yaml           # Orchestration: resources, tmpfs, flow_export, egress, nested_containers
│           ├── allowlist.yaml        # Generic hostname allowlist (pypi/npm/github/apt)
│           ├── token_replacer.yaml   # Generic token-redaction config
│           └── dependencies/         # Dependency command templates
│               ├── agent/commands.sh
│               └── root/commands.sh
│
├── coding-agent-proxy/            # Transparent proxy container
│   ├── Containerfile                 # mitmproxy + addon scripts/configs + pyyaml
│   ├── entrypoint.sh                 # iptables (redirect/DNAT, default-deny FORWARD) + mitmweb with addons
│   └── addons/
│       ├── allowlist/                # Hostname/IP allowlist addon (active)
│       └── token_replacer/           # Token redaction addon (API keys, Bearer tokens, cookies, JWTs) (active)
│
├── llama-server/                     # LLM server components
│   ├── models/                       # Downloaded GGUF model files (gitignored)
│   ├── logs/                         # llama-server log files (gitignored)
│   └── .locks/                       # Process lock files (gitignored)
│       └── local-gemma/              # Per-model lock directory
│           ├── .llama_server.pid
│           └── .llama_server_clients.json
│
├── docs/setups/                      # Model-specific setup directories
│   └── gemma-4-26b-a4b-it-qat-GGUF/  # Notes and config for specific model setups
│
└── .pa-container/                    # Per-project config (this repo's own; each workspace gets its own)
    ├── agent/                        # Agent launch config (sessions, config.json, …)
    ├── models.yaml                   # Allowed models allowlist
    ├── config.yaml                   # Orchestration: resource limits, tmpfs, flow-export, egress
    ├── token_replacer.yaml           # Token redaction rules (mounted into this project's proxy)
    ├── allowlist.yaml                # Hostname allowlist (mounted into this project's proxy)
    └── exports/                      # Captured flow history for this project (gitignored)