Server configuration
This page lists every setting of shippyd, the server on the box, with its default. Use it when you write shippyd.toml, tune a box, or hand out tokens.
Every setting has three names that mean the same thing: a key in the config file, a SHIPPYD_* variable and a command-line flag. A wrong value is refused the same way wherever it came from, with an error that names where it was, and shippyd does not start.
Quick start
shippyd config print-default > /etc/shippy/shippyd.toml # every key, its default, its variable
sudoedit /etc/shippy/shippyd.toml
shippyd config check # what a start would find; starts nothingWhere settings come from
- The file.
--config <file>, elseSHIPPYD_CONFIG, else/etc/shippy/shippyd.tomlwhen it exists. No file is fine if the required settings come from elsewhere. - The environment goes over the file. The systemd unit also reads
/etc/shippy/shippyd.env. An empty variable counts as unset. - Flags go over both.
A key in the file that is not a setting is an error, so a typo is never silently ignored. A change to the file needs systemctl restart shippyd. Only the tokens file is followed live.
Required settings
shippyd will not start without these three. shippyd config print-default writes them with the values shown.
| Key | Variable | Flag | Value in the default file |
|---|---|---|---|
listen | SHIPPYD_LISTEN | --listen | 127.0.0.1:7447 |
store.path | SHIPPYD_STORE | --store | /var/lib/shippy/store |
tokens.file | SHIPPYD_TOKENS | --tokens | /etc/shippy/tokens.toml |
listenis an address and a port. Inside a tailnet, use the box's Tailscale address.store.pathis the store directory. The sandbox's scratch space is inside it, so its inputs are hard links: the filesystem must allow executables (nonoexec). A tmpfs keeps the cache in memory.tokens.fileholds the hashes of accepted tokens. Keep it out of/usrand the toolchains' directories, which the sandbox can read.
All settings
Durations are written like 30s, 15m, 12h, 30d. Sizes like 200GB or 64GiB.
Top level and [store]
| Setting | Default | What it does |
|---|---|---|
listen (SHIPPYD_LISTEN, --listen) | required | Address and port to serve on. |
store.path (SHIPPYD_STORE, --store) | required | The store directory. |
store.max_size (SHIPPYD_MAX_STORE_SIZE, --max-store-size) | auto | How large the store may get: a size, off, or auto (three quarters of the store's own size plus the free space of its filesystem). |
store.max_age (SHIPPYD_MAX_AGE, --max-age) | off | Also remove results nobody used for this long, and claims and packs this old. |
store.evict_grace (SHIPPYD_EVICT_GRACE, --evict-grace) | 15m | Never remove what was stored or used within this long. It does not protect what an untrusted principal sent when the store is over its limit. |
store.principal_share (SHIPPYD_PRINCIPAL_SHARE, --principal-share) | 10 | Percent (1 to 100) of max_size one untrusted principal may hold in claims and blobs no public result names. A request past it gets 507. |
store.evict_every (SHIPPYD_EVICT_EVERY, --evict-every) | 30s | How often the store is checked against its limits. |
See eviction.
[exec]
A claim is a result an untrusted principal uploaded. The box re-runs its recipe (the full description of one rustc call) in a sandbox before anyone else gets it. These settings control those re-runs.
| Setting | Default | What it does |
|---|---|---|
exec.verify (SHIPPYD_VERIFY, --verify, --no-verify) | true | false keeps claims unverified: only their claimants see them. |
exec.workers (SHIPPYD_VERIFY_WORKERS, --verify-workers) | half the cores | Sandboxed re-runs of claims at once. |
exec.queue (SHIPPYD_VERIFY_QUEUE, --verify-queue) | 16384 | Claims waiting for a re-run in memory, for everyone. One more waits on disk. |
exec.queue_per_principal (SHIPPYD_VERIFY_QUEUE_PER_PRINCIPAL, --verify-queue-per-principal) | 4096 | The same, for one principal. |
exec.timeout (SHIPPYD_COMPILE_TIMEOUT, --exec-timeout, --exec-timeout-secs) | 15m | How long one sandboxed compilation may take. |
exec.local (SHIPPYD_EXEC_LOCAL, --exec-local) | true | false: the box runs no recipe itself, only its workers do. |
[workers]
How the box treats its worker machines. See workers.
| Setting | Default | What it does |
|---|---|---|
workers.audit (SHIPPYD_WORKERS_AUDIT, --workers-audit) | 5% | Percent (0 to 100) of workers' runs that are run a second time elsewhere and compared. |
workers.audit_first (SHIPPYD_WORKERS_AUDIT_FIRST, --workers-audit-first) | 5 | How many of a worker's first runs are always run a second time. |
workers.wait (SHIPPYD_WORKERS_WAIT, --workers-wait) | 30s | How long a run waits for a worker that could take it to appear. |
workers.retries (SHIPPYD_WORKERS_RETRIES, --workers-retries) | 2 | How often a run goes to another worker after one lost it. |
workers.heartbeat (SHIPPYD_WORKERS_HEARTBEAT, --workers-heartbeat) | 5s | How often a worker says a lease is alive. A lease is lost after three times this without a word. |
workers.max_slots (SHIPPYD_WORKERS_MAX_SLOTS, --workers-max-slots) | 64 | The most runs one worker is leased at once, whatever it says it takes. |
[worker]
Read by shippyd worker alone, on a worker machine. A worker also reads store.path (its cache of inputs and scratch space), exec.timeout, [toolchains], [log], [lifecycle] and [env].
| Setting | Default | What it does |
|---|---|---|
worker.store (SHIPPYD_WORKER_STORE, --worker-store) | required | The box's address, such as http://box:7447. |
worker.token_file (SHIPPYD_WORKER_TOKEN_FILE, --worker-token-file) | required | The file that holds the worker's token. The worker refuses one in a directory every sandbox can read (/usr, a toolchain's directory, the entries of /etc a sandbox has). |
worker.slots (SHIPPYD_WORKER_SLOTS, --worker-slots) | half the cores | Runs at once. |
worker.cache_size (SHIPPYD_WORKER_CACHE_SIZE, --worker-cache-size) | 20GiB | The most its cache of inputs may hold. |
[toolchains]
A claim is re-run with the exact compiler its recipe names. These say which toolchains rustup may install on the box.
| Setting | Default | What it does |
|---|---|---|
toolchains.install (SHIPPYD_INSTALL_TOOLCHAINS, --install-toolchains) | [] | Toolchains to install when anyone's recipe names one: a release (1.100.0), a dated channel (nightly-2026-11-10) or a channel (stable, beta, nightly). A list in the file, comma separated elsewhere. Without it only a trusted principal's recipe installs one. Through a whole channel, an untrusted principal's recipe has only a toolchain near the newest one of that channel here installed: a stable release no older than two before the newest, a nightly or a beta of the twelve weeks before the newest (before today, where none of that channel is here), no stable release at all where none is here, and none into the last quarter of the places: those are for a trusted principal's recipe and for a toolchain named here itself. |
toolchains.max_installed (SHIPPYD_MAX_TOOLCHAINS, --max-toolchains) | 8 | Install nothing while this many toolchains are installed. |
[tokens], [log], [metrics], [lifecycle]
| Setting | Default | What it does |
|---|---|---|
tokens.file (SHIPPYD_TOKENS, --tokens) | required | The tokens file that shippyd token writes. |
log.format (SHIPPYD_LOG_FORMAT, --log-format) | text | text for people, json for one object per line. |
log.level (SHIPPYD_LOG, --log-level) | info | trace, debug, info, warn or error. |
metrics.listen (SHIPPYD_METRICS_LISTEN, --metrics-listen) | none | A second address serving /metrics, /healthz and /readyz without a token, such as 127.0.0.1:9447. Keep it on loopback. |
lifecycle.shutdown_grace (SHIPPYD_SHUTDOWN_GRACE, --shutdown-grace) | 30s | After SIGTERM, how long running requests and sandboxes get to finish. |
lifecycle.selftest_every (SHIPPYD_SELFTEST_EVERY, --selftest-every) | 5m | How often the sandbox self-test behind /readyz runs again. |
The [env] table
Some parts of the server read the environment themselves: the sandbox's limits, remote execution's quotas and build scripts. Put those variables in the file's [env] table. shippyd copies them into its own environment before anything starts, unless the real environment already has them. Only SHIPPYD_* names are allowed.
[env]
SHIPPYD_SANDBOX_MEMORY = "16GiB"
SHIPPYD_EXEC_SLOTS = 48Sandbox limits
What one sandboxed compilation may use. A run past a limit is ended and its claim stays a claim. Memory and CPU limits need a cgroup per run (the systemd unit's Delegate=yes).
| Setting | Default | What it does |
|---|---|---|
SHIPPYD_SANDBOX_MEMORY | a quarter of the machine's memory, 2 to 32 GiB | Memory of the whole run, its /tmp included. |
SHIPPYD_SANDBOX_CPUS | no limit | CPUs a run uses at once, such as 4 or 2.5. |
SHIPPYD_SANDBOX_JOBS | no cap but the box's cores | Threads a run's compiler is lent through its jobserver. Never more than SHIPPYD_SANDBOX_CPUS rounded up. |
SHIPPYD_SANDBOX_PIDS | 4096 | Processes and threads of a run. |
SHIPPYD_SANDBOX_OPEN_FILES | 16384 | Open files per process. |
SHIPPYD_SANDBOX_OUTPUT_BYTES | 8GiB | What a run may leave in its output directories. |
SHIPPYD_SANDBOX_OUTPUT_FILES | 20000 | How many files it may leave there. |
SHIPPYD_SANDBOX_TMP_BYTES | 2GiB | A run's /tmp, which is memory. |
SHIPPYD_SANDBOX_STRICT_INPUTS | 1 | 1: only the inputs the recipe declares. 0 also adds undeclared dependency metadata that public results lead to. Leave it on. |
SHIPPYD_SANDBOX_SECCOMP | 1 | 0 runs the sandbox without its system call filter. |
SHIPPYD_CGROUP | the parent of shippyd's own cgroup | The delegated cgroup each run's group is created in. |
SHIPPYD_IMAGE_ID | computed from the system directories | A name for the system image verified results are sealed with, so boxes provisioned alike agree. |
Remote execution
Limits of /v1/exec, the route that compiles a client's cache misses (remote execution). Sizes are bytes.
| Setting | Default | What it does |
|---|---|---|
SHIPPYD_EXEC_SLOTS | one per core | Compilations at once, for everyone. |
SHIPPYD_EXEC_PRINCIPAL_SLOTS | all the slots | Compilations at once for one principal. |
SHIPPYD_EXEC_PRINCIPAL_QUEUE | 256 | Requests one principal may have waiting. One more gets 429. |
SHIPPYD_EXEC_QUEUE | 4096 | Requests waiting, for everyone. |
SHIPPYD_EXEC_PRINCIPAL_STREAMS | 256 | Answers one principal may have open as streams beyond its requests that run or wait. One more gets 429. |
SHIPPYD_EXEC_RETRY_AFTER | 5 | Seconds a refused client is told to stay away. |
SHIPPYD_EXEC_MAX_RECIPE_BYTES | 16777216 | The longest recipe. |
SHIPPYD_EXEC_MAX_INLINE_BYTES | 67108864 | Inputs one request may bring in its own body. |
SHIPPYD_EXEC_MAX_INPUTS | 100000 | Files a recipe may declare. |
SHIPPYD_EXEC_MAX_INPUT_BYTES | 34359738368 | Their size together. |
SHIPPYD_EXEC_MAX_LISTED_BYTES | 4294967296 | Packed directories one request may make the box read to find its inputs. Past it: 413. |
SHIPPYD_EXEC_MAX_LISTED_ENTRIES | 2097152 | The entries of those directories. |
SHIPPYD_NARROW | 1 | 0 compiles a request that declares a whole package with all of its files, without first finding what the compiler reads, and once more when the request set variables the crate does not read. With 0 every file a package was declared with is listed for everyone who builds the crate, read or not: leave it on where principals must not learn of each other's checkouts. |
Build scripts
The box runs build scripts in its sandbox (build scripts). A box with exec.local = false runs none.
| Setting | Default | What it does |
|---|---|---|
SHIPPYD_SCRIPTS | 1 | 0 runs no build scripts; clients run their own. |
SHIPPYD_SCRIPT_TIMEOUT | 1800 | Seconds one script may run. |
SHIPPYD_SCRIPT_JOBS | half of shippyd's CPUs, 2 to 32 | NUM_JOBS for a script, and its CPUs with a cgroup per run. |
SHIPPYD_SCRIPT_OUTPUT_BYTES | 8GiB | What one script may leave in OUT_DIR. |
SHIPPYD_SCRIPT_OUTPUT_FILES | 200000 | How many files and directories. |
Connections
| Setting | Default | What it does |
|---|---|---|
SHIPPYD_MAX_CONNECTIONS | 8192 | Connections the server holds at once. One more waits to be accepted. |
SHIPPYD_MAX_CONNECTIONS_PER_ADDRESS | half of the above | Connections at once from one address (an IPv4 address, an IPv6 /64). One more is closed as it is accepted. Behind a proxy every client has the proxy's address: set it to SHIPPYD_MAX_CONNECTIONS. |
A request's body that stands still for a minute, or comes slower than four kilobytes a second after its first minute of waiting, is given up and its request answered 400.
Token roles
A principal is one named identity (a person, a CI pool, a scraper) with one role. It may have several tokens. Results and claims are kept per principal, so use one principal per person or machine pool.
shippyd token add alice # untrusted, the default
shippyd token add ci-main --role trusted
shippyd token add prometheus --role metrics
shippyd token add worker-1 --role worker| Role | What it may do |
|---|---|
untrusted | Everything a developer or an agent needs. Its uploads are claims, visible to itself only, until the box has compiled them in its sandbox. The default. |
trusted | Its uploads are public as they are. Give it only to CI that builds protected branches on machines you control. |
metrics | GET /metrics and nothing else. |
worker | A worker machine of the shared pool: it leases sandbox runs and hands back their results, and reaches nothing else, /metrics included. |
trusted-worker | The same, for the trusted pool, which runs only what trusted principals ask for. |
--require-trusted-pool on a trusted token has it served only what a trusted principal uploaded or the trusted pool produced.
The tokens file
[[token]]
id = "3f9a1c2e" # the first hex digits of the hash; token revoke takes it
principal = "alice"
role = "untrusted" # trusted, untrusted, metrics, worker or trusted-worker
sha256 = "3f9a1c2e..."
created = "2026-11-13T20:00:00Z"
expires = "2027-02-11T20:00:00Z" # optionalThe file holds only the SHA-256 of each token, so a copy of it lets nobody in. Tokens start with shp_. A running shippyd looks at the file on every request: a new token works and a revoked one is refused from the next request. SIGHUP (systemctl reload shippyd) reads it again anyway. A file that cannot be read or parsed lets nobody in until it is fixed.
Go deeper
- docs/operations.md: configuration, sandbox limits, tokens
- deploy/shippyd.toml: the deployed config file
- docs/remote-exec.md: remote execution limits
- docs/workers.md: workers and their settings
- crates/shippyd/src/config.rs: where every key is defined