Command line

This page lists every command of the four programs shippy ships, with its flags. Use it to look up a command; the quickstart and the guides show them in context.

Program	Where it runs	What it is for
`shippy`	every machine that builds	runs cargo through the cache
`shippyd`	the box, and worker machines	the shared cache server, its tokens and its admin tools
`shippy-warm`	the box or a machine of its own	builds new commits ahead of people
`shippy-bench`	a developer machine	measures and tunes rebuild time

A box is a machine that runs shippyd and holds the shared cache. See the glossary for other terms.

shippy

shippy is one binary with two jobs. Called as shippy, it is the front end you type. Called under the name shippy-rustc, it is the compiler wrapper cargo runs for each crate (cargo's RUSTC_WRAPPER). You never call the wrapper yourself: the front end sets it up.

shippy <cargo arguments>
shippy setup --remote <url> --token-file <file> [--no-check]
shippy doctor
shippy push
shippy gc [--max-size SIZE] [--max-age AGE]
shippy stats
shippy wrapper
shippy broker
shippy --version
shippy help

shippy `<cargo arguments>`

shippy build
shippy test -p my-crate
shippy +nightly check

Runs cargo with the same arguments, through the cache. Anything that is not one of the subcommands below goes to cargo: build, check, test, run, clippy, doc and so on.

What the front end adds around cargo:

It reads shippy.toml at the workspace root and adds its [cargo] settings (toolchain, --config entries, extra arguments). A +toolchain you type yourself wins over the file's.
It points RUSTC_WRAPPER at the wrapper and passes the settings to it as SHIPPY_* variables.
With a remote, it prefetches: it downloads what the last build like this one used, before cargo starts.
While cargo runs, it uploads what this machine compiled, in rounds, and once more at the end.
At the end it prints one summary line, and may start a cleanup of the local store in the background.

The exit code is cargo's. Nothing the cache does changes it.

Loading diagram...

Settings come from client configuration.

shippy setup

shippy setup --remote http://box:7447 --token-file ~/Downloads/shippy-token

Points this user's shippy at a box, once. It writes two files to $XDG_CONFIG_HOME/shippy, else ~/.config/shippy: config.toml with the remote's URL, and token with your token. Both are readable by you alone. Then it asks the box whether it takes the token.

Flag	What it does
`--remote <url>`	The box, such as `http://box:7447`. Must start with `http://` or `https://`. Required.
`--token-file <file>`	A file that holds your token on one line, or `-` to read it from standard input. Required.
`--no-check`	Do not ask the box whether it takes the token.

Flags also take the --flag=value form. SHIPPY_REMOTE and SHIPPY_TOKEN in the environment still win over the file.

shippy doctor

shippy doctor

Checks the toolchain, the wrapper, the store, where the remote and token come from, the proxy, the box's latency (median of five /healthz), its clock, whether it takes the token and whether it is ready. Each problem comes with what to do. It also says whether the box compiles misses and whether its linker is this machine's. Exit code 1 when something is wrong. No flags.

shippy push

shippy push

Uploads results still waiting in the outbox, in one round. Use it after a build that could not reach the box. Exit code 1 when anything is still queued afterwards. It needs a remote: SHIPPY_REMOTE, or the user file from shippy setup. It sends what was queued for that remote: a result a build queued for another box waits for a push to that box.

shippy gc

shippy gc
shippy gc --max-size 20GB --max-age 14d

Removes what the local store holds beyond its limits. Results go least recently used first, each with the blobs only it used. Nothing stored or used in the last 15 minutes goes, nor a result queued for upload in the last day.

Flag	What it does
`--max-size SIZE`	How large the store may be: `50GB`, `500MB`, `8GiB`, or `off` for no limit.
`--max-age AGE`	Also remove results not used for this long: `14d`, `12h`, or `off`.

Without flags the limits are SHIPPY_MAX_SIZE and SHIPPY_MAX_AGE, else max_size and max_age under [cache] in your user file (~/.config/shippy/config.toml), else half of the store's own size plus the free space of its filesystem (at most 50GB) and no age limit.

shippy stats

shippy stats

Prints what the cache did in the last run here: the summary line of the newest session log in the store. Exit code 1 when no run is recorded yet.

shippy wrapper

export RUSTC_WRAPPER="$(shippy wrapper)"
cargo build

Prints the path to set RUSTC_WRAPPER to by hand: a link named shippy-rustc in <store>/bin. Plain cargo then uses the cache, but without the front end's prefetch, uploads or shippy.toml.

shippy broker

shippy broker

Starts a broker without a build. A broker keeps a few connections to the box open and lets the wrappers of a build share them over a Unix socket. The front end starts one for every build; this command is for cargo run by hand with shippy wrapper. It prints SHIPPY_BROKER_SOCKET=... and SHIPPY_BROKER_FOR=..., which the wrappers need in their environment, and serves until its standard input closes. It needs a remote. See docs/broker.md.

Internal entry points

The binary also answers to two flags that only shippy itself uses: --run-build-script and --run-build-script-here. They are launchers that sit in front of a build script when its run is cached or kept on this machine. Do not call them by hand. See build scripts.

shippyd

shippyd is the server on the box. With no command it serves.

shippyd [serve] [--config <shippyd.toml>] [settings]
shippyd worker [--config <shippyd.toml>] [settings]
shippyd config check | print-default
shippyd token add <principal> [--role trusted|untrusted|metrics|worker|trusted-worker]
                  [--expires 90d] [--require-trusted-pool]
shippyd token list
shippyd token revoke <principal|id>
shippyd admin who --key <action key>
shippyd admin purge (--key <action key> | --principal <name> [--include-verified]) [--dry-run]
shippyd admin workers [--clear <worker>]
shippyd admin quarantine [--clear <action key>]
shippyd help | --help | -h
shippyd version | --version | -V

Flags take --flag value or --flag=value. A bad argument exits with code 2.

shippyd serve

shippyd --store /dev/shm/shippy --listen 0.0.0.0:7447 --tokens tokens.toml
shippyd --config /etc/shippy/shippyd.toml

Serves the cache over HTTP until SIGTERM or Ctrl-C, then lets running requests finish for --shutdown-grace. SIGHUP reads the tokens file again. serve is the default command.

Every setting has a flag, a key in the config file and a SHIPPYD_* variable. The full list, with defaults, is in server configuration. The flags:

Flag	What it sets
`--config <file>`	The config file to read.
`--store`	The store directory.
`--listen`	Address and port to serve on.
`--tokens`	The tokens file.
`--verify-workers`	Sandboxed re-runs of claims at once.
`--verify-queue`	Claims waiting for a re-run in memory, for everyone.
`--verify-queue-per-principal`	The same, for one principal.
`--verify <bool>`	Whether claims are re-run.
`--no-verify`	Keep claims unverified: only their claimants see them.
`--exec-timeout`	Time one sandboxed compilation may take, such as `15m`.
`--exec-timeout-secs`	The same in seconds.
`--install-toolchains`	Toolchains rustup may install when any recipe names one, comma separated. Given twice, the lists add up.
`--max-toolchains`	Install no toolchain while this many are installed.
`--max-store-size`	How large the store may get.
`--max-age`	Also remove results nobody used for this long.
`--evict-grace`	Never remove what was stored or used within this long.
`--principal-share`	Percent of the store one untrusted principal may hold.
`--evict-every`	How often the store is checked against its limits.
`--log-format`	`text` or `json`.
`--log-level`	`trace`, `debug`, `info`, `warn` or `error`.
`--metrics-listen`	A second address serving `/metrics`, `/healthz` and `/readyz` without a token.
`--shutdown-grace`	Time running requests and sandboxes get after SIGTERM.
`--selftest-every`	How often the sandbox self-test behind `/readyz` runs.
`--exec-local`	`false`: the box runs no recipe itself, only its workers do.
`--workers-audit`	Percent of workers' runs run a second time and compared.
`--workers-audit-first`	How many of a worker's first runs are always run twice.
`--workers-wait`	How long a run waits for a fitting worker to appear.
`--workers-retries`	How often a run goes to another worker after one lost it.
`--workers-heartbeat`	How often a worker says a lease is alive.
`--workers-max-slots`	The most runs one worker is leased at once.

shippyd worker

shippyd worker --config /etc/shippy/shippyd-worker.toml

Runs on a worker machine: it asks the box for sandbox runs (leases), runs them and hands back the results. It holds no store, only a cache of inputs. It stops on SIGTERM and lets runs in flight finish for the shutdown grace. It reads the same kind of config file as the server.

Flag	What it sets
`--worker-store`	The box's address, such as `http://box:7447`. Required.
`--worker-token-file`	The file that holds the worker's token. Required.
`--worker-slots`	Runs at once. Default: half the cores.
`--worker-cache-size`	The most its cache of inputs may hold. Default: `20GiB`.
`--store`	The directory of that cache and of the runs' scratch space. Required.

It also reads --exec-timeout, --install-toolchains, --max-toolchains, --log-format, --log-level, --shutdown-grace, --selftest-every and the file's [env] table. See workers.

shippyd config

shippyd config print-default > shippyd.toml
shippyd config check --config shippyd.toml

Command	What it does
`config print-default`	Prints a config file with every key, its default and the variable that overrides it.
`config check`	Reads the settings as a start would and says what it found: the listen address, the store, how many tokens, the sandbox's limits, whether bubblewrap can create a sandbox and whether a Rust toolchain is there. Starts nothing.

shippyd token

A token is a secret a client sends as Authorization: Bearer <token>. It stands for a principal: one named identity, such as a person or a CI pool, with one role. The tokens file stores only the SHA-256 of each token.

shippyd token add alice                          # a person: untrusted
shippyd token add alice-laptop2 --expires 90d
shippyd token add ci-main --role trusted         # CI on protected branches only
shippyd token add prometheus --role metrics      # reads /metrics, nothing else
shippyd token list
shippyd token revoke alice                       # every token of the principal
shippyd token revoke 3f9a1c2e                    # one token, by its id

Command	What it does
`token add <principal>`	Makes a token and prints it once, alone on standard output, so you can pipe it into a secret store.
`token list`	Shows id, principal, role, creation time and expiry of every token.
`token revoke <principal or id>`	Removes every token of a principal, or one token by its id. A running shippyd refuses it from the next request.

Flag of `token add`	Default	What it does
`--role`	`untrusted`	`trusted`, `untrusted`, `metrics`, `worker` or `trusted-worker`. See token roles.
`--expires`	never	How long the token lives, such as `90d`.
`--require-trusted-pool`	off	Only for a `trusted` token: serve it only what a trusted principal uploaded or the trusted pool produced.

The token commands need only the tokens file: --tokens, SHIPPYD_TOKENS, or tokens.file in the config file.

shippyd admin

These work on the store of a running shippyd; no restart is needed. They read the config file's [env] table as the server does.

sudo -u shippy shippyd admin who --key <action key>
sudo -u shippy shippyd admin purge --key <action key> --dry-run
sudo -u shippy shippyd admin purge --principal <name> [--include-verified] [--dry-run]
sudo -u shippy shippyd admin workers [--clear <worker>]
sudo -u shippy shippyd admin quarantine [--clear <action key>]

An action key is the 64-hex-character key a result is stored under. The session log of a build lists each crate's keys (~/.cache/shippy/sessions/).

Command	What it does
`admin who --key K`	Shows who produced the results of a key: trust, publisher, time, outputs, the crate, whose claim it was compiled for, and who has claims for it.
`admin purge --key K`	Removes the key's results, every result made from their outputs, and every claim for it or made from them.
`admin purge --principal P`	Removes everything the principal put on the box: claims, packs, kept manifests, marks on blobs, results it uploaded as trusted, and what was made from those.
`admin workers`	Shows the worker machines. `--clear <worker>` lets a suspended worker lease again.
`admin quarantine`	Shows quarantined keys (keys whose runs disagreed). `--clear <action key>` lets one be run again.

Flag	What it does
`--include-verified`	With `--principal`: also remove the verified results the box compiled for it or that its uploads matched.
`--dry-run`	Say what would go and remove nothing.

admin purge takes --key or --principal, not both. Blobs are left to eviction.

shippy-warm

The warmer builds every new commit of the branches it watches through shippy, so the box has a commit's crates before people pull it. Give it an untrusted token.

shippy-warm [run] --config <shippy-warm.toml>
shippy-warm once --config <shippy-warm.toml>
shippy-warm config check --config <shippy-warm.toml>
shippy-warm config print-default

Command	What it does
`run`	The default. Asks the watched branches for their newest commit every interval and on `POST /trigger`, and builds each new one.
`once`	Asks once, builds what is new, and exits: 0 when every branch is built at its newest commit.
`config check`	Reads the file and reports how many repositories and branches it names, and any problem.
`config print-default`	Prints a config file with every key.

The file is --config, else SHIPPY_WARM_CONFIG, else /etc/shippy-warm/shippy-warm.toml. --version and --help work as usual.

sudo -u shippy-warm shippy-warm config check --config /etc/shippy-warm/shippy-warm.toml
curl -s 127.0.0.1:9448/status | jq '.targets | map_values({built, pending, failures, tip})'

shippy-bench

shippy-bench measures how long a cargo project takes to rebuild after a source edit, under plain cargo and under other setups (variants), and searches for the fastest combination. It always puts the edited file back.

shippy-bench run    --config <bench.toml> [--variant NAME]... [--scenario NAME]...
                    [--runs N] [--out DIR] [--repo DIR]
shippy-bench tune   --config <bench.toml> [--knobs <knobs.toml>] [--scenario NAME]...
                    [--runs N] [--out DIR] [--repo DIR] [--min-gain 0.03] [--prune]
shippy-bench tune   --file <source file> [--repo DIR] [--knobs <knobs.toml>] [--runs N]
                    [--out DIR] [--min-gain 0.03] [--prune] -- <cargo arguments>
shippy-bench report [--out DIR]

Command	What it does
`run`	Measures each variant on each scenario: cold build, no-op, and timed edits. Writes `results.json` and `results.md`.
`tune`	Starts from the `cargo` variant and tries each value of each knob (one decision, such as the linker), keeping a value when it lowers the score. Writes `tune.md`, `best.toml` and the winner as a `shippy.toml`, which it also prints. With `--file` and a cargo command after `--` it needs no config.
`report`	Renders the results in `--out` again.

Flag	Default	What it does
`--config`	none	Scenarios and variants to measure.
`--variant`	all	Measure only these variants (`run`). Repeatable.
`--scenario`	all	Measure only these scenarios. Repeatable.
`--runs`	the config's `runs`	Timed edits per scenario.
`--out`	`./shippy-bench-out`	Results, logs and target directories.
`--repo`	the config's `repo`	The cargo workspace.
`--file`	none	The file each edit goes to, relative to the repo, for a `tune` or `run` without `--config`; the cargo command follows `--`.
`--knobs`	the built-in ones for stable Rust	Knobs to tune over (`tune`).
`--min-gain`	`0.03`	A knob value must lower the score by more than this fraction (0 to 1).
`--prune`	off	Delete the target directory of every candidate that lost (`tune`).

shippy-bench run  --config bench/examples/ripgrep.toml --repo /tmp/ripgrep --out /tmp/rg-out
shippy-bench tune --config bench/examples/ripgrep.toml --repo /tmp/ripgrep --out /tmp/rg-out \
                  --knobs bench/knobs.default.toml

The settings tune finds go into the [cargo] section of shippy.toml. See tune a project and benchmarks.

Go deeper

README.md: using shippy and sharing a cache
docs/operations.md: tokens, admin commands and the warmer
docs/workers.md: shippyd worker
docs/broker.md: shippy broker
bench/README.md: shippy-bench

Edit this page on GitHub