Build scripts

shippy caches the runs of build scripts (build.rs) like compilations, and the box runs them in its sandbox so one person's run can be everyone's. Read this when a cold build is slow even with every crate restored, or when you need to decide which of your own packages' scripts may be shared.

Why it matters

A cold build with every compilation restored can still be minutes of build scripts: C and C++ compiled by cc, cmake and make (rocksdb, jemalloc, aws-lc), and code generated into OUT_DIR.

On a small machine those scripts are most of a cold build. A client of a warm box that compiled rocksdb and jemalloc itself took 332 s. The same build with their runs restored took 136 s.

The runs of downloaded packages' scripts are cached with no extra switch. Without a remote they are kept in this machine's store, for a second checkout or a cleaned target directory. With one they are shared:

shippy setup --remote http://box:7447 --token-file token.txt
shippy build   # dependency build scripts are restored from the box when it has them

What a run's key covers

A run is one execution of a build script. Like a compilation, it has a recipe and a key. The key covers what cargo's own rules say a run depends on:

The first six rows are known before the script runs. They form the pre-key, so a package is looked up without running anything. The last row is learned from a run and picks among runs of one pre-key.

PATH and HOME are each machine's own. A script that names one depends on its machine, and its run is not shared. NUM_JOBS and the jobserver are in no key.

The result is OUT_DIR packed as one tree, plus what the script printed, which is replayed to cargo.

Which scripts are cached

SHIPPY_SCRIPTS=deps shippy build

In shippy.toml, build_scripts = true under [cache] is 1 and false is 0.

Remote execution changes who runs a script nobody has run yet, not whose runs are cached:

Shared and local runs

A run is shared (other machines may get it) or local (kept in this machine's store only). The two never have one key.

A shared run becomes local for this machine when the recipe cannot cover what it reads, for example:

• the script asks about its git repository (.git/, rev-parse);
• the package holds a link that leads out of it;
• the package is behind a link, such as ~/.cargo/registry on another disk;
• the package and its exports exceed SHIPPY_SCRIPT_MAX_BYTES (1 GiB) or SHIPPY_SCRIPT_MAX_FILES (50,000);
• the run names PATH or HOME, or a rerun-if-changed path the inputs do not hold.

To see what happened to each script, look at <store>/sessions/<session>.scripts (one line per script: package, shared or local, restored, run-on-the-box or ran-here, and why), or set SHIPPY_DEBUG=1.

How a shared run becomes public

Only the box's own run is ever public. There are two ways to get there.

• A claim. The script ran on a client and the run is uploaded with the files the box lacks. The box runs the recipe in its sandbox and compares. When they agree, the box's run is published, never the claim's bytes. When they differ, or the script fails in the sandbox, nothing is published and the reason is recorded.
• A request. With remote execution on, a missed shared run goes to POST /v1/exec. The box runs the script, publishes the run, and sends back the packed OUT_DIR, which is restored like a hit. If anything goes wrong, the script runs here.

Either way a run is published under the variables its script named with rerun-if-env-changed and the workspace files it named with rerun-if-changed, and nothing else its sender declared.

The claim path is the safer of the two: the box compares two runs before publishing. A request has no run to compare with.

What a published run may tell cargo

A restored run's directives act on the machine that restores it, so a run is published only when every directive means the same on any machine:

A script that links a system library (rustc-link-lib=ssl after a pkg-config probe) is the machine's own. Such scripts are usually quick and run on each machine, in every new target directory: the same rule decides whether a machine restores a run it made itself. One difference: a machine's own run may name a file outside the build with rerun-if-changed, as bindgen names the system headers it read. The run's key holds that file's content, so the run is restored while the header is the same and the script runs again once it has changed.

What is never shared

• A workspace package's script, unless named in share_scripts.
• The script of a git dependency that is one package of a larger repository, unless a trusted principal ran it. It can read the repository around its package, which no key covers and the sandbox does not hold.
• A script that asks about its git repository (vergen, built, shadow-rs, git describe). The sandbox has no repository.
• A script whose run names its machine: a system library, an absolute path, PATH.
• A script whose claimed run differs from the sandbox's (it stamps the time, or reads a variable it does not name).
• A script that needs the network or a tool the box lacks.

Settings

On the machine that builds:

On the box (in the [env] table of shippyd.toml):

Go deeper

• docs/build-scripts.md
• docs/operations.md: Build scripts, for tuning SHIPPYD_SCRIPT_JOBS on a busy box
• docs/remote-exec.md