What is cached
Short answer: libraries and proc macros always, executables by default, build scripts when you have a box. Here is the whole picture.
| What | Cached? | Switch |
|---|---|---|
Library crates (rlib) | Always | - |
| Proc-macro crates | Always | - |
Executables (bin) and C-compatible libraries (cdylib) | Yes, by default | SHIPPY_LINKED=0 or linked_outputs = false turns it off |
| Build-script runs of downloaded packages | Always | SHIPPY_SCRIPTS=0 turns it off |
| Build-script runs of your own packages | When asked | SHIPPY_SCRIPTS=1 or build_scripts = true |
| Tests and Rust dynamic libraries | No | Linked by rustc as before |
| Rebuilds from incremental state | No: left where the compiler wrote them | SHIPPY_KEEP_INCREMENTAL=1 stores them locally, SHIPPY_CLAIM_INCREMENTAL=1 uploads them too |
Executables
An executable's key lists the .rlib of every crate it links, the native libraries on its search paths and the host's linker. A library the linker finds elsewhere on the machine, through LIBRARY_PATH, in the system's directories or in Apple's SDK, is in the key by its content, together with the places the linker searched before it: when such a library changes, or another would be found first, the executable is linked again. The same goes for every file a linker script names in place of a library (glibc's libm.a is one, so a static executable links through it) and for a file a linker option reads, such as a version script. An executable linked with such a file stays on the machine that linked it. Caching executables matters more than you might think:
The catch is size: an executable is the largest file of a build (reth's dev binary is 448 MB), and every edit makes a new one. That is what the store's size limit is for: old edits' executables are the first to go. Caching the executable doesn't speed up the first build after an edit, but it makes undoing the edit a restore.
Build scripts
Even with every compilation restored, a cold build is still minutes of build scripts: C and C++ compiled by cc, cmake and make for crates like rocksdb, jemalloc and aws-lc. So shippy caches a build script's run, and the box runs scripts in its sandbox so one person's run can be everyone's.
Cargo runs build scripts itself, so the wrapper never sees the run. It sees the script being linked, and leaves a small launcher in its place that calls back into shippy. A run's key covers the script's executable, every file of its package, what its dependencies' scripts export, the variables cargo and cc give it, the C toolchain, and whatever it declared with rerun-if-env-changed and rerun-if-changed.
A build script can read anything on the machine, so sharing runs has rules of its own. Build scripts explains them.
Incremental rebuilds stay local
-C incremental stays on for workspace crates, because it's what makes a local rebuild fast. But a compilation that started from incremental state (a rebuild after an edit) is not uploaded:
- What rustc writes from earlier incremental state is never byte-identical to what it writes from scratch, which is how the box compiles.
- Uploading every edit would cost megabytes per edit and a compiler on the box for each, for a comparison that tells nothing.
A crate that was restored has no incremental state at all, so the first edit after a restore would compile it and everything above it from nothing. After a build that restored workspace crates, shippy compiles them once more in the background for their state alone: at idle priority, into a scratch directory that is removed, stopping as soon as another build starts in that target directory. The first rebuild after an edit is then as fast as the ones after it. It compiles only what your own build's compiler wrappers wrote down and your local store vouches for, never what a file in a target directory says, and without the token. SHIPPY_PRIME=0 (prime = false) turns it off, SHIPPY_PRIME=wait makes the build wait for it, and it is off where CI is set.
It is not stored on your machine either. The rebuild after an edit is the build you wait for all day, and storing its result would mean hashing and copying everything the compiler just wrote while cargo waits (on tempo 2.6 GB hashed and an 800 MB executable copied for one edit, and 1.7 GB more in the store each time). So shippy leaves those files where the compiler put them, and compiles the crates above them without a key or a lookup: nobody else has what they are compiled against. Undoing an edit, or switching back to a commit that was built from scratch or fetched, is still restored from the store. SHIPPY_KEEP_INCREMENTAL=1 (keep_incremental = true) stores incremental rebuilds too, if you go back and forth between two edited states and want the second visit restored. Measure before you rely on it: with split-debuginfo = "unpacked" a workspace crate's result grows by a debug-info file per codegen unit with every rebuild, and on tempo restoring an edited state took longer (23 to 45 s) than compiling it again from incremental state (10 to 15 s).
-C incremental is part of the key, so a cache filled for developers must be filled with incremental on.
Cranelift
rustc_codegen_cranelift generates unoptimized code faster than LLVM, but is nightly only, has no unwinding (catch_unwind and #[should_panic] abort), and aborts on some std::arch intrinsics. It is a setting for build-and-run loops on workspace crates, never for tests or releases. The backend is part of every key. See tune a project.