Skip to content

CI cache measurement rollout

Issue #206 requires evidence before adding an adopter, dependency, or optional-matrix cache. This first rollout changes no cache configuration. It creates a cache-telemetry-* Actions artifact for each job that installs packages: lint, every test interpreter, deps, optional-matrix, and adopter validation. Architecture is excluded because it installs nothing.

Each artifact contains a JSON record and its Markdown rendering. It names the runner, records cache restore and installation durations, records the actions/cache outcome, and records both non-loopback bytes received during the installation interval and local download-cache growth. Network RX can include registry metadata, while cache growth does not count a downloaded file that the package manager declines to retain; neither field is presented as the other.

actions/cache reports true for an exact key, false for a prefix restore, and an empty value for a miss. The report labels these exact-hit, prefix-hit, and miss instead of treating a valid restore-keys fallback as a cold cache. Jobs with no persisted cache are labelled not-configured; they provide the baseline for the later deps, optional-matrix, and adopter changes.

After collecting completed runs on each active runner, compare the same job and lockfile/input hash by outcome. State any saving as the observed difference between miss and restored (exact-hit or prefix-hit) installation medians, alongside the sample counts and cache-restore median. Decide whether shared storage is worth operating only from the observed per-runner miss rate and restore cost. Do not infer it from the number of runners.

The follow-up may cache only the package-manager download directories:

  • set UV_CACHE_DIR and cache it for adopter validation;
  • cache ~/.cache/pip for deps;
  • cache the existing optional-matrix --cache-dir across runs.

It must never cache .ci-venv, any generated environment, or installed site-packages. Every validation environment remains newly created.