Repository navigation
CI: build the variants from common layers, cache the layers of v5 - #428
Merged
Merged
Conversation
mistraloz
force-pushed
the
perf/ci-layers-cache
branch
from
November 5, 2026 21:12
386329f to
a6e5052
Compare
…does not use them as cache
Apache is started without apache2ctl, which creates it: mod_cgid failed to start in the slim-apache image and with the built-in Apache of the fpm variant. The fat apache image only had it because an extension package started apache2ctl during its installation.
The slim Dockerfiles start from Dockerfile.base (the layers common to all the images). The extensions are installed once on Dockerfile.base (Dockerfile.extensions), then the slim Dockerfile of each variant is applied on them to build the fat images: the extensions were installed 3 times per PHP version and architecture. The images keep the same content (configuration, files, packages and PHP extensions of each SAPI compared on PHP 8.4). The CI builds and tests a PHP version and an architecture per job (8 jobs instead of 24) and runs the test files in parallel (tests-suite/run_tests.sh). The NodeJS images are now tested: variant-node.sh was run without NODE.
A job publishes the 15 images of a PHP version and an architecture: pushing them one by one took about 3 minutes.
… week The runs of v5 read and write the layers in a registry (ghcr.io/<owner>/php-cache): a push or a manual run only rebuilds what changed. A separate workflow deletes the versions of the cache each week, 30 minutes before the scheduled run of v5 (or when run manually), so that the images take the updates of the packages each week. The pull requests only read the cache of v5, which no pull request can write. The GitHub Actions cache is not used: limited to 10 GB, and slow to write by 8 jobs at the same time. BuildKit 0.34 does not find in the cache the layers of a target built from another target (contexts = target:...) with a new builder, as on GitHub: the builder uses BuildKit 0.35.
The test waited 3 seconds: on a loaded runner, the container was still starting, and the entrypoint only handles the stop signal at the end of its current step.
mistraloz
force-pushed
the
perf/ci-layers-cache
branch
from
November 6, 2026 08:39
a6e5052 to
bdff28f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Note
The builder uses the release candidate
moby/buildkit:v0.35.0-rc1(see below): bump it inworkflow.ymlwhenBuildKit 0.35.0 is released.
This PR divides the runner minutes of the CI by 2 to 3.5, and makes every run faster (see the measures below):
instead of once per variant. The extensions are installed once, then the slim part of the variant is applied on them for
the fat image (see Contributing). The fat images no longer contain the layers of the slim
image of their variant, their content is unchanged.
tests-suite/run_tests.sh), the NodeJS images are now tested.ghcr.io/thecodingmachine/php-cache:v5read and write it, so a push or a manual run only rebuilds what changed;purge-cache.yml) deletes the cache each Sunday, 30 minutes before the weekly run, so thatthe images take the updates of the packages each week. It can also be run manually;
BuildKit 0.34 does not find in the cache the layers of a target built from another target, with a new builder as on
GitHub. BuildKit 0.35.0-rc1 fixes it.
It also fixes
mod_cgid(APACHE_EXTENSION_CGID=1), which did not start in the slim-apache image and with thebuilt-in Apache of the fpm variant: its socket directory was missing.
Duration of the whole workflow, from its start to its end, and runner minutes:
v5(from an empty cache)v5"Before" is measured on the last runs of this repository (weekly runs of October and November, CI of #429): their
24 build jobs waited up to 7 min in the queue. The new workflow runs 8 jobs at a time, they start within 5 s. The
cache does not exist in this repository yet, so the runs marked "fork" were measured on a fork (with the same queue
time). The purge of the cache (about 1 min) runs 30 min before the weekly run, it is not counted.
Build of PHP 8.4 measured locally with a new builder: 753 s of CPU without cache, 297 s with the cache.
Test plan (required)
directory of
mod_cgiddiffers.test_cgid(variant-apache.sh), it fails on the current images.reuse it. Locally, a new builder finds the 195 build steps of PHP 8.4 in the cache.
test_dockerStopwaited 3 seconds before stopping the container: it now waits until its processes are started (itfailed once on a loaded runner).
Checklist