Shared-layer lifecycle: accounting, eviction, recovery, materialization

[chruffins]

summary

Capstone stage of the image-storage project — finishes migration, eviction, accounting, recovery, and observability on the shared-layer path.

• Materialization in builds: after a pull, each manifest layer is materialized into images/layers/<digest>/ (best effort; the composed rootfs still comes from blobs, so an artifact failure is logged and only degrades sharing).
• Reference-protected eviction: image deletion evicts only layer artifacts unreferenced by any remaining manifest model; a 10-minute grace period keeps fresh artifacts safe from racing in-flight builds. Startup also evicts orphans left by an unclean shutdown.
• Interrupted-build recovery: stale .unpack-* / .install-* / .tag-stage-* temp directories are swept at startup (age-gated so live builds are untouched).
• Layer-aware accounting: layer artifact bytes are bucketed with the OCI cache in TotalOCICacheBytes, so CurrentImageStorageBytes (ready rootfs + cache/layers) counts them exactly once and capacity admission sees the real footprint.
• Observability: hypeman_images_layer_artifacts_evicted_total counter plus slog eviction summaries and per-layer materialization warnings.
• Legacy flattened images remain readable and bootable; they carry no manifest model and no layer references, and are retired only through the existing delete/promotion paths.

validation

• End-to-end lifecycle test: two images sharing a base layer materialize the base once; deleting one image keeps the shared layer; deleting the last reference evicts everything.
• Accounting, grace-period, reference-protection, and stale-temp-dir tests.
• go test ./lib/images ./lib/paths ./lib/builds ./lib/scopes green; cmd/api/api green except Docker Hub pulls (anonymous rate limit — confirmed via direct probe returning TOOMANYREQUESTS) and VM lifecycle tests (need bridge privileges; verified failing identically on unmodified main).
• CI is unavailable during this work; all validation is local.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 33c8ca3. Configure here.}

[cursor]

Eviction can wipe in-flight layers

Medium Severity

Layer eviction decides freshness from the digest directory ModTime, and it runs concurrently with materialization. Unpack and mkfs.erofs write inside child temp dirs, so the parent mtime goes stale; DeleteImage can then RemoveAll that tree while a build is still using it, especially during the later unlocked ExportRootfs window before the manifest is written.

Additional Locations (2)

• lib/images/manager.go#L534-L569
• lib/images/manager.go#L865-L867

 

^{Reviewed by Cursor Bugbot for commit 33c8ca3. Configure here.}

[cursor]

Layer bytes skipped after failed builds

Medium Severity

Materialized layer artifacts are written to the layer store before conversion, but refreshDiskUsageTotals only runs on successful finalize, delete, or startup. After a failed conversion the artifacts remain on disk while TotalImageBytes keeps the stale cached total, so capacity admission undercounts the real footprint.

Additional Locations (2)

• lib/images/manager.go#L900-L907
• lib/images/disk_usage.go#L110-L153

 

^{Reviewed by Cursor Bugbot for commit 33c8ca3. Configure here.}