Skip to content

fix: keep params named size or nfiles in the stage hash - #11079

Merged
skshetry merged 4 commits into
treeverse:mainfrom
adarshsm:fix/10296-params-named-like-meta
Nov 6, 2026
Merged

skshetry merged 4 commits into
treeverse:mainfrom
adarshsm:fix/10296-params-named-like-meta

Conversation

@adarshsm

@adarshsm adarshsm commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Fixes #10296.

Problem

A parameter named size or nfiles is invisible to the run-cache, so changing it does not
re-run the stage and the outputs from the previous run are checked out instead. Reproduced
on current main:

$ cat params.yaml
size: 30
count: 30

$ dvc stage add -n gen -p size,count -d gen.py -o out.txt "python gen.py"
$ dvc repro
$ cat out.txt
size=30 count=30

# changing an ordinary param re-runs the stage
$ sed -i '' 's/count: 30/count: 40/' params.yaml && dvc repro
Running stage 'gen': ...

# changing 'size' does not
$ sed -i '' 's/size: 30/size: 40/' params.yaml && dvc repro
Stage 'gen' is cached - skipping run, checking out outputs

$ cat out.txt
size=30 count=40     # stale: the new value never took effect

The failure is silent, which is what makes it nasty — the stage reports as cached and the
outputs look valid.

Cause

_get_cache_hash() excludes size and nfiles because they are file metadata rather than
part of the stage definition, but it passes the exclusion to dict_sha256() for the whole
lockfile, and dict_filter() is recursive:

return dict_sha256(cache, exclude=[Meta.PARAM_SIZE, Meta.PARAM_NFILES])

The lockfile has four top-level sections — cmd, deps, params, outs — and metadata
only ever appears under deps and outs. params holds arbitrary user keys, so a
parameter literally named size or nfiles was stripped along with the metadata and never
reached the hash.

This matches @skshetry's reading on the issue: "We are recursively excluding nfiles and
size before hashing for stage cache, which is incorrect. Most likely, we'll be able to
remove size and nfiles only from outputs that are not parameter dependencies."

Fix

Apply the exclusion to the deps and outs sections only.

Existing run-cache entries stay valid. For a stage without a parameter of either name,
the filtered dictionary is identical to what the old code produced, so the hash is
unchanged. Only stages that were mis-hashed get a new hash, which is the point.

Verified directly:

case result
ordinary stage, old hash vs new hash identical → run-cache preserved
size: 30 vs size: 40, old code identical → the bug
size: 30 vs size: 40, new code different → fixed
size under deps/outs changed identical → metadata still excluded

And end to end, on the reproduction above: size changes now re-run the stage and
out.txt tracks the parameter, while an unchanged pipeline still reports
Data and pipelines are up to date.

Tests

Two unit tests in tests/unit/stage/test_cache.py:

  • test_cache_hash_tracks_params_named_like_meta — the regression. Fails on main
    (the two hashes are equal), passes here.
  • test_cache_hash_still_excludes_deps_and_outs_meta — guards the behavior that must not
    change; passes both before and after.
pytest tests/unit/stage/                                                # 84 passed
pytest tests/func/test_run_cache.py tests/func/test_stage.py \
       tests/func/params/ tests/func/repro/                             # 150 passed

tests/unit/stage/ also reports two failures in my environment
(test_fill_from_lock_use_appropriate_checksum, test_dump_nondefault_hash), both
ImportError: s3 is supported, but requires 'dvc-s3' to be installed. They are unrelated:
the failing test ids are identical with and without this change.

ruff check, ruff format --check and mypy dvc/stage/cache.py are clean.

Note on scope

@TimCosemans reported the same symptom on the issue for parameters named time_dummies and
method. Those names are not excluded anywhere in the hashing path, so I believe that is a
different problem and have deliberately left it alone rather than widen this change; happy
to look at it separately if it is still reproducible.

size and nfiles are file metadata and are excluded from the run-cache
hash, but the exclusion was applied to the whole lockfile. A parameter
with either name was therefore dropped before hashing, so changing it
left the hash untouched: the stage was reported as cached, the outputs
were checked out from the previous run, and the new parameter value
silently never took effect.

Those fields only ever appear under deps and outs, so restrict the
exclusion to those two sections. A stage without such a parameter hashes
exactly as before, leaving existing run-cache entries valid.

Fixes treeverse#10296
@github-project-automation github-project-automation Bot moved this to Backlog in DVC Aug 7, 2026
@codecov

codecov Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.98%. Comparing base (2431ec6) to head (21facb2).
⚠️ Report is 220 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11079      +/-   ##
==========================================
+ Coverage   90.68%   90.98%   +0.30%     
==========================================
  Files         504      505       +1     
  Lines       39795    41177    +1382     
  Branches     3141     3263     +122     
==========================================
+ Hits        36087    37466    +1379     
- Misses       3042     3072      +30     
+ Partials      666      639      -27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

* origin/main:
  fix: fall back to full index when pull/checkout targets mix .dvc files and paths (treeverse#11076)
  [pre-commit.ci] pre-commit autoupdate (treeverse#11066)
  build(deps-dev): bump mypy from 1.19.1 to 2.4.1 (treeverse#11131)
  build(deps): bump rtCamp/action-slack-notify from 2.4.0 to 2.4.1 (treeverse#11130)
  build(deps): bump github/codeql-action from 4.37.4 to 4.39.1 (treeverse#11125)
  add support for 3.15 (treeverse#11135)
  completion: pass shtab the preamble for the selected shell (treeverse#11129)
Filter size/nfiles under deps and outs inline, without the StageParams
import and the separate meta_only tuple. Hashes are unchanged: the
existing tests that pin exact run-cache paths still pass.

Add a test that reproduces the issue end to end: changing a param named
size and running `dvc repro` again restored the stale output from the
run-cache.

Drop test_cache_hash_still_excludes_deps_and_outs_meta. The pinned-path
tests already fail if the exclusion is removed under deps, outs, or
both.

Refs: treeverse#10296
Copy params.yaml to the output instead of running a Python script, so
the test needs no extra file or dependency. params.yaml stays out of
deps: as a file dependency its md5 would change the run-cache key on
its own, and the test would pass without the fix.

@skshetry skshetry left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing! LGTM.

@skshetry
skshetry merged commit b59add9 into treeverse:main Nov 6, 2026
52 of 54 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in DVC Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipeline is not executed for parameter with name size or nfiles

2 participants