Skip to content

[dv] Fix directed-test configuration and scheduling - #2497

Open
kulan-pal wants to merge 3 commits into
lowRISC:masterfrom
kulan-pal:dv/directed-test-sim-opts
Open

kulan-pal wants to merge 3 commits into
lowRISC:masterfrom
kulan-pal:dv/directed-test-sim-opts

Conversation

@kulan-pal

Copy link
Copy Markdown

Three small fixes to the directed-test flow in dv/uvm/core_ibex. None changes which
tests run on the baseline; they are prerequisites for directed tests that need a
plusarg of their own (Zcmp and dummy-instruction tests in follow-up PRs, see #2461).

sim_opts is silently dropped from directed-test entries

scripts/run_rtl.py reads sim_opts from the test options for both riscv-dv and
directed tests, but the pydantic schema for directed_testlist.yaml
(scripts/directed_test_schema.py) has no such field, so pydantic's default
extra = "ignore" discarded it. A directed test could not pass a plusarg through its
own entry. The field is added to DConfig as Optional[str] = None: existing
entries stay valid, and a config default can still be overridden per test.

A test with N rtl_params entries is scheduled N times

In scripts/ibex_cmd.py::filter_tests_by_config the accepting append sat inside
the loop over rtl_params, so a test listing N parameters was appended N times:
make warned about duplicate targets and regr.log counted every seed N times.
Every existing entry lists one parameter, which hid it. The append moves to the
loop's else: clause, which runs once and only when no parameter caused a break.

directed_testlist.yaml regenerates differently per locale

directed_tests/gen_testlist.py listed the vendored test directories with ls,
whose order follows the locale's collation, so the committed YAML could not be
checked against its generator. The listings now come from sorted(os.listdir()),
which is code-point order on every machine. The regenerated YAML is included.

Verification

Baseline 34b07057.

  • Regeneration is byte-identical after this change. The 944 entries and their
    contents are unchanged; only the order of a few vendored tests differs. With the
    unmodified generator, running under the C locale produces a different file from the
    committed one.
  • Python only: filter_tests_by_config on the opentitan config returns a test with
    two rtl_params entries twice before this change and once after; one-parameter,
    no-parameter and rejected tests behave the same before and after.
  • End to end (VCS U-2023.03-SP2, Spike ibex_cosim, IBEX_CONFIG=opentitan): a
    directed test with two rtl_params entries and sim_opts: +disable_cosim=1 is
    scheduled once and the plusarg appears on the simulator command line. Existing
    directed tests and riscv_arithmetic_basic_test pass unchanged.
  • util/lint_commits.py --no-merges clean.

Disclosure

Per CLA section 9: the analysis and code in this PR were produced with the
assistance of an AI coding tool (Claude Code) and independently reviewed with a
second tool (OpenAI Codex). I have reviewed and understood every change and take
full responsibility for it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Rn6C7Eyq1LchFeP9w6up9P

run_rtl.py reads 'sim_opts' out of the test options for both riscv-dv
and directed tests, but the pydantic schema that validates a directed
test's configuration has no such field, so pydantic's default
extra="ignore" dropped it. A directed test therefore could not pass a
plusarg through its own testlist entry.

Add the field to DConfig, optional so every existing entry stays valid,
and on DConfig rather than DTest so a config can supply a default that
an individual test still overrides after the merge.

Signed-off-by: Kulan Palanichamy <kulan.palanichamy@opentitan.org>
The append that accepts a test sat inside the loop over its rtl_params,
so a test listing N parameters was scheduled N times: make warned that
its targets were given more than once and regr.log counted every seed N
times. Every existing entry lists a single parameter, which hid it.
Append in the loop's else clause instead, once all parameters matched.

Signed-off-by: Kulan Palanichamy <kulan.palanichamy@opentitan.org>
gen_testlist.py enumerated the vendored test directories with ls, whose
order follows the collation of the current locale: under C/POSIX
sh-misaligned sorts before shamt, under en_US after it. Regenerating
directed_testlist.yaml therefore produced a different file depending on
the environment, and the committed file could not be checked against
its generator.

List the directories with os.listdir() and sort the names in code point
order, which does not depend on the locale. Regenerate
directed_testlist.yaml: no entry changes, only the order of the
vendored tests.

Signed-off-by: Kulan Palanichamy <kulan.palanichamy@opentitan.org>
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@kulan-pal

Copy link
Copy Markdown
Author

I have read the CLA Document. By submitting this pull request comment, I am hereby confirming my acceptance of the terms of the CLA Document and my agreement to be legally bound by its terms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant