[JulieLeeMSFT]

Fixes #135557

JIT/opt/GuardedDevirtualization/badinlinee fails in the rolling runtime-coreclr jitstress pipeline (instrument_if_optimizing, jitstress1, jitstress1_tiered, jitstress_random_*, jitstress_isas_1_*) with Test allocated 24000 bytes. I reproduced it locally (windows-x64 Checked) and found three independent root causes. This PR fixes each of them instead of disabling the test.

1. JIT: optimized instrumented code never devirtualized the calls it probed (instrument_if_optimizing)

impImportCall called impConsiderCallProbe before trying to devirtualize and skipped impDevirtualizeCall for every call it marked for a class/method probe (there was a TODO: "if we can devirtualize we should be able to suppress the probe"). The instrumentor then spills this into a temp without class information, so late devirtualization fails (no type available (op=LCL_VAR)), the inliner reports target not direct, and in new Base().M() the allocation stays because the object is also passed to the CORINFO_HELP_CLASSPROFILE32 helper. This hits every optimized+instrumented compile: OSR with PGO, hot R2R methods promoted to the instrumented tier 1, TieredPGO_InstrumentedTierAlwaysOptimized, JitInstrumentIfOptimizing.

The fix tries to devirtualize first and only probes the calls that are still virtual afterwards. A call that is going to be probed still cannot become a guarded devirtualization candidate (the probe info and the candidate info share storage in GenTreeCall), so considerGuardedDevirtualization now bails out for such calls, which keeps the previous behavior there. impConsiderCallProbe no longer returns a value because nothing uses it anymore. Non-instrumented compiles are not affected.

Test() in the instrumented tier 1 goes from a virtual call plus allocation (61 bytes) to mov eax, 42; ret.

2. badinlinee cannot hold under JitStress

JitStress picks methods by hash: at DOTNET_JitStress=1 it selects Test for STRESS_MIN_OPTS (and for the forced NO_WAY fallback of STRESS_GENERIC_VARN, which also retries the compile with MinOpts), so the final tier is MinOpts, nothing is inlined and the allocation stays. Disabling only STRESS_MIN_OPTS is not enough, so the test now uses [SkipOnCoreClr(..., RuntimeTestModes.JitStress)] like other tests that check optimization results under JitStress (it needs a TestLibrary reference for that).

3. badinlinee depended on wall-clock timing

The old warm-up was a fixed 100 x (100 calls + Sleep(15)) per method. With TC_CallCountThreshold=1 the tier-up queue holds hundreds of CoreLib methods, so Poison (which needs two promotions) usually never reached tier 1 before the second phase started, i.e. the GDV scenario of #133993 was not exercised at all: re-introducing that bug (reporting the GDV inline failure against call->gtCallMethHnd) still passed the old test 3/3. It also failed with DOTNET_ForceRelocs=1 (the JIT does twice the work) because Test was still in tier 0. The test now keeps calling the method until JitInfo.GetCompiledMethodCount() has been unchanged for 20 iterations, first for Poison, then for Test, and asserts with Assert.Equal instead of printing and returning -1. With the re-introduced #133993 bug the new test fails 3/3 in the normal scenario.

New regression test

src/tests/JIT/Regression/JitBlue/Runtime_135557 is a minimal repro of (1) that does not depend on the GDV scenario: new Base().M() with DOTNET_JitInstrumentIfOptimizing=1 (Checked/Debug JIT only, ignored by a Release JIT) must not allocate once Test reaches its final tier. 10/10 runs fail with the old JIT (Expected: 0 Actual: 24000) and 10/10 pass with the fix.

Testing

• badinlinee and Runtime_135557 on windows-x64 Checked. I ran badinlinee in 49 scenarios of testenvironment.proj with the old and the new JIT. Old JIT with the old test: fails forcerelocs, jitstress1, jitstress1_tiered, jitstress_isas_1_* and instrument_if_optimizing. Old JIT with the new test: only instrument_if_optimizing fails. New JIT with the new test: everything passes. jitpartialcompilation_pgo_stress_random is excluded from these statements: it crashes (exit code 0x80000003) identically with the old JIT and the old test and is unrelated (other tests pass in that scenario).
• All of src/tests/JIT (priority 1, 22,288 tests, including the two changed ones) with the new JIT in the default configuration and in the instrument_if_optimizing and fullpgo_methodprofiling_always_optimized scenarios, which instrument every optimized compile and so exercise the new path heavily: 22,125 passed, 0 failed, 163 skipped, 2 active issues in each of them.
• SuperPMI (windows x64, Checked JIT): replay is clean on the standard collections. Replaying coreclr_tests.run reports missing data for 8179 of 620965 contexts, all ResolveVirtualMethod, i.e. the new JIT-EE query in optimized+instrumented compiles (the old JIT replays this collection clean). Asm diffs against the base JIT on benchmarks.run, benchmarks.run_pgo, benchmarks.run_pgo_optrepeat, coreclr_tests.run, libraries.crossgen2, libraries.pmi and realworld.run (1,466,464 contexts): no diffs, which confirms that non-instrumented compiles are unchanged. To see the actual effect I collected 15,616 contexts by running crossgen2 on CoreLib with TieredPGO and TieredPGO_InstrumentedTierAlwaysOptimized (5,153 of them Instrumented Tier1): clean replay and 45 diffs (8 size improvements, 37 regressions), +2,420 bytes (+0.04%) overall, all in instrumented tier 1 compiles that now devirtualize and inline calls with a known receiver.
• jitformat.py reports no formatting changes.

Note

This pull request description was generated with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[EgorBo]

https://helixr1107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-136100-merge-6e50f371f8b144cc96/System.Globalization.Nls.Tests/1/console.858e45f2.log?helixlogtype=result test failure might be related - fgInline throws an exception

[JulieLeeMSFT]

test failure might be related - fgInline throws an exception

The test failure looks like #136020.

[JulieLeeMSFT]

/ba-g known issues.