Conversation
There was a problem hiding this comment.
Pull request overview
Adds opt-in class-level parallel execution support to MSTest by allowing [Parallelize] on test classes, carrying those settings through discovery into execution, and scheduling only opted-in classes to run in parallel when no global parallelization is configured.
Changes:
- Allow
ParallelizeAttributeon classes (in addition to assemblies). - Propagate class-level parallelization settings via new
TestCaseproperties and update execution scheduling accordingly. - Add unit tests and an acceptance integration test validating class-level parallelization without assembly-level configuration.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/TestFramework/TestFramework/Attributes/Lifecycle/ParallelizeAttribute.cs |
Enables [Parallelize] usage on classes. |
src/Adapter/MSTestAdapter.PlatformServices/Discovery/TypeEnumerator.cs |
Reads class-level [Parallelize] and stores parallelization metadata on discovered tests. |
src/Adapter/MSTestAdapter.PlatformServices/ObjectModel/UnitTestElement.cs |
Carries class-level parallelization settings and serializes them into TestCase properties. |
src/Adapter/MSTestAdapter.PlatformServices/EngineConstants.cs |
Registers new hidden TestProperty entries for class-level parallelization metadata. |
src/Adapter/MSTestAdapter.PlatformServices/Execution/TestExecutionManager.cs |
Builds an execution plan to run only opted-in classes in parallel when no global parallelization is configured. |
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/ObjectModel/UnitTestElementTests.cs |
Verifies TestCase gets the new parallelization properties when present. |
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Discovery/TypeEnumeratorTests.cs |
Verifies discovery populates class-level parallelization settings from [Parallelize]. |
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestExecutionManagerTests.cs |
Verifies only class-level opted-in tests are executed in parallel when no global parallelization exists. |
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/ParallelizeClassLevelTests.cs |
Acceptance test project validating class-level parallelization behavior end-to-end. |
|
|
||
| // Instead of asking reflect helper to query the type for every method we have, we ask once for the type. | ||
| bool classDisablesParallelization = _reflectHelper.IsAttributeDefined<DoNotParallelizeAttribute>(_type); | ||
| ParallelizeAttribute? classParallelizeAttribute = _reflectHelper.GetSingleAttributeOrDefault<ParallelizeAttribute>(_type); |
There was a problem hiding this comment.
TypeEnumerator uses GetSingleAttributeOrDefault(_type), but reflection returns inherited attributes (inherit: true). If a base class and a derived class both specify [Parallelize], this will surface as multiple ParallelizeAttribute instances and GetSingleAttributeOrDefault will throw, potentially breaking discovery/execution. Consider retrieving only the directly-declared attribute (inherit: false), or enumerating all ParallelizeAttribute instances and selecting the closest/most-derived one (and optionally emitting a warning) rather than throwing.
| ParallelizeAttribute? classParallelizeAttribute = _reflectHelper.GetSingleAttributeOrDefault<ParallelizeAttribute>(_type); | |
| ParallelizeAttribute? classParallelizeAttribute = (ParallelizeAttribute?)global::System.Attribute.GetCustomAttribute(_type, typeof(ParallelizeAttribute), inherit: true); |
|
|
||
| #file UnitTest1.cs | ||
| namespace ClassLevelParallelizeAttributeProject; | ||
|
|
There was a problem hiding this comment.
The generated UnitTest1.cs asset code uses DateTimeOffset.UtcNow but doesn't include using System;. Since this asset targets net462 on Windows (TargetFrameworks.All), it will fail to compile without implicit usings. Add using System; to the generated source (or enable implicit usings in the generated csproj).
| using System; |
| /// Specification for parallelization level for a test run. | ||
| /// </summary> |
There was a problem hiding this comment.
The XML doc comment still describes ParallelizeAttribute only as a specification for parallelization level for a test run (assembly-level). Since the attribute is now also valid on classes, the documentation should be updated to mention class-level opt-in semantics and how Workers/Scope are interpreted when applied to a class.
| /// Specification for parallelization level for a test run. | |
| /// </summary> | |
| /// Specifies parallelization settings for a test run at the assembly or test-class level. | |
| /// </summary> | |
| /// <remarks> | |
| /// When applied at the assembly level, this attribute configures the default degree and scope of parallel execution | |
| /// for all tests in the assembly through the <see cref="Workers"/> and <see cref="Scope"/> properties. | |
| /// When applied to a test class, this attribute opts that class into parallel execution and can override the | |
| /// assembly-level settings for that class. In that case, <see cref="Workers"/> limits the maximum number of tests | |
| /// from that class that may run concurrently, and <see cref="Scope"/> is interpreted relative to that class | |
| /// (for example, <see cref="ExecutionScope.MethodLevel"/> enables parallel execution of test methods within the class). | |
| /// </remarks> |
Youssef1313
left a comment
There was a problem hiding this comment.
This adds a lot of extra complexity with not enough user ask (IMO) and creates room for conflicting configurations that's hard to reason about.
|
I agree overall but let's chat about it next week. Putting aside the refactoring on the execution manager, this is mainly just adding some extra properties to the test method object. |
|
Closing the PR, it was good to make investigations and see limitations and potential conficts. |
I need a little more reviews and manual tests
Fixes #5555