Add artifact merge engines and Kind metadata flow (RFC 018 phase 1)#9805
Draft
Evangelink wants to merge 1 commit into
Draft
Add artifact merge engines and Kind metadata flow (RFC 018 phase 1)#9805Evangelink wants to merge 1 commit into
Evangelink wants to merge 1 commit into
Conversation
Implements the first testfx-side pieces of RFC 018 (artifact post-processing for dotnet test / MTP): - Report merge engines (pure, invocation-agnostic, no I/O in the core): - TrxReportEngine.Merge / MergeToFileAsync (XML-level TRX merge). - JUnitReportMerger (unions <testsuite>, counters derived per-suite). - CtrfReportMerger (JSON merge; summary derived from merged tests[]). - Kind metadata flow so the dotnet/sdk orchestrator can group artifacts by format: SessionFileArtifact gains an optional kind (experimental), the FileArtifactMessage IPC record + serializer carry Kind (field id 7), the dotnet-test consumer propagates it, and report producers tag their kind (trx/junit/ctrf/html). Unit tests cover all three merge engines and the IPC round-trip. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds report-merging foundations and propagates artifact format metadata through MTP’s artifact and IPC layers.
Changes:
- Adds TRX, JUnit, and CTRF merge engines with unit tests.
- Adds experimental
SessionFileArtifact.Kindmetadata and IPC serialization. - Tags TRX, JUnit, CTRF, and HTML report artifacts by format.
Show a summary per file
| File | Description |
|---|---|
test/UnitTests/Microsoft.Testing.Platform.UnitTests/IPC/ProtocolTests.cs |
Covers IPC kind serialization. |
test/UnitTests/Microsoft.Testing.Platform.UnitTests/IPC/ProtocolEdgeCaseTests.cs |
Tests nullable kind round trips. |
test/UnitTests/Microsoft.Testing.Platform.DotnetTestProtocolContract.UnitTests/DotnetTestProtocolSerializerTests.cs |
Verifies protocol-contract serialization. |
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/TrxReportEngineMergeTests.cs |
Tests TRX merging. |
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/JUnitReportMergerTests.cs |
Tests JUnit merging. |
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/CtrfReportMergerTests.cs |
Tests CTRF merging. |
src/Platform/SharedExtensionHelpers/ReportGeneratorBase.cs |
Publishes report artifact kinds. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Serializers/FileArtifactMessagesSerializer.cs |
Serializes the new kind field. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/ObjectFieldIds.cs |
Assigns IPC field ID 7. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Models/FileArtifactMessages.cs |
Adds kind to the wire model. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/DotnetTestDataConsumer.cs |
Propagates session artifact kinds. |
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt |
Tracks the experimental public API. |
src/Platform/Microsoft.Testing.Platform/Messages/FileArtifacts.cs |
Adds the kind-aware constructor and property. |
src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt |
Tracks IPC internal API changes. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxReportEngine.Merge.cs |
Implements TRX merging. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxDataConsumer.cs |
Tags in-process TRX artifacts. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks TRX internal APIs. |
src/Platform/Microsoft.Testing.Extensions.Retry/InternalAPI/InternalAPI.Unshipped.txt |
Tracks linked IPC field API. |
src/Platform/Microsoft.Testing.Extensions.MSBuild/InternalAPI.Unshipped.txt |
Tracks linked IPC field API. |
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportMerger.cs |
Implements JUnit merging. |
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs |
Tags JUnit artifacts. |
src/Platform/Microsoft.Testing.Extensions.JUnitReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks JUnit internal APIs. |
src/Platform/Microsoft.Testing.Extensions.HtmlReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks shared generator API. |
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportGenerator.cs |
Tags HTML artifacts. |
src/Platform/Microsoft.Testing.Extensions.HangDump/InternalAPI/InternalAPI.Unshipped.txt |
Tracks linked IPC field API. |
src/Platform/Microsoft.Testing.Extensions.CtrfReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks CTRF internal APIs. |
src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportMerger.cs |
Implements CTRF merging. |
src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportGenerator.cs |
Tags CTRF artifacts. |
Review details
- Files reviewed: 28/28 changed files
- Comments generated: 4
- Review effort level: Medium
| if (!TrxModeHelpers.ShouldUseOutOfProcessTrxGeneration(_commandLineOptionsService)) | ||
| { | ||
| await _messageBus.PublishAsync(this, new SessionFileArtifact(testSessionContext.SessionUid, new FileInfo(reportFileName), ExtensionResources.TrxReportArtifactDisplayName, ExtensionResources.TrxReportArtifactDescription)).ConfigureAwait(false); | ||
| await _messageBus.PublishAsync(this, new SessionFileArtifact(testSessionContext.SessionUid, new FileInfo(reportFileName), ExtensionResources.TrxReportArtifactDisplayName, ExtensionResources.TrxReportArtifactDescription, "microsoft.testing.trx")).ConfigureAwait(false); |
| /// <item><description><c>TestLists</c> are deduplicated by <c>id</c> (the well-known lists are shared across files).</description></item> | ||
| /// <item><description><c>Counters</c> attributes are summed; <c>Times</c> use the earliest start and latest finish.</description></item> | ||
| /// <item><description>The result summary outcome is <c>Failed</c> if any input failed, otherwise <c>Completed</c>.</description></item> | ||
| /// <item><description>Attachment/result-file paths are preserved verbatim (they are absolute today).</description></item> |
| mergedTestRun.Add(mergedTestDefinitions); | ||
| mergedTestRun.Add(mergedTestEntries); | ||
| mergedTestRun.Add(mergedTestLists); | ||
| mergedTestRun.Add(BuildResultSummary(anyFailure ? "Failed" : "Completed", counterAttributeOrder, counterSums)); |
Comment on lines
+69
to
+71
| CloneChildrenInto(FindChild(testRun, "Results"), mergedResults); | ||
| CloneChildrenInto(FindChild(testRun, "TestDefinitions"), mergedTestDefinitions); | ||
| CloneChildrenInto(FindChild(testRun, "TestEntries"), mergedTestEntries); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the first testfx-side pieces of RFC 018 — Artifact post-processing for
dotnet test(MTP). Two independent, self-contained foundations that every later phase (dispatcher tool, SDK election) reuses. No SDK dependency; the SDK orchestration lands separately once this merges.1. Report merge engines (pure, invocation-agnostic — no I/O in the core)
Validated the RFC's "collect same-kind inputs → re-aggregate → emit one" shape across both data models:
TrxReportEngine.Merge/MergeToFileAsyncResults/TestDefinitions/TestEntriesTimesearliest/latest,TestListsdeduped by id, outcomeFailedif any input failedJUnitReportMerger<testsuite><testsuite>rootsCtrfReportMergerJsonNode)results.tests[]summaryderived from the mergedtests[]sosummary.testsalways matches the array;start/stopmin/maxHTML is intentionally not file-merged (it's a rendered template with an embedded JSON blob) — this validates the design's opt-in / return-null semantics rather than breaking it.
2.
Kindmetadata flow (producer → IPC wire)So the
dotnet testorchestrator can group artifacts by format without inspecting file contents:SessionFileArtifactgains an optionalkind(experimental-gated,[TPEXP]); the shipped 4-arg constructor is preserved for binary compatibility.FileArtifactMessage.Kind+FileArtifactMessageFieldsId.Kind = 7+ serializer read/write/field-count (forward/backward compatible — old readers skip the unknown field).DotnetTestDataConsumerpropagatesKindonto the wire message.microsoft.testing.trx, JUnitmicrosoft.testing.junit, CTRFmicrosoft.testing.ctrf, HTMLmicrosoft.testing.html.Tests
Kindsurvives the pipe.WasiPlaygroundsample fails on a missing workload); Platform IPC/protocol, contract-package, and Extensions unit tests all green.Not in this PR (future phases, per the RFC)
IArtifactPostProcessortyped contract wrapping the engines.ITool+ tool-host-over-pipe composition.merge-trx, …) and the dotnet/sdk orchestration.cc @Evangelink
Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com