Subject: Possible GitHub Copilot SDK artifact-capture/session reliability issue during long-running certification work
Hi team,
I’d like to bring a possible GitHub Copilot SDK or agent-harness reliability problem onto your radar. We encountered the same failure twice while running a tightly controlled, long-running model certification workflow.
What we are doing
We use GitHub Copilot as the execution host for a code-review certification process. Each certification job:
Creates a fresh Copilot session.
Requests an exact model and reasoning-effort combination.
Gives the model a blind code-review assignment.
Requires the model to submit a structured result through a registered tool named pwi_submit_artifact.
Captures the tool payload, validates it, and persists a receipt containing the observed model identity and artifact hash.
The launcher uses one session at a time, unique artifact paths, explicit timeouts, and immutable attempt directories.
What happened
Two independent attempts stopped with the same failure:
copilot_worker_missing_artifact
In both cases:
A real Copilot session was created.
The requested model and reasoning effort were observed.
The session reached a terminal response.
No pwi_submit_artifact payload was captured by the application.
The same launcher successfully captured artifacts from the preceding sessions.
There were no overlapping session IDs or concurrent workers.
The attempts used different session IDs and different model routes.
Affected sessions:
Attempt 001
Session: 93a97bac-c176-4dab-8e37-031d993c74f1
Observed route: gpt-5.6-luna / low
Attempt 002
Session: 5ad8cf1f-8e07-4e58-9e67-e7ffc55281c3
Observed route: gpt-5.6-sol / low
Across the two attempts, 18 fresh sessions were consumed. The failure occurred on the ninth session of each attempt.
The second occurrence makes us concerned that this is not simply a one-off model response.
What we cannot currently determine
The SDK-facing application tells us that no structured artifact was captured, but the available application evidence does not clearly distinguish among these possibilities:
The model never invoked pwi_submit_artifact.
The model invoked the tool, but the tool event was not delivered or captured.
The model returned the payload in its final response rather than invoking the tool.
The tool invocation occurred, but parsing or SDK event handling failed.
The session terminated before the final tool event was persisted.
Our wrapper currently receives the returned response content, but when structured ArtifactCapture is empty, the operation raises a technical failure. The response/event material available to the wrapper is insufficiently durable to diagnose the boundary conclusively after the fact.
We are examining the local Copilot session-state and process logs for both session IDs. We will provide a redacted diagnostic inventory if it contains useful evidence.
Expected behavior
We would like the SDK or harness to provide durable and unambiguous terminal evidence that distinguishes:
model_did_not_submit_artifact
tool_invocation_observed_but_capture_failed
tool_payload_parse_failed
session_terminated_without_final_response
artifact_validation_failed
| Field |
Value |
| App version |
1.1.14 |
| OS |
Windows 10.0.26200 |
| Theme |
Espresso |
| Path |
/chat |
| Tenure |
Week 3 |
Subject: Possible GitHub Copilot SDK artifact-capture/session reliability issue during long-running certification work
Hi team,
I’d like to bring a possible GitHub Copilot SDK or agent-harness reliability problem onto your radar. We encountered the same failure twice while running a tightly controlled, long-running model certification workflow.
What we are doing
We use GitHub Copilot as the execution host for a code-review certification process. Each certification job:
Creates a fresh Copilot session.
Requests an exact model and reasoning-effort combination.
Gives the model a blind code-review assignment.
Requires the model to submit a structured result through a registered tool named pwi_submit_artifact.
Captures the tool payload, validates it, and persists a receipt containing the observed model identity and artifact hash.
The launcher uses one session at a time, unique artifact paths, explicit timeouts, and immutable attempt directories.
What happened
Two independent attempts stopped with the same failure:
copilot_worker_missing_artifact
In both cases:
A real Copilot session was created.
The requested model and reasoning effort were observed.
The session reached a terminal response.
No pwi_submit_artifact payload was captured by the application.
The same launcher successfully captured artifacts from the preceding sessions.
There were no overlapping session IDs or concurrent workers.
The attempts used different session IDs and different model routes.
Affected sessions:
Attempt 001
Session: 93a97bac-c176-4dab-8e37-031d993c74f1
Observed route: gpt-5.6-luna / low
Attempt 002
Session: 5ad8cf1f-8e07-4e58-9e67-e7ffc55281c3
Observed route: gpt-5.6-sol / low
Across the two attempts, 18 fresh sessions were consumed. The failure occurred on the ninth session of each attempt.
The second occurrence makes us concerned that this is not simply a one-off model response.
What we cannot currently determine
The SDK-facing application tells us that no structured artifact was captured, but the available application evidence does not clearly distinguish among these possibilities:
The model never invoked pwi_submit_artifact.
The model invoked the tool, but the tool event was not delivered or captured.
The model returned the payload in its final response rather than invoking the tool.
The tool invocation occurred, but parsing or SDK event handling failed.
The session terminated before the final tool event was persisted.
Our wrapper currently receives the returned response content, but when structured ArtifactCapture is empty, the operation raises a technical failure. The response/event material available to the wrapper is insufficiently durable to diagnose the boundary conclusively after the fact.
We are examining the local Copilot session-state and process logs for both session IDs. We will provide a redacted diagnostic inventory if it contains useful evidence.
Expected behavior
We would like the SDK or harness to provide durable and unambiguous terminal evidence that distinguishes:
model_did_not_submit_artifact
tool_invocation_observed_but_capture_failed
tool_payload_parse_failed
session_terminated_without_final_response
artifact_validation_failed