Skip to content

Replace CI scan curl access - #132753

Draft
vitek-karas wants to merge 3 commits into
dotnet:mainfrom
vitek-karas:vitek-karas-isolate-ci-evidence-reader
Draft

Replace CI scan curl access#132753
vitek-karas wants to merge 3 commits into
dotnet:mainfrom
vitek-karas:vitek-karas-isolate-ci-evidence-reader

Conversation

@vitek-karas

Copy link
Copy Markdown
Member

Motivation

This replaces direct HTTP reads in the CI failure scan with one repository-owned helper. It also creates a clear place to later redirect CI evidence inputs for deterministic evals, without changing the commands the scan uses.

Changes

  • Add ci-evidence-reader with fixed commands for Azure DevOps and Helix evidence.
  • Update ci-failure-scan to install and use the helper instead of curl.
  • Keep build definition IDs open while retaining the fixed build query shape and paging limits.
  • Use a table to define the direct request commands and add focused tests for the command routing.

Validation

  • py -3 .github\workflows\tests\test_ci_evidence_reader.py (20 tests)
  • gh-aw v0.86.2 compile ci-failure-scan --strict --validate

Note

This pull request description was created by GitHub Copilot.

vitek-karas and others added 3 commits August 25, 2026 16:30
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 65c2a187-ce76-4264-9d3e-ee20fc9e7565
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 65c2a187-ce76-4264-9d3e-ee20fc9e7565
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
15 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the area-skills Agent Skills label Aug 25, 2026
@vitek-karas
vitek-karas requested a review from PureWeen August 25, 2026 16:10
```

Do NOT retry an inline URL hoping the rejection clears. Switch to the variable pattern immediately.
All AzDO and Helix reads must use the repository-owned

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the Helix reads use the hlx MCP server?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably - but it would require quite a bit of changes to the agent, since the MCP tools are different shape/process than what the agent does today. I would also need to test it if it will work without auth, technically it should, but I didn't try to deploy it into actions.
Finally it makes the "mocking" for deterministic evals a bit more difficult/complex.

Using the MCP would likely mean less new code, but it would also mean a bigger change to the agent itself. I would not combine the two changes even though they're related.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hlx only requires auth when it is required by the endpoint. I don't need to block the work here but I'm happy to build in features to support evals and add url filtering to hlx, lets chat about what would be useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-skills Agent Skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants