Skip to content

feat(evals): link Braintrust experiments to their CI run and the PR - #384

Merged
mattrossman merged 31 commits into
mainfrom
mattrossman/ai-1301-link-braintrust-experiments-to-their-ci-run-and-the-job
Oct 9, 2026
Merged

mattrossman merged 31 commits into
mainfrom
mattrossman/ai-1301-link-braintrust-experiments-to-their-ci-run-and-the-job

Conversation

@mattrossman

@mattrossman mattrossman commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Improves cross-linking between Braintrust <> GitHub Actions <> GitHub PR. This makes it easier to e.g. debug CI logs from a trace, or debug the source of a run.

  • Braintrust now includes the ci_run_url when relevant GitHub env vars are present
  • GitHub Actions job summary now links to the related PRs (and links to braintrust are reformatted for consistency)

Note

Manual workflows are associated with branches, not PRs directly. The job summary lists the PR that triggered a PR-synced run (like with run-evals-changed), and for a manual dispatch it lists every open PR for the target branch, or (none).

ci_run_url in Experiment Related PRs in job summary Related PRs when the branch has multiple PRs (job summary)
CleanShot 2026-10-09 at 11 03 31 AM@2x CleanShot 2026-10-09 at 1 42 23 PM@2x CleanShot 2026-10-09 at 1 42 49 PM@2x

The existing pr_url in Braintrust metadata is sourced only for PR triggers, so it doesn't map cleanly to the way I check for multiple PRs in the job summary. I've left it as-is to avoid overloading concepts.

Closes AI-1301

@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
evals Ignored Ignored Preview Oct 9, 2026 5:53pm UTC

Request Review

@mattrossman
mattrossman marked this pull request as ready for review October 9, 2026 17:44
@mattrossman
mattrossman requested a review from a team as a code owner October 9, 2026 17:44
@mattrossman
mattrossman merged commit ab6fe4c into main Oct 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants