feat(ci): add AI-powered release notes generation by sfreudenthaler · Pull Request #35028 · dotCMS/core

sfreudenthaler · 2026-03-18T14:33:06Z

Summary

Adds an automated pipeline that generates developer-facing release notes for every dotCMS GitHub release. Closes #35011.

TypeScript data-gathering script extracts PR details, labels, and categorization from the GitHub API
Claude AI writes polished changelog prose from the structured JSON data
GitHub Actions orchestrates the pipeline as a non-blocking job in the existing release workflow

Release pipeline integration

The new release-notes job slots into the existing release pipeline after release completes:

release-prepare → build → deployment → release → release-notes (non-blocking)
                                                ↘              ↘
                                          finalize (always)   report (always)

Release notes generation flow

Within the release-notes job, these steps execute sequentially:

┌─────────────────────────────────────────────────────────────────┐
│  1. Validate release exists                                     │
│     └─ gh release view $TAG                                     │
│                                                                 │
│  2. Gather release data (TypeScript)                            │
│     ├─ Resolve previous tag from GitHub releases API            │
│     ├─ Fetch commit range via Compare API (with pagination)     │
│     ├─ Extract PR numbers from commit messages                  │
│     ├─ Batch-fetch PR details (title, labels, body)             │
│     ├─ Detect label signals:                                    │
│     │   ├─ Changelog: Skip → omit from output                  │
│     │   └─ Not Safe To Rollback → [!CAUTION] warning           │
│     ├─ Pre-categorize into 4 sections                           │
│     └─ Output structured JSON to /tmp/release-data.json         │
│                                                                 │
│  3. Assemble prompt                                             │
│     └─ prompt-template.md + release-data.json → prompt string   │
│                                                                 │
│  4. Claude writes release notes (Write tool only)               │
│     └─ Outputs /tmp/release-notes.md                            │
│                                                                 │
│  5. Update release description (deterministic shell step)       │
│     └─ gh release edit $TAG --notes-file /tmp/release-notes.md  │
└─────────────────────────────────────────────────────────────────┘

Backfill workflow (standalone)

For populating notes on past releases or re-generating notes manually:

workflow_dispatch(release_tag, previous_tag?)
  └─ calls cicd_comp_ai-release-notes-phase.yml
       └─ same flow as above

Tag filtering (monorepo)

The workflow skips non-standard releases automatically:

dotcms-cli-* → skipped (CLI product)
*_lts_* → skipped (LTS releases)
Must start with v → skipped otherwise

Key design decisions

Separation of concerns: Claude only writes prose (Write tool). The gh release edit runs as a deterministic shell step — auditable and retryable
Non-blocking: release-notes job uses if: success() and doesn't block finalize or report
Always-overwrite: Re-running produces consistent results
Shell-injection safe: Tag values referenced via env: variables, not direct ${{ }} interpolation

Files added/changed

File	Purpose
`.github/scripts/gather-release-data/src/`	TypeScript source (index, github, categorize, types)
`.github/scripts/gather-release-data/src/*.test.ts`	Jest tests (30 passing)
`.github/scripts/gather-release-data/prompt-template.md`	Prompt template for Claude
`.github/scripts/gather-release-data/package.json`	Dependencies (`@octokit/rest`)
`.github/scripts/gather-release-data/test/create-test-harness.sh`	Reproducible test harness
`.github/workflows/cicd_comp_ai-release-notes-phase.yml`	Reusable workflow component
`.github/workflows/cicd_ai-release-notes-backfill.yml`	Standalone backfill workflow
`.github/workflows/cicd_6-release.yml`	Added `release-notes` job (13 lines)

Secrets required

ANTHROPIC_API_KEY — already exists in dotCMS/core ✅

Test plan

Tested end-to-end in dotCMS/core-workflow-test:

Backfill workflow (run #440): Generated notes and updated release v26.03.17-02
LTS tag filter: Workflow correctly skipped for v26.03.17_lts_v01
CLI tag filter: Workflow correctly skipped for dotcms-cli-26.03.17-01
Changelog: Skip label: PR chore: add internal CI helper for build matrix generation core-workflow-test#449 omitted from output
Infrastructure categorization: PR ci: upgrade base Docker image to eclipse-temurin:21-jre-noble core-workflow-test#450 correctly placed in Infrastructure section
Jest tests: 30 passing (categorization logic, PR extraction, tag resolution)
Validate on next real release — or trigger backfill against a recent tag after merge

🤖 Generated with Claude Code

This PR fixes: #35011

Add an automated pipeline that generates developer-facing release notes for every dotCMS GitHub release. A TypeScript script gathers structured PR data from the GitHub API, Claude writes polished changelog prose, and a deterministic shell step updates the release description. Components: - TypeScript data-gathering script (.github/scripts/gather-release-data/) - Prompt template for Claude changelog writing - Reusable workflow component (cicd_comp_ai-release-notes-phase.yml) - Backfill workflow for past releases (cicd_ai-release-notes-backfill.yml) - New release-notes job in cicd_6-release.yml (non-blocking) - Reproducible test harness script Tested end-to-end in dotCMS/core-workflow-test: - Backfill workflow: generates notes and updates release description - Tag filtering: CLI and LTS releases correctly skipped - Label signals: Changelog:Skip omits PRs, Not Safe To Rollback triggers warning - 30 Jest tests passing for categorization and GitHub API helpers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

claude · 2026-03-18T14:34:06Z

Claude finished @sfreudenthaler's task in 1m 29s —— View job

Rollback Safety Analysis

Gather context and understand request
Read rollback-unsafe categories reference
Get full PR diff
Analyze diff against unsafe categories
Applied label: AI: Safe To Rollback

Result: ✅ Safe to Roll Back

Analyzed the diff (68e93ac...8bc9e78) against all rollback-unsafe categories (C-1 through M-4). No unsafe categories matched.

What this PR changes:

File(s)	Type	Rollback risk
`.github/scripts/gather-release-data/**`	New TypeScript scripts + tests	None — not deployed to dotCMS runtime
`.github/workflows/cicd_comp_ai-release-notes-phase.yml`	New reusable workflow	None — CI pipeline only
`.github/workflows/cicd_ai-release-notes-backfill.yml`	New standalone workflow	None — CI pipeline only
`.github/workflows/cicd_6-release.yml`	Adds `release-notes` job (14 lines)	None — non-blocking CI job

Why it's safe:

Zero database changes (no DDL, no runonce tasks, no migrations)
Zero Elasticsearch mapping changes
Zero content model changes (ImmutableContentlet, contentlet_as_json untouched)
Zero REST/GraphQL API contract changes
Zero OSGi interface changes
Zero binary storage changes

Rolling back to N-1 removes these workflow files and the release-notes job. The only effect is that the AI release-notes generation no longer runs — no application data or behavior is affected.

The 500-char cap was discarding 72% of PR body content across the repo, causing Claude to hallucinate categorization from titles alone. Analysis of the last 1000 merged PRs shows the max body is ~31K chars, so 50K is a safe ceiling with negligible token cost impact. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

sfreudenthaler

happy with it after these and will mark as ready for review when addressed

- Use org-standard runner version variable instead of hardcoded ubuntu-24.04 - Bump Node runtime from 20 to 22 to match @types/node ^22 in package.json - Replace static PROMPT_EOF heredoc delimiter with a random one generated via openssl at runtime, preventing PR body content from poisoning the GITHUB_ENV multiline variable Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- PR fetch errors now warn and continue instead of failing the entire run — a single deleted/transferred PR no longer blocks note generation - Remove dead label type guard (GitHub REST API always returns objects) - Remove unreachable commit cap warning (pagination fetches all commits) - Add untrusted data warning to prompt template to defend against prompt injection via PR bodies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add build: conventional commit prefix to title heuristics so these PRs land in Infrastructure instead of uncategorized. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b127e3fac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- Move continue-on-error to an allow_failure input (default false) so the backfill workflow fails visibly while the release pipeline stays non-blocking - Rollback-unsafe PRs can no longer be skipped via Changelog: Skip — prevents orphaned caution blocks referencing PRs absent from the notes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

erickgonzalez

We need to create these labels:
- Human: Not Safe To Rollback
Probably we need to automate and promote the use of Changelog: Skip
We also have the label Doc: Not Needed. Should we consider that or maybe delete it and only use the Changelog: Skip
We should automate PR title with Claude code, since by looking at the last 50 PRs, around 10 PRs don't follow conventional commits
We should include task and refactor since these are used as conventional commits as well.

Addresses some of Ericks comments. Decided to keep chore/task in `feature` because it's called features and enhancements and i'm not sure there's a distinction from task/chore output there from a user perspective

sfreudenthaler · 2026-03-19T15:38:54Z

We need to create these labels:

Human: Not Safe To Rollback

addressed in 8bc9e78

We also have the label Doc: Not Needed. Should we consider that or maybe delete it and only use the Changelog: Skip

8bc9e78

Probably we need to automate and promote the use of Changelog: Skip

We should automate PR title with Claude code, since by looking at the last 50 PRs, around 10 PRs don't follow conventional commits

Great call, Will make sure we capture in our Ai process improvement roadmap

We should include task and refactor since these are used as conventional commits as well.

ok i put task in feature cuz from customer perspective it's a feature or enhancement most likely. refactor i put in infra but I suppose it could be skipped too.

## Summary Adds an automated pipeline that generates developer-facing release notes for every dotCMS GitHub release. Closes #35011. - **TypeScript data-gathering script** extracts PR details, labels, and categorization from the GitHub API - **Claude AI** writes polished changelog prose from the structured JSON data - **GitHub Actions** orchestrates the pipeline as a non-blocking job in the existing release workflow ### Release pipeline integration The new `release-notes` job slots into the existing release pipeline after `release` completes: ``` release-prepare → build → deployment → release → release-notes (non-blocking) ↘ ↘ finalize (always) report (always) ``` ### Release notes generation flow Within the `release-notes` job, these steps execute sequentially: ``` ┌─────────────────────────────────────────────────────────────────┐ │ 1. Validate release exists │ │ └─ gh release view $TAG │ │ │ │ 2. Gather release data (TypeScript) │ │ ├─ Resolve previous tag from GitHub releases API │ │ ├─ Fetch commit range via Compare API (with pagination) │ │ ├─ Extract PR numbers from commit messages │ │ ├─ Batch-fetch PR details (title, labels, body) │ │ ├─ Detect label signals: │ │ │ ├─ Changelog: Skip → omit from output │ │ │ └─ Not Safe To Rollback → [!CAUTION] warning │ │ ├─ Pre-categorize into 4 sections │ │ └─ Output structured JSON to /tmp/release-data.json │ │ │ │ 3. Assemble prompt │ │ └─ prompt-template.md + release-data.json → prompt string │ │ │ │ 4. Claude writes release notes (Write tool only) │ │ └─ Outputs /tmp/release-notes.md │ │ │ │ 5. Update release description (deterministic shell step) │ │ └─ gh release edit $TAG --notes-file /tmp/release-notes.md │ └─────────────────────────────────────────────────────────────────┘ ``` ### Backfill workflow (standalone) For populating notes on past releases or re-generating notes manually: ``` workflow_dispatch(release_tag, previous_tag?) └─ calls cicd_comp_ai-release-notes-phase.yml └─ same flow as above ``` ### Tag filtering (monorepo) The workflow skips non-standard releases automatically: - `dotcms-cli-*` → skipped (CLI product) - `*_lts_*` → skipped (LTS releases) - Must start with `v` → skipped otherwise ### Key design decisions - **Separation of concerns**: Claude only writes prose (`Write` tool). The `gh release edit` runs as a deterministic shell step — auditable and retryable - **Non-blocking**: `release-notes` job uses `if: success()` and doesn't block `finalize` or `report` - **Always-overwrite**: Re-running produces consistent results - **Shell-injection safe**: Tag values referenced via `env:` variables, not direct `${{ }}` interpolation ### Files added/changed | File | Purpose | |------|---------| | `.github/scripts/gather-release-data/src/` | TypeScript source (index, github, categorize, types) | | `.github/scripts/gather-release-data/src/*.test.ts` | Jest tests (30 passing) | | `.github/scripts/gather-release-data/prompt-template.md` | Prompt template for Claude | | `.github/scripts/gather-release-data/package.json` | Dependencies (`@octokit/rest`) | | `.github/scripts/gather-release-data/test/create-test-harness.sh` | Reproducible test harness | | `.github/workflows/cicd_comp_ai-release-notes-phase.yml` | Reusable workflow component | | `.github/workflows/cicd_ai-release-notes-backfill.yml` | Standalone backfill workflow | | `.github/workflows/cicd_6-release.yml` | Added `release-notes` job (13 lines) | ### Secrets required - `ANTHROPIC_API_KEY` — already exists in dotCMS/core ✅ ## Test plan Tested end-to-end in [dotCMS/core-workflow-test](https://github.com/dotCMS/core-workflow-test): - [x] **Backfill workflow** ([run #440](https://github.com/dotCMS/core-workflow-test/actions/runs/23248206205)): Generated notes and updated [release v26.03.17-02](https://github.com/dotCMS/core-workflow-test/releases/tag/v26.03.17-02) - [x] **LTS tag filter**: Workflow correctly skipped for `v26.03.17_lts_v01` - [x] **CLI tag filter**: Workflow correctly skipped for `dotcms-cli-26.03.17-01` - [x] **Changelog: Skip label**: PR dotCMS#449 omitted from output - [x] **Infrastructure categorization**: PR dotCMS#450 correctly placed in Infrastructure section - [x] **Jest tests**: 30 passing (categorization logic, PR extraction, tag resolution) - [ ] Validate on next real release — or trigger backfill against a recent tag after merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

github-project-automation bot added this to dotCMS - Product Planning Mar 18, 2026

github-actions bot mentioned this pull request Mar 18, 2026

epic(ci): Automated release notes for GitHub releases #35011

Open

github-actions bot added Area : CI/CD PR changes GitHub Actions/workflows Area : Documentation PR changes documentation files labels Mar 18, 2026

claude bot added the AI: Safe To Rollback label Mar 18, 2026

sfreudenthaler requested a review from erickgonzalez March 18, 2026 15:31

sfreudenthaler commented Mar 18, 2026

View reviewed changes

Comment thread .github/scripts/gather-release-data/src/github.ts Outdated

sfreudenthaler commented Mar 18, 2026

View reviewed changes

sfreudenthaler and others added 7 commits March 18, 2026 14:45

Update .github/workflows/cicd_ai-release-notes-backfill.yml

b83e466

Update .github/workflows/cicd_comp_ai-release-notes-phase.yml

ab07e4d

Update .github/workflows/cicd_comp_ai-release-notes-phase.yml

c066bd2

Update .github/workflows/cicd_comp_ai-release-notes-phase.yml

8ecde49

Update .github/scripts/gather-release-data/prompt-template.md

55bc947

sfreudenthaler marked this pull request as ready for review March 18, 2026 20:21

sfreudenthaler requested a review from a team as a code owner March 18, 2026 20:21

fix(ci): categorize build: prefix as infrastructure

13ec3ea

Add build: conventional commit prefix to title heuristics so these PRs land in Infrastructure instead of uncategorized. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

chatgpt-codex-connector bot reviewed Mar 18, 2026

View reviewed changes

Comment thread .github/workflows/cicd_comp_ai-release-notes-phase.yml

Comment thread .github/workflows/cicd_comp_ai-release-notes-phase.yml Outdated

Comment thread .github/workflows/cicd_comp_ai-release-notes-phase.yml

erickgonzalez reviewed Mar 19, 2026

View reviewed changes

addresses comments

8bc9e78

Addresses some of Ericks comments. Decided to keep chore/task in `feature` because it's called features and enhancements and i'm not sure there's a distinction from task/chore output there from a user perspective

sfreudenthaler requested a review from erickgonzalez March 19, 2026 15:38

erickgonzalez approved these changes Mar 19, 2026

View reviewed changes

oidacra approved these changes Mar 19, 2026

View reviewed changes

sfreudenthaler enabled auto-merge March 20, 2026 12:35

sfreudenthaler added this pull request to the merge queue Mar 20, 2026

yolabingo removed this pull request from the merge queue due to a manual request Mar 20, 2026

spbolton added this pull request to the merge queue Mar 20, 2026

Merged via the queue into main with commit bf4c4c7 Mar 20, 2026
44 checks passed

spbolton deleted the issue-35011-ai-release-notes branch March 20, 2026 15:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ci): add AI-powered release notes generation#35028

feat(ci): add AI-powered release notes generation#35028
spbolton merged 12 commits intomainfrom
issue-35011-ai-release-notes

sfreudenthaler commented Mar 18, 2026 •

edited

Loading

Uh oh!

claude bot commented Mar 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

sfreudenthaler left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erickgonzalez left a comment •

edited

Loading

Uh oh!

sfreudenthaler commented Mar 19, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sfreudenthaler commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Release pipeline integration

Release notes generation flow

Backfill workflow (standalone)

Tag filtering (monorepo)

Key design decisions

Files added/changed

Secrets required

Test plan

Uh oh!

claude bot commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rollback Safety Analysis

Result: ✅ Safe to Roll Back

Uh oh!

Uh oh!

sfreudenthaler left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erickgonzalez left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sfreudenthaler commented Mar 19, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sfreudenthaler commented Mar 18, 2026 •

edited

Loading

claude bot commented Mar 18, 2026 •

edited

Loading

erickgonzalez left a comment •

edited

Loading