Update pre-release publishing architecture

[ottonomy]

I think we need to move to a new iteration of how documents in this repository are built and published
across the spec lifecycle: editor's latest drafts, pull‑request previews, and
final releases. This concept supersedes the legacy GitHub Actions workflows in
.github/workflows/*.disabled.

---

1. Goals

1. Multiple active spec lines. Today only ob_v3p0/ is active.
   Imminently we will add ob_v3p1/ and profiles/trustedcredential/v1p0/.
   The pipeline must scale to N active specs without per‑spec workflow files. There is a pain point that the staging server state could get messy if different concepts for .lines are pushed to it.
2. Editor's Latest Draft on every push to develop. Each active spec's
   document set is rebuilt and published to a stable, well‑known URL.
3. A unique preview URL per pull request, updated on every commit, with
   the URLs surfaced both in the workflow output and in a single,
   continuously‑updated PR comment (no comment spam).
4. Status banner reflects the trigger. Drafts and previews must not render
   as Final Release; the 1EdTech ReSpec banner should clearly say
   "Editor's Draft" or "Pull Request #N Preview".
5. Final releases are out of scope for this workflow. They are published
   by 1EdTech's separate publishing system (the main branch is the input to
   that system; we do not touch it here).
6. Minimal config to add a new spec. Adding ob_v3p1 or a new profile
   should be a one‑PR change to a single manifest file plus the spec source
   directory itself.

2. Hosting backend: Cloudflare Pages

We could use Cloudflare Pages with Direct Upload via cloudflare/wrangler-action
from GitHub Actions. We do not use Cloudflare's GitHub‑integration build
runner — all builds happen in our own GitHub Actions where we control the
toolchain (ReSpec rendering, manifest matrix, status injection).

flowchart LR
  subgraph Repo["Repository (1EdTech/openbadges-specification)"]
    Dev[develop branch]
    Feat["feature/* branches (canonical repo)"]
    Fork[forked PR head]
  end

  subgraph GHA["GitHub Actions (gated by IS_PUBLISHING_REPO=true)"]
    EWF[editors-draft.yml]
    PWF[pr-preview.yml]
    FWF["pr-preview-fork.yml (label-gated)"]
    CWF[pr-cleanup.yml]
  end

  subgraph CFP["Cloudflare Pages project: openbadges"]
    Prod["production branch = develop"]
    PRn["branch alias pr-N"]
  end

  subgraph URLs["Public URLs"]
    DraftsURL["drafts.openbadges.org/<spec>/..."]
    PrevURL["pr-N.openbadges.pages.dev/<spec>/..."]
  end

  Dev -- "push" --> EWF
  Feat -- "pull_request" --> PWF
  Fork -- "pull_request_target + label" --> FWF
  Repo -- "pull_request: closed" --> CWF

  EWF -- "wrangler --branch=develop" --> Prod
  PWF -- "wrangler --branch=pr-N" --> PRn
  FWF -- "wrangler --branch=pr-N" --> PRn
  CWF -- "wrangler deployment delete pr-N" --> PRn

  Prod --> DraftsURL
  PRn --> PrevURL

  ForkRepo((forks of this repo)) -. "all jobs short-circuit (no IS_PUBLISHING_REPO)" .-x GHA

Loading

The Cloudflare Pages project is named openbadges.

URL	Purpose	How
https://drafts.openbadges.org/<spec>/...	Editor's Latest Draft of every active spec	Production deploy, triggered by push to develop
https://pr-<N>.openbadges.pages.dev/<changed-spec>/...	Pull request preview of changed spec(s)	Preview deploy, branch alias pr-<N> set by wrangler --branch=pr-<N>
https://<hash>.openbadges.pages.dev/...	Atomic, immutable per‑build URL	Auto, returned by every Cloudflare Pages deploy

Notes:

• Production branch on the Pages project = develop. Anything pushed to
  develop updates <project>.pages.dev and the drafts.openbadges.org
  custom domain.
• pr-<N> is the alias we set explicitly via wrangler pages deploy ./out --branch=pr-<N>. We do not rely on the actual git branch name, so PRs
  from forks (with maintainer approval — see §8) and PRs with awkward branch
  names ("feature/foo-bar") still get a clean preview URL.
• X-Robots-Tag: noindex is added by Cloudflare automatically to every
  preview deployment — no SEO duplicate‑content risk.
• Custom branch aliases / wildcard preview hostnames (e.g.
  pr-123.preview.openbadges.org) are deferred to a future iteration. The
  default pages.dev URL is sufficient for v1.
• Fork safety. Every workflow's first job is gated on
  if: vars.IS_PUBLISHING_REPO == 'true'. That repository variable is set
  only on the canonical 1EdTech/openbadges-specification repo, so anyone
  who forks the repo gets the workflow files but every job short‑circuits.
  No fork CI ever attempts to deploy. See §8 for setup.

Why Cloudflare Pages over GitHub Pages

GitHub Pages serves a single site per repo from a single branch. To keep N
parallel PR previews on it, we'd have to commit each preview as a subfolder
into the gh-pages branch and clean them up by hand — exactly what the legacy
workflows did, with all the friction that produced. Cloudflare Pages models
preview deployments natively: each is its own immutable snapshot with its own
URL and an auto‑updating branch alias, with no commits and no cleanup churn in
git.

3. URL layout

We mirror the repo's source directory structure into the published site root.

<published-root>/
├── index.html                      # landing page listing active specs
├── ob_v3p0/
│   ├── ob_v3p0.html                # main spec
│   ├── errata/ob_err_v3p0.html
│   ├── cert/ob-cert-v3p0.html
│   ├── impl/ob_impl_v3p0.html
│   ├── examples.html
│   ├── vocab.html
│   ├── context.json
│   ├── context-3.0.3.json
│   ├── openapi/openapi.json
│   ├── openapi/openapi.yaml
│   └── schema/*.json
├── ob_v3p1/                        # added when active
│   └── ...
└── profiles/
    └── trustedcredential/
        └── v1p0/                   # added when active
            └── ...

PR preview deploys reuse the same internal paths under their pr-<N> host.
Because each PR preview deploy contains only the specs touched by that PR
(see §6), unchanged specs return 404 in the preview deploy. That is
intentional and acceptable: the spec documents do not link sideways to other
spec lines, so a 404 on an unrelated path is harmless. Maintainers wanting to
spot‑check a sibling spec can always click through to the editor's draft URL
on drafts.openbadges.org.

4. Spec manifest

A single file at .github/specs.yaml declares which specs are active, what
files belong to each one, and any per‑spec overrides. The publish workflow
reads this manifest to build its job matrix.

# .github/specs.yaml
specs:
  - id: ob_v3p0
    title: "Open Badges 3.0"
    active: true
    src_dir: ob_v3p0
    dest_dir: ob_v3p0          # default = src_dir; override if path needs aliasing
    documents:
      - source: ob_v3p0.html
        destination: ob_v3p0.html
        kind: main
      - source: errata/ob_err_v3p0.html
        destination: errata/ob_err_v3p0.html
        kind: errata
      - source: cert/ob-cert-v3p0.html
        destination: cert/ob-cert-v3p0.html
        kind: cert
      - source: impl/ob_impl_v3p0.html
        destination: impl/ob_impl_v3p0.html
        kind: impl
      - source: examples.html
        destination: examples.html
        kind: examples
    assets:                     # copied as-is, no ReSpec render
      - context*.json
      - vocab.html
      - vocab.ttl
      - vocabulary.ttl
      - images/**
    sideload:                   # optional; sent to datamodels API before render
      - ob_v3p0.lines
      - common_credentials.lines
    fetched_assets:             # optional; produced by API pulls
      - dest: openapi/openapi.json
        from: "https://datamodels-staging.imsglobal.org/openapischema/org.1edtech.ob.v3p0.model?binding=json"
      - dest: openapi/openapi.yaml
        from: "https://datamodels-staging.imsglobal.org/openapischema/org.1edtech.ob.v3p0.model?binding=yaml"
      - dest: schema/achievementcredential.json
        from: "https://datamodels-staging.imsglobal.org/jsonschema/org.1edtech.ob.v3p0.achievementcredential.class"
      # ... etc
    spec_status:
      develop: "Editor's Draft"
      pr: "Pull Request #${PR_NUMBER} Preview"
      # If a spec is final and we only edit errata, override develop here:
      # develop: "Final Release"
    shared_files:               # changes here also trigger a build of this spec
      - .github/specs.yaml

  - id: ob_v3p1
    title: "Open Badges 3.1"
    active: true
    src_dir: ob_v3p1
    # ... same shape

  - id: trustedcredential_v1p0
    title: "Trusted Credential Profile 1.0"
    active: true
    src_dir: profiles/trustedcredential/v1p0
    # ... same shape

  - id: ob_v2p1
    title: "Open Badges 2.1"
    active: false               # excluded from all builds

A spec's "document set" is the union of documents + assets +
fetched_assets. The workflow computes per‑PR change sets by globbing
src_dir against the PR's changed files (plus any shared_files matches).

5. ReSpec status injection

The 1EdTech ReSpec fork (loaded from https://purl.imsglobal.org/respec/respec-1edtech.js)
takes any string for specStatus and renders it in the document header. We
inject the right string per trigger before rendering.

A small Node script tools/inject-status.mjs runs as a build step and rewrites
the inline respecConfig block of each ReSpec source HTML in‑tree (in a
working copy, never committed):

// pseudo-code
const cfg = readRespecConfig(html);
cfg.specStatus = process.env.SPEC_STATUS;       // e.g. "Editor's Draft"
cfg.specDate   = process.env.SPEC_DATE;         // today, or PR commit date
if (process.env.SOTD_HTML) {
  insertSotdAside(html, process.env.SOTD_HTML); // a colored note at top of doc
}
writeBack(html);

Default status strings (overridable per spec via spec_status in the
manifest):

Trigger	specStatus	SOTD banner
Push to develop	Editor's Draft	"This is the editor's latest draft of <spec title>. The authoritative final release is published at imsglobal.org."
pull_request (canonical)	Pull Request #N Preview	"This is an automated preview of pull request #N at commit <short‑sha>. Not an official release."
pull_request_target (fork, label‑gated)	Pull Request #N Preview (Fork)	Same banner with explicit "from external contributor" notation.

6. Workflows

Four workflows live under .github/workflows/:

File	Trigger	Purpose
editors-draft.yml	push to develop	Build all active specs, deploy to production branch
pr-preview.yml	pull_request from same‑repo branches	Build changed specs, deploy to pr-<N>, comment on PR
pr-preview-fork.yml	pull_request_target + label safe-to-preview	Same as above for fork PRs, secret‑safe
pr-cleanup.yml	pull_request: closed	Delete pr-<N> Cloudflare deployment

Fork-skip guard (applies to all four). Each workflow's first job uses
if: vars.IS_PUBLISHING_REPO == 'true'. Subsequent jobs depend on it via
needs:, so when the variable is absent (i.e., on any fork of this repo)
every job is skipped and no deploy is ever attempted from a fork. Sketch:

jobs:
  preflight:
    if: vars.IS_PUBLISHING_REPO == 'true'
    runs-on: ubuntu-latest
    steps:
      - run: echo "Publishing repo confirmed; proceeding."
  build:
    needs: preflight
    # ... matrix
  publish:
    needs: build
    # ... wrangler deploy

6.1 editors-draft.yml (push → develop)

flowchart LR
  P[push to develop] --> M[load specs.yaml]
  M --> X[matrix: one job per active spec]
  X --> S[ReSpec render + asset copy + status injection]
  S --> A[upload-artifact per spec]
  A --> Q[publish job: combine artifacts]
  Q --> W[wrangler pages deploy ./out --branch=develop]
  W --> R[invalidate cache, post status to commit]

Loading

• The matrix is built from specs[*] filtered by active: true. No path
  filter — every push to develop rebuilds every active spec, which is fine
  for the volume we expect.
• The matrix job sideloads each spec's sideload files to
  datamodels-staging.imsglobal.org before rendering, so the editor's draft
  always reflects the current .lines model. This is the only place
  sideload happens (see §6.5).
• The publish job concatenates the matrix outputs into one ./out/ tree and
  performs a single Cloudflare deploy (one <hash>.<project>.pages.dev
  snapshot) with --branch=develop. Since develop is the production branch
  on the Pages project, this also updates drafts.openbadges.org.

6.2 pr-preview.yml (same‑repo PRs)

flowchart LR
  P[pull_request opened/synchronize] --> D[diff: changed files]
  D --> F[filter via specs.yaml: which specs are affected?]
  F --> X[matrix: one job per affected spec]
  X --> S[ReSpec render + status injection PR-N]
  S --> A[upload-artifact]
  A --> Q[publish job: combine artifacts]
  Q --> W["wrangler pages deploy ./out --branch=pr-N"]
  W --> C[upsert sticky PR comment with URLs]

Loading

Change detection rules (hybrid path filter):

1. For each active spec, mark it affected if any changed file matches
   src_dir/** or any glob in its shared_files.
2. Special case: changes to .github/specs.yaml mark all specs affected.
3. Maintainer escape hatch: if the PR carries the label build-all, all
   active specs are built regardless of diff.
4. If no spec is affected (e.g., only README.md changed) the workflow exits
   successfully without deploying.

6.3 pr-preview-fork.yml (fork PRs, gated)

Uses pull_request_target (which gives the workflow the base repo's
secrets) but only runs when:

• The PR has the safe-to-preview label, and
• The label was added by a user with write permission on the repo (verified
  in‑workflow via the GitHub API).

To keep secrets safe, the workflow checks out the merge commit of the PR
without running any package install scripts from the PR head, and runs
wrangler from the base repo's pinned tooling. Build matches §6.2 otherwise.
The sticky PR comment makes clear that this is a fork preview and lists the
commit SHA actually built.

Why two workflows instead of one? pull_request_target runs in the context
of the base repo and cannot be conditioned cleanly on label presence inside
a single workflow without giving fork code partial access to secrets. Two
separate workflows with mutually exclusive triggers avoids that footgun.

6.4 pr-cleanup.yml (PR closed)

flowchart LR
  P["pull_request: closed (merged or not)"] --> L["wrangler pages deployment list"]
  L --> F["jq filter: branch == pr-N"]
  F --> D["wrangler pages deployment delete (xN)"]
  D --> S["sticky comment: 'Preview removed'"]

Loading

Concretely:

wrangler pages deployment list --project-name=openbadges \
  | jq -r '.[] | select(.deployment_trigger.metadata.branch=="pr-'"$PR"'") | .id' \
  | xargs -n1 wrangler pages deployment delete --project-name=openbadges --yes

This removes both the pr-<N> alias and every hash deployment behind it,
keeping the project tidy and avoiding stale links surviving long after the PR.

6.5 Datamodel sideload policy

Sideloading *.lines files to datamodels-staging.imsglobal.org is a
develop‑only operation. PR preview builds do not sideload; instead,
the PR preview's OpenAPI/JSON Schema sections are rendered against whatever
model is currently staged (i.e., what the latest push to develop
uploaded).

Why: the staging service is a shared mutable resource keyed by model ID
(e.g., org.1edtech.ob.v3p0.model). Letting every PR upload to it would
race other PRs and the editor's draft, polluting each other's previews.

Implications for PR authors:

• Editorial / prose / example PRs render normally — they don't touch the
  model.
• PRs that change *.lines will see their model changes reflected in the
  preview's <section data-model="..."> blocks only after the PR is
  merged to develop. Until then, the preview shows the previous model.
  The PR template should call this out so authors aren't surprised.
• If a PR genuinely needs the staging service to reflect its model changes
  for review, a maintainer can manually trigger a one-shot
  workflow_dispatch of editors-draft.yml against the PR ref (the
  workflow accepts a ref input for this purpose), accepting that staging
  will be temporarily desynced from develop until the next merge.

7. PR comment behavior (sticky comment)

A single comment per PR, created on first preview build, edited in place on
every subsequent build. We use
marocchino/sticky-pull-request-comment
with header: spec-preview so that the same comment is updated regardless of
which workflow run posts it.

Comment body format:

### 📄 Preview: Pull Request #123

Built from commit `abc1234` at 2026-04-17 12:34 UTC.

| Spec | Document | Preview URL |
| --- | --- | --- |
| Open Badges 3.1 | Main | https://pr-123.openbadges.pages.dev/ob_v3p1/ob_v3p1.html |
| Open Badges 3.1 | Errata | https://pr-123.openbadges.pages.dev/ob_v3p1/errata/ob_err_v3p1.html |
| Open Badges 3.1 | Examples | https://pr-123.openbadges.pages.dev/ob_v3p1/examples.html |

Atomic snapshot: https://6f4e2a1b.openbadges.pages.dev/

This preview will be deleted when the PR is closed.

The same URL list (and the atomic snapshot URL) is also written to the
GitHub Actions job summary ($GITHUB_STEP_SUMMARY) and exposed as workflow
outputs so other workflows or status checks can consume them.

8. Required secrets and one‑time setup

Repository secrets (Settings → Secrets and variables → Actions):

Secret	Purpose
CLOUDFLARE_API_TOKEN	Pages: Edit token, scoped to the project
CLOUDFLARE_ACCOUNT_ID	1EdTech Cloudflare account ID
API_KEY	Existing 1EdTech datamodels staging API key (sideload + fetched_assets)
GIT_CRYPT_KEY	Existing git‑crypt key for unlocking encrypted secrets in the repo (only if the build still requires it)

One‑time Cloudflare setup:

1. Create a Cloudflare Pages project named openbadges using
   "Direct Upload" mode (no GitHub repo connection).
2. Set Production branch = develop in Pages project settings.
3. Add custom domain drafts.openbadges.org to the project; Cloudflare
   provisions the cert and creates the proxied DNS record automatically
   (assuming the DNS zone is on Cloudflare).
4. Optional: enable Cloudflare Access on preview deployments only if we want
   private previews. Default = public (recommended).

GitHub setup:

1. Add the four secrets above (Settings → Secrets and variables → Actions →
   Secrets).
2. Add a repository variable (Settings → Secrets and variables → Actions
   → Variables) named IS_PUBLISHING_REPO with value true. This is the
   master switch that gates every workflow so forks short‑circuit. Do not
   set this variable on forks. GitHub doesn't copy variables when a repo
   is forked, so the default state on a fork is correctly "publishing
   disabled."
3. Create the labels safe-to-preview and build-all.
4. Branch protection on develop: keep current rules; the new workflows do
   not push to git.

9. Migration / rollout plan

Phased to keep the existing site working while the new pipeline ramps up.

1. Phase 0 (this PR). Land this docs/publishing.md. No code yet.
2. Phase 1 — pipeline scaffolding. Add tools/inject-status.mjs,
   .github/specs.yaml (with only ob_v3p0 active), and the four workflows,
   wired to a dummy Cloudflare project so we can iterate without exposing the
   real drafts.openbadges.org URL.
3. Phase 2 — cutover. Provision the real Pages project, point
   drafts.openbadges.org at it, enable the workflows for real, retire the
   *.disabled legacy workflows.
4. Phase 3 — onboard ob_v3p1 and trustedcredential v1p0. Add their
   manifest entries when the source directories land.
5. Phase 4 — gh-pages retirement (at cutover). As part of the Phase 2
   cutover, replace the contents of the gh-pages branch with a single
   index.html that redirects to drafts.openbadges.org (with sub-path
   redirects for the handful of historically-linked files: ob_v3p0.html,
   examples.html, vocab.html, context*.json, openapi/*, schema/*).
   Then disable GitHub Pages on the repo. The gh-pages branch is left in
   git history for archival but receives no further automated writes.

10. Open questions

These are deliberately left for a follow‑up PR or design discussion:

1. Wildcard preview hostname. Should we add
   pr-<N>.preview.openbadges.org later, or is pr-<N>.openbadges.pages.dev
   permanently fine?
2. Per‑PR CDN cost / quota. Cloudflare Pages free plan caps deployments
   per project per month. At our PR volume this is unlikely to bind, but we
   should confirm the 1EdTech Cloudflare plan tier before cutover.
3. ReSpec render tool. w3c/spec-prod@v2 works for our 1EdTech ReSpec
   fork (the source HTML loads its own respec script), but we may prefer a
   thinner wrapper around respec/respec CLI for build determinism. Decide
   in Phase 1.

11. Out of scope

• Final release publication. Final and Candidate Final releases of any spec
  are produced by 1EdTech's separate publishing system, fed by the main
  branch of this repo. Nothing in this document changes that flow.
• Authoring tooling (lint, link checks, vocabulary validation). Tracked
  separately.

[xaviaracil]

At 1EdTech, we’re implementing a new publishing process that includes a pre-release workflow. This workflow grants access control to draft versions of documents.

I’ll ensure the agreed requirements from this issue are also met in this new process.

[xaviaracil]

I think we can avoid having a manifest file by setting some conventions in the repository. For instance, a particular version can have this folder structure:

version
    |–––– cert
    |      |––– assets.      // images, etc of  the cert document
    |      |––– index.md.    // cert document
    |–––– errata
    |      |––– assets.      // images, etc of  the errata document
    |      |––– index.md.    // errata document
    |–––– impl
    |      |––– assets.      // images, etc of  the implementation guide
    |      |––– index.md.    // implementation guide
    |–––– assets
    |      |––– context.json
    |      |––– images
    |      |––– vocab.html
    |      |––– vocab.ttl
    |      |––– rendered.yaml // contains the list of rendered assets (OpenApi, JSON Schemas ,etc)
    |–––– examples            // examples
    |–––– index.md            // specification document
    |–––– spec.lines          // specification model

We can extract the status of the specification from version/index.md and sideload the models only when that status is not "final".

Therefore, our workflow can identify the folder of the files being pushed and thus, the version of the spec to publish, extract the status from the spec document and sideload models if needed, and publish the documents accordingly.