Skip to content

ci: shard and harden daily translations - #152

Open
BZ-D wants to merge 1 commit into
mainfrom
agent/fix-daily-translation-ci
Open

ci: shard and harden daily translations#152
BZ-D wants to merge 1 commit into
mainfrom
agent/fix-daily-translation-ci

Conversation

@BZ-D

@BZ-D BZ-D commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • split each target language into two weight-balanced shards, with at most three translation jobs running concurrently
  • pin detection, translation, and finalization to one workflow and upstream source commit
  • merge structured artifacts only after validating shard metadata, commit consistency, statistics, failures, and deletion lists
  • propagate upstream deletions and rename sources to every language without advancing the baseline after a failed translation
  • add pull-request validation with read-only permissions and isolate PR concurrency from production runs

Root cause

The scheduled workflow translated all six languages in one job and committed only after every translation completed. Recent runs reached GitHub Actions six-hour per-job limit while translation was still progressing. The timed-out runs never advanced last-sync.json, so each following run repeated the growing backlog.

Language-only parallelism was still too close to the limit for the current backlog. The workflow now creates twelve language-by-shard jobs. Source byte size is used to balance each pair of shards while max-parallel: 3 preserves the existing API load ceiling.

Reliability details

  • newly created translation files are included in artifacts
  • any per-file failure fails the job before changelog or baseline updates
  • upstream deletions and rename sources remove stale source and translated files
  • the final merger rejects missing or inconsistent artifacts before commit
  • only the final publish job receives contents: write; PR validation is read-only
  • a late main update is rebased before the bot pushes

Validation

  • translator TypeScript build
  • 4 Node tests covering shard merge, cross-commit rejection, delete/rename parsing, and deterministic balancing
  • Actionlint validation of the workflow
  • structured real upstream sync --detect-only run (75 changed documents)
  • invalid shard argument failure check
  • git diff --check
  • full website production build: 1652 static routes generated and 1632 pages indexed

The real model API path requires the repository secret and can be exercised by manually dispatching the workflow after merge.

@BZ-D
BZ-D force-pushed the agent/fix-daily-translation-ci branch from 80568eb to 4501381 Compare July 14, 2026 07:00
@BZ-D BZ-D changed the title ci: parallelize daily translations by language ci: shard and harden daily translations Jul 14, 2026
@BZ-D
BZ-D marked this pull request as ready for review July 14, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant