Repository navigation
feat(omnigraph): pin 0.14.0, storage format 16 and HTTP contract 0.14 - #573
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Writes with unknown effects can be replayed, the core-version mismatch leaves dependency checks failing, and paths containing spaces can cause incorrect retries.
3 open issues
What changed in this PR
Upgrades Witan’s Omnigraph integration to 0.14.0, aligning the installer, container images and HTTP client with storage format 16 and contract 0.14. This follows the earlier CLI-addressing, patched-S3 and remote-branch work and requires a coordinated data-tier-first cutover.
Changes:
- Updates binary pins, checksums and image labels; enables the patched data-tier CLI.
- Adapts CLI error classification to 0.14 output and exit codes.
- Raises both servers’ core dependency floors and updates compatibility tests and changelog fragments.
| File | Description |
|---|---|
| testsupport/omnigraph_server.py | Uses version-aware cluster targeting in served tests. |
| packages/witan-core/witan_core/omnigraph.py | Handles 0.14 error output, target reports and exit codes. |
| packages/witan-core/witan_core/omnigraph_install.py | Pins 0.14.0 assets and declares storage format 16. |
| packages/witan-core/witan_core/omnigraph_http.py | Selects contract 0.14 and adds an unknown-outcome kind. |
| packages/witan-core/tests/test_omnigraph.py | Covers 0.14 CLI failure classification. |
| packages/witan-core/tests/test_omnigraph_http.py | Updates HTTP contract expectations. |
| packages/witan-core/tests/test_binary_contract.py | Adds format-16 expectations and target-aware upgrade calls. |
| packages/witan-core/changelog.d/20261029_125338_blarghmatey_omnigraph_0_14_pin.md | Records core compatibility and error-handling changes. |
| mcp/servers/witan/tests/test_remote_proxy.py | Explicitly selects the legacy dialect for S3 export coverage. |
| mcp/servers/witan/tests/test_group_commit_served.py | Uses shared HTTP contract constants. |
| mcp/servers/witan/pyproject.toml | Raises the core dependency floor to 0.50. |
| mcp/servers/witan/changelog.d/20261029_125338_blarghmatey_omnigraph_0_14_pin.md | Documents the dependency and image updates. |
| mcp/servers/witan-code/tests/test_store.py | Uses a schema change that still requires bridge recreation. |
| mcp/servers/witan-code/pyproject.toml | Raises the code server’s core dependency floor to 0.50. |
| mcp/servers/witan-code/changelog.d/20261029_125338_blarghmatey_omnigraph_0_14_pin.md | Documents the new compatibility floor. |
| docker/witan.Dockerfile | Updates CLI pins, checksums and storage-format label. |
| docker/omnigraph-server.Dockerfile | Updates server pins and selects the patched CLI. |
| docker/omnigraph_server_entrypoint.py | Updates documentation for the default 0.14 image. |
| .github/workflows/omnigraph-cli-patch.yml | Updates comments describing patched-image builds. |
Review effort: Balanced
The installer and both images pinned omnigraph 0.13.0 (format 14, HTTP contract 0.13). This moves all three to the released v0.14.0 assets. The binary reports "internal-schema 16 (serves v14 to v16)" and http-contract 0.14, so `_OMNIGRAPH_INTERNAL_SCHEMA` moves to 16 (the release notes say 15) and `HTTP_API_CONTRACT` to 0.14. The data-tier image selects the patched CLI, which `check-omnigraph-pins` requires once the release tag equals the patch base. Both servers' witan-core floors move to 0.50. Two client changes the 0.14 CLI needs: - 0.14 prints the resolved target on stderr for every call, so stderr is never empty, and `_cli_failure` classified stderr whenever it had text. Every read's error object on stdout (recovery barrier, conflict, unreachable server) was then classified from the target line alone and reported as fatal. The error object now wins whenever there is one. - `_classify_cli_error` reads the new exits: 2 (partial or unknown effects) is never retried and a write raises WriteIndeterminate, 64 (a refused command line) is fatal, and 75 from a 503 gets the unavailable-server budget rather than the admission cap's. Tests: a format-16 row in the binary contract table (measured on 0.14.0, same as 14's), `upgrade` addressed by `--target`, the served-server helper applies and unlocks with 0.14's `--target`, the bridge-recreate test uses a type change 0.14 still refuses in place (it now drops an indexed property in place), and the s3 export test pins a pre-0.14 CLI. The first write a 0.14 binary makes to a format-14 graph stamps it 16, and 0.13 binaries then refuse it. This merges only in the cutover window, with the data tier moving before the witan clients. Claude-Session: https://claude.ai/code/session_01FGZebFvLxbZTiKey4rCL6B
…fication 0.14 prints `target <name> (<provenance>): ...` and `target write scope: branch=<name>` on stderr before every command, and 0.13's `--quiet` that suppressed them is gone. Those lines carry the graph id, branch and paths, so a marker without spaces (`recovery_required`, `too_many_requests`) could match a name and misroute an ordinary failure. The message keeps them; only the classified text drops them. Claude-Session: https://claude.ai/code/session_01FGZebFvLxbZTiKey4rCL6B
The witan-policy CI job installs the pinned omnigraph, so with the pin at 0.14.0 policy/check.sh failed at its first cluster command with "target_required". 0.14 takes the cluster root as --target on cluster apply (--config is only the source) and replaced --cluster with --target on policy validate and policy test. Claude-Session: https://claude.ai/code/session_01FGZebFvLxbZTiKey4rCL6B
Both servers' floors move to witan-core>=0.50 in this PR, so check-core-floor needs the workspace at 0.50.0. Collects the 0.14 pin fragment and #572's StoreQuarantined removal. Claude-Session: https://claude.ai/code/session_01FGZebFvLxbZTiKey4rCL6B
d95aa41 to
47a9bdf
Compare
…ects landed 0.14 answers a /load whose fork published before its rows were refused with HTTP 409 and disposition effects "partial". The 409 rule classified that RETRYABLE, so the retry loop could replay a write that had already changed the graph. A retried kind (409/429/503) on a non-idempotent request now becomes OUTCOME_UNKNOWN when the disposition says partial or unknown, and the write raises WriteIndeterminate. FATAL responses are left alone, because 0.14 reports "unknown" for every owned write that isn't a typed refusal (a plain 400 included) and they aren't retried anyway. The 0.14 target-report regex matched the label as \S+, so an inline path with a space left the line in the classified text, where a marker such as too_many_requests could turn a precondition failure into a retried admission cap. It now matches up to the provenance delimiter. Claude-Session: https://claude.ai/code/session_01FGZebFvLxbZTiKey4rCL6B
feoh
left a comment
There was a problem hiding this comment.
Code looks right. I checked the exit-code mapping against upstream exit_for at v0.14.0, and the digests against the release API. Two merge-time problems are inline. Minor items:
- The body is out of date: the witan-core bump is already done (
47a9bdf), andf391196fixes the deferred "HTTP still retries every 409" low, sotk-classify-omnigraph-failures-from-the-0-14-dispos-2ae3camay be closable. - Unchanged code, but it bites now:
_download_omnigraphreplaces any non-pinned binary, so an older witan'ssetupputs 0.13 back over 0.14. That build then refuses local stores already stamped 16. Recoverable fromomnigraph-0.14.0, but there's no warning.
ol-infrastructure's preview refuses an image whose internal-schema label differs from the stack's served format, so the 16 label fails every stack still on fmt14 even though 0.14 opens format 14 in place. This declares the low end of `omnigraph version`'s "serves v14 to v16" as _OMNIGRAPH_MIN_INTERNAL_SCHEMA and stamps it on both images as edu.mit.ol.omnigraph.min-internal-schema, so the guard can accept any served root inside the range. check-omnigraph-format compares it against the binary and check-omnigraph-pins keeps the three declarations and the runtime-stage labels in step. Raised in review: #573 (comment) Claude-Session: https://claude.ai/code/session_01FGZebHdoMkhaQ1ewuGV2Ni
A version line whose "serves" clause isn't in the "vN to vM" shape read as serving only the written format, so check-omnigraph-format advised raising the min label to it, which would make ol-infrastructure refuse every fmt14 root. It now raises instead. check-omnigraph-pins also refuses a min above the written format, and the format check's remedy says to update the min in the same change. Claude-Session: https://claude.ai/code/session_01FGZebHdoMkhaQ1ewuGV2Ni
|
@feoh On the minor items: the body is updated. |
Both servers now pass `actor=` to OmnigraphClient, a keyword this change adds to witan-core. With the floors at 0.50 an external install could resolve 0.50.0, which imports cleanly and then raises TypeError on the first deployed request or ingest. The floor checker accepts an unpublished floor only when it equals the workspace version, so the bump and the floors move together, as in #573. 0.51.0 is a minor bump because it also removes reported_internal_schema() and reported_internal_schema_range() (#583). Claude-Session: https://claude.ai/code/session_01FGZebE9nnZy6bBL3eprDdQ
* feat(omnigraph): assert the actor on per-actor requests witan resolves a different bearer token per actor on every request (ADR-0004), and a mapping that hands out the wrong token writes under someone else's name. HTTP contract 0.14 lets the client say which actor it expects (Omnigraph-If-Actor), and the server refuses a mismatch with 412 actor_mismatch before graph lookup or any effect. OmnigraphClient takes the actor its token belongs to. The pooled transport sends it on every request, and a write over the 0.14 CLI passes --as, which the CLI sends as the same header. witan's per-actor client and witan-code's ingest client set it. A 0.13.0 server ignores the header, so it goes out whatever contract the transport speaks. ACTOR_MISMATCH is its own kind because it shares 412 with a failed precondition and means the opposite thing for a CAS caller: re-reading does not fix a wrong token, so it is terminal and never a lost race, and group commit fails the whole group rather than the fenced member. Task: tk-send-omnigraph-if-actor-on-per-actor-requests-so-f4fc67 Claude-Session: https://claude.ai/code/session_01FGZebE9nnZy6bBL3eprDdQ * fix(witan-code): pass actor to witan-core only when there is one witan-code's OmnigraphClient forwarded actor= to the base class on every construction, so against the published witan-core 0.50.0, which has no such keyword, every witan-code client raised TypeError, local use included. It now leaves the keyword off without an actor, so only a deployment's per-actor ingest client needs the witan-core release that adds it. Claude-Session: https://claude.ai/code/session_01FGZebE9nnZy6bBL3eprDdQ * test(witan): make the served actor test refuse an actual write task_create runs a duplicate search first, and a 0.14 server checks the Omnigraph-If-Actor assertion on reads too, so over HTTP the refusal came from the search and the test passed without attempting the write. It now stubs the search and records change_many, so both transports have to refuse the mutation itself. Claude-Session: https://claude.ai/code/session_01FGZebE9nnZy6bBL3eprDdQ * version: witan-core 0.51.0, raise both server floors to it Both servers now pass `actor=` to OmnigraphClient, a keyword this change adds to witan-core. With the floors at 0.50 an external install could resolve 0.50.0, which imports cleanly and then raises TypeError on the first deployed request or ingest. The floor checker accepts an unpublished floor only when it equals the workspace version, so the bump and the floors move together, as in #573. 0.51.0 is a minor bump because it also removes reported_internal_schema() and reported_internal_schema_range() (#583). Claude-Session: https://claude.ai/code/session_01FGZebE9nnZy6bBL3eprDdQ


What are the relevant tickets?
witan task
tk-bump-the-omnigraph-pin-to-0-14-contract-0-14-sto-e7e5c7, the pin-bump step of the omnigraph 0.14 upgrade (spec indocs/internals/design/omnigraph-0-14-upgrade-spec.md, rollout step 3). Follows #564, #568, #569 and #571.Description (What does it do?)
The installer and both images pin omnigraph 0.13.0 (storage format 14, HTTP contract 0.13). This moves all three to the released v0.14.0 assets (tag commit
60bfd19). The digests were hashed locally and matched against each published.sha256and the release API.omnigraph versionon the binary readsinternal-schema 16 (serves v14 to v16)andhttp-contract 0.14, so_OMNIGRAPH_INTERNAL_SCHEMAand both image labels move 14 to 16 (the release notes say 15) andHTTP_API_CONTRACTmoves to0.14. The data-tier image now selects the patched CLI from #569 (OMNIGRAPH_CLI_SOURCE=patched), whichcheck-omnigraph-pinsrequires once the release tag equals the patch base. The witan image andwitan setupkeep the stock release binary.Both images also carry a new label,
edu.mit.ol.omnigraph.min-internal-schema(14). It's the low end of that served range, declared as_OMNIGRAPH_MIN_INTERNAL_SCHEMAand checked against the binary bycheck-omnigraph-formatand across all three declarations bycheck-omnigraph-pins. ol-infrastructure's preview guard refuses any image whoseinternal-schemalabel differs from the stack's served format, so the 16 label alone fails every stack still onfmt14. mitodl/ol-infrastructure#6801 makes the guard accept a served format anywhere in the image's range.Merging this is one-way for any graph once 0.14 publishes to it. 0.14 opens a format-14 graph in place, but the first write a 0.14 binary makes stamps that branch 16, and every 0.13 binary refuses it from then on. So this has to merge in the cutover window, and there is no rollback short of restoring the pre-cutover backup.
Merging also auto-deploys CI's
pulumi-omnigraphandpulumi-witanin no fixed order, and publishes witan-core 0.50.0 to PyPI. A 0.13 server and a pinned 0.14 client refuse each other's contract header, but #580 (stacked on this branch) makes witan-core negotiate the contract per server, 0.14 or 0.13, so the two stacks can deploy in either order and the merge doesn't need witan's CI deploy paused. The ol-infrastructure council probe pins0.13until mitodl/ol-infrastructure#6811, which makes it discover the contract the same way, is deployed. The published witan-council 0.46.0 (witan-core>=0.48,<1) and witan-code 0.27.1 (>=0.46,<1) resolve 0.50.0 on a fresh install or upgrade. That doesn't put the 0.14 contract on any deployed data tier: omnigraph-server is ClusterIP-only with no HTTPRoute (ol-infrastructureapplications/witan/ci_indexer.py), so users reach it only through the deployed witan MCP tier, whose image builds from this workspace. What it does change is local:witan setupinstalls the 0.14 binary, which stamps a developer's local stores 16 on their first write.Two client changes ride along, because the 0.14 CLI breaks the existing failure handling:
target <store> (inline): direct graph …), so stderr is never empty._cli_failureclassified stderr whenever it had text, so every read's JSON error object on stdout (a recovery barrier, a conflict, an unreachable server) was classified from the target line alone and reported as fatal. This is what failedtest_apply_schema_upgrades_an_old_store. The error object now wins whenever there is one._classify_cli_errorreads 0.14's exits (command_outcome.rsexit_forat v0.14.0). 2 (partial or unknown effects) is never retried, and a write raisesWriteIndeterminate; it is checked before the retryable markers so a dropped connection can't turn it into a retry. 64 (a refused command line) is fatal. 75 now covers any refusal witheffects: none, action: retry, so one that carries a 503 gets the unavailable-server budget rather than the admission cap's.target <name> (<provenance>): ...,target write scope: branch=<name>) are kept in the message but left out of the classified text. They carry graph ids, branches and paths, and 0.13's--quietthat suppressed them is gone.The 0.13-worded markers needed no change: 0.14 prints the same sentences with
0.14in place of0.13, checked on the binary and ingraph_http.rs. witan-core is bumped to 0.50.0 here, and both servers' witan-core floors move to>=0.50for the new contract.A write whose HTTP error carries a 0.14 disposition with
effectspartialorunknownis nowOUTCOME_UNKNOWNand raisesWriteIndeterminateinstead of being retried. 0.14 answers a load whose fork published before its rows were refused with a 409 that sayspartial, which the 409 rule would otherwise have replayed. A FATAL response is left alone: 0.14 reportsunknownfor every owned write that isn't a typed refusal, a plain 400 included, and group commit replays a FATAL composed group one write at a time. Whether that replay is safe stays withtk-classify-omnigraph-failures-from-the-0-14-dispos-2ae3ca.Test changes for 0.14:
test_binary_contract.pygets a format-16 row, measured on 0.14.0 (every surface matches format 14), and addressesupgradewith--targetas witan does.testsupport/omnigraph_server.pyapplies and force-unlocks with 0.14's--target, so the served tests run against a 0.14 server with the 0.14 contract.String?toStringchange instead, which both 0.13.0 and 0.14.0 refuse in place (OG-MF-106on 0.14.0; the test passes against both binaries).s3://store and witan-core already tests that refusal.How can this be tested?
Run against the 0.14.0 binaries (a scratch
HOMEwhose~/.local/binholds the release'somnigraphandomnigraph-server,TMPDIRon disk):just test-all: witan-core 888 passed, 7 skipped; witan-council 1801 passed, 4 skipped, 1 failed; witan-code 711 passed, 1 failed; agent-config-kit 564 passed; ol-agent-kit 1 passed. The two failures are harness-only:test_pre_upgrade_candidates_exclude_the_current_binary(twoomnigraphbinaries on PATH under the scratch HOME) andtest_rebuild_refuses_a_file_path_with_real_git(a longTMPDIRwraps the asserted phrase). Both pass in the default environment. The served group-commit and branch-view tests ran against a real 0.14 server.witan-code/tests/test_store.pyalso pass against the 0.13.0 binary.just check-omnigraph-pins,just check-omnigraph-format,just check-versions,just docs-checkanduv run prek run --all-filespass.just check-core-floorpasses against the bumped workspace witan-core.just check-omnigraph-format --binary <0.14.0>reports "opens 14 to 16 in place, as declared", andtest_omnigraph_install.pycovers the 0.14 and 0.11 version lines. After the label commit, witan-core's suite was re-run with the 0.13.0 binary on PATH: 880 passed, plus one expected failure (test_the_binary_on_path_is_the_declared_format, which needs 0.14).check-omnigraph-pins,check-versions,docs-checkand prek on the changed files pass too.Not verified here: the Concourse image builds (the patched CLI stage was qualified by
omnigraph-cli-patch.ymlin #569), and anything against a deployed data tier. The cutover itself istk-cut-over-ci-qa-then-production-to-omnigraph-0-14-be7663.Checklist:
fmt14tk-spike-let-witan-core-negotiate-the-omnigraph-htt-3c94fbsettled: gohttps://claude.ai/code/session_01FGZebFvLxbZTiKey4rCL6B