[blarghmatey]

What are the relevant tickets?

witan task tk-witan-core-negotiate-the-omnigraph-http-contract-e29072, the go from spike tk-spike-let-witan-core-negotiate-the-omnigraph-htt-3c94fb. Stacked on #573: it merges into feat/omnigraph-0.14-pin, not main, because a 0.13-built client lacks #573's disposition handling and would replay a 409 partial load on a 0.14 server.

Description (What does it do?)

Today a 0.14 witan and a 0.13 data tier refuse each other (400 api_contract_mismatch), and the omnigraph and witan stacks deploy from separate pipelines in no fixed order, so whichever deploys first breaks witan until the other catches up. This makes witan-core speak whichever of 0.14 and 0.13 the server stamps, so the 0.14 cutover needs no pause of witan's CI deploy.

• PooledTransport probes HEAD /healthz on first use and sends the server's stamp when it is in SUPPORTED_HTTP_API_CONTRACTS (0.14, 0.13).
• A 400 api_contract_mismatch stamped with another supported contract switches the transport and resends once, writes included. A second mismatch on the same call stays CONTRACT_VIOLATION.
• RESPONSE_CONTRACT_REQUIRED and the 0.11 tolerance behind it are gone.
• The served suites run once per configured server, and CI installs the 0.13.0 server beside the pinned one.

Implementation details

The evidence for resending a write is from the spike (memory pf-omnigraph-0-13-and-0-14-servers-announce-their-h-229d01). Both servers stamp Omnigraph-Http-Api on every response, the 400 mismatch and the public HEAD /healthz included. require_contract runs after bearer auth and before graph resolution and body work at both tags, and 0.14's disposition lists the mismatch as effects: none.

• The contract is per transport, behind a lock, since shared_transport keeps one transport per server URL per process. Connections stay per thread.
• The probe uses its own short-lived connection with a 5s timeout (PROBE_TIMEOUT_SECONDS), outside the lock, so a server that accepts connections and never answers doesn't queue every thread behind one probe. A failed probe, or one without exactly one supported stamp, is not cached: the request goes out with the pinned contract and classifies its own failure, so a down server still reads as UNAVAILABLE, and the next call probes again.
• The resend goes out on a fresh connection, carrying the stamp the refusing server returned. Run against 0.13.0 and 0.14.0, a small mutate's connection survives a mismatch, but a 6 MB load/ndjson sent with the wrong contract fails at send with a broken pipe. A reused connection failing mid-flight would make a write's resend FATAL, while a fresh one fails at connect, before anything was sent.
• load_ndjson re-probes before every load. Because a wrong-contract load is cut at send, a long-lived process that cached 0.13 would otherwise fail every large load after the data tier moved, until some small call learned the new contract from its 400.
• check-omnigraph-format now also asserts the binary's http-contract equals HTTP_API_CONTRACT. Before this, nothing checked it, so a release that moved the contract without moving the storage format would have passed.
• Probing before the first request matters for load/ndjson, where the server refuses a wrong contract at the middleware without reading the body. A large upload could be cut mid-send and come back with no stamp to learn from.
• _switch_contract only adopts the stamp if the transport is still on the contract the refused request sent. If another thread already moved it, the caller resends with the transport's current contract.
• A WARNING is logged when the transport adopts a contract other than the pinned one, once per change.
• The CLI subprocess path does not negotiate. In the deployed tier it carries nothing to the server: query, mutate, load, export, commit list and branch statements all go over HTTP when remote.
• bin/install_omnigraph_server.py --release previous --dest … fetches the v0.13.0 server and CLI, checked against the digests main pinned for v0.13.0. It refuses to run without --dest, so a second omnigraph never lands on PATH. An import-time assert ties it to SUPPORTED_HTTP_API_CONTRACTS.
• The 0.14 spec's D6 now records the decision, and rollout step 4.4 rolls the images in either order. The ol-infrastructure council probe still pins 0.13 until tk-council-probe-discover-the-omnigraph-http-contra-9c8302 lands.

How can this be tested?

Run against the 0.14.0 release binaries (a scratch HOME whose ~/.local/bin holds them), with OMNIGRAPH_PREVIOUS_SERVER_BIN pointing at a v0.13.0 omnigraph-server that has its omnigraph CLI beside it.

• just test-witan-core: 903 passed, 7 skipped.
• With WITAN_REQUIRE_OMNIGRAPH=1, the served suites pass against both servers (each test id carries omnigraph-0.14.0 or omnigraph-0.13.0): test_group_commit_served.py 6 and 6, test_migrate.py::test_store_merge_into_a_served_graph_never_shells_out 1 and 1, witan-code's test_branches_served.py 3 and 3.
• The full witan-council and witan-code runs fail only test_pre_upgrade_candidates_exclude_the_current_binary, which fails because the scratch HOME leaves two omnigraph binaries on PATH. Two witan-code test_rebuild_refuses_* tests also fail, because the long TMPDIR wraps the phrase they assert. All three pass in the default environment.
• uv run prek run on the changed files, just docs-check, just check-versions, just check-omnigraph-pins and just check-omnigraph-format --binary <0.14.0> pass. Against a stub binary printing http-contract 0.15, check-omnigraph-format exits 1 with the remedy.

The new CI step that installs the 0.13.0 server is only proven by this PR's own witan-tests run.

https://claude.ai/code/session_01FGZebHdoMkhaQ1ewuGV2Ni

[Copilot]

🟡 Changes recommended

The refresh can overwrite a newer contract with a stale probe result and reuse a connection to the replaced server, leaving large uploads vulnerable to broken-pipe failures.

2 open issues

• Prevent stale re-probes from overwriting newer contracts · New
• Discard pooled connections before re-probing and uploading · New

What changed in this PR

This PR lets witan-core negotiate Omnigraph HTTP contracts per server, allowing the witan and data tiers to deploy in either order during the 0.13-to-0.14 cutover.

Changes:

• Adds contract discovery, a single resend after a supported mismatch, and re-probing before bulk loads.
• Runs served tests against both supported releases and checks the pinned binary’s HTTP contract.
• Updates compatibility notes and rollout documentation.

File	Description
testsupport/​omnigraph_server.py	Parameterizes served tests across configured binaries.
packages/​witan-core/​witan_core/​omnigraph_install.py	Updates deployment-order comments.
packages/​witan-core/​witan_core/​omnigraph_http.py	Implements per-server contract negotiation and resends.
packages/​witan-core/​tests/​test_omnigraph_http.py	Tests discovery, resends, refreshes, and shared contract state.
packages/​witan-core/​CHANGELOG.md	Documents negotiation behavior.
mcp/​servers/​witan/​tests/​test_group_commit_served.py	Discovers the contract for commit-count requests.
mcp/​servers/​witan/​tests/​conftest.py	Runs served fixtures against each configured server.
mcp/​servers/​witan/​pyproject.toml	Updates compatibility-floor comments only.
mcp/​servers/​witan/​changelog.d/​20261029_125338_blarghmatey_omnigraph_0_14_pin.md	Clarifies negotiated contract support.
mcp/​servers/​witan-code/​tests/​test_branches_served.py	Parameterizes branch tests across server releases.
mcp/​servers/​witan-code/​pyproject.toml	Updates compatibility-floor comments only.
mcp/​servers/​witan-code/​changelog.d/​20261029_125338_blarghmatey_omnigraph_0_14_pin.md	Clarifies negotiated contract support.
docs/​internals/​design/​omnigraph-0-14-upgrade-spec.md	Records negotiation and revised rollout ordering.
bin/​install_omnigraph_server.py	Adds isolated installation of the previous release.
bin/​check_omnigraph_format.py	Validates the binary’s declared HTTP contract.
.github/​workflows/​witan-tests.yml	Installs and enables the previous server for integration tests.

_{Review effort: Balanced}