Cross-SDK and load scenarios

Interop tests of the Java SDK against the TypeScript, Rust, Python and Kotlin ACP SDKs over stdio, Streamable HTTP and WebSocket, plus load runs of the Java listener. Each scenario starts real processes in several languages, checks what they print, and tears them all down.

This directory is not a Maven module and the root pom.xml does not reference it, so ./mvnw verify and the release build never run it. It runs locally with the scripts below, and in CI through .github/workflows/cross-sdk.yml (per peer; manual dispatch plus nightly), .github/workflows/interop-java.yml (the Java<->Java cells on every push and pull request) and .github/workflows/framework-smoke.yml (the framework smoke matrix, the release gate: every push to main, nightly, and on dispatch for any branch).

There are four kinds of scenarios:

Generated cells, configs/x-<client>-<agent>-<transport>[-unstable].json: one feature-driven agent and one client program per language, a shared step catalogue (steps.json), and configs generated from matrix.json by GenConfigs.java. The Contracts section is everything a language package needs to take part.
Conformance scenarios, configs/conf-java-*.json: the raw JSON-RPC driver (programs/raw) against the Java agent and client, generated by programs/raw/gen_conf.py.
Framework smoke cells, configs/fw-*.json and configs/load-10-fw-*.json: the SDK hosted by Spring Boot (servlet and WebFlux), Micronaut and Quarkus against the TypeScript SDK, generated from smoke.json by GenConfigs.java (Framework smoke matrix).
Load scenarios, configs/load-*.json, hand-written (except load-10-fw-*).

Prerequisites

Tool	Version	Used for
JDK (with `jcmd`)	17+	the SDK, the Java programs, the runner, heap and thread samples
JDK	21	the Kotlin SDK's Gradle toolchain (Kotlin cells only)
JBang	0.135+	`RunScenario.java`, `GenConfigs.java`
Node.js and npm	20	the TypeScript SDK and its programs
Rust (`cargo`)	stable	the Rust SDK and its programs
Python	3.12 (`python3 -m venv`)	the Python SDK (plus Hypercorn) and its programs
git	any	cloning the peer SDKs

run-all.sh checks only the tools the selected scenarios need. The first run clones the peer SDKs it needs from GitHub and builds them (npm, cargo, a venv, Gradle), which takes a few minutes. Later runs reuse the checkouts in .cache/ and rebuild a peer only when its ref resolves to a new commit.

Running

Every scenario tests this working tree: the SDK is installed from the checkout (./mvnw -DskipTests install) before the Java programs are built against it.

One scenario (from integration-testing/):

cd integration-testing
jbang RunScenario.java x-java-java-stdio            # installs the SDK first
jbang RunScenario.java load-300 --skip-sdk-install  # reuse the SDK already installed
jbang RunScenario.java --list

A set of scenarios, with a pass/fail table at the end (exits non-zero on any unexpected failure):

integration-testing/scripts/run-all.sh                                     # the default set
integration-testing/scripts/run-all.sh --only x-java-java-http,load-50     # names
integration-testing/scripts/run-all.sh --only 'x-*-python-*,x-python-*'    # globs
integration-testing/scripts/run-all.sh --tag python                        # name tokens
integration-testing/scripts/run-all.sh --tag stdio --exclude 'x-java-*' --list

--only takes comma-separated names or globs; --tag takes comma-separated tags, where a scenario's tags are the dash-separated tokens of its name (x-java-python-stdio has x, java, python, stdio). Both together select the union; --exclude (globs) removes from it. --list prints the selection and exits.
With no --only/--tag, every config runs except self-*, smoke-* and *-unstable. Those run only when a pattern or tag targets them (--only 'self-*', --only '*-unstable', --tag self, --tag unstable) or a name selects one exactly.
An empty selection fails unless --allow-empty is given; a name with no config is an error.
Parallel by default. run-all.sh installs the SDK once, then prepares every peer checkout and program build the selection needs, once and serially (RunScenario --prepare). It then runs the scenarios --jobs at a time (default: half the CPUs, at most 8; --jobs 1 for one at a time), each with --prepared: no git, no build, its own logs/<scenario>/, its own temp directory and its own port range. load-* scenarios run afterwards, alone and serially, since they measure throughput. With more than one job each scenario's console goes to logs/run-all/<scenario>.console.log and a line is printed as each finishes; the table at the end is the same. --prepare-only stops after the preparation (CI warms its caches with it).

Against a tag or other ref of the peer SDKs (the default, from peers.json, is main):

# every peer at the same ref
integration-testing/scripts/run-all.sh --peers-ref v1.0.0
# one peer at a tag, the others at main
integration-testing/scripts/run-all.sh --peer typescript-sdk=v0.5.0
jbang RunScenario.java x-java-python-http --peer python-sdk=1.0.0rc2

A ref can be a branch, a tag or a commit SHA. Each ref gets its own checkout, .cache/peers/<name>@<ref>, so switching refs does not throw away a build.

Logs for each run go to logs/<scenario>/: one file per process (agent.log, server.log, client.log, clients.log), per build (build-*.log, peer-*.log) and result.txt. run-all.sh also writes logs/run-all/summary.txt. CI uploads the whole logs/ directory as an artifact.

Scenarios

Scenario	What runs
`x-<client>-<agent>-<transport>`	generated: the client program of one language against the agent program of another over one transport, running every stable catalogue step (`steps.json`). Pairs are Java<->Java and Java<->X for X in TypeScript, Rust, Python and Kotlin; transports are stdio, Streamable HTTP and WebSocket (Kotlin: stdio and WebSocket)
`x-<client>-<agent>-<transport>-unstable`	the same pair running the `unstable` steps (`session.fork`); on demand only
`conf-java-agent-<t>`, `conf-java-client-<t>`, `conf-java-client-catalogue-<t>`	the raw driver probing the Java agent, the Java client's raw mode, and the Java client running catalogue steps against the raw agent (programs/raw/gen_conf.py)
`quarkus-typescript-http`, `quarkus-typescript-ws`	hand-written: the TypeScript client program against the Quarkus-hosted smoke agent (`programs/quarkus`: the shared `programs/framework` handlers as an `@AcpAgent` bean served by `acp-quarkus` on the Quarkus HTTP server), running the catalogue steps that agent implements (initialize, sessions, echo chunks, stop reasons, -32601, 1 MB and 8 MB prompts and updates). Run in the `typescript-http`/`typescript-ws` legs of `cross-sdk.yml`
`load-50`, `load-300`, `load-1000`	N Java clients on their own connections, each: initialize, session/new, then 10 (or 5) prompts streaming two updates each
`load-shared-300`	300 clients sharing one HttpClient, so one HTTP/2 connection
`fw-<framework>-agent-<t>`, `fw-<framework>-client-<t>`, `load-10-fw-<framework>`	generated from `smoke.json`: the framework smoke matrix

Over Streamable HTTP the cells also assert, through matrix.json facts, what the retired hand-written interop-* scenarios checked: the negotiated HTTP version on both sides (h2c between the Java client and the Java and Python agents, HTTP/1.1 everywhere else), and the DELETE on close answered 202. http.reconnect, in every HTTP cell, loads and prompts a session on a new connection.

Load thresholds (first baselines, from the 2026-09-25 pre-release run): errors == 0, every prompt answered (ok == expected), every update delivered (updates == 2*ok), at most 4 SDK threads (acp-*) on the server at peak and after close, server heap after a full GC at most 256 MB at peak and 64 MB after close. p50/p99 latency is recorded in the results and never asserted. The load scenarios measure the server, so the load generator's client readers use platform threads (virtualThreads(false)) on every JDK; the SDK client's default stays virtual threads on JDK 21 and later.

Framework smoke matrix

The release gate for the framework integrations: the SDK as an application uses it, hosted by Spring Boot (acp-spring-boot-starter), Micronaut (acp-micronaut) and Quarkus (acp-quarkus), against the TypeScript SDK. A release candidate must have a green framework smoke matrix on the commit being released (framework-smoke.yml, or the local command below), next to the usual ./mvnw clean verify; the full cross-SDK matrix (cross-sdk.yml) should be green on it too.

smoke.json defines it once; GenConfigs.java writes its configs (and --check keeps them in step):

agentSteps, run by the TypeScript client against each framework-hosted agent on each of that framework's transports (fw-<framework>-agent-<t>): init.initialize (with the agentInfo the @AcpAgent annotation declares), init.agent-capabilities (capabilities derived from the handler annotations: the agent has no @Initialize method), init.agent-info, session.new, session.load, session.multi, update.agent_message_chunk, perm.selected, elicit.form, cancel.prompt, ext.client-request (an @ExtRequest method), error.method-not-found, http.reconnect (HTTP only), stdio.eof-exit (stdio only) and conn.close: 14 steps per cell, 13 over WebSocket. The agent-side assertions (STEP agent.perm.selected, agent.elicit.form, agent.cancel.prompt) are required, and the agent must log nothing at WARN or ERROR.
clientSteps, run by the client bean each framework builds from its own configuration (transport URI and advertised capabilities; the handlers come from an AcpClientCustomizer bean) against the TypeScript agent (fw-<framework>-client-<t>): init.initialize, init.agent-info, session.new, update.agent_message_chunk, perm.selected, fs.read, elicit.form, cancel.prompt, ext.client-request, conn.close.
load: 10 Java load clients x 10 prompts against each framework-hosted agent over HTTP (load-10-fw-<framework>): no errors, every prompt and update delivered, clean close, no ERROR on the server. Latency is recorded, not asserted.

Framework	Agent transports	Client	Program
Spring Boot 4.1	stdio (`spring.main.keep-alive=true`), HTTP and WebSocket (the SDK servlet, `acp-http-servlet`, on `server.port` in a servlet web application)	HTTP	programs/spring
Spring Boot 4.1, WebFlux (`spring-webflux`)	HTTP and WebSocket (the SDK's WebFlux host, `acp-http-webflux`, routed on `server.port` by Reactor Netty in a reactive web application)	none	programs/spring, `--web reactive`
Micronaut 4	stdio, HTTP, WebSocket (the SDK listener)	HTTP	programs/micronaut
Quarkus	stdio, HTTP, WebSocket (the Quarkus HTTP server); one package per build-time transport	HTTP	programs/quarkus

The agent handlers (programs/framework/.../SmokeAgent.java) and the client steps (SmokeClient.java) are shared sources that every framework program compiles in; each framework adds only an @AcpAgent subclass with its own bean annotation and a main class. A framework entry in smoke.json may reuse another's program with agentArgs (flags its agent.sh gets before --transport, as spring-webflux runs programs/spring with --web reactive) and agentName (the @AcpAgent name the init.initialize step expects, interop-<agentName>-agent). To run the whole matrix, locally or against another branch:

git checkout <branch>                                  # the candidate under test
integration-testing/scripts/run-all.sh --tag fw        # every fw-* and load-10-fw-* scenario
integration-testing/scripts/run-all.sh --only 'fw-spring-*,load-10-fw-spring'   # one framework
gh workflow run framework-smoke.yml --ref <branch>     # the same in CI, on that branch

It needs JDK 17+, Node 20 and JBang (the TypeScript SDK is cloned and built on the first run); on a warm machine it takes about 3 minutes.

Contracts

What a language package (an agent and a client program for one SDK) must implement to take part in the generated cells. Nothing here needs the runner's source. The step catalogue, steps.json, is the normative list of steps; this section defines everything around it.

1. Files a language package owns

programs/<lang>/launch/build.sh    build the programs (may do nothing)
programs/<lang>/launch/agent.sh    exec the agent
programs/<lang>/launch/client.sh   exec the client
programs/<lang>/...                sources, project files
expectations/<lang>.json           the package's expected failures (expectations/README.md)

<lang> directories are those of matrix.json: java, node (typescript), rust, python, kotlin. Do not name a directory bin/: the repository .gitignore ignores it.

The three scripts are bash, executable, and work from any working directory (resolve paths from $(dirname "${BASH_SOURCE[0]}")). agent.sh and client.sh end in exec <program> "$@", so the process the runner (or a stdio client) starts is the program itself and a kill reaches it; they print nothing themselves. They read the environment below and nothing else.

Variable	Set for	Value
`MVNW`, `ACP_VERSION`, `CACHE`, `IT_ROOT`	`build.sh`	the Maven wrapper (pinned to `IT_M2_REPO`), the SDK version under test, `integration-testing/.cache`, `integration-testing/`
the language's `env` in `matrix.json`	`build.sh`, and every process of a cell that uses the language	e.g. `TS_SDK`, `PY_SDK`, `RUST_SDK` + `CARGO_TARGET_DIR`, `KOTLIN_SDK`: the peer checkout (and build dir) for the ref under test
`STEPS`	client	comma-separated step ids, in order
`AGENT_CMD`	client, stdio only	a shell command line that starts the agent
`STEP_TIMEOUT_MS`	client, optional	per-step timeout, default 15000

In a stdio cell both languages' variables are on the client process, and the client's environment is inherited by the agent it spawns. The build runs once per cell before any process starts (build.sh of both languages in a stdio cell); it must be idempotent and fast when nothing changed.

2. Command lines and the transport flag

agent.sh  --transport stdio
agent.sh  --transport http --port <port>
agent.sh  --transport ws   --port <port>
client.sh --transport stdio                       (AGENT_CMD in the environment)
client.sh --transport http --url http://127.0.0.1:<port>/acp
client.sh --transport ws   --url ws://127.0.0.1:<port>/acp

Agent, http/ws. Listen on 127.0.0.1:<port> (0 means any), serve ACP at /acp, and print READY <port> on stdout once accepting connections. ws is a GET /acp with Upgrade: websocket on that endpoint; an SDK whose one server handles both (Java, TypeScript, Rust, Python) may serve both transports for either flag. Run until killed (SIGTERM, then SIGKILL after 5 s). Optionally log one [http] <METHOD> <path> <protocol> -> <status> ... line per request, which matrix.json facts can assert on.
Agent, stdio. Newline-delimited JSON-RPC on stdin/stdout. Stdout carries nothing else: every log line, diagnostic and STEP agent.* line goes to stderr (logging frameworks included). Exit when stdin reaches EOF (any exit code), or on SIGTERM.
Client, stdio. Spawn the agent as bash -c "exec $AGENT_CMD" with the client's environment, using the SDK's own process transport where it has one (Java StdioAcpClientTransport, Rust AcpAgent, Python spawn_agent_process; TypeScript child_process.spawn + ndJsonStream; Kotlin the sample's createProcessStdioTransport). Relay the child's stderr to the client's stdout, line by line: a line that starts with STEP verbatim, every other line prefixed agent| . The runner's STEP matcher is anchored at the start of the line, so this is how the agent's assertions reach it in a stdio cell. End the child when the run ends.
An unknown flag or a missing required value: print usage to stderr and exit 2.

3. What the programs print

Client, on stdout, one line per step in STEPS order, then one RESULT line, then exit 0 (whatever the step outcomes; a non-zero exit means the program itself broke):

STEP <id> PASS (<ms> ms) -> <detail>
STEP <id> FAIL (<ms> ms) -> <detail>
RESULT pass=<n> fail=<n> updates_total=<n> upd_<sessionUpdate>=<n> ...

<detail> is one line. A step that timed out has TIMEOUT in its detail; the runner reports their count as a hang count, a distinct kind of finding.
A step id the program does not implement prints STEP <id> FAIL (0 ms) -> unknown-step, so a program that lags the catalogue shows up rather than passing silently.
pass and fail count the client's own steps only. updates_total counts every session/update received on any connection, and upd_<kind> the ones of each sessionUpdate value (upd_agent_message_chunk=8); unknown kinds count as upd_other.
Other output (updates, requests, logs) is free-form, but must not start with STEP or RESULT .

Agent: its own assertions, on stderr, never RESULT:

STEP agent.<id> PASS (<ms> ms) -> <detail>
STEP agent.<id> FAIL (<ms> ms) -> <detail>

An agent prints STEP agent.<id> exactly when it handles that step's directive, for the steps whose pass.agent is set in steps.json. The generated config requires the PASS line (on the agent process for HTTP/WS, on the relaying client for stdio), and any undeclared FAIL fails the scenario.

4. The step catalogue, `steps.json`

conventions: the step timeout (15 s), the grace for updates that trail a response (1 s) and for load replay (2 s), placeholders, and the mandatory first (init.initialize) and last (conn.close) steps, which every cell runs.
fixtures: the exact values both sides use: the client and agent initialize profiles, modes, config options, permission options and the client's selection rule, #emit payloads, file contents, extension method names, _meta, elicitation schemas. Programs hard-code these.
directives: what the agent does for each prompt text. The agent keeps no script: the client drives it through the prompt. If the first text block of a prompt starts with #, it is #<name> <args> (single spaces; for #fs write and #len the last argument is the rest of the line); otherwise it is plain text, answered with the chunks "echo: " and the text. An unknown directive is answered with a JSON-RPC error -32602, message unknown directive: #<name>.
steps, in run order. Each has id, group, client (what the client does), pass.client (when its STEP <id> passes) and optionally prompt (the prompt text it sends), agent and pass.agent (the agent behaviour and its STEP agent.<id> rule), requires (capabilities the step depends on: client.<path> in the client's initialize, agent.<path> in the agent's), transports (default all three), only ({"client": [...], "agent": [...]}: languages the step applies to, e.g. Java SDK policy), profile (stable default, or unstable), phaseB (the open Java Phase B item that blocks it, exactly the steps with see: "P<n>" entries in expectations/java.json; the item's commit removes both; informative) and seed (implemented by the WP0 Java seed).

Rules for every step:

Self-contained. A step opens its own session (or connection) when it needs state; one failure must not cascade. Only init.initialize opens the main connection, and only conn.close closes it.
{dir} in a prompt is the client's scratch directory: absolute, created once per run, no spaces. Agent and client always run on the same host.
A program advertises in initialize only what it really implements, taking the values from fixtures.client / fixtures.agent. An agent obeys the client's capabilities: when a directive needs one the client did not advertise, it prints STEP agent.<id> FAIL and answers without calling the client.
Updates may trail the prompt response (nothing orders their dispatch on HTTP): wait up to updateGraceMs before failing on a missing update, replayGraceMs for a load replay, and accept the replay on either side of the response.

Step ids are stable. A new behaviour is a new id; a changed pass rule is a contract change that every package must follow, so it goes through review like an API change.

5. Cells, `matrix.json` and `GenConfigs.java`

A cell is one generated config, configs/x-<client>-<agent>-<transport>[-unstable].json:

pairs are J<->J, J->X and X->J for X in TypeScript, Rust, Python and Kotlin;
transports are those both languages support (Kotlin has no Streamable HTTP: stdio and ws);
the profile is stable, or unstable (the unstable steps only, plus the mandatory first and last; suffix -unstable, run on demand).

matrix.json per language: enabled (cells are generated only when both languages are enabled; all five are), dir, peers (from peers.json), env, transports, readyTimeoutSec. stepSelection is all (seed, only the "seed": true steps, or a comma-separated id list also work). facts add transport assertions to the cells their when matches, with roles client and agent mapped to processes: the HTTP version each HTTP pair negotiates (h2c between the Java client and the Java and Python agents, HTTP/1.1 otherwise, on both sides), DELETE -> 202 on every agent, the WebSocket 101 on the Java agent.

Each cell is an ordinary runner config: on HTTP/WS an agent process (role server, ready on READY) and a client process; on stdio a single client process with AGENT_CMD. The checks are client RESULT pass == <steps - expected client failures> and client RESULT fail == <expected client failures>, plus the required agent STEP lines, the facts and the expected failures from expectations/*.json (expectations/README.md). RunScenario, Checks, Scenario and Proc are unchanged.

cd integration-testing
jbang GenConfigs.java              # write the cells of the enabled languages (commit them)
jbang GenConfigs.java --check      # CI: exit 1 if a committed cell is stale, missing or extra
jbang GenConfigs.java --list       # print cells and step counts, write nothing
# local development of a language package: its cells and self-pairs, every catalogue step
jbang GenConfigs.java --enable python --self --steps all
scripts/run-all.sh --only 'self-python-*'
scripts/run-all.sh --only 'x-*-python-*,x-python-*'

--self writes self-<lang>-<transport> cells (the language against itself: develop a package before the Java side is ready, and separate a peer SDK bug from a Java one); they are git-ignored and never part of the nightly. Language packages never commit generated configs: only the integration step regenerates and commits them, after enabling the languages. --check reports locally generated extra cells as "extra", which is the reminder.

6. The Java programs and the expectations

The Java programs, programs/java (interop.Agent, interop.Client), are every package's partner: every cell has a Java client or a Java agent. They implement the whole catalogue on all three transports and advertise the full fixtures.client / fixtures.agent profiles. The seed flag marks the eight steps the first Java seed implemented; matrix.json now selects every step ("stepSelection": "all"), and --steps seed still generates the seed subset.

A step that fails in a cell is either fixed or declared in exactly one expectations file (expectations/README.md), chosen by whose code has to change:

A Java gap goes in expectations/java.json, with when.client: java or when.agent: java and see: "P<n>", the open Phase B item that closes it. The step's phaseB in steps.json names the same item. Today there is none: every Phase B item the catalogue depends on has landed, and java.json is empty.
A peer gap goes in the peer's own file (typescript.json, rust.json, python.json, kotlin.json), with see: "peer:<sdk> <file:line>" in that SDK, or see: "spec:<where>" when the RFD or schema contradicts itself and the peer and Java read it differently.
A spec disagreement that only a Java pair shows goes in java.json with see: "spec:..." (none today).

The Kotlin package has stdio and WebSocket only: the Kotlin SDK has no Streamable HTTP transport, so matrix.json lists "transports": ["stdio", "ws"] for it and no x-*-kotlin-http cell exists. Kotlin's Gradle build needs JDK 21 while everything else stays on 17.

expectations/raw.json belongs to the raw driver, which is no matrix language: its expectations array stays empty (GenConfigs.java reads it like every other file), and its conformance array holds the expected failures of the conf-java-* scenarios, which only programs/raw/gen_conf.py reads (its header documents the fields: case, when.side, when.transport, contains, see, reason). gen_conf.py --check runs next to GenConfigs.java --check in CI.

7. Isolation: every scenario must be safe to run next to any other

run-all.sh runs scenarios concurrently, so a program must assume that other cells, of its own language too, run on the same host at the same time.

Ports. Never a fixed port. An agent binds exactly the --port it is given: the runner takes ${PORT} from a range reserved for the scenario (IT_PORT_RANGE, 100 ports per scenario from the run's port block, below the Linux ephemeral range). --port 0 must also work (print the bound port in READY). A client connects only to the --url it is given.
Concurrent runs: ports. Several checkouts may run run-all.sh on one host at once. Each run-all takes an exclusive 10,000-port block for its lifetime: the first of 20000, 30000, 40000 and 50000 whose lock file (${XDG_RUNTIME_DIR:-/tmp}/acp-it-ports-<base>.lock) it can flock, held by a file descriptor until the run and everything it started exit (a kill releases it too); with all four taken it waits. IT_PORT_BASE=<n> overrides the choice (no lock). Scenario i gets <base> + 100 * (i mod 100), 100 ports wide. RunScenario.java run directly, without IT_PORT_RANGE, uses an OS-chosen ephemeral port.
Concurrent runs: Maven repository. One knob, IT_M2_REPO, is the local repository for everything the suite resolves or installs: the SDK install, every program build (${MVNW} is scripts/mvnw.sh, the checkout's ./mvnw with -Dmaven.repo.local=$IT_M2_REPO) and JBang (JBANG_REPO is set to it). Unset, it is integration-testing/.cache/m2 (per checkout, git-ignored), so concurrent checkouts never see each other's 0.19.0-SNAPSHOT; with CI=true it is ~/.m2/repository, which the CI cache keeps. A cold per-checkout repository downloads the SDK's third-party dependencies once (a few minutes); copying another checkout's .cache/m2 warms it, since the SNAPSHOT is reinstalled by every run. Never call ./mvnw or mvn directly from a build; use ${MVNW}.
Files. Never a fixed path. Temporary files go under $TMPDIR, which the runner sets for every process to logs/<scenario>/tmp/, or under the client's {dir} (created with a unique name). Nothing is written into programs/<lang>/ at run time.
Logs. Only stdout and stderr; the runner files them under logs/<scenario>/. No log files elsewhere.
Builds. Only build.sh builds, fetches or installs, and it runs before any scenario starts (serially, once per distinct build in a parallel run). agent.sh and client.sh never build; in a --prepared run nothing else does either.
Processes. No daemons and no detached children: everything a program starts dies with it (a stdio client ends its agent; the runner kills each process tree at the end of a scenario).
No shared mutable state in programs: no lock files, fixed sockets or named pipes, no global caches written at run time.

8. CI

.github/workflows/interop-java.yml, on every push to main and every pull request: JDK 17 only, GenConfigs.java --check and programs/raw/gen_conf.py --check, then the x-java-java-* cells on all three transports.
.github/workflows/cross-sdk.yml, nightly and on manual dispatch (optionally for one peer): a prepare <peer> job per peer SDK fetches and builds the peer and its programs once (run-all.sh --prepare-only) and caches them under the peer's resolved commit (scripts/peer-sha.sh); then one job per peer and transport (typescript-stdio, typescript-http, ..., kotlin-ws) restores that build, installs only its own toolchain, and runs run-all.sh --only 'x-java-<peer>-<t>*,x-<peer>-java-<t>*'. A java job runs the Java pair, conformance, load and both --checks. Each job uploads logs/ as cross-sdk-logs-<leg>; fail-fast is off.
.github/workflows/framework-smoke.yml, on every push to main, nightly and on manual dispatch for any branch: GenConfigs.java --check, then run-all.sh --tag fw on JDK 17. This is the release gate (Framework smoke matrix); release.yml does not check it itself, so the maintainer confirms it is green on the release commit before dispatching a release.
Neither interop-java.yml nor cross-sdk.yml gates a release.

How a scenario is described

configs/<scenario>.json:

{
  "name": "x-java-typescript-http",
  "timeoutSec": 565,
  "peers": ["typescript-sdk"],            // prepared from peers.json before anything is built
  "processes": [
    { "name": "agent", "role": "server", "language": "typescript",
      "dir": "${ROOT}/programs/node", "build": "... ${ROOT}/programs/node/launch/build.sh",
      "run": "${ROOT}/programs/node/launch/agent.sh --transport http --port ${PORT}",
      "env": { "TS_SDK": "${peer.typescript-sdk}" },
      "ready": { "line": "READY", "timeoutSec": 60 } },
    { "name": "client", "role": "client", "language": "java",
      "dir": "${ROOT}/programs/java", "build": "... ${ROOT}/programs/java/launch/build.sh",
      "run": "${ROOT}/programs/java/launch/client.sh --transport http --url http://127.0.0.1:${PORT}/acp",
      "env": { "STEPS": "init.initialize,...,conn.close" }, "timeoutSec": 445 }
  ],
  "assertions": {
    "requiredOutput":  { "agent": ["STEP agent.fs.write PASS"], "client": ["negotiated HTTP_1_1"] },
    "forbiddenOutput": { "client": ["negotiated HTTP_2"] },
    "requiredPatterns": { "agent": ["\\[http\\] DELETE /acp HTTP/1\\.1 .*-> 202"] },
    "checks": ["client RESULT pass == 65", "client RESULT fail == 0"],
    "expectedFailures": [],     // from expectations/*.json, e.g. { "process": "client", "step": "...", "contains": "-32601", "reason": "...", "see": "peer:..." }
    "report": { "client": ["TIMEOUT"] }
  }
}

Order and lifetime. Every build runs first (identical builds run once). Servers start in order, and each must print its ready line in time. Clients then run in order under their timeoutSec and must exit with exitCode (default 0). Every process tree is killed in a finally, including on Ctrl-C.
Variables. ${PORT} (a free port, one per scenario; from IT_PORT_RANGE when set), ${ROOT} (this directory), ${REPO}, ${MVNW}, ${ACP_VERSION} (from the root pom), ${CACHE}, ${LOG_DIR}, ${TMP} (logs/<scenario>/tmp, also every process's TMPDIR), and for each peer ${peer.<name>} (checkout path), ${peerRef.<name>} and ${peerKey.<name>}. An unknown variable is an error.
Samples. A server can list samples, which are jcmd samples taken on ready, on a line another process prints, or on another process's exit (after delayMs). Each sample is a full GC, then heap used, total threads and SDK threads. It is recorded as SAMPLE <label> threads=.. acp_threads=.. heap_mb=.. rss_mb=.. in that server's output.
Checks. A check reads <process> RESULT <lhs> <op> <rhs> or <process> SAMPLE <label> <lhs> <op> <rhs>. A term is a number, a key, or <number>*<key>, and the operators are == != <= >= < >. Values come from the key=value pairs on the process's RESULT lines (all of them merged) or its last SAMPLE <label> line.
Steps. Every STEP <name> FAIL line fails the scenario unless it is declared in expectedFailures. A declared failure that passes also fails the scenario.
Report. report counts lines containing a substring and shows the count in the results without asserting on it. It is for known, timing-dependent behaviour: in load-shared-300, Jetty's HTTP/2 rate control can send GOAWAY when all 300 clients close at once.

Layout

RunScenario.java          JBang entry point: runs one configs/<scenario>.json
GenConfigs.java           JBang: generates configs/x-*.json from the three inputs below, and fw-* from smoke.json
steps.json                the step catalogue (Contracts)
matrix.json               languages, pairs, transports, profiles, transport facts
smoke.json                the framework smoke matrix: step subsets, frameworks and transports, load
expectations/             <lang>.json expected failures, one file per language package
jbang-lib/                config model, peer checkouts, processes, jcmd sampler, assertions
peers.json                peer SDK git URLs, default refs and build commands
configs/                  one JSON file per scenario (x-*, fw-*, load-10-fw-* generated, the rest hand-written)
programs/<lang>/          each language's programs; launch/{build,agent,client}.sh for the cells
programs/<framework>/     spring, micronaut, quarkus: the smoke programs, built on programs/framework
scripts/run-all.sh        a selection of scenarios plus a summary table
.cache/, logs/            generated, git-ignored

Navigation Menu

Repository navigation

Cross-SDK and load scenarios

Prerequisites

Running

Scenarios

Framework smoke matrix

Contracts

1. Files a language package owns

2. Command lines and the transport flag

3. What the programs print

4. The step catalogue, `steps.json`

5. Cells, `matrix.json` and `GenConfigs.java`

6. The Java programs and the expectations

7. Isolation: every scenario must be safe to run next to any other

8. CI

How a scenario is described

Layout

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Cross-SDK and load scenarios

Prerequisites

Running

Scenarios

Framework smoke matrix

Contracts

1. Files a language package owns

2. Command lines and the transport flag

3. What the programs print

4. The step catalogue, steps.json

5. Cells, matrix.json and GenConfigs.java

6. The Java programs and the expectations

7. Isolation: every scenario must be safe to run next to any other

8. CI

How a scenario is described

Layout

4. The step catalogue, `steps.json`

5. Cells, `matrix.json` and `GenConfigs.java`