[Tracking] In-process (FFI) items to be cleaned up #1934

Summary

Several CopilotClientOptions are implemented by lowering them to environment variables set on the spawned CLI child process. With the new in-process (FFI) transport (RuntimeConnection.ForInProcess() / Transport::InProcess / forInProcess()), there is no child process the SDK controls the environment of for these values, so these options are not honored in-process. This affects all SDKs adding the in-process transport (.NET already merged; TypeScript and Rust in progress on stevesa/ffi-inproc-rust-ts).

This issue tracks the gap and the fact that the current workaround is to set the corresponding environment variable on the host/parent process yourself before constructing the client.

Why it happens

For stdio/TCP, the SDK spawns a child process and populates startInfo.Environment (or the equivalent), so per-client values are coherent — each client owns its own OS process.

For in-process, the native runtime cdylib is loaded into the shared host process, and host-side native reads consult the ambient host process environment, not a per-client env block. .NET's ValidateEnvironmentOptions therefore rejects Environment/Telemetry for in-process, and the in-process start path passes an effectively empty env to host_start — so the remaining env-lowered options are simply never applied.

Complete list of options lowered to env vars (from dotnet/src/Client.cs)

Note: UseLoggedInUser, SessionIdleTimeoutSeconds, EnableRemoteSessions, LogLevel are lowered to CLI args (--no-auto-login, --session-idle-timeout, --remote, --log-level), not env vars — but the FFI host_start argv is [entrypoint, --embedded-host] and does not forward these either, so they are a related (separate) in-process gap worth confirming.

Current workaround

Set the environment variable on the host/parent process before creating the client, e.g.:

• Auth: export COPILOT_SDK_AUTH_TOKEN (or ensure ambient GH_TOKEN/GITHUB_TOKEN are valid) instead of GitHubToken.
• Home: export COPILOT_HOME instead of BaseDirectory.
• Keytar: export COPILOT_DISABLE_KEYTAR=1 instead of relying on Mode = Empty.
• Telemetry: export the COPILOT_OTEL_* / OTEL_* vars instead of Telemetry.

Proper long-term fix

These should all become first-class server-level options, not reliant on being passed as env vars. The runtime code can continue to look for env vars as an override for back-compat.

Acceptance / follow-ups

• Decide, cross-SDK, whether each env-lowered option should be honored-for-worker, rejected, or documented-as-unsupported in-process (currently inconsistent: Environment/Telemetry throw; GitHubToken/BaseDirectory/keytar silently no-op).
• Runtime: consume host_start env_json in host-side reads so per-client env is coherent in-process.
• Confirm CLI-arg-lowered options (UseLoggedInUser, idle timeout, remote, log level) in-process.

Refs: dotnet/src/Client.cs ValidateEnvironmentOptions (~L215), in-process start (~L339-353), child-process env block (~L2079-2105), ApplyTelemetryEnvironment (~L1971), auth arg (~L2029). Related PRs: #1901, #1920, #1929, #1930.

Other issues

• Windows+Inproc CI legs are disabled for .NET and Rust due to napi-oop cleanup race. This likely doesn't affect real apps but we do need to re-enable these CI legs for Windows and be sure it's robust after removing napi-oop.
• Underlying runtime doesn't close sqlite connections after a forceStop (or possibly even a session disposal). This leaves locked files on Windows. It might be that in cases where sqlite is shut down gracefully it's slow - this is my leading explanation for why the inproc CI legs all run 25-50% longer than the out-of-proc equivalents.
• API changes:
  • For 2.0, we should actually remove cwd/env options from CopilotClient across all languages instead of just deprecating. The only way to set those should be on the stdio/tcp transports.
  • In the Rust SDK, we've left cwd/env on ClientOptions instead of moving them to transport because it would be a breaking change (language limitation: can't add fields to units/structs that weren't declared #[non_exhaustive] #[derive(Default)]). We should take the breaking change for v2.
• The Rust build.rs is forked between inproc and out-of-proc modes for backcompat, since inproc needs the .node binary and that's not included in the GitHub CLI release (so for inproc we get it from npmjs.org, but that might not be viable long term). We should update the underlying runtime's build/publish mechanism to publish the raw native binaries as separate items on GitHub Releases.
• Rust has program and extra_args on ClientOptions, unlike the other languages which only have this on the child-process transport. These options make no sense outside child-process transport, so we should move them as a breaking change.
• The Python SDK still does PATH resolution (see commit 05dd60e). We should remove this and require bundling or specifying a path explicitly for child-process transports.
• Rust and Go both have build-time flags that control whether the inproc binary is embedded in the app bundle or not (because it duplicates the copilot SEA binary that's also embedded). Once the Node dep is gone we should get to the point where we embed a minimal bootstrapping executable and the dylib, then we don't need to vary what gets embedded at compile time.
• For most SDKs, the base class/interface common to stdio and tcp is called "child process" transport, which seems like a misnomer since it includes unrelated-process TCP. We should rename this to "out of process" transport. In the case of unrelated-process TCP it can reject path/args at runtime.
• Rust's child proc env option is inconsistent with other languages - the others all treat a nonempty set as meaning "replace the entire env with this set" whereas Rust treats it as "add these to the existing env set".
• Python has both copilot.generated.rpc and copilot.rpc. The latter is the same as the former except it excludes internal things. So we should rename copilot.generated.rpc to copilot._generated.rpc and require people only to use copilot.rpc.