From 1c8c289059d2879c85224efed625c3c22425530b Mon Sep 17 00:00:00 2001 From: Dan Lapid Date: Fri, 14 Aug 2026 12:58:49 -0400 Subject: [PATCH] kj-rs-io: KJ async I/O interfaces backed by tokio Third piece of the Rust I/O backend split (on top of kj-rs-tokio): implements KJ's async I/O surface over tokio sockets, driven by a KJ event loop on a TokioEventPort. - kj::AsyncIoStream over tokio TCP/Unix streams (readiness-based, cancel- safe: dropping a pending promise releases the interest), listeners with restrictPeers() enforcement and KJ-parity accept semantics (TCP_NODELAY, peer-address race tolerance), acceptAuthenticated() with KJ-identical NetworkPeerIdentity/LocalPeerIdentity, kj::Network / address parsing following KJ's grammar, and the low-level fd-wrapping providers. - kj_rs_io::setupTokioAsyncIo(): the drop-in kj::setupAsyncIo() replacement (event loop + providers + network), used by the final piece to swap workerd's I/O backend. - Signal watching (tokio's process-global signal registry routed through a runtime task so its cross-thread broadcast never touches the bridge's loop-thread-only wakers), an inotify/kqueue file watcher over AsyncFd, PeerFilter (a faithful port of kj::_::NetworkFilter, which lives in a KJ-internal header -- refcounted here, so networks, addresses, receivers, and restrictPeers() chains share ownership of the filter instead of upstream's outlive-the-derived-network reference contract), the stream unwrap fast path for recovering native tokio sockets, and serve_kj_stream() for Rust servers consuming KJ streams. - Signal watching awaits tokio's signal streams directly from the bridged future: the kj-rs waker bridge is thread-safe, so the process-global registry's broadcast may run on another runtime's thread (workerd's inspector thread) or Windows' console-ctrl thread and still be delivered. Includes a multi-runtime regression test reproducing the SIGTERM-hang scenario (a second tokio-ported KJ loop parked on another thread racing for the signal's wake byte). resolve_host keeps its forwarding-task shape as a same-thread fast-path optimization (on the loop's LocalSet, so it is cancelled with the loop), documented as such. - kj_rs_io::TokioAsyncIoContext composes kj_rs_tokio::TokioAsyncIoContext (port, loop, wait scope and their teardown) with the providers; teardown is member order. - Multi-piece write() is one bridged operation using vectored writes (writev): the pieces cross as an opaque KjPieces handle read through two accessors (no raw pointers), and IoSlice::advance_slices resumes a partial writev at the exact byte. shutdownWrite() uses a socket2::SockRef borrow on every platform instead of dup'ing the fd on unix. - sockaddr_from_bytes validates length against family: an AF_INET in four bytes is rejected rather than decoded out of zero-filled storage. - FileWatcher::onChange()'s coroutine co-owns the refcounted Impl, so the promise may outlive the FileWatcher object: no outlive-the-watcher contract. - No per-poll runtime entering: the loop thread is permanently inside the port's runtime context (kj-rs-tokio), so the with_runtime wrapper and every Handle::enter() around resource creation are gone; what remains is a one-time precondition check that turns "no TokioEventPort on this thread" into a kj::Exception instead of a tokio panic. - TokioConnectionReceiver holds exactly one kj::Own: a share of the PeerFilter chain (kj::Rc::toOwn), or, on the wrapListenSocketFd path, the caller's filter behind kj::NullDisposer -- KJ's idiom for an explicitly non-owning Own -- instead of an owning handle plus a reference that may alias it. The KJ interface's caller-keeps-it-alive rule is unchanged. - whenWriteDisconnected's second fd is documented as inherent to tokio's cached-readiness model (a disconnect waiter clearing WRITABLE on the socket's own registration would park concurrent try_write callers on an edge that never comes), with its cost stated. - TokioAddress::from_socket_addrs: a programmatic multi-address constructor, the counterpart of a DNS answer. - Extensive C++-driven tests: stream contract, transfers, cancellation, DNS/connect ordering, refused connects (instrumented for a Windows CI wedge), unix sockets, peer identity, file watching, HTTP and Cap'n Proto RPC over the tokio backend. Adds bytes and socket2 to the workspace, plus tokio's io-util, macros, and signal features. Review hardening (re-review against the prep-review lens): - TokioStream tracks in-flight I/O with a RefCell borrow instead of a caller contract: every operation holds a shared borrow across its awaits and unwrap/take needs the exclusive borrow, so unwrapping a stream with I/O in flight is a checked Err rather than aliasing a live &mut. unwrap_kj_stream is a safe fn now; the crate's public surface has no unsafe fn. - The per-identity allow-all PeerFilter is created per identity, not as a process-wide static kj::Rc (non-atomic refcount raced by accept loops on multiple threads). - kjStreamShutdownWrite returns Result: it can throw, and a C++ exception across a non-Result cxx shim would abort. - FileWatcher readiness goes through an owned TokioFdWatcher (owns a dup of the notification fd and one I/O-driver registration) instead of a borrowed raw fd with keep-open / register-once contracts in comments. - ServeIo::Duplex cross-thread docs corrected (thread-safe since the waker rework; a cross-thread wake is a perf hop, not UB) and the net.rs single-thread-axiom / getFd runCatchingExceptions / winsock windows-sanity / #elifdef nits fixed. - Tests: Rust unit tests for the errno->exception table, the parseAddress grammar, sockaddr round-trips, and the hollow/in-flight stream guards, with send/sync static_assertions; C++ tests for concurrent accept on two loops, a restrictPeers child outliving its parent, cancel-mid-connect and cancel-mid-accept, teardown with an in-flight DNS lookup, a zero-length write, and driving a ServeIo::Duplex from another thread. The Windows connect-wedge instrumentation (a lost-wake diagnosis fixed by the waker rework) is retired. Clean under --config=asan (C++) and --config=tsan-macos. - Object-relationship overview in lib.rs. Second re-review hardening: - ~TokioAsyncIoContext guards cancelSpawnedTasks() with kj::UnwindDetector and deletes its move-assignment (same reasons as kj-rs-tokio's context). - The #[cfg(windows)] when_write_disconnected arm carries the same #[expect(await_holding_refcell_ref)] as its siblings and drops the unreachable!() (deny(clippy::unreachable)), so a Windows clippy run stays clean. - serve_helpers.rs drive() joins the foreign echo-consumer thread even on the pump-error path (was a test-only unjoined-thread leak). - Tests: concurrent read+write both in flight (the shared-borrow invariant the RefCell in-flight guard rests on) and hollow into_serve_io -> None. Test-gap pass: - New peer-filter-test.c++: direct PeerFilter::shouldAllow coverage of the CIDR grammar -- public/network/local/private, explicit CIDRs, the allow/deny specificity tie-break, IPv6, unix/unix-abstract, nested-chain enforcement, and the deny-'public'/'network' rejections. - Stream/net: write() to a reset peer -> DISCONNECTED, write_all backpressure against a non-reading peer, IPv6 getSockaddr. - Signals: multiple concurrent watchers on one signum, different-signum isolation, dropped-pending-watch, and an unwatchable signum erroring. - take_kj_socket on an already-hollow stream is refused; two FileWatcher scenarios (multiple files in one watcher, two independent watchers). - Rust units: get_sockaddr unsupported-family and runtime_handle no-port arms. Coverage pass: - Vectored writes: pieces totalling several MiB with empty pieces mixed in against a concurrent reader (partial writev + backpressure), and all-empty pieces. - Network: an unresolvable host fails with a getaddrinfo exception; listen() on an existing unix socket path fails (no unlink, like KJ); acceptAuthenticated() through a restricted listener drops disallowed peers. - Pump: a DISCONNECTED read and a DISCONNECTED write are treated as EOF, not errors; a consumer dropping its ServeIo without shutdown() half-closes the kj side. - FileWatcher: delete then recreate (fires on both backends; the recreated file is tracked by inotify, not by kqueue -- documented). - Rust units: sockaddr family/length mismatch, own_socket_from_raw happy and rejection paths; two seeded randomized tests (20k iterations each, no new dependencies): sockaddr_from_bytes never panics on arbitrary bytes and accepts only family/length-consistent input, and the address grammar never panics on arbitrary strings with every accepted literal surviving a display round trip. - connect()'s per-address fallback is tested deterministically: a two-address list whose first port is provably closed connects through the second, and an all-refused list propagates the last DISCONNECTED error. The localhost DNS test only claims DNS. - Shared C++ test helpers (connected pairs, patterned data, chunked write/verify, timer-bounded waits) in tests/io-test-helpers.h. - A TLS-shaped byte-transforming wrapper (exposing its transport's fd) is pumped correctly by serve_kj_stream, never read off its fd; the onChange() promise outlives its FileWatcher. The three I/O-heavy suites are `medium`: they run in ~2 s alone but exceeded small's 3 s budget under parallel build load. Co-Authored-By: Claude Fable 5.1 --- deps/rust/Cargo.lock | 35 + deps/rust/Cargo.toml | 4 +- src/rust/cxx/AGENTS.md | 4 + src/rust/cxx/kj-rs-io/BUILD.bazel | 77 ++ src/rust/cxx/kj-rs-io/async-io.c++ | 531 ++++++++ src/rust/cxx/kj-rs-io/async-io.h | 292 +++++ src/rust/cxx/kj-rs-io/error.rs | 243 ++++ src/rust/cxx/kj-rs-io/ffi.rs | 1073 ++++++++++++++++ src/rust/cxx/kj-rs-io/file-watcher.c++ | 228 ++++ src/rust/cxx/kj-rs-io/file-watcher.h | 57 + src/rust/cxx/kj-rs-io/lib.rs | 111 ++ src/rust/cxx/kj-rs-io/net.rs | 841 +++++++++++++ src/rust/cxx/kj-rs-io/peer-filter.c++ | 203 +++ src/rust/cxx/kj-rs-io/peer-filter.h | 55 + src/rust/cxx/kj-rs-io/readiness.rs | 94 ++ src/rust/cxx/kj-rs-io/runtime.rs | 61 + src/rust/cxx/kj-rs-io/serve.rs | 459 +++++++ src/rust/cxx/kj-rs-io/signal.rs | 94 ++ src/rust/cxx/kj-rs-io/stream.rs | 848 +++++++++++++ src/rust/cxx/kj-rs-io/tests/BUILD.bazel | 170 +++ src/rust/cxx/kj-rs-io/tests/async-io-test.c++ | 1109 +++++++++++++++++ .../cxx/kj-rs-io/tests/capnp-rpc-test.c++ | 79 ++ .../cxx/kj-rs-io/tests/file-watcher-test.c++ | 368 ++++++ src/rust/cxx/kj-rs-io/tests/http-test.c++ | 147 +++ src/rust/cxx/kj-rs-io/tests/io-test-helpers.h | 108 ++ src/rust/cxx/kj-rs-io/tests/lib.rs | 105 ++ .../cxx/kj-rs-io/tests/peer-filter-test.c++ | 193 +++ src/rust/cxx/kj-rs-io/tests/serve-test.c++ | 422 +++++++ src/rust/cxx/kj-rs-io/tests/serve_helpers.rs | 185 +++ src/rust/cxx/kj-rs-io/tests/test_helpers.rs | 80 ++ src/rust/cxx/kj-rs-io/unwrap.h | 95 ++ 31 files changed, 8369 insertions(+), 2 deletions(-) create mode 100644 src/rust/cxx/kj-rs-io/BUILD.bazel create mode 100644 src/rust/cxx/kj-rs-io/async-io.c++ create mode 100644 src/rust/cxx/kj-rs-io/async-io.h create mode 100644 src/rust/cxx/kj-rs-io/error.rs create mode 100644 src/rust/cxx/kj-rs-io/ffi.rs create mode 100644 src/rust/cxx/kj-rs-io/file-watcher.c++ create mode 100644 src/rust/cxx/kj-rs-io/file-watcher.h create mode 100644 src/rust/cxx/kj-rs-io/lib.rs create mode 100644 src/rust/cxx/kj-rs-io/net.rs create mode 100644 src/rust/cxx/kj-rs-io/peer-filter.c++ create mode 100644 src/rust/cxx/kj-rs-io/peer-filter.h create mode 100644 src/rust/cxx/kj-rs-io/readiness.rs create mode 100644 src/rust/cxx/kj-rs-io/runtime.rs create mode 100644 src/rust/cxx/kj-rs-io/serve.rs create mode 100644 src/rust/cxx/kj-rs-io/signal.rs create mode 100644 src/rust/cxx/kj-rs-io/stream.rs create mode 100644 src/rust/cxx/kj-rs-io/tests/BUILD.bazel create mode 100644 src/rust/cxx/kj-rs-io/tests/async-io-test.c++ create mode 100644 src/rust/cxx/kj-rs-io/tests/capnp-rpc-test.c++ create mode 100644 src/rust/cxx/kj-rs-io/tests/file-watcher-test.c++ create mode 100644 src/rust/cxx/kj-rs-io/tests/http-test.c++ create mode 100644 src/rust/cxx/kj-rs-io/tests/io-test-helpers.h create mode 100644 src/rust/cxx/kj-rs-io/tests/lib.rs create mode 100644 src/rust/cxx/kj-rs-io/tests/peer-filter-test.c++ create mode 100644 src/rust/cxx/kj-rs-io/tests/serve-test.c++ create mode 100644 src/rust/cxx/kj-rs-io/tests/serve_helpers.rs create mode 100644 src/rust/cxx/kj-rs-io/tests/test_helpers.rs create mode 100644 src/rust/cxx/kj-rs-io/unwrap.h diff --git a/deps/rust/Cargo.lock b/deps/rust/Cargo.lock index c9979292e20..cf20a1e2e7a 100644 --- a/deps/rust/Cargo.lock +++ b/deps/rust/Cargo.lock @@ -476,6 +476,7 @@ dependencies = [ "scratch", "serde", "serde_json", + "socket2", "static_assertions", "swc_common", "swc_ts_fast_strip", @@ -550,6 +551,16 @@ version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + [[package]] name = "fastrand" version = "2.5.0" @@ -1601,6 +1612,16 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + [[package]] name = "simd-adler32" version = "0.3.10" @@ -2105,13 +2126,27 @@ version = "1.53.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed" dependencies = [ + "bytes", "libc", "mio", "pin-project-lite", + "signal-hook-registry", "socket2", + "tokio-macros", "windows-sys", ] +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.4", +] + [[package]] name = "tracing" version = "0.1.44" diff --git a/deps/rust/Cargo.toml b/deps/rust/Cargo.toml index 9aa2a97789c..69d6ba04c95 100644 --- a/deps/rust/Cargo.toml +++ b/deps/rust/Cargo.toml @@ -54,9 +54,9 @@ ruff_python_parser = { git = "https://github.com/astral-sh/ruff", tag = "0.12.1" # param_extractor depends on unbounded_depth feature serde_json = { version = "1", features = ["unbounded_depth"] } serde = { version = "1", features = ["derive"] } +socket2 = "0.6" thiserror = "2" -# tokio is huge, let's enable only features when we actually need them. -tokio = { version = "1", default-features = false, features = ["net", "rt", "rt-multi-thread", "sync", "time"] } +tokio = { version = "1", default-features = false, features = ["io-util", "macros", "net", "rt", "rt-multi-thread", "signal", "sync", "time"] } tracing = { version = "0", default-features = false, features = ["std"] } swc_common = "25" swc_ts_fast_strip = "57" diff --git a/src/rust/cxx/AGENTS.md b/src/rust/cxx/AGENTS.md index c01f74f67fd..4cb7ff9759b 100644 --- a/src/rust/cxx/AGENTS.md +++ b/src/rust/cxx/AGENTS.md @@ -36,6 +36,10 @@ Bazel module, Cargo workspace, toolchain configuration, or external `workerd-cxx - `kj-rs-tokio/` — `TokioEventPort`: a `kj::EventPort` backed by a per-thread tokio `current_thread` runtime, plus `setupTokioAsyncIo()` (no I/O providers) and `kj_rs_tokio::spawn()` +- `kj-rs-io/` — KJ async I/O interfaces over tokio (`kj::AsyncIoStream`, listeners, + `kj::Network`, providers, signals, file watching), `kj_rs_io::setupTokioAsyncIo()` as the + drop-in `kj::setupAsyncIo()` replacement, the stream unwrap fast path, and + `serve_kj_stream()` for Rust servers consuming KJ streams - `tests/` and `kj-rs/tests/` — Rust and C++ bridge integration tests - `tools/bazel/` — Bazel bridge-generation macro used by this component's tests diff --git a/src/rust/cxx/kj-rs-io/BUILD.bazel b/src/rust/cxx/kj-rs-io/BUILD.bazel new file mode 100644 index 00000000000..9f15efc6c29 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/BUILD.bazel @@ -0,0 +1,77 @@ +load("@rules_rust//rust:defs.bzl", "rust_library") +load("//:build/wd_cc_library.bzl", "wd_cc_library") +load("//:build/wd_rust_test.bzl", "wd_rust_test") +load("//src/rust/cxx/tools/bazel:rust_cxx_bridge.bzl", "rust_cxx_bridge") + +wd_cc_library( + name = "kj-rs-io-lib", + srcs = glob(["*.c++"]), + hdrs = glob(["*.h"]), + include_prefix = "kj-rs-io", + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + strip_include_prefix = "/src/rust/cxx/kj-rs-io", + visibility = ["//visibility:public"], + deps = [ + ":bridge", + "//src/rust/cxx/kj-rs-tokio:kj-rs-tokio-lib", + ], +) + +rust_library( + name = "kj-rs-io", + srcs = glob(["*.rs"]), + compile_data = glob(["*.h"]), + edition = "2024", + link_deps = [ + ":bridge", + ":kj-rs-io-lib", + ], + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + visibility = ["//visibility:public"], + deps = [ + "//src/rust/cxx", + "//src/rust/cxx/kj-rs", + "//src/rust/cxx/kj-rs-tokio", + "@crates_vendor//:bytes", + "@crates_vendor//:libc", + "@crates_vendor//:socket2", + "@crates_vendor//:static_assertions", + "@crates_vendor//:tokio", + ], +) + +wd_rust_test( + name = "kj-rs-io_test", + crate = "kj-rs-io", + edition = "2024", +) + +rust_cxx_bridge( + name = "bridge", + src = "ffi.rs", + hdrs = ["unwrap.h"], + include_prefix = "kj-rs-io", + visibility = ["//visibility:public"], + deps = [ + "//src/rust/cxx/kj-rs", + "@capnp-cpp//src/kj:kj", + # kj-rs-io IS the kj-side implementation of the rust I/O backend: it derives from the + # abstract kj::AsyncIoStream / kj::Network / kj::LowLevelAsyncIoProvider interfaces and + # reuses portable helpers (kj::newOneWayPipe, kj::CidrRange, the AsyncInputStream / + # AsyncOutputStream default-method vtable slots). Those live in the ABSTRACT layers + # :kj-async-core (Promise machinery) + :kj-async-io (abstract streams / Network / CIDR; + # no event loop). It does NOT call kj::setupAsyncIo / kj::UnixEventPort, so it never + # needs :kj-async-os (the concrete OS event loop) -- the tokio-backed EventPort + # (kj-rs-tokio) replaces it. A stray setupAsyncIo would be an undefined-symbol link + # error, the intended backstop. + "@capnp-cpp//src/kj:kj-async-core", + "@capnp-cpp//src/kj:kj-async-io", + "//src/rust/cxx:core", + ], +) diff --git a/src/rust/cxx/kj-rs-io/async-io.c++ b/src/rust/cxx/kj-rs-io/async-io.c++ new file mode 100644 index 00000000000..93cb99b9e5b --- /dev/null +++ b/src/rust/cxx/kj-rs-io/async-io.c++ @@ -0,0 +1,531 @@ +#include "kj-rs-io/async-io.h" + +#include + +#include + +#if _WIN32 +#include +// windows.h (pulled in by winsock2.h) defines ERROR as a macro, which breaks KJ_LOG(ERROR). +#include +#else +#include +#include +#include +#include +#if __APPLE__ || __FreeBSD__ || __OpenBSD__ || __NetBSD__ || __DragonFly__ +#include +#endif +#endif + +namespace kj_rs_io { + +// ======================================================================================= +// TokioAsyncIoStream + +kj::Promise TokioAsyncIoStream::tryRead(void *buffer, size_t minBytes, size_t maxBytes) { + return stream_try_read( + *inner, ::rust::Slice(reinterpret_cast(buffer), maxBytes), minBytes); +} + +kj::Promise TokioAsyncIoStream::write(kj::ArrayPtr buffer) { + return stream_write(*inner, ::rust::Slice(buffer.begin(), buffer.size())); +} + +kj::Promise TokioAsyncIoStream::write( + kj::ArrayPtr> pieces) { + return writePieces(pieces); +} + +kj::Promise TokioAsyncIoStream::writePieces( + kj::ArrayPtr> pieces) { + // One bridged operation for all pieces (writev on the Rust side). The bridged future borrows + // the KjPieces it is handed, so it lives in this coroutine's frame; the pieces themselves are + // the caller's, valid until the promise settles per the kj::AsyncOutputStream contract. + KjPieces owned{pieces}; + co_await stream_write_pieces(*inner, owned); +} + +kj::Promise TokioAsyncIoStream::whenWriteDisconnected() { + return stream_when_write_disconnected(*inner); +} + +void TokioAsyncIoStream::shutdownWrite() { + stream_shutdown_write(*inner); +} + +void TokioAsyncIoStream::getsockopt(int level, int option, void *value, kj::uint *length) { + // The platform seam lives on the Rust side (stream_getsockopt); errors surface as kj + // exceptions, like KJ_SYSCALL. Raw socklen in/out semantics: the syscall's reported length + // is mirrored back verbatim. + *length = stream_getsockopt( + *inner, level, option, ::rust::Slice(reinterpret_cast(value), *length)); +} + +void TokioAsyncIoStream::setsockopt(int level, int option, const void *value, kj::uint length) { + stream_setsockopt(*inner, level, option, + ::rust::Slice(reinterpret_cast(value), length)); +} + +void TokioAsyncIoStream::getsockname(struct sockaddr *addr, kj::uint *length) { + auto bytes = stream_local_addr(*inner); + // Mirror the raw syscall's truncation semantics: copy what fits into the caller's buffer, + // report the address's full length. + memcpy(addr, bytes.data(), kj::min(bytes.size(), *length)); + *length = bytes.size(); +} + +void TokioAsyncIoStream::getpeername(struct sockaddr *addr, kj::uint *length) { + auto bytes = stream_peer_addr(*inner); + memcpy(addr, bytes.data(), kj::min(bytes.size(), *length)); + *length = bytes.size(); +} + +kj::Maybe TokioAsyncIoStream::getFd() const { +#if _WIN32 + // On Windows the underlying handle is a winsock SOCKET, not a Unix fd; it is exposed via + // getWin32Handle() below instead. (Validated by Windows CI.) + return kj::none; +#else + // On unix the raw socket handle is the fd, widened losslessly to int64 by the bridge; -1 + // means the wrapper is hollow (unwrapped). + int64_t handle = stream_try_raw_handle(*inner); + if (handle < 0) return kj::none; + return static_cast(handle); +#endif +} + +#if _WIN32 +// Validated by Windows CI; mirrors the unix getFd() arm (and kj's own win32 AsyncStreamFd, +// which returns its SOCKET cast to void* -- capnproto async-io-win32.c++). +kj::Maybe TokioAsyncIoStream::getWin32Handle() const { + int64_t handle = stream_try_raw_handle(*inner); + if (handle < 0) return kj::none; + return reinterpret_cast(static_cast(handle)); +} +#endif + +::rust::Box unwrapTokioStream(kj::AsyncIoStream &stream) { + KJ_IF_SOME(tokioStream, kj::dynamicDowncastIfAvailable(stream)) { + return tokioStream.unwrap(); + } + KJ_FAIL_REQUIRE("stream is not a kj-rs-io tokio-backed stream; cannot unwrap"); +} + +// ======================================================================================= +// TokioConnectionReceiver + +namespace { + +// Builds the accepted connection's kj::PeerIdentity, mirroring KJ's SocketAddress::getIdentity() +// (kj/async-io-unix.c++): NetworkPeerIdentity wrapping the peer's address for TCP peers (its +// toString() is "ip:port" / "[v6]:port", byte-identical to KJ's format -- workerd's HTTP +// listener puts this string in the cf blob's clientIp), LocalPeerIdentity with the peer's +// process credentials for unix sockets, UnknownPeerIdentity otherwise. +kj::Own peerIdentityFromSockaddr( + struct sockaddr *sa, kj::uint addrlen, [[maybe_unused]] kj::AsyncIoStream &stream) { + switch (sa->sa_family) { + case AF_INET: + case AF_INET6: { + // The identity's NetworkAddress uses an allow-all filter (not the listener's): it exists + // for toString()/getAddress(); restrictPeers enforcement on this listener already happened + // in the accept loop. (KJ instead threads the listener's filter through, which only + // matters if a caller connect()s back through the identity address.) A fresh filter per + // identity, NOT a process-wide static: kj::Rc's refcount is not atomic, and accept loops + // run on every tokio-ported loop thread, so a shared one would race. + auto address = network_get_sockaddr( + ::rust::Slice(reinterpret_cast(sa), addrlen)); + return kj::NetworkPeerIdentity::newInstance( + kj::heap(kj::mv(address), kj::rc())); + } +#if !_WIN32 + case AF_UNIX: { + // Same credential sources and invalid-value handling as KJ (SO_PEERCRED on Linux, + // LOCAL_PEERCRED/LOCAL_PEERPID on BSDs/macOS; OpenBSD defines SO_PEERCRED but with a + // different interface, so it uses the LOCAL_PEERCRED arm). + kj::LocalPeerIdentity::Credentials result; +#if defined(SO_PEERCRED) && !__OpenBSD__ + struct ucred creds; + kj::uint length = sizeof(creds); + stream.getsockopt(SOL_SOCKET, SO_PEERCRED, &creds, &length); + if (creds.pid > 0) { + result.pid = creds.pid; + } + if (creds.uid != static_cast(-1)) { + result.uid = creds.uid; + } +#elifdef LOCAL_PEERCRED + struct xucred creds; + kj::uint length = sizeof(creds); + stream.getsockopt(SOL_LOCAL, LOCAL_PEERCRED, &creds, &length); + KJ_ASSERT(length == sizeof(creds)); + if (creds.cr_uid != static_cast(-1)) { + result.uid = creds.cr_uid; + } +#ifdef LOCAL_PEERPID + pid_t pid; + length = sizeof(pid); + stream.getsockopt(SOL_LOCAL, LOCAL_PEERPID, &pid, &length); + KJ_ASSERT(length == sizeof(pid)); + if (pid > 0) { + result.pid = pid; + } +#endif +#endif + return kj::LocalPeerIdentity::newInstance(result); + } +#endif // !_WIN32 + default: + return kj::UnknownPeerIdentity::newInstance(); + } +} + +} // namespace + +kj::Promise> TokioConnectionReceiver::accept() { + return acceptImpl(false).then( + [](kj::AuthenticatedStream authenticated) { return kj::mv(authenticated.stream); }); +} + +kj::Promise TokioConnectionReceiver::acceptAuthenticated() { + return acceptImpl(true); +} + +kj::Promise TokioConnectionReceiver::acceptImpl(bool authenticated) { + for (;;) { + auto stream = co_await listener_accept(*inner); + // restrictPeers / NetworkFilter enforcement, mirroring KJ: a connection from a disallowed + // peer is silently dropped and we keep accepting. + struct sockaddr_storage addr; + memset(&addr, 0, sizeof(addr)); + kj::uint addrlen = 0; + KJ_IF_SOME(exception, kj::runCatchingExceptions([&]() { + auto bytes = stream_peer_addr(*stream); + KJ_ASSERT(bytes.size() <= sizeof(addr), "sockaddr too large"); + memcpy(&addr, bytes.data(), bytes.size()); + addrlen = bytes.size(); + })) { + // The peer can reset the connection between tokio's accept() and this call, in which + // case getpeername fails (EINVAL on macOS, ENOTCONN elsewhere). The connection is dead; + // drop it and keep accepting. This must NOT throw: an exception here propagates out of + // the server's accept loop and takes down the whole process (observed as a fatal + // uncaught kj::Exception under client abort storms). Unlike KJ's native accept path, + // which gets the peer address atomically from accept4(), we re-derive it and so must + // tolerate the race. Log at INFO (off by default) for observability under abort storms. + KJ_LOG(INFO, "dropping accepted connection; could not read peer address", exception); + continue; + } + if (!filter->shouldAllow(reinterpret_cast(&addr), addrlen)) { + // Drop the disallowed connection and wait for the next one. + continue; + } + kj::AuthenticatedStream result; + result.stream = kj::heap(kj::mv(stream)); + if (authenticated) { + result.peerIdentity = peerIdentityFromSockaddr( + reinterpret_cast(&addr), addrlen, *result.stream); + } else { + result.peerIdentity = kj::UnknownPeerIdentity::newInstance(); + } + co_return kj::mv(result); + } +} + +kj::uint TokioConnectionReceiver::getPort() { + return listener_port(*inner); +} + +void TokioConnectionReceiver::getsockopt(int level, int option, void *value, kj::uint *length) { + *length = listener_getsockopt( + *inner, level, option, ::rust::Slice(reinterpret_cast(value), *length)); +} + +void TokioConnectionReceiver::setsockopt( + int level, int option, const void *value, kj::uint length) { + listener_setsockopt(*inner, level, option, + ::rust::Slice(reinterpret_cast(value), length)); +} + +void TokioConnectionReceiver::getsockname(struct sockaddr *addr, kj::uint *length) { + auto bytes = listener_local_addr(*inner); + // Mirror the raw syscall's truncation semantics (see TokioAsyncIoStream::getsockname). + memcpy(addr, bytes.data(), kj::min(bytes.size(), *length)); + *length = bytes.size(); +} + +// ======================================================================================= +// TokioNetworkAddress / TokioNetwork + +kj::Promise> TokioNetworkAddress::connect() { + // KJ contract (NetworkAddressImpl::connect() in kj/async-io-unix.c++): callers may drop the + // NetworkAddress while the returned promise is still pending. We honor this by cloning the + // resolved address list into a coroutine frame local: the frame owns the copy, so it + // survives every co_await and is dropped on completion/cancellation. connect() may be called + // repeatedly on the same address, so we clone rather than move `*inner` out of `this`. + // The filter share is frame-owned for the same reason: `this` may be gone after the first + // co_await, so the frame keeps the chain alive itself rather than reading the member again. + auto addr = address_clone(*inner); + auto localFilter = filter.addRef(); + size_t count = address_count(*addr); + KJ_REQUIRE(count > 0, "no addresses to connect to"); + + // Try each resolved address in order; a filter block or connect error falls through to the + // next one, and the last address's exception propagates (KJ parity). + kj::Maybe lastException; + for (size_t i = 0; i < count; i++) { + kj::Maybe> stream; + try { + auto raw = address_raw_sockaddr(*addr, i); + // Copy into sockaddr_storage for alignment (rust::Vec data is 1-aligned). + struct sockaddr_storage storage; + memset(&storage, 0, sizeof(storage)); + KJ_REQUIRE(raw.size() <= sizeof(storage), "sockaddr too large"); + memcpy(&storage, raw.data(), raw.size()); + if (!localFilter->shouldAllow(reinterpret_cast(&storage), raw.size())) { + // Exact KJ error text; error-string parity matters to callers. + lastException = KJ_EXCEPTION(FAILED, "connect() blocked by restrictPeers()"); + } else { + stream = kj::heap(co_await address_connect_index(*addr, i)); + } + } catch (...) { + // Note: getCaughtExceptionAsKj() rethrows kj::CanceledException, so cancellation still + // propagates out of this coroutine instead of being folded into lastException. + lastException = kj::getCaughtExceptionAsKj(); + } + KJ_IF_SOME(s, stream) { + co_return kj::mv(s); + } + } + + kj::throwFatalException(kj::mv(KJ_ASSERT_NONNULL(lastException))); +} + +kj::Own TokioNetworkAddress::listen() { + return kj::heap(address_listen(*inner), filter.addRef()); +} + +kj::Own TokioNetworkAddress::clone() { + return kj::heap(address_clone(*inner), filter.addRef()); +} + +kj::String TokioNetworkAddress::toString() { + auto text = address_to_string(*inner); + return kj::heapString(text.data(), text.size()); +} + +kj::Promise> TokioNetwork::parseAddress( + kj::StringPtr addr, kj::uint portHint) { + KJ_REQUIRE(portHint < 65536, "port hint too large", portHint); + // The Rust side takes an owned copy: the caller's buffer need not outlive this call. + // + // Note: unlike KJ, disallowed (restrictPeers) DNS results are not dropped here; they are + // rejected at connect()/accept() time instead. See TokioNetworkAddress. + return network_parse_address( + ::rust::String(addr.begin(), addr.size()), static_cast(portHint)) + .then([filter = filter.addRef()]( + ::rust::Box address) mutable -> kj::Own { + // The continuation owns its share of the filter chain (no `this` capture): the returned + // promise is independent of this network's lifetime. + return kj::heap(kj::mv(address), kj::mv(filter)); + }); +} + +kj::Own TokioNetwork::getSockaddr(const void *sockaddr, kj::uint len) { + // KJ parity: getSockaddr() rejects filtered addresses eagerly (same error text as KJ). + KJ_REQUIRE(filter->shouldAllow(reinterpret_cast(sockaddr), len), + "address blocked by restrictPeers()"); + return kj::heap(network_get_sockaddr(::rust::Slice( + reinterpret_cast(sockaddr), len)), + filter.addRef()); +} + +kj::Own TokioNetwork::restrictPeers( + kj::ArrayPtr allow, kj::ArrayPtr deny) { + // The child owns a share of this network's filter chain (see TokioNetwork's constructor), so + // it remains valid even if this network is destroyed first. + return kj::heap(*this, allow, deny); +} + +// ======================================================================================= +// TokioLowLevelAsyncIoProvider + +namespace { + +#if _WIN32 +// Normalizes KJ's fd-wrapping flags so Rust always receives a SOCKET it owns, in non-blocking +// mode: the windows arm of prepareFd, mirroring the unix arm below in *effect*, not mechanism. +// kj's own win32 provider (capnproto async-io-win32.c++: OwnedFd, NEW_FD_FLAGS) never dups a +// borrowed socket -- it merely skips closesocket() on destruction when TAKE_OWNERSHIP is +// absent -- ignores ALREADY_CLOEXEC entirely (there is no CLOEXEC on Windows; handle +// inheritance is the analogue), and never toggles non-blocking mode (it uses overlapped I/O, +// not readiness). Rust's OwnedSocket has no "don't close" mode, so a borrowed socket is +// duplicated (WSADuplicateSocketW + WSASocketW, non-inheritable) into a handle Rust can own; +// and tokio/mio's readiness model requires non-blocking sockets, so FIONBIO is set unless the +// caller declared ALREADY_NONBLOCK (the duplicate shares the underlying socket state, so this +// is observed through the caller's handle too, matching the unix dup()+O_NONBLOCK behavior). +// Validated by Windows CI. +uintptr_t prepareFd(uintptr_t fd, kj::uint flags) { + SOCKET sock = static_cast(fd); + if ((flags & kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP) == 0) { + WSAPROTOCOL_INFOW info; + KJ_WINSOCK(WSADuplicateSocketW(sock, GetCurrentProcessId(), &info)); + SOCKET duped = WSASocketW(FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO, &info, 0, + WSA_FLAG_OVERLAPPED | WSA_FLAG_NO_HANDLE_INHERIT); + if (duped == INVALID_SOCKET) { + KJ_FAIL_WIN32("WSASocketW()", WSAGetLastError()); + } + sock = duped; + } + // ALREADY_NONBLOCK does not exist on Windows (kj declares it under #if !_WIN32), so callers + // cannot assert pre-set non-blocking mode; always enable it (idempotent). + u_long mode = 1; + KJ_WINSOCK(ioctlsocket(sock, FIONBIO, &mode)); + return static_cast(sock); +} +#else +// Normalizes KJ's fd-wrapping flags so Rust always receives an fd it owns, with CLOEXEC set and +// in non-blocking mode. +int prepareFd(int fd, kj::uint flags) { + if ((flags & kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP) == 0) { + // dup() shares the open file description — the O_NONBLOCK set below is observed through the + // caller's fd too, matching KJ (which sets O_NONBLOCK on the caller's fd directly) — while + // giving Rust a descriptor it can own and close. + int duped; + KJ_SYSCALL(duped = ::dup(fd)); + fd = duped; + KJ_SYSCALL(fcntl(fd, F_SETFD, FD_CLOEXEC)); + } else if ((flags & kj::LowLevelAsyncIoProvider::ALREADY_CLOEXEC) == 0) { + KJ_SYSCALL(fcntl(fd, F_SETFD, FD_CLOEXEC)); + } + if ((flags & kj::LowLevelAsyncIoProvider::ALREADY_NONBLOCK) == 0) { + int fl; + KJ_SYSCALL(fl = fcntl(fd, F_GETFL)); + if ((fl & O_NONBLOCK) == 0) { + KJ_SYSCALL(fcntl(fd, F_SETFL, fl | O_NONBLOCK)); + } + } + return fd; +} + +// kj::AsyncInputStream over an arbitrary readable fd (pipe, socket, character device). +class TokioInputStreamFd final: public kj::AsyncInputStream { + public: + explicit TokioInputStreamFd(::rust::Box inner): inner(kj::mv(inner)) {} + + kj::Promise tryRead(void *buffer, size_t minBytes, size_t maxBytes) override { + return input_fd_try_read( + *inner, ::rust::Slice(reinterpret_cast(buffer), maxBytes), minBytes); + } + + private: + ::rust::Box inner; +}; + +// kj::AsyncOutputStream over an arbitrary writable fd. +class TokioOutputStreamFd final: public kj::AsyncOutputStream { + public: + explicit TokioOutputStreamFd(::rust::Box inner): inner(kj::mv(inner)) {} + + kj::Promise write(kj::ArrayPtr buffer) override { + return output_fd_write(*inner, ::rust::Slice(buffer.begin(), buffer.size())); + } + + kj::Promise write(kj::ArrayPtr> pieces) override { + for (auto piece: pieces) { + co_await write(piece); + } + } + + // Pipes/arbitrary fds have no portable disconnect detection here; KJ allows a never-resolving + // promise for such streams. + kj::Promise whenWriteDisconnected() override { + return kj::NEVER_DONE; + } + + private: + ::rust::Box inner; +}; +#endif // _WIN32 (prepareFd platform arms; the pipe-fd stream classes above are unix-only) + +} // namespace + +kj::Own TokioLowLevelAsyncIoProvider::wrapInputFd(Fd fd, kj::uint flags) { +#if _WIN32 + // KJ parity: on win32, LowLevelAsyncIoProvider::Fd is documented as a SOCKET (async-io.h: + // "On Windows, the `fd` parameter to each of these methods must be a SOCKET"), and kj's own + // win32 provider implements wrapInputFd/wrapOutputFd *identically* to wrapSocketFd + // (async-io-win32.c++: all three wrap the SOCKET in AsyncStreamFd) -- even kj's "pipes" on + // Windows are loopback-TCP socketpairs (newOsSocketpair). So there is no pipe-HANDLE tier to + // implement; delegate to the tested socket path. Validated by Windows CI. + return kj::heap(wrap_socket_fd(static_cast(prepareFd(fd, flags)))); +#else + return kj::heap(wrap_input_fd(prepareFd(fd, flags))); +#endif +} + +kj::Own TokioLowLevelAsyncIoProvider::wrapOutputFd(Fd fd, kj::uint flags) { +#if _WIN32 + // See wrapInputFd above: win32 Fd is a SOCKET and kj's win32 wrapOutputFd == wrapSocketFd. + // Validated by Windows CI. + return kj::heap(wrap_socket_fd(static_cast(prepareFd(fd, flags)))); +#else + return kj::heap(wrap_output_fd(prepareFd(fd, flags))); +#endif +} + +kj::Own TokioLowLevelAsyncIoProvider::wrapSocketFd(Fd fd, kj::uint flags) { + // `Fd` is int on unix and uintptr_t (SOCKET) on win32; the bridge carries it widened to + // int64 (a "raw socket handle") either way, with -1 == INVALID_SOCKET as the one sentinel. + return kj::heap(wrap_socket_fd(static_cast(prepareFd(fd, flags)))); +} + +kj::Promise> TokioLowLevelAsyncIoProvider::wrapConnectingSocketFd( + Fd fd, const struct sockaddr *addr, kj::uint addrlen, kj::uint flags) { + int64_t prepared = static_cast(prepareFd(fd, flags)); + // The Rust side takes an owned copy of the sockaddr: the caller's pointer need not outlive + // this call (KJ's own implementation copies too). + ::rust::Vec addrCopy; + addrCopy.reserve(addrlen); + const uint8_t *addrBytes = reinterpret_cast(addr); + for (kj::uint i = 0; i < addrlen; i++) { + addrCopy.push_back(addrBytes[i]); + } + return wrap_connecting_socket_fd(prepared, kj::mv(addrCopy)) + .then([](::rust::Box stream) -> kj::Own { + return kj::heap(kj::mv(stream)); + }); +} + +kj::Own TokioLowLevelAsyncIoProvider::wrapListenSocketFd( + Fd fd, NetworkFilter &filter, kj::uint flags) { + // `filter` applies to accepted connections (KJ parity); it must outlive the receiver. + return kj::heap( + wrap_listen_fd(static_cast(prepareFd(fd, flags))), filter); +} + +// ======================================================================================= +// TokioAsyncIoProvider / setup + +kj::AsyncIoProvider::PipeThread TokioAsyncIoProvider::newPipeThread( + kj::Function startFunc) { + KJ_UNIMPLEMENTED("kj-rs-io does not implement newPipeThread() (workerd does not use it)"); +} + +TokioAsyncIoContext setupTokioAsyncIo() { + auto base = kj_rs_tokio::setupTokioAsyncIo(); + auto &timer = base.getTimer(); + auto lowLevelProvider = kj::heap(timer); + auto provider = kj::heap(timer); + return TokioAsyncIoContext(kj::mv(base), kj::mv(lowLevelProvider), kj::mv(provider)); +} + +// ======================================================================================= +// Signals + +kj::Promise onSignal(int signum) { + // The bridged future is eager-by-default, so the tokio signal handler is registered as soon + // as the event loop runs, even if the caller parks the promise without awaiting it immediately. + return wait_for_signal(signum); +} + +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/async-io.h b/src/rust/cxx/kj-rs-io/async-io.h new file mode 100644 index 00000000000..b2573ee7fcb --- /dev/null +++ b/src/rust/cxx/kj-rs-io/async-io.h @@ -0,0 +1,292 @@ +#pragma once +// kj-rs-io: tokio-backed implementations of KJ's async I/O interfaces. +// +// Everything here wraps an opaque Rust object (a native tokio TcpStream/UnixStream/TcpListener/ +// address list) and implements the corresponding KJ interface by calling `async fn`s across the +// cxx bridge, which return kj::Promises. Design points: +// +// - All promises must be awaited on the thread owning the kj_rs_tokio::TokioEventPort: the +// tokio I/O driver that delivers readiness for these sockets only runs while that KJ loop +// sleeps in the port's wait()/poll(). +// - Cancellation: dropping a returned kj::Promise drops the underlying Rust future, which +// releases the socket's readiness interest. A stream with a canceled read remains usable. +// - Unwrap fast path: every Rust-originated stream can be recovered as its native tokio object +// (see unwrapTokioStream), so Rust servers can serve a connection natively +// instead of crossing the FFI per read. Foreign kj streams are not unwrappable. +// +// Known stubs (all throw UNIMPLEMENTED, documented per method): newPipeThread(), capability +// streams (SCM_RIGHTS fd passing), datagram sockets, and named-service / abstract-unix-socket +// address forms. restrictPeers() IS implemented (via PeerFilter, a port of KJ's NetworkFilter); see +// TokioNetwork for enforcement points and the parse-time-filtering deviation. + +#include "kj-rs-io/ffi.rs.h" +#include "kj-rs-io/peer-filter.h" +#include "kj-rs-tokio/tokio-event-port.h" + +#include +#include +#include + +namespace kj_rs_io { + +// A kj::AsyncIoStream backed by a native tokio TcpStream or UnixStream. +class TokioAsyncIoStream final: public kj::AsyncIoStream { + public: + explicit TokioAsyncIoStream(::rust::Box inner): inner(kj::mv(inner)) {} + + // AsyncInputStream. tryRead honors KJ's min-bytes contract: resolves with >= minBytes unless + // EOF is reached first (in which case the short count signals EOF). + kj::Promise tryRead(void *buffer, size_t minBytes, size_t maxBytes) override; + + // AsyncOutputStream. Both overloads have write-all semantics; the multi-piece overload is one + // bridged operation using vectored writes (writev), like KJ's own socket streams. + kj::Promise write(kj::ArrayPtr buffer) override; + kj::Promise write(kj::ArrayPtr> pieces) override; + + // Resolves when new writes are doomed (peer reset/hangup observed). Does not fire on a mere + // half-close (peer FIN), mirroring KJ. On non-Unix platforms the promise never resolves + // (KJ-on-Windows behavior). Safe to call multiple times concurrently. + kj::Promise whenWriteDisconnected() override; + + // AsyncIoStream. + void shutdownWrite() override; + void getsockopt(int level, int option, void *value, kj::uint *length) override; + void setsockopt(int level, int option, const void *value, kj::uint length) override; + void getsockname(struct sockaddr *addr, kj::uint *length) override; + void getpeername(struct sockaddr *addr, kj::uint *length) override; + kj::Maybe getFd() const override; +#if _WIN32 + // Validated by Windows CI; mirrors getFd(): on Windows the underlying socket is a winsock + // SOCKET, exposed as a void* handle (kj convention; getFd() returns none there). + kj::Maybe getWin32Handle() const override; +#endif + + // Unwrap fast path: moves the native tokio stream out, leaving this wrapper hollow (all + // further operations throw). Throws if I/O promises are in flight -- the Rust side tracks + // in-flight operations, so this is checked rather than a caller contract. Prefer the free + // function unwrapTokioStream() when holding only a kj::AsyncIoStream&. + ::rust::Box unwrap() { + return stream_take(*inner); + } + + private: + kj::Promise writePieces(kj::ArrayPtr> pieces); + + ::rust::Box inner; +}; + +// A kj::ConnectionReceiver backed by a native tokio TcpListener or UnixListener. Incoming +// connections from peers disallowed by `filter` (restrictPeers) are silently dropped and the +// accept loop continues, mirroring KJ. +class TokioConnectionReceiver final: public kj::ConnectionReceiver { + public: + // Receiver for one of our own networks/addresses: shares ownership of the PeerFilter chain, so + // there is no lifetime coupling to the network that created it. + TokioConnectionReceiver(::rust::Box inner, kj::Rc filter) + : inner(kj::mv(inner)), + filter(filter.toOwn()) {} + + // Receiver over a caller-provided filter (the kj::LowLevelAsyncIoProvider::wrapListenSocketFd + // entry point). KJ's interface hands the filter by reference and makes the caller responsible + // for keeping it alive for the receiver's lifetime, exactly as with KJ's native providers; we + // hold it as a kj::Own with kj::NullDisposer, KJ's idiom for an explicitly non-owning Own, so + // the receiver has exactly one filter member with one meaning rather than an owning handle and + // a reference that may alias it. + TokioConnectionReceiver( + ::rust::Box inner, kj::LowLevelAsyncIoProvider::NetworkFilter &filter) + : inner(kj::mv(inner)), + filter(&filter, kj::NullDisposer::instance) {} + + kj::Promise> accept() override; + kj::Promise acceptAuthenticated() override; + kj::uint getPort() override; + void getsockopt(int level, int option, void *value, kj::uint *length) override; + void setsockopt(int level, int option, const void *value, kj::uint length) override; + void getsockname(struct sockaddr *addr, kj::uint *length) override; + + private: + kj::Promise acceptImpl(bool authenticated); + + ::rust::Box inner; + // A share of our PeerFilter chain, or (wrapListenSocketFd) the caller's filter behind a + // NullDisposer -- see the constructors. + kj::Own filter; +}; + +// A kj::NetworkAddress holding pre-resolved socket addresses (DNS happens at parseAddress time, +// like KJ). connect() tries each address in order; listen() binds the first. +// +// connect() honors KJ's lifetime contract (NetworkAddressImpl::connect() in +// kj/async-io-unix.c++): the returned promise is a coroutine whose frame owns a copy of the +// resolved address list, so the caller may drop this NetworkAddress while the connect is still +// pending. +// +// `filter` is the restrictPeers filter chain of the kj::Network this address came from +// (allow-all for an unrestricted network); this address co-owns it, so it stays valid for this +// address and any promises it returns regardless of the network's lifetime. Filtering is +// enforced at connect() time per address ("connect() blocked by restrictPeers()", KJ parity) +// and at accept() time on listeners; unlike KJ, disallowed DNS results are not already dropped +// at parse time (they fail at connect instead). +class TokioNetworkAddress final: public kj::NetworkAddress { + public: + TokioNetworkAddress(::rust::Box inner, kj::Rc filter) + : inner(kj::mv(inner)), + filter(kj::mv(filter)) {} + + kj::Promise> connect() override; + kj::Own listen() override; + kj::Own clone() override; + kj::String toString() override; + + private: + ::rust::Box inner; + kj::Rc filter; +}; + +// The tokio-backed kj::Network. Supports the KJ address grammar subset workerd uses; see +// net.rs for the exact forms and documented deviations (no named services, no unix-abstract, +// no IPv6 scope IDs). +// +// restrictPeers() uses PeerFilter, a faithful port of KJ's NetworkFilter (semantics identical to +// kj::setupAsyncIo()'s networks). The returned network owns a share of this one's filter chain +// (kj::Rc), so derived networks, addresses, and receivers all remain valid regardless of which +// order the networks are destroyed in. Enforcement points: per-address connect()-time checks and +// accept()-time peer checks; parse-time DNS-result dropping is NOT implemented (blocked +// addresses fail at connect instead) -- see TokioNetworkAddress. +class TokioNetwork final: public kj::Network { + public: + // Allow-everything root network (matches KJ's root networks). + TokioNetwork(): filter(kj::rc()) {} + TokioNetwork(TokioNetwork &parent, + kj::ArrayPtr allow, + kj::ArrayPtr deny) + : filter(kj::rc(allow, deny, parent.filter.addRef())) {} + + kj::Promise> parseAddress( + kj::StringPtr addr, kj::uint portHint) override; + kj::Own getSockaddr(const void *sockaddr, kj::uint len) override; + kj::Own restrictPeers( + kj::ArrayPtr allow, kj::ArrayPtr deny) override; + + private: + kj::Rc filter; +}; + +// The tokio-backed kj::LowLevelAsyncIoProvider. Implements the socket-wrapping entry points on +// Unix and Windows (each wrap*Fd normalizes KJ's TAKE_OWNERSHIP/ALREADY_CLOEXEC/ALREADY_NONBLOCK +// flags, then hands an owned, non-blocking raw socket handle -- a Unix fd or a win32 SOCKET, +// widened to int64 -- to Rust). The pipe tier (wrapInputFd/wrapOutputFd) is Unix-only for now. +// wrapUnixSocketFd (capability streams) and wrapDatagramSocketFd keep their default-throwing +// implementations. +class TokioLowLevelAsyncIoProvider final: public kj::LowLevelAsyncIoProvider { + public: + explicit TokioLowLevelAsyncIoProvider(kj::Timer &timer): timer(timer) {} + + kj::Own wrapInputFd(Fd fd, kj::uint flags) override; + kj::Own wrapOutputFd(Fd fd, kj::uint flags) override; + kj::Own wrapSocketFd(Fd fd, kj::uint flags) override; + kj::Promise> wrapConnectingSocketFd( + Fd fd, const struct sockaddr *addr, kj::uint addrlen, kj::uint flags) override; + // `filter` applies to accepted connections (disallowed peers are dropped and the accept + // loop continues, like KJ); it must outlive the returned receiver. + kj::Own wrapListenSocketFd( + Fd fd, NetworkFilter &filter, kj::uint flags) override; + kj::Timer &getTimer() override { + return timer; + } + + private: + kj::Timer &timer; +}; + +// The tokio-backed kj::AsyncIoProvider. Pipes are KJ's in-memory pipes (port-agnostic, like +// kj::newOneWayPipe/newTwoWayPipe themselves); newPipeThread throws UNIMPLEMENTED (workerd does +// not use it); newCapabilityPipe keeps its default-throwing implementation. +class TokioAsyncIoProvider final: public kj::AsyncIoProvider { + public: + explicit TokioAsyncIoProvider(kj::Timer &timer): timer(timer) {} + + kj::OneWayPipe newOneWayPipe() override { + return kj::newOneWayPipe(); + } + kj::TwoWayPipe newTwoWayPipe() override { + return kj::newTwoWayPipe(); + } + kj::Network &getNetwork() override { + return network; + } + PipeThread newPipeThread( + kj::Function startFunc) + override; + kj::Timer &getTimer() override { + return timer; + } + + private: + TokioNetwork network; + kj::Timer &timer; +}; + +// Mirror of kj::AsyncIoContext (kj/async-io.h) for the tokio-backed loop: a drop-in replacement +// for kj::setupAsyncIo() at workerd.c++:1570. Composes kj_rs_tokio::TokioAsyncIoContext (which +// owns the event port, the kj::EventLoop and the kj::WaitScope, and orders their teardown) with +// the tokio-backed I/O providers. +// +// Teardown is member order: the providers (which borrow the port's timer) go first, then the +// base context -- spawned tasks cancelled while the WaitScope is alive, then WaitScope, then the +// port (loop, runtime, timer). I/O objects created *through* the providers (streams, listeners, +// addresses) must be destroyed before the context, as with kj::setupAsyncIo(). +struct TokioAsyncIoContext { + TokioAsyncIoContext(kj_rs_tokio::TokioAsyncIoContext base, + kj::Own lowLevelProvider, + kj::Own provider) + : base(kj::mv(base)), + lowLevelProvider(kj::mv(lowLevelProvider)), + provider(kj::mv(provider)) {} + // Same move rules as the base context (move-constructible for return-by-value; no + // move-assignment, which would bypass the base's teardown ordering). + TokioAsyncIoContext(TokioAsyncIoContext &&) = default; + TokioAsyncIoContext &operator=(TokioAsyncIoContext &&) = delete; + KJ_DISALLOW_COPY(TokioAsyncIoContext); + + kj_rs_tokio::TokioAsyncIoContext base; + kj::Own lowLevelProvider; + kj::Own provider; + + kj_rs_tokio::TokioEventPort &getPort() { + return base.getPort(); + } + kj::EventLoop &getLoop() { + return base.getLoop(); + } + kj::WaitScope &getWaitScope() { + return base.getWaitScope(); + } + kj::Timer &getTimer() { + return base.getTimer(); + } + kj::Network &getNetwork() { + return provider->getNetwork(); + } + kj::AsyncIoProvider &getProvider() { + return *provider; + } + kj::LowLevelAsyncIoProvider &getLowLevelProvider() { + return *lowLevelProvider; + } +}; + +// Sets up the current thread with a tokio-driven KJ event loop plus tokio-backed I/O providers: +// the kj::setupAsyncIo() equivalent for the tokio loop. One per thread. +TokioAsyncIoContext setupTokioAsyncIo(); + +// Resolves when the process receives signal `signum`: the tokio-loop replacement for +// kj::UnixEventPort::onSignal() (workerd's SIGTERM graceful drain). Must be awaited on the +// thread owning the TokioEventPort. Unlike UnixEventPort, KJ does not block/capture the signal +// beforehand: the tokio handler is registered when the promise is first polled, so a signal +// delivered before the event loop first runs takes its default disposition (see signal.rs). +// On Windows, SIGTERM/SIGINT are mapped to the ctrl_shutdown/ctrl_c console control events; +// the promise rejects for other signums. +kj::Promise onSignal(int signum); + +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/error.rs b/src/rust/cxx/kj-rs-io/error.rs new file mode 100644 index 00000000000..bf38a5d3791 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/error.rs @@ -0,0 +1,243 @@ +//! Error mapping: `std::io::Error` -> `kj::Exception`, preserving KJ's exception-type taxonomy. +//! +//! kj-http and capnp RPC change behavior based on `kj::Exception::Type` (e.g. `DISCONNECTED` +//! failures are treated as clean peer hangups rather than bugs), so the mapping of connection +//! errors matters for behavioral parity with `kj::setupAsyncIo()`. + +use cxx::IntoKjException; +use cxx::KjError; +use cxx::KjException; +use cxx::KjExceptionType; + +pub type Result = std::result::Result; + +/// An `std::io::Error` (plus operation context) that converts into a `kj::Exception` with an +/// appropriate exception type. +#[derive(Debug)] +pub struct KjIoError { + /// Name of the failing operation, included in the exception description the way KJ's + /// `KJ_SYSCALL` includes the syscall name (e.g. "`connect()`: Connection refused ..."). + op: &'static str, + inner: std::io::Error, +} + +impl KjIoError { + pub(crate) fn other(op: &'static str, message: impl std::fmt::Display) -> Self { + Self { + op, + inner: std::io::Error::other(message.to_string()), + } + } +} + +/// Attaches an operation name to `io::Error`s, for use with `Result::map_err`. +pub fn op(name: &'static str) -> impl Fn(std::io::Error) -> KjIoError { + move |inner| KjIoError { op: name, inner } +} + +fn exception_type(error: &std::io::Error) -> KjExceptionType { + use std::io::ErrorKind; + // Primary classification: by raw errno, mirroring KJ's own table (`typeOfErrno()` in + // kj/debug.c++) errno-for-errno. Consumers (kj-http, capnp-rpc) change behavior on the + // exception type, so the classes must match `kj::setupAsyncIo()` exactly — e.g. ETIMEDOUT + // is OVERLOADED in KJ (retry-later), NOT DISCONNECTED (clean peer hangup), and std's + // `ErrorKind` buckets have no stable kinds at all for KJ's fd/memory-exhaustion OVERLOADED + // set (EMFILE/ENFILE/ENOBUFS/...), hence the raw match. + #[cfg(unix)] + if let Some(errno) = error.raw_os_error() { + return errno_exception_type(errno); + } + // Fallback for synthetic (non-OS) errors — and all errors on Windows, where + // `raw_os_error()` is a Win32/WSA code, not an errno (KJ classifies those in + // `typeOfWin32Error()`; the buckets below agree with it for the kinds tokio surfaces). + match error.kind() { + // KJ's DISCONNECTED class: connection teardown, treated as a clean peer hangup. + ErrorKind::ConnectionRefused + | ErrorKind::ConnectionReset + | ErrorKind::ConnectionAborted + | ErrorKind::BrokenPipe + | ErrorKind::NotConnected + | ErrorKind::UnexpectedEof + | ErrorKind::HostUnreachable + | ErrorKind::NetworkUnreachable + | ErrorKind::NetworkDown => KjExceptionType::Disconnected, + // KJ's OVERLOADED class: temporary lack of resources (ETIMEDOUT/WSAETIMEDOUT and + // ENOMEM land here in KJ's tables). + ErrorKind::TimedOut | ErrorKind::OutOfMemory => KjExceptionType::Overloaded, + ErrorKind::Unsupported => KjExceptionType::Unimplemented, + _ => KjExceptionType::Failed, + } +} + +/// Exact mirror of KJ's `typeOfErrno()` (kj/debug.c++), so `kj::Exception::Type` matches the +/// native `kj::setupAsyncIo()` backend errno-for-errno. +#[cfg(unix)] +fn errno_exception_type(errno: i32) -> KjExceptionType { + // Errnos that are `#ifdef`-conditional in KJ's table for platform reasons, mirrored here + // with `cfg`: ENONET exists only on Linux; EOPNOTSUPP aliases ENOTSUP on Linux (KJ compiles + // its case only `#if EOPNOTSUPP != ENOTSUP` — an or-pattern with both would be an + // unreachable pattern there). + #[cfg(any(target_os = "linux", target_os = "android"))] + if errno == libc::ENONET { + return KjExceptionType::Disconnected; + } + #[cfg(not(any(target_os = "linux", target_os = "android")))] + if errno == libc::EOPNOTSUPP { + return KjExceptionType::Unimplemented; + } + match errno { + // OVERLOADED: the call failed because of a temporary lack of resources. + libc::EDQUOT + | libc::EMFILE + | libc::ENFILE + | libc::ENOBUFS + | libc::ENOLCK + | libc::ENOMEM + | libc::ENOSPC + | libc::ETIMEDOUT + | libc::EUSERS => KjExceptionType::Overloaded, + // DISCONNECTED: communication over a connection that has been lost. + libc::ENOTCONN + | libc::ECONNABORTED + | libc::ECONNREFUSED + | libc::ECONNRESET + | libc::EHOSTDOWN + | libc::EHOSTUNREACH + | libc::ENETDOWN + | libc::ENETRESET + | libc::ENETUNREACH + | libc::EPIPE => KjExceptionType::Disconnected, + // UNIMPLEMENTED: the "not supported" family (ENOTSOCK is really "syscall not + // implemented for non-sockets", per KJ's own comment). + libc::ENOSYS | libc::ENOTSUP | libc::ENOPROTOOPT | libc::ENOTSOCK => { + KjExceptionType::Unimplemented + } + _ => KjExceptionType::Failed, + } +} + +impl From for KjError { + fn from(error: KjIoError) -> Self { + let description = format!("{}: {}", error.op, error.inner); + Self::new(exception_type(&error.inner), description) + } +} + +impl IntoKjException for KjIoError { + fn into_kj_exception(self, file: &str, line: u32) -> KjException { + KjError::from(self).into_kj_exception(file, line) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn kind_error(kind: std::io::ErrorKind) -> std::io::Error { + std::io::Error::new(kind, "synthetic") + } + + #[test] + fn errorkind_fallback_matches_kj_classes() { + use std::io::ErrorKind; + for kind in [ + ErrorKind::ConnectionRefused, + ErrorKind::ConnectionReset, + ErrorKind::ConnectionAborted, + ErrorKind::BrokenPipe, + ErrorKind::NotConnected, + ErrorKind::UnexpectedEof, + ] { + assert_eq!( + exception_type(&kind_error(kind)), + KjExceptionType::Disconnected, + "{kind:?}" + ); + } + assert_eq!( + exception_type(&kind_error(ErrorKind::TimedOut)), + KjExceptionType::Overloaded + ); + assert_eq!( + exception_type(&kind_error(ErrorKind::OutOfMemory)), + KjExceptionType::Overloaded + ); + assert_eq!( + exception_type(&kind_error(ErrorKind::Unsupported)), + KjExceptionType::Unimplemented + ); + assert_eq!( + exception_type(&kind_error(ErrorKind::Other)), + KjExceptionType::Failed + ); + assert_eq!( + exception_type(&kind_error(ErrorKind::InvalidInput)), + KjExceptionType::Failed + ); + } + + /// The raw-errno table must mirror KJ's `typeOfErrno()` errno-for-errno, since consumers + /// (kj-http, capnp-rpc) change behavior on the class. Spot-checks one member of every + /// class plus the classic confusables (ETIMEDOUT is OVERLOADED, not DISCONNECTED). + #[cfg(unix)] + #[test] + fn errno_table_matches_kj() { + let of = |errno: i32| exception_type(&std::io::Error::from_raw_os_error(errno)); + // OVERLOADED + for errno in [ + libc::EMFILE, + libc::ENFILE, + libc::ENOBUFS, + libc::ENOMEM, + libc::ETIMEDOUT, + ] { + assert_eq!(of(errno), KjExceptionType::Overloaded, "errno {errno}"); + } + // DISCONNECTED + for errno in [ + libc::ECONNREFUSED, + libc::ECONNRESET, + libc::EPIPE, + libc::ENOTCONN, + libc::EHOSTUNREACH, + libc::ENETDOWN, + ] { + assert_eq!(of(errno), KjExceptionType::Disconnected, "errno {errno}"); + } + // UNIMPLEMENTED + for errno in [ + libc::ENOSYS, + libc::ENOTSUP, + libc::ENOPROTOOPT, + libc::ENOTSOCK, + ] { + assert_eq!(of(errno), KjExceptionType::Unimplemented, "errno {errno}"); + } + // FAILED (everything else) + for errno in [libc::EINVAL, libc::EACCES, libc::EBADF, libc::EEXIST] { + assert_eq!(of(errno), KjExceptionType::Failed, "errno {errno}"); + } + // Platform-conditional entries mirror KJ's #ifdefs. + #[cfg(any(target_os = "linux", target_os = "android"))] + assert_eq!(of(libc::ENONET), KjExceptionType::Disconnected); + #[cfg(not(any(target_os = "linux", target_os = "android")))] + assert_eq!(of(libc::EOPNOTSUPP), KjExceptionType::Unimplemented); + } + + #[test] + fn description_is_op_colon_message() { + let err = KjIoError::other("connect()", "boom"); + let kj = KjError::from(err); + assert_eq!(kj.description(), "connect(): boom"); + assert_eq!(kj.exception_type(), KjExceptionType::Failed); + + let err = op("read()")(kind_error(std::io::ErrorKind::ConnectionReset)); + let kj = KjError::from(err); + assert!( + kj.description().starts_with("read(): "), + "{}", + kj.description() + ); + assert_eq!(kj.exception_type(), KjExceptionType::Disconnected); + } +} diff --git a/src/rust/cxx/kj-rs-io/ffi.rs b/src/rust/cxx/kj-rs-io/ffi.rs new file mode 100644 index 00000000000..05abb117af6 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/ffi.rs @@ -0,0 +1,1073 @@ +//! The FFI island of kj-rs-io: the `#[cxx::bridge]` wire plus the crate's hand-written `unsafe` +//! boundary, in one dedicated file (file-top `#![allow(unsafe_code)]`). +//! +//! It holds: +//! +//! - the `#[cxx::bridge] mod bridge` (namespace `kj_rs_io`) — the cxx-generated C++ <-> Rust wire, +//! re-exported as `crate::ffi::*`; and +//! - every hand-written `unsafe` the macro does not generate, so the serve / net / stream modules +//! can carry the crate-root `#![deny(unsafe_code)]` and be *compiler-proven* free of hand-written +//! unsafe. Three kinds of raw thing cross the FFI boundary and are laundered into safe Rust types +//! here: +//! +//! 1. **Raw OS socket handles** arriving from C++ as an `i64` (a Unix fd or a win32 `SOCKET`; +//! see [`own_socket_from_raw`], the one platform conversion point) — the C++ side has +//! already normalized KJ's `TAKE_OWNERSHIP` / `ALREADY_CLOEXEC` flags, dup'ing when not +//! transferring ownership — plus [`dup_raw_fd`] (dup a borrowed fd into an owned one; the +//! unix-only handle tier of [`take_kj_socket`]) and [`own_fd_from_raw`] (the unix-only +//! pipe tier's `i32` fds). +//! 2. **`struct sockaddr` bytes** crossing in both directions — [`sockaddr_to_bytes`] / +//! [`sockaddr_from_bytes`]. +//! 3. **Raw `getsockopt(2)` / `setsockopt(2)`** — the socket-option passthrough behind +//! `kj::AsyncIoStream` / `kj::ConnectionReceiver` carries caller-owned option buffers whose +//! length semantics (socklen in/out) no safe std/socket2 API expresses, so the raw syscalls +//! live here ([`stream_getsockopt`] and friends). +//! +//! Because this file opts back into `unsafe` (`#![allow(unsafe_code)]`), it is the one module in +//! the crate whose soundness must be audited by hand; the rest is enforced by the compiler. +#![allow(unsafe_code)] + +use core::pin::Pin; + +/// Opaque binding of `kj::AsyncIoStream`, and the cxx-bridged operations on it (shared-receiver +/// shims; safe to call). Re-exported as `crate::ffi::*` so the rest of the crate (and the +/// crate-root re-exports) keep using `ffi::`. +pub use bridge::KjAsyncIoStream; +pub use bridge::KjPieces; +pub use bridge::kj_piece; +pub use bridge::kj_pieces_count; +pub use bridge::kj_stream_get_handle; +pub use bridge::unwrap_tokio_stream; +use cxx::KjException; +use kj_rs::KjOwn; + +use crate::error::Result; +use crate::error::op; +use crate::net::TokioAddress; +use crate::net::TokioListener; +use crate::net::address_clone; +use crate::net::address_connect_index; +use crate::net::address_count; +use crate::net::address_listen; +use crate::net::address_raw_sockaddr; +use crate::net::address_to_string; +use crate::net::listener_accept; +use crate::net::listener_local_addr; +use crate::net::listener_port; +use crate::net::network_get_sockaddr; +use crate::net::network_parse_address; +use crate::net::wrap_connecting_socket_fd; +use crate::net::wrap_listen_fd; +use crate::net::wrap_socket_fd; +use crate::readiness::TokioFdWatcher; +use crate::readiness::new_fd_watcher; +use crate::signal::wait_for_signal; +use crate::stream::TokioInputFd; +use crate::stream::TokioOutputFd; +use crate::stream::TokioStream; +use crate::stream::input_fd_try_read; +use crate::stream::output_fd_write; +use crate::stream::stream_local_addr; +use crate::stream::stream_peer_addr; +use crate::stream::stream_raw_handle; +use crate::stream::stream_shutdown_write; +use crate::stream::stream_take; +use crate::stream::stream_try_raw_handle; +use crate::stream::stream_try_read; +use crate::stream::stream_when_write_disconnected; +use crate::stream::stream_write; +use crate::stream::stream_write_pieces; +use crate::stream::wrap_input_fd; +use crate::stream::wrap_output_fd; + +#[cxx::bridge(namespace = "kj_rs_io")] +// FFI island: the cxx bridge macro generates the `unsafe` extern shims, and this module declares +// the hand-written `unsafe extern "C++"` / `async unsafe fn` bridge surface. +// unnecessary_box_returns: returning an opaque Rust type to C++ as `Box` is the cxx idiom. +#[expect(clippy::unnecessary_box_returns)] +// missing_safety_doc fires (or not) deep inside the macro expansion depending on which bridge +// items are publicly re-exported, so an `#[expect]` could go unfulfilled. +#[expect(clippy::allow_attributes)] +#[allow(clippy::missing_safety_doc)] +mod bridge { + extern "Rust" { + type TokioStream; + type TokioListener; + type TokioAddress; + type TokioInputFd; + type TokioOutputFd; + + // ================================================================================== + // Streams (TCP or Unix domain, behind kj::AsyncIoStream) + + /// Reads until at least `min_bytes` are available (or EOF), up to `buf.len()`. Returns + /// the number of bytes read; fewer than `min_bytes` indicates EOF. The kj `tryRead` + /// contract. + async unsafe fn stream_try_read<'a>( + stream: &'a TokioStream, + buf: &'a mut [u8], + min_bytes: usize, + ) -> Result; + + /// Writes the entire buffer (write-all semantics). + async unsafe fn stream_write<'a>(stream: &'a TokioStream, buf: &'a [u8]) -> Result<()>; + + /// Writes every piece, in order, with write-all semantics, using vectored writes + /// (`writev`) so a multi-piece `kj::AsyncOutputStream::write()` is one bridged + /// operation and as few syscalls as the kernel allows. + async unsafe fn stream_write_pieces<'a>( + stream: &'a TokioStream, + pieces: &'a KjPieces, + ) -> Result<()>; + + /// Resolves when the stream has become disconnected such that new writes will fail. + /// See `TokioStream::when_write_disconnected` for the mechanism and platform caveats. + async unsafe fn stream_when_write_disconnected<'a>(stream: &'a TokioStream) -> Result<()>; + + /// `shutdown(SHUT_WR)`: cleanly shut down the write end, keeping the read end open. + fn stream_shutdown_write(stream: &TokioStream) -> Result<()>; + + /// The underlying raw OS socket handle (fd on unix, `SOCKET` on windows) as an `i64`, + /// backing `kj::AsyncIoStream::getFd()` / `getWin32Handle()`. + fn stream_raw_handle(stream: &TokioStream) -> Result; + + /// Raw `struct sockaddr` bytes of the socket's locally-bound address (the + /// `getsockname()` passthrough). + fn stream_local_addr(stream: &TokioStream) -> Result>; + + /// Raw `struct sockaddr` bytes of the connected peer's address (the `getpeername()` + /// passthrough, also used by the accept-loop peer-filter check). + fn stream_peer_addr(stream: &TokioStream) -> Result>; + + /// `getsockopt(2)` on the underlying socket. `value.len()` is the caller's in-length + /// (the kernel truncates the option value to it); returns the syscall's reported + /// out-length, which the caller must mirror back exactly (raw socklen in/out + /// semantics). + fn stream_getsockopt( + stream: &TokioStream, + level: i32, + option: i32, + value: &mut [u8], + ) -> Result; + + /// `setsockopt(2)` on the underlying socket. + fn stream_setsockopt( + stream: &TokioStream, + level: i32, + option: i32, + value: &[u8], + ) -> Result<()>; + + /// Moves the native stream out, leaving `stream` hollow (all further ops error). Fails + /// (rather than aliasing live borrows) if any I/O future is in flight on `stream`: the + /// `TokioStream` tracks in-flight operations itself, so this is a checked operation, + /// not a caller contract. + fn stream_take(stream: &TokioStream) -> Result>; + /// Like `stream_raw_handle`, but -1 instead of an error when the wrapper is hollow (for + /// the `kj::Maybe`-returning `getFd()`/`getWin32Handle()`). + fn stream_try_raw_handle(stream: &TokioStream) -> i64; + + // ================================================================================== + // Network addresses (kj::Network::parseAddress grammar subset) + + /// Parses a KJ address string ("1.2.3.4:80", "[::1]:80", "host:80", "*", "*:80", + /// "unix:/path"), resolving hostnames via DNS. `port_hint` fills in a missing port. + /// (Owned `String`: the future must not borrow the caller's buffer across the DNS + /// suspension.) + async fn network_parse_address(addr: String, port_hint: u16) -> Result>; + + /// Builds an address from a raw `struct sockaddr` (AF_INET / AF_INET6 / AF_UNIX). + fn network_get_sockaddr(sockaddr: &[u8]) -> Result>; + + /// Connects to exactly the `index`th resolved address (no fallback). The C++ side + /// drives the try-each-address loop so it can apply restrictPeers() filtering per + /// address (KJ parity). + async unsafe fn address_connect_index<'a>( + addr: &'a TokioAddress, + index: usize, + ) -> Result>; + + /// Number of resolved socket addresses behind this address (>= 1). + fn address_count(addr: &TokioAddress) -> usize; + + /// Raw `struct sockaddr` bytes of the `index`th resolved address, for C++-side + /// kj::_::NetworkFilter (restrictPeers) checks. + fn address_raw_sockaddr(addr: &TokioAddress, index: usize) -> Result>; + + /// Binds + listens on the (first) address. Wildcard addresses bind dual-stack. + fn address_listen(addr: &TokioAddress) -> Result>; + + fn address_clone(addr: &TokioAddress) -> Box; + fn address_to_string(addr: &TokioAddress) -> String; + + // ================================================================================== + // Listeners (kj::ConnectionReceiver) + + async unsafe fn listener_accept<'a>( + listener: &'a TokioListener, + ) -> Result>; + + /// The locally-bound port (0 for Unix domain sockets, mirroring KJ). + fn listener_port(listener: &TokioListener) -> Result; + + /// Raw `struct sockaddr` bytes of the listener's bound address (the `getsockname()` + /// passthrough). + fn listener_local_addr(listener: &TokioListener) -> Result>; + + /// `getsockopt(2)` on the listening socket; same length semantics as + /// `stream_getsockopt`. + fn listener_getsockopt( + listener: &TokioListener, + level: i32, + option: i32, + value: &mut [u8], + ) -> Result; + + /// `setsockopt(2)` on the listening socket. + fn listener_setsockopt( + listener: &TokioListener, + level: i32, + option: i32, + value: &[u8], + ) -> Result<()>; + + // ================================================================================== + // Socket-handle wrapping (kj::LowLevelAsyncIoProvider). + // + // The `i64` is a raw OS socket handle: a Unix fd or a win32 `SOCKET` (`i64` fits both + // losslessly, with `-1` ≡ `INVALID_SOCKET` as the shared sentinel). All of these take + // ownership of the handle. The C++ side normalizes + // TAKE_OWNERSHIP/ALREADY_CLOEXEC/ALREADY_NONBLOCK flags (dup'ing when not taking + // ownership) before calling in; [`own_socket_from_raw`] is the single point where the + // raw handle becomes an owned socket. + + /// Wraps a connected stream socket handle (TCP or Unix domain, detected automatically). + fn wrap_socket_fd(handle: i64) -> Result>; + + /// Wraps a bound+listening socket handle (TCP or Unix domain, detected automatically). + fn wrap_listen_fd(handle: i64) -> Result>; + + /// Wraps an unconnected TCP socket handle and connects it to `sockaddr` (a raw + /// `struct sockaddr`, AF_INET/AF_INET6 only; owned copy, since the caller's pointer + /// need not outlive the call). + async fn wrap_connecting_socket_fd( + handle: i64, + sockaddr: Vec, + ) -> Result>; + + /// Wraps a readable fd (pipe, character device, socket). Regular files are rejected by + /// the OS readiness API (same as KJ's epoll-based provider). Unix only (the pipe tier + /// keeps `i32` fds). + fn wrap_input_fd(fd: i32) -> Result>; + + async unsafe fn input_fd_try_read<'a>( + stream: &'a TokioInputFd, + buf: &'a mut [u8], + min_bytes: usize, + ) -> Result; + + /// Wraps a writable fd (pipe, character device, socket). + fn wrap_output_fd(fd: i32) -> Result>; + + async unsafe fn output_fd_write<'a>(stream: &'a TokioOutputFd, buf: &'a [u8]) + -> Result<()>; + + // ================================================================================== + // Signals (kj::UnixEventPort::onSignal replacement; see signal.rs for semantics) + + /// Resolves when the process receives signal `signum` (on Windows: the mapped + /// SIGTERM/SIGINT console control event). + async fn wait_for_signal(signum: i32) -> Result<()>; + + // ================================================================================== + // Fd readiness (kj::UnixEventPort::FdObserver::whenBecomesReadable replacement, + // backing kj_rs_io::FileWatcher in file-watcher.h; see readiness.rs for semantics) + + /// A readiness watcher over a notification fd (inotify / kqueue). Owns its own dup of + /// the fd and a single registration with the I/O driver, so there is no "keep the fd + /// open" or "register once" contract for C++ to uphold. Unix only. + type TokioFdWatcher; + /// Create a watcher for `fd`: the fd is only borrowed for the duration of this call + /// (to dup it); the watcher owns the dup. + fn new_fd_watcher(fd: i32) -> Result>; + /// Resolves when the fd becomes readable (readiness already pending is reported + /// immediately). Multiple concurrent callers are fine. + async unsafe fn readable<'a>(self: &'a TokioFdWatcher) -> Result<()>; + } + + unsafe extern "C++" { + include!("kj-rs-io/unwrap.h"); + + /// The pieces of a `kj::AsyncOutputStream::write(pieces)` call (unwrap.h), read through + /// the two accessors below. Owned by the C++ coroutine frame awaiting + /// `stream_write_pieces`, so the borrow the bridged future holds is always valid. + type KjPieces; + #[cxx_name = "kjPiecesCount"] + fn kj_pieces_count(pieces: &KjPieces) -> usize; + /// The `index`th piece. Borrowed from `pieces`; `index < kj_pieces_count(pieces)`. + #[cxx_name = "kjPiece"] + fn kj_piece<'a>(pieces: &'a KjPieces, index: usize) -> &'a [u8]; + + /// `kj::AsyncIoStream`, opaque. Used by [`unwrap_kj_stream`]. + #[namespace = "kj"] + #[cxx_name = "AsyncIoStream"] + type KjAsyncIoStream; + + /// Implemented in `async-io.c++`: downcasts to the kj-rs-io wrapper and moves the native + /// stream out. Throws (surfaced as `Err`) for foreign streams. + #[cxx_name = "unwrapTokioStream"] + fn unwrap_tokio_stream(stream: Pin<&mut KjAsyncIoStream>) -> Result>; + + // Bridged operations on a foreign `kj::AsyncIoStream`, backing `serve_kj_stream`'s + // duplex-pump fallback (serve.rs). Shared receivers (`&KjAsyncIoStream`, const_cast + // shims in unwrap.h): a kj two-way stream supports one concurrent read and one write, + // which the pump models as concurrent shared borrows of the stream it owns. All + // returned futures must be polled on the KJ event-loop thread owning the stream. + + /// Corresponds to `kj::AsyncIoStream::tryRead(buffer, min_bytes, buffer.len())`. + #[cxx_name = "kjStreamTryRead"] + async fn kj_stream_try_read( + stream: &KjAsyncIoStream, + buffer: &mut [u8], + min_bytes: usize, + ) -> Result; + + /// Corresponds to `kj::AsyncIoStream::write(buffer)` (write-all semantics). + #[cxx_name = "kjStreamWrite"] + async fn kj_stream_write(stream: &KjAsyncIoStream, buffer: &[u8]) -> Result<()>; + + /// Corresponds to `kj::AsyncIoStream::shutdownWrite()`. `Result`: the C++ side can + /// throw (e.g. `shutdown(2)` on an already-reset socket), and a C++ exception crossing a + /// non-`Result` shim would abort the process. + #[cxx_name = "kjStreamShutdownWrite"] + fn kj_stream_shutdown_write(stream: &KjAsyncIoStream) -> Result<()>; + + /// The stream's underlying raw OS socket handle (fd on unix, `SOCKET` on windows; + /// `kj::AsyncIoStream::getFd()` / `getWin32Handle()`) as an `i64`, or -1 if it exposes + /// none. Backs the handle tier of [`take_kj_socket`]. + #[cxx_name = "kjStreamGetHandle"] + fn kj_stream_get_handle(stream: &KjAsyncIoStream) -> i64; + } +} + +// ====================================================================================== +// Raw socket handles / file descriptors. + +/// Materializes an owned file descriptor from a raw `i32` that arrived across the FFI bridge +/// (the unix-only pipe tier — `wrap_input_fd`/`wrap_output_fd`; sockets go through +/// [`own_socket_from_raw`]). +/// +/// Callable from safe code: the invariant it relies on (`fd` is open and its ownership has been +/// transferred to us) is structurally upheld by the cxx bridge — the C++ side normalizes KJ's +/// `TAKE_OWNERSHIP` / `ALREADY_CLOEXEC` flags and dup's the fd when the caller is not handing +/// over ownership. The returned `OwnedFd` becomes the sole owner and closes it on drop. +#[cfg(unix)] +#[must_use] +pub fn own_fd_from_raw(fd: i32) -> std::os::fd::OwnedFd { + use std::os::fd::FromRawFd; + // `OwnedFd`'s invariant is "an open fd, never -1" (-1 is its niche), so a negative value + // here would be library-level UB rather than an error. C++ callers normalize through + // prepareFd, but its all-flags-set path (TAKE_OWNERSHIP|ALREADY_CLOEXEC|ALREADY_NONBLOCK) + // performs no syscall that would catch a bad fd — enforce the contract at THE conversion + // point instead of inheriting the UB. + assert!(fd >= 0, "invalid fd crossed the FFI bridge: {fd}"); + // Safety: per the bridge contract `fd` is open and owned by us from this point on. + unsafe { std::os::fd::OwnedFd::from_raw_fd(fd) } +} + +/// Materializes an owned socket from a raw OS socket handle that arrived across the FFI bridge +/// as an `i64`: a Unix fd here, a win32 `SOCKET` in the `cfg(windows)` twin below. This is THE +/// one platform conversion point — behind it everything is a uniform `socket2::Socket` / +/// std/tokio socket type. +/// +/// Callable from safe code: the invariant it relies on (`handle` is an open socket whose +/// ownership has been transferred to us) is structurally upheld by the cxx bridge — the C++ +/// side normalizes KJ's fd-wrapping flags, dup'ing when the caller is not handing over +/// ownership. The returned socket becomes the sole owner and closes it on drop. +#[cfg(unix)] +#[must_use] +pub fn own_socket_from_raw(handle: i64) -> socket2::Socket { + // A unix fd is a non-negative int: the bridge widened it losslessly to i64, so the + // narrowing back to i32 cannot truncate for any legitimate handle. Enforce that (rejecting + // -1/garbage) here at THE conversion point rather than inheriting `OwnedFd`'s niche UB. + assert!( + (0..=i64::from(i32::MAX)).contains(&handle), + "invalid socket fd crossed the FFI bridge: {handle}" + ); + #[expect(clippy::cast_possible_truncation)] + let fd = handle as i32; + socket2::Socket::from(own_fd_from_raw(fd)) +} + +/// The `cfg(windows)` twin of [`own_socket_from_raw`]: the raw handle is a winsock `SOCKET` +/// (`u64`-shaped `RawSocket`; a live SOCKET fits in an `i64` without colliding with the -1 +/// sentinel, which is `INVALID_SOCKET` and never crosses the bridge as an owned handle). +// Validated by Windows CI; mirrors the unix arm. +#[cfg(windows)] +#[must_use] +pub fn own_socket_from_raw(handle: i64) -> socket2::Socket { + use std::os::windows::io::FromRawSocket; + use std::os::windows::io::OwnedSocket; + use std::os::windows::io::RawSocket; + // Live SOCKET values are non-negative in i64 (the -1 sentinel is INVALID_SOCKET, which + // `OwnedSocket` forbids as its niche and which must never cross the bridge as an owned + // handle) — enforce at THE conversion point rather than inheriting the niche UB. + assert!( + handle >= 0, + "invalid SOCKET crossed the FFI bridge: {handle}" + ); + // The bridge carries the SOCKET's bits verbatim. + #[allow(clippy::cast_sign_loss)] + let raw = handle as RawSocket; + // Safety: per the bridge contract `handle` is an open SOCKET owned by us from this point on. + let owned = unsafe { OwnedSocket::from_raw_socket(raw) }; + socket2::Socket::from(owned) +} + +/// Duplicates a *borrowed* raw fd into an independently-owned fd (`F_DUPFD_CLOEXEC`), for the +/// handle tier of [`take_kj_socket`]: the kj stream keeps its own fd, we get a fresh dup. +/// +/// Unix only, deliberately: no windows twin is needed. The handle tier only fires for *foreign* +/// handle-backed kj streams, and under the all-rust mode on Windows every socket-backed stream +/// originates in kj-rs-io (tier-1 unwrap); the other in-process dup users don't dup on windows +/// either (`when_write_disconnected` is unix-only — never-resolving on windows, KJ parity — and +/// windows `shutdown_write` borrows via `SockRef` instead of dup'ing). If a windows twin is +/// ever needed, `std::os::windows::io::BorrowedSocket::try_clone_to_owned` +/// (`WSADuplicateSocketW`) is the same-process equivalent. +/// +/// # Errors +/// +/// Returns the `dup()` `io::Error` (mapped to a `kj::Exception`) if the syscall fails. +#[cfg(unix)] +pub fn dup_raw_fd(fd: i32) -> Result { + // Safety: the caller owns whatever `fd` belongs to (take_kj_socket holds the kj stream; + // new_fd_watcher is called from FileWatcher::Impl's constructor, which owns the fd) and so + // keeps it open for the duration of the call; we immediately dup it into an + // independently-owned fd and never touch `fd` again. + unsafe { std::os::fd::BorrowedFd::borrow_raw(fd) } + .try_clone_to_owned() + .map_err(op("dup()")) +} + +// ====================================================================================== +// `struct sockaddr` <-> bytes. + +/// Copies a `socket2::SockAddr`'s initialized `struct sockaddr` bytes into an owned `Vec`, to +/// hand across the bridge for the C++ side's `kj::_::NetworkFilter` (restrictPeers) checks. +#[must_use] +pub fn sockaddr_to_bytes(sockaddr: &socket2::SockAddr) -> Vec { + // Safety: as_ptr()/len() delimit an initialized sockaddr owned by `sockaddr`. + let bytes = unsafe { + std::slice::from_raw_parts(sockaddr.as_ptr().cast::(), sockaddr.len() as usize) + }; + bytes.to_vec() +} + +/// Decodes raw `struct sockaddr` bytes (arriving from C++) into a `socket2::SockAddr`. +/// +/// # Errors +/// +/// Errors if the byte length is too short to hold a family, exceeds `sockaddr_storage`, or (on +/// unix) is shorter than the family's own address struct -- an `AF_INET` in four bytes is +/// garbage, not an address. +#[cfg(any(unix, windows))] +pub fn sockaddr_from_bytes(bytes: &[u8]) -> Result { + use crate::error::KjIoError; + + let mut storage = socket2::SockAddrStorage::zeroed(); + let storage_size = std::mem::size_of::(); + // Every sockaddr starts with its family (on BSDs preceded by a length byte); anything + // shorter than that header cannot even be classified. + #[cfg(unix)] + let header_size = std::mem::offset_of!(libc::sockaddr, sa_data); + #[cfg(not(unix))] + let header_size = std::mem::size_of::(); + if bytes.len() < header_size || bytes.len() > storage_size { + return Err(KjIoError::other("sockaddr", "invalid sockaddr length")); + } + // Safety: SockAddrStorage is plain-old-data large enough for any sockaddr; we copy + // `bytes.len() <= size_of::()` bytes into it. + unsafe { + std::ptr::copy_nonoverlapping( + bytes.as_ptr(), + std::ptr::from_mut(&mut storage).cast::(), + bytes.len(), + ); + } + #[expect(clippy::cast_possible_truncation)] + let len = bytes.len() as socket2::socklen_t; + // Safety: `storage` is a zeroed sockaddr_storage with the caller's `len` bytes copied in, + // satisfying SockAddr::new's layout/length requirements. Reading a known family's struct + // out of it below is always in bounds (storage is full-size and zero-filled), so a + // length/family mismatch is a wrong address, never an out-of-bounds read; it is rejected + // as an error just after. Families socket2 does not understand (e.g. AF_NETLINK) make the + // accessors `as_socket()`/`as_pathname()` return `None`, and callers surface that as an + // "unsupported sockaddr family" error (see `net.rs::network_get_sockaddr`). + let addr = unsafe { socket2::SockAddr::new(storage, len) }; + #[cfg(unix)] + { + let min_len = match i32::from(addr.family()) { + libc::AF_INET => std::mem::size_of::(), + libc::AF_INET6 => std::mem::size_of::(), + libc::AF_UNIX => std::mem::offset_of!(libc::sockaddr_un, sun_path), + _ => 0, + }; + if bytes.len() < min_len { + return Err(KjIoError::other( + "sockaddr", + format!( + "sockaddr too short for its family: {} bytes, family {} needs at least {min_len}", + bytes.len(), + addr.family() + ), + )); + } + } + Ok(addr) +} + +// ====================================================================================== +// Raw `getsockopt(2)` / `setsockopt(2)`. +// +// The socket-option passthrough behind `kj::AsyncIoStream::get/setsockopt` and +// `kj::ConnectionReceiver::get/setsockopt`. The option buffer is caller-owned opaque bytes with +// raw socklen in/out semantics (the caller's buffer may be smaller than the option value, and the +// syscall's reported length must be surfaced verbatim), which no safe std/socket2 API expresses — +// so the raw syscalls are declared and called here, in the unsafe island. + +/// Raw `getsockopt(2)` on a borrowed socket fd. `value.len()` is passed as the in `optlen` (the +/// kernel truncates the option value to it); the syscall's reported out `optlen` is returned so +/// the C++ caller can mirror `*length = socklen` exactly as `KJ_SYSCALL(::getsockopt(...))` did. +#[cfg(unix)] +fn getsockopt_raw( + fd: std::os::fd::BorrowedFd<'_>, + level: i32, + option: i32, + value: &mut [u8], +) -> Result { + use core::ffi::c_int; + use core::ffi::c_void; + use std::os::fd::AsRawFd; + unsafe extern "C" { + fn getsockopt( + sockfd: c_int, + level: c_int, + optname: c_int, + optval: *mut c_void, + optlen: *mut socket2::socklen_t, + ) -> c_int; + } + #[expect(clippy::cast_possible_truncation)] + let mut optlen = value.len() as socket2::socklen_t; + // Safety: simple syscall wrapper. `fd` is a live socket fd (borrowed from the tokio object + // for the duration of the call); `value.as_mut_ptr()` with in-`optlen == value.len()` + // delimits writable caller memory the kernel fills (never past `optlen`); `&raw mut optlen` + // is a valid in/out pointer for the call. + let rc = unsafe { + getsockopt( + fd.as_raw_fd(), + level, + option, + value.as_mut_ptr().cast::(), + &raw mut optlen, + ) + }; + if rc != 0 { + return Err(op("getsockopt()")(std::io::Error::last_os_error())); + } + Ok(optlen as usize) +} + +/// Raw `setsockopt(2)` on a borrowed socket fd. +#[cfg(unix)] +fn setsockopt_raw( + fd: std::os::fd::BorrowedFd<'_>, + level: i32, + option: i32, + value: &[u8], +) -> Result<()> { + use core::ffi::c_int; + use core::ffi::c_void; + use std::os::fd::AsRawFd; + unsafe extern "C" { + fn setsockopt( + sockfd: c_int, + level: c_int, + optname: c_int, + optval: *const c_void, + optlen: socket2::socklen_t, + ) -> c_int; + } + #[expect(clippy::cast_possible_truncation)] + let optlen = value.len() as socket2::socklen_t; + // Safety: simple syscall wrapper. `fd` is a live socket fd (borrowed from the tokio object + // for the duration of the call); `value.as_ptr()` with `optlen == value.len()` delimits + // readable caller memory the kernel only reads. + let rc = unsafe { + setsockopt( + fd.as_raw_fd(), + level, + option, + value.as_ptr().cast::(), + optlen, + ) + }; + if rc != 0 { + return Err(op("setsockopt()")(std::io::Error::last_os_error())); + } + Ok(()) +} + +/// Raw ws2_32 `getsockopt` on a borrowed `SOCKET`. Same socklen in/out semantics as the unix +/// arm above: `value.len()` is passed as the in `optlen`, and the reported out `optlen` is +/// returned verbatim. +// Validated by Windows CI; mirrors the unix arm. +#[cfg(windows)] +fn getsockopt_raw( + sock: std::os::windows::io::BorrowedSocket<'_>, + level: i32, + option: i32, + value: &mut [u8], +) -> Result { + use core::ffi::c_char; + use core::ffi::c_int; + use std::os::windows::io::AsRawSocket; + use std::os::windows::io::RawSocket; + #[link(name = "ws2_32")] + unsafe extern "system" { + fn getsockopt( + s: RawSocket, + level: c_int, + optname: c_int, + optval: *mut c_char, + optlen: *mut c_int, + ) -> c_int; + } + #[allow(clippy::cast_possible_truncation, clippy::cast_possible_wrap)] + let mut optlen = value.len() as c_int; + // Safety: simple syscall wrapper. `sock` is a live `SOCKET` (borrowed from the tokio object + // for the duration of the call); `value.as_mut_ptr()` with in-`optlen == value.len()` + // delimits writable caller memory winsock fills (never past `optlen`); `&raw mut optlen` is + // a valid in/out pointer for the call. + let rc = unsafe { + getsockopt( + sock.as_raw_socket(), + level, + option, + value.as_mut_ptr().cast::(), + &raw mut optlen, + ) + }; + if rc != 0 { + // rc is SOCKET_ERROR (-1); `last_os_error()` reads `WSAGetLastError()` on Windows. + return Err(op("getsockopt()")(std::io::Error::last_os_error())); + } + // The out-length winsock reports is non-negative (and bounded by the in-length). + #[allow(clippy::cast_sign_loss)] + let reported = optlen as usize; + Ok(reported) +} + +/// Raw ws2_32 `setsockopt` on a borrowed `SOCKET`. +// Validated by Windows CI; mirrors the unix arm. +#[cfg(windows)] +fn setsockopt_raw( + sock: std::os::windows::io::BorrowedSocket<'_>, + level: i32, + option: i32, + value: &[u8], +) -> Result<()> { + use core::ffi::c_char; + use core::ffi::c_int; + use std::os::windows::io::AsRawSocket; + use std::os::windows::io::RawSocket; + #[link(name = "ws2_32")] + unsafe extern "system" { + fn setsockopt( + s: RawSocket, + level: c_int, + optname: c_int, + optval: *const c_char, + optlen: c_int, + ) -> c_int; + } + #[allow(clippy::cast_possible_truncation, clippy::cast_possible_wrap)] + let optlen = value.len() as c_int; + // Safety: simple syscall wrapper. `sock` is a live `SOCKET` (borrowed from the tokio object + // for the duration of the call); `value.as_ptr()` with `optlen == value.len()` delimits + // readable caller memory winsock only reads. + let rc = unsafe { + setsockopt( + sock.as_raw_socket(), + level, + option, + value.as_ptr().cast::(), + optlen, + ) + }; + if rc != 0 { + // rc is SOCKET_ERROR (-1); `last_os_error()` reads `WSAGetLastError()` on Windows. + return Err(op("setsockopt()")(std::io::Error::last_os_error())); + } + Ok(()) +} + +pub fn stream_getsockopt( + stream: &TokioStream, + level: i32, + option: i32, + value: &mut [u8], +) -> Result { + #[cfg(unix)] + { + stream.with_borrowed_fd(|fd| getsockopt_raw(fd, level, option, value))? + } + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + { + stream.with_borrowed_socket(|sock| getsockopt_raw(sock, level, option, value))? + } + #[cfg(not(any(unix, windows)))] + { + let _ = (stream, level, option, value); + Err(crate::error::KjIoError::other( + "getsockopt", + "not implemented by kj-rs-io on this platform", + )) + } +} + +pub fn stream_setsockopt( + stream: &TokioStream, + level: i32, + option: i32, + value: &[u8], +) -> Result<()> { + #[cfg(unix)] + { + stream.with_borrowed_fd(|fd| setsockopt_raw(fd, level, option, value))? + } + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + { + stream.with_borrowed_socket(|sock| setsockopt_raw(sock, level, option, value))? + } + #[cfg(not(any(unix, windows)))] + { + let _ = (stream, level, option, value); + Err(crate::error::KjIoError::other( + "setsockopt", + "not implemented by kj-rs-io on this platform", + )) + } +} + +pub fn listener_getsockopt( + listener: &TokioListener, + level: i32, + option: i32, + value: &mut [u8], +) -> Result { + #[cfg(unix)] + { + getsockopt_raw(listener.as_borrowed_fd(), level, option, value) + } + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + { + getsockopt_raw(listener.as_borrowed_socket(), level, option, value) + } + #[cfg(not(any(unix, windows)))] + { + let _ = (listener, level, option, value); + Err(crate::error::KjIoError::other( + "getsockopt", + "not implemented by kj-rs-io on this platform", + )) + } +} + +pub fn listener_setsockopt( + listener: &TokioListener, + level: i32, + option: i32, + value: &[u8], +) -> Result<()> { + #[cfg(unix)] + { + setsockopt_raw(listener.as_borrowed_fd(), level, option, value) + } + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + { + setsockopt_raw(listener.as_borrowed_socket(), level, option, value) + } + #[cfg(not(any(unix, windows)))] + { + let _ = (listener, level, option, value); + Err(crate::error::KjIoError::other( + "setsockopt", + "not implemented by kj-rs-io on this platform", + )) + } +} + +// ====================================================================================== +// Typed read/write halves of a pumped `kj::AsyncIoStream`. +// +// kj's stream contract — at most one read and one write may be in flight at once — is prose in +// kj; these halves make the borrow checker enforce it. Each half's operations take `&mut self`, +// so an in-flight operation's future exclusively borrows its half (a second overlapping read is +// a compile error), and `split_kj_stream` takes the owner's `&mut`, so while the halves live +// nothing else (an unwrap, another split) can touch the stream. The bridged operations behind +// them are not re-exported: the halves are the only way to drive a foreign stream. + +/// The read direction of a pumped stream. See the module comment above. +pub struct KjStreamReadHalf<'a>(&'a KjAsyncIoStream); + +/// The write direction of a pumped stream (writes and the write-side shutdown). See the module +/// comment above. +pub struct KjStreamWriteHalf<'a>(&'a KjAsyncIoStream); + +/// Splits the owned stream into its two directions. Holding the owner's `&mut` for the halves' +/// lifetime proves exactly one pair exists and reserves the stream for them. +// The unused `&mut` is the point (see the doc comment): it reserves the stream for the halves. +#[expect(clippy::needless_pass_by_ref_mut)] +pub fn split_kj_stream( + stream: &mut KjOwn, +) -> (KjStreamReadHalf<'_>, KjStreamWriteHalf<'_>) { + let stream = &**stream; + (KjStreamReadHalf(stream), KjStreamWriteHalf(stream)) +} + +impl KjStreamReadHalf<'_> { + /// `kj::AsyncIoStream::tryRead(buffer, min_bytes, buffer.len())`. + // The unused `&mut self` is the point: an in-flight read's future exclusively borrows the + // read half (kj's one-read-in-flight contract), see the section comment above. + #[expect(clippy::needless_pass_by_ref_mut)] + pub(crate) async fn try_read( + &mut self, + buf: &mut [u8], + min_bytes: usize, + ) -> std::result::Result { + bridge::kj_stream_try_read(self.0, buf, min_bytes).await + } +} + +impl KjStreamWriteHalf<'_> { + /// `kj::AsyncIoStream::write(buffer)` (write-all semantics). + // The unused `&mut self` is the point: an in-flight write's future exclusively borrows the + // write half (kj's one-write-in-flight contract), see the section comment above. + #[expect(clippy::needless_pass_by_ref_mut)] + pub(crate) async fn write(&mut self, buf: &[u8]) -> std::result::Result<(), KjException> { + bridge::kj_stream_write(self.0, buf).await + } + + /// `kj::AsyncIoStream::shutdownWrite()`. + // The unused `&mut self` is the point: an exclusive borrow of the write half serializes + // write-side operations (kj's one-write-in-flight contract), see the section comment above. + #[expect(clippy::needless_pass_by_ref_mut)] + pub(crate) fn shutdown_write(&mut self) -> std::result::Result<(), KjException> { + bridge::kj_stream_shutdown_write(self.0) + } +} + +// ====================================================================================== +// Borrow-based unwrap entry point (`Pin<&mut kj::AsyncIoStream>`). Safe: the in-flight-I/O +// conflict it must avoid is detected by TokioStream's RefCell guard (see stream.rs). +// +// The owning native-serve entry points (`take_kj_socket`, `serve_kj_stream`) are safe fns in +// [`crate::serve`] — ownership arrives as a `KjOwn` and no raw pointer crosses the crate's +// public surface. Only the borrow-based unwrap remains here: C++ keeps the (hollow) wrapper, +// so its "no I/O in flight" precondition cannot be expressed structurally. + +/// Recovers the native [`TokioStream`] out of a `kj::AsyncIoStream` that was created by +/// kj-rs-io, leaving the C++ wrapper hollow (any further I/O through it fails). +/// +/// # Errors +/// +/// Returns an error if the stream is not a kj-rs-io tokio-backed stream, was already unwrapped, +/// or has I/O operations (reads, writes, `whenWriteDisconnected`) in flight -- their futures +/// borrow the native object this function moves out, and `TokioStream` tracks those borrows, +/// so the conflict is detected rather than being a caller contract. +pub fn unwrap_kj_stream( + stream: Pin<&mut KjAsyncIoStream>, +) -> std::result::Result, KjException> { + bridge::unwrap_tokio_stream(stream) +} + +#[cfg(test)] +mod tests { + #[cfg(unix)] + use cxx::KjError; + + use super::*; + + #[test] + fn sockaddr_round_trips_v4_and_v6() { + for text in ["1.2.3.4:80", "[::1]:443", "[fe80::1]:0"] { + let addr: std::net::SocketAddr = text.parse().unwrap(); + let original = socket2::SockAddr::from(addr); + let bytes = sockaddr_to_bytes(&original); + assert_eq!(bytes.len(), original.len() as usize); + let decoded = sockaddr_from_bytes(&bytes).unwrap(); + assert_eq!(decoded.as_socket(), Some(addr), "{text}"); + } + } + + #[cfg(unix)] + #[test] + fn sockaddr_round_trips_unix_paths() { + let original = socket2::SockAddr::unix("/tmp/kj-rs-io-test.sock").unwrap(); + let bytes = sockaddr_to_bytes(&original); + let decoded = sockaddr_from_bytes(&bytes).unwrap(); + assert!(decoded.is_unix()); + assert_eq!( + decoded.as_pathname(), + Some(std::path::Path::new("/tmp/kj-rs-io-test.sock")) + ); + } + + #[test] + fn sockaddr_from_bytes_rejects_bad_lengths() { + assert!(sockaddr_from_bytes(&[]).is_err()); + assert!( + sockaddr_from_bytes(&[0u8]).is_err(), + "shorter than a family" + ); + let too_long = vec![0u8; std::mem::size_of::() + 1]; + assert!(sockaddr_from_bytes(&too_long).is_err()); + } + + /// A family whose struct does not fit in the given length is garbage, not an address: a + /// truncated `sockaddr_in`, or an `AF_INET6` claimed in `sockaddr_in`'s size, must be rejected + /// rather than decoded out of the zero-filled tail of the storage. + #[cfg(unix)] + #[test] + fn sockaddr_from_bytes_rejects_family_length_mismatch() { + let v4: std::net::SocketAddr = "1.2.3.4:80".parse().unwrap(); + let v4_bytes = sockaddr_to_bytes(&socket2::SockAddr::from(v4)); + // Truncated sockaddr_in (family present, address cut off). + let truncated = &v4_bytes[..std::mem::size_of::() - 1]; + let err = KjError::from(sockaddr_from_bytes(truncated).unwrap_err()); + assert!( + err.description().contains("too short for its family"), + "{}", + err.description() + ); + + // An AF_INET6 family in only sockaddr_in's worth of bytes. + let v6: std::net::SocketAddr = "[::1]:443".parse().unwrap(); + let v6_bytes = sockaddr_to_bytes(&socket2::SockAddr::from(v6)); + let short_v6 = &v6_bytes[..std::mem::size_of::()]; + assert!(sockaddr_from_bytes(short_v6).is_err()); + + // The exact struct sizes are accepted. + assert!(sockaddr_from_bytes(&v4_bytes[..std::mem::size_of::()]).is_ok()); + assert!( + sockaddr_from_bytes(&v6_bytes[..std::mem::size_of::()]).is_ok() + ); + + // A unix sockaddr needs at least its header (family + sun_path offset). + let un = sockaddr_to_bytes(&socket2::SockAddr::unix("/tmp/x").unwrap()); + assert!( + sockaddr_from_bytes(&un[..std::mem::offset_of!(libc::sockaddr_un, sun_path) - 1]) + .is_err() + ); + } + + /// Randomized: `sockaddr_from_bytes` must never panic on arbitrary input, and whatever it + /// accepts must be self-consistent (family and length agree; accessors do not read past + /// the given length's worth of meaning). Seeded xorshift, so a failure is reproducible. + #[cfg(unix)] + #[test] + fn sockaddr_from_bytes_never_panics_on_random_input() { + let mut state: u64 = 0x9e37_79b9_7f4a_7c15; + let mut next = move || { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + state + }; + let max = std::mem::size_of::() + 4; + for _ in 0..20_000 { + #[expect(clippy::cast_possible_truncation)] + let len = (next() % (max as u64 + 1)) as usize; + let mut bytes: Vec = (0..len).map(|_| (next() & 0xff) as u8).collect(); + // Bias the family field toward the interesting ones half of the time. + if len >= 2 && next() % 2 == 0 { + let fam = [ + libc::AF_INET, + libc::AF_INET6, + libc::AF_UNIX, + libc::AF_UNSPEC, + ][(next() % 4) as usize]; + #[expect(clippy::cast_possible_truncation, clippy::cast_sign_loss)] + let fam = fam as socket2::sa_family_t; + let off = std::mem::offset_of!(libc::sockaddr, sa_family); + let fam_bytes = fam.to_ne_bytes(); + bytes[off..off + fam_bytes.len().min(len - off)] + .copy_from_slice(&fam_bytes[..fam_bytes.len().min(len - off)]); + } + if let Ok(addr) = sockaddr_from_bytes(&bytes) { + let min_len = match i32::from(addr.family()) { + libc::AF_INET => std::mem::size_of::(), + libc::AF_INET6 => std::mem::size_of::(), + libc::AF_UNIX => std::mem::offset_of!(libc::sockaddr_un, sun_path), + _ => 0, + }; + assert!( + len >= min_len, + "accepted {len} bytes for family {}", + addr.family() + ); + // Accessors must be safe to call on anything accepted. + let _ = addr.as_socket(); + let _ = addr.as_pathname(); + let _ = sockaddr_to_bytes(&addr); + } + } + } + + #[cfg(unix)] + #[test] + #[should_panic(expected = "invalid fd crossed the FFI bridge")] + fn own_fd_from_raw_rejects_negative_fds() { + // -1 is OwnedFd's niche: turning it into an OwnedFd would be library UB, so the single + // conversion point must refuse it loudly (a panic here becomes a kj::Exception). + let _ = own_fd_from_raw(-1); + } + + #[cfg(unix)] + #[test] + #[should_panic(expected = "invalid socket fd crossed the FFI bridge")] + fn own_socket_from_raw_rejects_negative_handles() { + let _ = own_socket_from_raw(-1); + } + + #[cfg(unix)] + #[test] + #[should_panic(expected = "invalid socket fd crossed the FFI bridge")] + fn own_socket_from_raw_rejects_handles_that_do_not_fit_an_fd() { + let _ = own_socket_from_raw(i64::from(i32::MAX) + 1); + } + + /// The happy path of THE conversion point: an fd released by std becomes a socket2 socket + /// that owns it (closes it on drop) and is fully usable. + #[cfg(unix)] + #[test] + fn own_socket_from_raw_takes_ownership_of_a_live_socket() { + use std::os::fd::AsRawFd; + use std::os::fd::IntoRawFd; + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + let port = listener.local_addr().unwrap().port(); + let raw = listener.into_raw_fd(); + let socket = own_socket_from_raw(i64::from(raw)); + assert_eq!(socket.as_raw_fd(), raw); + assert_eq!( + socket.local_addr().unwrap().as_socket().unwrap().port(), + port + ); + // `socket` is the sole owner: dropping it closes the fd (socket2::Socket's drop glue). + } +} diff --git a/src/rust/cxx/kj-rs-io/file-watcher.c++ b/src/rust/cxx/kj-rs-io/file-watcher.c++ new file mode 100644 index 00000000000..3e469f14ff8 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/file-watcher.c++ @@ -0,0 +1,228 @@ +// kj_rs_io::FileWatcher implementation. Each platform backend is a line-for-line port of +// workerd's kj-mode FileWatcher (workerd.c++), with the one difference that waiting for the +// notification fd to become readable goes through tokio's AsyncFd (TokioFdWatcher, see +// readiness.rs) instead of kj::UnixEventPort::FdObserver::whenBecomesReadable(). Everything +// else -- which fds get created, which watch masks are used, how events are drained and +// filtered -- is kept identical so that --watch behaves the same on either event loop. + +#include "kj-rs-io/file-watcher.h" + +#include "kj-rs-io/ffi.rs.h" + +#include +#include +#include +#include +#include + +#include + +#if __linux__ +#include +#include +#include +#elif __APPLE__ || __FreeBSD__ || __OpenBSD__ || __NetBSD__ || __DragonFly__ +#define KJ_RS_IO_USE_KQUEUE_FOR_FILE_WATCHER 1 +#include +#include +#include +#include +#include +#endif + +namespace kj_rs_io { + +#if __linux__ + +// inotify backend. Watches each file's parent directory (IN_DELETE | IN_MODIFY | IN_MOVE | +// IN_CREATE) and filters events by basename, so files replaced by rename (editors' atomic +// saves) or deleted-and-recreated keep firing, and the watched file itself need not exist yet. +struct FileWatcher::Impl: public kj::Refcounted { + kj::OwnFd inotifyFd; + // Owns a dup of `inotifyFd` and its one registration with the tokio I/O driver (see + // readiness.rs): declared after the fd it was created from, destroyed before it. + ::rust::Box watcher; + + kj::HashMap watches; + kj::HashMap> filesWatched; + + Impl() + : inotifyFd(KJ_SYSCALL_FD(inotify_init1(IN_NONBLOCK | IN_CLOEXEC))), + watcher(new_fd_watcher(inotifyFd.get())) {} + + bool isSupported() { + return true; + } + + void watch(kj::PathPtr path, kj::Maybe file) { + // The inotify backend doesn't use `file`; it watches the parent directory. + + auto pathStr = path.parent().toNativeString(true); + + int wd = watches.findOrCreate(pathStr, [&]() { + int wd; + uint32_t mask = IN_DELETE | IN_MODIFY | IN_MOVE | IN_CREATE; + KJ_SYSCALL(wd = inotify_add_watch(inotifyFd, pathStr.cStr(), mask)); + return decltype(watches)::Entry{kj::mv(pathStr), wd}; + }); + + auto &files = + filesWatched.findOrCreate(wd, [&]() { return decltype(filesWatched)::Entry{wd, {}}; }); + + files.upsert(kj::str(path.basename()[0]), [](auto &&...) {}); + } + + // `self` is this Impl's own refcount, held by the coroutine frame: the frame reads `this` + // across every co_await, so it keeps its referent alive itself rather than relying on the + // FileWatcher outliving the promise. + kj::Promise onChange(kj::Own self) { + kj::byte buffer[4096]{}; + + for (;;) { + ssize_t n; + KJ_NONBLOCKING_SYSCALL(n = read(inotifyFd, buffer, sizeof(buffer))); + + if (n < 0) { + // No more data to read: wait for the inotify fd to become readable again. + co_await watcher->readable(); + continue; + } + + kj::byte *ptr = buffer; + while (n > 0) { + KJ_ASSERT(n >= sizeof(struct inotify_event)); + + auto &event = *reinterpret_cast(ptr); + size_t eventSize = sizeof(struct inotify_event) + event.len; + KJ_ASSERT(n >= eventSize); + KJ_ASSERT(eventSize % sizeof(void *) == 0); + ptr += eventSize; + n -= eventSize; + + if (event.len > 0 && event.name[0] != '\0') { + auto &watched = KJ_ASSERT_NONNULL(filesWatched.find(event.wd)); + if (watched.find(kj::StringPtr(event.name)) != kj::none) { + // HIT! We saw a change. + co_return; + } + } + } + } + } +}; + +#elif KJ_RS_IO_USE_KQUEUE_FOR_FILE_WATCHER + +// kqueue backend. One EVFILT_VNODE registration per watched file (dup of the already-open +// config fd when available, else opened by path -- so the path must exist). NOTE_DELETE / +// NOTE_RENAME on the old inode cover atomic-rename saves. kqueue doesn't scale to whole +// directory trees, but we only watch the specific files opened while parsing the config. +struct FileWatcher::Impl: public kj::Refcounted { + kj::OwnFd kqueueFd; + // Owns a dup of `kqueueFd` and its one registration with the tokio I/O driver (see + // readiness.rs): declared after the fd it was created from, destroyed before it. + ::rust::Box watcher; + kj::Vector filesWatched; + + Impl(): kqueueFd(makeKqueue()), watcher(new_fd_watcher(kqueueFd.get())) {} + + bool isSupported() { + return true; + } + + void watch(kj::PathPtr path, kj::Maybe file) { + KJ_IF_SOME(f, file) { + KJ_IF_SOME(fd, f.getFd()) { + // We need to duplicate the fd because the original will probably be closed later, and + // closing the fd unregisters it from kqueue. + watchFd(KJ_SYSCALL_FD(dup(fd))); + return; + } + } + + // No existing file, open from disk. + watchFd(KJ_SYSCALL_FD(open(path.toNativeString(true).cStr(), O_RDONLY))); + } + + // `self` is this Impl's own refcount, held by the coroutine frame (see the inotify backend). + kj::Promise onChange(kj::Own self) { + for (;;) { + struct kevent event; + struct timespec timeout; + memset(&event, 0, sizeof(event)); + memset(&timeout, 0, sizeof(timeout)); + + int n; + KJ_SYSCALL(n = kevent(kqueueFd, nullptr, 0, &event, 1, &timeout)); + + if (n == 0) { + // No events: wait for the kqueue fd to become readable, indicating an event has been + // delivered. + co_await watcher->readable(); + continue; + } else { + // We only registered for events that indicate changes in the first place, so there's + // no need to examine the event: it definitely means something changed. + co_return; + } + } + } + + static kj::OwnFd makeKqueue() { + auto fd = KJ_SYSCALL_FD(kqueue()); + KJ_SYSCALL(fcntl(fd, F_SETFD, FD_CLOEXEC)); + return kj::mv(fd); + } + + void watchFd(kj::OwnFd fd) { + KJ_SYSCALL(fcntl(fd, F_SETFD, FD_CLOEXEC)); + + struct kevent change; + memset(&change, 0, sizeof(change)); + change.ident = fd.get(); + change.filter = EVFILT_VNODE; + change.flags = EV_ADD | EV_CLEAR; + change.fflags = NOTE_WRITE | NOTE_EXTEND | NOTE_DELETE | NOTE_RENAME; + KJ_SYSCALL(kevent(kqueueFd, &change, 1, nullptr, 0, nullptr)); + filesWatched.add(kj::mv(fd)); + } +}; + +#else + +// Dummy backend for platforms without an implementation (Windows, ...), mirroring workerd's: +// isSupported() returns false, which workerd surfaces as a clean CLI error for --watch +// ("File watching is not yet implemented on your OS") rather than a crash. A real Windows +// backend (e.g. ReadDirectoryChangesW, perhaps via the notify crate) is a potential follow-up. +struct FileWatcher::Impl: public kj::Refcounted { + bool isSupported() { + return false; + } + + void watch(kj::PathPtr path, kj::Maybe file) {} + + kj::Promise onChange(kj::Own) { + return kj::NEVER_DONE; + } +}; + +#endif + +FileWatcher::FileWatcher(): impl(kj::refcounted()) {} +FileWatcher::~FileWatcher() noexcept(false) = default; + +bool FileWatcher::isSupported() { + return impl->isSupported(); +} + +void FileWatcher::watch(kj::PathPtr path, kj::Maybe file) { + impl->watch(path, file); +} + +kj::Promise FileWatcher::onChange() { + // The coroutine frame co-owns the Impl (see Impl::onChange), so the returned promise stays + // valid even if this FileWatcher is destroyed first: no outlive-the-watcher contract. + return impl->onChange(kj::addRef(*impl)); +} + +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/file-watcher.h b/src/rust/cxx/kj-rs-io/file-watcher.h new file mode 100644 index 00000000000..21689a806a2 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/file-watcher.h @@ -0,0 +1,57 @@ +#pragma once +// kj_rs_io::FileWatcher: the tokio-loop replacement for workerd's `--watch` file watcher. +// +// Watches a set of individual files and resolves onChange() when any of them changes. The +// platform backends mirror workerd's kj FileWatcher exactly — inotify on the parent directory +// on Linux, kqueue EVFILT_VNODE per open file on macOS/BSD — but the notification fd's +// readiness is awaited through tokio's AsyncFd (kj_rs_io::wait_fd_readable) instead of +// kj::UnixEventPort::FdObserver, so it works on the tokio-backed event loop where no +// UnixEventPort exists. +// +// Behavior notes (all matching the kj version): +// - Multiple rapid changes coalesce: onChange() resolves once for whatever is queued; calling +// it again drains the queue before waiting, so changes are never lost between calls. +// - Linux: the watched file itself need not exist (only its parent directory must), and a +// file deleted and re-created is picked up again. macOS/BSD: watch() opens the file (or +// dups the provided already-open fd), so watching a nonexistent file throws; a +// replaced-by-rename file still fires on the old inode. +// - All internal fds are CLOEXEC: --watch reloads via execve(), which must not leak them. +// - Unsupported platforms (Windows, ...): isSupported() returns false, watch() is a no-op and +// onChange() never resolves, mirroring workerd's dummy watcher. +// +// onChange() must be awaited on the thread owning the kj_rs_tokio::TokioEventPort, and at most +// one onChange() promise may be outstanding at a time (workerd awaits it sequentially). The +// promise co-owns the watcher's state, so it may outlive the FileWatcher object itself. + +#include +#include + +namespace kj_rs_io { + +class FileWatcher { + public: + FileWatcher(); + ~FileWatcher() noexcept(false); + KJ_DISALLOW_COPY_AND_MOVE(FileWatcher); + + // False on platforms with no watcher implementation (callers should report an error). + bool isSupported(); + + // Adds `path` to the watched set. `file`, if provided, is an already-open handle for the + // same path (the kqueue backend watches it directly via a dup'd fd; the inotify backend + // ignores it and watches the parent directory by name). + void watch(kj::PathPtr path, kj::Maybe file); + + // Resolves the next time any watched file changes (immediately, if a change is already + // queued). Eagerly evaluated, per kj-rs-io convention for I/O promises. The promise co-owns + // the watcher's state (the notification fd, its I/O-driver registration, the watched set), so + // destroying the FileWatcher while it is pending is safe: the state lives until the promise + // settles or is dropped. + kj::Promise onChange(); + + private: + struct Impl; + kj::Own impl; +}; + +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/lib.rs b/src/rust/cxx/kj-rs-io/lib.rs new file mode 100644 index 00000000000..680246299d3 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/lib.rs @@ -0,0 +1,111 @@ +//! Rust half of kj-rs-io: tokio-backed implementations of KJ's async I/O interfaces. +//! +//! The C++ side (`async-io.h`) implements `kj::AsyncIoStream`, `kj::ConnectionReceiver`, +//! `kj::NetworkAddress`, `kj::Network`, `kj::AsyncIoProvider` and `kj::LowLevelAsyncIoProvider` +//! as thin wrappers over the opaque Rust types in this crate. All async operations are plain +//! `async fn`s bridged to `kj::Promise` by workerd-cxx; dropping the promise drops the Rust +//! future, which releases any tokio readiness interest (cancellation is implicit). +//! +//! Every future returned from this crate must be polled on the thread that owns the +//! `kj_rs_tokio::TokioEventPort` runtime: tokio I/O objects register with that runtime's I/O +//! driver, which is only driven while the KJ event loop sleeps inside the port's +//! `wait()`/`poll()`. +//! +//! A Rust-originated stream wrapped as `kj::AsyncIoStream` can be recovered as its native +//! tokio object so Rust servers can drive the connection without crossing the +//! FFI per read: [`unwrap_kj_stream`] from Rust, `kj_rs_io::unwrapTokioStream()` from C++. +//! Foreign streams fail to unwrap with a `kj::Exception`. The native-serve entry points +//! ([`serve_kj_stream`], [`take_kj_socket`]) build on the [`serve`] module's `ServeIo` / pump +//! machinery. +//! +//! # Object relationships +//! +//! C++ owns the KJ-facing objects; each holds an opaque Rust object by `rust::Box`: +//! +//! ```text +//! TokioAsyncIoContext (C++) -- kj::setupAsyncIo() analogue: composes +//! │ kj_rs_tokio::TokioAsyncIoContext (port -> loop, +//! │ runtime, timer; WaitScope) with the providers below; +//! │ teardown is member order (providers, then the base) +//! ├── kj_rs_tokio::TokioAsyncIoContext -- the loop's tokio runtime (see kj-rs-tokio) +//! ├── TokioLowLevelAsyncIoProvider -- wrap*Fd(): owned fd/SOCKET as i64 -> Rust owns it +//! └── TokioAsyncIoProvider +//! └── TokioNetwork -- Rc (restrictPeers chain; children +//! │ share ownership, nothing outlives anything by +//! │ convention) +//! ├── TokioNetworkAddress -- Box + Rc share +//! │ └── TokioConnectionReceiver -- Box + filter +//! └── TokioAsyncIoStream -- Box +//! +//! TokioStream (this crate) -- RefCell>: every I/O op holds a +//! │ shared borrow across its await; take() needs the +//! │ exclusive borrow, so unwrapping with I/O in flight +//! │ is an Err, not aliasing. None = hollow wrapper. +//! └── Inner::Tcp / Inner::Unix -- the native tokio socket +//! +//! serve_kj_stream(KjOwn) -> ServedKjStream +//! ├── native path: unwrap -> ServeIo::Tcp/Unix, hollow wrapper destroyed +//! └── pump path: ServeIo::Duplex (consumer end) + StreamPump (!Send) owning the KjOwn +//! and the other duplex end, polled on the KJ thread +//! +//! FileWatcher::Impl (C++) -- owns the inotify/kqueue fd, parses events +//! └── Box -- owns a dup of that fd + one I/O-driver registration +//! ``` + +// Safety & panic enforcement walls. Test code exempted. +// +// `unsafe` is quarantined into a single named FFI island: the crate root denies `unsafe_code`, so +// the serve / net / stream / error / runtime / readiness / signal business logic is +// *compiler-proven* free of hand-written unsafe. The one island that opts back in via +// `#![allow(unsafe_code)]` is `ffi.rs`: it holds the `#[cxx::bridge] mod bridge` (re-exported as +// `crate::ffi::*`) plus all the fd / sockaddr laundering. The public surface has no `unsafe fn` +// at all: `unwrap_kj_stream` is a safe fn (the in-flight-I/O conflict it used to leave to the +// caller is detected on the Rust side), and the owning entry points (`take_kj_socket`, +// `serve_kj_stream`) are safe fns in `serve.rs`: ownership arrives as a `KjOwn`, the pump drives +// it through compiler-checked shared borrows (shared-receiver shims, see unwrap.h), and no raw +// pointer crosses the public surface. +#![deny(unsafe_op_in_unsafe_fn)] +#![deny(unsafe_code)] +#![deny(clippy::undocumented_unsafe_blocks)] +#![deny( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::unreachable, + clippy::todo, + clippy::unimplemented +)] +#![cfg_attr( + test, + allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::unreachable, + clippy::todo, + clippy::unimplemented + ) +)] + +pub use net::TokioAddress; +pub use stream::TokioStream; + +mod error; +mod ffi; +mod net; +mod readiness; +mod runtime; +pub mod serve; +mod signal; +mod stream; + +/// Opaque binding of `kj::AsyncIoStream` (see [`unwrap_kj_stream`]). +pub use ffi::KjAsyncIoStream; +pub use ffi::unwrap_kj_stream; +pub use serve::ServeIo; +pub use serve::ServePath; +pub use serve::ServedKjStream; +pub use serve::StreamPump; +pub use serve::TakeSocketError; +pub use serve::serve_kj_stream; +pub use serve::take_kj_socket; diff --git a/src/rust/cxx/kj-rs-io/net.rs b/src/rust/cxx/kj-rs-io/net.rs new file mode 100644 index 00000000000..95be54a6e16 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/net.rs @@ -0,0 +1,841 @@ +//! Tokio-backed `kj::Network` / `kj::NetworkAddress` / `kj::ConnectionReceiver` backends. +//! +//! Address-string grammar follows KJ's `SocketAddress::parse` (kj/async-io-unix.c++) for the +//! subset workerd feeds it: +//! +//! - IPv4: `"1.2.3.4"`, `"1.2.3.4:80"` +//! - IPv6: `"1234:5678::abcd"`, `"[1234:5678::abcd]:80"` +//! - Wildcard (dual-stack): `"*"`, `"*:80"` +//! - Hostnames (DNS via blocking `getaddrinfo` on tokio's blocking pool, its completion delivered +//! same-thread through a tokio runtime task — see [`resolve_host`]): `"example.com"`, +//! `"example.com:80"` +//! - Unix domain: `"unix:/path/to/socket"` (Unix only) +//! +//! Known deviations from KJ, all erroring loudly rather than misbehaving: named services +//! (`"host:http"`), `unix-abstract:` addresses, and IPv6 scope IDs (`"fe80::1%eth0"`) are not +//! supported. + +use std::net::IpAddr; +use std::net::SocketAddr; +use std::net::ToSocketAddrs; + +use tokio::net::TcpListener; +use tokio::net::TcpStream; +#[cfg(unix)] +use tokio::net::UnixListener; +#[cfg(unix)] +use tokio::net::UnixStream; + +use crate::error::KjIoError; +use crate::error::Result; +use crate::error::op; +use crate::runtime::on_loop_runtime; +use crate::runtime::require_loop_runtime; +use crate::stream::TokioStream; + +const LISTEN_BACKLOG: i32 = 1024; + +/// A parsed network address: one or more socket addresses to try in order. +pub struct TokioAddress { + spec: Spec, +} + +#[derive(Clone)] +enum Spec { + Ip { + /// Resolved addresses, tried in order by `connect()`; `listen()` binds the first one + /// (mirroring KJ, which also only listens on the first result). + addrs: Vec, + /// `"*"`: listen on `[::]` with `IPV6_V6ONLY` disabled (dual-stack), reject `connect()`. + wildcard: bool, + }, + #[cfg(unix)] + Unix { path: std::path::PathBuf }, +} + +impl TokioAddress { + /// An address that resolves to exactly `addrs`, tried in that order by `connect()`. The + /// programmatic counterpart of what DNS resolution produces; `listen()` binds the first. + #[must_use] + pub fn from_socket_addrs(addrs: Vec) -> Self { + Self { + spec: Spec::Ip { + addrs, + wildcard: false, + }, + } + } + + async fn parse(text: &str, port_hint: u16) -> Result { + if let Some(path) = text.strip_prefix("unix:") { + #[cfg(unix)] + { + return Ok(Self { + spec: Spec::Unix { path: path.into() }, + }); + } + #[cfg(not(unix))] + { + let _ = path; + return Err(KjIoError::other( + "parseAddress", + "Unix domain sockets are not supported on this platform", + )); + } + } + if text.starts_with("unix-abstract:") { + return Err(KjIoError::other( + "parseAddress", + "abstract Unix domain sockets are not implemented by kj-rs-io", + )); + } + + // Split into address and port parts, exactly like KJ's SocketAddress::parse. + let (addr_part, port_part) = if let Some(rest) = text.strip_prefix('[') { + // Bracketed IPv6, optionally "[..]:port". + let close = rest.rfind(']').ok_or_else(|| { + KjIoError::other("parseAddress", format!("Unclosed '[' in address: {text}")) + })?; + let addr = &rest[..close]; + let tail = &rest[close + 1..]; + if tail.is_empty() { + (addr, None) + } else if let Some(port) = tail.strip_prefix(':') { + (addr, Some(port)) + } else { + return Err(KjIoError::other( + "parseAddress", + format!("Expected port suffix after ']': {text}"), + )); + } + } else if let Some(colon) = text.find(':') { + if text[colon + 1..].contains(':') { + // Two or more colons, no brackets: a bare IPv6 address with no port. + (text, None) + } else { + // Exactly one colon: ip4/hostname with port. + (&text[..colon], Some(&text[colon + 1..])) + } + } else { + (text, None) + }; + + let port = match port_part { + Some(port_text) => port_text.parse::().map_err(|_| { + // KJ falls back to getaddrinfo service-name resolution here; tokio's resolver + // only accepts numeric ports. + KjIoError::other( + "parseAddress", + format!("invalid port (named services are not supported): {port_text}"), + ) + })?, + None => port_hint, + }; + + if addr_part == "*" { + return Ok(Self { + spec: Spec::Ip { + addrs: vec![SocketAddr::new( + IpAddr::V6(std::net::Ipv6Addr::UNSPECIFIED), + port, + )], + wildcard: true, + }, + }); + } + + if let Ok(ip) = addr_part.parse::() { + return Ok(Self { + spec: Spec::Ip { + addrs: vec![SocketAddr::new(ip, port)], + wildcard: false, + }, + }); + } + + // Not a literal: resolve the hostname via getaddrinfo on tokio's blocking pool. The + // completion is absorbed by a runtime task and forwarded to the loop thread so this + // await resumes same-thread -- an optimization (a cross-thread wake would go through + // the waker bridge's cross-thread fulfiller, which is legal but slower); see + // `resolve_host` for the full rationale. + let addrs: Vec = resolve_host(addr_part, port).await?; + if addrs.is_empty() { + return Err(KjIoError::other( + "getaddrinfo()", + format!("no addresses found for host: {addr_part}"), + )); + } + Ok(Self { + spec: Spec::Ip { + addrs, + wildcard: false, + }, + }) + } + + async fn connect_index(&self, index: usize) -> Result> { + match &self.spec { + Spec::Ip { addrs, wildcard } => { + if *wildcard { + return Err(KjIoError::other( + "connect()", + "cannot connect() to a wildcard address", + )); + } + let addr = addrs + .get(index) + .ok_or_else(|| KjIoError::other("connect()", "address index out of range"))?; + let stream = TcpStream::connect(addr).await.map_err(op("connect()"))?; + Ok(Box::new(TokioStream::from_tcp(stream))) + } + #[cfg(unix)] + Spec::Unix { path } => { + if index != 0 { + return Err(KjIoError::other("connect()", "address index out of range")); + } + let stream = UnixStream::connect(path).await.map_err(op("connect()"))?; + Ok(Box::new(TokioStream::from_unix(stream))) + } + } + } + + fn count(&self) -> usize { + match &self.spec { + Spec::Ip { addrs, .. } => addrs.len(), + #[cfg(unix)] + Spec::Unix { .. } => 1, + } + } + + fn raw_sockaddr(&self, index: usize) -> Result> { + let sockaddr: socket2::SockAddr = match &self.spec { + Spec::Ip { addrs, .. } => (*addrs + .get(index) + .ok_or_else(|| KjIoError::other("sockaddr", "address index out of range"))?) + .into(), + #[cfg(unix)] + Spec::Unix { path } => { + if index != 0 { + return Err(KjIoError::other("sockaddr", "address index out of range")); + } + socket2::SockAddr::unix(path).map_err(op("sockaddr"))? + } + }; + Ok(crate::ffi::sockaddr_to_bytes(&sockaddr)) + } + + fn listen(&self) -> Result> { + require_loop_runtime()?; + match &self.spec { + Spec::Ip { addrs, wildcard } => { + let addr = *addrs + .first() + .ok_or_else(|| KjIoError::other("listen()", "no addresses to bind"))?; + let domain = socket2::Domain::for_address(addr); + let socket = socket2::Socket::new(domain, socket2::Type::STREAM, None) + .map_err(op("socket()"))?; + // KJ parity: SO_REUSEADDR on listeners; wildcard sockets accept both address + // families (IPV6_V6ONLY off). + socket.set_reuse_address(true).map_err(op("setsockopt()"))?; + if *wildcard { + socket.set_only_v6(false).map_err(op("setsockopt()"))?; + } + socket.bind(&addr.into()).map_err(op("bind()"))?; + socket.listen(LISTEN_BACKLOG).map_err(op("listen()"))?; + socket.set_nonblocking(true).map_err(op("fcntl()"))?; + let listener = TcpListener::from_std(socket.into()).map_err(op("wrap listener"))?; + Ok(Box::new(TokioListener { + inner: ListenerInner::Tcp(listener), + })) + } + #[cfg(unix)] + Spec::Unix { path } => { + // Like KJ, no unlink(): binding an existing path fails. + let listener = + std::os::unix::net::UnixListener::bind(path).map_err(op("bind()"))?; + listener.set_nonblocking(true).map_err(op("fcntl()"))?; + let listener = UnixListener::from_std(listener).map_err(op("wrap listener"))?; + Ok(Box::new(TokioListener { + inner: ListenerInner::Unix(listener), + })) + } + } + } + + fn to_display_string(&self) -> String { + match &self.spec { + Spec::Ip { addrs, wildcard } => { + if *wildcard { + format!("*:{}", addrs[0].port()) + } else { + let parts: Vec = addrs.iter().map(ToString::to_string).collect(); + parts.join(",") + } + } + #[cfg(unix)] + Spec::Unix { path } => format!("unix:{}", path.display()), + } + } +} + +/// A listening socket (`kj::ConnectionReceiver` backend). +pub struct TokioListener { + inner: ListenerInner, +} + +enum ListenerInner { + Tcp(TcpListener), + #[cfg(unix)] + Unix(UnixListener), +} + +impl TokioListener { + async fn accept(&self) -> Result> { + match &self.inner { + ListenerInner::Tcp(listener) => { + let (stream, _peer) = listener.accept().await.map_err(op("accept()"))?; + let _ = stream.set_nodelay(true); + Ok(Box::new(TokioStream::from_tcp(stream))) + } + #[cfg(unix)] + ListenerInner::Unix(listener) => { + let (stream, _peer) = listener.accept().await.map_err(op("accept()"))?; + Ok(Box::new(TokioStream::from_unix(stream))) + } + } + } + + fn port(&self) -> Result { + match &self.inner { + ListenerInner::Tcp(listener) => { + Ok(listener.local_addr().map_err(op("getsockname()"))?.port()) + } + // KJ returns 0 for non-IP listeners. + #[cfg(unix)] + ListenerInner::Unix(_) => Ok(0), + } + } + + /// Borrows the live listener socket's fd (tokio listeners implement `AsFd`), for the + /// sockopt/sockname passthrough behind `kj::ConnectionReceiver`. + #[cfg(unix)] + pub(crate) fn as_borrowed_fd(&self) -> std::os::fd::BorrowedFd<'_> { + use std::os::fd::AsFd; + match &self.inner { + ListenerInner::Tcp(listener) => listener.as_fd(), + ListenerInner::Unix(listener) => listener.as_fd(), + } + } + + /// Borrows the live listener socket's `SOCKET` (tokio's `TcpListener` implements + /// `AsSocket`): the Windows counterpart of [`TokioListener::as_borrowed_fd`]. On Windows + /// only the Tcp variant of `ListenerInner` exists. + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + pub(crate) fn as_borrowed_socket(&self) -> std::os::windows::io::BorrowedSocket<'_> { + use std::os::windows::io::AsSocket; + match &self.inner { + ListenerInner::Tcp(listener) => listener.as_socket(), + } + } + + /// Raw `struct sockaddr` bytes of the listener's bound address (the `getsockname()` + /// passthrough behind `kj::ConnectionReceiver::getsockname`). + #[cfg(any(unix, windows))] + fn local_addr_bytes(&self) -> Result> { + #[cfg(unix)] + let sock = self.as_borrowed_fd(); + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + let sock = self.as_borrowed_socket(); + let addr = socket2::SockRef::from(&sock) + .local_addr() + .map_err(op("getsockname()"))?; + Ok(crate::ffi::sockaddr_to_bytes(&addr)) + } + + #[cfg(not(any(unix, windows)))] + fn local_addr_bytes(&self) -> Result> { + Err(KjIoError::other( + "getsockname", + "not implemented by kj-rs-io on this platform", + )) + } +} + +// ====================================================================================== +// Bridge entry points (see lib.rs). + +/// Resolves a hostname via blocking `getaddrinfo` on tokio's blocking pool. A task on the loop's +/// `LocalSet` owns the blocking `JoinHandle`, so the blocking-pool completion wakes tokio's own +/// scheduler waker and the result comes back over a oneshot, waking the awaiting future +/// same-thread. The kj-rs waker bridge is thread-safe, so `tokio::net::lookup_host` (whose +/// `JoinHandle` wake lands on the caller's waker cross-thread) would also be *correct*; this +/// shape is kept as an optimization — it keeps every bridged-waker wake on the loop thread's +/// fast path instead of a cross-thread fulfiller hop per lookup. Same blocking `getaddrinfo` +/// call and NSS/`/etc/hosts` parity as `lookup_host`. +async fn resolve_host(host: &str, port: u16) -> Result> { + let host = host.to_owned(); + let (tx, rx) = tokio::sync::oneshot::channel::>>(); + + // The forwarding task's waker is tokio's own scheduler waker (Send + Sync), so the + // cross-thread completion from the blocking pool terminates inside tokio's scheduler + // (unparking this loop), never at a rust cross-thread waker. The task forwards the result on + // the loop thread, waking the awaiting future same-thread. Spawned onto the loop's LocalSet + // (kj_rs_tokio::spawn) so it is cancelled with the loop rather than with the runtime. + let task = kj_rs_tokio::spawn(async move { + let resolved = match tokio::task::spawn_blocking(move || { + (host.as_str(), port) + .to_socket_addrs() + .map(std::iter::Iterator::collect::>) + }) + .await + { + Ok(result) => result, + Err(_) => Err(std::io::Error::other("getaddrinfo task failed")), + }; + let _ = tx.send(resolved); + }); + // If this future is dropped (KJ promise cancelled), abort the forwarding task rather than + // leaving it to run to completion for nobody. The blocking getaddrinfo call itself cannot be + // interrupted once started (an OS limitation shared with KJ's own resolver and tokio's + // `lookup_host`), so its blocking-pool slot is reclaimed only when the syscall returns. + let _abort_guard = crate::runtime::AbortOnDrop(task); + + match rx.await { + Ok(result) => result.map_err(op("getaddrinfo()")), + Err(_) => Err(KjIoError::other( + "getaddrinfo()", + "DNS resolver task dropped", + )), + } +} + +pub async fn network_parse_address(addr: String, port_hint: u16) -> Result> { + on_loop_runtime(async move { Ok(Box::new(TokioAddress::parse(&addr, port_hint).await?)) }).await +} + +pub fn network_get_sockaddr(sockaddr: &[u8]) -> Result> { + #[cfg(any(unix, windows))] + { + let addr = sockaddr_from_bytes(sockaddr)?; + if let Some(socket_addr) = addr.as_socket() { + return Ok(Box::new(TokioAddress { + spec: Spec::Ip { + addrs: vec![socket_addr], + wildcard: false, + }, + })); + } + #[cfg(unix)] + if let Some(path) = addr.as_pathname() { + return Ok(Box::new(TokioAddress { + spec: Spec::Unix { path: path.into() }, + })); + } + Err(KjIoError::other( + "getSockaddr", + "unsupported sockaddr family", + )) + } + #[cfg(not(any(unix, windows)))] + { + let _ = sockaddr; + Err(KjIoError::other( + "getSockaddr", + "not implemented on this platform", + )) + } +} + +/// Connects to exactly the `index`th resolved address (no fallback). The C++ side drives the +/// try-each-address loop itself so it can apply `restrictPeers()` filtering per address before +/// initiating each connection attempt (KJ parity: a blocked address contributes a +/// "`connect()` blocked by `restrictPeers()`" failure; only the last address's error propagates). +pub async fn address_connect_index(addr: &TokioAddress, index: usize) -> Result> { + on_loop_runtime(addr.connect_index(index)).await +} + +/// Number of resolved socket addresses behind this address (>= 1). +pub fn address_count(addr: &TokioAddress) -> usize { + addr.count() +} + +/// Raw `struct sockaddr` bytes of the `index`th resolved address, for the C++ side's +/// `kj::_::NetworkFilter` (restrictPeers) checks. +pub fn address_raw_sockaddr(addr: &TokioAddress, index: usize) -> Result> { + addr.raw_sockaddr(index) +} + +pub fn address_listen(addr: &TokioAddress) -> Result> { + addr.listen() +} + +#[expect(clippy::unnecessary_box_returns)] // Opaque cxx types must cross the bridge boxed. +pub fn address_clone(addr: &TokioAddress) -> Box { + Box::new(TokioAddress { + spec: addr.spec.clone(), + }) +} + +pub fn address_to_string(addr: &TokioAddress) -> String { + addr.to_display_string() +} + +pub async fn listener_accept(listener: &TokioListener) -> Result> { + on_loop_runtime(listener.accept()).await +} + +pub fn listener_port(listener: &TokioListener) -> Result { + listener.port() +} + +pub fn listener_local_addr(listener: &TokioListener) -> Result> { + listener.local_addr_bytes() +} + +// ====================================================================================== +// Socket-handle wrapping. All handles arrive owned and non-blocking as an `i64` "raw socket +// handle" — a Unix fd or a win32 SOCKET (the C++ side normalizes KJ's TAKE_OWNERSHIP / +// ALREADY_CLOEXEC / ALREADY_NONBLOCK flags, dup'ing when not taking ownership). The platform +// split lives entirely in `ffi::own_socket_from_raw` (the one conversion point); everything +// here operates on the uniform `socket2::Socket` / std / tokio types. + +#[cfg(any(unix, windows))] +fn socket_from_raw(handle: i64) -> socket2::Socket { + crate::ffi::own_socket_from_raw(handle) +} + +#[cfg(any(unix, windows))] +fn sockaddr_from_bytes(bytes: &[u8]) -> Result { + crate::ffi::sockaddr_from_bytes(bytes) +} + +/// The handle tier of [`crate::take_kj_socket`] (unix only; see that function's docs): wraps +/// an *owned*, connected stream-socket fd (TCP or Unix domain, detected automatically) as a +/// [`crate::serve::ServeIo`]. Unlike [`wrap_socket_fd`] the fd is a fresh dup of a kj stream's +/// socket, so non-blocking mode is forced rather than assumed (the original may have come from +/// anywhere). +#[cfg(unix)] +pub fn serve_io_from_owned_fd(fd: std::os::fd::OwnedFd) -> Result { + let socket = socket2::Socket::from(fd); + socket.set_nonblocking(true).map_err(op("fcntl()"))?; + let local = socket.local_addr().map_err(op("getsockname()"))?; + require_loop_runtime()?; + match local.domain() { + socket2::Domain::IPV4 | socket2::Domain::IPV6 => { + let stream = TcpStream::from_std(socket.into()).map_err(op("takeKjSocket"))?; + Ok(crate::serve::ServeIo::Tcp(stream)) + } + socket2::Domain::UNIX => { + let stream = UnixStream::from_std(socket.into()).map_err(op("takeKjSocket"))?; + Ok(crate::serve::ServeIo::Unix(stream)) + } + _ => Err(KjIoError::other( + "takeKjSocket", + "unsupported socket family", + )), + } +} + +pub fn wrap_socket_fd(handle: i64) -> Result> { + #[cfg(any(unix, windows))] + { + let socket = socket_from_raw(handle); + let local = socket.local_addr().map_err(op("getsockname()"))?; + require_loop_runtime()?; + match local.domain() { + socket2::Domain::IPV4 | socket2::Domain::IPV6 => { + let stream = TcpStream::from_std(socket.into()).map_err(op("wrapSocketFd"))?; + Ok(Box::new(TokioStream::from_tcp(stream))) + } + #[cfg(unix)] + socket2::Domain::UNIX => { + let stream = UnixStream::from_std(socket.into()).map_err(op("wrapSocketFd"))?; + Ok(Box::new(TokioStream::from_unix(stream))) + } + _ => Err(KjIoError::other( + "wrapSocketFd", + "unsupported socket family", + )), + } + } + #[cfg(not(any(unix, windows)))] + { + let _ = handle; + Err(KjIoError::other( + "wrapSocketFd", + "not implemented on this platform", + )) + } +} + +pub fn wrap_listen_fd(handle: i64) -> Result> { + #[cfg(any(unix, windows))] + { + let socket = socket_from_raw(handle); + let local = socket.local_addr().map_err(op("getsockname()"))?; + require_loop_runtime()?; + match local.domain() { + socket2::Domain::IPV4 | socket2::Domain::IPV6 => { + let listener = + TcpListener::from_std(socket.into()).map_err(op("wrapListenSocketFd"))?; + Ok(Box::new(TokioListener { + inner: ListenerInner::Tcp(listener), + })) + } + #[cfg(unix)] + socket2::Domain::UNIX => { + let listener = + UnixListener::from_std(socket.into()).map_err(op("wrapListenSocketFd"))?; + Ok(Box::new(TokioListener { + inner: ListenerInner::Unix(listener), + })) + } + _ => Err(KjIoError::other( + "wrapListenSocketFd", + "unsupported socket family", + )), + } + } + #[cfg(not(any(unix, windows)))] + { + let _ = handle; + Err(KjIoError::other( + "wrapListenSocketFd", + "not implemented on this platform", + )) + } +} + +pub async fn wrap_connecting_socket_fd(handle: i64, sockaddr: Vec) -> Result> { + #[cfg(any(unix, windows))] + { + on_loop_runtime(async move { + let addr = sockaddr_from_bytes(&sockaddr)?; + let socket_addr = addr.as_socket().ok_or_else(|| { + KjIoError::other( + "wrapConnectingSocketFd", + "only AF_INET/AF_INET6 sockaddrs are supported", + ) + })?; + let socket = socket_from_raw(handle); + // TcpSocket::connect handles the nonblocking connect dance (EINPROGRESS, wait for + // writability, check SO_ERROR) and registers with the I/O driver. + let tcp_socket = tokio::net::TcpSocket::from_std_stream(socket.into()); + let stream = tcp_socket + .connect(socket_addr) + .await + .map_err(op("connect()"))?; + Ok(Box::new(TokioStream::from_tcp(stream))) + }) + .await + } + #[cfg(not(any(unix, windows)))] + { + let _ = (handle, sockaddr); + Err(KjIoError::other( + "wrapConnectingSocketFd", + "not implemented on this platform", + )) + } +} + +#[cfg(test)] +mod tests { + use std::net::IpAddr; + use std::net::Ipv4Addr; + use std::net::Ipv6Addr; + use std::net::SocketAddr; + + use cxx::KjError; + + use super::*; + + /// `TokioAddress::parse` is `async` only because hostnames go through DNS; every literal + /// form below resolves without ever awaiting, so a single poll with a no-op waker suffices. + fn parse_literal(text: &str, port_hint: u16) -> Result { + let mut fut = std::pin::pin!(TokioAddress::parse(text, port_hint)); + let mut cx = std::task::Context::from_waker(std::task::Waker::noop()); + match fut.as_mut().poll(&mut cx) { + std::task::Poll::Ready(result) => result, + std::task::Poll::Pending => panic!("literal address {text:?} should not await"), + } + } + + fn parse_ok(text: &str, port_hint: u16) -> TokioAddress { + match parse_literal(text, port_hint) { + Ok(addr) => addr, + Err(e) => panic!( + "{text:?} failed to parse: {}", + KjError::from(e).description() + ), + } + } + + fn parse_err(text: &str, port_hint: u16) -> KjError { + match parse_literal(text, port_hint) { + Ok(_) => panic!("{text:?} unexpectedly parsed"), + Err(e) => KjError::from(e), + } + } + + fn ip_addrs(addr: &TokioAddress) -> (&[SocketAddr], bool) { + match &addr.spec { + Spec::Ip { addrs, wildcard } => (addrs, *wildcard), + #[cfg(unix)] + Spec::Unix { .. } => panic!("expected an IP address"), + } + } + + #[test] + fn ipv4_literal_with_and_without_port() { + let addr = parse_ok("1.2.3.4:80", 0); + let (addrs, wildcard) = ip_addrs(&addr); + assert_eq!( + addrs, + &[SocketAddr::new(IpAddr::V4(Ipv4Addr::new(1, 2, 3, 4)), 80)] + ); + assert!(!wildcard); + + // No port: the port hint fills in. + let addr = parse_ok("1.2.3.4", 8080); + assert_eq!(ip_addrs(&addr).0[0].port(), 8080); + } + + #[test] + fn ipv6_literal_forms() { + // Bracketed with port. + let addr = parse_ok("[::1]:443", 0); + assert_eq!( + ip_addrs(&addr).0, + &[SocketAddr::new(IpAddr::V6(Ipv6Addr::LOCALHOST), 443)] + ); + // Bracketed without port: hint applies. + let addr = parse_ok("[::1]", 7); + assert_eq!(ip_addrs(&addr).0[0].port(), 7); + // Bare IPv6 (two or more colons, no brackets) means "no port" -- never "host:port". + let addr = parse_ok("fe80::1", 9); + assert_eq!( + ip_addrs(&addr).0[0], + SocketAddr::new("fe80::1".parse::().unwrap(), 9) + ); + } + + #[test] + fn wildcard_forms() { + let addr = parse_ok("*", 1234); + let (addrs, wildcard) = ip_addrs(&addr); + assert!(wildcard); + assert_eq!( + addrs, + &[SocketAddr::new(IpAddr::V6(Ipv6Addr::UNSPECIFIED), 1234)] + ); + let addr = parse_ok("*:80", 0); + let (addrs, wildcard) = ip_addrs(&addr); + assert!(wildcard); + assert_eq!(addrs[0].port(), 80); + } + + #[cfg(unix)] + #[test] + fn unix_forms() { + match &parse_ok("unix:/tmp/sock", 0).spec { + Spec::Unix { path } => assert_eq!(path, std::path::Path::new("/tmp/sock")), + Spec::Ip { .. } => panic!("expected a unix address"), + } + // Abstract sockets are a documented gap. + assert!( + parse_err("unix-abstract:foo", 0) + .description() + .contains("abstract") + ); + } + + #[cfg(unix)] + #[test] + fn get_sockaddr_rejects_unsupported_family() { + // A zeroed sockaddr (family AF_UNSPEC) is neither AF_INET/6 nor AF_UNIX, so + // network_get_sockaddr must reject it rather than fabricate an address. Length is a + // valid sockaddr_in size so it passes the length check and reaches the family branch. + let bogus = vec![0u8; core::mem::size_of::()]; + let err = match network_get_sockaddr(&bogus) { + Ok(_) => panic!("a zeroed (AF_UNSPEC) sockaddr must be rejected"), + Err(e) => KjError::from(e), + }; + assert!( + err.description().contains("unsupported sockaddr family"), + "{}", + err.description() + ); + } + + /// Randomized: the address grammar must never panic on arbitrary strings, and every literal + /// it accepts must survive a display round trip. Hostname-shaped inputs are allowed to go + /// Pending (they start a DNS lookup on the loop's runtime, which needs a port; dropping the + /// future aborts it). Seeded xorshift, so a failure is reproducible. + #[test] + fn parse_never_panics_on_random_input_and_literals_round_trip() { + const ALPHABET: &[u8] = b"0123456789abcdef:.[]*%-/xu"; + let _port = kj_rs_tokio::TokioPort::new(); + let mut state: u64 = 0x2545_f491_4f6c_dd1d; + let mut next = move || { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + state + }; + let cx_waker = std::task::Waker::noop(); + for _ in 0..20_000 { + let len = usize::try_from(next() % 24).unwrap(); + let text: String = (0..len) + .map(|_| ALPHABET[usize::try_from(next() % ALPHABET.len() as u64).unwrap()] as char) + .collect(); + let hint = u16::try_from(next() & 0xffff).unwrap(); + let mut fut = std::pin::pin!(TokioAddress::parse(&text, hint)); + let mut cx = std::task::Context::from_waker(cx_waker); + let std::task::Poll::Ready(Ok(addr)) = fut.as_mut().poll(&mut cx) else { + continue; // rejected, or a hostname now resolving: both fine + }; + // Round trip: what it prints must parse back to the same addresses. + let shown = addr.to_display_string(); + let Spec::Ip { addrs, wildcard } = &addr.spec else { + continue; + }; + if *wildcard { + assert!(shown.starts_with("*:"), "{text:?} -> {shown:?}"); + continue; + } + let expected = addrs[0]; + let reparsed = parse_ok(&shown, 0); + assert_eq!(ip_addrs(&reparsed).0[0], expected, "{text:?} -> {shown:?}"); + } + } + + #[test] + fn documented_rejections() { + // Named services: tokio's resolver only takes numeric ports. + let err = parse_err("1.2.3.4:http", 0); + assert!( + err.description().contains("named services"), + "{}", + err.description() + ); + // Unclosed bracket. + let err = parse_err("[::1", 0); + assert!( + err.description().contains("Unclosed"), + "{}", + err.description() + ); + // Junk after the closing bracket. + let err = parse_err("[::1]x", 0); + assert!( + err.description().contains("Expected port suffix"), + "{}", + err.description() + ); + // Out-of-range port. + let _ = parse_err("1.2.3.4:70000", 0); + } +} diff --git a/src/rust/cxx/kj-rs-io/peer-filter.c++ b/src/rust/cxx/kj-rs-io/peer-filter.c++ new file mode 100644 index 00000000000..27f3ee1d96e --- /dev/null +++ b/src/rust/cxx/kj-rs-io/peer-filter.c++ @@ -0,0 +1,203 @@ +// Port of KJ's kj::_::NetworkFilter (kj/async-io.c++, MIT-licensed, Sandstorm Development +// Group and contributors) — see peer-filter.h for why this is a port rather than a reuse. +// Behavior must be kept in lockstep with upstream KJ. + +#include "kj-rs-io/peer-filter.h" + +#include + +#if _WIN32 +#include +// windows.h (pulled in by winsock2.h) defines ERROR as a macro, which breaks KJ_LOG(ERROR). +#include +#else +#include +#include +#include +#endif + +namespace kj_rs_io { +namespace { + +using kj::CidrRange; + +kj::ArrayPtr localCidrs() { + static const CidrRange result[] = { + // localhost + "127.0.0.0/8"_kj, + "::1/128"_kj, + + // Trying to *connect* to 0.0.0.0 on many systems is equivalent to connecting to + // localhost. (wat) + "0.0.0.0/32"_kj, + "::/128"_kj, + }; + return kj::arrayPtr(result, kj::size(result)); +} + +kj::ArrayPtr privateCidrs() { + static const CidrRange result[] = { + "10.0.0.0/8"_kj, // RFC1918 reserved for internal network + "100.64.0.0/10"_kj, // RFC6598 "shared address space" for carrier-grade NAT + "169.254.0.0/16"_kj, // RFC3927 "link local" (auto-configured LAN in absence of DHCP) + "172.16.0.0/12"_kj, // RFC1918 reserved for internal network + "192.168.0.0/16"_kj, // RFC1918 reserved for internal network + + "fc00::/7"_kj, // RFC4193 unique private network + "fe80::/10"_kj, // RFC4291 "link local" (auto-configured LAN in absence of DHCP) + }; + return kj::arrayPtr(result, kj::size(result)); +} + +kj::ArrayPtr reservedCidrs() { + // Address ranges reserved by RFCs for specific alternative protocols. These are not + // considered part of "public", "private", "network", nor "local". But, we will allow apps to + // explicitly allowlist CIDRs in this range if they really want, because some people actually + // use these ranges as if they were private ranges. + static const CidrRange result[] = { + "192.0.0.0/24"_kj, // RFC6890 reserved for special protocols + "224.0.0.0/4"_kj, // RFC1112 multicast + "240.0.0.0/4"_kj, // RFC1112 multicast / reserved for future use + "255.255.255.255/32"_kj, // RFC0919 broadcast address + + "2001::/23"_kj, // RFC2928 reserved for special protocols + "ff00::/8"_kj, // RFC4291 multicast + }; + return kj::arrayPtr(result, kj::size(result)); +} + +bool matchesAny(kj::ArrayPtr cidrs, const struct sockaddr *addr) { + for (auto &cidr: cidrs) { + if (cidr.matches(addr)) return true; + } + return false; +} + +#if !_WIN32 +// sockaddr_un::sun_path is not required to have a NUL terminator, so it must be read carefully. +kj::ArrayPtr safeUnixPath(const struct sockaddr_un *addr, kj::uint addrlen) { + KJ_REQUIRE(addr->sun_family == AF_UNIX, "not a unix address"); + KJ_REQUIRE(addrlen >= offsetof(sockaddr_un, sun_path), "invalid unix address"); + + size_t maxPathlen = addrlen - offsetof(sockaddr_un, sun_path); + + size_t pathlen; + if (maxPathlen > 0 && addr->sun_path[0] == '\0') { + // Linux "abstract" unix address + pathlen = strnlen(addr->sun_path + 1, maxPathlen - 1) + 1; + } else { + pathlen = strnlen(addr->sun_path, maxPathlen); + } + return kj::arrayPtr(addr->sun_path, pathlen); +} +#endif // !_WIN32 + +} // namespace + +PeerFilter::PeerFilter(): allowUnix(true), allowAbstractUnix(true) { + allowCidrs.add(CidrRange::inet4({0, 0, 0, 0}, 0)); + allowCidrs.add(CidrRange::inet6({}, {}, 0)); +} + +PeerFilter::PeerFilter(kj::ArrayPtr allow, + kj::ArrayPtr deny, + kj::Rc next) + : allowUnix(false), + allowAbstractUnix(false), + next(kj::mv(next)) { + for (auto rule: allow) { + if (rule == "local") { + allowCidrs.addAll(localCidrs()); + } else if (rule == "network") { + // Can't be represented as a simple union of CIDRs, so we handle in shouldAllow(). + allowNetwork = true; + } else if (rule == "private") { + allowCidrs.addAll(privateCidrs()); + allowCidrs.addAll(localCidrs()); + } else if (rule == "public") { + // Can't be represented as a simple union of CIDRs, so we handle in shouldAllow(). + allowPublic = true; + } else if (rule == "unix") { + allowUnix = true; + } else if (rule == "unix-abstract") { + allowAbstractUnix = true; + } else { + allowCidrs.add(CidrRange(rule)); + } + } + + for (auto rule: deny) { + if (rule == "local") { + denyCidrs.addAll(localCidrs()); + } else if (rule == "network") { + KJ_FAIL_REQUIRE("don't deny 'network', allow 'local' instead"); + } else if (rule == "private") { + denyCidrs.addAll(privateCidrs()); + } else if (rule == "public") { + // Tricky: What if we allow 'network' and deny 'public'? + KJ_FAIL_REQUIRE("don't deny 'public', allow 'private' instead"); + } else if (rule == "unix") { + allowUnix = false; + } else if (rule == "unix-abstract") { + allowAbstractUnix = false; + } else { + denyCidrs.add(CidrRange(rule)); + } + } +} + +bool PeerFilter::shouldAllow(const struct sockaddr *addr, kj::uint addrlen) { + KJ_REQUIRE(addrlen >= sizeof(addr->sa_family)); + +#if !_WIN32 + if (addr->sa_family == AF_UNIX) { + auto path = safeUnixPath(reinterpret_cast(addr), addrlen); + if (path.size() > 0 && path[0] == '\0') { + return allowAbstractUnix; + } else { + return allowUnix; + } + } +#endif + + bool allowed = false; + kj::uint allowSpecificity = 0; + + if (allowPublic) { + if ((addr->sa_family == AF_INET || addr->sa_family == AF_INET6) && + !matchesAny(privateCidrs(), addr) && !matchesAny(localCidrs(), addr) && + !matchesAny(reservedCidrs(), addr)) { + allowed = true; + // Don't adjust allowSpecificity as this match has an effective specificity of zero. + } + } + + if (allowNetwork) { + if ((addr->sa_family == AF_INET || addr->sa_family == AF_INET6) && + !matchesAny(localCidrs(), addr) && !matchesAny(reservedCidrs(), addr)) { + allowed = true; + // Don't adjust allowSpecificity as this match has an effective specificity of zero. + } + } + + for (auto &cidr: allowCidrs) { + if (cidr.matches(addr)) { + allowSpecificity = kj::max(allowSpecificity, cidr.getSpecificity()); + allowed = true; + } + } + if (!allowed) return false; + for (auto &cidr: denyCidrs) { + if (cidr.matches(addr)) { + if (cidr.getSpecificity() >= allowSpecificity) return false; + } + } + + KJ_IF_SOME(n, next) { + return n->shouldAllow(addr, addrlen); + } else { + return true; + } +} + +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/peer-filter.h b/src/rust/cxx/kj-rs-io/peer-filter.h new file mode 100644 index 00000000000..754120d721c --- /dev/null +++ b/src/rust/cxx/kj-rs-io/peer-filter.h @@ -0,0 +1,55 @@ +#pragma once +// PeerFilter: a faithful port of KJ's kj::_::NetworkFilter (kj/async-io.c++), backing +// kj-rs-io's Network::restrictPeers() support. +// +// Ported rather than reused because kj::_::NetworkFilter lives in KJ's internal header +// (kj/async-io-internal.h), whose quoted includes ("vector.h") only resolve inside the KJ +// source tree — it is not includable through Bazel's virtual include dirs. The allow/deny +// grammar ("public"/"private"/"local"/"network"/"unix"/"unix-abstract"/CIDRs), the RFC CIDR +// tables, the specificity tie-breaking between allow and deny rules, and the filter-chaining +// semantics are kept identical so restrictPeers() behaves exactly like kj::setupAsyncIo()'s +// networks. kj::CidrRange itself IS reused (kj/cidr.h is a clean public header). + +#include +#include +#include +#include + +namespace kj_rs_io { + +class PeerFilter final: public kj::LowLevelAsyncIoProvider::NetworkFilter, public kj::Refcounted { + public: + // Allow-everything filter (matches KJ's root networks). + PeerFilter(); + + // Restriction layered on `next`, which the new filter OWNS: a restrictPeers() chain keeps its + // parent filters alive, so there is no outlive-me contract between networks. Grammar identical + // to kj::Network::restrictPeers(). + PeerFilter(kj::ArrayPtr allow, + kj::ArrayPtr deny, + kj::Rc next); + + // Read-only despite the non-const signature: this override matches + // kj::LowLevelAsyncIoProvider::NetworkFilter::shouldAllow (declared non-const upstream), but the + // implementation only *reads* the CIDR tables / flags and recurses into `next` — it mutates no + // member and has no interior mutability, so concurrent callers sharing a filter are safe. Keep + // it read-only. + bool shouldAllow(const struct sockaddr *addr, kj::uint addrlen) override; + + // Refcounted (always created via kj::rc()) and immobile: every holder — networks, + // addresses, receivers, derived filters' `next` — shares ownership via kj::Rc, so a filter can + // never be destroyed or moved out from under its chain. + KJ_DISALLOW_COPY_AND_MOVE(PeerFilter); + + private: + kj::Vector allowCidrs; + kj::Vector denyCidrs; + bool allowUnix; + bool allowAbstractUnix; + bool allowPublic = false; + bool allowNetwork = false; + + kj::Maybe> next; +}; + +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/readiness.rs b/src/rust/cxx/kj-rs-io/readiness.rs new file mode 100644 index 00000000000..2c7e9dbe336 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/readiness.rs @@ -0,0 +1,94 @@ +//! tokio-backed fd readiness watching. +//! +//! This backs `kj_rs_io::FileWatcher` (file-watcher.h), the tokio-loop replacement for +//! workerd's `--watch` file watcher. The C++ side owns the platform notification fd (inotify on +//! Linux, kqueue on macOS/BSD) and does all the event parsing; the Rust side only supplies +//! "resolve when this fd becomes readable", replacing +//! `kj::UnixEventPort::FdObserver::whenBecomesReadable()`. +//! +//! Ownership: a [`TokioFdWatcher`] owns its own `dup(2)` of the notification fd and ONE +//! registration of it with the tokio I/O driver, created when the C++ `FileWatcher::Impl` is +//! constructed and released when it is destroyed. Nothing about the original fd is borrowed past +//! the constructor call, and C++ never has to keep an fd open for a pending promise or avoid +//! registering it twice -- the two contracts the previous borrowed-raw-fd design left to +//! comments. +//! +//! Semantics of [`TokioFdWatcher::readable`]: +//! +//! - Readiness that already exists when it is called is reported immediately (both epoll and +//! kqueue report existing readiness at registration; tokio remembers it thereafter). +//! - The wake consumes tokio's cached readiness (`clear_ready`) BEFORE resolving, so after C++ +//! drains the fd (via the original) the next call sleeps until a genuinely new event. An +//! event that lands between the clear and the drain is drained, and costs at most one +//! spurious wake later (C++'s read then sees EAGAIN and calls again) -- never a lost wake +//! and never a busy loop. +//! - Dropping a pending future just removes its waker; the registration stays. + +use crate::error::Result; + +/// See the module docs. Unix only; on other platforms construction fails. +pub struct TokioFdWatcher { + #[cfg(unix)] + afd: tokio::io::unix::AsyncFd, +} + +/// Creates a watcher for `fd`, which is borrowed only for the duration of this call. +/// +/// # Errors +/// +/// Fails if the fd cannot be duplicated or registered with the loop runtime's I/O driver, or on +/// non-Unix platforms. +pub fn new_fd_watcher(fd: i32) -> Result> { + #[cfg(unix)] + { + use tokio::io::Interest; + use tokio::io::unix::AsyncFd; + + use crate::error::op; + // The caller (C++ FileWatcher::Impl) owns `fd` and it is open for this call; we keep + // only the dup (see `dup_raw_fd`). + let owned = crate::ffi::dup_raw_fd(fd)?; + crate::runtime::require_loop_runtime()?; + let afd = AsyncFd::with_interest(owned, Interest::READABLE).map_err(op("AsyncFd"))?; + Ok(Box::new(TokioFdWatcher { afd })) + } + #[cfg(not(unix))] + { + use crate::error::KjIoError; + let _ = fd; + Err(KjIoError::other( + "new_fd_watcher", + "kj-rs-io fd readiness watching is only implemented on Unix", + )) + } +} + +impl TokioFdWatcher { + /// Resolves when the watched fd becomes readable. See the module docs for the exact + /// semantics. + /// + /// # Errors + /// + /// Fails if the I/O driver reports an error for the fd, or on non-Unix platforms. + pub async fn readable(&self) -> Result<()> { + #[cfg(unix)] + { + use crate::error::op; + use crate::runtime::on_loop_runtime; + on_loop_runtime(async { + let mut guard = self.afd.readable().await.map_err(op("readable"))?; + guard.clear_ready(); + Ok(()) + }) + .await + } + #[cfg(not(unix))] + { + use crate::error::KjIoError; + Err(KjIoError::other( + "readable", + "kj-rs-io fd readiness watching is only implemented on Unix", + )) + } + } +} diff --git a/src/rust/cxx/kj-rs-io/runtime.rs b/src/rust/cxx/kj-rs-io/runtime.rs new file mode 100644 index 00000000000..0db5c6b7ab9 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/runtime.rs @@ -0,0 +1,61 @@ +//! Loop-runtime precondition: every kj-rs-io operation runs on a thread owning a +//! `kj_rs_tokio::TokioEventPort`, whose tokio runtime context that thread is permanently inside +//! (see `kj_rs_tokio`'s `EnteredRuntime`). tokio resources therefore register with the loop's +//! I/O driver and timers without any per-call `Handle::enter()`; the only thing left to do here +//! is turn "no port on this thread" into a `kj::Exception` with a useful message instead of the +//! panic tokio would raise. + +use std::future::Future; + +use crate::error::KjIoError; +use crate::error::Result; + +/// Errors (`kj::Exception`-convertible) unless this thread owns a `TokioEventPort`. +pub fn require_loop_runtime() -> Result<()> { + if kj_rs_tokio::current_handle().is_some() { + Ok(()) + } else { + Err(KjIoError::other( + "kj_rs_io", + "no kj-rs-tokio runtime on this thread; kj-rs-io requires a TokioEventPort \ + (see kj_rs_io::setupTokioAsyncIo())", + )) + } +} + +/// Aborts the wrapped tokio task when dropped. Used by the "task forwards a result over a +/// oneshot" pattern (see `net.rs::resolve_host`): if the awaiting bridged future is dropped (KJ +/// promise cancelled), the forwarding task is aborted instead of lingering until its underlying +/// operation completes on its own. +pub struct AbortOnDrop(pub tokio::task::JoinHandle<()>); + +impl Drop for AbortOnDrop { + fn drop(&mut self) { + self.0.abort(); + } +} + +/// Runs `fut` after checking the loop-runtime precondition (the check happens once, when the +/// future is first polled -- not per poll). +pub async fn on_loop_runtime(fut: impl Future>) -> Result { + require_loop_runtime()?; + fut.await +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn require_loop_runtime_errors_without_a_port() { + // A thread with no TokioEventPort has no loop runtime; the precondition must be a + // kj::Exception-convertible error naming it, not a tokio panic. + let err = cxx::KjError::from(require_loop_runtime().unwrap_err()); + assert!( + err.description() + .contains("no kj-rs-tokio runtime on this thread"), + "{}", + err.description() + ); + } +} diff --git a/src/rust/cxx/kj-rs-io/serve.rs b/src/rust/cxx/kj-rs-io/serve.rs new file mode 100644 index 00000000000..df2c2bece32 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/serve.rs @@ -0,0 +1,459 @@ +//! The native-serve entry points: give a Rust server (any tokio consumer) the +//! best-available tokio-side byte stream for an owned `kj::AsyncIoStream`. +//! +//! Three tiers, two entry points: +//! (1) kj-rs-io-originated streams give up their native tokio socket (the hollow +//! wrapper is destroyed); (2) [`take_kj_socket`] only — a foreign fd-backed stream's fd is +//! duplicated into a fresh tokio socket; (3) [`serve_kj_stream`] only — any other foreign +//! stream is bridged through an in-memory duplex plus a pump future that owns it. Ownership +//! arrives as a [`KjOwn`], so both entry points are safe functions: there is no +//! keep-it-alive caller contract, and the stream is destroyed by the tier that consumed it. +//! See the two entry points' docs for the remaining semantics, in particular why the fd tier +//! is caller-asserted and never automatic. +//! +//! # Pump semantics (matching the hand-built pumps this subsumes) +//! +//! - Bidirectional; each direction ends independently. +//! - Half-close propagates both ways: kj-side EOF shuts down the duplex write half (the tokio +//! consumer reads EOF); the consumer shutting down (or dropping) its duplex end results in +//! `shutdownWrite()` on the kj stream. +//! - Peer-teardown-shaped kj failures (DISCONNECTED reads/writes) are treated as normal EOF, +//! not errors — abrupt client disconnects are normal server load. +//! - Dropping the pump future cancels the in-flight bridged kj promises synchronously, drops +//! the kj-side duplex end (the tokio consumer observes EOF), and destroys the owned kj +//! stream — the peer observes teardown, not a zombie half-open connection (abort-on-drop). + +use std::future::Future; +use std::pin::Pin; +use std::task::Context; +use std::task::Poll; + +use cxx::KjError; +use cxx::KjException; +use cxx::KjExceptionType; +use kj_rs::KjOwn; +use tokio::io::AsyncRead; +use tokio::io::AsyncReadExt; +use tokio::io::AsyncWrite; +use tokio::io::AsyncWriteExt; +use tokio::io::DuplexStream; +use tokio::io::ReadBuf; +use tokio::net::TcpStream; +#[cfg(unix)] +use tokio::net::UnixStream; + +use crate::ffi::KjAsyncIoStream; +#[cfg(unix)] +use crate::ffi::dup_raw_fd; +#[cfg(unix)] +use crate::ffi::kj_stream_get_handle; +use crate::ffi::split_kj_stream; +use crate::ffi::unwrap_tokio_stream; + +/// Read chunk size for the pump fallback. +const PUMP_BUF: usize = 8192; + +/// In-memory buffer per direction of the pump's duplex (how far the two sides may run ahead +/// of each other before backpressure). +pub(crate) const DUPLEX_CAPACITY: usize = 4 * PUMP_BUF; + +/// Which transport path [`serve_kj_stream`] produced (perf observability: `Pumped` costs FFI +/// promise round-trips per buffer, `Native` costs none). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ServePath { + /// A native tokio socket (unwrap fast path). + Native, + /// An in-memory duplex fed by the FFI stream pump. + Pumped, +} + +impl std::fmt::Display for ServePath { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(match self { + Self::Native => "native", + Self::Pumped => "pumped", + }) + } +} + +/// The tokio-side byte stream for a served kj stream: a native socket or the consumer end of +/// the pump's duplex. +/// +/// Implements `AsyncRead + AsyncWrite`, so it drops into any tokio consumer, on any runtime +/// thread -- see [`ServedKjStream::io`] for the (performance, not soundness) note on driving +/// `Duplex` off the KJ event-loop thread. +pub enum ServeIo { + Tcp(TcpStream), + #[cfg(unix)] + Unix(UnixStream), + Duplex(DuplexStream), +} + +impl ServeIo { + /// Which transport path this stream is on (see [`ServePath`]). + #[must_use] + pub fn path(&self) -> ServePath { + match self { + Self::Tcp(_) => ServePath::Native, + #[cfg(unix)] + Self::Unix(_) => ServePath::Native, + Self::Duplex(_) => ServePath::Pumped, + } + } + + /// Sets `TCP_NODELAY` on the underlying socket where applicable (a no-op for Unix-domain + /// and duplex transports, which have no Nagle to disable). + /// + /// # Errors + /// + /// Returns the underlying `std::io::Error` if setting the socket option fails. + pub fn set_nodelay(&self, nodelay: bool) -> std::io::Result<()> { + match self { + Self::Tcp(s) => s.set_nodelay(nodelay), + #[cfg(unix)] + Self::Unix(_) => Ok(()), + Self::Duplex(_) => Ok(()), + } + } +} + +impl AsyncRead for ServeIo { + fn poll_read( + self: Pin<&mut Self>, + cx: &mut Context<'_>, + buf: &mut ReadBuf<'_>, + ) -> Poll> { + match self.get_mut() { + Self::Tcp(s) => Pin::new(s).poll_read(cx, buf), + #[cfg(unix)] + Self::Unix(s) => Pin::new(s).poll_read(cx, buf), + Self::Duplex(s) => Pin::new(s).poll_read(cx, buf), + } + } +} + +impl AsyncWrite for ServeIo { + fn poll_write( + self: Pin<&mut Self>, + cx: &mut Context<'_>, + buf: &[u8], + ) -> Poll> { + match self.get_mut() { + Self::Tcp(s) => Pin::new(s).poll_write(cx, buf), + #[cfg(unix)] + Self::Unix(s) => Pin::new(s).poll_write(cx, buf), + Self::Duplex(s) => Pin::new(s).poll_write(cx, buf), + } + } + + fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + match self.get_mut() { + Self::Tcp(s) => Pin::new(s).poll_flush(cx), + #[cfg(unix)] + Self::Unix(s) => Pin::new(s).poll_flush(cx), + Self::Duplex(s) => Pin::new(s).poll_flush(cx), + } + } + + fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + match self.get_mut() { + Self::Tcp(s) => Pin::new(s).poll_shutdown(cx), + #[cfg(unix)] + Self::Unix(s) => Pin::new(s).poll_shutdown(cx), + Self::Duplex(s) => Pin::new(s).poll_shutdown(cx), + } + } + + fn poll_write_vectored( + self: Pin<&mut Self>, + cx: &mut Context<'_>, + bufs: &[std::io::IoSlice<'_>], + ) -> Poll> { + match self.get_mut() { + Self::Tcp(s) => Pin::new(s).poll_write_vectored(cx, bufs), + #[cfg(unix)] + Self::Unix(s) => Pin::new(s).poll_write_vectored(cx, bufs), + Self::Duplex(s) => Pin::new(s).poll_write_vectored(cx, bufs), + } + } + + fn is_write_vectored(&self) -> bool { + match self { + Self::Tcp(s) => s.is_write_vectored(), + #[cfg(unix)] + Self::Unix(s) => s.is_write_vectored(), + Self::Duplex(s) => s.is_write_vectored(), + } + } +} + +/// The KJ-side pump future of the fallback path. +/// +/// Not `Send`: it awaits bridged `kj::Promise`s and must be polled on the KJ event-loop thread +/// owning the stream. Resolves when both directions are done; dropping it aborts the +/// connection bridge (see the module docs). +pub type StreamPump = Pin>>>; + +/// The result of [`serve_kj_stream`]. +pub struct ServedKjStream { + /// The tokio-side stream. + /// + /// Thread affinity: every variant may be handed to a connection task on any runtime thread. + /// The NATIVE variants (`Tcp`/`Unix`) stay registered with the I/O driver that created them + /// (for kj-rs-io streams, this thread's KJ-loop runtime) and wake their consumer via + /// tokio's own task waker. The [`ServeIo::Duplex`] variant's peer end lives inside `pump`, + /// which is polled as a bridged future, so the waker parked in the duplex's internal waker + /// slots is a `kj_rs` `FutureWakerCell` clone -- atomically refcounted and honoring the full + /// `Waker: Send + Sync` contract: a read/write/drop of the duplex from another thread wakes + /// the pump through the cell's cross-thread fulfiller. That hop is a bit more expensive + /// than a same-thread wake, so co-locating a `Duplex` consumer with the KJ event-loop + /// thread is a performance recommendation (check [`ServedKjStream::path`]), not a + /// soundness requirement. + pub io: ServeIo, + /// Present iff `io` is [`ServeIo::Duplex`]: the pump that actually moves the bytes, owning + /// the kj stream it bridges. The caller must poll it on the KJ event-loop thread until it + /// settles or is dropped; dropping it destroys the stream (see the module docs). + pub pump: Option, +} + +impl ServedKjStream { + /// Which transport path was taken (see [`ServePath`]). + #[must_use] + pub fn path(&self) -> ServePath { + self.io.path() + } +} + +// ======================================================================================= +// Entry points + +/// [`take_kj_socket`]'s error: the failure, plus the untouched stream handed back so the +/// caller can fall back to [`serve_kj_stream`]'s pump tier (or destroy it). +pub struct TakeSocketError { + /// The stream `take_kj_socket` consumed, returned untouched. + pub stream: KjOwn, + /// Why the socket could not be taken natively. + pub error: KjError, +} + +impl std::fmt::Debug for TakeSocketError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("TakeSocketError") + .field("error", &self.error) + .finish_non_exhaustive() + } +} + +impl std::fmt::Display for TakeSocketError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{:?}", self.error) + } +} + +/// Drops the handed-back stream (on the current — KJ event-loop — thread) and keeps the error. +impl From for KjError { + fn from(e: TakeSocketError) -> Self { + e.error + } +} + +/// Tier-1 unwrap: if the owned stream is kj-rs-io-originated, moves its native tokio socket +/// out (leaving the C++ wrapper hollow) and returns it. `None` for a foreign stream. +fn unwrap_native(stream: &mut KjOwn) -> Option { + unwrap_tokio_stream(stream.as_mut()) + .ok() + .and_then(|native| native.into_serve_io()) +} + +/// Takes the stream's socket natively (tiers 1 + 2). +/// +/// Tier 1 moves the native tokio object out of a kj-rs-io stream; tier 2 — unix only — +/// duplicates the stream's OS fd (`F_DUPFD_CLOEXEC`, forced non-blocking) into a fresh tokio +/// socket. Either way the result is an owned, KJ-independent [`ServeIo`] (never +/// [`ServeIo::Duplex`]) and the consumed kj stream is destroyed before returning. +/// +/// The handle tier (tier 2) is `cfg(unix)`: under the all-rust mode on Windows every +/// socket-backed stream originates in kj-rs-io, so tier 1 always applies and a windows dup arm +/// would be dead code (see `dup_raw_fd`'s docs for the analysis and the +/// `BorrowedSocket::try_clone_to_owned` escape hatch should that ever change). +/// +/// The caller asserts the stream is a **plain stream socket**: if it exposes an fd, that fd +/// carries the stream's own bytes. Byte-transforming wrappers (TLS — `kj::TlsConnection` +/// forwards `getFd()` to the ciphertext transport socket) violate this and must go through +/// [`serve_kj_stream`] instead. As with destroying any kj stream, no I/O promises may be +/// outstanding on it when ownership is handed over (for kj-rs-io streams the unwrap detects +/// and rejects that; for foreign streams it remains KJ's own contract). +/// +/// # Errors +/// +/// Errors when the stream is neither kj-rs-io native nor fd-backed (in-memory pipes, promised +/// streams, non-Unix platforms' foreign streams): such transports can only be served through +/// [`serve_kj_stream`]'s pump tier — the error hands the stream back for exactly that +/// fallback. Also errors if the dup or tokio registration fails. +pub fn take_kj_socket( + stream: KjOwn, +) -> std::result::Result { + let mut stream = stream; + if let Some(io) = unwrap_native(&mut stream) { + // Tier 1: the wrapper is hollow; destroy it now. + drop(stream); + return Ok(io); + } + #[cfg(unix)] + { + let handle = kj_stream_get_handle(&stream); + if handle >= 0 { + // A unix fd is a non-negative int, widened losslessly to i64 by the bridge. + #[expect(clippy::cast_possible_truncation)] + let fd = handle as i32; + let result = dup_raw_fd(fd) + .map_err(KjError::from) + .and_then(|owned| crate::net::serve_io_from_owned_fd(owned).map_err(KjError::from)); + return match result { + Ok(io) => { + // Tier 2: the dup is independent; the original stream (and its fd) can go. + drop(stream); + Ok(io) + } + Err(error) => Err(TakeSocketError { stream, error }), + }; + } + } + Err(TakeSocketError { + stream, + error: KjError::new( + KjExceptionType::Failed, + "cannot take the stream's socket natively: not a kj-rs-io stream and no underlying \ + OS fd (foreign non-socket transport; serve it through serve_kj_stream's pump instead)" + .to_owned(), + ), + }) +} + +/// Yields the best-available tokio-side stream for the owned `stream`. +/// +/// That is the native tokio object when `stream` originated in kj-rs-io, else an in-memory +/// duplex bridged by a pump future that owns the stream. Never extracts a foreign stream's fd +/// — kj wrappers forward `getFd()` to their transport socket, so a byte-transforming wrapper's +/// fd carries the wrong bytes (TLS ciphertext); callers that can assert a plain socket should +/// prefer [`take_kj_socket`]. +/// +/// On the native path the hollow wrapper is destroyed before returning; on the pump path the +/// stream lives inside the pump and is destroyed when the pump settles or is dropped. The pump +/// must only be polled from the KJ event-loop thread owning the stream. As with destroying any +/// kj stream, no I/O promises may be outstanding on it when ownership is handed over (for +/// kj-rs-io streams the unwrap detects and rejects that; for foreign streams it remains KJ's +/// own contract). +#[must_use] +pub fn serve_kj_stream(stream: KjOwn) -> ServedKjStream { + let mut stream = stream; + if let Some(io) = unwrap_native(&mut stream) { + // Native path: the wrapper is hollow; destroy it now. + drop(stream); + return ServedKjStream { io, pump: None }; + } + + // Foreign stream (or, pathologically, an already-hollow wrapper, whose pump reads will + // surface the "already unwrapped" error): bridge through a duplex pump owning the stream. + let (consumer_end, kj_end) = tokio::io::duplex(DUPLEX_CAPACITY); + let pump = Box::pin(pump_kj_stream(stream, kj_end)); + ServedKjStream { + io: ServeIo::Duplex(consumer_end), + pump: Some(pump), + } +} + +/// Whether a bridged kj exception is peer-teardown-shaped (treated as EOF by the pump). +fn is_disconnected(exception: &KjException) -> bool { + exception.r#type() == KjExceptionType::Disconnected +} + +/// The duplex pump: bridges the owned `stream` (via bridged `kj::io` promises, on the calling +/// KJ thread) to `kj_end`, the pump-side end of the consumer's duplex. Owns the stream: it is +/// destroyed when this future settles or is dropped. +/// +/// Unsafe-free and compiler-checked end to end: the stream is split into typed read/write +/// halves ([`split_kj_stream`]), so the borrow checker enforces kj's stream contract — at most +/// one read and one write in flight, nothing else touching the stream while the halves live — +/// and proves the owner outlives every in-flight bridged promise. +pub(crate) async fn pump_kj_stream( + mut stream: KjOwn, + kj_end: DuplexStream, +) -> Result<(), KjError> { + let (mut rd, mut wr) = split_kj_stream(&mut stream); + let (mut from_consumer, mut to_consumer) = tokio::io::split(kj_end); + + // kj stream -> consumer. Ends (shutting down the duplex write half, i.e. EOF to the + // consumer) at kj-side EOF — or when the peer disconnects abruptly (DISCONNECTED read + // failures are normal client behavior, treated as EOF). + let kj_to_consumer = async { + let mut buf = vec![0u8; PUMP_BUF]; + loop { + let n = match rd.try_read(&mut buf, 1).await { + Ok(n) => n, + Err(e) if is_disconnected(&e) => 0, + Err(e) => return Err(KjError::from(e)), + }; + if n == 0 { + let _ = to_consumer.shutdown().await; + return Ok::<(), KjError>(()); + } + if to_consumer.write_all(&buf[..n]).await.is_err() { + // The consumer dropped its duplex end: the connection was abandoned + // deliberately; nothing more to deliver in this direction. + return Ok(()); + } + } + }; + + // consumer -> kj stream. Ends (with a kj-side shutdownWrite) when the consumer shuts + // down or drops its end — or, without an error, when the kj peer already went away (a + // DISCONNECTED write failure: the consumer's remaining output has nowhere to go). + let consumer_to_kj = async { + let mut buf = vec![0u8; PUMP_BUF]; + loop { + let n = match from_consumer.read(&mut buf).await { + // Duplex reads only fail if the consumer end vanished ungracefully; either + // way this direction is over. + Ok(0) | Err(_) => 0, + Ok(n) => n, + }; + if n == 0 { + // The consumer is done writing: half-close the kj side. A peer that already + // vanished (DISCONNECTED) is the same outcome for this direction. + match wr.shutdown_write() { + Ok(()) => {} + Err(e) if is_disconnected(&e) => {} + Err(e) => return Err(KjError::from(e)), + } + return Ok::<(), KjError>(()); + } + match wr.write(&buf[..n]).await { + Ok(()) => {} + Err(e) if is_disconnected(&e) => return Ok(()), + Err(e) => return Err(KjError::from(e)), + } + } + }; + + tokio::try_join!(kj_to_consumer, consumer_to_kj).map(|((), ())| ()) +} + +#[cfg(test)] +mod send_guards { + use static_assertions::assert_impl_all; + use static_assertions::assert_not_impl_any; + + use super::*; + + // The pump awaits bridged kj::Promises and owns a KjOwn: it must be polled on the KJ + // event-loop thread, which the type system enforces only while this stays true. + assert_not_impl_any!(StreamPump: Send, Sync); + + // The consumer-side stream may be handed to a connection task on any runtime thread + // (native variants: tokio's own wakers; Duplex: the thread-safe kj-rs waker cell). + assert_impl_all!(ServeIo: Send); + + // Hands a KjOwn back to the caller: a KJ object, single-loop, never Send. + assert_not_impl_any!(TakeSocketError: Send, Sync); +} diff --git a/src/rust/cxx/kj-rs-io/signal.rs b/src/rust/cxx/kj-rs-io/signal.rs new file mode 100644 index 00000000000..e6b52e10c4f --- /dev/null +++ b/src/rust/cxx/kj-rs-io/signal.rs @@ -0,0 +1,94 @@ +//! tokio-backed signal watching: POSIX signals on Unix, the corresponding console control +//! events on Windows. +//! +//! This backs `kj_rs_io::onSignal()` (async-io.h), the tokio-loop replacement for +//! `kj::UnixEventPort::onSignal()` -- workerd uses it for SIGTERM graceful drain. +//! +//! Semantics differences vs `UnixEventPort::onSignal()` (acceptable for the drain use case): +//! +//! - No `siginfo_t` is reported; the promise just resolves. +//! - The handler is registered when the returned future is first polled (tokio registers with +//! the process-global signal registry at `signal()` time), not at call time, and KJ does not +//! block the signal beforehand the way `UnixEventPort::captureSignal()` does. A signal +//! delivered before the first poll takes its default disposition. +//! - tokio's signal registration is process-wide and persists for the life of the process +//! (dropping the future stops *watching*, but does not restore `SIG_DFL`). +//! +//! Cross-thread delivery note: tokio's signal registry is process-global, and its broadcast can +//! run on a different thread — another runtime's loop thread on unix (whichever runtime's driver +//! consumes the signal's wake byte, e.g. workerd's inspector thread), or the OS-spawned +//! console-ctrl thread on Windows. That is fine: the kj-rs waker bridge is thread-safe (a +//! cross-thread wake is delivered through the `FutureWakerCell`'s cross-thread fulfiller; see +//! kj-rs/waker.h), so the streams are awaited directly from the bridged future here. The +//! multi-runtime case is covered by the "onSignal is delivered even when another runtime's +//! thread consumes the signal" test in tests/async-io-test.c++ — the scenario that, before the +//! bridge was thread-safe, made workerd ignore SIGTERM whenever the inspector thread's runtime +//! won the race. +//! +//! On Windows the signums workerd actually passes are mapped to their conventional console +//! control events: SIGTERM -> `ctrl_shutdown`, SIGINT -> `ctrl_c`. Anything else errors. + +use crate::error::KjIoError; +use crate::error::Result; +use crate::runtime::on_loop_runtime; + +/// Resolves when the process receives signal `signum` (on Windows: the console control event +/// conventionally mapped to it). Errors immediately for unmapped signums / other platforms. +pub async fn wait_for_signal(signum: i32) -> Result<()> { + #[cfg(unix)] + { + use crate::error::op; + on_loop_runtime(async move { + let kind = tokio::signal::unix::SignalKind::from_raw(signum); + let mut sig = tokio::signal::unix::signal(kind).map_err(op("signal"))?; + sig.recv() + .await + .ok_or_else(|| KjIoError::other("signal", "signal stream closed unexpectedly"))?; + Ok(()) + }) + .await + } + // Validated by Windows CI. tokio's `SetConsoleCtrlHandler` handler broadcasts from an + // OS-spawned console-ctrl thread; the thread-safe waker bridge absorbs that (see the module + // doc), so this arm awaits directly too. + #[cfg(windows)] + { + use crate::error::op; + // `` values as the C++ callers pass them (MSVC defines SIGINT=2, SIGTERM=15). + // workerd's only caller passes SIGTERM (graceful drain; server/cli-io-backend.c++); + // SIGINT is mapped for completeness. + const SIGINT: i32 = 2; + const SIGTERM: i32 = 15; + on_loop_runtime(async move { + // SIGTERM -> ctrl_shutdown, SIGINT -> ctrl_c (the conventional mappings). + let received = match signum { + SIGTERM => { + let mut sig = tokio::signal::windows::ctrl_shutdown().map_err(op("signal"))?; + sig.recv().await + } + SIGINT => { + let mut sig = tokio::signal::windows::ctrl_c().map_err(op("signal"))?; + sig.recv().await + } + _ => { + return Err(KjIoError::other( + "signal", + "kj-rs-io only watches SIGTERM/SIGINT on Windows", + )); + } + }; + received + .ok_or_else(|| KjIoError::other("signal", "signal stream closed unexpectedly"))?; + Ok(()) + }) + .await + } + #[cfg(not(any(unix, windows)))] + { + let _ = signum; + Err(KjIoError::other( + "signal", + "kj-rs-io signal watching is not implemented on this platform", + )) + } +} diff --git a/src/rust/cxx/kj-rs-io/stream.rs b/src/rust/cxx/kj-rs-io/stream.rs new file mode 100644 index 00000000000..723270be4b9 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/stream.rs @@ -0,0 +1,848 @@ +//! Tokio-backed byte streams behind KJ's stream interfaces. +//! +//! `TokioStream` (TCP or Unix domain) implements the `kj::AsyncIoStream` operations; all I/O +//! uses tokio's `&self` readiness API (`ready()` + `try_read`/`try_write`), which supports +//! concurrent reads and writes on one stream and is cancel-safe: dropping a pending future +//! (i.e. dropping the wrapping `kj::Promise`) merely deregisters the waker, releasing the +//! readiness interest so the stream can be reused or dropped sanely. + +use std::cell::Ref; +use std::cell::RefCell; +use std::io::IoSlice; +use std::io::Read; +use std::io::Write; + +use tokio::io::Interest; +use tokio::net::TcpStream; +#[cfg(unix)] +use tokio::net::UnixStream; + +use crate::error::KjIoError; +use crate::error::Result; +use crate::error::op; +use crate::runtime::on_loop_runtime; + +/// A native tokio stream (the "unwrap fast path" object). +/// +/// C++ holds one of these inside every kj-rs-io `kj::AsyncIoStream`; Rust code can take it +/// back out via [`crate::unwrap_kj_stream`] and drive the connection natively. +/// +/// # Ownership of the native stream while I/O is in flight +/// +/// Every I/O operation borrows the native stream for its whole duration (across its awaits), +/// and [`TokioStream::take`] moves it out. Those two must never overlap -- the moved-out object +/// is exactly what the pending operation is using. Rather than making that a caller contract +/// ("no I/O promises may be in flight when unwrapping"), the `RefCell` makes it checked: each +/// operation holds a shared borrow (`Ref`) of the slot for as long as it runs, and `take()` +/// needs the exclusive borrow, so unwrapping with I/O in flight fails with an error instead of +/// aliasing live borrows. This is why `TokioStream` is `!Sync`: it is a KJ-loop-thread object +/// (its owner, the C++ `kj::AsyncIoStream` wrapper, is), never shared across threads. +pub struct TokioStream { + /// `None` after the native stream has been moved out by [`TokioStream::take`] (the C++ + /// wrapper is then "hollow" and every operation fails). + inner: RefCell>, +} + +enum Inner { + Tcp(TcpStream), + #[cfg(unix)] + Unix(UnixStream), +} + +impl Inner { + async fn ready(&self, interest: Interest) -> Result<()> { + match self { + Self::Tcp(s) => s.ready(interest).await, + #[cfg(unix)] + Self::Unix(s) => s.ready(interest).await, + } + .map_err(op("poll()"))?; + Ok(()) + } + + fn try_read(&self, buf: &mut [u8]) -> std::io::Result { + match self { + Self::Tcp(s) => s.try_read(buf), + #[cfg(unix)] + Self::Unix(s) => s.try_read(buf), + } + } + + fn try_write(&self, buf: &[u8]) -> std::io::Result { + match self { + Self::Tcp(s) => s.try_write(buf), + #[cfg(unix)] + Self::Unix(s) => s.try_write(buf), + } + } + + fn try_write_vectored(&self, bufs: &[IoSlice<'_>]) -> std::io::Result { + match self { + Self::Tcp(s) => s.try_write_vectored(bufs), + #[cfg(unix)] + Self::Unix(s) => s.try_write_vectored(bufs), + } + } + + /// Borrows the live tokio socket's fd (tokio streams implement `AsFd`), for dup-based + /// operations that must not conjure a raw fd out of an integer. + #[cfg(unix)] + fn as_borrowed_fd(&self) -> std::os::fd::BorrowedFd<'_> { + use std::os::fd::AsFd; + match self { + Self::Tcp(s) => s.as_fd(), + Self::Unix(s) => s.as_fd(), + } + } + + /// Borrows the live tokio socket's `SOCKET` (tokio's `TcpStream` implements `AsSocket`): + /// the Windows counterpart of [`Inner::as_borrowed_fd`]. On Windows only the Tcp variant + /// exists. Validated by Windows CI. + #[cfg(windows)] + fn as_borrowed_socket(&self) -> std::os::windows::io::BorrowedSocket<'_> { + use std::os::windows::io::AsSocket; + match self { + Self::Tcp(s) => s.as_socket(), + } + } +} + +fn hollow() -> KjIoError { + KjIoError::other("kj_rs_io", "stream was unwrapped (hollow wrapper)") +} + +impl TokioStream { + fn new(inner: Inner) -> Self { + Self { + inner: RefCell::new(Some(inner)), + } + } + + #[must_use] + pub fn from_tcp(stream: TcpStream) -> Self { + Self::new(Inner::Tcp(stream)) + } + + #[cfg(unix)] + #[must_use] + pub fn from_unix(stream: UnixStream) -> Self { + Self::new(Inner::Unix(stream)) + } + + /// Recovers the native tokio TCP stream, if this is a (non-hollow) TCP stream. + #[must_use] + pub fn into_tcp_stream(self) -> Option { + match self.inner.into_inner() { + Some(Inner::Tcp(stream)) => Some(stream), + _ => None, + } + } + + /// Recovers the native tokio Unix-domain stream, if this is one. + #[cfg(unix)] + #[must_use] + pub fn into_unix_stream(self) -> Option { + match self.inner.into_inner() { + Some(Inner::Unix(stream)) => Some(stream), + _ => None, + } + } + + /// Borrows the live native stream for the duration of an operation. The returned `Ref` is + /// what makes a concurrent [`TokioStream::take`] fail (see the type docs). Errors if the + /// wrapper is hollow. + fn live(&self) -> Result> { + // `take()` is synchronous and never yields while holding the exclusive borrow, so a + // failed shared borrow here cannot happen in practice; report it rather than panic. + let slot = self + .inner + .try_borrow() + .map_err(|_| KjIoError::other("kj_rs_io", "stream is being unwrapped concurrently"))?; + Ref::filter_map(slot, Option::as_ref).map_err(|_| hollow()) + } + + /// Runs `f` on the live socket's borrowed fd. Errors if the wrapper is hollow. + #[cfg(unix)] + pub(crate) fn with_borrowed_fd( + &self, + f: impl FnOnce(std::os::fd::BorrowedFd<'_>) -> T, + ) -> Result { + let inner = self.live()?; + Ok(f(inner.as_borrowed_fd())) + } + + /// Runs `f` on the live socket's borrowed `SOCKET`. Errors if the wrapper is hollow. + /// Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + pub(crate) fn with_borrowed_socket( + &self, + f: impl FnOnce(std::os::windows::io::BorrowedSocket<'_>) -> T, + ) -> Result { + let inner = self.live()?; + Ok(f(inner.as_borrowed_socket())) + } + + /// KJ `tryRead` semantics: loop until at least `min_bytes` (or EOF), up to `buf.len()`. + // Holding the `Ref` across awaits is the whole point (see the type docs): it reserves the + // native stream for this operation so a concurrent `take()`/unwrap fails cleanly. Sound + // here: the only conflicting accessor is `take()`, which uses `try_borrow_mut` and returns + // an error rather than panicking, and concurrent read+write just stack shared `Ref`s. + #[expect( + clippy::await_holding_refcell_ref, + reason = "intentional in-flight guard; see above" + )] + async fn try_read_min(&self, buf: &mut [u8], min_bytes: usize) -> Result { + let inner = self.live()?; + let min_bytes = min_bytes.min(buf.len()); + let mut total = 0; + while total < min_bytes { + match inner.try_read(&mut buf[total..]) { + Ok(0) => break, // EOF: return what we have (< min_bytes signals EOF to KJ). + Ok(n) => total += n, + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => { + inner.ready(Interest::READABLE).await?; + } + Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {} + Err(e) => return Err(op("read()")(e)), + } + } + Ok(total) + } + + /// Write-all semantics. + // Holding the `Ref` across awaits is the whole point (see the type docs): it reserves the + // native stream for this operation so a concurrent `take()`/unwrap fails cleanly. Sound + // here: the only conflicting accessor is `take()`, which uses `try_borrow_mut` and returns + // an error rather than panicking, and concurrent read+write just stack shared `Ref`s. + #[expect( + clippy::await_holding_refcell_ref, + reason = "intentional in-flight guard; see above" + )] + async fn write_all(&self, buf: &[u8]) -> Result<()> { + let inner = self.live()?; + let mut written = 0; + while written < buf.len() { + match inner.try_write(&buf[written..]) { + Ok(0) => { + // try_write on a socket signals "would block" via Err(WouldBlock), so a + // zero-byte result for a non-empty buffer means the connection is gone. + return Err(KjIoError::other( + "write()", + "wrote zero bytes (connection closed)", + )); + } + Ok(n) => written += n, + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => { + inner.ready(Interest::WRITABLE).await?; + } + Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {} + Err(e) => return Err(op("write()")(e)), + } + } + Ok(()) + } + + /// Write-all semantics over several pieces, as one operation: `writev` until every piece is + /// fully written. `IoSlice::advance_slices` drops fully-written leading pieces and trims a + /// partially-written one, so a short write resumes exactly where the kernel stopped. + // Holding the `Ref` across awaits is the whole point (see the type docs): it reserves the + // native stream for this operation so a concurrent `take()`/unwrap fails cleanly. Sound + // here: the only conflicting accessor is `take()`, which uses `try_borrow_mut` and returns + // an error rather than panicking, and concurrent read+write just stack shared `Ref`s. + #[expect( + clippy::await_holding_refcell_ref, + reason = "intentional in-flight guard; see above" + )] + async fn write_all_pieces(&self, pieces: &crate::ffi::KjPieces) -> Result<()> { + let inner = self.live()?; + let count = crate::ffi::kj_pieces_count(pieces); + let mut slices: Vec> = (0..count) + .map(|index| IoSlice::new(crate::ffi::kj_piece(pieces, index))) + .collect(); + let mut bufs: &mut [IoSlice<'_>] = &mut slices; + loop { + // All-empty remainder (including all-empty input): nothing left to write. Checked + // before the syscall because writev of zero bytes returns 0, which below means + // "connection closed". + if bufs.iter().all(|piece| piece.is_empty()) { + return Ok(()); + } + match inner.try_write_vectored(bufs) { + Ok(0) => { + return Err(KjIoError::other( + "writev()", + "wrote zero bytes (connection closed)", + )); + } + Ok(n) => IoSlice::advance_slices(&mut bufs, n), + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => { + inner.ready(Interest::WRITABLE).await?; + } + Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {} + Err(e) => return Err(op("writev()")(e)), + } + } + } + + /// Resolves once new writes are doomed to fail (peer reset / hangup observed). + /// + /// tokio has no direct primitive for this, so we register a *duplicate* of the socket fd + /// with the I/O driver for WRITABLE interest and wait — explicitly clearing plain-writable + /// readiness — until the OS reports write-closed (kqueue: `EV_EOF` on the write filter; + /// epoll: `EPOLLHUP`/`EPOLLERR`) or an error. Like KJ's own implementation this does *not* + /// fire on a mere half-close (peer FIN / `EPOLLRDHUP`): reads hitting EOF must not count as + /// write-disconnect. + /// + /// Why a second fd rather than the socket's own registration: tokio caches readiness per + /// registration, and `try_write` consults that cache before making the syscall -- if the + /// cached WRITABLE bit is clear it returns `WouldBlock` and the writer parks in + /// `ready(WRITABLE)` until the *next* epoll/kqueue edge. This waiter must clear WRITABLE to + /// avoid spinning on an always-writable socket, so sharing the registration with concurrent + /// writers would park them on an edge that never comes (the socket stays writable): a hang, + /// not a slowdown. A `dup(2)` is its own registration, so clearing its readiness disturbs no + /// one. The cost is one extra fd and one extra driver registration for as long as the wait + /// is pending; kj-http holds one per server connection. + /// + /// Windows behavior (see the arm below): a never-resolving future, which IS KJ + /// parity — KJ's *current* Windows behavior (`whenWriteDisconnected` returns `NEVER_DONE`). + /// Win32 has no documented primitive for detecting disconnect without a read/write; KJ's own + /// TODO points at the undocumented-but-stable `IOCTL_AFD_POLL` ioctl (capnproto + /// `async-io-win32.c++:289` — the mechanism `select()` itself is built on). An AFD-poll + /// implementation remains an optional upgrade over this documented parity behavior. + #[cfg(unix)] + // Holding the `Ref` across awaits is the whole point (see the type docs): it reserves the + // native stream for this operation so a concurrent `take()`/unwrap fails cleanly. Sound + // here: the only conflicting accessor is `take()`, which uses `try_borrow_mut` and returns + // an error rather than panicking, and concurrent read+write just stack shared `Ref`s. + #[expect( + clippy::await_holding_refcell_ref, + reason = "intentional in-flight guard; see above" + )] + async fn when_write_disconnected(&self) -> Result<()> { + use std::os::fd::AsRawFd; + + // Borrow the live socket's fd directly (tokio streams implement `AsFd`) and dup it, so + // no raw fd is ever materialized without an owner. The `Ref` is held for the whole wait: + // this IS an in-flight operation as far as `take()` is concerned. + let inner = self.live()?; + let borrowed = inner.as_borrowed_fd(); + let owned = borrowed.try_clone_to_owned().map_err(op("dup()"))?; + debug_assert_ne!(owned.as_raw_fd(), borrowed.as_raw_fd()); + // The dup shares the underlying open socket (and its O_NONBLOCK status), but has its own + // registration with the I/O driver, so clearing readiness here never disturbs reads or + // writes on the primary registration. + let async_fd = tokio::io::unix::AsyncFd::with_interest(owned, Interest::WRITABLE) + .map_err(op("whenWriteDisconnected"))?; + loop { + let mut guard = async_fd + .ready(Interest::WRITABLE) + .await + .map_err(op("whenWriteDisconnected"))?; + let ready = guard.ready(); + if ready.is_write_closed() || ready.is_error() { + return Ok(()); + } + // Plain "writable": clear it so the next wait sleeps until an actual state-change + // event (edge-triggered), rather than spinning on an always-writable socket. + guard.clear_ready(); + } + } + + /// Never resolves: KJ parity, not a gap — capnproto's win32 `whenWriteDisconnected` returns + /// `NEVER_DONE` today (its `IOCTL_AFD_POLL` idea is only a TODO; see the Windows-behavior note + /// on the unix arm above). Validated by Windows CI. + #[cfg(windows)] + #[expect( + clippy::await_holding_refcell_ref, + reason = "intentional in-flight guard; see above" + )] + async fn when_write_disconnected(&self) -> Result<()> { + // Holds the in-flight guard (like the unix arm) and never resolves. `pending()` infers + // the Result<()> return type, so there is no unreachable tail (crate-level + // deny(clippy::unreachable)). + let _inner = self.live()?; + std::future::pending().await + } + + #[cfg(not(any(unix, windows)))] + async fn when_write_disconnected(&self) -> Result<()> { + self.live()?; + Err(KjIoError::other( + "whenWriteDisconnected", + "not implemented by kj-rs-io on this platform", + )) + } + + fn shutdown_write(&self) -> Result<()> { + // `shutdown(2)` acts on the socket, not on a descriptor, so a `SockRef` borrow of the + // live socket is all it needs: no dup, no owning std type, identical on unix and windows. + #[cfg(any(unix, windows))] + { + self.with_sock_ref("shutdown(SHUT_WR)", |sock| { + sock.shutdown(std::net::Shutdown::Write) + }) + } + #[cfg(not(any(unix, windows)))] + { + Err(KjIoError::other( + "shutdownWrite", + "not implemented on this platform", + )) + } + } + + /// Runs `f` on a `socket2::SockRef` borrowing the live socket — the shared body of the + /// `getsockname()`/`getpeername()` passthroughs; only the socket borrow is per-platform. + fn with_sock_ref( + &self, + op_name: &'static str, + f: impl FnOnce(&socket2::SockRef<'_>) -> std::io::Result, + ) -> Result { + #[cfg(unix)] + { + self.with_borrowed_fd(|fd| f(&socket2::SockRef::from(&fd)))? + .map_err(op(op_name)) + } + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + { + self.with_borrowed_socket(|sock| f(&socket2::SockRef::from(&sock)))? + .map_err(op(op_name)) + } + #[cfg(not(any(unix, windows)))] + { + let _ = f; + self.live()?; + Err(KjIoError::other( + op_name, + "not implemented by kj-rs-io on this platform", + )) + } + } + + /// Raw `struct sockaddr` bytes of the socket's locally-bound address (the `getsockname()` + /// passthrough behind `kj::AsyncIoStream::getsockname`). + fn local_addr_bytes(&self) -> Result> { + let addr = self.with_sock_ref("getsockname()", |sock| sock.local_addr())?; + Ok(crate::ffi::sockaddr_to_bytes(&addr)) + } + + /// Raw `struct sockaddr` bytes of the connected peer's address (the `getpeername()` + /// passthrough behind `kj::AsyncIoStream::getpeername` and the accept-loop peer-filter + /// check). + fn peer_addr_bytes(&self) -> Result> { + let addr = self.with_sock_ref("getpeername()", |sock| sock.peer_addr())?; + Ok(crate::ffi::sockaddr_to_bytes(&addr)) + } + + /// The underlying raw OS socket handle, widened to `i64`: a Unix fd + /// (`kj::AsyncIoStream::getFd()`) or a win32 `SOCKET` (`getWin32Handle()`). + fn raw_handle(&self) -> Result { + #[cfg(unix)] + { + use std::os::fd::AsRawFd; + self.with_borrowed_fd(|fd| i64::from(fd.as_raw_fd())) + } + // Validated by Windows CI; mirrors the unix arm. + #[cfg(windows)] + { + use std::os::windows::io::AsRawSocket; + // A live SOCKET fits in i64 (Windows handles fit in 32 bits); the bridge carries + // its bits verbatim. + #[allow(clippy::cast_possible_wrap)] + self.with_borrowed_socket(|sock| sock.as_raw_socket() as i64) + } + #[cfg(not(any(unix, windows)))] + { + self.live()?; + Err(KjIoError::other( + "getFd", + "file descriptors are not available on this platform", + )) + } + } + + /// Recovers whichever native tokio object this is, as a [`crate::serve::ServeIo`] + /// (the unwrap fast path of [`crate::serve_kj_stream`]). `None` if hollow. + pub(crate) fn into_serve_io(self) -> Option { + match self.inner.into_inner()? { + Inner::Tcp(stream) => Some(crate::serve::ServeIo::Tcp(stream)), + #[cfg(unix)] + Inner::Unix(stream) => Some(crate::serve::ServeIo::Unix(stream)), + } + } + + /// Moves the native stream out into a fresh wrapper, leaving this one hollow. Fails if the + /// wrapper is already hollow, or if any I/O operation is in flight (see the type docs): + /// that is the checked replacement for the old "no I/O promises may be outstanding" caller + /// contract. + fn take(&self) -> Result> { + let mut slot = self.inner.try_borrow_mut().map_err(|_| { + KjIoError::other( + "kj_rs_io", + "cannot unwrap a stream while I/O operations are in flight on it", + ) + })?; + let inner = slot.take().ok_or_else(|| { + KjIoError::other("kj_rs_io", "stream was already unwrapped (hollow wrapper)") + })?; + Ok(Box::new(Self::new(inner))) + } +} + +// ====================================================================================== +// Bridge entry points (see lib.rs). Every async fn checks the loop-runtime precondition once +// (`on_loop_runtime`); the loop thread is permanently inside the runtime's context, so resources +// created while polling on the KJ thread reach the loop runtime's I/O driver on their own. + +pub async fn stream_try_read( + stream: &TokioStream, + buf: &mut [u8], + min_bytes: usize, +) -> Result { + on_loop_runtime(stream.try_read_min(buf, min_bytes)).await +} + +pub async fn stream_write(stream: &TokioStream, buf: &[u8]) -> Result<()> { + on_loop_runtime(stream.write_all(buf)).await +} + +pub async fn stream_write_pieces( + stream: &TokioStream, + pieces: &crate::ffi::KjPieces, +) -> Result<()> { + on_loop_runtime(stream.write_all_pieces(pieces)).await +} + +pub async fn stream_when_write_disconnected(stream: &TokioStream) -> Result<()> { + on_loop_runtime(stream.when_write_disconnected()).await +} + +pub fn stream_shutdown_write(stream: &TokioStream) -> Result<()> { + stream.shutdown_write() +} + +pub fn stream_raw_handle(stream: &TokioStream) -> Result { + stream.raw_handle() +} + +/// Non-throwing variant for `kj::AsyncIoStream::getFd()`/`getWin32Handle()`, which return a +/// `kj::Maybe`: -1 when the wrapper is hollow (or the platform has no handle), so C++ does not +/// have to use exception catching as control flow. +pub fn stream_try_raw_handle(stream: &TokioStream) -> i64 { + stream.raw_handle().unwrap_or(-1) +} + +pub fn stream_local_addr(stream: &TokioStream) -> Result> { + stream.local_addr_bytes() +} + +pub fn stream_peer_addr(stream: &TokioStream) -> Result> { + stream.peer_addr_bytes() +} + +pub fn stream_take(stream: &TokioStream) -> Result> { + stream.take() +} + +// ====================================================================================== +// Arbitrary readable/writable fds (kj::LowLevelAsyncIoProvider::wrapInputFd/wrapOutputFd). +// Unix only: implemented over AsyncFd, which supports pipes, character devices and sockets +// (regular files are rejected by epoll/kqueue, matching KJ's fd-observer-based provider). +// Deliberately no windows arm: kj's win32 LowLevelAsyncIoProvider has no pipe-fd tier — its +// `Fd` is documented as a SOCKET (capnproto async-io.h) and its wrapInputFd/wrapOutputFd are +// implemented identically to wrapSocketFd (async-io-win32.c++) — so the C++ side +// (async-io.c++) routes win32 wrapInputFd/wrapOutputFd through the tested socket path +// (`wrap_socket_fd`) and never calls these entry points there; the `not(unix)` arms below are +// totality backstops only. + +#[cfg(unix)] +type FdIo = tokio::io::unix::AsyncFd; + +pub struct TokioInputFd { + #[cfg(unix)] + inner: FdIo, +} + +pub struct TokioOutputFd { + #[cfg(unix)] + inner: FdIo, +} + +#[cfg(unix)] +fn fd_io_from_raw(fd: i32, interest: Interest) -> Result { + let owned = crate::ffi::own_fd_from_raw(fd); + crate::runtime::require_loop_runtime()?; + tokio::io::unix::AsyncFd::with_interest(std::fs::File::from(owned), interest) + .map_err(op("wrapFd")) +} + +pub fn wrap_input_fd(fd: i32) -> Result> { + #[cfg(unix)] + { + Ok(Box::new(TokioInputFd { + inner: fd_io_from_raw(fd, Interest::READABLE)?, + })) + } + #[cfg(not(unix))] + { + let _ = fd; + Err(KjIoError::other( + "wrapInputFd", + "not implemented on this platform", + )) + } +} + +pub fn wrap_output_fd(fd: i32) -> Result> { + #[cfg(unix)] + { + Ok(Box::new(TokioOutputFd { + inner: fd_io_from_raw(fd, Interest::WRITABLE)?, + })) + } + #[cfg(not(unix))] + { + let _ = fd; + Err(KjIoError::other( + "wrapOutputFd", + "not implemented on this platform", + )) + } +} + +pub async fn input_fd_try_read( + stream: &TokioInputFd, + buf: &mut [u8], + min_bytes: usize, +) -> Result { + #[cfg(unix)] + { + on_loop_runtime(async move { + let min_bytes = min_bytes.min(buf.len()); + let mut total = 0; + while total < min_bytes { + let mut guard = stream + .inner + .ready(Interest::READABLE) + .await + .map_err(op("poll()"))?; + match guard.try_io(|inner| { + let mut file: &std::fs::File = inner.get_ref(); + file.read(&mut buf[total..]) + }) { + Ok(Ok(0)) => break, // EOF + Ok(Ok(n)) => total += n, + Ok(Err(e)) if e.kind() == std::io::ErrorKind::Interrupted => {} + Ok(Err(e)) => return Err(op("read()")(e)), + Err(_would_block) => {} + } + } + Ok(total) + }) + .await + } + #[cfg(not(unix))] + { + let _ = (stream, buf, min_bytes); + Err(KjIoError::other( + "read()", + "not implemented on this platform", + )) + } +} + +pub async fn output_fd_write(stream: &TokioOutputFd, buf: &[u8]) -> Result<()> { + #[cfg(unix)] + { + on_loop_runtime(async move { + let mut written = 0; + while written < buf.len() { + let mut guard = stream + .inner + .ready(Interest::WRITABLE) + .await + .map_err(op("poll()"))?; + match guard.try_io(|inner| { + let mut file: &std::fs::File = inner.get_ref(); + file.write(&buf[written..]) + }) { + Ok(Ok(0)) => { + return Err(KjIoError::other("write()", "wrote zero bytes")); + } + Ok(Ok(n)) => written += n, + Ok(Err(e)) if e.kind() == std::io::ErrorKind::Interrupted => {} + Ok(Err(e)) => return Err(op("write()")(e)), + Err(_would_block) => {} + } + } + Ok(()) + }) + .await + } + #[cfg(not(unix))] + { + let _ = (stream, buf); + Err(KjIoError::other( + "write()", + "not implemented on this platform", + )) + } +} + +#[cfg(test)] +mod tests { + use std::task::Context; + use std::task::Waker; + + use cxx::KjError; + use static_assertions::assert_impl_all; + use static_assertions::assert_not_impl_any; + + use super::*; + + // `TokioStream` is a KJ-loop-thread object whose in-flight-operation tracking lives in a + // `RefCell` (see the type docs): it must never become `Sync` by accident. It stays `Send`, + // like the native tokio sockets inside it, so `unwrap_kj_stream`'s `Box` can be + // handed to a connection task. + assert_not_impl_any!(TokioStream: Sync); + assert_impl_all!(TokioStream: Send); + + /// A connected localhost TCP pair as tokio streams registered with `port`'s runtime. + // Takes the port only to make the caller prove one exists: the thread must be inside the + // port's runtime context for `TcpStream::from_std` to find the I/O driver. + fn connected_pair(_port: &kj_rs_tokio::TokioPort) -> (TokioStream, std::net::TcpStream) { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + let client = std::net::TcpStream::connect(listener.local_addr().unwrap()).unwrap(); + let (server, _) = listener.accept().unwrap(); + server.set_nonblocking(true).unwrap(); + // No `enter()`: the thread is inside the port's runtime context for the port's life. + ( + TokioStream::from_tcp(TcpStream::from_std(server).unwrap()), + client, + ) + } + + #[test] + fn hollow_wrapper_rejects_every_operation_and_second_take() { + let port = kj_rs_tokio::TokioPort::new(); + let (stream, _client) = connected_pair(&port); + + let taken = match stream.take() { + Ok(taken) => taken, + Err(e) => panic!("first take: {}", KjError::from(e).description()), + }; + assert!( + taken.live().is_ok(), + "the taken wrapper holds the live stream" + ); + + // The original is hollow now. + let err = |r: Result<()>| match r { + Ok(()) => panic!("expected an error from a hollow wrapper"), + Err(e) => KjError::from(e).description().to_owned(), + }; + assert!(stream.live().is_err()); + assert!(err(stream.shutdown_write()).contains("hollow")); + assert!(stream.raw_handle().is_err()); + assert_eq!(stream_try_raw_handle(&stream), -1); + assert!(stream.local_addr_bytes().is_err()); + assert!(err(stream.take().map(drop)).contains("already unwrapped")); + assert!(stream.into_tcp_stream().is_none()); + } + + #[test] + fn take_while_an_operation_is_in_flight_is_an_error_not_aliasing() { + let port = kj_rs_tokio::TokioPort::new(); + let (stream, _client) = connected_pair(&port); + + // Start a read and park it: nothing has been written, so it registers readiness + // interest and returns Pending while holding its borrow of the native stream. The + // runtime guard is held across the poll so the tokio reactor is reachable. + let mut buf = [0u8; 8]; + let mut read = Box::pin(stream.try_read_min(&mut buf, 1)); + let mut cx = Context::from_waker(Waker::noop()); + assert!(read.as_mut().poll(&mut cx).is_pending()); + + // Unwrapping now would alias that live borrow; it is refused instead. + let err = match stream.take() { + Ok(_) => panic!("take() must fail while a read is in flight"), + Err(e) => KjError::from(e), + }; + assert!( + err.description().contains("in flight"), + "{}", + err.description() + ); + + // Once the operation is gone the unwrap goes through, and the native stream is intact. + drop(read); + let taken = match stream.take() { + Ok(taken) => taken, + Err(e) => panic!( + "take after the read was dropped: {}", + KjError::from(e).description() + ), + }; + assert!(taken.into_tcp_stream().is_some()); + } + + #[test] + fn concurrent_read_and_write_both_in_flight_share_the_borrow() { + // The load-bearing invariant of the RefCell design: shared borrows stack, so a read and + // a write can be in flight at once (kj's one-read + one-write contract), and `take()` + // fails while EITHER is alive, succeeding only once BOTH are dropped. + let port = kj_rs_tokio::TokioPort::new(); + let (stream, _client) = connected_pair(&port); + + let mut buf = [0u8; 8]; + let mut read = Box::pin(stream.try_read_min(&mut buf, 1)); + // 1 MiB: larger than the socket buffer, so the write cannot drain in one go and the + // future stays pending, holding its borrow. + let payload = vec![0u8; 1024 * 1024]; + let mut write = Box::pin(stream.write_all(&payload)); + let mut cx = Context::from_waker(Waker::noop()); + assert!(read.as_mut().poll(&mut cx).is_pending()); + assert!(write.as_mut().poll(&mut cx).is_pending()); + // Both borrows are live and coexist; take() is refused. + assert!( + stream.take().is_err(), + "take() must fail while read+write are both in flight" + ); + drop(read); + assert!( + stream.take().is_err(), + "take() must still fail while the write is in flight" + ); + drop(write); + assert!( + stream.take().is_ok(), + "take() succeeds once both operations are gone" + ); + } + + #[test] + fn hollow_into_serve_io_is_none() { + let port = kj_rs_tokio::TokioPort::new(); + let (stream, _client) = connected_pair(&port); + let _taken = stream.take().expect("first take"); + // The original is hollow; recovering a ServeIo yields None (the into_inner()? == None arm). + assert!(stream.into_serve_io().is_none()); + } + + #[cfg(unix)] + #[test] + fn write_disconnected_wait_counts_as_in_flight() { + let port = kj_rs_tokio::TokioPort::new(); + let (stream, _client) = connected_pair(&port); + let mut wait = Box::pin(stream.when_write_disconnected()); + let mut cx = Context::from_waker(Waker::noop()); + assert!(wait.as_mut().poll(&mut cx).is_pending()); + assert!( + stream.take().is_err(), + "whenWriteDisconnected holds the stream too" + ); + drop(wait); + assert!(stream.take().is_ok()); + } +} diff --git a/src/rust/cxx/kj-rs-io/tests/BUILD.bazel b/src/rust/cxx/kj-rs-io/tests/BUILD.bazel new file mode 100644 index 00000000000..71f3feb7632 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/BUILD.bazel @@ -0,0 +1,170 @@ +load("@rules_cc//cc:cc_test.bzl", "cc_test") +load("@rules_rust//rust:defs.bzl", "rust_library") +load("//src/rust/cxx/tools/bazel:rust_cxx_bridge.bzl", "rust_cxx_bridge") + +rust_library( + name = "tests", + srcs = glob(["*.rs"]), + edition = "2024", + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":bridge", + "//src/rust/cxx", + "//src/rust/cxx/kj-rs", + "//src/rust/cxx/kj-rs-io", + "//src/rust/cxx/kj-rs-tokio", + "@crates_vendor//:bytes", + "@crates_vendor//:tokio", + ], +) + +rust_cxx_bridge( + name = "bridge", + src = "lib.rs", + include_prefix = "kj-rs-io-test", + deps = [ + "//src/rust/cxx/kj-rs", + "//src/rust/cxx/kj-rs-io:bridge", + ], +) + +cc_test( + name = "async-io-test", + # medium: several tests do real socket I/O plus timer-bounded waits (backpressure, signals, + # DNS), which can exceed small's 60 s budget under load / sanitizers. + size = "medium", + srcs = [ + "async-io-test.c++", + "io-test-helpers.h", + ], + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":bridge", + ":tests", + "//src/rust/cxx/kj-rs-io:kj-rs-io-lib", + "//src/rust/cxx/third-party:runtime", + "@capnp-cpp//src/kj:kj-test", + ], +) + +cc_test( + name = "file-watcher-test", + # medium: real I/O plus timer-bounded waits; small's 3 s budget is exceeded under parallel + # build load even though the suite runs in ~2 s alone. + size = "medium", + srcs = ["file-watcher-test.c++"], + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":tests", + "//src/rust/cxx/kj-rs-io:kj-rs-io-lib", + "//src/rust/cxx/third-party:runtime", + "@capnp-cpp//src/kj:kj-test", + ], +) + +cc_test( + name = "http-test", + # medium: real I/O plus timer-bounded waits; small's 3 s budget is exceeded under parallel + # build load even though the suite runs in ~2 s alone. + size = "medium", + srcs = ["http-test.c++"], + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":tests", + "//src/rust/cxx/kj-rs-io:kj-rs-io-lib", + "//src/rust/cxx/third-party:runtime", + "@capnp-cpp//src/kj:kj-test", + "@capnp-cpp//src/kj/compat:kj-http", + ], +) + +cc_test( + name = "serve-test", + size = "small", + srcs = [ + "io-test-helpers.h", + "serve-test.c++", + ], + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":bridge", + ":tests", + "//src/rust/cxx/kj-rs-io:kj-rs-io-lib", + "//src/rust/cxx/third-party:runtime", + "@capnp-cpp//src/kj:kj-test", + ], +) + +cc_test( + name = "peer-filter-test", + size = "small", + srcs = ["peer-filter-test.c++"], + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":tests", + "//src/rust/cxx/kj-rs-io:kj-rs-io-lib", + "//src/rust/cxx/third-party:runtime", + "@capnp-cpp//src/kj:kj-test", + ], +) + +cc_test( + name = "capnp-rpc-test", + # medium: real I/O plus timer-bounded waits; small's 3 s budget is exceeded under parallel + # build load even though the suite runs in ~2 s alone. + size = "medium", + srcs = ["capnp-rpc-test.c++"], + linkstatic = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + target_compatible_with = select({ + "@//build/config:no_build": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + deps = [ + ":tests", + "//src/rust/cxx/kj-rs-io:kj-rs-io-lib", + "//src/rust/cxx/third-party:runtime", + "@capnp-cpp//src/capnp:capnp-rpc", + "@capnp-cpp//src/kj:kj-test", + ], +) diff --git a/src/rust/cxx/kj-rs-io/tests/async-io-test.c++ b/src/rust/cxx/kj-rs-io/tests/async-io-test.c++ new file mode 100644 index 00000000000..96320dfed07 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/async-io-test.c++ @@ -0,0 +1,1109 @@ +// Tests for kj-rs-io: tokio-backed implementations of KJ's async I/O interfaces, driven by a +// kj::EventLoop on a TokioEventPort. Following kj-rs conventions, C++ KJ_TESTs drive; Rust +// helpers (tests/lib.rs) provide the native-side behaviors (unwrap fast path, pre-bound fds). + +#include "io-test-helpers.h" +#include "kj-rs-io-test/lib.rs.h" +#include "kj-rs-io/async-io.h" + +#include +#include +#include +#include +#include + +#include + +#if _WIN32 +#include // Win32 APIs used by the Windows-only test arms below. + +// After windows.h: un-breaks macros it leaks over KJ's, notably ERROR (which otherwise breaks +// the KJ_LOG(ERROR, ...) inside KJ_FAIL_* expansions). +#include +#else +#include +#include +#include +#include +#include +#include +#endif + +namespace kj_rs_io_test { +namespace { + +using kj_rs_io::setupTokioAsyncIo; +using kj_rs_io::TokioAsyncIoContext; + +// ======================================================================================= +// Helpers (shared ones: io-test-helpers.h) + +::rust::Slice toRust(kj::ArrayPtr data) { + return ::rust::Slice(data.begin(), data.size()); +} + +::rust::Vec toRustVec(kj::ArrayPtr data) { + ::rust::Vec vec; + vec.reserve(data.size()); + for (auto b: data) { + vec.push_back(b); + } + return vec; +} + +// Waits for `connectPromise` (a connect to a certainly-closed port) to fail and returns the +// exception, bounded by a KJ timer so a never-settling connect fails the test with a message +// instead of eating the binary's bazel timeout. (An earlier version of this helper carried a +// watchdog thread and CPU accounting to diagnose a Windows CI wedge: a lost connect-readiness +// wake caused by the then single-threaded waker bridge. That bridge is thread-safe now and the +// wedge is gone with it; the timer bound is kept as a plain test hygiene measure.) +kj::Exception expectConnectFailure( + TokioAsyncIoContext &io, kj::Promise> connectPromise) { + auto timeout = + io.getTimer().afterDelay(30 * kj::SECONDS).then([]() -> kj::Own { + KJ_FAIL_ASSERT("connect() to a closed port neither succeeded nor failed within 30s"); + }); + return KJ_ASSERT_NONNULL(kj::runCatchingExceptions([&]() { + connectPromise.exclusiveJoin(kj::mv(timeout)).wait(io.getWaitScope()); + }), + "connect() to a closed port unexpectedly succeeded"); +} + +// ======================================================================================= +// Stream contract + +KJ_TEST("tryRead waits for minBytes, then returns what is available up to " + "maxBytes") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + kj::byte buffer[16]; + + // Exactly-min: 3 bytes written, min 3 -> resolves with 3. + pair.client->write("abc"_kjb).wait(ws); + KJ_EXPECT(pair.server->tryRead(buffer, 3, sizeof(buffer)).wait(ws) == 3); + KJ_EXPECT(kj::ArrayPtr(buffer, 3) == "abc"_kjb); + + // Blocks until minBytes: 2 available < min 5 -> pending; 3 more arrive -> resolves with 5. + pair.client->write("de"_kjb).wait(ws); + auto readPromise = pair.server->tryRead(buffer, 5, sizeof(buffer)); + KJ_EXPECT(!readPromise.poll(ws)); + pair.client->write("fgh"_kjb).wait(ws); + KJ_EXPECT(readPromise.wait(ws) == 5); + KJ_EXPECT(kj::ArrayPtr(buffer, 5) == "defgh"_kjb); +} + +KJ_TEST("EOF before minBytes returns a short count; half-close keeps the other " + "direction usable") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + pair.client->write("ab"_kjb).wait(ws); + pair.client->shutdownWrite(); + + // EOF-before-min: only 2 bytes then FIN -> tryRead(min 5) resolves with 2. + kj::byte buffer[16]; + KJ_EXPECT(pair.server->tryRead(buffer, 5, sizeof(buffer)).wait(ws) == 2); + KJ_EXPECT(kj::ArrayPtr(buffer, 2) == "ab"_kjb); + // Subsequent reads keep reporting EOF. + KJ_EXPECT(pair.server->tryRead(buffer, 1, sizeof(buffer)).wait(ws) == 0); + + // Half-close: server -> client direction still works after client's shutdownWrite. + pair.server->write("reply"_kjb).wait(ws); + KJ_EXPECT(pair.client->tryRead(buffer, 5, sizeof(buffer)).wait(ws) == 5); + KJ_EXPECT(kj::ArrayPtr(buffer, 5) == "reply"_kjb); +} + +KJ_TEST("multi-megabyte transfers in both directions with concurrent read+write " + "per stream") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + constexpr size_t SIZE = 8 * 1024 * 1024; + auto dataA = makePatternedData(SIZE, 1); + auto dataB = makePatternedData(SIZE, 2); + + // All four directions at once: each stream is simultaneously reading and writing, and each + // transfer is far larger than the socket buffers (forcing many readiness round-trips). + auto builder = kj::heapArrayBuilder>(4); + builder.add(writeChunked(*pair.client, dataA)); + builder.add(readExact(*pair.server, dataA)); + builder.add(writeChunked(*pair.server, dataB)); + builder.add(readExact(*pair.client, dataB)); + kj::joinPromisesFailFast(builder.finish()).wait(ws); +} + +KJ_TEST("multi-piece write() writes all pieces in order") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + const kj::ArrayPtr pieces[] = {"one,"_kjb, "two,"_kjb, "three"_kjb}; + pair.client->write(kj::arrayPtr(pieces, 3)).wait(ws); + + kj::byte buffer[32]; + KJ_EXPECT(pair.server->tryRead(buffer, 13, sizeof(buffer)).wait(ws) == 13); + KJ_EXPECT(kj::ArrayPtr(buffer, 13) == "one,two,three"_kjb); +} + +KJ_TEST("canceling a blocked read releases the socket for reuse") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + kj::byte buffer[16]; + { + // A read blocked in tokio (registered with the I/O driver, no data available)... + auto blocked = pair.server->tryRead(buffer, 1, sizeof(buffer)); + KJ_EXPECT(!blocked.poll(ws)); + // ...is canceled by dropping the promise, which must drop the Rust future and release the + // read interest. + } + + // The stream remains fully usable: a fresh read gets the next bytes. + pair.client->write("later"_kjb).wait(ws); + KJ_EXPECT(pair.server->tryRead(buffer, 5, sizeof(buffer)).wait(ws) == 5); + KJ_EXPECT(kj::ArrayPtr(buffer, 5) == "later"_kjb); + + // Canceling mid-large-write also leaves the process sane (bytes may be lost, like KJ). + { + auto data = makePatternedData(16 * 1024 * 1024, 7); + auto bigWrite = pair.client->write(data); + if (bigWrite.poll(ws)) { + bigWrite.wait(ws); + } + } +} + +#if !_WIN32 +KJ_TEST("whenWriteDisconnected resolves on peer reset, not on half-close") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + auto disconnected = pair.client->whenWriteDisconnected(); + KJ_EXPECT(!disconnected.poll(ws)); + + // A peer half-close (FIN) must NOT count as write-disconnect: the client can still write. + pair.server->shutdownWrite(); + kj::byte buffer[16]; + KJ_EXPECT(pair.client->tryRead(buffer, 1, sizeof(buffer)).wait(ws) == 0); // observe EOF + KJ_EXPECT(!disconnected.poll(ws)); + + // Destroying the server end with SO_LINGER=0 sends an RST; now writes are doomed. + struct linger lin; + lin.l_onoff = 1; + lin.l_linger = 0; + pair.server->setsockopt(SOL_SOCKET, SO_LINGER, &lin, sizeof(lin)); + pair.server = nullptr; + + disconnected.wait(ws); +} +#endif + +KJ_TEST("acceptAuthenticated reports the TCP peer's NetworkPeerIdentity") { + // workerd's HTTP listener builds the cf blob's clientIp (-> the CF-Connecting-IP header) from + // this identity; UnknownPeerIdentity (kj's base-class default) silently yields an empty + // client IP. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto listener = parseNow(io, "127.0.0.1")->listen(); + auto acceptPromise = listener->acceptAuthenticated(); + auto client = parseNow(io, kj::str("127.0.0.1:", listener->getPort()))->connect().wait(ws); + auto server = acceptPromise.wait(ws); + + auto &identity = + KJ_ASSERT_NONNULL(kj::tryDowncast(*server.peerIdentity)); + // KJ's "ip:port" format, byte-identical to the native backend. + auto text = identity.toString(); + KJ_EXPECT(text.startsWith("127.0.0.1:"), text); +} + +#if !_WIN32 +KJ_TEST("acceptAuthenticated reports LocalPeerIdentity credentials on unix sockets") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // /tmp rather than TEST_TMPDIR: sun_path is limited to ~104 bytes. + auto path = kj::str("/tmp/kj-rs-io-auth-test-", getpid(), ".sock"); + auto addr = parseNow(io, kj::str("unix:", path)); + + auto listener = addr->listen(); + auto acceptPromise = listener->acceptAuthenticated(); + auto client = addr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + + auto &identity = KJ_ASSERT_NONNULL(kj::tryDowncast(*server.peerIdentity)); + auto creds = identity.getCredentials(); + // The peer is this very process. + KJ_EXPECT(KJ_ASSERT_NONNULL(creds.pid) == getpid()); + KJ_EXPECT(KJ_ASSERT_NONNULL(creds.uid) == getuid()); + + unlink(path.cStr()); +} +#endif + +KJ_TEST("sockname/peername/sockopt/getFd passthrough") { +#if _WIN32 + return; +#else + auto io = setupTokioAsyncIo(); + auto pair = makeTcpPair(io); + + // getFd is populated. + KJ_EXPECT(KJ_ASSERT_NONNULL(pair.client->getFd()) >= 0); + + // The client's peer is the server's local socket. + struct sockaddr_in peer, local; + kj::uint peerLen = sizeof(peer), localLen = sizeof(local); + pair.client->getpeername(reinterpret_cast(&peer), &peerLen); + pair.server->getsockname(reinterpret_cast(&local), &localLen); + KJ_EXPECT(peer.sin_port == local.sin_port); + KJ_EXPECT(peer.sin_addr.s_addr == local.sin_addr.s_addr); + + // setsockopt/getsockopt round-trip (this is also how setNoDelay-style options are applied). + int on = 1; + pair.client->setsockopt(IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on)); + int result = 0; + kj::uint resultLen = sizeof(result); + pair.client->getsockopt(IPPROTO_TCP, TCP_NODELAY, &result, &resultLen); + KJ_EXPECT(result != 0); +#endif +} + +// ======================================================================================= +// Network / addresses + +KJ_TEST("parseAddress handles IP literals, port hints, and toString round-trips") { + auto io = setupTokioAsyncIo(); + + KJ_EXPECT(parseNow(io, "1.2.3.4:80")->toString() == "1.2.3.4:80"); + KJ_EXPECT(parseNow(io, "1.2.3.4", 99)->toString() == "1.2.3.4:99"); + KJ_EXPECT(parseNow(io, "[1234:5678::abcd]:80")->toString() == "[1234:5678::abcd]:80"); + KJ_EXPECT(parseNow(io, "1234:5678::abcd", 80)->toString() == "[1234:5678::abcd]:80"); + KJ_EXPECT(parseNow(io, "*:80")->toString() == "*:80"); + KJ_EXPECT(parseNow(io, "*")->toString() == "*:0"); + + // clone() produces an equivalent address. + auto addr = parseNow(io, "127.0.0.1:1234"); + KJ_EXPECT(addr->clone()->toString() == addr->toString()); +} + +KJ_TEST("wildcard listen binds dual-stack and reports its port; port 0 picks a " + "free port") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto listener = parseNow(io, "*:0")->listen(); + kj::uint port = listener->getPort(); + KJ_EXPECT(port != 0); + + // Reachable over both IPv4 and IPv6 loopback (IPV6_V6ONLY off, like KJ). + kj::String addrTexts[] = {kj::str("127.0.0.1:", port), kj::str("[::1]:", port)}; + for (auto &addrText: addrTexts) { + auto acceptPromise = listener->accept(); + auto client = parseNow(io, addrText)->connect().wait(ws); + auto server = acceptPromise.wait(ws); + client->write("ping"_kjb).wait(ws); + kj::byte buffer[4]; + KJ_EXPECT(server->tryRead(buffer, 4, sizeof(buffer)).wait(ws) == 4); + } +} + +KJ_TEST("parseAddress resolves hostnames via DNS") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // Listen on IPv4 loopback only. "localhost" resolves to 127.0.0.1 (and, on most systems, ::1 + // as well, which connect()'s per-address fallback -- tested deterministically below -- skips). + auto listener = parseNow(io, "127.0.0.1")->listen(); + auto addr = parseNow(io, kj::str("localhost:", listener->getPort())); + + auto acceptPromise = listener->accept(); + auto client = addr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + client->write("dns!"_kjb).wait(ws); + kj::byte buffer[4]; + KJ_EXPECT(server->tryRead(buffer, 4, sizeof(buffer)).wait(ws) == 4); + KJ_EXPECT(kj::ArrayPtr(buffer, 4) == "dns!"_kjb); +} + +KJ_TEST("connect() tries each resolved address in order, falling through refused ones") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // A port that is certainly closed: bind, note the port, close. + kj::uint closedPort = parseNow(io, "127.0.0.1")->listen()->getPort(); + auto listener = parseNow(io, "127.0.0.1")->listen(); + + // Two addresses, the first refused: a deterministic multi-result "DNS" answer. + uint16_t ports[] = { + static_cast(closedPort), static_cast(listener->getPort())}; + auto addr = kj::heap( + address_from_loopback_ports(::rust::Slice(ports, kj::size(ports))), + kj::rc()); + auto acceptPromise = listener->accept(); + auto client = addr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + client->write("2nd"_kjb).wait(ws); + kj::byte buffer[3]; + KJ_EXPECT(server->tryRead(buffer, 3, 3).wait(ws) == 3); + + // All refused: the LAST address's exception propagates (KJ parity). + kj::uint closedPort2 = parseNow(io, "127.0.0.1")->listen()->getPort(); + uint16_t closedPorts[] = {static_cast(closedPort), static_cast(closedPort2)}; + auto allClosed = kj::heap( + address_from_loopback_ports( + ::rust::Slice(closedPorts, kj::size(closedPorts))), + kj::rc()); + auto exception = expectConnectFailure(io, allClosed->connect()); + KJ_EXPECT(exception.getType() == kj::Exception::Type::DISCONNECTED, exception); + KJ_EXPECT(exception.getDescription().contains("connect()"), exception.getDescription()); +} + +KJ_TEST("connect to a closed port surfaces a DISCONNECTED kj::Exception " + "mentioning the refusal") { + auto io = setupTokioAsyncIo(); + + // Find a port that is certainly closed: bind one, note it, close it. + kj::uint port; + { + auto listener = parseNow(io, "127.0.0.1")->listen(); + port = listener->getPort(); + } + + auto addr = parseNow(io, kj::str("127.0.0.1:", port)); + auto exception = expectConnectFailure(io, addr->connect()); + // Exact text (recorded): "connect(): Connection refused (os error 61)" on macOS / + // "... (os error 111)" on Linux. KJ's native text would be "connect(): Connection refused". + KJ_EXPECT( + strstr(exception.getDescription().cStr(), "refused") != nullptr, exception.getDescription()); + KJ_EXPECT(exception.getType() == kj::Exception::Type::DISCONNECTED); +} + +KJ_TEST("the address may be dropped while connect() is pending (KJ lifetime " + "contract)") { + // Upstream KJ heap-copies the resolved address list into the connect promise + // (NetworkAddressImpl::connect() in kj/async-io-unix.c++), so callers may legally drop + // the kj::NetworkAddress right after calling connect(). Verify this port honors the same + // contract, on both the success path and the error/retry path. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // Success path: drop the address immediately, then complete the connect. + { + auto listener = parseNow(io, "127.0.0.1")->listen(); + auto acceptPromise = listener->accept(); + + kj::Promise> connectPromise = nullptr; + { + auto addr = parseNow(io, kj::str("127.0.0.1:", listener->getPort())); + connectPromise = addr->connect(); + // `addr` is destroyed here, while the connect is still in flight. + } + + auto client = connectPromise.wait(ws); + auto server = acceptPromise.wait(ws); + client->write("hello"_kjb).wait(ws); + kj::byte buffer[5] = {}; + KJ_EXPECT(server->tryRead(buffer, 5, 5).wait(ws) == 5); + KJ_EXPECT(kj::arrayPtr(buffer, 5) == "hello"_kjb); + } + + // Error path: connect to a certainly-closed port with the address already dropped; the + // failure continuation (which re-reads the address list) must still be safe and surface + // the normal exception. + { + kj::uint port; + { + auto listener = parseNow(io, "127.0.0.1")->listen(); + port = listener->getPort(); + } + + kj::Promise> connectPromise = nullptr; + { + auto addr = parseNow(io, kj::str("127.0.0.1:", port)); + connectPromise = addr->connect(); + } + + auto exception = expectConnectFailure(io, kj::mv(connectPromise)); + KJ_EXPECT(strstr(exception.getDescription().cStr(), "refused") != nullptr, + exception.getDescription()); + KJ_EXPECT(exception.getType() == kj::Exception::Type::DISCONNECTED); + } +} + +KJ_TEST("connecting to a wildcard address is an error") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto addr = parseNow(io, "*:1234"); + KJ_EXPECT_THROW_MESSAGE("wildcard", addr->connect().wait(ws)); +} + +#if !_WIN32 +KJ_TEST("unix domain sockets: parse, listen, connect, toString") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // Note: /tmp rather than TEST_TMPDIR because sun_path is limited to ~104 bytes. + auto path = kj::str("/tmp/kj-rs-io-test-", getpid(), ".sock"); + auto addrText = kj::str("unix:", path); + + auto addr = parseNow(io, addrText); + KJ_EXPECT(addr->toString() == addrText); + + auto listener = addr->listen(); + KJ_EXPECT(listener->getPort() == 0); // KJ reports 0 for non-IP listeners. + auto acceptPromise = listener->accept(); + auto client = addr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + + client->write("via unix"_kjb).wait(ws); + client->shutdownWrite(); + kj::byte buffer[16]; + KJ_EXPECT(server->tryRead(buffer, 16, sizeof(buffer)).wait(ws) == 8); + KJ_EXPECT(kj::ArrayPtr(buffer, 8) == "via unix"_kjb); + + unlink(path.cStr()); +} + +KJ_TEST("getSockaddr builds a connectable address from a raw struct sockaddr") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto listener = parseNow(io, "127.0.0.1")->listen(); + + struct sockaddr_in sin; + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_port = htons(static_cast(listener->getPort())); + sin.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + auto addr = io.getNetwork().getSockaddr(&sin, sizeof(sin)); + KJ_EXPECT(addr->toString() == kj::str("127.0.0.1:", listener->getPort())); + + auto acceptPromise = listener->accept(); + auto client = addr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + client->write("hi"_kjb).wait(ws); + kj::byte buffer[2]; + KJ_EXPECT(server->tryRead(buffer, 2, sizeof(buffer)).wait(ws) == 2); +} +#endif + +KJ_TEST("newPipeThread is a documented stub") { + auto io = setupTokioAsyncIo(); + KJ_EXPECT_THROW_MESSAGE("newPipeThread", + io.getProvider().newPipeThread( + [](kj::AsyncIoProvider &, kj::AsyncIoStream &, kj::WaitScope &) {})); +} + +// ======================================================================================= +// Provider odds and ends + +KJ_TEST("provider pipes (in-memory) and timer work under the tokio loop") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto pipe = io.getProvider().newTwoWayPipe(); + auto writePromise = pipe.ends[0]->write("pipe data"_kjb).eagerlyEvaluate(nullptr); + kj::byte buffer[16]; + KJ_EXPECT(pipe.ends[1]->tryRead(buffer, 9, sizeof(buffer)).wait(ws) == 9); + writePromise.wait(ws); + + auto &timer = io.getProvider().getTimer(); + auto before = timer.now(); + timer.afterDelay(5 * kj::MILLISECONDS).wait(ws); + KJ_EXPECT(timer.now() - before >= 5 * kj::MILLISECONDS); +} + +// ======================================================================================= +// Unwrap fast path + +KJ_TEST("unwrap fast path: recover the native tokio stream and write from Rust") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Recover the native tokio TcpStream out of the kj wrapper (free-function form, as a Rust + // server would after being handed a kj::AsyncIoStream&)... + auto native = kj_rs_io::unwrapTokioStream(*pair.server); + + // ...the hollow wrapper now refuses I/O... + kj::byte buffer[32]; + KJ_EXPECT_THROW_MESSAGE("unwrapped", pair.server->tryRead(buffer, 1, sizeof(buffer)).wait(ws)); + + // ...and Rust can drive the connection natively: it writes via the tokio readiness API and + // closes; C++ reads the bytes plus EOF through the (still wrapped) client end. + auto writeDone = native_write_via_unwrap(kj::mv(native), toRustVec("native write"_kjb)); + KJ_EXPECT(pair.client->tryRead(buffer, 12, sizeof(buffer)).wait(ws) == 12); + KJ_EXPECT(kj::ArrayPtr(buffer, 12) == "native write"_kjb); + writeDone.wait(ws); + KJ_EXPECT(pair.client->tryRead(buffer, 1, sizeof(buffer)).wait(ws) == 0); // EOF +} + +KJ_TEST("unwrap fast path: Rust-side unwrap_kj_stream() from a " + "kj::AsyncIoStream&") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Rust receives only a kj::AsyncIoStream& and performs the unwrap + native write itself. + auto writeDone = native_write_via_kj_unwrap(*pair.server, toRust("rust unwrap"_kjb)); + kj::byte buffer[32]; + KJ_EXPECT(pair.client->tryRead(buffer, 11, sizeof(buffer)).wait(ws) == 11); + KJ_EXPECT(kj::ArrayPtr(buffer, 11) == "rust unwrap"_kjb); + writeDone.wait(ws); + + // Unwrapping a foreign (non-kj-rs-io) stream fails cleanly. + auto pipe = io.getProvider().newTwoWayPipe(); + KJ_EXPECT_THROW_MESSAGE("cannot unwrap", kj_rs_io::unwrapTokioStream(*pipe.ends[0])); +} + +// ======================================================================================= +// Fd wrapping (kj::LowLevelAsyncIoProvider) + +#if !_WIN32 +KJ_TEST("wrapListenSocketFd accepts connections on a pre-bound listener (the " + "--socket-fd case)") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // Rust binds a *blocking* std listener (like an fd inherited from a supervisor) and hands us + // the raw fd; wrapListenSocketFd must take ownership and make it usable. + auto prebound = create_prebound_listener_fd(); + auto receiver = io.getLowLevelProvider().wrapListenSocketFd( + prebound.fd, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP); + KJ_EXPECT(receiver->getPort() == prebound.port); + + auto acceptPromise = receiver->accept(); + auto client = parseNow(io, kj::str("127.0.0.1:", prebound.port))->connect().wait(ws); + auto server = acceptPromise.wait(ws); + + client->write("fd listen"_kjb).wait(ws); + kj::byte buffer[16]; + KJ_EXPECT(server->tryRead(buffer, 9, sizeof(buffer)).wait(ws) == 9); + KJ_EXPECT(kj::ArrayPtr(buffer, 9) == "fd listen"_kjb); +} + +KJ_TEST("wrapSocketFd wraps both ends of a socketpair") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + int fds[2]; + KJ_SYSCALL(socketpair(AF_UNIX, SOCK_STREAM, 0, fds)); + auto end0 = + io.getLowLevelProvider().wrapSocketFd(fds[0], kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP); + auto end1 = + io.getLowLevelProvider().wrapSocketFd(fds[1], kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP); + + end0->write("socketpair"_kjb).wait(ws); + kj::byte buffer[16]; + KJ_EXPECT(end1->tryRead(buffer, 10, sizeof(buffer)).wait(ws) == 10); + KJ_EXPECT(kj::ArrayPtr(buffer, 10) == "socketpair"_kjb); +} + +KJ_TEST("wrapConnectingSocketFd completes a nonblocking connect") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto listener = parseNow(io, "127.0.0.1")->listen(); + + int fd; + KJ_SYSCALL(fd = socket(AF_INET, SOCK_STREAM, 0)); + struct sockaddr_in sin; + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_port = htons(static_cast(listener->getPort())); + sin.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + + auto acceptPromise = listener->accept(); + auto client = io.getLowLevelProvider() + .wrapConnectingSocketFd(fd, reinterpret_cast(&sin), + sizeof(sin), kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP) + .wait(ws); + auto server = acceptPromise.wait(ws); + + client->write("connected"_kjb).wait(ws); + kj::byte buffer[16]; + KJ_EXPECT(server->tryRead(buffer, 9, sizeof(buffer)).wait(ws) == 9); +} + +KJ_TEST("wrapInputFd/wrapOutputFd move bytes through an OS pipe and observe EOF") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + int fds[2]; + KJ_SYSCALL(pipe(fds)); + auto input = + io.getLowLevelProvider().wrapInputFd(fds[0], kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP); + auto output = + io.getLowLevelProvider().wrapOutputFd(fds[1], kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP); + + // Blocked read completes once data is written ("pipe fd io" is 10 bytes; cap the first read + // at 9 so one byte remains). + kj::byte buffer[16]; + auto readPromise = input->tryRead(buffer, 9, 9); + KJ_EXPECT(!readPromise.poll(ws)); + auto writePromise = output->write("pipe fd io"_kjb); + KJ_EXPECT(readPromise.wait(ws) == 9); + writePromise.wait(ws); + KJ_EXPECT(input->tryRead(buffer, 1, sizeof(buffer)).wait(ws) == 1); // "o" + + // Dropping the output stream closes the write end -> EOF. + output = nullptr; + KJ_EXPECT(input->tryRead(buffer, 1, sizeof(buffer)).wait(ws) == 0); +} + +KJ_TEST("restrictPeers blocks disallowed connect() with KJ's error text") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + kj_rs_io::TokioNetwork network; + auto restricted = network.restrictPeers({"public"_kj}, {}); + + // Loopback is not "public": blocked before any connection attempt. + auto blockedAddr = restricted->parseAddress("127.0.0.1:1").wait(ws); + KJ_EXPECT_THROW_MESSAGE("connect() blocked by restrictPeers()", blockedAddr->connect().wait(ws)); + + // getSockaddr is rejected eagerly, like KJ. + struct sockaddr_in sin; + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_port = htons(1); + sin.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + KJ_EXPECT_THROW_MESSAGE( + "address blocked by restrictPeers()", restricted->getSockaddr(&sin, sizeof(sin))); + + // An allowing restriction still connects. + auto allowed = network.restrictPeers({"private"_kj}, {}); + auto listener = network.parseAddress("127.0.0.1", 0).wait(ws)->listen(); + auto acceptPromise = listener->accept(); + auto client = allowed->parseAddress(kj::str("127.0.0.1:", listener->getPort())) + .wait(ws) + ->connect() + .wait(ws); + auto server = acceptPromise.wait(ws); + client->write("ok"_kjb).wait(ws); + kj::byte buffer[2]; + KJ_EXPECT(server->tryRead(buffer, 2, 2).wait(ws) == 2); +} + +KJ_TEST("restrictPeers filters accepted peers (disallowed peers are dropped, " + "accept keeps waiting)") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + kj_rs_io::TokioNetwork network; + auto restricted = network.restrictPeers({"public"_kj}, {}); + + auto listener = restricted->parseAddress("127.0.0.1", 0).wait(ws)->listen(); + auto acceptPromise = listener->accept(); + + // Connect via the unrestricted network; the loopback peer is not "public", so the listener + // silently drops it: accept() stays pending and the client observes EOF. + auto client = network.parseAddress(kj::str("127.0.0.1:", listener->getPort()), 0) + .wait(ws) + ->connect() + .wait(ws); + KJ_EXPECT(!acceptPromise.poll(ws)); + kj::byte buffer[1]; + KJ_EXPECT(client->tryRead(buffer, 1, 1).wait(ws) == 0); +} + +KJ_TEST("onSignal is delivered even when another runtime's thread consumes the signal") { + // Regression test for a SIGTERM hang observed in workerd: tokio's signal registry is + // process-global, and whichever runtime's driver consumes the signal's wake byte performs the + // broadcast. With a second tokio runtime parked on another thread (workerd's inspector thread, + // in the wild), the broadcast often runs on THAT thread — a cross-thread wake of this loop's + // waker. Before the kj-rs waker bridge was thread-safe, that wake was lost and workerd ignored + // SIGTERM until killed; now it must always be delivered (kj-rs/waker.h's cross-thread path). + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // A second, idle tokio-ported KJ loop parked on another thread for the duration of the test. + // The shutdown promise must be created ON that thread's loop (kj promises are single-loop + // objects); only its CrossThreadPromiseFulfiller half comes back to this thread. + kj::MutexGuarded>>> shutdown; + kj::Thread otherLoop([&shutdown]() { + auto io2 = setupTokioAsyncIo(); + auto paf = kj::newPromiseAndCrossThreadFulfiller(); + *shutdown.lockExclusive() = kj::mv(paf.fulfiller); + paf.promise.wait(io2.getWaitScope()); + }); + KJ_DEFER({ + auto lock = shutdown.lockExclusive(); + if (*lock != kj::none) { + KJ_ASSERT_NONNULL(*lock)->fulfill(); + } + }); + // Wait until the other loop is up and parked (and the shutdown fulfiller exists) before + // raising any signals, so its runtime genuinely participates in the wake-byte race. + shutdown.when([](auto &maybe) { return maybe != kj::none; }, [](auto &) {}); + + // Several rounds, giving each runtime chances to win the wake-byte race. Bounded so a lost + // wake fails with a diagnosis instead of eating the binary's bazel timeout. + for (int i = 0; i < 5; i++) { + auto promise = kj_rs_io::onSignal(SIGUSR2); + // Pump the loop so the handler is installed before we raise (see the test below). + KJ_EXPECT(!promise.poll(ws)); + KJ_SYSCALL(kill(getpid(), SIGUSR2)); + promise + .exclusiveJoin(io.getTimer().afterDelay(20 * kj::SECONDS).then([]() { + KJ_FAIL_ASSERT("onSignal wake was lost (cross-thread waker regression)"); + })).wait(ws); + } +} + +KJ_TEST("onSignal resolves when the process receives the signal") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto promise = kj_rs_io::onSignal(SIGUSR2); + // Pump the loop so the (cold, first-poll-registered) tokio signal handler is installed + // before we raise; raising first would take SIGUSR2's default disposition (terminate). + KJ_EXPECT(!promise.poll(ws)); + + KJ_SYSCALL(kill(getpid(), SIGUSR2)); + promise.wait(ws); + + // A second watcher works too (the process-global registration is reusable). + auto again = kj_rs_io::onSignal(SIGUSR2); + KJ_EXPECT(!again.poll(ws)); + KJ_SYSCALL(kill(getpid(), SIGUSR2)); + again.wait(ws); +} +#endif + +// ======================================================================================= +// Review-driven hardening tests (re-review of Part2 against the prep-review lens). + +KJ_TEST("concurrent acceptAuthenticated on two tokio-ported loops does not race the filter") { + // Regression guard for the per-identity allow-all filter: it must NOT be a process-wide + // static kj::Rc (non-atomic refcount) shared across accept loops on different threads. Two + // loops, each accepting a TCP connection and building a NetworkPeerIdentity (which creates + // that filter), running concurrently. TSAN target; also ASAN-visible as a double-free if the + // refcount ever races. + constexpr kj::uint N = 40; + auto runOne = []() noexcept { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + for (kj::uint i = 0; i < N; i++) { + auto listener = parseNow(io, "127.0.0.1")->listen(); + auto connectAddr = parseNow(io, kj::str("127.0.0.1:", listener->getPort())); + auto acceptPromise = listener->acceptAuthenticated(); + auto client = connectAddr->connect().wait(ws); + auto authed = acceptPromise.wait(ws); + // Touch the identity so the allow-all filter is actually built and addRef'd. + KJ_EXPECT(authed.peerIdentity->toString() != nullptr); + } + }; + kj::Thread other(runOne); + runOne(); +} + +KJ_TEST("restrictPeers: a child network (and its addresses) outlive the parent network") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // Build a restricted child, then DROP the parent network while keeping the child and an + // address parsed from it. The refcounted PeerFilter chain must keep the parent's filter alive + // through the child's Rc, so the child stays usable. + kj::Own addr; + kj::Own child; + { + auto parent = io.getNetwork().restrictPeers({"public"_kj}, {}); + child = parent->restrictPeers({"private"_kj}, {}); + addr = child->parseAddress("127.0.0.1:1").wait(ws); + // `parent` drops here. + } + // The child still works (its filter chain is intact): a private address connect is blocked + // with KJ's error text, proving the (grand)parent rules still apply. + auto blocked = kj::runCatchingExceptions([&]() { addr->connect().wait(ws); }); + KJ_EXPECT(blocked != kj::none); + KJ_EXPECT(KJ_ASSERT_NONNULL(blocked).getDescription().contains("restrictPeers")); +} + +KJ_TEST("dropping a just-started connect() then tearing down the context is clean") { + // Start a connect(), kick the machinery with one poll, then drop the promise and destroy the + // whole context -- exercising cancellation of the connect's readiness registration and the + // teardown that follows (ASAN target). Whether the connect has settled by the poll is + // environment-dependent and irrelevant: either way the drop + teardown must be clean. + // 198.51.100.1 is TEST-NET-2 (RFC 5737), non-routable, so it usually stays pending. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto addr = parseNow(io, "198.51.100.1:80"); + auto connectPromise = addr->connect(); + connectPromise.poll(ws); + { auto dropped = kj::mv(connectPromise); } + KJ_EXPECT(kj::evalLater([]() { return 1; }).wait(ws) == 1); +} + +KJ_TEST("canceling a pending accept() then accepting again works") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto listener = parseNow(io, "127.0.0.1")->listen(); + auto connectAddr = parseNow(io, kj::str("127.0.0.1:", listener->getPort())); + + // Start an accept with no client, then drop it. + { + auto pending = listener->accept(); + KJ_EXPECT(!pending.poll(ws)); + } + + // The listener is still usable: a fresh accept completes against a new client. + auto acceptPromise = listener->accept(); + auto client = connectAddr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + KJ_EXPECT(server->write("x"_kjb).then([]() { return true; }).wait(ws)); +} + +KJ_TEST("context teardown while a DNS parseAddress() is in flight is clean") { + // parseAddress() of a hostname spawns a runtime task (getaddrinfo on the blocking pool). Drop + // the promise mid-lookup and tear the context down: the spawned task must be cancelled without + // touching freed KJ state (ASAN target; the kj-rs-tokio teardown-order fix covers this). + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto parsePromise = io.getNetwork().parseAddress("example.invalid:80"); + // `.invalid` never resolves to success, but the lookup is in flight after one poll. + parsePromise.poll(ws); + { auto dropped = kj::mv(parsePromise); } + KJ_EXPECT(kj::evalLater([]() { return 2; }).wait(ws) == 2); +} + +KJ_TEST("zero-length write() is a no-op that succeeds") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + pair.client->write(kj::ArrayPtr()).wait(ws); + // The stream is still fully usable afterwards. + pair.client->write("hi"_kjb).wait(ws); + kj::byte buf[2]; + KJ_EXPECT(pair.server->tryRead(buf, 2, 2).wait(ws) == 2); +} + +#if !_WIN32 +KJ_TEST("write() to a reset peer surfaces a DISCONNECTED exception") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // RST the server end (SO_LINGER=0 close), then write from the client until the RST is + // observed. The first write may still succeed into the local send buffer, so loop; the + // failure, when it comes, must be DISCONNECTED (EPIPE/ECONNRESET), not FAILED. + struct linger lin; + lin.l_onoff = 1; + lin.l_linger = 0; + pair.server->setsockopt(SOL_SOCKET, SO_LINGER, &lin, sizeof(lin)); + pair.server = nullptr; + + auto chunk = kj::heapArray(64 * 1024); + memset(chunk.begin(), 0, chunk.size()); + kj::Maybe maybeException; + for (int i = 0; i < 100 && maybeException == kj::none; i++) { + maybeException = kj::runCatchingExceptions([&]() { pair.client->write(chunk).wait(ws); }); + } + auto &exception = KJ_ASSERT_NONNULL(maybeException, "write to a reset peer should fail"); + KJ_EXPECT(exception.getType() == kj::Exception::Type::DISCONNECTED, exception.getDescription()); +} + +KJ_TEST("write_all applies backpressure: it stays pending against a peer that never reads") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Write far more than the combined send+recv socket buffers to a peer that never reads: the + // write must NOT complete (write_all's try_write hits WouldBlock and awaits WRITABLE). + auto payload = kj::heapArray(16 * 1024 * 1024); + memset(payload.begin(), 0x5a, payload.size()); + auto write = pair.client->write(payload); + // Give the loop real turns; a correct write stays pending under backpressure. + for (int i = 0; i < 5; i++) { + io.getTimer().afterDelay(5 * kj::MILLISECONDS).wait(ws); + } + KJ_EXPECT(!write.poll(ws), "write_all must not complete while the peer never reads"); + + // Now drain on the peer; the write completes. + auto drain = [](kj::AsyncIoStream &s, size_t total) -> kj::Promise { + auto buf = kj::heapArray(256 * 1024); + size_t got = 0; + while (got < total) { + size_t n = co_await s.tryRead(buf.begin(), 1, buf.size()); + if (n == 0) break; + got += n; + } + }(*pair.server, payload.size()); + write.exclusiveJoin(kj::mv(drain)).wait(ws); +} + +KJ_TEST("getSockaddr builds a connectable IPv6 address from a raw sockaddr_in6") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + auto listener = parseNow(io, "[::1]")->listen(); + + struct sockaddr_in6 sin6; + memset(&sin6, 0, sizeof(sin6)); + sin6.sin6_family = AF_INET6; + sin6.sin6_port = htons(static_cast(listener->getPort())); + sin6.sin6_addr = in6addr_loopback; + auto addr = io.getNetwork().getSockaddr(&sin6, sizeof(sin6)); + KJ_EXPECT(addr->toString() == kj::str("[::1]:", listener->getPort())); + + auto acceptPromise = listener->accept(); + auto client = addr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + client->write("v6"_kjb).wait(ws); + kj::byte buf[2]; + KJ_EXPECT(server->tryRead(buf, 2, 2).wait(ws) == 2); +} + +KJ_TEST("multiple concurrent onSignal for the same signum all fire") { + // tokio broadcasts a signal to every live stream for that signum, so two concurrent + // onSignal(SIGUSR2) must both resolve on a single delivery. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto a = kj_rs_io::onSignal(SIGUSR2); + auto b = kj_rs_io::onSignal(SIGUSR2); + KJ_EXPECT(!a.poll(ws)); // both handlers installed before we raise + KJ_EXPECT(!b.poll(ws)); + KJ_SYSCALL(kill(getpid(), SIGUSR2)); + a.wait(ws); + b.wait(ws); +} + +KJ_TEST("onSignal isolates different signums") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto usr1 = kj_rs_io::onSignal(SIGUSR1); + auto usr2 = kj_rs_io::onSignal(SIGUSR2); + KJ_EXPECT(!usr1.poll(ws)); + KJ_EXPECT(!usr2.poll(ws)); + KJ_SYSCALL(kill(getpid(), SIGUSR2)); + usr2.wait(ws); + // Only SIGUSR2 was raised; the SIGUSR1 watcher stays pending. + KJ_EXPECT(!usr1.poll(ws)); +} + +KJ_TEST("dropping a pending onSignal does not break later watches") { + // Cancel a registered-but-unfired signal watch, then confirm a fresh watch still delivers -- + // the dropped tokio signal stream must not disturb the process-global registration. ASAN- + // relevant (the drop cancels the stream). + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + { + auto dropped = kj_rs_io::onSignal(SIGUSR2); + KJ_EXPECT(!dropped.poll(ws)); + } + auto again = kj_rs_io::onSignal(SIGUSR2); + KJ_EXPECT(!again.poll(ws)); + KJ_SYSCALL(kill(getpid(), SIGUSR2)); + again.wait(ws); +} + +KJ_TEST("onSignal for an unwatchable signum errors instead of aborting") { + // SIGKILL/SIGSTOP cannot have handlers; tokio's signal() rejects them, which must surface as a + // catchable kj::Exception (a rejected promise), never a crash. No signal is raised. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + KJ_EXPECT_THROW_MESSAGE("signal", kj_rs_io::onSignal(SIGKILL).wait(ws)); +} +#endif // !_WIN32 + +// ======================================================================================= +// Added coverage: vectored writes under backpressure, DNS failure, unix bind collisions, +// acceptAuthenticated through a restricted listener. + +KJ_TEST("multi-piece write() larger than the socket buffer arrives intact and in order " + "(partial writev + backpressure)") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Several MiB across pieces of very different sizes, with empty pieces mixed in: the kernel + // will accept only part of the iovec array per writev, so the Rust side must resume from the + // exact byte the previous writev stopped at, across pieces. + auto a = makePatternedData(3 * 1024 * 1024, 1); + auto b = makePatternedData(7, 2); + auto c = makePatternedData(2 * 1024 * 1024, 3); + kj::ArrayPtr empty; + kj::ArrayPtr pieces[] = {empty, a, b, empty, c, empty}; + + auto expected = kj::heapArray(a.size() + b.size() + c.size()); + memcpy(expected.begin(), a.begin(), a.size()); + memcpy(expected.begin() + a.size(), b.begin(), b.size()); + memcpy(expected.begin() + a.size() + b.size(), c.begin(), c.size()); + + auto write = pair.client->write(kj::arrayPtr(pieces, kj::size(pieces))); + // The reader runs concurrently: nothing drains the socket otherwise, so the write must hit + // WouldBlock partway through the array and pick up where it left off. + kj::joinPromisesFailFast(kj::arr(kj::mv(write), readExact(*pair.server, expected))).wait(ws); +} + +KJ_TEST("multi-piece write() of only empty pieces succeeds without touching the socket") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + kj::ArrayPtr pieces[] = {{}, {}}; + pair.client->write(kj::arrayPtr(pieces, 2)).wait(ws); + // Nothing was written: a short read times out rather than returning bytes. + kj::byte buffer[1]; + auto read = pair.server->tryRead(buffer, 1, 1); + KJ_EXPECT(!read.poll(ws)); +} + +KJ_TEST("parseAddress of an unresolvable host fails with a getaddrinfo exception") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // RFC 6761 reserves .invalid: resolvers must answer NXDOMAIN without asking upstream. + auto exception = KJ_ASSERT_NONNULL(kj::runCatchingExceptions([&]() { + boundedBy(io, io.getNetwork().parseAddress("nonexistent.invalid:80"), 30 * kj::SECONDS, + "DNS failure to be reported") + .wait(ws); + }), + "nonexistent.invalid unexpectedly resolved"); + KJ_EXPECT(exception.getDescription().contains("getaddrinfo"), exception.getDescription()); +} + +#if !_WIN32 +KJ_TEST("listen() on a unix socket path that already exists fails (no unlink, like KJ)") { + auto io = setupTokioAsyncIo(); + auto path = freshUnixSocketPath("bind-twice"); + KJ_DEFER(::unlink(path.cStr())); + auto addr = parseNow(io, kj::str("unix:", path)); + auto first = addr->listen(); + KJ_EXPECT_THROW_MESSAGE("bind()", addr->listen()); + // The first listener still works. + KJ_EXPECT(first->getPort() == 0); +} +#endif // !_WIN32 + +KJ_TEST("acceptAuthenticated() through a restricted listener drops disallowed peers too") { + // Mirrors the accept() test above through the other entry point: both share acceptImpl, and + // the filter must run before any peer identity is minted. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + kj_rs_io::TokioNetwork network; + auto restricted = network.restrictPeers({"public"_kj}, {}); + + auto listener = restricted->parseAddress("127.0.0.1", 0).wait(ws)->listen(); + auto acceptPromise = listener->acceptAuthenticated(); + + auto client = network.parseAddress(kj::str("127.0.0.1:", listener->getPort()), 0) + .wait(ws) + ->connect() + .wait(ws); + kj::byte buffer[1]; + // The listener drops the loopback (non-"public") peer: the client sees EOF, and the accept is + // still pending afterwards (the EOF proves the drop happened, so this poll is meaningful). + KJ_EXPECT(client->tryRead(buffer, 1, 1).wait(ws) == 0); + KJ_EXPECT(!acceptPromise.poll(ws)); +} + +} // namespace +} // namespace kj_rs_io_test diff --git a/src/rust/cxx/kj-rs-io/tests/capnp-rpc-test.c++ b/src/rust/cxx/kj-rs-io/tests/capnp-rpc-test.c++ new file mode 100644 index 00000000000..b1ea271c8b3 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/capnp-rpc-test.c++ @@ -0,0 +1,79 @@ +// capnp RPC integration: capnp::TwoPartyServer / capnp::TwoPartyClient running over kj-rs-io +// (tokio-backed) streams on the tokio event loop — bootstrap plus call round-trips. Uses a +// schema-less capability (raw dispatchCall / typelessRequest with Text payloads) to avoid +// needing capnp codegen in this repo. + +#include "kj-rs-io/async-io.h" + +#include +#include +#include +#include +#include +#include + +namespace kj_rs_io_test { +namespace { + +using kj_rs_io::setupTokioAsyncIo; + +constexpr uint64_t ECHO_INTERFACE_ID = 0xabcd1234abcd1234ull; +constexpr uint16_t ECHO_METHOD_ID = 0; + +// A schema-less capability: method 0 takes a Text param and returns "echo:" + text. +class EchoCapability final: public capnp::Capability::Server { + public: + DispatchCallResult dispatchCall(uint64_t interfaceId, + uint16_t methodId, + capnp::CallContext context) override { + KJ_ASSERT(interfaceId == ECHO_INTERFACE_ID); + KJ_ASSERT(methodId == ECHO_METHOD_ID); + auto params = kj::str(context.getParams().getAs()); + context.releaseParams(); + context.getResults(capnp::MessageSize{16, 0}).setAs(kj::str("echo:", params)); + return DispatchCallResult{kj::READY_NOW, false, true}; + } +}; + +KJ_TEST("capnp two-party RPC bootstrap and call round-trips over kj-rs-io " + "streams on the tokio loop") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // Server: TwoPartyServer accepting from a kj-rs-io ConnectionReceiver. + capnp::TwoPartyServer server(kj::heap()); + auto listener = io.getNetwork().parseAddress("127.0.0.1", 0).wait(ws)->listen(); + auto listenTask = server.listen(*listener).eagerlyEvaluate( + [](kj::Exception &&e) { KJ_FAIL_EXPECT("RPC server failed", e); }); + + // Client: TwoPartyClient over a kj-rs-io connection. + auto addr = io.getNetwork().parseAddress(kj::str("127.0.0.1:", listener->getPort())).wait(ws); + auto connection = addr->connect().wait(ws); + capnp::TwoPartyClient client(*connection); + auto cap = client.bootstrap(); + + // Single call round trip. + { + auto request = cap.typelessRequest(ECHO_INTERFACE_ID, ECHO_METHOD_ID, kj::none, {}); + request.setAs("hello tokio"); + auto response = request.send().wait(ws); + KJ_EXPECT(response.getAs() == "echo:hello tokio"); + } + + // A pile of pipelined calls in flight at once. + { + constexpr int COUNT = 64; + auto builder = kj::heapArrayBuilder>(COUNT); + for (int i = 0; i < COUNT; i++) { + auto request = cap.typelessRequest(ECHO_INTERFACE_ID, ECHO_METHOD_ID, kj::none, {}); + request.setAs(kj::str("msg", i)); + builder.add(request.send().then([i](capnp::Response response) { + KJ_EXPECT(response.getAs() == kj::str("echo:msg", i)); + })); + } + kj::joinPromisesFailFast(builder.finish()).wait(ws); + } +} + +} // namespace +} // namespace kj_rs_io_test diff --git a/src/rust/cxx/kj-rs-io/tests/file-watcher-test.c++ b/src/rust/cxx/kj-rs-io/tests/file-watcher-test.c++ new file mode 100644 index 00000000000..ac376dfa130 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/file-watcher-test.c++ @@ -0,0 +1,368 @@ +// Tests for kj_rs_io::FileWatcher (file-watcher.h), the tokio-loop replacement for workerd's +// --watch FileWatcher. Exercises the behaviors workerd depends on: plain modification, atomic +// replace-by-rename (editor saves), event queueing/coalescing across onChange() calls, the +// already-open-fd watch path (kqueue backends), missing-file handling, and teardown/cancel +// while a watch promise is armed. + +#include "kj-rs-io/async-io.h" +#include "kj-rs-io/file-watcher.h" + +#include +#include +#include +#include + +#include +#include + +#if !_WIN32 +#include +#include +#include +#endif + +namespace kj_rs_io_test { +namespace { + +using kj_rs_io::FileWatcher; +using kj_rs_io::setupTokioAsyncIo; +using kj_rs_io::TokioAsyncIoContext; + +#if !_WIN32 + +// ======================================================================================= +// Helpers + +// Waits for `promise` to resolve, returning true, or false after `timeout`. +bool resolvesWithin(kj::Promise promise, TokioAsyncIoContext &io, kj::Duration timeout) { + auto timedOut = io.getTimer().afterDelay(timeout).then([]() { return false; }); + return promise.then([]() { return true; }) + .exclusiveJoin(kj::mv(timedOut)) + .wait(io.getWaitScope()); +} + +// Generous bound for "the change fires"; file events are near-immediate on both backends. +constexpr kj::Duration FIRE_TIMEOUT = 5 * kj::SECONDS; +// Short bound for "nothing fires" checks. +constexpr kj::Duration QUIET_TIMEOUT = 200 * kj::MILLISECONDS; + +struct TempDir { + kj::String path; + + TempDir() { + const char *base = getenv("TEST_TMPDIR"); + if (base == nullptr) base = "/tmp"; + auto tmpl = kj::str(base, "/kj-rs-io-file-watcher-test.XXXXXX"); + KJ_ASSERT(mkdtemp(tmpl.begin()) != nullptr, strerror(errno)); + path = kj::mv(tmpl); + } + + ~TempDir() noexcept(false) { + // Best-effort cleanup; TEST_TMPDIR is wiped by bazel anyway. + auto cmd = kj::str("rm -rf ", path); + (void)system(cmd.cStr()); + } + + kj::String fileName(kj::StringPtr name) { + return kj::str(path, "/", name); + } + + kj::Path filePath(kj::StringPtr name) { + auto full = fileName(name); + KJ_ASSERT(full.startsWith("/")); + return kj::Path::parse(full.slice(1)); + } +}; + +void writeFile(kj::StringPtr path, kj::StringPtr content) { + kj::OwnFd fd = KJ_SYSCALL_FD(open(path.cStr(), O_WRONLY | O_CREAT | O_TRUNC, 0644)); + KJ_SYSCALL(write(fd, content.begin(), content.size())); +} + +void appendFile(kj::StringPtr path, kj::StringPtr content) { + kj::OwnFd fd = KJ_SYSCALL_FD(open(path.cStr(), O_WRONLY | O_APPEND)); + KJ_SYSCALL(write(fd, content.begin(), content.size())); +} + +// After a change fired, drains any further already-queued events so the next onChange() call +// starts from a quiet state (mirrors what workerd's waitForChanges() settle loop achieves). +void drain(FileWatcher &watcher, TokioAsyncIoContext &io) { + while (resolvesWithin(watcher.onChange(), io, QUIET_TIMEOUT)) {} +} + +// ======================================================================================= +// Tests + +KJ_TEST("FileWatcher: supported on this platform") { + auto io = setupTokioAsyncIo(); + FileWatcher watcher; + KJ_EXPECT(watcher.isSupported()); +} + +KJ_TEST("FileWatcher: modification fires onChange") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + auto change = watcher.onChange(); + appendFile(dir.fileName("a.txt"), " two"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: change before onChange() is called is not lost") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + // Modify before anyone is waiting: the event queues in the kernel. + appendFile(dir.fileName("a.txt"), " two"); + KJ_EXPECT(resolvesWithin(watcher.onChange(), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: atomic replace-by-rename fires onChange") { + // Editors typically save by writing a temporary file and rename(2)ing it over the target. + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + auto change = watcher.onChange(); + writeFile(dir.fileName("a.txt.tmp"), "two"); + KJ_SYSCALL(rename(dir.fileName("a.txt.tmp").cStr(), dir.fileName("a.txt").cStr())); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: watching via an already-open file handle") { + // workerd passes the config files' already-open kj::ReadableFile to watch(); the kqueue + // backend watches a dup of that fd (the inotify backend ignores it and uses the path). + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + auto file = kj::newDiskReadableFile(KJ_SYSCALL_FD(open(dir.fileName("a.txt").cStr(), O_RDONLY))); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), *file); + file = nullptr; // The original handle may be closed; the watch must survive. + + auto change = watcher.onChange(); + appendFile(dir.fileName("a.txt"), " two"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: rapid changes coalesce; watcher stays armed for later " + "changes") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + // A burst of changes produces one resolution per onChange() call (not one per event), ... + auto change = watcher.onChange(); + appendFile(dir.fileName("a.txt"), " two"); + appendFile(dir.fileName("a.txt"), " three"); + appendFile(dir.fileName("a.txt"), " four"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); + + // ... and once the queue is drained, the watcher is quiet ... + drain(watcher, io); + + // ... but still armed: a fresh change fires a fresh onChange(). + auto later = watcher.onChange(); + appendFile(dir.fileName("a.txt"), " five"); + KJ_EXPECT(resolvesWithin(kj::mv(later), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: deleting the watched file fires; a recreated file is tracked where the " + "backend can") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + // Delete: both backends report it (inotify IN_DELETE on the directory; kqueue NOTE_DELETE on + // the open fd). + auto change = watcher.onChange(); + KJ_SYSCALL(unlink(dir.fileName("a.txt").cStr())); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); + drain(watcher, io); + + // Recreate and modify. The inotify backend watches the directory by name, so the new file is + // picked up; the kqueue backend watches the deleted inode's fd and cannot see the new file + // (workerd's --watch re-execs on the first change anyway, so this only matters for coverage + // of the two backends' documented difference). + auto later = watcher.onChange(); + writeFile(dir.fileName("a.txt"), "two"); + appendFile(dir.fileName("a.txt"), " three"); +#if __linux__ + KJ_EXPECT(resolvesWithin(kj::mv(later), io, FIRE_TIMEOUT)); +#else + KJ_EXPECT(!resolvesWithin(kj::mv(later), io, QUIET_TIMEOUT)); +#endif +} + +KJ_TEST("FileWatcher: the onChange() promise outlives the FileWatcher object") { + // The promise co-owns the watcher state, so destroying the FileWatcher first must neither + // crash nor invalidate the pending promise: a change made afterwards still resolves it. + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + kj::Promise change = nullptr; + { + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + change = watcher.onChange(); + KJ_EXPECT(!change.poll(io.getWaitScope())); + } + appendFile(dir.fileName("a.txt"), " two"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: unrelated files in the same directory do not fire " + "(inotify filtering)") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + writeFile(dir.fileName("other.txt"), "other"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + appendFile(dir.fileName("other.txt"), " more"); + KJ_EXPECT(!resolvesWithin(watcher.onChange(), io, QUIET_TIMEOUT)); +} + +#if __linux__ +KJ_TEST("FileWatcher: watching a not-yet-existing file fires when it is created") { + // The inotify backend watches the parent directory, so the file itself need not exist yet. + // (The kqueue backend opens the file and so requires it to exist; see the test below.) + auto io = setupTokioAsyncIo(); + TempDir dir; + + FileWatcher watcher; + watcher.watch(dir.filePath("missing.txt"), kj::none); + + auto change = watcher.onChange(); + writeFile(dir.fileName("missing.txt"), "now it exists"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); +} +#else +KJ_TEST("FileWatcher: watching a nonexistent file throws (kqueue backend)") { + // Same behavior as workerd's kj-mode kqueue watcher: watch() opens the path with + // KJ_SYSCALL, which throws if it doesn't exist. + auto io = setupTokioAsyncIo(); + TempDir dir; + + FileWatcher watcher; + auto exception = + kj::runCatchingExceptions([&]() { watcher.watch(dir.filePath("missing.txt"), kj::none); }); + KJ_EXPECT(exception != kj::none); +} +#endif + +KJ_TEST("FileWatcher: canceling an armed onChange() and re-arming works") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + + { + auto armed = watcher.onChange(); + KJ_EXPECT(!armed.poll(io.getWaitScope())); + // Dropped here while armed (fd registered with the tokio I/O driver). + } + + auto change = watcher.onChange(); + appendFile(dir.fileName("a.txt"), " two"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); +} + +KJ_TEST("FileWatcher: multiple files in one watcher each fire on change") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "a"); + writeFile(dir.fileName("b.txt"), "b"); + + FileWatcher watcher; + watcher.watch(dir.filePath("a.txt"), kj::none); + watcher.watch(dir.filePath("b.txt"), kj::none); + + { + auto change = watcher.onChange(); + appendFile(dir.fileName("a.txt"), "1"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); + } + // Drain any residual events from the first change before testing the second file. + while (resolvesWithin(watcher.onChange(), io, QUIET_TIMEOUT)) {} + { + auto change = watcher.onChange(); + appendFile(dir.fileName("b.txt"), "2"); + KJ_EXPECT(resolvesWithin(kj::mv(change), io, FIRE_TIMEOUT)); + } +} + +KJ_TEST("FileWatcher: two independent watchers do not cross-fire") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "a"); + writeFile(dir.fileName("b.txt"), "b"); + + FileWatcher w1; + w1.watch(dir.filePath("a.txt"), kj::none); + FileWatcher w2; + w2.watch(dir.filePath("b.txt"), kj::none); + + // Changing a.txt fires w1 but must NOT fire w2 (inotify filters by basename; kqueue watches + // only b.txt's own fd). + auto c2 = w2.onChange(); + { + auto c1 = w1.onChange(); + appendFile(dir.fileName("a.txt"), "1"); + KJ_EXPECT(resolvesWithin(kj::mv(c1), io, FIRE_TIMEOUT)); + } + KJ_EXPECT(!resolvesWithin(kj::mv(c2), io, QUIET_TIMEOUT)); +} + +KJ_TEST("FileWatcher: teardown while a watch promise is armed") { + auto io = setupTokioAsyncIo(); + TempDir dir; + writeFile(dir.fileName("a.txt"), "one"); + + auto watcher = kj::heap(); + watcher->watch(dir.filePath("a.txt"), kj::none); + + auto armed = watcher->onChange(); + KJ_EXPECT(!armed.poll(io.getWaitScope())); + + // Promise first (it borrows the watcher's fd), then the watcher itself. + armed = nullptr; + watcher = nullptr; +} + +#else // _WIN32 + +KJ_TEST("FileWatcher: reports unsupported on this platform") { + auto io = setupTokioAsyncIo(); + FileWatcher watcher; + KJ_EXPECT(!watcher.isSupported()); +} + +#endif + +} // namespace +} // namespace kj_rs_io_test diff --git a/src/rust/cxx/kj-rs-io/tests/http-test.c++ b/src/rust/cxx/kj-rs-io/tests/http-test.c++ new file mode 100644 index 00000000000..55dfb9506f9 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/http-test.c++ @@ -0,0 +1,147 @@ +// kj-http integration ("the acid test"): a real kj::HttpServer serving on a kj-rs-io listener +// and a kj::HttpClient over a kj-rs-io connection, all driven by the tokio event loop. Proves +// that kj-http works unchanged over tokio-backed streams. + +#include "kj-rs-io/async-io.h" + +#include +#include +#include +#include + +#include + +namespace kj_rs_io_test { +namespace { + +using kj_rs_io::setupTokioAsyncIo; + +// Echoes the request body back as the response body, streaming (pumpTo), preserving the +// content length when known. +class EchoService final: public kj::HttpService { + public: + explicit EchoService(kj::HttpHeaderTable &table): table(table) {} + + kj::Promise request(kj::HttpMethod method, + kj::StringPtr url, + const kj::HttpHeaders &headers, + kj::AsyncInputStream &requestBody, + Response &response) override { + kj::HttpHeaders responseHeaders(table); + auto body = response.send(200, "OK", responseHeaders, requestBody.tryGetLength()); + co_await requestBody.pumpTo(*body); + } + + private: + kj::HttpHeaderTable &table; +}; + +kj::Array makeBody(size_t size) { + auto data = kj::heapArray(size); + for (size_t i = 0; i < size; i++) { + data[i] = static_cast(('A' + i / 8192 + i * 13) & 0xff); + } + return data; +} + +// Writes `data` in chunks to the request body stream, then closes it. +kj::Promise writeBody( + kj::Own body, kj::ArrayPtr data) { + constexpr size_t CHUNK = 128 * 1024; + size_t offset = 0; + while (offset < data.size()) { + size_t n = kj::min(CHUNK, data.size() - offset); + co_await body->write(data.slice(offset, offset + n)); + offset += n; + } + // Dropping `body` (coroutine frame teardown) finishes the request body. +} + +KJ_TEST("bodyless GET works (regression: kj-http never awaits its header-write " + "queue for bodyless requests, relying on KJ hot-promise write semantics)") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + kj::HttpHeaderTable table; + EchoService service(table); + kj::HttpServer server(io.getTimer(), table, service); + auto listener = io.getNetwork().parseAddress("127.0.0.1", 0).wait(ws)->listen(); + auto listenTask = server.listenHttp(*listener).eagerlyEvaluate(nullptr); + auto addr = io.getNetwork().parseAddress(kj::str("127.0.0.1:", listener->getPort())).wait(ws); + auto connection = addr->connect().wait(ws); + auto client = kj::newHttpClient(table, *connection); + kj::HttpHeaders headers(table); + auto request = client->request(kj::HttpMethod::GET, "/x", headers, static_cast(0)); + auto response = request.response.wait(ws); + KJ_EXPECT(response.statusCode == 200); + auto body = response.body->readAllBytes().wait(ws); + KJ_EXPECT(body.size() == 0); +} + +KJ_TEST("kj-http round trip with streaming bodies over kj-rs-io streams on the " + "tokio loop") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + kj::HttpHeaderTable table; + EchoService service(table); + kj::HttpServer server(io.getTimer(), table, service); + + // Server side: kj::HttpServer accepting from a kj-rs-io ConnectionReceiver. + auto listener = io.getNetwork().parseAddress("127.0.0.1", 0).wait(ws)->listen(); + auto listenTask = server.listenHttp(*listener).eagerlyEvaluate( + [](kj::Exception &&e) { KJ_FAIL_EXPECT("HTTP server failed", e); }); + + // Client side: kj::HttpClient over a kj-rs-io connection. + auto addr = io.getNetwork().parseAddress(kj::str("127.0.0.1:", listener->getPort())).wait(ws); + auto connection = addr->connect().wait(ws); + auto client = kj::newHttpClient(table, *connection); + + // Round trip 1: 4 MB POST with a streamed request body, echoed back and read while the + // request body is still being written (full-duplex streaming through the tokio loop). + { + constexpr size_t SIZE = 4 * 1024 * 1024; + auto data = makeBody(SIZE); + + kj::HttpHeaders headers(table); + auto request = + client->request(kj::HttpMethod::POST, "/echo", headers, static_cast(SIZE)); + + auto writeTask = writeBody(kj::mv(request.body), data).eagerlyEvaluate(nullptr); + auto response = request.response.wait(ws); + KJ_EXPECT(response.statusCode == 200); + KJ_EXPECT(KJ_ASSERT_NONNULL(response.body->tryGetLength()) == SIZE); + + auto echoed = response.body->readAllBytes(SIZE + 1).wait(ws); + KJ_ASSERT(echoed.size() == SIZE); + KJ_ASSERT(memcmp(echoed.begin(), data.begin(), SIZE) == 0); + writeTask.wait(ws); + } + + // Round trip 2 on the same connection (keep-alive): small GET, empty echoed body. + { + kj::HttpHeaders headers(table); + auto request = + client->request(kj::HttpMethod::GET, "/again", headers, static_cast(0)); + auto response = request.response.wait(ws); + KJ_EXPECT(response.statusCode == 200); + auto body = response.body->readAllBytes().wait(ws); + KJ_EXPECT(body.size() == 0); + } + + // Round trip 3: chunked request body (no expected size -> Transfer-Encoding: chunked). + { + kj::HttpHeaders headers(table); + auto request = client->request(kj::HttpMethod::POST, "/chunked", headers); + auto data = makeBody(64 * 1024); + auto writeTask = writeBody(kj::mv(request.body), data).eagerlyEvaluate(nullptr); + auto response = request.response.wait(ws); + KJ_EXPECT(response.statusCode == 200); + auto echoed = response.body->readAllBytes().wait(ws); + KJ_ASSERT(echoed.size() == data.size()); + KJ_ASSERT(memcmp(echoed.begin(), data.begin(), data.size()) == 0); + writeTask.wait(ws); + } +} + +} // namespace +} // namespace kj_rs_io_test diff --git a/src/rust/cxx/kj-rs-io/tests/io-test-helpers.h b/src/rust/cxx/kj-rs-io/tests/io-test-helpers.h new file mode 100644 index 00000000000..a0888ad59c1 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/io-test-helpers.h @@ -0,0 +1,108 @@ +#pragma once +// Shared helpers for the kj-rs-io C++ tests (async-io-test, serve-test): connected stream pairs +// over the tokio-backed network, patterned payloads, and chunked write / verify pumps. + +#include "kj-rs-io/async-io.h" + +#include +#include +#include +#include + +#include + +#if !_WIN32 +#include // getpid()/unlink() for the unix-socket pair helper +#endif + +namespace kj_rs_io_test { + +struct ConnectedPair { + kj::Own listener; + kj::Own client; + kj::Own server; +}; + +inline kj::Own parseNow( + kj_rs_io::TokioAsyncIoContext &io, kj::StringPtr addr, kj::uint portHint = 0) { + return io.getNetwork().parseAddress(addr, portHint).wait(io.getWaitScope()); +} + +// A connected loopback TCP pair from the kj-rs-io network. +inline ConnectedPair makeTcpPair(kj_rs_io::TokioAsyncIoContext &io) { + auto &ws = io.getWaitScope(); + auto listener = parseNow(io, "127.0.0.1")->listen(); + auto connectAddr = parseNow(io, kj::str("127.0.0.1:", listener->getPort())); + auto acceptPromise = listener->accept(); + auto client = connectAddr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + return ConnectedPair{kj::mv(listener), kj::mv(client), kj::mv(server)}; +} + +#if !_WIN32 +// A short /tmp unix-socket path unique per process + call, so parallel/repeated runs never +// collide. Stale files are unlinked first. +inline kj::String freshUnixSocketPath(kj::StringPtr tag) { + static uint counter = 0; + auto path = kj::str("/tmp/kj-rs-io-", tag, "-", ::getpid(), "-", counter++, ".sock"); + ::unlink(path.cStr()); + return path; +} + +// A connected AF_UNIX stream-socket pair from the kj-rs-io network's `unix:` support. The bound +// path is unlinked once both ends are connected. +inline ConnectedPair makeUnixPair(kj_rs_io::TokioAsyncIoContext &io) { + auto &ws = io.getWaitScope(); + auto path = freshUnixSocketPath("pair"); + auto addr = kj::str("unix:", path); + auto listener = parseNow(io, addr)->listen(); + auto connectAddr = parseNow(io, addr); + auto acceptPromise = listener->accept(); + auto client = connectAddr->connect().wait(ws); + auto server = acceptPromise.wait(ws); + ::unlink(path.cStr()); + return ConnectedPair{kj::mv(listener), kj::mv(client), kj::mv(server)}; +} +#endif // !_WIN32 + +inline kj::Array makePatternedData(size_t size, kj::byte seed) { + auto data = kj::heapArray(size); + for (size_t i = 0; i < size; i++) { + data[i] = static_cast((i * 31 + seed) & 0xff); + } + return data; +} + +// Writes `data` to `out` in 64 KiB chunks (exercising write-all + backpressure). +inline kj::Promise writeChunked( + kj::AsyncOutputStream &out, kj::ArrayPtr data) { + constexpr size_t CHUNK = 64 * 1024; + size_t offset = 0; + while (offset < data.size()) { + size_t n = kj::min(CHUNK, data.size() - offset); + co_await out.write(data.slice(offset, offset + n)); + offset += n; + } +} + +// Reads exactly `expected.size()` bytes from `in` and verifies they match `expected`. +inline kj::Promise readExact( + kj::AsyncInputStream &in, kj::ArrayPtr expected) { + auto buffer = kj::heapArray(expected.size()); + size_t total = co_await in.tryRead(buffer.begin(), buffer.size(), buffer.size()); + KJ_ASSERT(total == expected.size(), total, expected.size()); + KJ_ASSERT(memcmp(buffer.begin(), expected.begin(), expected.size()) == 0); +} + +// `promise`, failing loudly (rather than hanging until bazel's timeout) if it is still pending +// after `timeout` on the context's KJ timer. +template +kj::Promise boundedBy(kj_rs_io::TokioAsyncIoContext &io, + kj::Promise promise, + kj::Duration timeout, + kj::StringPtr what) { + return promise.exclusiveJoin(io.getTimer().afterDelay(timeout).then( + [what]() -> T { KJ_FAIL_ASSERT("timed out waiting for", what); })); +} + +} // namespace kj_rs_io_test diff --git a/src/rust/cxx/kj-rs-io/tests/lib.rs b/src/rust/cxx/kj-rs-io/tests/lib.rs new file mode 100644 index 00000000000..1bece888fbc --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/lib.rs @@ -0,0 +1,105 @@ +#![allow(clippy::unused_async)] +#![allow(clippy::missing_errors_doc)] +#![allow(clippy::must_use_candidate)] +#![allow(clippy::missing_panics_doc)] +#![allow(clippy::missing_safety_doc)] +#![allow(clippy::unnecessary_box_returns)] // cxx bridge functions return Box by contract + +mod serve_helpers; +mod test_helpers; + +use serve_helpers::ServeEchoSession; +use serve_helpers::start_serve_drop_consumer; +use serve_helpers::start_serve_echo; +use serve_helpers::start_serve_echo_foreign_thread; +use serve_helpers::start_take_socket_echo; +use test_helpers::address_from_loopback_ports; +use test_helpers::create_prebound_listener_fd; +use test_helpers::native_write_via_kj_unwrap; +use test_helpers::native_write_via_unwrap; + +#[cxx::bridge(namespace = "kj_rs_io_test")] +mod ffi { + /// A pre-bound, *blocking* std TCP listener handed to C++ as a raw fd (the `--socket-fd` + /// scenario for `wrapListenSocketFd`). + struct PreboundListener { + fd: i32, + port: u16, + } + + extern "Rust" { + /// Recovers the native tokio TcpStream from an unwrapped kj-rs-io stream Box and writes + /// `data` natively (tokio readiness API, no FFI-per-byte), then closes the connection. + async fn native_write_via_unwrap(stream: Box, data: Vec) -> Result<()>; + + /// Same, but starts from a `kj::AsyncIoStream&`: unwraps it from the Rust side via + /// `kj_rs_io::unwrap_kj_stream` (the API a native Rust server's glue will use), then writes + /// natively. Leaves the C++ wrapper hollow. + async unsafe fn native_write_via_kj_unwrap<'a>( + stream: Pin<&'a mut KjAsyncIoStream>, + data: &'a [u8], + ) -> Result<()>; + + /// Binds 127.0.0.1:0 with std (blocking mode, like an inherited `--socket-fd` listener) + /// and releases it as a raw fd owned by the caller. + fn create_prebound_listener_fd() -> Result; + + /// A kj-rs-io address resolving to `127.0.0.1:` for each of `ports`, in order: + /// a deterministic stand-in for a multi-result DNS lookup, for testing connect()'s + /// try-each-address fallback. + fn address_from_loopback_ports(ports: &[u16]) -> Box; + + // --- serve_kj_stream (serve_helpers.rs) + + /// One echo server over a served kj stream: `start_serve_echo` picks the transport + /// path (unwrap fast path or duplex pump) via `kj_rs_io::serve_kj_stream` — taking + /// ownership of the stream — and spawns the echo consumer on the loop runtime; + /// `drive()` runs the connection (the pump, on the pumped path) to completion — + /// dropping the `drive()` promise mid-connection is the abort-on-drop path, which + /// also destroys the owned stream. + type ServeEchoSession; + + fn start_serve_echo(stream: KjOwn) -> Box; + + /// Like `start_serve_echo`, but the consumer reads one message and then DROPS its + /// `ServeIo` without calling `shutdown()`: the pump must turn the dropped duplex end into + /// `shutdownWrite()` on the kj stream. + fn start_serve_drop_consumer(stream: KjOwn) -> Box; + + /// Like `start_serve_echo`, but the echo consumer runs on a separate OS thread with its + /// own tokio runtime, driving a pumped stream's `ServeIo::Duplex` end off the KJ + /// event-loop thread (cross-thread waker path; TSAN target). + fn start_serve_echo_foreign_thread(stream: KjOwn) + -> Box; + + /// Like `start_serve_echo`, but through the native-only `take_kj_socket` entry point + /// (unwrap tier, else fd-dup tier): errors — instead of pumping — for streams that + /// are neither. The consumed stream is destroyed before this returns. + fn start_take_socket_echo(stream: KjOwn) -> Result>; + + /// Whether the unwrap fast path was taken (perf observability surface). + fn is_native(self: &ServeEchoSession) -> bool; + + /// Runs the connection to completion (see the type's docs). May only be called once. + async unsafe fn drive<'a>(self: &'a ServeEchoSession) -> Result<()>; + + /// Resolves once the echo task has exited — observing that dropping `drive()` EOFs + /// the pumped consumer. + async unsafe fn wait_echo_done<'a>(self: &'a ServeEchoSession); + } + + extern "Rust" { + #[namespace = "kj_rs_io"] + type TokioStream = kj_rs_io::TokioStream; + #[namespace = "kj_rs_io"] + type TokioAddress = kj_rs_io::TokioAddress; + } + + unsafe extern "C++" { + include!("kj-rs-io/unwrap.h"); + + #[namespace = "kj"] + #[cxx_name = "AsyncIoStream"] + type KjAsyncIoStream = kj_rs_io::KjAsyncIoStream; + } +} diff --git a/src/rust/cxx/kj-rs-io/tests/peer-filter-test.c++ b/src/rust/cxx/kj-rs-io/tests/peer-filter-test.c++ new file mode 100644 index 00000000000..2e19a08827a --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/peer-filter-test.c++ @@ -0,0 +1,193 @@ +// Direct unit tests for PeerFilter::shouldAllow — the faithful port of kj::_::NetworkFilter that +// backs restrictPeers(). The network-level tests (async-io-test.c++) only exercise +// {"public"}/{"private"} end to end; this file drives the grammar directly against hand-built +// sockaddrs (no event loop, no sockets), covering every rule class, the allow/deny specificity +// tie-break, IPv6, unix/unix-abstract, and filter chaining. + +#include "kj-rs-io/peer-filter.h" + +#include +#include + +#include + +#if _WIN32 +#include +#include + +#include +#else +#include +#include +#include +#endif + +namespace kj_rs_io { +namespace { + +// A filter with `allow`/`deny` rules layered on an allow-everything parent, so the parent never +// changes the verdict and the rules under test are what decides. +kj::Rc filter( + kj::ArrayPtr allow, kj::ArrayPtr deny = nullptr) { + return kj::rc(allow, deny, kj::rc()); +} + +// Does `f` allow the given numeric IP (v4 if it has no ':', else v6), port 1? +bool allowsIp(PeerFilter& f, kj::StringPtr ip) { + if (ip.findFirst(':') != kj::none) { + struct sockaddr_in6 sin6; + memset(&sin6, 0, sizeof(sin6)); + sin6.sin6_family = AF_INET6; + sin6.sin6_port = htons(1); + KJ_ASSERT(inet_pton(AF_INET6, ip.cStr(), &sin6.sin6_addr) == 1, ip); + return f.shouldAllow(reinterpret_cast(&sin6), sizeof(sin6)); + } else { + struct sockaddr_in sin; + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_port = htons(1); + KJ_ASSERT(inet_pton(AF_INET, ip.cStr(), &sin.sin_addr) == 1, ip); + return f.shouldAllow(reinterpret_cast(&sin), sizeof(sin)); + } +} + +KJ_TEST("PeerFilter: default filter allows everything") { + auto f = kj::rc(); + KJ_EXPECT(allowsIp(*f, "8.8.8.8")); + KJ_EXPECT(allowsIp(*f, "127.0.0.1")); + KJ_EXPECT(allowsIp(*f, "10.0.0.1")); + KJ_EXPECT(allowsIp(*f, "::1")); +} + +KJ_TEST("PeerFilter: 'public' allows public IPs, blocks private/local/reserved") { + auto f = filter({"public"_kj}); + KJ_EXPECT(allowsIp(*f, "8.8.8.8")); // public + KJ_EXPECT(allowsIp(*f, "1.1.1.1")); // public + KJ_EXPECT(!allowsIp(*f, "10.0.0.1")); // RFC1918 private + KJ_EXPECT(!allowsIp(*f, "192.168.1.1")); // RFC1918 private + KJ_EXPECT(!allowsIp(*f, "172.16.5.5")); // RFC1918 private + KJ_EXPECT(!allowsIp(*f, "127.0.0.1")); // local + KJ_EXPECT(!allowsIp(*f, "224.0.0.1")); // reserved (multicast) + KJ_EXPECT(!allowsIp(*f, "169.254.1.1")); // link-local (private) +} + +KJ_TEST("PeerFilter: 'network' allows public+private, blocks local/reserved") { + auto f = filter({"network"_kj}); + KJ_EXPECT(allowsIp(*f, "8.8.8.8")); // public + KJ_EXPECT(allowsIp(*f, "10.0.0.1")); // private — the difference from 'public' + KJ_EXPECT(allowsIp(*f, "192.168.1.1")); // private + KJ_EXPECT(!allowsIp(*f, "127.0.0.1")); // local + KJ_EXPECT(!allowsIp(*f, "224.0.0.1")); // reserved +} + +KJ_TEST("PeerFilter: 'local' allows loopback only") { + auto f = filter({"local"_kj}); + KJ_EXPECT(allowsIp(*f, "127.0.0.1")); + KJ_EXPECT(allowsIp(*f, "127.5.5.5")); // 127/8 + KJ_EXPECT(!allowsIp(*f, "8.8.8.8")); + KJ_EXPECT(!allowsIp(*f, "10.0.0.1")); +} + +KJ_TEST("PeerFilter: 'private' allows RFC1918 + local, blocks public") { + auto f = filter({"private"_kj}); + KJ_EXPECT(allowsIp(*f, "10.1.2.3")); + KJ_EXPECT(allowsIp(*f, "192.168.0.1")); + KJ_EXPECT(allowsIp(*f, "127.0.0.1")); // 'private' includes local + KJ_EXPECT(!allowsIp(*f, "8.8.8.8")); +} + +KJ_TEST("PeerFilter: explicit CIDR allow") { + auto f = filter({"10.0.0.0/8"_kj}); + KJ_EXPECT(allowsIp(*f, "10.1.2.3")); + KJ_EXPECT(allowsIp(*f, "10.255.255.255")); + KJ_EXPECT(!allowsIp(*f, "11.0.0.1")); + KJ_EXPECT(!allowsIp(*f, "8.8.8.8")); +} + +KJ_TEST("PeerFilter: allow + more-specific deny (specificity tie-break)") { + // allow 10/8 but deny the more-specific 10.1/16: 10.1.x blocked, other 10.x allowed. + auto f = filter({"10.0.0.0/8"_kj}, {"10.1.0.0/16"_kj}); + KJ_EXPECT(allowsIp(*f, "10.2.3.4")); // allowed by /8, no deny matches + KJ_EXPECT(!allowsIp(*f, "10.1.2.3")); // deny /16 is more specific than allow /8 + KJ_EXPECT(allowsIp(*f, "10.255.0.1")); +} + +KJ_TEST("PeerFilter: equal-specificity deny wins over allow (>= tie-break)") { + // Same prefix length on both sides: deny's `>=` means the deny wins. + auto f = filter({"10.1.0.0/16"_kj}, {"10.1.0.0/16"_kj}); + KJ_EXPECT(!allowsIp(*f, "10.1.2.3")); +} + +KJ_TEST("PeerFilter: IPv6 public/private/explicit rules") { + auto pub = filter({"public"_kj}); + KJ_EXPECT(allowsIp(*pub, "2606:4700:4700::1111")); // public v6 + KJ_EXPECT(!allowsIp(*pub, "::1")); // v6 loopback (local) + KJ_EXPECT(!allowsIp(*pub, "fc00::1")); // v6 unique-local (private) + + auto priv = filter({"private"_kj}); + KJ_EXPECT(allowsIp(*priv, "fc00::1")); // fc00::/7 private + KJ_EXPECT(allowsIp(*priv, "::1")); // local included in private + KJ_EXPECT(!allowsIp(*priv, "2606:4700:4700::1111")); + + auto cidr = filter({"fc00::/7"_kj}); + KJ_EXPECT(allowsIp(*cidr, "fc00::1")); + KJ_EXPECT(!allowsIp(*cidr, "2606:4700:4700::1111")); +} + +KJ_TEST("PeerFilter: nested filter chain enforces BOTH levels") { + // Child allows all private; parent (next) allows only local. An address the child allows but + // the parent denies must be blocked — the chain is an AND. + auto parent = kj::rc(kj::arr("local"_kj), nullptr, kj::rc()); + auto child = kj::rc(kj::arr("private"_kj), nullptr, kj::mv(parent)); + KJ_EXPECT(allowsIp(*child, "127.0.0.1")); // allowed by both child (private⊇local) and parent + KJ_EXPECT(!allowsIp(*child, "10.0.0.1")); // allowed by child, DENIED by parent → blocked +} + +KJ_TEST("PeerFilter: denying 'network' or 'public' is rejected") { + KJ_EXPECT_THROW_MESSAGE("don't deny 'network'", + kj::rc(nullptr, kj::arr("network"_kj), kj::rc())); + KJ_EXPECT_THROW_MESSAGE("don't deny 'public'", + kj::rc(nullptr, kj::arr("public"_kj), kj::rc())); +} + +#if !_WIN32 +// Builds a sockaddr_un for `path` (abstract if `abstractLeadingNul`) and returns the verdict. +bool allowsUnix(PeerFilter& f, kj::StringPtr path, bool abstractLeadingNul = false) { + struct sockaddr_un su; + memset(&su, 0, sizeof(su)); + su.sun_family = AF_UNIX; + size_t off = 0; + if (abstractLeadingNul) { + su.sun_path[0] = '\0'; + off = 1; + } + memcpy(su.sun_path + off, path.begin(), path.size()); + kj::uint addrlen = + static_cast(offsetof(struct sockaddr_un, sun_path) + off + path.size()); + return f.shouldAllow(reinterpret_cast(&su), addrlen); +} + +KJ_TEST("PeerFilter: unix and unix-abstract allow/deny") { + // Default allows both. + auto def = kj::rc(); + KJ_EXPECT(allowsUnix(*def, "/tmp/sock")); + KJ_EXPECT(allowsUnix(*def, "abstract-name", true)); + + // "unix" allows pathname sockets, not abstract; "unix-abstract" the reverse. + auto pathOnly = filter({"unix"_kj}); + KJ_EXPECT(allowsUnix(*pathOnly, "/tmp/sock")); + KJ_EXPECT(!allowsUnix(*pathOnly, "abstract-name", true)); + + auto abstractOnly = filter({"unix-abstract"_kj}); + KJ_EXPECT(!allowsUnix(*abstractOnly, "/tmp/sock")); + KJ_EXPECT(allowsUnix(*abstractOnly, "abstract-name", true)); + + // Deny turns them off even from the allow-everything default. + auto denyUnix = kj::rc( + kj::arr("private"_kj, "unix"_kj), kj::arr("unix"_kj), kj::rc()); + KJ_EXPECT(!allowsUnix(*denyUnix, "/tmp/sock")); +} +#endif // !_WIN32 + +} // namespace +} // namespace kj_rs_io diff --git a/src/rust/cxx/kj-rs-io/tests/serve-test.c++ b/src/rust/cxx/kj-rs-io/tests/serve-test.c++ new file mode 100644 index 00000000000..855c8d1cca3 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/serve-test.c++ @@ -0,0 +1,422 @@ +// Tests for kj_rs_io::serve_kj_stream (serve.rs): the native-serve entry +// point Rust servers use to drive a kj::AsyncIoStream's connection natively. Following kj-rs +// conventions, C++ KJ_TESTs drive; Rust helpers (tests/serve_helpers.rs) run the echo server +// side over whichever transport path the entry point picks. + +#include "io-test-helpers.h" +#include "kj-rs-io-test/lib.rs.h" +#include "kj-rs-io/async-io.h" + +#include +#include +#include +#include +#include + +#include + +namespace kj_rs_io_test { +namespace { + +using kj_rs_io::setupTokioAsyncIo; + +// The client side of an echo round trip: write `data` (in chunks) and concurrently read the +// echo back and verify it (concurrent, so bounded transports -- the pump duplex, socket +// buffers -- never deadlock on payloads larger than their buffering); then half-close and +// expect EOF. +kj::Promise echoRoundTrip( + kj::AsyncIoStream &clientStream, kj::ArrayPtr data) { + static auto constexpr readBack = [](kj::AsyncIoStream &s, + kj::ArrayPtr expected) -> kj::Promise { + auto buffer = kj::heapArray(expected.size()); + size_t total = co_await s.tryRead(buffer.begin(), buffer.size(), buffer.size()); + KJ_ASSERT(total == expected.size(), total, expected.size()); + KJ_ASSERT(memcmp(buffer.begin(), expected.begin(), expected.size()) == 0); + }; + + static auto constexpr writeAll = [](kj::AsyncIoStream &s, + kj::ArrayPtr data) -> kj::Promise { + constexpr size_t CHUNK = 64 * 1024; + size_t offset = 0; + while (offset < data.size()) { + size_t n = kj::min(CHUNK, data.size() - offset); + co_await s.write(data.slice(offset, offset + n)); + offset += n; + } + s.shutdownWrite(); // half-close: the echo server sees EOF and finishes flushing + }; + + co_await kj::joinPromisesFailFast( + kj::arr(writeAll(clientStream, data), readBack(clientStream, data))); + kj::byte extra; + KJ_EXPECT(co_await clientStream.tryRead(&extra, 1, 1) == 0); // EOF after echo completes +} + +// ======================================================================================= +// Unwrap fast path + +KJ_TEST("serve_kj_stream takes the native path for kj-rs-io TCP streams and " + "echoes") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Native path: the connection now belongs to the Rust side (the hollow wrapper was + // destroyed by serve_kj_stream). + auto session = start_serve_echo(kj::mv(pair.server)); + KJ_EXPECT(session->is_native()); + + auto data = makePatternedData(256 * 1024, 7); + auto drive = session->drive(); + auto client = echoRoundTrip(*pair.client, data); + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(client))).wait(ws); +} + +// ======================================================================================= +// Duplex pump fallback (foreign streams) + +KJ_TEST("serve_kj_stream pumps foreign streams: bidirectional echo + half-close") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + + // An in-memory kj pipe is the canonical foreign stream: not kj-rs-io-originated. + auto pipe = kj::newTwoWayPipe(); + + auto session = start_serve_echo(kj::mv(pipe.ends[0])); + KJ_EXPECT(!session->is_native()); + + auto data = makePatternedData(512 * 1024, 3); + auto drive = session->drive(); + auto client = echoRoundTrip(*pipe.ends[1], data); + // The pump owns `pipe.ends[0]` now; only the client end stays with the test. + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(client))).wait(ws); +} + +KJ_TEST("serve_kj_stream pump: dropping the pump aborts the bridge (drop-abort)") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pipe = kj::newTwoWayPipe(); + + auto session = start_serve_echo(kj::mv(pipe.ends[0])); + KJ_EXPECT(!session->is_native()); + + { + // Prove the bridge is live: one small round trip, driving the pump only while the + // client operation runs, then *drop* the drive promise mid-connection. + auto drive = session->drive(); + auto oneRoundTrip = [](kj::AsyncIoStream &s) -> kj::Promise { + co_await s.write("ping"_kjb); + kj::byte buffer[4]; + size_t n = co_await s.tryRead(buffer, 4, 4); + KJ_ASSERT(n == 4); + KJ_ASSERT(memcmp(buffer, "ping", 4) == 0); + }(*pipe.ends[1]); + // exclusiveJoin: when the round trip finishes, `drive` is cancelled (dropped). + oneRoundTrip.exclusiveJoin(kj::mv(drive)).wait(ws); + } + + // Dropping the pump dropped the kj-side duplex end: the echo consumer reads EOF and + // finishes... + session->wait_echo_done().wait(ws); + + // ...and the pump destroyed the kj stream it owned: the peer observes teardown (a rejected + // write), not a zombie half-open pipe. + auto orphanWrite = kj::evalNow([&]() { return pipe.ends[1]->write("anyone there?"_kjb); }); + KJ_EXPECT(orphanWrite.poll(ws)); + orphanWrite + .then([]() { KJ_FAIL_EXPECT("write to a torn-down pipe unexpectedly succeeded"); }, + [](kj::Exception &&) { + }).wait(ws); +} + +KJ_TEST("serve_kj_stream pump: the Duplex consumer may be driven from another thread") { + // ServedKjStream::io's docs promise the Duplex variant is safe to drive off the KJ event-loop + // thread since the waker bridge became thread-safe. Here the echo consumer runs on its own OS + // thread and runtime, so every read/write/drop on the Duplex wakes the pump (parked on this + // KJ loop) cross-thread through the FutureWakerCell. TSAN target. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pipe = kj::newTwoWayPipe(); + + auto session = start_serve_echo_foreign_thread(kj::mv(pipe.ends[0])); + KJ_EXPECT(!session->is_native()); + + auto data = makePatternedData(512 * 1024, 5); + auto drive = session->drive(); // drives the pump here; the consumer echoes on its own thread + auto client = echoRoundTrip(*pipe.ends[1], data); + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(client))).wait(ws); +} + +// A foreign stream over one end of a kj two-way pipe whose read side reports a DISCONNECTED +// failure once the pipe's data is gone (instead of a clean EOF), and whose write side may be +// made to fail DISCONNECTED as well: the shape of an abruptly-reset TCP peer, as seen through a +// non-kj-rs-io kj stream. +class DisconnectingStream final: public kj::AsyncIoStream { + public: + DisconnectingStream(kj::Own inner, bool writesDisconnected) + : inner(kj::mv(inner)), + writesDisconnected(writesDisconnected) {} + + kj::Promise tryRead(void *buffer, size_t minBytes, size_t maxBytes) override { + size_t n = co_await inner->tryRead(buffer, minBytes, maxBytes); + if (n < minBytes) { + // Where a well-behaved peer would half-close, this one was reset. + kj::throwFatalException(KJ_EXCEPTION(DISCONNECTED, "peer reset the connection")); + } + co_return n; + } + kj::Promise write(kj::ArrayPtr buffer) override { + if (writesDisconnected) return KJ_EXCEPTION(DISCONNECTED, "peer reset the connection"); + return inner->write(buffer); + } + kj::Promise write(kj::ArrayPtr> pieces) override { + if (writesDisconnected) return KJ_EXCEPTION(DISCONNECTED, "peer reset the connection"); + return inner->write(pieces); + } + kj::Promise whenWriteDisconnected() override { + return inner->whenWriteDisconnected(); + } + void shutdownWrite() override { + if (writesDisconnected) { + kj::throwFatalException(KJ_EXCEPTION(DISCONNECTED, "peer reset the connection")); + } + inner->shutdownWrite(); + } + + private: + kj::Own inner; + bool writesDisconnected; +}; + +KJ_TEST("serve_kj_stream pump: a DISCONNECTED read is treated as EOF, not as an error") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pipe = kj::newTwoWayPipe(); + + auto session = start_serve_echo(kj::heap(kj::mv(pipe.ends[0]), false)); + KJ_EXPECT(!session->is_native()); + + auto drive = session->drive(); + // One message goes through, then the client goes away abruptly: the pump's read fails + // DISCONNECTED. The echo consumer must see EOF (and echo back what it got), and the pump must + // settle Ok -- abrupt client disconnects are normal load, not failures. + pipe.ends[1]->write("ping"_kjb).wait(ws); + kj::byte buffer[4]; + KJ_EXPECT(pipe.ends[1]->tryRead(buffer, 4, 4).wait(ws) == 4); + pipe.ends[1]->shutdownWrite(); // the wrapper turns this EOF into DISCONNECTED + KJ_EXPECT(pipe.ends[1]->tryRead(buffer, 1, 1).wait(ws) == 0); // consumer shut down -> EOF back + boundedBy(io, kj::mv(drive), 10 * kj::SECONDS, "the pump to settle").wait(ws); + session->wait_echo_done().wait(ws); +} + +KJ_TEST("serve_kj_stream pump: a DISCONNECTED write ends the direction without an error") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pipe = kj::newTwoWayPipe(); + + auto session = start_serve_echo(kj::heap(kj::mv(pipe.ends[0]), true)); + auto drive = session->drive(); + // The consumer's echo of "ping" is written to a peer that already reset: the pump must not + // fail. Half-close so the read direction finishes normally too. + pipe.ends[1]->write("ping"_kjb).wait(ws); + pipe.ends[1]->shutdownWrite(); + boundedBy(io, kj::mv(drive), 10 * kj::SECONDS, "the pump to settle").wait(ws); + session->wait_echo_done().wait(ws); +} + +KJ_TEST("serve_kj_stream pump: the consumer dropping its end (no shutdown) half-closes the kj " + "side") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pipe = kj::newTwoWayPipe(); + + // A consumer that reads one message and then simply drops its ServeIo: no shutdown() call. + auto session = start_serve_drop_consumer(kj::mv(pipe.ends[0])); + KJ_EXPECT(!session->is_native()); + auto drive = session->drive(); + + pipe.ends[1]->write("ping"_kjb).wait(ws); + // The drop must surface as shutdownWrite() on the kj stream: the client reads EOF. + kj::byte buffer[1]; + KJ_EXPECT(boundedBy(io, pipe.ends[1]->tryRead(buffer, 1, 1), 10 * kj::SECONDS, + "EOF from the pump's shutdownWrite") + .wait(ws) == 0); + // The kj->consumer direction is still waiting on the client; closing the client end lets the + // pump finish both directions. + pipe.ends[1] = nullptr; + boundedBy(io, kj::mv(drive), 10 * kj::SECONDS, "the pump to settle").wait(ws); +} + +// A byte-transforming wrapper over a kj-rs-io TCP stream: XORs everything in both directions. +// The shape of kj::TlsConnection as far as serve_kj_stream is concerned -- it forwards getFd() +// to its transport socket, whose bytes are NOT the stream's bytes. +class XorStream final: public kj::AsyncIoStream { + public: + explicit XorStream(kj::Own inner): inner(kj::mv(inner)) {} + + kj::Promise tryRead(void *buffer, size_t minBytes, size_t maxBytes) override { + size_t n = co_await inner->tryRead(buffer, minBytes, maxBytes); + auto bytes = reinterpret_cast(buffer); + for (size_t i = 0; i < n; i++) bytes[i] ^= KEY; + co_return n; + } + kj::Promise write(kj::ArrayPtr buffer) override { + auto copy = kj::heapArray(buffer.size()); + for (size_t i = 0; i < buffer.size(); i++) copy[i] = buffer[i] ^ KEY; + co_await inner->write(copy); + } + kj::Promise write(kj::ArrayPtr> pieces) override { + for (auto piece: pieces) co_await write(piece); + } + kj::Promise whenWriteDisconnected() override { + return inner->whenWriteDisconnected(); + } + void shutdownWrite() override { + inner->shutdownWrite(); + } + kj::Maybe getFd() const override { + return inner->getFd(); // the transport's fd: ciphertext, not this stream's bytes + } + + private: + static constexpr kj::byte KEY = 0x5a; + kj::Own inner; +}; + +KJ_TEST("serve_kj_stream pumps a byte-transforming wrapper (TLS-shaped) correctly, never its " + "fd") { + // The wrapper exposes its transport's fd, so an fd-tier shortcut would serve the transformed + // bytes. serve_kj_stream must take the pump path and echo the plaintext the client sees. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + auto session = start_serve_echo(kj::heap(kj::mv(pair.server))); + KJ_EXPECT(!session->is_native()); + + // The client speaks through its own XorStream, so plaintext round-trips only if the server + // side was pumped through the wrapper (and not read off the raw socket). + XorStream client(kj::mv(pair.client)); + auto data = makePatternedData(128 * 1024, 13); + auto drive = session->drive(); + auto echo = echoRoundTrip(client, data); + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(echo))).wait(ws); +} + +// ======================================================================================= +// take_kj_socket (native-only: unwrap tier, else fd-dup tier) + +KJ_TEST("take_kj_socket unwraps kj-rs-io TCP streams (tier 1) and echoes") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Native socket taken; the (hollow) kj stream was destroyed inside take_kj_socket. + auto session = start_take_socket_echo(kj::mv(pair.server)); + KJ_EXPECT(session->is_native()); + + auto data = makePatternedData(256 * 1024, 5); + auto drive = session->drive(); + auto client = echoRoundTrip(*pair.client, data); + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(client))).wait(ws); +} + +#if !_WIN32 +// A foreign kj::AsyncIoStream that exposes only its OS fd: unwrap must fail (it is not a +// kj-rs-io wrapper) and any per-read FFI would abort the test -- proving take_kj_socket's fd +// tier does all its I/O on the dup'd socket, never through the kj stream. +class FdOnlyStream final: public kj::AsyncIoStream { + public: + explicit FdOnlyStream(kj::Own inner): inner(kj::mv(inner)) {} + + kj::Maybe getFd() const override { + return inner->getFd(); + } + + kj::Promise tryRead(void *, size_t, size_t) override { + KJ_UNIMPLEMENTED("FdOnlyStream must not be read through the FFI"); + } + kj::Promise write(kj::ArrayPtr) override { + KJ_UNIMPLEMENTED("FdOnlyStream must not be written through the FFI"); + } + kj::Promise write(kj::ArrayPtr>) override { + KJ_UNIMPLEMENTED("FdOnlyStream must not be written through the FFI"); + } + kj::Promise whenWriteDisconnected() override { + KJ_UNIMPLEMENTED("FdOnlyStream must not be observed through the FFI"); + } + void shutdownWrite() override { + KJ_UNIMPLEMENTED("FdOnlyStream must not be shut down through the FFI"); + } + + private: + kj::Own inner; +}; + +KJ_TEST("take_kj_socket dups the fd of foreign fd-backed streams (tier 2); the " + "original stream may be dropped") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Hide the kj-rs-io origin behind a foreign wrapper: only getFd() is reachable. + auto foreign = kj::heap(kj::mv(pair.server)); + + // Fd tier: the dup is independent; take_kj_socket destroys the wrapper (and with it the + // original socket's owner) before returning, which must not tear the served connection down. + auto session = start_take_socket_echo(kj::mv(foreign)); + KJ_EXPECT(session->is_native()); + + auto data = makePatternedData(256 * 1024, 9); + auto drive = session->drive(); + auto client = echoRoundTrip(*pair.client, data); + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(client))).wait(ws); +} + +// A unix-domain (AF_UNIX) socket must be served natively too: take_kj_socket's fd tier dups the +// fd and detects the family (getsockname), producing a tokio UnixStream (ServeIo::Unix). +KJ_TEST("take_kj_socket serves a unix-domain (AF_UNIX) socket via its fd tier and echoes") { + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeUnixPair(io); + + // Hide the kj-rs-io origin behind a foreign wrapper so take_kj_socket must use its fd tier + // (dup + family detection), not the unwrap fast path -- proving the AF_UNIX -> UnixStream + // branch of serve_io_from_owned_fd. + auto foreign = kj::heap(kj::mv(pair.server)); + + auto session = start_take_socket_echo(kj::mv(foreign)); + KJ_EXPECT(session->is_native()); // ServeIo::Unix is a native path (no pump) + + auto data = makePatternedData(256 * 1024, 11); + auto drive = session->drive(); + auto client = echoRoundTrip(*pair.client, data); + kj::joinPromisesFailFast(kj::arr(kj::mv(drive), kj::mv(client))).wait(ws); +} +#endif // !_WIN32 + +KJ_TEST("take_kj_socket on an already-unwrapped (hollow) kj-rs-io stream is refused") { + // Unwrap the stream first (leaving the C++ wrapper hollow), then take_kj_socket it: tier 1 + // (unwrap) fails because it is hollow, and tier 2 (fd dup) finds no fd (getFd -> none on a + // hollow wrapper), so it is refused rather than crashing. + auto io = setupTokioAsyncIo(); + auto &ws = io.getWaitScope(); + auto pair = makeTcpPair(io); + + // Unwrap + drop the native stream; pair.server is now a hollow wrapper. + native_write_via_kj_unwrap(*pair.server, ::rust::Slice()).wait(ws); + + KJ_EXPECT_THROW_MESSAGE( + "cannot take the stream's socket natively", start_take_socket_echo(kj::mv(pair.server))); +} + +KJ_TEST("take_kj_socket refuses fd-less foreign streams (no pump tier)") { + auto io = setupTokioAsyncIo(); + auto pipe = kj::newTwoWayPipe(); + + KJ_EXPECT_THROW_MESSAGE( + "cannot take the stream's socket natively", start_take_socket_echo(kj::mv(pipe.ends[0]))); +} + +} // namespace +} // namespace kj_rs_io_test diff --git a/src/rust/cxx/kj-rs-io/tests/serve_helpers.rs b/src/rust/cxx/kj-rs-io/tests/serve_helpers.rs new file mode 100644 index 00000000000..ecb7130a3cf --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/serve_helpers.rs @@ -0,0 +1,185 @@ +//! Rust helpers for the `serve_kj_stream` `KJ_TEST`s (`serve-test.c++`): +//! echo sessions over each transport path, driven by the C++ tests. + +use std::cell::RefCell; + +use cxx::KjError; +use kj_rs::KjOwn; +use kj_rs_io::serve::ServeIo; +use kj_rs_io::serve::ServePath; +use kj_rs_io::serve::StreamPump; +use tokio::io::AsyncWriteExt; +use tokio::sync::watch; + +use crate::ffi::KjAsyncIoStream; + +type Result = std::result::Result; + +fn kj_err(message: impl std::fmt::Display) -> KjError { + KjError::new(cxx::KjExceptionType::Failed, message.to_string()) +} + +/// The echo task shared by every path: copy everything read back to the writer, then +/// propagate the half-close. Reports completion through `done_tx`. +async fn echo(io: ServeIo, done_tx: watch::Sender) -> std::io::Result { + let result = async { + let (mut rd, mut wr) = tokio::io::split(io); + let n = tokio::io::copy(&mut rd, &mut wr).await?; + wr.shutdown().await?; + Ok(n) + } + .await; + let _ = done_tx.send(true); + result +} + +/// A consumer that reads one message and then drops the stream WITHOUT `shutdown()` (see +/// `start_serve_drop_consumer` in lib.rs). Reports completion through `done_tx`. +async fn read_then_drop(mut io: ServeIo, done_tx: watch::Sender) -> std::io::Result { + use tokio::io::AsyncReadExt; + let result = async { + let mut buf = [0u8; 64]; + let n = io.read(&mut buf).await?; + drop(io); + Ok(n as u64) + } + .await; + let _ = done_tx.send(true); + result +} + +/// One echo server over a served kj stream; see `start_serve_echo` in lib.rs. +pub struct ServeEchoSession { + native: bool, + /// Present for the pumped path; taken by `drive()`. + pump: RefCell>, + /// The echo task's completion signal (fires even if the task failed). + done_rx: watch::Receiver, + /// Taken by `drive()`. + echo: RefCell>>>, + /// For the foreign-thread variant: the OS thread running the echo consumer, joined by + /// `drive()`. `None` for the loop-runtime variant. + foreign: RefCell>>>, +} + +pub fn start_serve_echo(stream: KjOwn) -> Box { + let served = kj_rs_io::serve_kj_stream(stream); + Box::new(ServeEchoSession::new(served.io, served.pump)) +} + +pub fn start_serve_drop_consumer(stream: KjOwn) -> Box { + let served = kj_rs_io::serve_kj_stream(stream); + let native = served.io.path() == ServePath::Native; + let (done_tx, done_rx) = watch::channel(false); + let consumer = kj_rs_tokio::spawn(read_then_drop(served.io, done_tx)); + Box::new(ServeEchoSession { + native, + pump: RefCell::new(served.pump), + done_rx, + echo: RefCell::new(Some(consumer)), + foreign: RefCell::new(None), + }) +} + +/// Like `start_serve_echo`, but the echo consumer runs on a SEPARATE OS thread with its own +/// plain tokio runtime instead of this thread's KJ-loop runtime. For a pumped (Duplex) stream +/// this drives the `ServeIo::Duplex` end entirely off the KJ event-loop thread: every read / +/// write / drop on it wakes the pump (parked on the KJ loop) cross-thread through the kj-rs +/// `FutureWakerCell`. That is the scenario `ServedKjStream::io`'s docs describe as legal since +/// the waker bridge became thread-safe; this proves it (and is a TSAN target). +pub fn start_serve_echo_foreign_thread(stream: KjOwn) -> Box { + let served = kj_rs_io::serve_kj_stream(stream); + Box::new(ServeEchoSession::new_foreign_thread(served.io, served.pump)) +} + +/// Like `start_serve_echo`, but through the native-only entry point (`take_kj_socket`, +/// tiers 1 + 2): errors for streams that are neither kj-rs-io native nor fd-backed (the +/// handed-back stream is dropped with the error). +pub fn start_take_socket_echo(stream: KjOwn) -> Result> { + let io = kj_rs_io::take_kj_socket(stream).map_err(KjError::from)?; + Ok(Box::new(ServeEchoSession::new(io, None))) +} + +impl ServeEchoSession { + fn new(io: ServeIo, pump: Option) -> Self { + let native = io.path() == ServePath::Native; + let (done_tx, done_rx) = watch::channel(false); + // The echo consumer runs on this thread's KJ-loop runtime: the one-runtime shape + // (native sockets are registered with this runtime's I/O driver anyway). + let echo = kj_rs_tokio::spawn(echo(io, done_tx)); + Self { + native, + pump: RefCell::new(pump), + done_rx, + echo: RefCell::new(Some(echo)), + foreign: RefCell::new(None), + } + } + + fn new_foreign_thread(io: ServeIo, pump: Option) -> Self { + let native = io.path() == ServePath::Native; + let (done_tx, done_rx) = watch::channel(false); + // Run the echo consumer on its own OS thread with a private current-thread runtime, so + // the Duplex end is driven entirely off the KJ event-loop thread. + let foreign = std::thread::spawn(move || { + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .expect("build echo-consumer runtime"); + rt.block_on(echo(io, done_tx)) + }); + Self { + native, + pump: RefCell::new(pump), + done_rx, + echo: RefCell::new(None), + foreign: RefCell::new(Some(foreign)), + } + } + + pub fn is_native(&self) -> bool { + self.native + } + + /// Runs the connection to completion: drives the pump (if any) and waits for the echo + /// task to finish. Dropping the returned promise mid-connection drops the pump — + /// the abort-on-drop path. + pub async fn drive(&self) -> Result<()> { + let pump = self.pump.borrow_mut().take(); + let echo = self.echo.borrow_mut().take(); + // Await the pump but do NOT `?` yet: on the foreign-thread variant we must still join + // the consumer OS thread even if the pump errored, or we leak an unjoined thread. + let pump_result = match pump { + Some(pump) => pump.await, + None => Ok(()), + }; + if let Some(handle) = echo { + handle + .await + .map_err(|e| kj_err(format!("echo task panicked: {e}")))? + .map_err(|e| kj_err(format!("echo failed: {e}")))?; + } + // Foreign-thread variant: wait for the consumer to finish (dropping the pump above + // EOFs it), then join. Waiting on the done signal first keeps the join near-instant. + let foreign = self.foreign.borrow_mut().take(); + if let Some(handle) = foreign { + self.wait_echo_done().await; + handle + .join() + .map_err(|_| kj_err("echo consumer thread panicked"))? + .map_err(|e| kj_err(format!("echo failed: {e}")))?; + } + pump_result?; + Ok(()) + } + + /// Resolves once the echo task has finished (however `drive()` fared) — used by the + /// drop-abort test to observe that dropping the pump EOFs the consumer. + pub async fn wait_echo_done(&self) { + let mut rx = self.done_rx.clone(); + // Cannot fail: the sender is owned by the echo task, which always sends before exit; + // even if it panicked, the closed channel resolves wait_for with an error we ignore + // after checking the flag. + let _ = rx.wait_for(|done| *done).await; + } +} diff --git a/src/rust/cxx/kj-rs-io/tests/test_helpers.rs b/src/rust/cxx/kj-rs-io/tests/test_helpers.rs new file mode 100644 index 00000000000..269bbd7166b --- /dev/null +++ b/src/rust/cxx/kj-rs-io/tests/test_helpers.rs @@ -0,0 +1,80 @@ +use std::io; +use std::pin::Pin; + +use cxx::KjError; +use kj_rs_io::TokioStream; +use tokio::io::Interest; +use tokio::net::TcpStream; + +use crate::ffi::KjAsyncIoStream; +use crate::ffi::PreboundListener; + +type Result = std::result::Result; + +fn kj_err(message: impl std::fmt::Display) -> KjError { + KjError::new(cxx::KjExceptionType::Failed, message.to_string()) +} + +/// Write-all over the native tokio readiness API. +async fn native_write_all(stream: &TcpStream, data: &[u8]) -> io::Result<()> { + let mut written = 0; + while written < data.len() { + match stream.try_write(&data[written..]) { + Ok(n) => written += n, + Err(e) if e.kind() == io::ErrorKind::WouldBlock => { + stream.ready(Interest::WRITABLE).await?; + } + Err(e) => return Err(e), + } + } + Ok(()) +} + +pub async fn native_write_via_unwrap(stream: Box, data: Vec) -> Result<()> { + let tcp = stream + .into_tcp_stream() + .ok_or_else(|| kj_err("expected a TCP stream"))?; + native_write_all(&tcp, &data).await.map_err(kj_err)?; + // Dropping `tcp` closes the connection; the C++ side observes data followed by EOF. + Ok(()) +} + +pub async fn native_write_via_kj_unwrap( + stream: Pin<&mut KjAsyncIoStream>, + data: &[u8], +) -> Result<()> { + // unwrap_kj_stream is safe now: it detects in-flight I/O and returns an error rather than + // aliasing (the C++ test also keeps no I/O in flight here). + let native = kj_rs_io::unwrap_kj_stream(stream).map_err(KjError::from)?; + let tcp = native + .into_tcp_stream() + .ok_or_else(|| kj_err("expected a TCP stream"))?; + native_write_all(&tcp, data).await.map_err(kj_err)?; + Ok(()) +} + +pub fn create_prebound_listener_fd() -> Result { + let listener = std::net::TcpListener::bind("127.0.0.1:0").map_err(kj_err)?; + let port = listener.local_addr().map_err(kj_err)?.port(); + #[cfg(unix)] + { + use std::os::fd::IntoRawFd; + Ok(PreboundListener { + fd: listener.into_raw_fd(), + port, + }) + } + #[cfg(not(unix))] + { + Err(kj_err("not supported on this platform")) + } +} + +/// See the bridge doc on `address_from_loopback_ports` (lib.rs). +pub fn address_from_loopback_ports(ports: &[u16]) -> Box { + let addrs = ports + .iter() + .map(|&port| std::net::SocketAddr::from((std::net::Ipv4Addr::LOCALHOST, port))) + .collect(); + Box::new(kj_rs_io::TokioAddress::from_socket_addrs(addrs)) +} diff --git a/src/rust/cxx/kj-rs-io/unwrap.h b/src/rust/cxx/kj-rs-io/unwrap.h new file mode 100644 index 00000000000..9c3033d1279 --- /dev/null +++ b/src/rust/cxx/kj-rs-io/unwrap.h @@ -0,0 +1,95 @@ +#pragma once +// Declarations needed by the kj-rs-io cxx bridge (lib.rs) itself. The full C++ API lives in +// kj-rs-io/async-io.h; this header only exposes what the generated bridge code references: +// kj::AsyncIoStream (as an opaque extern C++ type), the unwrap hook, and the bridged stream +// operations backing serve_kj_stream()'s pump fallback (serve.rs). + +#include + +#include + +namespace kj_rs_io { + +struct TokioStream; // Opaque Rust type, defined in the generated lib.rs.h. + +// Recovers the native Rust stream out of a kj::AsyncIoStream created by kj-rs-io (the "unwrap +// fast path"), leaving the wrapper hollow: any further I/O through the wrapper throws. Throws if +// `stream` is not a kj-rs-io stream, was already unwrapped, or has I/O promises in flight (the +// Rust side tracks in-flight operations, so this is detected, not a caller contract). +// +// Implemented in async-io.c++. Rust code calls this through kj_rs_io::unwrap_kj_stream(). +::rust::Box unwrapTokioStream(kj::AsyncIoStream &stream); + +// --- Bridged operations on a *foreign* kj::AsyncIoStream (one that did not originate in +// kj-rs-io and therefore cannot be unwrapped). These back the duplex-pump fallback of +// serve_kj_stream() (serve.rs): the pump owns the stream and reads and writes it through +// concurrent Rust *shared* borrows — kj two-way streams support one read and one write in +// flight at once. + +// A Rust shared borrow (&KjAsyncIoStream) arrives in C++ as a const&: the constness is cxx's +// wire format for "shared", not a property of the object — the stream is uniquely owned by the +// pump and never actually const. NOTE the meaning mismatch with KJ convention: KJ const means +// *thread-safe* (Rust's Sync), but this shared-ness is the weaker property — reentrant use from +// one thread's async tasks (the bridge types are !Send/!Sync, so Rust can never move these +// borrows off the KJ event-loop thread that owns the stream). Recover the callable reference +// here, once. +inline kj::AsyncIoStream &pumpStream(const kj::AsyncIoStream &stream) { + return const_cast(stream); +} + +// The pieces of a kj::AsyncOutputStream::write(pieces) call, handed to Rust for a vectored write +// (stream_write_pieces, ffi.rs). Opaque to cxx; Rust reads it through the two accessors below. +// Owned by the C++ coroutine frame that awaits the bridged future (TokioAsyncIoStream:: +// writePieces); the piece buffers are the caller's, valid until that promise settles. +struct KjPieces { + kj::ArrayPtr> pieces; +}; + +inline size_t kjPiecesCount(const KjPieces &pieces) { + return pieces.pieces.size(); +} + +inline ::rust::Slice kjPiece(const KjPieces &pieces, size_t index) { + auto piece = pieces.pieces[index]; + return ::rust::Slice(piece.begin(), piece.size()); +} + +// Corresponds to kj::AsyncIoStream::tryRead(buffer, minBytes, buffer.size()). +inline kj::Promise kjStreamTryRead( + const kj::AsyncIoStream &stream, ::rust::Slice buffer, size_t minBytes) { + return pumpStream(stream).tryRead(buffer.data(), minBytes, buffer.size()); +} + +// Corresponds to kj::AsyncIoStream::write(buffer) (write-all semantics). +inline kj::Promise kjStreamWrite( + const kj::AsyncIoStream &stream, ::rust::Slice buffer) { + return pumpStream(stream).write(kj::arrayPtr(buffer.data(), buffer.size())); +} + +// Corresponds to kj::AsyncIoStream::shutdownWrite(). +inline void kjStreamShutdownWrite(const kj::AsyncIoStream &stream) { + pumpStream(stream).shutdownWrite(); +} + +// The stream's underlying raw OS socket handle -- a Unix fd (kj::AsyncIoStream::getFd()) or a +// win32 SOCKET (kj::AsyncIoStream::getWin32Handle()) -- widened to int64, or -1 if it exposes +// none. int64 fits both losslessly with one sentinel: a Unix fd is a non-negative int, and +// INVALID_SOCKET (~0 as UINT_PTR) is exactly -1 as int64 (live win64 SOCKET values fit in 32 +// bits per the Windows handle-interoperability guarantee, so they never collide with -1). +// Backs the handle tier of take_kj_socket() (ffi.rs); see that function's docs for the +// caller-asserted "the handle carries the stream's own bytes" contract (wrappers such as +// kj::TlsConnection forward getFd() to their *transport* socket, which this tier must never be +// used on). +inline int64_t kjStreamGetHandle(const kj::AsyncIoStream &stream) { +#if _WIN32 + // Validated by Windows CI; mirrors the unix arm. + KJ_IF_SOME(handle, stream.getWin32Handle()) { + return static_cast(reinterpret_cast(handle)); + } + return -1; +#else + return stream.getFd().orDefault(-1); +#endif +} + +} // namespace kj_rs_io