UDF Runner V2: Interface skeleton

doc/design/v2/runner/context_implementation.md

@@ -174,7 +174,7 @@ shared atomic byte allowance plus a generation used for timed waiting; it is not
 
 1. Run synchronous worker-side dispatch before inspecting the grant.
 2. If this is not the first record batch in the direction and the grant is zero, wait for a positive grant, the optional
-   send deadline, or terminal state. A timeout returns `timed_out` without consuming the builder or its Arrow
+   send deadline, or terminal state. A timeout returns `TimedOut` without consuming the builder or its Arrow
    ownership. The first batch does not wait for `Next` credit.
 3. Measure the whole unsplit batch's Arrow buffer bytes and consume that amount from the worker grant. A positive grant
    admits the current batch even if this one submission overshoots it.
@@ -542,8 +542,8 @@ The terminal operation statuses distinguish at least:
 
 ```text
 closed
-transport_error
-protocol_error
+TransportError
+ProtocolError
 ```
 
 Timeout remains a normal non-terminal receive result. It is not converted into a context exception.

doc/design/v2/runner/context_interface.md

@@ -20,24 +20,24 @@ Readiness operations report status without consuming a message:
 
 ```text
 MessageStatus =
-    message_available
-  | no_message
-  | timed_out
-  | cancelled
-  | peer_closed
-  | connection_error
+    MessageAvailable
+  | NoMessage
+  | TimedOut
+  | Cancelled
+  | PeerClosed
+  | ConnectionError
 ```
 
 Consuming operations return either a value or a terminal result:
 
 ```text
 OperationStatus =
-    ok
-  | timed_out
-  | cancelled
-  | peer_closed
-  | protocol_error
-  | transport_error
+    Ok
+  | TimedOut
+  | Cancelled
+  | PeerClosed
+  | ProtocolError
+  | TransportError
 ```
 
 The conceptual generic result type is:
@@ -50,7 +50,7 @@ Result<T> {
 }
 ```
 
-`Result<void>` represents an operation that has no value on success. A successful result has `status == ok` and a
+`Result<void>` represents an operation that has no value on success. A successful result has `status == Ok` and a
 value where applicable. A failed result has no value and may include an error code and message.
 
 All timeout parameters use the conceptual type `optional<Duration>`. An absent timeout waits indefinitely; a zero
@@ -113,7 +113,7 @@ Call {
 `Call::receive_status()` is non-blocking, observes only this call's stream, and never consumes a message.
 
 `Call::receive(timeout)` waits only for this call's stream. An omitted timeout means wait indefinitely. It returns the
-next complete call message, returns `timed_out` when the timeout expires, and returns a terminal result for
+next complete call message, returns `TimedOut` when the timeout expires, and returns a terminal result for
 cancellation, peer closure, or connection failure.
 
 The first successful `Call::receive()` returns the opening `CallMessageView`, including its `open_call` field and any
@@ -128,7 +128,7 @@ If a control message or another call's message arrives while `Call::receive()` i
 to its own stream and the call receive continues waiting. It does not return unrelated traffic.
 
 `Call::send()` waits for the context to grant permission for the next data batch when required by flow control. An
-absent timeout waits indefinitely, a zero timeout does not block, and an expired timeout returns `timed_out` without
+absent timeout waits indefinitely, a zero timeout does not block, and an expired timeout returns `TimedOut` without
 consuming the builder or its Arrow ownership. The background thread owns the authoritative credit and any internal
 batch splitting; the worker-facing send operation submits one logical batch.
 
@@ -138,7 +138,7 @@ The worker-facing data-schema value is:
 
 ```text
 DataSchema {
-    ArrowSchema schema
+    ArrowSchemaPtr schema
     bool has_group_id
     bool has_row_id
 }
@@ -150,7 +150,7 @@ The worker-facing record-batch value is:
 
 ```text
 RecordBatch {
-    ArrowArray array
+    ArrowArrayPtr array
     bool is_end_of_group
 }
 ```
@@ -225,7 +225,7 @@ ControlMessage {
 ```
 
 The control stream supports capabilities, keepalive, connection-level payloads, errors, and connection close. It must
-not expose `OpenCall`, `CloseCall`, `Next`, `ArrowSchema`, or `ArrowArray` objects.
+not expose `OpenCall`, `CloseCall`, `Next`, `ArrowSchemaPtr`, or `ArrowArrayPtr` objects.
 
 An error without `close_connection` is a non-terminal diagnostic. `CloseConnection` starts or acknowledges connection
 shutdown and may be combined with an error or other permitted connection-level fields.
@@ -241,9 +241,9 @@ and prevents connection-only fields from being sent on a call stream.
 CallMessageBuilder {
     open_call(call_name) -> CallMessageBuilder&
     payloads(payloads) -> CallMessageBuilder&
-    data_schema(ArrowSchema*, has_group_id, has_row_id) -> CallMessageBuilder&
+    data_schema(ArrowSchemaPtr schema, has_group_id, has_row_id) -> CallMessageBuilder&
     next(byte_budget, reset, row_id) -> CallMessageBuilder&
-    record_batch(ArrowArray*, is_end_of_group) -> CallMessageBuilder&
+    record_batch(ArrowArrayPtr array, is_end_of_group) -> CallMessageBuilder&
     error(code, message) -> CallMessageBuilder&
     close_call() -> CallMessageBuilder&
 }
@@ -271,40 +271,41 @@ The control builder cannot construct `OpenCall`, `CloseCall`, `Next`, `DataSchem
 ```text
 CallMessageView {
     has_open_call() -> bool
-    open_call() -> const OpenCall*
+    open_call() -> std::optional<std::reference_wrapper<const OpenCall>>
     has_payloads() -> bool
-    payloads() -> const Payloads*
+    payloads() -> std::optional<std::reference_wrapper<const protocol::Payloads>>
     has_data_schema() -> bool
     data_schema() -> DataSchemaView
     has_next() -> bool
-    next() -> NextView
+    next() -> std::optional<std::reference_wrapper<const Next>>
     has_record_batch() -> bool
     record_batch() -> RecordBatchView
     has_error() -> bool
-    error() -> const Error*
+    error() -> std::optional<std::reference_wrapper<const ErrorInfo>>
     has_close_call() -> bool
-    close_call() -> const CloseCall*
 }
 ```
 
-`CallMessageView` and `ControlMessageView` are the primary specialized message views. Their nested protocol metadata
-accessors return generated FlatBuffer table pointers, except for `NextView`, which intentionally hides the internal
-transport byte budget. `DataSchemaView` and `RecordBatchView` remain explicit Arrow C Data Interface adapters because
-the wire metadata must be converted to Arrow schema/array ownership for the worker.
+`CallMessageView` and `ControlMessageView` expose three distinct layers of accessors. High-level accessors return
+worker-facing `OpenCall`, `Next`, and `ErrorInfo` values through optional reference wrappers. The
+`server_capabilities()` and `payloads()` accessors return optional references to generated FlatBuffer `protocol`
+tables. The `DataSchemaView` and `RecordBatchView` accessors are Arrow C Data Interface adapters because the wire
+metadata must be converted to Arrow schema/array ownership for the worker. The interface defines these adapters even
+where their transport implementation is deferred.
 
 `DataSchemaView` exposes the Arrow schema and its correlation flags. `RecordBatchView` exposes the Arrow array and
 `is_end_of_group`.
 
-### Received Next view
+### Received Next value
 
 ```text
-NextView {
+Next {
     reset() -> bool
     row_id() -> uint64
 }
 ```
 
-`NextView` does not expose `byte_budget`. The background thread consumes the received wire-level byte budget for
+`Next` does not expose `byte_budget`. The background thread consumes the received wire-level byte budget for
 authoritative flow control before the message is delivered to the worker. The outbound
 `CallMessageBuilder.next(byte_budget, reset, row_id)` operation remains available because the worker may grant credit
 to its peer.
@@ -314,15 +315,13 @@ to its peer.
 ```text
 ControlMessageView {
     has_server_capabilities() -> bool
-    server_capabilities() -> const ServerCapabilities*
+    server_capabilities() -> std::optional<std::reference_wrapper<const protocol::ServerCapabilities>>
     has_keep_alive() -> bool
-    keep_alive() -> const KeepAlive*
     has_payloads() -> bool
-    payloads() -> const Payloads*
+    payloads() -> std::optional<std::reference_wrapper<const protocol::Payloads>>
     has_error() -> bool
-    error() -> const Error*
+    error() -> std::optional<std::reference_wrapper<const ErrorInfo>>
     has_close_connection() -> bool
-    close_connection() -> const CloseConnection*
 }
 ```
 
@@ -334,8 +333,9 @@ Each top-level specialized view retains a verified, immutable storage lease for
 FlatBuffer accessors point into that storage and remain valid until the top-level view is destroyed. Receiving another
 message does not invalidate an existing view. Malformed FlatBuffer data produces an error rather than a view.
 
-The view also retains the Arrow schema and array ownership leases required by its fields. Arrow release callbacks are
-invoked exactly once.
+Where a view exposes Arrow data, it retains the Arrow schema and array ownership leases required by its fields. Arrow
+release callbacks are invoked exactly once. A frame that contains only wire-level data metadata must not claim to
+provide a usable Arrow object until the corresponding adapter has populated the owned pointer.
 
 ### Builder and view implementation contract
 
@@ -355,8 +355,12 @@ If validation, finalization, or transport acceptance fails before ownership tran
 
 Data schemas and record batches are transferred through the Arrow C Data Interface, not as serialized FlatBuffer
 objects. The conceptual `data_schema` field contains the interface-level `DataSchema` value, whose `schema` field is
-an `ArrowSchema`. The conceptual `record_batch` field contains the interface-level `RecordBatch` value, whose
-`array` field is an `ArrowArray`.
+an `ArrowSchemaPtr`. The conceptual `record_batch` field contains the interface-level `RecordBatch` value, whose
+`array` field is an `ArrowArrayPtr`.
+
+`ArrowSchemaPtr` and `ArrowArrayPtr` are move-only `std::unique_ptr` aliases with custom deleters. Their deleters call
+the Arrow C Data Interface `release` callback, when present; they never apply C++ `delete` to an Arrow object. This
+makes ownership explicit while preserving the C Data Interface as the worker-facing type.
 
 `DataSchema.schema` describes one direction of a call's data stream. It is sent once per direction, before or together
 with that direction's first `RecordBatch`. A later `RecordBatch` uses the previously announced schema and flags and
@@ -385,11 +389,13 @@ transport-specific buffer handling remain hidden.
 
 Ownership follows the Arrow release contract:
 
-- received `DataSchema.schema` and `RecordBatch.array` objects become owned by the caller;
+- received `DataSchema.schema` and `RecordBatch.array` objects become owned by the caller once the receive-side Arrow
+  adapters are implemented;
 - the caller releases each received Arrow object exactly once;
-- sent objects remain caller-owned until the context accepts the send;
-- after successful acceptance, release responsibility transfers to the context;
-- a failed send before acceptance leaves ownership with the caller;
+- sent `ArrowSchemaPtr` and `ArrowArrayPtr` objects remain owned by the builder/caller until the context accepts the
+  send;
+- after successful acceptance, the context takes the unique pointers and therefore the release responsibility;
+- a failed send before acceptance leaves the unique pointers with the builder/caller;
 - no object may be released more than once.
 
 ## Dispatch and consumption rules
@@ -537,8 +543,8 @@ The design is complete when it is clear that:
 - call fields can be combined or sent separately;
 - errors can be sent alone;
 - close messages can carry other permitted fields;
-- record batches use an Arrow `ArrowArray` plus `is_end_of_group` metadata;
-- data schemas use `ArrowSchema` and preserve `has_group_id` and `has_row_id`;
+- record batches use an `ArrowArrayPtr` plus `is_end_of_group` metadata;
+- data schemas use an `ArrowSchemaPtr` and preserve `has_group_id` and `has_row_id`;
 - Arrow ownership and release responsibility are explicit;
 - cancellation, closure, timeout, and failure behavior is defined;
 - one worker thread owns each context and concurrent operations on that context are outside the contract;

udf-runner-cpp/v2/BUILD.bazel

@@ -86,8 +86,12 @@ genrule(
 cc_library(
     name = "udf_protocol",
     srcs = ["udf_protocol.cc"],
-    hdrs = ["udf_protocol.hpp", ":udf_protocol_generated"],
+    hdrs = [
+        "include/exasol/udf/v2/udf_protocol.hpp",
+        ":udf_protocol_generated",
+    ],
     copts = ["-std=c++23"],
+    includes = ["include"],
     # Do not depend on the ordinary runtime headers: the generated protocol
     # header must resolve all runtime includes through the isolated copy.
     deps = [":private_flatbuffers_runtime"],
@@ -113,14 +117,22 @@ cc_test(
     copts = ["-std=c++23"],
     tags = ["no-mull"],
     deps = [
+        ":interface",
         ":udf_protocol",
         "@googletest//:gtest_main",
     ],
 )
 
+cc_test(
+    name = "interface_test",
+    srcs = ["interface_test.cc"],
+    copts = ["-std=c++23"],
+    tags = ["no-mull"],
+    deps = [":interface"],
+)
+
 cc_binary(
     name = "udf_protocol_shared",
-    srcs = ["udf_protocol.cc"],
     copts = ["-std=c++23"],
     linkshared = 1,
     deps = [":udf_protocol"],
@@ -227,6 +239,40 @@ cc_library(
     deps = ["@v2_json_schema_validator//:json_schema_validator"],
 )
 
+cc_library(
+    name = "interface",
+    srcs = [
+        "call.cc",
+        "context.cc",
+        "control_stream.cc",
+        "call_message_builder.cc",
+        "control_message_builder.cc",
+        "message_view.cc",
+    ],
+    hdrs = [
+        "include/exasol/udf/v2/call.hpp",
+        "include/exasol/udf/v2/context.hpp",
+        "include/exasol/udf/v2/content_types.hpp",
+        "include/exasol/udf/v2/control_stream.hpp",
+        "include/exasol/udf/v2/interface.hpp",
+        "include/exasol/udf/v2/message_builder/call_message_builder.hpp",
+        "include/exasol/udf/v2/message_builder/control_message_builder.hpp",
+        "include/exasol/udf/v2/message_view/call_message_view.hpp",
+        "include/exasol/udf/v2/message_view/control_message_view.hpp",
+        "include/exasol/udf/v2/message_view/data_schema_view.hpp",
+        "include/exasol/udf/v2/message_view/record_batch_view.hpp",
+        "include/exasol/udf/v2/types.hpp",
+    ],
+    includes = [
+        "include",
+    ],
+    copts = ["-std=c++23"],
+    deps = [
+        ":arrow_core",
+        ":udf_protocol",
+    ],
+)
+
 cc_test(
     name = "json_schema_validation_test",
     srcs = ["json_schema_validation_test.cc"],

udf-runner-cpp/v2/MODULE.bazel

@@ -4,7 +4,7 @@ module(
 )
 
 bazel_dep(name = "rules_cc", version = "0.2.17")
-bazel_dep(name = "platforms", version = "1.0.0")
+bazel_dep(name = "platforms", version = "1.1.0")
 bazel_dep(name = "flatbuffers", version = "25.2.10")
 bazel_dep(name = "googletest", version = "1.15.0")
 bazel_dep(name = "google_benchmark", version = "1.9.5")

udf-runner-cpp/v2/call.cc

@@ -0,0 +1,5 @@
+#include <exasol/udf/v2/call.hpp>
+namespace exasol::udf::v2
+{
+Call::~Call() = default;
+} // namespace exasol::udf::v2