egress: add egress gateway support - atenet-egress by LiorLieberman · Pull Request #693 · agent-substrate/substrate

Lior Lieberman (LiorLieberman) · 2026-08-02T03:48:28Z

Added pluggable egress PEP support. (Feedback on atenet-egress name is welcomed - will open a separate PR to rename atenet-router to atenet-ingress )

#559 shipped the actor egress data path without any egress gateway. This adds an Envoy deployment that terminates actor CONNECTs. It requires downstream mTLS, so only a worker's atunnel can reach it.

The gateway consists of an Envoy and an atenet router --standalone ext_proc sidecar.

The same ext_proc binary now serves both directions. Direction is decided by
the accepting listener via the xds.listener_name CEL attribute, not by
anything in the request, so an ingress client cannot reach the egress handler
by crafting a CONNECT (and maybe thats not ideal? feedback is welcomed!).

handleEgressRequestHeaders checks the method,
validates the worker-asserted actor identity headers, authenticates them
against the control plane with GetActor, and rejects actors that are not
RUNNING or whose asserted version is stale.

With that change, atelet's Run/Restore requests grow two fields it now
populates on the way down to ateom:

egress_gateway_address, from the new cluster-wide
--egress-gateway-address flag, which the manifests point at
atenet-egress.ate-system.svc:443. This is the value that arms the
nftables REDIRECT; from here actor TCP egress leaves through the gateway
rather than the worker's masquerade.
actor_version, the Actor resource version ate-api observed when it
assigned the worker. atunnel asserts it to the gateway, and the gateway
fails closed when its own GetActor read is older, so a stale control-plane
view cannot authorize egress for an Actor whose assignment has since
moved. atelet rejects Run/Restore without it.

Tests pass
Appropriate changes to documentation are included in the PR

The gateway terminates actor's CONNECT request. It requires downstream mTLS, so only a worker's atunnel can reach it. The gateway consists of an Envoy and an `atenet router --standalone` ext_proc sidecar.

Lior Lieberman (LiorLieberman) · 2026-08-02T03:58:34Z

FYI Dmitry Berkovich (@dberkov)

demos/egress is a small Actor that fetches a URL it is given and echoes the upstream status and body back, which makes the egress path observable from outside the sandbox. hack/install-demo-egress.sh registers it as a --deploy-demo-egress fixture and hack/verify-egress-demo.sh drives it and checks the atenet-egress logs for the corresponding authorized CONNECT. TestActorEgress in the networking suite covers the same path automatically: it creates an Actor from the demo template, POSTs a fetch request through atenet-router, and asserts 200. The suite's actor helper is parameterised by template so the ingress test keeps using the counter fixture.

haiyanmeng · 2026-08-04T01:11:51Z

+      # Envoy calls it over localhost to authenticate actor identity against the
+      # ate API on every CONNECT. This mirrors the ingress gateway topology
+      # (Envoy + ext_proc in one pod); a shared/standalone ext_proc is a future step.
+      - name: ext-proc


Let us make ext-proc a native sidecar (initContainers + restartPolicy: Always).
Regular containers get SIGTERM together, ext_proc has no drain and will exit first,
and with failure_mode_allow: false every CONNECT arriving during Envoy's drain
window fails closed with a 503.

haiyanmeng · 2026-08-04T01:14:00Z

+      # Co-located ext_proc server (the atenet router, ext_proc-only). The egress
+      # Envoy calls it over localhost to authenticate actor identity against the
+      # ate API on every CONNECT. This mirrors the ingress gateway topology
+      # (Envoy + ext_proc in one pod); a shared/standalone ext_proc is a future step.


a shared/standalone ext_proc is a future step: can we first move the exe_proc logic into a separate binary?

ext_proc is already separate

I don't think so. See Line 268 in this file:

image: ko://github.com/agent-substrate/substrate/cmd/atenet

Bowei Du (bowei) · 2026-08-04T21:07:17Z

implementation itself looks ok, but would like to fix a few organization things:

Make it so that egress, ingress for ext_proc can be enabled separately (add flags and gated deployment). We currently deploy both in the same deployment, but at scale, you would probably want to have separate ingress and egress deployments as they will be allocated resources very differently.
Put ingress,egress logic in .../atenet/router/{ingress,egress} packages.
router/ext_proc just has mux logic

probably something like:

switch {
case EgressEnabled && IsEgressRequest(...):
case IngressEnabled:
default:
   // send 4xx nothing is enabled.
}

Bowei Du (bowei)

You need to scrub through all of the comments -- there seems to some internal notes to yourself and your friend Claude.

Bowei Du (bowei) · 2026-08-04T16:51:33Z

 	localhostRegistryReplacement = pflag.String("localhost-registry-replacement", "", "The replacement registry endpoint for localhost and/or loopback IP addresses, useful for local development. for example kind-registry:5000")
 	imageCacheDir                = pflag.String("image-cache-dir", ateompath.ImageCacheDir, "Directory for the node-local OCI image layer cache. Must be on the volume shared with the ateom pods (the cached layers are their overlay lowerdirs), and on a disk sized for both capacity and IOPS: unpack throughput is gated by the volume's IOPS.")

+	// SEE(lior): both sides kept — main added --log-level here while this branch


What is this?

Bowei Du (bowei) · 2026-08-04T16:52:01Z

 	imageCacheDir                = pflag.String("image-cache-dir", ateompath.ImageCacheDir, "Directory for the node-local OCI image layer cache. Must be on the volume shared with the ateom pods (the cached layers are their overlay lowerdirs), and on a disk sized for both capacity and IOPS: unpack throughput is gated by the volume's IOPS.")

+	// SEE(lior): both sides kept — main added --log-level here while this branch
+	// added --egress-gateway-address. Independent flags, no interaction.


probably want to scrub your comments

Bowei Du (bowei) · 2026-08-04T16:53:24Z

+	// When set, actors whose Run/Restore request does not already carry an egress
+	// gateway address have this address injected, causing ateom to redirect actor
+	// TCP egress through atunnel to the egress gateway. Empty keeps egress off.
+	egressGatewayAddress = pflag.String("egress-gateway-address", "", "Address (host:port) of the egress gateway. When set, actor TCP egress is transparently tunneled through atunnel to this gateway. Empty disables egress.")


Your comment and the help text are different.

As I understand it -- this sets a default egress if not specified on the Actor resource explicitly?

Can you fix it to match up?

Bowei Du (bowei) · 2026-08-04T20:43:53Z

 		case *extprocv3.ProcessingRequest_RequestHeaders:
 			start := time.Now()
-			hResponse, rqm, target, tmplNs, tmplName, resumeOutcome, err := s.handleRequestHeaders(stream.Context(), reqType.RequestHeaders)
+			// One ext_proc server handles both directions: actor egress


Let's factor this so you could run it in this way, but in a real deployment, I think you would keep them as separate deployments as they will likely have different scaling parameters.

Bowei Du (bowei) · 2026-08-04T20:52:52Z

+	// filter chain that accepted the request. The egress Envoy asks for it via
+	// request_attributes on its ext_proc filter.
+	//
+	// SEE(lior): this was xds.listener_name, which reads more naturally but


you should scrub for these.

Bowei Du (bowei) · 2026-08-05T06:06:31Z

Needs rebase

egress: add an Envoy egress gateway.

7a02250

The gateway terminates actor's CONNECT request. It requires downstream mTLS, so only a worker's atunnel can reach it. The gateway consists of an Envoy and an `atenet router --standalone` ext_proc sidecar.

Lior Lieberman (LiorLieberman) requested review from Bowei Du (bowei) and John Howard (howardjohn) August 2, 2026 03:48

Lior Lieberman (LiorLieberman) force-pushed the pr2-atenet-egress branch from 305b232 to 598a548 Compare August 3, 2026 19:02

haiyanmeng reviewed Aug 4, 2026

View reviewed changes

Bowei Du (bowei) reviewed Aug 4, 2026

View reviewed changes

Navigation Menu

Uh oh!

egress: add egress gateway support - atenet-egress - #693

egress: add egress gateway support - atenet-egress#693
Lior Lieberman (LiorLieberman) wants to merge 2 commits into
agent-substrate:mainfrom
LiorLieberman:pr2-atenet-egress

Lior Lieberman (LiorLieberman) commented Aug 2, 2026

Uh oh!

Lior Lieberman (LiorLieberman) commented Aug 2, 2026

Uh oh!

haiyanmeng Aug 4, 2026

Uh oh!

haiyanmeng Aug 4, 2026

Uh oh!

Bowei Du (bowei) Aug 4, 2026

Uh oh!

haiyanmeng Aug 4, 2026

Uh oh!

Bowei Du (bowei) commented Aug 4, 2026

Uh oh!

Bowei Du (bowei) left a comment

Uh oh!

Bowei Du (bowei) Aug 4, 2026

Uh oh!

Bowei Du (bowei) Aug 4, 2026

Uh oh!

Bowei Du (bowei) Aug 4, 2026

Uh oh!

Bowei Du (bowei) Aug 4, 2026

Uh oh!

Bowei Du (bowei) Aug 4, 2026

Uh oh!

Bowei Du (bowei) commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Lior Lieberman (LiorLieberman) commented Aug 2, 2026

Uh oh!

Lior Lieberman (LiorLieberman) commented Aug 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bowei Du (bowei) commented Aug 4, 2026

Uh oh!

Bowei Du (bowei) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bowei Du (bowei) commented Aug 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants