This is informational rather than a bug report. The OpenTelemetry Java agent instruments pekko-http to create HTTP server and client spans and to propagate context, and it does so by attaching bytecode advice to specific methods. Most of those methods are internal to pekko-http, so a refactor that is invisible to users can silently stop the instrumentation from working. Filing this so the list is written down somewhere the pekko-http project can see it.
Why "silently": the agent's muzzle checks verify the classes and methods its advice code calls, and disable the instrumentation when they no longer exist. They do not verify the method matchers. If a matched method is renamed, inlined, or restructured, the advice simply never applies — no error, no log, just missing telemetry.
What is used
Public API, no concern, listed for completeness:
| Symbol |
Used for |
HttpExt.bindAndHandle, HttpExt.singleRequest |
server and client entry points |
Http.IncomingConnection.handleWith |
server bindings created from a connection source |
Directive.tapply, PathMatcher.apply, PathMatcher$.apply |
deriving http.route |
Internal API:
| Symbol |
Declared |
Used for |
HttpServerBluePrint.requestPreparation |
@InternalApi private[http] object |
wraps the http/1.1 request/response bidi to start and end server spans |
Http2Ext.bindAndHandleAsync |
@InternalApi private[http] final class |
same for http/2, which does not go through bindAndHandle |
Http2Ext.system |
same |
obtaining a dispatcher for the response future |
Http2.streamId |
@InternalApi private[http] object |
recognising a request that was replayed through the http/2 stack after an h2c upgrade |
PoolMasterActor.startPoolInterface |
private def in @InternalApi private[http] final class |
clearing the context when a connection pool actor starts, so pool setup is not attributed to the request that triggered it |
The last one is the fragile case. Being a private method it has no stable name, and the agent matches the mangled org$apache$pekko$http$impl$engine$client$PoolMasterActor$$startPoolInterface. That name exists only because a closure in startPoolInterface captures it. If that closure is refactored away the method stops being mangled, or is inlined and disappears from the bytecode entirely.
That is not hypothetical: while instrumenting pekko remoting we found EndpointReader.dispatchMessage in the sources but not in the bytecode, because scala had inlined it.
What would help
Nothing urgent, and no API change is being requested. Two things would reduce the risk if the project is willing:
- Awareness that these five are load-bearing for an out-of-tree consumer, so a refactor can mention it in release notes.
- If any of them are considered stable in practice,
@InternalStableApi would say so explicitly. Http2Ext.bindAndHandleAsync and Http2Blueprint.handleWithStreamIdHeader already carry @noinline with comments about instrumentation, so the project clearly anticipates this kind of consumer.
Related OpenTelemetry work: open-telemetry/opentelemetry-java-instrumentation#19817 (http/2 server support).
This is informational rather than a bug report. The OpenTelemetry Java agent instruments pekko-http to create HTTP server and client spans and to propagate context, and it does so by attaching bytecode advice to specific methods. Most of those methods are internal to pekko-http, so a refactor that is invisible to users can silently stop the instrumentation from working. Filing this so the list is written down somewhere the pekko-http project can see it.
Why "silently": the agent's muzzle checks verify the classes and methods its advice code calls, and disable the instrumentation when they no longer exist. They do not verify the method matchers. If a matched method is renamed, inlined, or restructured, the advice simply never applies — no error, no log, just missing telemetry.
What is used
Public API, no concern, listed for completeness:
HttpExt.bindAndHandle,HttpExt.singleRequestHttp.IncomingConnection.handleWithDirective.tapply,PathMatcher.apply,PathMatcher$.applyhttp.routeInternal API:
HttpServerBluePrint.requestPreparation@InternalApi private[http] objectHttp2Ext.bindAndHandleAsync@InternalApi private[http] final classbindAndHandleHttp2Ext.systemHttp2.streamId@InternalApi private[http] objectPoolMasterActor.startPoolInterfaceprivate defin@InternalApi private[http] final classThe last one is the fragile case. Being a private method it has no stable name, and the agent matches the mangled
org$apache$pekko$http$impl$engine$client$PoolMasterActor$$startPoolInterface. That name exists only because a closure instartPoolInterfacecaptures it. If that closure is refactored away the method stops being mangled, or is inlined and disappears from the bytecode entirely.That is not hypothetical: while instrumenting pekko remoting we found
EndpointReader.dispatchMessagein the sources but not in the bytecode, because scala had inlined it.What would help
Nothing urgent, and no API change is being requested. Two things would reduce the risk if the project is willing:
@InternalStableApiwould say so explicitly.Http2Ext.bindAndHandleAsyncandHttp2Blueprint.handleWithStreamIdHeaderalready carry@noinlinewith comments about instrumentation, so the project clearly anticipates this kind of consumer.Related OpenTelemetry work: open-telemetry/opentelemetry-java-instrumentation#19817 (http/2 server support).