Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/otel-conformance-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,13 @@ jobs:
actions: write
contents: read
id-token: write
uses: aws/aws-durable-execution-conformance-tests/.github/workflows/opentelemetry-orchestrator.yml@f9998f305f1e26423baf0c58148a3bd69120d5ef
uses: aws/aws-durable-execution-conformance-tests/.github/workflows/opentelemetry-orchestrator.yml@02d6dca971a38c13d94d6233d12f687e55b2a572
with:
language: java
resource_prefix: j
sdk_repository: aws/aws-durable-execution-sdk-java
sdk_ref: ${{ github.event.pull_request.head.sha || github.sha }}
conformance_test_ref: ${{ inputs.conformance_test_ref || '91740c98b496409fa9f1bb8e8e6c329ca8b0185f' }}
conformance_test_ref: ${{ inputs.conformance_test_ref || '02d6dca971a38c13d94d6233d12f687e55b2a572' }}
checkout_sdk: true
# Build the handlers from this repo's checked-out module instead of the conformance repo's
# bundled examples/java. Path is relative to the conformance workspace where the SDK is
Expand Down
60 changes: 39 additions & 21 deletions otel-plugin/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# AWS Durable Execution SDK - OpenTelemetry Plugin

OpenTelemetry instrumentation plugin for the AWS Lambda Durable Execution SDK for Java. Emits a deterministic Workflow trace for durable-execution correlation while keeping each Invocation span in the ambient Lambda trace.
OpenTelemetry instrumentation plugin for the AWS Lambda Durable Execution SDK for Java. Anchors every durable execution on one trace so the Workflow span and its per-invocation spans stay correlated, joining the propagated backend trace when one is present.

## Features

- **Deterministic Workflow Traces**: Workflow trace IDs are derived from the execution start time and ARN; stable span IDs are derived from the ARN
- **Ambient Invocation Traces**: Invocation spans inherit the active Lambda/X-Ray context, or receive a fresh provider-generated root trace ID
- **Backend-parented execution trace**: The Workflow span parents onto the execution ancestor resolved at invocation start — a propagated remote context, or a synthetic execution root — for one trace ID that is stable across all invocations, plus a stable span ID derived from the ARN
- **Ambient Invocation Traces**: Invocation spans inherit the active Lambda/X-Ray context, or join the execution ancestor so they stay on the execution trace
- **Scoped ID Generation**: Unrelated instrumentation scopes retain their provider's normal root trace ID generation
- **Span-per-Operation**: Each durable operation (step, wait, map, etc.) gets its own span with accurate timing
- **Attempt Spans**: Each user function execution (step attempt, child context run) gets a span, including retries
Expand Down Expand Up @@ -92,7 +92,7 @@ Build the plugin layer ZIP with the OTel plugin JAR at `java/lib/aws-durable-exe

### 2. AWS X-Ray Active Tracing

Enable active tracing on your Lambda function so the `_X_AMZN_TRACE_ID` environment variable is populated at invocation time. The plugin uses this header to parent Invocation spans to the ambient Lambda/X-Ray trace. The Workflow trace remains independent and deterministic.
Enable active tracing on your Lambda function so the `_X_AMZN_TRACE_ID` environment variable is populated at invocation time. The plugin uses this header both to parent Invocation spans to the ambient Lambda/X-Ray trace and to anchor the execution trace on the propagated context when it carries a complete parent and an explicit sampling decision.

**AWS Console:** Lambda > Configuration > Monitoring and operations tools > Active tracing > Enable

Expand Down Expand Up @@ -157,29 +157,47 @@ The function's execution role needs the `AWSXRayDaemonWriteAccess` managed polic

## Trace Structure

With `InvocationOtelPlugin`, the plugin creates two correlated traces:
The whole execution shares one trace, anchored at the execution ancestor resolved at invocation start. When the backend propagates a valid remote server span (`Root` and `Parent`), that span is the ancestor and the Workflow and Invocation spans nest under it, alongside the ambient Lambda spans on the same trace:

```
Workflow trace:
Workflow (deterministic trace/span IDs, exported once)

Ambient invocation trace:
Lambda/X-Ray parent
└── Invocation
├── fetch-data
│ └── fetch-data attempt 1
├── cool-down
└── process
└── process attempt 1
Remote backend server span (Root / Parent)
├── Workflow (stable span ID, exported once)
├── Ambient Lambda span 1
│ └── Invocation 1
├── Ambient Lambda span 2
│ └── Invocation 2
└── Invocation N (direct child when no same-trace ambient span exists)
```

- **Workflow span** — one logical root per durable execution with a deterministic, X-Ray-compatible trace ID derived from the execution start time and ARN, plus a stable span ID derived from the ARN. Exported only on the terminal invocation (SUCCEEDED/FAILED).
- **Invocation span** — one per Lambda invocation, parented to ambient context when available
When no valid remote parent can be constructed, a synthetic execution root anchors the trace instead and both spans parent onto it:

```
Synthetic execution root
├── Workflow
├── Invocation 1
├── Invocation 2
└── Invocation N
```

- **Execution ancestor** — the common parent both the Workflow and Invocation spans resolve onto. A valid remote server span (`Root` and `Parent`) is used directly, whether or not `Sampled` is present; only when a valid remote parent cannot be constructed does a synthetic execution root take its place. It is a non-recording context, not an exported span.
- **Workflow span** — one logical span per durable execution, joining the execution trace with a stable span ID derived from the ARN. Exported only on the terminal invocation (SUCCEEDED/FAILED).
- **Invocation span** — one per Lambda invocation, parented to the ambient span only when it is on the execution trace, otherwise to the execution ancestor
- **Operation span** — one per durable operation, named after your step/wait names
- **Attempt span** — one per user function execution (retries produce additional attempt spans)

Operation and attempt spans link to the Workflow span. `ExecutionOtelPlugin` reverses that relationship: operations are children of Workflow and link to the current Invocation span.

### Sampling

The plugin decides sampling once per invocation and applies that single decision to every durable span (Workflow, Invocation, operation, attempt), so the configured sampler is not re-invoked per span and the full decision — including `RECORD_ONLY` — is preserved. The decision follows this precedence, highest first:

1. **Backend decision** — `Sampled=1` / `Sampled=0` in the propagated header is authoritative and always preserved, regardless of the configured sampler.
2. **Same-trace ambient span** — when the header carries no usable `Sampled` value but a valid ambient span (for example an auto-instrumentation Lambda handler span) is already on the execution's trace, the plugin follows that span's decision: sampled → sampled; unsampled but still recording → `RECORD_ONLY`; unsampled and not recording → dropped.
3. **Configured sampler (application-owned provider)** — when you pass a `SdkTracerProvider` to the plugin, its sampler is read directly and evaluated once with the trace ID, span name, and attributes. A trace-ID-ratio sampler therefore produces a stable decision across reinvocations (the trace ID is stable).
4. **Installed sampler (Java-agent path)** — when the agent owns the provider, it is behind a classloader boundary and its *effective* sampler (which another agent extension may have wrapped or replaced) cannot be reliably read at decision time. Rather than guess, the plugin **defers**: it installs a delegating sampler through the agent's autoconfiguration and lets that wrapper consult the agent's real sampler. The delegate's decision is honored in full — if your configured policy is `always_off`, a rate limiter, or a remote sampler (`xray`, `jaeger_remote`) that returns drop, the durable spans are dropped; they are **not** force-sampled. To avoid consuming a stateful or quota-based sampler once per span, the wrapper consults the delegate once per execution (keyed by trace ID) and reuses that decision for the execution's remaining durable spans within the invocation.

For precise, provider-independent control, set an explicit `Sampled` value upstream (for example by enabling X-Ray active tracing) — that backend decision takes precedence over everything else.

## Span Attributes

### Invocation Span
Expand Down Expand Up @@ -305,7 +323,7 @@ The plugin's spans do not appear as nested subsegments of the Lambda platform se

### Workflow Span

The Workflow span appears in a separate deterministic trace because it uses `setNoParent()`. Invocation spans remain in the ambient Lambda/X-Ray trace. Links correlate durable operations with the other trace.
The Workflow span joins the execution trace by parenting onto the execution ancestor: the propagated remote server span when one is valid, otherwise a synthetic execution root. Either way it shares the execution trace ID and keeps its stable, ARN-derived span ID.

## Verification

Expand All @@ -314,8 +332,8 @@ After deploying your function with the plugin configured:
1. **Invoke your durable function** — trigger at least one execution that includes multiple steps or a wait/resume cycle.

2. **Check CloudWatch console** — Navigate to CloudWatch > Traces. Enable "Group by nodes" to see:
- A deterministic Workflow trace covering the entire execution
- Ambient Lambda traces containing one Invocation span per Lambda invocation
- One execution trace covering the whole execution, with the Workflow span and each Invocation span sharing its trace ID
- One Invocation span per Lambda invocation
- Child spans for each durable operation (named after your step names)
- Links between durable Workflow/operation spans and Invocation spans

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,33 @@
package software.amazon.lambda.durable.otel;

/**
* Extracts trace context from the Lambda runtime environment.
* Extracts the durable execution's propagated trace context from the Lambda runtime environment.
*
* <p>Implementations read trace context from various sources (X-Ray trace header, W3C traceparent, etc.) and return an
* {@link ExtractedContext} containing the trace ID and optional parent span ID.
*
* <p>Plugins use a valid ambient OpenTelemetry span as the invocation parent when one is available. This extractor is
* consulted only when no ambient span context is available, providing fallback propagation context from the runtime
* environment.
* <p><strong>When it is called:</strong> the plugin invokes {@link #extract()} once at the start of every invocation,
* unconditionally — including when an ambient OpenTelemetry span is active. The extracted context is the durable
* execution's identity and is resolved with the following precedence:
*
* <ol>
* <li>a valid extracted backend context anchors the execution trace (this is what makes the durable spans share one
* stable trace across reinvocations, so it takes precedence over the per-invocation ambient span);
* <li>otherwise the execution is anchored on a deterministic synthetic root derived from the execution ARN;
* <li>the ambient span is never adopted as the execution trace. It is used only to parent the Invocation span when it
* is already on the resolved trace, and otherwise correlated with a span link.
* </ol>
*
* <p><strong>Implementation contract:</strong> because {@code extract()} runs on every invocation, implementations must
* be side-effect-free (or idempotent) and cheap, and must return the durable execution's own context — returning stale
* or unrelated context will displace the correct execution trace.
*/
@FunctionalInterface
public interface ContextExtractor {

/**
* Extracts fallback trace context from the runtime environment.
* Extracts the durable execution's propagated trace context from the runtime environment. Called once per
* invocation, unconditionally.
*
* @return the extracted context, or {@code null} if no context is available
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,22 @@ String generateTraceIdForExecution(String arn, Instant executionStartTime) {
}

String generateWorkflowSpanId(String arn) {
var seed = "workflow:" + (arn != null ? arn : "");
return deterministicSpanId("workflow:" + (arn != null ? arn : ""));
}

/**
* Generates the deterministic span ID for the synthetic execution root from the execution ARN, using a seed
* namespace distinct from the Workflow and operation span IDs. Stable across reinvocations so the synthetic root is
* the same common ancestor every invocation.
*
* @param arn the durable execution ARN
* @return a deterministic 16-char hex span ID
*/
String generateExecutionRootSpanId(String arn) {
return deterministicSpanId("execution-root:" + (arn != null ? arn : ""));
}

private static String deterministicSpanId(String seed) {
var spanId = sha256(seed).substring(0, 16);
if (spanId.equals("0000000000000000")) {
spanId = "0000000000000001";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.lambda.durable.otel;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.context.Context;
import io.opentelemetry.sdk.trace.SdkTracerProviderBuilder;
import io.opentelemetry.sdk.trace.data.LinkData;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import io.opentelemetry.sdk.trace.samplers.SamplingResult;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

/**
* A delegating sampler that applies the durable execution's precomputed decision to durable spans and leaves every
* other span to the wrapped sampler.
*
* <p>The durable plugins decide sampling for the whole execution exactly once per invocation and attach the resulting
* {@link SamplingResult} to the {@link Context} used as the parent of each durable span (via
* {@link DurableSamplingDecision}). When {@code shouldSample} sees that decision on the parent context it returns it
* verbatim, so:
*
* <ul>
* <li>the wrapped (customer-configured or ADOT/community default) sampler is invoked at most once per invocation for
* durable spans, which is safe for stateful or quota-based samplers that would otherwise be consumed multiple
* times; and
* <li>the full decision is preserved, including {@code RECORD_ONLY}, rather than being reduced to a sampled/dropped
* bit and re-derived at span creation.
* </ul>
*
* <p>Spans without the durable decision on their parent context (ordinary application or auto-instrumentation spans)
* are delegated to the wrapped sampler unchanged, so the customer's sampling configuration governs everything outside
* the durable execution's own spans.
*/
final class DurableSampler implements Sampler {

// Cap on the deferred-decision cache. The sampler is long-lived (installed once), so an unbounded map would grow
// per execution on a warm container. A modest LRU cap bounds memory while keeping the "consult once per execution"
// guarantee for the handful of executions active on one Lambda instance; an evicted entry at worst re-consults the
// delegate for a later span of that execution, which is a rare, benign degradation rather than a correctness bug.
private static final int MAX_CACHED_DEFERRED_DECISIONS = 256;

private final Sampler delegate;
// Caches the delegate's decision for a deferred durable execution, keyed by canonical trace ID, so a stateful or
// quota-based delegate is consulted once per execution rather than per span. Access-ordered LRU, size-capped and
// synchronized (contention is low: at most one miss per execution).
private final Map<String, SamplingResult> deferredDecisions =
Collections.synchronizedMap(new LinkedHashMap<>(16, 0.75f, true) {
@Override
protected boolean removeEldestEntry(Map.Entry<String, SamplingResult> eldest) {
return size() > MAX_CACHED_DEFERRED_DECISIONS;
}
});

private DurableSampler(Sampler delegate) {
this.delegate = delegate;
}

/**
* Wraps {@code delegate} so durable spans use the precomputed decision. A null delegate or an already-wrapped
* delegate is handled defensively: wrapping is idempotent, and a null delegate falls back to
* {@link Sampler#parentBased(Sampler)} of {@link Sampler#alwaysOn()} (the OTel/ADOT default) so ordinary spans
* still have a sampler.
*/
static DurableSampler wrap(Sampler delegate) {
if (delegate instanceof DurableSampler durableSampler) {
return durableSampler;
}
var effectiveDelegate = delegate != null ? delegate : Sampler.parentBased(Sampler.alwaysOn());
return new DurableSampler(effectiveDelegate);
}

/**
* Installs the durable sampler on an application-owned {@link SdkTracerProviderBuilder} by wrapping its configured
* sampler. The builder exposes only a setter, so the configured sampler is read reflectively (mirroring how the
* deterministic ID generator wraps the builder's ID generator) and replaced with a wrapper that delegates to it.
* Installation is idempotent: a builder whose sampler is already a {@link DurableSampler} is left unchanged.
*/
static void installOn(SdkTracerProviderBuilder builder) {
var configured = configuredSampler(builder);
if (configured instanceof DurableSampler) {
return;
}
builder.setSampler(wrap(configured));
}

private static Sampler configuredSampler(SdkTracerProviderBuilder builder) {
for (var field : builder.getClass().getDeclaredFields()) {
if (java.lang.reflect.Modifier.isStatic(field.getModifiers())
|| !Sampler.class.isAssignableFrom(field.getType())) {
continue;
}
try {
if (!field.trySetAccessible()) {
break;
}
return (Sampler) field.get(builder);
} catch (IllegalAccessException e) {
throw new IllegalStateException("Unable to read the configured OpenTelemetry sampler", e);
}
}
throw new IllegalStateException("Unable to locate the configured OpenTelemetry sampler");
}

@Override
public SamplingResult shouldSample(
Context parentContext,
String traceId,
String name,
SpanKind spanKind,
Attributes attributes,
List<LinkData> parentLinks) {
var intent = DurableSamplingDecision.get(parentContext);
if (intent == null) {
// Not a durable span: the customer's sampler governs it unchanged.
return delegate.shouldSample(parentContext, traceId, name, spanKind, attributes, parentLinks);
}
if (!intent.isDeferred()) {
// A resolved decision (explicit upstream, same-trace ambient, or a locally reproduced sampler): return it
// verbatim, preserving RECORD_ONLY and never re-invoking the delegate.
return intent.resolved();
}
// Deferred (agent path, real sampler not reproducible here): evaluate the actual delegate once per execution
// and reuse it, so an installed drop/rate-limit policy is honored and consulted only once.
return deferredDecisions.computeIfAbsent(
intent.deferredTraceId(),
key -> delegate.shouldSample(Context.root(), key, name, spanKind, attributes, Collections.emptyList()));
Comment thread
ayushiahjolia marked this conversation as resolved.
}

@Override
public String getDescription() {
return "DurableSampler{" + delegate.getDescription() + "}";
}
}
Loading
Loading