feat(telemetry): align spans and metrics with OTel GenAI semantic conventions

[ajbozarth]

Pull Request

Issue

Fixes #1209

Description

Aligns Mellea's tracing and metrics with the OTel GenAI semantic conventions and namespaces the remaining flat span attributes. Part of the tracing epic #444.

Breaking: nearly every span attribute and client metric is renamed or restructured. This is a hard break — old names are removed outright, with no dual-emit and no deprecation window. Any dashboard, alert, or query built on the previous mellea.* metric names or span attributes will stop matching and must be updated to the new names below. The public record_* helpers also gain a required operation arg, so direct callers must update their calls.

Spans:

• Backend span name is now {operation} {model}; model/provider surfaced on the chat pre-call payload for parity with the batch path
• Backend spans use SpanKind.CLIENT for remote providers (INTERNAL for in-process backends such as huggingface)
• Emit gen_ai.request.stream (replaces the mellea.streaming bool)
• Map watsonx to ibm.watsonx.ai for gen_ai.provider.name at emit sites
• gen_ai.output.type now "json" (was the invalid "json_schema")
• gen_ai.usage.total_tokens → mellea.usage.total_tokens (not a spec attr)
• Drop the redundant mellea.has_format (superseded by gen_ai.output.type)
• Add gen_ai.response.time_to_first_chunk to streaming spans

Metrics:

• Token counters → single gen_ai.client.token.usage histogram, split by gen_ai.token.type, with spec bucket boundaries
• mellea.llm.request.duration → gen_ai.client.operation.duration and mellea.llm.ttfb → gen_ai.client.operation.time_to_first_chunk, with spec buckets
• Add the required gen_ai.operation.name dimension to the backend metrics
• Record gen_ai.client.operation.duration on failed operations too, tagged with error.type
• streaming dimension → gen_ai.request.stream (matches the span attribute)
• Error counter's non-standard error_type key → mellea.error.category
• Add the gen_ai.client.operation.time_per_output_chunk histogram for streaming (opt-in via MELLEA_GENERATION_CHUNK_EVENTS)

Span attributes:

• Namespace every remaining flat mellea.* span attribute by its owning subsystem (mellea.<scope>.<leaf>), matching the convention already used for event attributes and metrics
• Promote the session's model and provider to gen_ai.request.model / gen_ai.provider.name

Not included: the server.address / server.port span and metric items from the checklist — the real endpoint isn't reliably available without per-backend SDK introspection, disproportionate for Recommended-only attributes.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

• Component
• Requirement
• Sampling Strategy
• Tool

[ajbozarth]

@ reviewers if you could prioritize this, I'd like to see if I can get it merged before I leave on my trip and am out for a week (ie by EOD tomorrow, Tuesday) and want to make sure I have time to address any feedback

[planetf1]

Client latency metrics are not yet safe to use for application-performance diagnosis; see inline.

[planetf1]

gen_ai.client.operation.duration is intended to be Mellea's client-observed model-call latency, but this timestamp is taken only after stream consumption, post-processing, parsing, and validation. TTFB is likewise recorded after queue.get*() and chunk intervals after _gen.process() (mellea/core/base.py:1133-1153, 1189-1207). The producer puts raw chunks into the queue without a receipt timestamp (mellea/helpers/async_helpers.py:79-125), so a slow renderer, paused agent loop, or validation step becomes apparent provider latency.

Please timestamp chunk receipt immediately after anext() returns (and completion on the producer side) using a monotonic clock, propagate those timestamps through the queue, and derive TTFB, inter-chunk timing, and client duration from them. Consumer processing and strategy time already have Mellea orchestration spans. Please add a regression test that delays the consumer after a chunk is produced and verifies these client metrics do not increase.

[ajbozarth]

I'm digging into this more (I had looked into it myself before opening the PR), but I see a few options on address this:

1. I rip out the entire feature, span attributed and metric histogram and we just don't emit this value
2. I rename it so it doesn't claim to be a gen_ai standard value
3. I plumb the correct value to the metric. I had Claude do an initial attempt at this just now and it requires changes to 8 files. I'm still investigating, but I'm not convinced this metric is worth all that new plumbing.

I will note that if we go with option 2 we have the option to do option 3 in a follow up PR if desired. Though technically option 1 is the most correct

[ajbozarth]

I have also now realized that there are separate issues here:

1. the existing span attr that I renamed in this PR (this diff line)
2. the new metric I added by the same name.

I'll continue to dig into these, but would you be open to me filing a follow up PR to fix the values rather than blocking this entire PR on poor plumbing of an existing value for one item?

[ajbozarth]

Here's the proposed followup issue I'd open if you're ok with a followup unblocking this. I would pick up work on it once I get back from my trip next week. Full draft folded below.

Proposed follow-up issue — Client latency telemetry (TTFC / duration / per-output-chunk) is measured consumer-side, not at provider receipt

Context

Split out of #1551 (align spans and metrics with OTel GenAI semantic conventions) to keep that PR scoped to the conventions rename, with the measurement corrections tracked here. Raised in review by @planetf1: the client latency instruments are timestamped on the consumer side of the streaming pipeline, so a slow renderer, paused agent loop, or client-side post-processing inflates what is reported as provider latency.

Root cause

Mellea streams through a producer→queue→consumer pipeline: backends push chunks into an asyncio.Queue from send_to_queue() (mellea/helpers/async_helpers.py) while ModelOutputThunk.astream() (mellea/core/base.py) drains it. All three latency instruments take their timestamps in the consumer — after the chunk is dequeued, and for some after it is processed — so they measure when the application handled the data, not when the provider delivered it.

Two structural problems compound, and they have different fix locations:

• (A) Measurement location — timestamps are taken consumer-side instead of at chunk receipt in the producer. Straightforward to move.
• (B) Bounded-queue backpressure — the queue is asyncio.Queue(maxsize=20); once full, the producer's put() blocks, which in turn paces its own anext(). So even a producer-side timestamp is dragged by a stalled consumer on any stream longer than the queue. Removing this needs decoupling the producer's drain from the bounded queue (buffer receipts independently, or use provider-reported timing).

Fixes required

The three are related — all "measure provider time, not consumer time," all enabled by producer-side receipt timestamps — but land in separate places with different scope.

1. time_to_first_chunk — fully fixable (location only, Issue A)

Surfaces: gen_ai.client.operation.time_to_first_chunk (metric) and gen_ai.response.time_to_first_chunk (span attr), both derived from ModelOutputThunk.generation.ttfb_ms.

ttfb_ms is recorded in _record_ttfb() at queue.get() time, so it includes the gap between request dispatch and the consumer draining the first chunk. The spec definition is explicit:

Time to receive the first chunk, measured from when the client issues the generation request to when the first chunk is received in the response stream.

Consumer-side stamping violates that definition. Fix location: send_to_queue() — stamp a monotonic receipt time the instant the first anext() returns (before enqueue) and derive ttfb_ms as first_receipt − request_dispatch. The queue is empty at the first chunk so put() never blocks — this is immune to Issue B and is completely solvable.

2. operation.duration — location fix + backpressure decoupling (Issues A + B)

Surface: gen_ai.client.operation.duration (metric).

latency_ms is _elapsed_ms() read at the generation_post_call hook — after full consumption, post-processing, parsing, and validation — and because that hook fires whenever the consumer finishes, it also absorbs idle time (e.g. a paused agent loop between astream() calls). Note the spec only defines this as "GenAI operation duration" (client perspective) and does not pin the boundary, so including post-processing is arguably within latitude; including arbitrary idle time is not defensible under any reading.

Fix location:

• (a) send_to_queue() — stamp completion at last-chunk receipt; removes post-processing/parse/validate. Same location as Better @generative documentation in the tutorial #1.
• (b) Decouple the producer drain from the bounded queue to remove the Issue B backpressure contamination on long streams.

Resolution options: (i) keep the gen_ai.* spec name and implement (a) + (b); or (ii) rename to an honest mellea metric instead of re-measuring — viable for duration because the current value is a coherent quantity (end-to-end wall time) that only carries the wrong name, so an honest name makes it correct-by-definition and (b) is not needed.

3. time_per_output_chunk / time_since_last_chunk_ms — location + per-item propagation + backpressure (Issues A + B, plus queue-payload rework)

Surfaces: gen_ai.client.operation.time_per_output_chunk (metric) and time_since_last_chunk_ms on chunk_processed span events (opt-in via MELLEA_GENERATION_CHUNK_EVENTS).

time_since_last_chunk_ms is computed in the consumer after await self._gen.process(self, chunk), from datetime.now() deltas at processing time; the consumer also drains in batches, so the deltas reflect processing/batching cadence, not provider inter-arrival.

Fix location:

• (a) Producer captures per-chunk receipt timestamps (send_to_queue()).
• (b) Propagate them per-item through the queue so the consumer can emit them alongside chunk_index/chunk_text_length. The queue currently carries bare items and discriminates end-of-stream/error by identity (is None / isinstance Exception), so this needs a wrapper type or a parallel timestamp channel — the most invasive piece.
• (c) Subject to the same Issue B backpressure ceiling as Easier RCA for failed requirements #2.

Most involved of the three.

Note: these surfaces are new and opt-in (MELLEA_GENERATION_CHUNK_EVENTS, no existing consumers), so an alternative to shipping approximate values is to gate them off until (a) + (b) + (c) land. Renaming does not help here (unlike duration): the current value is a batching-distorted consumer-processing cadence, not a coherent quantity, so an honest name yields a truthful-but-low-value metric rather than a correct one.

Acceptance

• TTFC, duration, and per-chunk timing are derived from provider-side receipt timestamps rather than consumer-side stamps.
• Regression test: stall the consumer after chunks are produced and assert the client latency values do not increase.
• The instruments distinguish provider latency from client-side/orchestration time, so they're usable for performance diagnosis (the original review concern).

[planetf1]

Thanks for this — the rest of the PR looks good to me.

One thing to check on the latency metrics: the OTel GenAI spec defines the operation duration as ending when the response is fully received, not after post-processing. Currently we stop the clock after the response has been consumed from the queue and validated (at base.py:1246, after stream consumption, post-processing, and parsing), so the values will include that overhead.

Moving the measurement point to chunk receipt in send_to_queue in a follow-up PR, or adding a note in the docs stating that the values are consumer-observed (including post-processing), would help clarify and improve the metric.

[ajbozarth]

Follow up open: #1566

[planetf1]

LGTM — good work on the semconv alignment. Follow-up on the latency measurement point noted in the inline comment.