feat(producer,cli): full telemetry visibility for DE parallel-router/inversion failures#2115
Open
vanceingalls wants to merge 1 commit into
Open
feat(producer,cli): full telemetry visibility for DE parallel-router/inversion failures#2115vanceingalls wants to merge 1 commit into
vanceingalls wants to merge 1 commit into
Conversation
…inversion failures render_error previously carried zero DE-cohort context — a hard failure while routed (worker crash, OOM, capture timeout from the fixed 3-worker pin overriding calibration) was indistinguishable from any other failure. The data existed (RenderCaptureObservability is mutated live and survives into job.errorDetails on the failure path) but was never projected into the render_error payload, which only ever drew de_* fields from perfSummary (success-only). - RenderCaptureObservability now also records dePreInversionWorkers / dePreRouterWorkers — the worker count calibration would have picked absent the experiment — so a resource-pressure failure can be correlated with the router overriding a lower calibrated count. - New capture-sourced de_* fields on RenderObservabilityTelemetryPayload, shared by trackRenderComplete and trackRenderError. Explicit perfSummary-sourced fields still win on render_complete (spread moved first in the event object) — this is purely a failure-path fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2095. Closes a gap found while assessing default-flip readiness:
render_errorcarried zero DE-cohort context, so a hard failure while routed (worker crash, OOM, capture timeout from the router's fixed 3-worker pin overriding calibration) was indistinguishable from any unrelated failure in PostHog.RenderCaptureObservabilitynow recordsdePreInversionWorkers/dePreRouterWorkers— the worker count calibration would have picked absent the experiment — set live at the same pointdeWorkerInversion/deParallelRouterare set, so it survives intojob.errorDetailson a hard failure, not just into the success-pathperfSummary.de_*fields to the sharedRenderObservabilityTelemetryPayload, consumed by bothtrackRenderCompleteandtrackRenderError. Onrender_complete, the existing explicit perfSummary-sourced fields still win (moved the shared-payload spread to the front of the event object) — this addition is purely a failure-path fallback, no change to existing success-path values.de_pre_router_workers(was calibration overridden from a lower count?) and withde_self_verify_fallback/de_worker_count/de_memory_exhaustion_detected(already-existing fields, now reachable on failure too via the same fallback path).Test plan
bunx vitest runonrenderOrchestrator.test.ts,observability.test.ts,events.test.ts,renderObservability.test.ts— 150/150 passrender_errorcarriesde_parallel_router/de_pre_router_workers/capture_worker_count/capture_memory_exhaustion_detectedon a hard failurede_worker_inversiononrender_completewins over a divergent capture-observability fallback valuerenderObservabilityTelemetryPayloadprojects router/inversion cohort + pre-worker-count + self-verify-fallback correctlytsc --noEmit,oxlint,oxfmtclean on all touched files🤖 Generated with Claude Code