Skip to content

[METRICS SDK] Drop aggregation behaves as a no-op - #4515

Open
dbarker wants to merge 9 commits into
open-telemetry:mainfrom
dbarker:perf_metrics_sync_storage_drop_aggregation
Open

[METRICS SDK] Drop aggregation behaves as a no-op#4515
dbarker wants to merge 9 commits into
open-telemetry:mainfrom
dbarker:perf_metrics_sync_storage_drop_aggregation

Conversation

@dbarker

@dbarker dbarker commented Sep 1, 2026

Copy link
Copy Markdown
Member

Fixes #4514

When a view configures drop aggregation for an instrument (sync or async) the meter should not create a storage for that view.

When multiple views are configured for an instrument (a catch-all view and an instrument specific view), the SDK should detect semantic errors and prevent conflicting metrics.

Producers SHOULD prevent the presence of multiple Metric identities for a given name with the same Resource and Scope attributes. Producers are expected to aggregate data for identical Metric objects as a basic feature, so the appearance of multiple Metric, considered a “semantic error”, generally requires duplicate conflicting instrument registration to have occurred somewhere.

https://opentelemetry.io/docs/specs/otel/metrics/data-model/#opentelemetry-protocol-data-model-producer-recommendations

Changes

  • Check if a view configures drop aggregation before creating storage
  • Log a warning if the views will create duplicate conflicting metric storage objects.
  • Fix bugs that allowed the meter to add the same storage multiple times to a single instrument. This would cause unexpected results when multiple views match the same instrument/stream.
  • Update meter test to verify drop aggregation (including catch-all view cases) and general cleanup/enhancement.
  • Updates sync instruments benchmark results

For significant contributions please make sure you have completed the following items:

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.22034% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.45%. Comparing base (3dc106f) to head (ac90f8c).

Files with missing lines Patch % Lines
...dk/include/opentelemetry/sdk/metrics/instruments.h 75.00% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4515      +/-   ##
==========================================
- Coverage   83.46%   83.45%   -0.00%     
==========================================
  Files         521      521              
  Lines       20412    20467      +55     
==========================================
+ Hits        17034    17078      +44     
- Misses       3378     3389      +11     
Files with missing lines Coverage Δ
sdk/include/opentelemetry/sdk/metrics/meter.h 57.15% <ø> (ø)
...telemetry/sdk/metrics/state/multi_metric_storage.h 92.73% <100.00%> (+2.49%) ⬆️
sdk/src/metrics/meter.cc 82.82% <100.00%> (+1.46%) ⬆️
...dk/include/opentelemetry/sdk/metrics/instruments.h 75.61% <75.00%> (-0.14%) ⬇️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…antic error warning for view configurations that duplicate metrics. Update meter tests to verify drop aggregation and multiple views.
@dbarker
dbarker force-pushed the perf_metrics_sync_storage_drop_aggregation branch from e66422a to 8be812a Compare September 2, 2026 05:00
@dbarker dbarker changed the title [METRICS SDK] Metrics sync instrument drop aggregation behaves as a no-op [METRICS SDK] Drop aggregation behaves as a no-op Sep 2, 2026
@dbarker
dbarker marked this pull request as ready for review September 2, 2026 11:58
@dbarker
dbarker requested a review from a team as a code owner September 2, 2026 11:58
// ENABLE_METRICS_EXEMPLAR_PREVIEW = ON
// OPENTELEMETRY_HAVE_METRICS_BOUND_INSTRUMENTS_PREVIEW = OFF
// -------------------------------------------------------------------------------------------------------
// ~/build/sdk/test/metrics/sync_instruments_benchmark

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop aggregation cases with ABIv1 (bound instruments preview OFF). Instruments with drop aggregation are now within ~2ns of the fully disabled meter case and latency no longer scales with the number of threads recording with the instrument.

Benchmark Threads main (ns) this PR (ns) Speedup
BM_Record_Counter_Drop_ByThreads 1 181 2.29 ~79x
BM_Record_Counter_Drop_ByThreads 2 336 2.29 ~147x
BM_Record_Counter_Drop_ByThreads 4 548 2.35 ~233x
BM_Record_Histogram_Drop_ByThreads 1 180 2.50 ~72x
BM_Record_Histogram_Drop_ByThreads 2 198 2.52 ~79x
BM_Record_Histogram_Drop_ByThreads 4 556 2.56 ~217x

// -------------------------------------------------------------------------------------------------------
// ~/build/sdk/test/metrics/sync_instruments_benchmark
// 2026-08-22T19:58:12+00:00
// 2026-09-02T04:38:03+00:00

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ABIv2 (bound instruments preview ON)

Benchmark Threads main (ns) this PR (ns) Speedup
BM_Record_Counter_Drop_ByThreads 1 291 1.75 ~166x
BM_Record_Counter_Drop_ByThreads 2 545 1.77 ~308x
BM_Record_Counter_Drop_ByThreads 4 1222 1.80 ~679x
BM_Record_Histogram_Drop_ByThreads 1 288 2.12 ~136x
BM_Record_Histogram_Drop_ByThreads 2 540 2.16 ~250x
BM_Record_Histogram_Drop_ByThreads 4 1423 2.16 ~659x
BM_Record_Gauge_Drop_ByThreads 1 286 1.77 ~162x
BM_Record_Gauge_Drop_ByThreads 2 546 1.76 ~310x
BM_Record_Gauge_Drop_ByThreads 4 1200 1.88 ~638x
BM_Record_BoundCounter_Drop_ByThreads 1 5.81 1.14 ~5.1x
BM_Record_BoundCounter_Drop_ByThreads 2 34.8 1.16 ~30x
BM_Record_BoundCounter_Drop_ByThreads 4 123 1.20 ~103x
BM_Record_BoundHistogram_Drop_ByThreads 1 6.19 1.76 ~3.5x
BM_Record_BoundHistogram_Drop_ByThreads 2 32.6 1.76 ~19x
BM_Record_BoundHistogram_Drop_ByThreads 4 155 1.79 ~87x

Comment thread sdk/test/metrics/meter_test.cc Outdated

auto &point_data1 = point_data_attr[0].point_data;
auto &point_data2 = point_data_attr[1].point_data;
TEST_F(MeterCreateInstrumentTest, SyncInstrumentWithConflictingAggregation)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this PR, this case would result in duplicate counts due to duplicate storages added to the multi storage object of the instrument. When this happened, each value passed to counter->Add would be added N times, where N is the number of non-drop views corresponding to a single metric. This bug was fixed by checking if the storage configured by a view already exists in the Multi{Sync,Async}Storage before adding it again.

AddView("counter_one", "", InstrumentType::kCounter, "", AggregationType::kSum);

// Catch-all view
AddView("*", "", InstrumentType::kCounter, "", AggregationType::kSum);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: both views use Sum, so this test verifies deduplication but cannot distinguish aggregation selection. Please keep this case and add same-name Histogram/Sum cases in both registration orders, asserting the exported point-data type and the ignored aggregation in the warning.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback. I updated the tests to include a case for duplicate aggregation types and one for conflicting aggregation types (in which case the first view wins). The tests now verify the collected point data sums by aggregation type and fail when unexpected types of point data sums are collected.

Comment thread sdk/src/metrics/meter.cc Outdated
storage_registry_.insert({view_instr_desc, sync_storage});
}
auto sync_multi_storage = static_cast<SyncMultiMetricStorage *>(storages.get());
if (sync_multi_storage->HasStorage(sync_storage))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: with [A], inserting B scans in Meter and again in AddStorage(); a duplicate A returns after the first check. Let AddStorage() return insertion success, use that result for the warning, and keep HasStorage() private. This is registration-time cleanup, not a recording-path bug.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I updated the multi storage classes to return a bool from AddStorage and made the HasStorage methods private.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[METRICS SDK] Synchronous instruments with drop aggregation must behave as no-op

2 participants