Categorize cluster (de)serialization metrics and drop per-op allocation in metric wrappers

[thjaeckle]

What & why

Two related improvements around cluster (de)serialization metrics: better insight into where serialization traffic comes from, and a lower-cost, allocation-free metric increment path.

1. Categorize cluster (de)serialization metrics

The <serializer>_serializer_messages counter (in AbstractJsonifiableWithDittoHeadersSerializer) currently only carries a direction (in/out) tag. That makes it hard to explain spikes in cluster CBOR/JSON serialization: a doubling of the counter could equally be event fan-out to more subscribers, a burst of inbound commands, a wave of policy sudo fetches, or response traffic — all indistinguishable.

This adds two low-cardinality tags derived at the (de)serialization site:

• category — coarse signal kind: event, command, response, acknowledgement, announcement, error, other (via instanceof, ordered so Acknowledgement is classified before CommandResponse).
• resource_type — from WithResource#getResourceType() (thing, policy, connectivity, message, thing-search, and the *-sudo variants…), or other for non-signal envelopes.

Together these let operators attribute spikes directly — e.g. category=event,resource_type=thing for live/streaming fan-out vs category=command,resource_type=thing for write load, and resource_type=policy-sudo/thing-sudo to spot cross-node enforcement policy fetches. direction is unchanged, so existing dashboards keep working; the new tags are additive.

Cardinality stays modest: ~7 categories × ~18 resource types × 2 directions per serializer, and most combinations never co-occur. The fully-tagged counters are cached per (category, resource_type) so the hot path is a map lookup + increment.

2. Memoize the resolved Kamon instrument in the metric wrappers

KamonCounter, KamonGauge, KamonHistogram and PreparedKamonTimer re-resolved their underlying Kamon instrument on every operation:

Kamon.counter(name).withTags(KamonTagSetConverter.getKamonTagSet(tags)).increment();

That is a metric-registry lookup plus a per-call allocation of a Kamon TagSet (the getKamonTagSet builder) on every increment/record — even though (name, tags) is invariant for a given wrapper. This is now memoized in a volatile field, so the hot path becomes a volatile read + the actual op, with no per-op allocation.

Counter/Gauge/Histogram have immutable tags (copy-on-write tag()/tags()), so a plain lazy memo is safe. PreparedKamonTimer mutates its tags in place, so the cache is invalidated in tag()/tags(). StoppedKamonTimer is single-use per stop with dynamic segment tags and is intentionally left unchanged.

This benefits every metric in Ditto, and in particular the now-per-message serializer counter on the cluster (de)serialization hot path.

Micro-benchmark (JMH, AverageTime, JDK, 1 fork · 3 warmup · 5 measurement iterations)

Benchmark	ns/op	vs. old
re-resolve per increment (old)	83.8 ± 1.9	1.0×
re-resolve, tags converted once	33.3 ± 0.8	2.5×
memoized wrapper (this PR)	7.9 ± 1.1	10.6×
pre-resolved instrument (theoretical floor)	7.9 ± 2.1	10.6×

The memoized path matches the theoretical floor (the volatile read is free), and is ~10.6× faster than re-resolving. Of the ~84 ns saved, roughly ~50 ns was the per-call TagSet allocation/conversion and ~25 ns the registry withTags lookup. The allocation removal also reduces GC pressure on the hot path.

Compatibility

• Existing metric names and the direction tag are unchanged; the category/resource_type tags are additive.
• No public API changes; the Kamon memoization is an internal implementation detail of the metric wrappers.

[thjaeckle]

System tests passed: https://github.com/eclipse-ditto/ditto/actions/runs/29031970314