Add OpenTelemetry-native host and process monitoring

[mike-realuptime]

Description

Resolves #14046.

This PR adds OpenTelemetry-native host and process monitoring for Linux and Windows while preserving the existing Prometheus exporter based monitoring paths.

The implementation keeps the final SkyWalking data model and metric aggregation in OAP/MAL. The OpenTelemetry Collector is responsible for collection, normalization of source attributes, transport, and local process cardinality control.

Host monitoring

The existing host MAL rules are extended instead of introducing parallel rules owning the same SkyWalking target metrics:

• vm.yaml supports both:

  • Prometheus node-exporter metrics
  • OpenTelemetry hostmetrics for Linux

• windows.yaml supports both:

  • Prometheus windows-exporter metrics
  • OpenTelemetry hostmetrics for Windows

Equivalent source metrics are mapped to the same canonical SkyWalking metrics only when their semantics match.

The existing node-exporter/windows-exporter monitoring paths remain supported.

Process monitoring

This PR adds:

• process-hostmetrics-linux.yaml
• process-hostmetrics-windows.yaml

Processes are modeled as logical SkyWalking instances using a normalized process name rather than PID.

This avoids PID-driven entity churn and keeps process cardinality bounded.

Multiple operating-system processes belonging to the same normalized process group are aggregated before being sent to OAP. SkyWalking then performs the final metric modeling through MAL.

Process metrics include:

• process count
• thread count
• CPU utilization
• resident memory usage
• resident memory utilization
• open handles / file descriptors
• oldest process uptime

OpenTelemetry Collector

The reference configuration targets OpenTelemetry Collector Contrib 0.158.0.

The Collector:

• uses the hostmetrics receiver
• detects host.name
• adds the SkyWalking job identity
• normalizes process names
• groups equivalent processes
• aggregates process metrics before OTLP export
• sends host and process metrics through OTLP to OAP

Backward compatibility

The existing node-exporter path remains intact.

The existing vm.yaml rule is extended rather than replaced, so existing Linux VM metrics continue using their current canonical SkyWalking metric names.

Node-exporter-specific metrics without an equivalent OpenTelemetry hostmetric remain sourced from node-exporter.

Validation

A new E2E case validates the OpenTelemetry hostmetrics path using OpenTelemetry Collector Contrib 0.158.0.

It verifies:

• Linux host service discovery
• memory metrics
• CPU metrics
• logical CPU count
• normalized CPU utilization
• process logical-instance discovery
• process count aggregation
• thread aggregation
• resident memory
• open handles
• oldest process uptime

The process test starts three operating-system processes with the same executable name and verifies that SkyWalking exposes one logical process instance with a process count of 3.

The E2E test passes with:

12 passed
0 failed
0 skipped

The existing node-exporter path was also regression-tested with the unified vm.yaml, including successful OS_LINUX service discovery and values for host memory, CPU, and node-exporter-specific tcp_alloc.

Scope

This PR focuses on OAP/MAL, Collector configuration, and E2E coverage.

Horizon UI dashboards for the new process metrics will be submitted separately to the SkyWalking Horizon UI repository.

[Copilot]

🟡 Changes recommended

Several YAML files have an incomplete Apache 2.0 license header (missing the leading “Licensed to the Apache Software Foundation (ASF)…” line).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds OpenTelemetry-native host and process monitoring support (Linux + Windows) by extending existing MAL rules to accept either exporter-based metrics or OTel hostmetrics, and introduces a new E2E case that validates the OTel hostmetrics + pre-aggregated process pipeline end-to-end.

Changes:

• Extend Linux vm.yaml and Windows windows.yaml MAL rules to map both exporter and OTel hostmetrics sources into the same canonical SkyWalking meter targets (avoiding rule ownership collisions).
• Add new process MAL rules (process-hostmetrics-{linux,windows}.yaml) modeling normalized process groups as logical SkyWalking instances.
• Add a new E2E case (vm/otel-hostmetrics) including collector config, compose wiring, and workflow integration.

File summaries

File	Description
test/e2e-v2/cases/vm/prometheus-node-exporter/otel-rules/vm.yaml	Extends Linux VM MAL to accept both node-exporter and OTel hostmetrics under vm-monitoring.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml	Adds Windows VM MAL rules for the OTel-hostmetrics E2E case.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/vm.yaml	Adds Linux VM MAL rules for the OTel-hostmetrics E2E case.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/process-hostmetrics-windows.yaml	Adds Windows process MAL rules mapping normalized process groups to instances.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/process-hostmetrics-linux.yaml	Adds Linux process MAL rules mapping normalized process groups to instances.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-collector-config.yaml	Adds an otelcol-contrib config for host + process collection, normalization, grouping, and aggregation.
test/e2e-v2/cases/vm/otel-hostmetrics/expected/process-count-3.yml	Adds an assertion verifying grouped process count aggregation (=3).
test/e2e-v2/cases/vm/otel-hostmetrics/entrypoint.sh	Starts 3 sleep processes and runs the collector in the E2E container.
test/e2e-v2/cases/vm/otel-hostmetrics/e2e.yaml	Defines the new E2E verification steps for hostmetrics + process metrics.
test/e2e-v2/cases/vm/otel-hostmetrics/Dockerfile.otelcol	Builds a minimal image bundling otelcol-contrib plus the E2E entrypoint.
test/e2e-v2/cases/vm/otel-hostmetrics/docker-compose.yml	Wires OAP + BanyanDB + collector container; mounts MAL rules and collector config.
oap-server/server-starter/src/main/resources/otel-rules/windows.yaml	Updates production Windows infrastructure MAL to accept both windows-exporter and OTel hostmetrics sources.
oap-server/server-starter/src/main/resources/otel-rules/vm.yaml	Updates production Linux infrastructure MAL to accept both node-exporter and OTel hostmetrics sources.
oap-server/server-starter/src/main/resources/otel-rules/process-hostmetrics-windows.yaml	Adds production Windows process MAL rules for normalized process groups.
oap-server/server-starter/src/main/resources/otel-rules/process-hostmetrics-linux.yaml	Adds production Linux process MAL rules for normalized process groups.
.github/workflows/skywalking.yaml	Adds the new “VM OpenTelemetry Hostmetrics” E2E job entry and pins otelcol-contrib version via env.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.