LOG-8386: Initialize custom Loki label keys in VRL remap

[vparfonov]

Description

When users specify custom labelKeys in Loki output, they were not initialized in the VRL remap transform, causing empty strings in Loki. Add remapLabelKeys() to initialize both container metadata fields and custom user-specified fields that are read directly from log records.

/cc @cahartma @Clee2691
/assign @jcantrill

Links

• Depending on PR(s):
• GitHub issue:
• JIRA: https://redhat.atlassian.net/browse/LOG-8386
• Enhancement proposal:

[qodo-code-review]

Review Summary by Qodo

Initialize custom Loki label keys in VRL remap transform

✨ Enhancement

Walkthroughs

Description

• Initialize custom Loki label keys in VRL remap transform
• Add remapLabelKeys() function to include container metadata and custom fields
• Prevent missing field errors when users specify custom labelKeys in Loki output
• Update remap labels to use dynamic label key initialization

Diagram

flowchart LR
  A["User Custom LabelKeys"] --> B["remapLabelKeys Function"]
  C["Container Metadata Labels"] --> B
  B --> D["VRL Remap Transform"]
  D --> E["Loki Output with Initialized Fields"]

Loading

File Changes

1. internal/generator/vector/output/loki/loki.go ✨ Enhancement +19/-1

Add dynamic label key initialization function

• Added remapLabelKeys() function to dynamically initialize label keys
• Function combines container metadata labels with custom user-specified labelKeys
• Filters out keys that are environment variables or mapped to other fields
• Updated RemapLabels() to use remapLabelKeys() instead of static containerLabels

internal/generator/vector/output/loki/loki.go

---

2. internal/generator/vector/output/loki/with_custom_labels.toml ⚙️ Configuration changes +6/-3

Initialize additional Kubernetes label fields in VRL

• Added VRL initialization checks for kubernetes.container_name and kubernetes.labels.app
• Reordered initialization checks for consistency
• Ensures all custom label fields are initialized to empty strings if missing

internal/generator/vector/output/loki/with_custom_labels.toml

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1)   📘 Rule violations (0)   📎 Requirement gaps (0)   🎨 UX Issues (0)

🐞\ ≡ Correctness (1)

1. Invalid VRL for labelKeys 🐞 ≡

Description

remapLabelKeys() inserts raw spec.Loki.LabelKeys into remapLabelsVrl(), which emits
exists(.<key>) and assignments using the unsanitized key string. For Kubernetes labelKeys
containing slashes/dashes/dots (e.g. kubernetes.labels.app.kubernetes.io/name), this produces
syntactically invalid VRL (and does not initialize the sanitized field that the Loki sink template
actually reads), which can prevent the collector from starting.

Code

internal/generator/vector/output/loki/loki.go[R286-306]

+func remapLabelKeys(l *obs.Loki) []string {
+	if l == nil || len(l.LabelKeys) == 0 {
+		return containerLabels
+	}
+	keys := *sets.NewString(containerLabels...)
+	for _, k := range l.LabelKeys {
+		// Only add custom keys that are read directly from the record,
+		// skip keys that are env vars or mapped to other fields.
+		if generateCustomLabelValues(k) == "" {
+			keys.Insert(k)
+		}
+	}
+	return keys.List()
+}
+
func RemapLabels(id string, o obs.OutputSpec, inputs []string) Element {
	return Remap{
		ComponentID: id,
		Inputs:      vectorhelpers.MakeInputs(inputs...),
-		VRL:         remapLabelsVrl(containerLabels),
+		VRL:         remapLabelsVrl(remapLabelKeys(o.Loki)),
	}

Evidence

The generated VRL remap source uses the label key verbatim as a VRL field path. However, elsewhere
the generator explicitly sanitizes/quotes Kubernetes label keys for template access, and there is an
existing functional test that configures labelKeys with slashes/dashes and expects the collector to
start. With this PR, those same labelKeys will now be injected into the VRL remap initialization,
yielding invalid field paths like .kubernetes.labels.app.kubernetes.io/name and
.kubernetes.labels.prefix-cloud_com_platform-stage.

internal/generator/vector/output/loki/loki.go[260-269]
internal/generator/vector/output/loki/loki.go[272-280]
internal/generator/vector/output/loki/loki.go[283-299]
internal/generator/vector/output/loki/loki.go[301-306]
test/functional/outputs/loki/application_logs_vector_test.go[70-81]
internal/generator/vector/output/loki/loki_test.go[14-21]
internal/generator/vector/output/loki/loki_test.go[34-37]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The remap VRL initializer is generated from raw `Loki.LabelKeys`, but VRL field paths cannot contain characters like `/` or `-` unquoted. This breaks configurations where users specify Kubernetes labelKeys like `kubernetes.labels.app.kubernetes.io/name` or `kubernetes.labels.prefix-cloud_com_platform-stage` (covered by an existing functional test).

### Issue Context
The Loki sink label *template* path already sanitizes these Kubernetes labelKeys (replacing `/` and `.` with `_` and quoting the resulting key segment). The VRL remap initializer must initialize the same sanitized/quoted record path that the template reads, and must never emit invalid VRL.

### Fix Focus Areas
- internal/generator/vector/output/loki/loki.go[260-306]

### Suggested fix approach
1. Introduce a helper that converts a labelKey into a VRL-safe field path **for initialization**:
  - For `kubernetes.labels.*` and `kubernetes.namespace_labels.*`:
    - Split at `labels.`
    - Replace `/` and `.` with `_` in the suffix (same as `formatLokiLabelValue`)
    - Return `kubernetes.labels."<sanitized>"` **without backslashes** in the actual VRL output, i.e. `kubernetes.labels."..."` should become `kubernetes.labels."..."` in Go but result in VRL: `.kubernetes.labels."..."`? (Prefer building the VRL as `kubernetes.labels."..."` in the string so the emitted VRL is `.kubernetes.labels."..."` with real quotes, not backslashes.)
    - Ensure keys containing `-` are quoted (these prefixes should always be quoted).
  - For other keys, keep as-is.
2. In `remapLabelKeys()`, insert the VRL-safe path (not the raw key) into the set.
3. Add/adjust unit coverage for remap initialization generation using a complex key like `kubernetes.labels.app.kubernetes.io/name` to prevent regressions.
4. Update expected TOML fixtures if the emitted VRL changes (e.g., `.kubernetes.labels."app"` vs `.kubernetes.labels.app`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more

[openshift-ci-robot]

@vparfonov: This pull request references LOG-8386 which is a valid jira issue.

Details

In response to this:

Description

When users specify custom labelKeys in Loki output, they were not initialized in the VRL remap transform, causing empty strings in Loki. Add remapLabelKeys() to initialize both container metadata fields and custom user-specified fields that are read directly from log records.

/cc @cahartma @Clee2691
/assign @jcantrill

Links

• Depending on PR(s):
• GitHub issue:
• JIRA: https://redhat.atlassian.net/browse/LOG-8386
• Enhancement proposal:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vparfonov
Once this PR has been reviewed and has the lgtm label, please assign xperimental for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[vparfonov]

/retest

[openshift-ci]

@vparfonov: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.