Embedding ETW Name-GUID Mapping for Log Forward Service by marma-dev · Pull Request #2617 · microsoft/hcsshim

marma-dev · 2026-03-03T08:48:40Z

As per the updated design below,

Adding a ETW name-GUID map in hcsshim along with a list of default log sources.
Adding code to embed the two json files, read them once and create a map for lookup.
Adding code in sidecar-GCS to unmarshal the ModifyServiceSettings appropriately and append GUIDs to the providers in Log Sources before forwarding the request to inbox-GCS

The user should only provide ETW Provider Name only for well-known GUIDs (in etw-map.json).
HCSShim will append the default list of providers (if the annotation is not set to disable default log sources) with or without GUID depending on whether its WCOW or CWCOW respectively and send it in the ModifyServiceSettings message over the HCSShim-GCS bridge.

For CWCOW, the sidecar GCS will intercept the message with the list of providers, check it against the policy (implementation pending) and add the GUIDs by looking up the etw-map to the allowed providers before forwarding it to the inbox GCS within the UVM.

Annotations have been updated as per the design discussion and wcow term has been removed from the nomenclature.

"io.microsoft.virtualmachine.forwardlogs.disable": "Master Switch for Enabling Log Forwarding". Annotation to disable forwarding logs to the host,
Defaults to false for (non-confidential) WCOW, meaning logs will be forwarded to the host if LogSources is set.
Defaults to true for C-WCOW confidential containers, meaning logs will not be forwarded to the host by default. "Master Switch for Enabling Log Forwarding"
"io.microsoft.virtualmachine.forwardlogs.defaultsources.disable" - Specifies whether to disable default providers or not.
Defaults to true.
The user no longer needs to add the default log sources as a base64 encoded json in an annotation. If the user needs to add custom sources, they can use the annotation below
"io.microsoft.virtualmachine.forwardlogs.sources" : Customer defined log sources can be specified as base64 encoded json. Preferably without mentioning ProviderGUID, unless the Name-GUID mapping is not present in etw-map.json

ANNOTATIONS REMOVED

"io.microsoft.virtualmachine.wcow.logsources"
"io.microsoft.virtualmachine.wcow.forwardlogs"

Design

Confidential WCOW

A user will have to specify the log settings in the pod configuration using the annotations above, and in the security policy as well (so that the request from hcshim with the log settings is allowed/verified against)

The Security Policy will contain analogous options for these.
- Boolean Master Switch whether policy would allow any request for log forwarding or not
- Boolean Specifies whether the log providers in the "default-log-sources.json" would be allowed
- List of ProviderNames if a customer wants to explicitly allow certain providers (or ProviderName regex e.g Microsoft.HyperV.*)

Workflow

HCSShim will make a combined list from the log sources specified by io.microsoft.virtualmachine.forwardlogs.sources, if any, and from "default-log-sources.json" (unless io.microsoft.virtualmachine.forwardlogs.defaultsources.disable is set to true). NOTE: HCSShim will remove ProviderGUID if it has a match for its corresponding ProviderName in its own map etw-map.json
HCSShim will send the ModifyServiceSettings with this "combined list" over the HCSShim-GCS bridge.
Gcs-Sidecar intercepts this message and will verify this list against the security policy. The high-level flow is expected as follows
- If policy does not allow (Boolean master Switch is not present/set) Drop all ModifyServiceSettings messages meant for log forward service. Otherwise,
- Construct an "AllowList" by combining the default-list (if the default sources policy boolean is set) and the user-defined matching rules from the policy.
- For every provider in each log source, verify the match against the "AllowList", and add to "ApprovedList" if matched
- Create the request message UpdateLogSources() with the ApprovedList and includeGuids=$true to include the ProviderGUIDs and forward the request to inbox-GCS
Inbox-GCS will receive this request and forward it to Log Forwarding Service which will configure itself to collect events from the Log Sources
HcsShim will send another ModifyServiceSettings message for Starting the Log Forwarding, which the sidecar GCS will forward to inbox GCS if Boolean master switch is set. This gets propagated to LogForwardService and it then starts forwarding the events from the Log Sources to the HcsShim on the host over HvSocket.

WCOW

Since sidecar-GCS is not present for WCOW, the flow skips the sidecar-GCS part and remains same otherwise.
A user will have to specify the log setting in the pod configuration using the annotations above.

Workflow

HCSShim will make a combined list from the log sources specified by io.microsoft.virtualmachine.forwardlogs.sources, if any, and from "default-log-sources.json" (unless io.microsoft.virtualmachine.forwardlogs.defaultsources.disable is set to true). HCSShim will ADD ProviderGUIDs by looking up the etw-map.json
HCSShim will send the ModifyServiceSettings with this "combined list including the GUIDs" over the HCSShim-GCS bridge.
Inbox-GCS receives this message and forwards this to Logging Service which will configure itself to collect events from the Log Sources
HcsShim will send another ModifyServiceSettings message for Starting the Log Forwarding, which gets propagated similarly to LogForwardService and it then starts forwarding the events from the Log Sources to the HcsShim on the host over HvSocket.

internal/vm/vmutils/etw/default-logsources.json

internal/uvm/log_wcow.go

Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>

rawahars

Uber comment is about moving JSON to a Golang file which declares the entire struct.
The provider_map.go will be significantly reduced and optimized with the change above.

In case you are looking to guard against future modifications of the files, I suggest that you add a test file. In the test file, you can write a test which does the validation we are looking for. That way we can capture the errors during build time and ensure that our runtime code is simple.

Lastly, please add tests for provider_map.go once you have the final version.

internal/gcs-sidecar/handlers.go

rawahars · 2026-03-13T01:58:23Z

internal/gcs-sidecar/handlers.go

+	// Todo: Add policy enforcement for modifying service settings
+	modifyRequest, err := unmarshalModifyServiceSettings(req)
+	if err != nil {
+		return err


nit: For consistency in the method, either wrap the errors or return the errors and let caller handle them.
In this case, you can effectively wrap the error stating that unmarshall failed.

rawahars · 2026-03-13T02:24:21Z

internal/vm/vmutils/etw/provider_map.go

+	guidToName   map[string]string // STATIC
+)
+
+// Log Sources JSON structure


We already have a well-defined LogSources structure here.
We can directly create a file default_logsources.go which includes the entire config in native Go format.

That is the preferred mechanism in Go and would also do away with parsing the json altogether.
Reference- https://github.com/kubernetes/kubernetes/blob/4221620945b1f1f73f651c7936456baffcf0327f/pkg/features/kube_features.go

I've added the two go files for default sources and etw map. Lmk if its in line with what you had in mind

Yes that's perfect!

rawahars · 2026-03-13T02:25:44Z

internal/vm/vmutils/etw/provider_map.go

+}
+
+// NormalizeGUID takes a GUID string in various formats and normalizes it to the standard 8-4-4-4-12 format with uppercase letters. It returns an error if the input string is not a valid GUID.
+func NormalizeGUID(in string) (string, error) {


This method need not be exported if it is not used elsewhere.

rawahars · 2026-03-13T02:26:35Z

internal/vm/vmutils/etw/provider_map.go

+			return
+		}
+
+		var cfg EtwInfo


If we have direct Go mapping, we can skip the logic here with parsing the JSON.
In such a case, we can remove this method and directly use the variables.

Added a go for the map as well as default sources

rawahars · 2026-03-13T02:28:00Z

internal/vm/vmutils/etw/provider_map.go

+				continue
+			}
+
+			// Duplicate check


Same here. Since these are hardcoded values, we can ensure not to add duplicate entries.
Moreover, if it's a Go Map, then the case is automatically taken care of.

Added in a different go file

rawahars · 2026-03-13T02:30:08Z

internal/vm/vmutils/etw/provider_map.go

+			return
+		}
+
+		// Check if the default log sources have provider names. If they do, do not include GUIDs in the


Again, this are hardcoded values. We can ensure that correct values are added instead of the static check here at runtime.

I've added them as separate go files in the latest update.

internal/vm/vmutils/etw/provider_map.go

KenGordon · 2026-03-13T09:37:01Z

internal/gcs-sidecar/handlers.go

+					settings.Settings = allowedLogSources
+				}
+			default:
+				log.G(req.ctx).Tracef("modifyServiceSettings for LogForwardService with RPCType: %v, skipping policy enforcement", settings.RPCType)


Can this ever happen? If so what sort of thing?

Same for the other skipping cases. Maybe produce a warning if this is not expected.

In the future if we add more RPCs and the sidecar is old and not updated (for some reason), then this could occur. But this is a stop-measure gap for the moment, till Mahati gets her changes for policy enforcement in.
Changed to raising a warning log

…d methods and addressing review comments Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>

MahatiC · 2026-03-13T15:17:51Z

pkg/annotations/annotations.go

-	// ForwardLogs specifies whether to forward logs to the host or not.
-	ForwardLogs = "io.microsoft.virtualmachine.wcow.forwardlogs"
+	// Specifies whether to disable forwarding logs to the host or not. Defaults to false for (non-confidential) WCOW, meaning logs will be forwarded to the host if LogSources is set. And true for confidential containers, meaning logs will not be forwarded to the host by default.
+	DisableForwardLogs = "io.microsoft.virtualmachine.forwardlogs.disable"


nit: maybe switch up *.disable to *.enable, easier to read than double-negative.

MahatiC · 2026-03-13T15:22:22Z

internal/gcs-sidecar/handlers.go

+					log.G(req.ctx).Tracef("Allowed log sources after policy enforcement: %v", allowedLogSources)
+
+					// Update the allowed log sources in the settings. This will be forwarded to inbox GCS which expects the log sources in a JSON string format with GUIDs for providers included.
+					allowedLogSources = etw.UpdateLogSources(req.ctx, allowedLogSources, false, true)


Given this is the func definition func UpdateLogSources(ctx context.Context, base64EncodedJSONLogConfig string, useDefaultLogSources bool, includeGUIDs bool) string {:

Are you defaulting useDefaultSources to false as this check is already done on the shim side? Ideally it would be policy based? And until we have that, shouldn't this be true?

MahatiC · 2026-03-13T15:24:57Z

internal/uvm/create_wcow.go

+		createOpts:               opts,
+		blockCIMMounts:           make(map[string]*UVMMountedBlockCIMs),
+		logSources:               opts.LogSources,
+		forwardLogs:              !opts.DisableLogForwarding,


nit: rename forwardLogs to match DisableLogForwarding to reflect this new name like you do below (or the other way around if you remove the double negative)

MahatiC · 2026-03-13T15:26:59Z

internal/uvm/types.go


-	forwardLogs bool   // Indicates whether to forward logs from the UVM to the host
-	logSources  string // ETW providers to enable for log forwarding
+	forwardLogs              bool   // Indicates whether to forward logs from the UVM to the host


nit: same here. naming consistency.

rawahars · 2026-03-16T15:51:58Z

internal/vm/vmutils/etw/default-sources.go

@@ -0,0 +1,66 @@
+package etw
+
+// defaultLogSourcesInfo is the native Go representation of the default-logsources.json file.


nit: Comment is misleading. There is no default-logsources.json file now.

rawahars · 2026-03-16T15:53:48Z

internal/vm/vmutils/etw/provider_map.go

+	guidToName   map[string]string // STATIC
+)
+
+// Log Sources JSON structure


Yes that's perfect!

rawahars · 2026-03-16T15:56:44Z

internal/vm/vmutils/etw/provider_map.go

+	if base64EncodedJSONLogConfig != "" {
+		jsonBytes, err := base64.StdEncoding.DecodeString(base64EncodedJSONLogConfig)
+		if err != nil {
+			log.G(ctx).Errorf("Error decoding base64 log config: %v", err)


Do we need to return error at all these places?

marma-dev · 2026-03-17T08:37:27Z

internal/vm/vmutils/etw/provider_map.go

+}
+
+// NormalizeGUID takes a GUID string in various formats and normalizes it to the standard 8-4-4-4-12 format with uppercase letters. It returns an error if the input string is not a valid GUID.
+func normalizeGUID(in string) (string, error) {


use guid.FromString instead

MahatiC reviewed Mar 3, 2026

View reviewed changes

internal/vm/vmutils/etw/default-logsources.json Outdated Show resolved Hide resolved

rawahars reviewed Mar 4, 2026

View reviewed changes

internal/uvm/log_wcow.go Outdated Show resolved Hide resolved

msscotb assigned rawahars and MahatiC Mar 5, 2026

marma-dev force-pushed the user/mmahanta/logsvc-shim-provider-map branch 2 times, most recently from 501b49d to 86e68a3 Compare March 11, 2026 07:07

Initial check-in for ETW Name-GUID Mapping code

8f00207

Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>

marma-dev force-pushed the user/mmahanta/logsvc-shim-provider-map branch 7 times, most recently from 57b670a to 3ebe430 Compare March 12, 2026 13:14

Manish Ranjan Mahanta added 2 commits March 12, 2026 20:05

Moving to VMUtils and static analysis fix

c65f0a3

Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>

sidecar-GCS changes to forward modifyServiceSettings

2eac7bb

Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>

marma-dev force-pushed the user/mmahanta/logsvc-shim-provider-map branch from f18f459 to 2eac7bb Compare March 12, 2026 14:35

marma-dev marked this pull request as ready for review March 12, 2026 16:53

marma-dev requested a review from a team as a code owner March 12, 2026 16:53

marma-dev assigned KenGordon Mar 12, 2026

marma-dev requested a review from rawahars March 12, 2026 17:30

rawahars requested changes Mar 13, 2026

View reviewed changes

KenGordon reviewed Mar 13, 2026

View reviewed changes

Making the default sources and map in native go, remvoing unreference…

c3c8e64

…d methods and addressing review comments Signed-off-by: Manish Ranjan Mahanta <mmahanta@microsoft.com>

MahatiC reviewed Mar 13, 2026

View reviewed changes

rawahars reviewed Mar 16, 2026

View reviewed changes

marma-dev commented Mar 17, 2026

View reviewed changes

		@@ -0,0 +1,66 @@
		package etw

		// defaultLogSourcesInfo is the native Go representation of the default-logsources.json file.

Conversation

marma-dev commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Design

Confidential WCOW

Workflow

WCOW

Workflow

Uh oh!

Uh oh!

Uh oh!

rawahars left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

marma-dev commented Mar 3, 2026 •

edited

Loading