diff --git a/docs/core/diagnostics/debug-highcpu.md b/docs/core/diagnostics/debug-highcpu.md index 2d42a26f3e990..d86f545bc3d1a 100644 --- a/docs/core/diagnostics/debug-highcpu.md +++ b/docs/core/diagnostics/debug-highcpu.md @@ -173,11 +173,29 @@ At this point, you can safely say the CPU is running higher than you expect. Ide ## Analyze High CPU with Profiler -When analyzing an app with high CPU usage, you need a diagnostics tool that can provide insights into what the code is doing. The usual choice is a profiler, and there are different profiler options to choose from. `dotnet-trace` can be used on all operating systems, however, its limitations of safe-point bias and managed-only callstacks result in more general information compared to a kernel-aware profiler like 'perf' for Linux or ETW for Windows. If your performance investigation involves only managed code, generally `dotnet-trace` will be sufficient. +When analyzing an app with high CPU usage, use a profiler to understand what the code is doing. `dotnet-trace collect` works on all operating systems, but safe-point bias and managed-only callstacks limit it to more general information than a kernel-aware profiler like ETW for Windows or `perf` for Linux. Depending on your operating system and .NET version, improved profiling capabilities might be available—see the platform-specific tabs that follow for detailed guidance. ### [Linux](#tab/linux) -The `perf` tool can be used to generate .NET Core app profiles. We will demonstrate this tool, although dotnet-trace could be used as well. Exit the previous instance of the [sample debug target](/samples/dotnet/samples/diagnostic-scenarios). +#### Use `dotnet-trace collect-linux` (.NET 10+) + +On .NET 10 and later, [`dotnet-trace collect-linux`](dotnet-trace.md#dotnet-trace-collect-linux) is the recommended profiling approach on Linux. It combines EventPipe with OS-level perf_events to produce a single unified trace that includes both managed and native callstacks, all without requiring a process restart. + +Ensure the [sample debug target](/samples/dotnet/samples/diagnostic-scenarios) is configured to target .NET 10 or later, then run it and exercise the high CPU endpoint (`https://localhost:5001/api/diagscenario/highcpu/60000`) again. While it's running within the 1-minute request, run `dotnet-trace collect-linux` to capture a machine-wide trace: + +```dotnetcli +sudo dotnet-trace collect-linux +``` + +Let it run for about 20-30 seconds, then press Ctrl+C or Enter to stop the collection. The result is a `.nettrace` file that includes both managed and native callstacks. + +Open the `.nettrace` with [`PerfView`](https://github.com/microsoft/perfview/blob/main/documentation/Downloading.md) and use the **CPU Stacks** view to identify the methods consuming the most CPU time. + +For information about resolving native runtime symbols in the trace, see [Get symbols for native runtime frames](dotnet-trace.md#get-symbols-for-native-runtime-frames). + +#### Use `perf` + +The `perf` tool can also be used to generate .NET Core app profiles. Exit the previous instance of the [sample debug target](/samples/dotnet/samples/diagnostic-scenarios). Set the `DOTNET_PerfMapEnabled` environment variable to cause the .NET app to create a `map` file in the `/tmp` directory. This `map` file is used by `perf` to map CPU addresses to JIT-generated functions by name. For more information, see [Export perf maps and jit dumps](../runtime-config/debugging-profiling.md#export-perf-maps-and-jit-dumps). diff --git a/docs/core/diagnostics/dotnet-trace.md b/docs/core/diagnostics/dotnet-trace.md index 227e1be1d89f0..f032010fc8d06 100644 --- a/docs/core/diagnostics/dotnet-trace.md +++ b/docs/core/diagnostics/dotnet-trace.md @@ -699,6 +699,30 @@ However, when you want to gain a finer control over the lifetime of the app bein ## (Linux-only) Collect a machine-wide trace using dotnet-trace +### Get symbols for native runtime frames + +`collect-linux` captures native frames in callstacks. To resolve native method names for runtime libraries (such as `libcoreclr.so`), place the corresponding debug symbol files on disk beside the libraries. Without these symbols, native frames appear as unresolved addresses in the trace. + +Unlike [`perfcollect`](./trace-perfcollect-lttng.md), `collect-linux` doesn't require you to set environment variables like `DOTNET_PerfMapEnabled` or `DOTNET_EnableEventLog` before starting your application. `collect-linux` dynamically enables perf map generation for JIT-compiled code when the trace begins, so you don't need to restart any .NET processes. + +To download native runtime symbols, use [dotnet-symbol](./dotnet-symbol.md): + +1. Install `dotnet-symbol`: + + ```dotnetcli + dotnet tool install -g dotnet-symbol + ``` + +1. Download the debug symbols for your runtime version. For example, if your runtime is installed at `/usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.0`: + + ```dotnetcli + dotnet-symbol --symbols /usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.0/lib*.so + ``` + +1. Place the downloaded `.so.dbg` files beside the runtime libraries they correspond to (for example, `libcoreclr.so.dbg` next to `libcoreclr.so`). By default, `dotnet-symbol` writes symbol files next to each input file. If your runtime libraries live under a protected path such as `/usr/share/dotnet/...`, run `dotnet-symbol` with elevated permissions (for example, by using `sudo`), or use the `-o`/`--output` option to write to a writable directory, then copy the `.so.dbg` files beside the runtime libraries. + +After you place the symbols, `collect-linux` resolves native method names when it collects the trace. + This example captures CPU samples for all processes on the machine. Any processes running .NET 10+ will also include some additional lightweight events describing GC, JIT, and Assembly loading behavior. ```output diff --git a/docs/core/diagnostics/eventpipe.md b/docs/core/diagnostics/eventpipe.md index 8ab863fc3a0ce..5db2c33fbeb45 100644 --- a/docs/core/diagnostics/eventpipe.md +++ b/docs/core/diagnostics/eventpipe.md @@ -25,18 +25,20 @@ To learn more about the NetTrace format, see the [NetTrace format documentation] EventPipe is part of the .NET runtime and is designed to work the same way across all the platforms .NET Core supports. This allows tracing tools based on EventPipe, such as `dotnet-counters`, `dotnet-gcdump`, and `dotnet-trace`, to work seamlessly across platforms. -However, because EventPipe is a runtime built-in component, its scope is limited to managed code and the runtime itself. EventPipe events include stacktraces with managed code frame information only. If you want events generated from other unmanaged user-mode libraries, CPU sampling for native code, or kernel events you should use OS-specific tracing tools such as ETW or perf_events. On Linux the [perfcollect tool](./trace-perfcollect-lttng.md) helps automate using perf_events and [LTTng](https://en.wikipedia.org/wiki/LTTng). +However, because EventPipe is a runtime built-in component, its scope is limited to managed code and the runtime itself. Without other tracing tools, EventPipe events include stack traces with managed code frame information only. To get events from other unmanaged user-mode libraries, CPU sampling for native code, or kernel events, use OS-specific tracing tools such as ETW or perf_events. On Linux, the [perfcollect tool](./trace-perfcollect-lttng.md) helps automate using perf_events and [LTTng](https://en.wikipedia.org/wiki/LTTng). -Another major difference between EventPipe and ETW/perf_events is admin/root privilege requirement. To trace an application using ETW or perf_events you need to be an admin/root. Using EventPipe, you can trace applications as long as the tracer (for example, `dotnet-trace`) is run as the same user as the user that launched the application. +Starting in .NET 10, EventPipe on Linux can emit events as [user_events](https://docs.kernel.org/trace/user_events.html), enabling collection of managed events, OS/kernel events, and native callstacks in a single unified trace. This mode requires admin/root privileges and Linux kernel 6.4+. For more information, see [`dotnet-trace collect-linux`](./dotnet-trace.md#dotnet-trace-collect-linux). + +Another major difference between EventPipe and ETW/perf_events is admin/root privilege requirement. To trace an application using ETW or perf_events you need to be an admin/root. Using standard EventPipe, you can trace applications as long as the tracer (for example, `dotnet-trace`) is run as the same user as the user that launched the application. The EventPipe (user_events) mode described earlier requires admin/root privileges because it interacts with OS-level tracing infrastructure. The following table is a summary of the differences between EventPipe and ETW/perf_events. -|Feature|EventPipe|ETW|perf_events| -|-------|---------|---|-----------| -|Cross-platform|Yes|No (only on Windows)|No (only on supported Linux distros)| -|Require admin/root privilege|No|Yes|Yes| -|Can get OS/kernel events|No|Yes|Yes| -|Can resolve native callstacks|No|Yes|Yes| +|Feature|EventPipe|EventPipe (user_events)|ETW|perf_events| +|-------|---------|----------------------|---|-----------| +|Cross-platform|Yes|No (only on supported Linux distros)|No (only on Windows)|No (only on supported Linux distros)| +|Require admin/root privilege|No|Yes|Yes|Yes| +|Can get OS/kernel events|No|Yes|Yes|Yes| +|Can resolve native callstacks|No|Yes|Yes|Yes| ## Use EventPipe to trace your .NET application