From 8d5b771408697bc4ce27999f9f49bc997ea042e0 Mon Sep 17 00:00:00 2001 From: Tony Redondo Date: Tue, 16 Jun 2026 16:19:48 +0200 Subject: [PATCH] Add Bazel Test Optimization setup docs --- config/_default/menus/main.en.yaml | 20 ++ content/en/tests/_index.md | 3 + content/en/tests/setup/_index.md | 3 + content/en/tests/setup/bazel/_index.md | 80 +++++++ content/en/tests/setup/bazel/go.md | 268 ++++++++++++++++++++++ content/en/tests/setup/bazel/java.md | 271 ++++++++++++++++++++++ content/en/tests/setup/bazel/python.md | 299 +++++++++++++++++++++++++ content/en/tests/setup/go.md | 3 + content/en/tests/setup/java.md | 6 +- content/en/tests/setup/python.md | 3 + 10 files changed, 955 insertions(+), 1 deletion(-) create mode 100644 content/en/tests/setup/bazel/_index.md create mode 100644 content/en/tests/setup/bazel/go.md create mode 100644 content/en/tests/setup/bazel/java.md create mode 100644 content/en/tests/setup/bazel/python.md diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml index 7b8f1907fac..c49b746344c 100644 --- a/config/_default/menus/main.en.yaml +++ b/config/_default/menus/main.en.yaml @@ -5956,6 +5956,26 @@ menu: parent: tests_setup identifier: tests_setup_junit_xml weight: 108 + - name: Bazel + url: tests/setup/bazel/ + parent: tests_setup + identifier: tests_setup_bazel + weight: 109 + - name: Java + url: tests/setup/bazel/java/ + parent: tests_setup_bazel + identifier: tests_setup_bazel_java + weight: 10901 + - name: Python + url: tests/setup/bazel/python/ + parent: tests_setup_bazel + identifier: tests_setup_bazel_python + weight: 10902 + - name: Go + url: tests/setup/bazel/go/ + parent: tests_setup_bazel + identifier: tests_setup_bazel_go + weight: 10903 - name: Network Settings url: tests/network/ parent: tests diff --git a/content/en/tests/_index.md b/content/en/tests/_index.md index 8a891a3277d..e4d9db935ed 100644 --- a/content/en/tests/_index.md +++ b/content/en/tests/_index.md @@ -71,6 +71,8 @@ Select an option to configure Test Optimization in Datadog:
+If you run Go, Java, or Python tests with Bazel, use Datadog's official [Bazel rules for Test Optimization][bazel-setup]. Bazel support is in Preview. + In addition to tests, Test Optimization provides visibility over the whole testing phase of your project. ### Supported features @@ -193,3 +195,4 @@ When you're evaluating failed or flaky tests, or the performance of a CI test, y [7]: /notebooks [8]: https://app.datadoghq.com/ci/test-runs [9]: /monitors/types/ci/ +[bazel-setup]: /tests/setup/bazel/ diff --git a/content/en/tests/setup/_index.md b/content/en/tests/setup/_index.md index 9dae35c2cfc..fbf4dcdd7d5 100644 --- a/content/en/tests/setup/_index.md +++ b/content/en/tests/setup/_index.md @@ -20,6 +20,8 @@ For information about configuration options for [Test Optimization][1], choose y
+If you run Go, Java, or Python tests with Bazel, use Datadog's official [Bazel rules for Test Optimization][5]. + If you run your tests in an environment with network restrictions, see the [Agent Network Traffic][2] or [Agentless Network Settings][3] guide for information on how to configure whitelisting. @@ -29,3 +31,4 @@ If you run your tests in a container, see the [Tests in Containers][4] guide for [2]: /agent/configuration/network/ [3]: /tests/network/ [4]: /tests/containers/ +[5]: /tests/setup/bazel/ diff --git a/content/en/tests/setup/bazel/_index.md b/content/en/tests/setup/bazel/_index.md new file mode 100644 index 00000000000..daab731e27f --- /dev/null +++ b/content/en/tests/setup/bazel/_index.md @@ -0,0 +1,80 @@ +--- +title: Configure Test Optimization with Bazel +description: Configure Test Optimization for Bazel test targets with Datadog's official Bazel rules. +further_reading: + - link: "/tests/setup/" + tag: "Documentation" + text: "Configure Test Optimization" + - link: "/tests/explorer/" + tag: "Documentation" + text: "Explore Test Results and Performance" + - link: "/tests/troubleshooting/" + tag: "Documentation" + text: "Troubleshooting Test Optimization" +--- + +
Bazel support for Test Optimization is in Preview.
+ +Datadog provides official Bazel rules for Test Optimization. Use these rules to configure Bazel test targets. The rules read Test Optimization metadata, write local payload files during test execution, and upload test results to Datadog after Bazel tests complete. + +The Bazel integration keeps Datadog metadata fetches outside test execution. During module or repository resolution, Bazel fetches Test Optimization metadata from Datadog and exposes it through a generated repository. During test execution, language-specific macros pass the metadata location to the test process and configure payloads to be written under `TEST_UNDECLARED_OUTPUTS_DIR`. After tests finish, run the doctor and uploader targets with `bazel run`. + +Test Impact Analysis is not supported for Bazel. + +## Language setup pages + +Use the language-specific setup page for your Bazel test targets: + +- [Java tests][1] +- [Python tests][2] +- [Go tests][3] + +## Compatibility + +This section includes setup pages for the following language test targets: + +| Language | Bazel macro | Notes | +|---|---|---| +| Java | `dd_topt_java_test` | Requires a `dd-java-agent` JAR label. | +| Python | `dd_topt_py_test` | Supports the managed `pytest` runner and repository-owned pytest wrappers. | +| Go | `dd_topt_go_test` | Use `test_optimization` mode for the faster standard-library `testing` path, or `general` mode for broader Orchestrion support. | + +Use `datadog-rules-test-optimization` version `1.2.0` and the commit pin shown on each language setup page. + +## How the Bazel flow works + +The Bazel setup has four parts: + +1. Add the `datadog-rules-test-optimization` module and the companion module for your language. +1. Configure a sync repository with your Datadog service name, runtime name, and runtime version. +1. Replace the language test rule with the Datadog Bazel macro for each instrumented test target. +1. Run tests, validate local payloads with the doctor target, validate enrichment with the uploader dry run, and then upload payloads. + +For remote execution, configure Bazel to download test outputs locally: + +```text +test:test-optimization --remote_download_outputs=all +``` + +Without local `test.outputs` directories, the doctor and uploader cannot inspect payload files after `bazel test`. + +## Upload payloads + +Run the upload flow after your Bazel tests complete. Replace `//tools/test_optimization` with the package where you declared the doctor and uploader targets: + +```bash +bazel test --config=test-optimization //... +bazel run --config=test-optimization //tools/test_optimization:dd_test_optimization_doctor +bazel run --config=test-optimization //tools/test_optimization:dd_upload_payloads -- --dry-run --validate-enrichment +DD_API_KEY= DD_SITE= bazel run --config=test-optimization //tools/test_optimization:dd_upload_payloads +``` + +Do not pass `DD_API_KEY`, `DD_SITE`, `DD_GIT_*`, or upload endpoint variables through `--test_env`. Forward sync metadata with `--repo_env`, and pass upload credentials only to the uploader runtime. + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} + +[1]: /tests/setup/bazel/java/ +[2]: /tests/setup/bazel/python/ +[3]: /tests/setup/bazel/go/ diff --git a/content/en/tests/setup/bazel/go.md b/content/en/tests/setup/bazel/go.md new file mode 100644 index 00000000000..80e18b3f815 --- /dev/null +++ b/content/en/tests/setup/bazel/go.md @@ -0,0 +1,268 @@ +--- +title: Bazel Rules for Go Tests +description: Configure Test Optimization for Go test targets in Bazel. +code_lang: go +type: multi-code-lang +code_lang_weight: 60 +further_reading: + - link: "/tests/setup/bazel/" + tag: "Documentation" + text: "Configure Test Optimization with Bazel" + - link: "/tests/explorer/" + tag: "Documentation" + text: "Explore Test Results and Performance" + - link: "/tests/troubleshooting/" + tag: "Documentation" + text: "Troubleshooting Test Optimization" +--- + +
Bazel support for Test Optimization is in Preview. Test Impact Analysis is not supported for Bazel.
+ +Datadog provides official Bazel rules for Go Test Optimization. Guided bootstrap writes a local `dd_go_test` wrapper that calls Datadog's `dd_topt_go_test` macro with the recommended Test Optimization defaults. Use `dd_go_test` in package `BUILD.bazel` files after running bootstrap. Use `dd_topt_go_test` directly when you maintain your own wrapper or need to set Datadog attributes such as `orchestrion_mode`. + +## Compatibility + +The Go Bazel rule supports two Orchestrion modes: + +| Mode | Use when | +|---|---| +| `test_optimization` | You want the faster Test Optimization path for tests that use the standard library `testing` package. Guided bootstrap configures this mode. | +| `general` | Your test target needs the broader Orchestrion path, such as tests that use frameworks or patterns outside the standard library `testing` package. | + +Use the following minimum versions: + +| Component | Version | +|---|---| +| `datadog-rules-test-optimization` | `>=1.2.0` | +| `datadog-rules-test-optimization-go` | `>=1.2.0` | +| `rules_go` | `0.60.0` | +| `dd-trace-go` | `>=v2.9.0-rc.2` | +| Orchestrion | `>=v1.9.0` | + +## Prerequisites + +Before setting up Test Optimization for Go tests in Bazel: + +- Configure a Datadog API key in your CI secret store. +- Set `DD_SITE` to your Datadog site, such as `datadoghq.com`. +- Use a Bazel workspace that can resolve the `datadog-rules-test-optimization` module. +- Use Go modules for the Go package under test. + +## Add modules + +Add the core module, the Go companion module, and `rules_go` to `MODULE.bazel`: + +```starlark +bazel_dep(name = "datadog-rules-test-optimization", version = "1.2.0") +git_override( + module_name = "datadog-rules-test-optimization", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", +) + +bazel_dep(name = "datadog-rules-test-optimization-go", version = "1.2.0") +git_override( + module_name = "datadog-rules-test-optimization-go", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", + strip_prefix = "modules/go", +) + +bazel_dep(name = "rules_go", version = "0.60.0") +``` + +Use the same commit SHA for the core module and the Go companion module. + +## Use `WORKSPACE` mode + +If Bzlmod is disabled, declare the core repository in `WORKSPACE`. Then use the public Go helper to declare the Go companion repository and the Orchestrion-enabled `rules_go` fork: + +```starlark +load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") + +git_repository( + name = "datadog-rules-test-optimization", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", +) + +load( + "@datadog-rules-test-optimization//tools/go:workspace_repositories.bzl", + "datadog_go_test_optimization_workspace_repositories", +) + +datadog_go_test_optimization_workspace_repositories( + rto_commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", + rules_go_repo_name = "io_bazel_rules_go", + rules_go_variant = "base", +) +``` + +In the `datadog_go_test_optimization_workspace_repositories(...)` call, keep `rules_go_variant = "base"`. Set `rules_go_variant = "complete"` only when the repository needs the extended monorepo compatibility variant. + +## Run guided bootstrap + +For a single-service Go workspace, run the guided bootstrap: + +```bash +bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \ + --guided \ + --service \ + --runtime-version 1.25.0 \ + --dd-trace-go-version v2.9.0-rc.2 \ + --write-bazelrc +``` + +The bootstrap creates local wrapper targets and a managed `.bazelrc` config named `test-optimization`. By default, bootstrap uses targeted Go module sync. This updates the Orchestrion and `dd-trace-go` tool requirements without running `go mod tidy` for the whole module. + +If your Go module lives below the workspace root, pass its path: + +```bash +bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \ + --guided \ + --service \ + --runtime-version 1.25.0 \ + --dd-trace-go-version v2.9.0-rc.2 \ + --go-module-dir path/to/go-module \ + --write-bazelrc +``` + +## Configure test targets + +After bootstrap writes the local wrapper, use the generated `dd_go_test` wrapper in package `BUILD.bazel` files. The wrapper calls `dd_topt_go_test` with `orchestrion_mode = "test_optimization"` and the bootstrap-managed Orchestrion pin files: + +```starlark +load("@rules_go//go:def.bzl", "go_library") +load("//tools/build:dd_go_test.bzl", "dd_go_test") + +go_library( + name = "pkg_lib", + srcs = ["main.go"], +) + +dd_go_test( + name = "pkg_go_test", + srcs = ["main_test.go"], + embed = [":pkg_lib"], +) +``` + +Use `embed = [":pkg_lib"]` so the macro can infer the Go import path from `rules_go` providers and select the matching per-module Test Optimization metadata. + +If you use a manual wrapper instead of guided bootstrap, call `dd_topt_go_test` and set `orchestrion_mode = "test_optimization"` for the faster standard-library `testing` path: + +```starlark +load("@rules_go//go:def.bzl", "go_library") +load("@datadog-rules-test-optimization-go//:topt_go_test.bzl", "dd_topt_go_test") +load("@test_optimization_data//:export.bzl", "topt_data") + +go_library( + name = "pkg_lib", + srcs = ["main.go"], +) + +dd_topt_go_test( + name = "pkg_go_test", + srcs = ["main_test.go"], + embed = [":pkg_lib"], + orchestrion_mode = "test_optimization", + topt_data = topt_data, +) +``` + +For nested packages, pass module-root pin files with `orchestrion_pin_files` when the BUILD file does not live beside `go.mod`. + +For tests that need the broader Orchestrion path, set `orchestrion_mode = "general"` in your manual wrapper or repo-local wrapper: + +```starlark +dd_topt_go_test( + name = "pkg_go_test", + srcs = ["main_test.go"], + embed = [":pkg_lib"], + orchestrion_mode = "general", + topt_data = topt_data, +) +``` + +## Run and validate tests + +Run tests with the generated Bazel config: + +```bash +bazel test --config=test-optimization //... +bazel run --config=test-optimization //:dd_test_optimization_doctor +bazel run --config=test-optimization //:dd_upload_payloads -- --dry-run --validate-enrichment +DD_API_KEY= DD_SITE= bazel run --config=test-optimization //:dd_upload_payloads +``` + +You do not need to set the Test Optimization runtime variables manually. The `dd_topt_go_test` macro adds them to the generated test target, including: + +`DD_CIVISIBILITY_ENABLED` +: Enables Test Optimization for the Go test process. + +`DD_TEST_OPTIMIZATION_MANIFEST_FILE` +: Points the Go test process to the synced Test Optimization metadata. + +`DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES` +: Configures payloads to be written as JSON files under `TEST_UNDECLARED_OUTPUTS_DIR`. + +If you pass an `env` attribute to `dd_topt_go_test`, keep these variables reserved for the macro. + +## Large repositories + +For large WORKSPACE repositories, use bootstrap `--workspace-mode` to print repository wiring and write local scaffolding without editing `WORKSPACE`: + +```bash +bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \ + --workspace-mode \ + --print-workspace-snippet \ + --service \ + --runtime-version 1.25.0 \ + --sync-repo-name test_optimization_data \ + --rto-commit 69953536d4ef1252c8181c267d16c61263f0aa4c \ + --rules-go-variant base \ + --rules-go-repo-name io_bazel_rules_go +``` + +Use `--rules-go-variant complete` only when the repository needs the extended monorepo compatibility variant. + +Keep repository-specific scheduling, tags, flaky policy, Docker defaults, and platform constraints in your local wrapper. The optimized wrapper should own only Test Optimization wiring, Orchestrion mode, and pin files. + +## Known limitations + +- Test Impact Analysis is not supported for Bazel. +- The faster `test_optimization` mode supports tests that use the standard library `testing` package. Use `general` mode for tests that need the broader Orchestrion path. +- Go module pins and Bazel tracer pins must resolve to the same `dd-trace-go` versions. + +## Troubleshooting + +### The doctor reports missing payloads + +This can happen when the instrumented test target did not run. It can also happen when test outputs were not downloaded locally, or when the test target uses the raw `go_test` rule instead of the Datadog wrapper. + +To fix this issue: + +1. Run the exact `dd_go_test` or `dd_topt_go_test` target before running the doctor. +1. For remote execution, add `test:test-optimization --remote_download_outputs=all` to `.bazelrc`. +1. Confirm the target uses `orchestrion_mode = "test_optimization"` or `orchestrion_mode = "general"` when using `dd_topt_go_test` directly. + +### The doctor reports `full_bundle_no_match` + +This can happen when the macro cannot map the Go test target to a per-module metadata bundle. + +To fix this issue, prefer `go_library` plus `embed = [":pkg_lib"]` so the macro can infer the same import path that `rules_go` uses. Use `module_label_override` only when the expected module label is known. + +### Go builds fail with a tracer version mismatch + +This can happen when Bazel and the local Go module resolve different `dd-trace-go` versions. + +To fix this issue, rerun bootstrap with the `dd-trace-go` version used by your workspace: + +```bash +bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \ + --dd-trace-go-version v2.9.0-rc.2 +``` + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} diff --git a/content/en/tests/setup/bazel/java.md b/content/en/tests/setup/bazel/java.md new file mode 100644 index 00000000000..12ce1d078f5 --- /dev/null +++ b/content/en/tests/setup/bazel/java.md @@ -0,0 +1,271 @@ +--- +title: Bazel Rules for Java Tests +description: Configure Test Optimization for Java test targets in Bazel. +code_lang: java +type: multi-code-lang +code_lang_weight: 10 +further_reading: + - link: "/tests/setup/bazel/" + tag: "Documentation" + text: "Configure Test Optimization with Bazel" + - link: "/tests/explorer/" + tag: "Documentation" + text: "Explore Test Results and Performance" + - link: "/tests/troubleshooting/" + tag: "Documentation" + text: "Troubleshooting Test Optimization" +--- + +
Bazel support for Test Optimization is in Preview. Test Impact Analysis is not supported for Bazel.
+ +Datadog provides official Bazel rules for Java Test Optimization. Use `dd_topt_java_test` to configure Java test targets to run with the Datadog Java tracer and write Test Optimization payloads during Bazel test execution. + +## Compatibility + +Use the following minimum versions: + +| Component | Version | +|---|---| +| `datadog-rules-test-optimization` | `>=1.2.0` | +| `datadog-rules-test-optimization-java` | `>=1.2.0` | +| Java runtime example | `>=17` | +| `dd-java-agent` example | `>=1.60.0` | + +The Datadog Bazel rule is the official Test Optimization setup path for Java tests that run with Bazel. + +## Prerequisites + +Before setting up Test Optimization for Java tests in Bazel: + +- Configure a Datadog API key in your CI secret store. +- Set `DD_SITE` to your Datadog site, such as `datadoghq.com`. +- Use a Bazel workspace that can resolve the `datadog-rules-test-optimization` module. +- Make the `dd-java-agent` JAR available to Bazel. + +## Add modules + +Add the core module and the Java companion module to `MODULE.bazel`: + +```starlark +bazel_dep(name = "datadog-rules-test-optimization", version = "1.2.0") +git_override( + module_name = "datadog-rules-test-optimization", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", +) + +bazel_dep(name = "datadog-rules-test-optimization-java", version = "1.2.0") +git_override( + module_name = "datadog-rules-test-optimization-java", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", + strip_prefix = "modules/java", +) +``` + +Use the same commit SHA for the core module and the Java companion module. + +## Use `WORKSPACE` mode + +If Bzlmod is disabled, declare the core repository in `WORKSPACE`. Then use the public Java helper to declare the Java companion repository: + +```starlark +load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") + +git_repository( + name = "datadog-rules-test-optimization", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", +) + +load( + "@datadog-rules-test-optimization//tools/java:workspace_repositories.bzl", + "datadog_java_test_optimization_workspace_repositories", +) + +datadog_java_test_optimization_workspace_repositories( + rto_commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", + rules_java_repo_name = "rules_java", +) +``` + +The consuming repository owns Java rules, Java toolchains, test framework dependencies, and the `dd-java-agent` artifact. + +## Configure Test Optimization metadata + +Configure a sync repository in `MODULE.bazel`: + +```starlark +topt = use_extension( + "@datadog-rules-test-optimization//tools/core:test_optimization_sync.bzl", + "test_optimization_sync_extension", +) + +topt.test_optimization_sync( + name = "test_optimization_data", + service = "", + runtime_name = "java", + runtime_version = "17", +) + +use_repo(topt, "test_optimization_data") +``` + +Declare the `dd-java-agent` JAR as a Bazel file dependency. The `dd_topt_java_test` macro requires an `agent_jar` label: + +```starlark +http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") + +http_file( + name = "dd_java_agent", + downloaded_file_path = "dd-java-agent.jar", + urls = ["https://repo1.maven.org/maven2/com/datadoghq/dd-java-agent/1.60.0/dd-java-agent-1.60.0.jar"], +) +``` + +## Add doctor and uploader targets + +Add a doctor and uploader pair. In large repositories, put these targets in a lightweight package such as `//tools/test_optimization`. + +```starlark +load("@datadog-rules-test-optimization//tools/core:test_optimization_targets.bzl", "dd_test_optimization_targets") + +dd_test_optimization_targets( + name = "test_optimization", + sync_repo_name = "test_optimization_data", + expected_targets = [ + "//java/pkg:pkg_java_test", + ], +) +``` + +This macro creates: + +- `//tools/test_optimization:dd_test_optimization_doctor` +- `//tools/test_optimization:dd_upload_payloads` + +## Configure Java test targets + +Use `dd_topt_java_test` in package `BUILD.bazel` files: + +```starlark +load("@datadog-rules-test-optimization-java//:topt_java_test.bzl", "dd_topt_java_test") +load("@test_optimization_data//:export.bzl", "topt_data") + +java_library( + name = "pkg_lib", + srcs = ["Hello.java"], +) + +dd_topt_java_test( + name = "pkg_java_test", + srcs = ["HelloTest.java"], + deps = [":pkg_lib"], + test_class = "com.example.pkg.HelloTest", + topt_data = topt_data, + agent_jar = "@dd_java_agent//file", +) +``` + +The macro injects the Datadog Java tracer with `-javaagent` and adds the agent JAR to test runtime data. It also enables Test Optimization for the Java test process and configures payloads to be written as JSON files under `TEST_UNDECLARED_OUTPUTS_DIR`. + +## Run and validate tests + +Add the required Bazel config: + +```text +common:test-optimization --repo_env=DD_API_KEY +common:test-optimization --repo_env=DD_SITE +common:test-optimization --repo_env=DD_GIT_REPOSITORY_URL +common:test-optimization --repo_env=DD_GIT_BRANCH +common:test-optimization --repo_env=DD_GIT_TAG +common:test-optimization --repo_env=DD_GIT_COMMIT_SHA +common:test-optimization --repo_env=DD_PR_NUMBER +test:test-optimization --remote_download_outputs=all +``` + +Run tests, validate local payloads, validate enrichment, and upload: + +```bash +bazel test --config=test-optimization //java/pkg:pkg_java_test +bazel run --config=test-optimization //tools/test_optimization:dd_test_optimization_doctor +bazel run --config=test-optimization //tools/test_optimization:dd_upload_payloads -- --dry-run --validate-enrichment +DD_API_KEY= DD_SITE= bazel run --config=test-optimization //tools/test_optimization:dd_upload_payloads +``` + +Do not pass upload credentials, upload endpoints, or `DD_GIT_*` values through `--test_env`. + +## Multi-service setup + +Use the multi-sync extension when one Bazel workspace reports Java tests for more than one Datadog service: + +```starlark +topt = use_extension( + "@datadog-rules-test-optimization//tools/core:test_optimization_multi_sync.bzl", + "test_optimization_multi_sync_extension", +) + +topt.test_optimization_multi_sync( + name = "test_optimization_data", + services = ["java-service-a", "java-service-b"], + runtime_name = "java", + runtime_version = "17", +) + +use_repo( + topt, + "test_optimization_data", + "test_optimization_data_java_service_a", + "test_optimization_data_java_service_b", +) +``` + +In test targets, pass `topt_data_by_service` and select the service: + +```starlark +load("@test_optimization_data//:export.bzl", "topt_data_by_service") + +dd_topt_java_test( + name = "pkg_java_test", + srcs = ["HelloTest.java"], + test_class = "com.example.pkg.HelloTest", + topt_data = topt_data_by_service, + topt_service = "java_service_a", + agent_jar = "@dd_java_agent//file", +) +``` + +## Known limitations + +- Test Impact Analysis is not supported for Bazel. +- `dd_topt_java_test` requires an `agent_jar` label. +- Automatic coverage configuration for Bazel is not supported. + +## Troubleshooting + +### The test target fails because `agent_jar` is missing + +This can happen when `dd_topt_java_test` does not receive a label that points to the Datadog Java tracer JAR. + +To fix this issue, declare the JAR with `http_file`, `maven_install`, or a repository-owned filegroup, and pass the label to `agent_jar`. + +### Tests run but Datadog does not show results + +This can happen when the test target uses the raw `java_test` rule. It can also happen when payload files were not downloaded locally, or when the uploader did not receive the synced context data. + +To fix this issue: + +1. Confirm the target uses `dd_topt_java_test`. +1. For remote execution, add `test:test-optimization --remote_download_outputs=all` to `.bazelrc`. +1. Run the doctor target before upload. +1. Run uploader dry-run enrichment validation before the real upload. + +### The doctor reports missing Git metadata + +This can happen when repository URL, commit SHA, branch, or tag metadata is not available during metadata sync. + +To fix this issue, pass Git metadata through `--repo_env`, not `--test_env`. + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} diff --git a/content/en/tests/setup/bazel/python.md b/content/en/tests/setup/bazel/python.md new file mode 100644 index 00000000000..a3a00ffdf68 --- /dev/null +++ b/content/en/tests/setup/bazel/python.md @@ -0,0 +1,299 @@ +--- +title: Bazel Rules for Python Tests +description: Configure Test Optimization for Python test targets in Bazel. +code_lang: python +type: multi-code-lang +code_lang_weight: 30 +further_reading: + - link: "/tests/setup/bazel/" + tag: "Documentation" + text: "Configure Test Optimization with Bazel" + - link: "/tests/explorer/" + tag: "Documentation" + text: "Explore Test Results and Performance" + - link: "/tests/troubleshooting/" + tag: "Documentation" + text: "Troubleshooting Test Optimization" +--- + +
Bazel support for Test Optimization is in Preview. Test Impact Analysis is not supported for Bazel.
+ +Datadog provides official Bazel rules for Python Test Optimization. Use `dd_topt_py_test` to configure `pytest` targets to read synced Test Optimization metadata and write payloads during Bazel test execution. + +## Compatibility + +Supported test frameworks: + +- `pytest` + +Use the following minimum versions: + +| Component | Version | +|---|---| +| `datadog-rules-test-optimization` | `>=1.2.0` | +| `datadog-rules-test-optimization-python` | `>=1.2.0` | +| Python runtime example | `>=3.12` | + +The repository that consumes the Bazel rule owns Python toolchains, `pytest`, `ddtrace`, and lockfiles. + +## Prerequisites + +Before setting up Test Optimization for Python tests in Bazel: + +- Configure a Datadog API key in your CI secret store. +- Set `DD_SITE` to your Datadog site, such as `datadoghq.com`. +- Use a Bazel workspace that can resolve the `datadog-rules-test-optimization` module. +- Add `pytest` and `ddtrace` to the Python dependency repository used by the test target. + +## Add modules + +Add the core module and the Python companion module to `MODULE.bazel`: + +```starlark +bazel_dep(name = "datadog-rules-test-optimization", version = "1.2.0") +git_override( + module_name = "datadog-rules-test-optimization", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", +) + +bazel_dep(name = "datadog-rules-test-optimization-python", version = "1.2.0") +git_override( + module_name = "datadog-rules-test-optimization-python", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", + strip_prefix = "modules/python", +) +``` + +Use the same commit SHA for the core module and the Python companion module. + +## Use `WORKSPACE` mode + +If Bzlmod is disabled, declare the core repository in `WORKSPACE`. Then use the public Python helper to declare the Python companion repository: + +```starlark +load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") + +git_repository( + name = "datadog-rules-test-optimization", + remote = "https://github.com/DataDog/rules_test_optimization.git", + commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", +) + +load( + "@datadog-rules-test-optimization//tools/python:workspace_repositories.bzl", + "datadog_python_test_optimization_workspace_repositories", +) + +datadog_python_test_optimization_workspace_repositories( + rto_commit = "69953536d4ef1252c8181c267d16c61263f0aa4c", + rules_python_repo_name = "rules_python", +) +``` + +The consuming repository owns Python toolchains, `pip_parse` or `pip.parse`, `pytest`, `ddtrace`, and lockfiles. + +## Configure Test Optimization metadata + +Configure a sync repository in `MODULE.bazel`: + +```starlark +topt = use_extension( + "@datadog-rules-test-optimization//tools/core:test_optimization_sync.bzl", + "test_optimization_sync_extension", +) + +topt.test_optimization_sync( + name = "test_optimization_data", + service = "", + runtime_name = "python", + runtime_version = "3.12", +) + +use_repo(topt, "test_optimization_data") +``` + +## Add doctor and uploader targets + +Add a doctor and uploader pair. In large repositories, put these targets in a lightweight package such as `//tools/test_optimization`. + +```starlark +load("@datadog-rules-test-optimization//tools/core:test_optimization_targets.bzl", "dd_test_optimization_targets") + +dd_test_optimization_targets( + name = "test_optimization", + sync_repo_name = "test_optimization_data", + expected_targets = [ + "//python/pkg:pkg_py_test", + ], +) +``` + +This macro creates: + +- `//tools/test_optimization:dd_test_optimization_doctor` +- `//tools/test_optimization:dd_upload_payloads` + +## Configure Python test targets + +Use `dd_topt_py_test` in package `BUILD.bazel` files: + +```starlark +load("@python_deps//:requirements.bzl", "requirement") +load("@datadog-rules-test-optimization-python//:topt_py_test.bzl", "dd_topt_py_test") +load("@test_optimization_data//:export.bzl", "topt_data") + +py_library( + name = "pkg_lib", + srcs = ["main.py"], +) + +dd_topt_py_test( + name = "pkg_py_test", + srcs = glob(["test_*.py"]), + imports = ["example/python/pkg"], + deps = [ + ":pkg_lib", + requirement("ddtrace"), + requirement("pytest"), + ], + topt_data = topt_data, +) +``` + +By default, `dd_topt_py_test` uses `runner_mode = "managed_pytest"`. In this mode, the macro runs the bundled pytest entry point. It sets `PYTEST_ADDOPTS=--ddtrace` when you have not already configured it, and passes the Test Optimization metadata path to the test process. + +## Use a repository-owned pytest wrapper + +Use `runner_mode = "consumer_runner"` when your repository already owns a Python test wrapper: + +```starlark +load("@python_deps//:requirements.bzl", "requirement") +load("@datadog-rules-test-optimization-python//:topt_py_test.bzl", "dd_topt_py_test") +load("@test_optimization_data//:export.bzl", "topt_data") +load("//tools/build:py_test.bzl", "repo_py_test") + +dd_topt_py_test( + name = "pkg_py_test", + py_test_rule = repo_py_test, + runner_mode = "consumer_runner", + module_identifier = "example.python.pkg", + srcs = glob(["test_*.py"]), + deps = [ + requirement("ddtrace"), + requirement("pytest"), + ], + topt_data = topt_data, +) +``` + +In `consumer_runner` mode, the repository-owned wrapper must execute pytest and keep the environment passed by `dd_topt_py_test`. If your wrapper sets `PYTEST_ADDOPTS`, include `--ddtrace` unless you intentionally disable the plugin with `--no-ddtrace`. + +## Run and validate tests + +Add the required Bazel config: + +```text +common:test-optimization --repo_env=DD_API_KEY +common:test-optimization --repo_env=DD_SITE +common:test-optimization --repo_env=DD_GIT_REPOSITORY_URL +common:test-optimization --repo_env=DD_GIT_BRANCH +common:test-optimization --repo_env=DD_GIT_TAG +common:test-optimization --repo_env=DD_GIT_COMMIT_SHA +common:test-optimization --repo_env=DD_PR_NUMBER +test:test-optimization --remote_download_outputs=all +``` + +Run tests, validate local payloads, validate enrichment, and upload: + +```bash +bazel test --config=test-optimization //python/pkg:pkg_py_test +bazel run --config=test-optimization //tools/test_optimization:dd_test_optimization_doctor +bazel run --config=test-optimization //tools/test_optimization:dd_upload_payloads -- --dry-run --validate-enrichment +DD_API_KEY= DD_SITE= bazel run --config=test-optimization //tools/test_optimization:dd_upload_payloads +``` + +Do not pass upload credentials, upload endpoints, or `DD_GIT_*` values through `--test_env`. + +## Multi-service setup + +Use the multi-sync extension when one Bazel workspace reports Python tests for more than one Datadog service: + +```starlark +topt = use_extension( + "@datadog-rules-test-optimization//tools/core:test_optimization_multi_sync.bzl", + "test_optimization_multi_sync_extension", +) + +topt.test_optimization_multi_sync( + name = "test_optimization_data", + services = ["py-service-a", "py-service-b"], + runtime_name = "python", + runtime_version = "3.12", +) + +use_repo( + topt, + "test_optimization_data", + "test_optimization_data_py_service_a", + "test_optimization_data_py_service_b", +) +``` + +In test targets, pass `topt_data_by_service` and select the service: + +```starlark +load("@test_optimization_data//:export.bzl", "topt_data_by_service") + +dd_topt_py_test( + name = "pkg_py_test", + srcs = glob(["test_*.py"]), + imports = ["example/python/pkg"], + deps = [ + requirement("ddtrace"), + requirement("pytest"), + ], + topt_data = topt_data_by_service, + topt_service = "py_service_a", +) +``` + +## Known limitations + +- Test Impact Analysis is not supported for Bazel. +- `dd_topt_py_test` supports `pytest`. +- In `consumer_runner` mode, the repository-owned wrapper must run pytest with the ddtrace plugin enabled. + +## Troubleshooting + +### The target passes but no tests appear in Datadog + +This can happen when a custom wrapper does not run pytest with the ddtrace plugin. It can also happen when the wrapper does not preserve the environment from `dd_topt_py_test`, or uses a raw `py_test` target instead of the Datadog macro. + +To fix this issue: + +1. Confirm the target uses `dd_topt_py_test`. +1. If you use `consumer_runner`, confirm the wrapper runs pytest with `--ddtrace`. +1. Run the doctor target before upload. +1. Run uploader dry-run enrichment validation before the real upload. + +### The doctor reports missing payloads + +This can happen when the instrumented target did not run, test outputs were not downloaded locally, or payloads were not written under `TEST_UNDECLARED_OUTPUTS_DIR`. + +To fix this issue: + +1. Run the exact `dd_topt_py_test` target before running the doctor. +1. For remote execution, add `test:test-optimization --remote_download_outputs=all` to `.bazelrc`. +1. Confirm the target depends on both `pytest` and `ddtrace`. + +### `consumer_runner` fails during analysis + +This can happen when `runner_mode = "consumer_runner"` uses the base `py_test` rule without a repository-owned pytest runner. + +To fix this issue, pass your repository's Python test wrapper with `py_test_rule`, pass an explicit `main` that runs pytest with the ddtrace plugin, or use `runner_mode = "managed_pytest"`. + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} diff --git a/content/en/tests/setup/go.md b/content/en/tests/setup/go.md index 4f10a087271..3bbce9ac1ea 100644 --- a/content/en/tests/setup/go.md +++ b/content/en/tests/setup/go.md @@ -28,6 +28,8 @@ Supported test frameworks: - `testing` package +If you run Go tests with Bazel, use Datadog's official [Bazel rules for Go tests][bazel-go]. + ## Configuring reporting method To report test results to Datadog, you need to configure the Datadog Go library: @@ -146,6 +148,7 @@ $ go test -toolexec 'orchestrion toolexec' -race . ``` [1]: https://github.com/datadog/orchestrion +[bazel-go]: /tests/setup/bazel/go/ ## Further reading {{< partial name="whats-next/whats-next.html" >}} diff --git a/content/en/tests/setup/java.md b/content/en/tests/setup/java.md index a1d3a58fbb2..ae8d3b1aa79 100644 --- a/content/en/tests/setup/java.md +++ b/content/en/tests/setup/java.md @@ -52,8 +52,11 @@ Supported build systems: |---|---| | Gradle | >= 2.0 | | Maven | >= 3.2.1 | +| Bazel | Preview | -Other build systems, such as Ant, Bazel, or SBT are supported with the following limitations: +Use Datadog's official [Bazel rules for Java tests][bazel-java] to configure Test Optimization for Java test targets in Bazel. + +Other build systems, such as Ant or SBT, are supported with the following limitations: - Automatic coverage configuration and reporting is not supported. - When building a multi-module project, every module is reported in a separate trace. @@ -571,3 +574,4 @@ To disable all integrations, augment the list of `-javaagent` arguments with `dd [8]: /tests/#parameterized-test-configurations [9]: https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests-display-names [10]: /tracing/trace_collection/compatibility/java#integrations +[bazel-java]: /tests/setup/bazel/java/ diff --git a/content/en/tests/setup/python.md b/content/en/tests/setup/python.md index 63762ef73d3..da508058f88 100644 --- a/content/en/tests/setup/python.md +++ b/content/en/tests/setup/python.md @@ -36,6 +36,8 @@ Supported test frameworks: | `pytest-benchmark` | >= 3.1.0 | | `unittest` | >= 3.7 | +If you run Python tests with Bazel, use Datadog's official [Bazel rules for Python tests][bazel-python]. + ## Configuring reporting method To report test results to Datadog, you need to configure the Datadog Python library: @@ -489,3 +491,4 @@ Datadog recommends you use up to one process at a time to prevent affecting test [1]: /tracing/trace_collection/dd_libraries/python/ [2]: /getting_started/tagging/unified_service_tagging [3]: /tracing/trace_collection/library_config/python/?tab=containers#configuration +[bazel-python]: /tests/setup/bazel/python/