[tbuffington7]

What does this PR do? What is the motivation?

Document how holdouts measure the combined effect of shipped changes. Cover setup, inclusion and analysis phases, exposure rules, and warehouse exposure models. Link the guide from experiment concepts, launch instructions, and results.

EX-3939

Merge readiness

• Ready for merge

AI assistance

Used Codex to draft and revise the documentation, review the changes, and run checks.

For Datadog employees:

• Your branch name MUST follow the <name>/<description> convention and include the forward slash (/). If you've already created your PR with an incorrect branch name, please rename your branch and open a fresh PR.
• New: Comment with /review to run an automated check that catches common issues before a Documentation team member reviews your PR.

Additional notes

Validation:

• Vale 3.9.1: no findings on added lines.
• Repository pre-commit checks passed.
• Frontmatter, local links, section anchors, and SQL example checks passed.
• Four independent reviews completed.

The SQL example was checked with a local fixture, not a live warehouse.

[github-actions]

Preview links (active after the build_preview check completes)

New or renamed files

• https://docs-staging.datadoghq.com/tbuffington7/EX-3939-holdout-documentation/experiments/holdouts

Modified Files

• https://docs-staging.datadoghq.com/tbuffington7/EX-3939-holdout-documentation/experiments/concepts/
• https://docs-staging.datadoghq.com/tbuffington7/EX-3939-holdout-documentation/experiments/plan_and_launch_experiments
• https://docs-staging.datadoghq.com/tbuffington7/EX-3939-holdout-documentation/experiments/reading_results

[maycmlee]

Created DOCS-16171 for review.

[git-thuerk-done]

Thanks for documenting holdouts! My main feedback is about navigation, structure, and repetition. Line-level suggestions are inline.

BLOCKING --> Add the page to the left nav under Concepts. experiments/holdouts doesn't have an entry in hugo/config/_default/menus/main.en.yaml, so it publishes without a left-nav link. The Concepts page already links to it, so nest it under Concepts, after Cumulative Impact:

    - name: Holdouts
      url: experiments/holdouts/
      parent: pa_experiments_concepts
      identifier: pa_experiments_holdouts
      weight: 5

Front-load requirements. The main constraints (Datadog Feature Flags, the same subject type, and a consistent subject ID on line 23, plus the Eligible experiments list on line 95) appear after a lot of concept material and after setup starts. Consider a short Requirements section near the top, or at least before Set up a holdout, so readers know right away whether holdouts work for them.

Reduce repetition. The overview (lines 8–21) says "combined effect of shipped changes" three times across several single-sentence paragraphs. Consider merging them into one or two paragraphs plus the two-group list. Across the page, "early directional read" and "both groups receive control until rollout completes" each appear four or five times (lines 21, 57, 61, 65, 81, 133, 149). Explain each idea once, where it fits best, and link to it from other sections.

Otherwise other comments are inline, let me know if you have any questions and when this is ready for re-review!