Skip to content

fix(ffe): report rejected flags as parse errors - #2339

Open
leoromanovsky wants to merge 3 commits into
mainfrom
agent/ffe-rejected-flag-errors
Open

fix(ffe): report rejected flags as parse errors#2339
leoromanovsky wants to merge 3 commits into
mainfrom
agent/ffe-rejected-flag-errors

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Motivation

Malformed per-flag UFC configuration is already isolated during ingestion: libdatadog stores a rejection error for that key while compiling valid neighboring flags normally. The FFI translation incorrectly exposed that rejection as a normal DEFAULT result with no error code, hiding a configuration bug from callers.

This implements the contract merged in DataDog/ffe-system-test-data#26: evaluating a rejected key returns the caller default with ERROR / PARSE_ERROR, while valid neighboring flags remain usable.

Changes

  • Map FlagConfigurationInvalid to the FFI ERROR reason and PARSE_ERROR error code.
  • Add regression coverage proving an invalid regex flag does not poison a valid neighboring flag.
  • Bump the canonical fixture submodule to the revision merged in DataDog/ffe-system-test-data#26.
  • Update the canonical adapter to expect PARSE_ERROR for rejected keys.
  • Compare semantic versions by precedence so build metadata does not change equality or ordering.
  • Correct the regex-ingestion documentation to match the parser behavior.

Decisions

  • Keep validation in libdatadog so the same Rust regex and UFC parsers used for evaluation own rejection decisions.
  • Retain only the per-key rejection error in the compiled configuration, not the malformed flag object.
  • Preserve FLAG_NOT_FOUND for keys absent from the configuration.
  • Use SemVer precedence comparisons, which ignore build metadata as required by the specification.

Validation

  • cargo test -p libdd-ffe
  • cargo test -p libdd-ffe-ffi rejected_flag_is_exposed_as_parse_error
  • cargo test -p libdd-ffe-test-suite --test canonical_fixtures
  • git diff --check

@github-actions

github-actions Bot commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 929 documentation warning(s) found

📦 libdd-ffe-ffi - 545 warning(s)

📦 libdd-ffe-test-suite - ✅ No warnings

📦 libdd-ffe - 384 warning(s)

Updated: 2026-08-11 01:38:08 UTC | Commit: bd144cf | missing-docs job results

@github-actions

github-actions Bot commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 10 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-ffe-ffi - 3 error(s)

Show output 
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:30:1
   │
30 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── (dev) libdd-ffe v0.1.0
         └── libdd-ffe-ffi v1.0.2

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:194:1
    │
194 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── (dev) libdd-ffe v0.1.0
      │   │   │   └── libdd-ffe-ffi v1.0.2
      │   │   └── libdd-remote-config v2.0.0
      │   │       ├── libdd-ffe v0.1.0 (*)
      │   │       └── (dev) libdd-remote-config v2.0.0 (*)
      │   ├── libdd-common-ffi v40.0.0
      │   │   └── libdd-ffe-ffi v1.0.2 (*)
      │   ├── libdd-ffe v0.1.0 (*)
      │   └── libdd-remote-config v2.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-common v5.1.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:260:1
    │
260 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── libdd-remote-config v2.0.0
          ├── libdd-ffe v0.1.0
          │   └── libdd-ffe-ffi v1.0.2
          └── (dev) libdd-remote-config v2.0.0 (*)

advisories FAILED, bans ok, sources ok

📦 libdd-ffe-test-suite - 4 error(s)

Show output 
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:24:1
   │
24 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── (dev) libdd-ffe v0.1.0
         └── (dev) libdd-ffe-test-suite v40.0.0

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:55:1
   │
55 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     └── (dev) libdd-ffe-test-suite v40.0.0

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:185:1
    │
185 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── (dev) libdd-ffe v0.1.0
      │   │   │   └── (dev) libdd-ffe-test-suite v40.0.0
      │   │   └── libdd-remote-config v2.0.0
      │   │       ├── libdd-ffe v0.1.0 (*)
      │   │       └── (dev) libdd-remote-config v2.0.0 (*)
      │   ├── libdd-ffe v0.1.0 (*)
      │   └── libdd-remote-config v2.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-common v5.1.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:249:1
    │
249 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── libdd-remote-config v2.0.0
          ├── libdd-ffe v0.1.0
          │   └── (dev) libdd-ffe-test-suite v40.0.0
          └── (dev) libdd-remote-config v2.0.0 (*)

advisories FAILED, bans ok, sources ok

📦 libdd-ffe - 3 error(s)

Show output 
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:22:1
   │
22 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── (dev) libdd-ffe v0.1.0

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:164:1
    │
164 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── (dev) libdd-ffe v0.1.0
      │   │   └── libdd-remote-config v2.0.0
      │   │       ├── libdd-ffe v0.1.0 (*)
      │   │       └── (dev) libdd-remote-config v2.0.0 (*)
      │   ├── libdd-ffe v0.1.0 (*)
      │   └── libdd-remote-config v2.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-common v5.1.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:225:1
    │
225 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── libdd-remote-config v2.0.0
          ├── libdd-ffe v0.1.0
          └── (dev) libdd-remote-config v2.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-08-11 01:38:15 UTC | Commit: bd144cf | dependency-check job results

@datadog-official

datadog-official Bot commented Aug 11, 2026 •
edited by datadog-prod-us1-6 Bot
Loading

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 76.05% (+0.07%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d20319f | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-08-11 01:48:08

Comparing candidate commit d20319f in PR branch agent/ffe-rejected-flag-errors with baseline commit 130ed6b in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 13 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:sdk_test_data/rules-based

  • 🟥 execution_time [+16.712µs; +18.904µs] or [+13.097%; +14.814%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz d20319f 1786412181 agent/ffe-rejected-flag-errors
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time 5.415ms 5.431ms ± 0.007ms 5.431ms ± 0.004ms 5.435ms 5.443ms 5.448ms 5.460ms 0.53% 0.609 1.092 0.13% 0.000ms 1 200
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput 457912.066op/s 460326.498op/s ± 579.493op/s 460332.154op/s ± 380.188op/s 460740.992op/s 461147.716op/s 461529.317op/s 461651.637op/s 0.29% -0.598 1.065 0.13% 40.976op/s 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time 7.926ms 7.941ms ± 0.008ms 7.940ms ± 0.003ms 7.943ms 7.951ms 7.974ms 7.988ms 0.60% 2.662 12.098 0.10% 0.001ms 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput 125193.278op/s 125930.889op/s ± 120.718op/s 125945.351op/s ± 53.974op/s 125998.816op/s 126071.336op/s 126107.091op/s 126173.996op/s 0.18% -2.642 11.962 0.10% 8.536op/s 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time 180.407µs 185.984µs ± 39.581µs 183.049µs ± 0.150µs 183.296µs 184.405µs 185.064µs 742.894µs 305.84% 14.032 194.925 21.23% 2.799µs 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput 134608.706op/s 543843.324op/s ± 29130.940op/s 546300.783op/s ± 447.661op/s 546694.519op/s 547127.890op/s 550307.959op/s 554303.369op/s 1.46% -13.966 193.696 5.34% 2059.869op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time [5.430ms; 5.432ms] or [-0.017%; +0.017%] None None None
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput [460246.186op/s; 460406.811op/s] or [-0.017%; +0.017%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time [7.940ms; 7.942ms] or [-0.013%; +0.013%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput [125914.158op/s; 125947.619op/s] or [-0.013%; +0.013%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time [180.498µs; 191.470µs] or [-2.949%; +2.949%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput [539806.056op/s; 547880.592op/s] or [-0.742%; +0.742%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz d20319f 1786412181 agent/ffe-rejected-flag-errors
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time 21.492ms 21.622ms ± 0.057ms 21.616ms ± 0.037ms 21.657ms 21.729ms 21.758ms 21.814ms 0.92% 0.471 0.107 0.26% 0.004ms 1 200
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput 114603.539op/s 115623.289op/s ± 304.536op/s 115657.708op/s ± 196.142op/s 115832.255op/s 116057.465op/s 116199.579op/s 116322.645op/s 0.57% -0.456 0.086 0.26% 21.534op/s 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time 97.117ms 97.440ms ± 0.291ms 97.397ms ± 0.099ms 97.525ms 97.702ms 97.743ms 100.895ms 3.59% 8.465 97.969 0.30% 0.021ms 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput 9911.342op/s 10262.820op/s ± 29.872op/s 10267.218op/s ± 10.412op/s 10275.659op/s 10289.115op/s 10292.041op/s 10296.885op/s 0.29% -8.214 94.004 0.29% 2.112op/s 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time 589.932µs 591.535µs ± 0.595µs 591.577µs ± 0.443µs 591.863µs 592.517µs 592.839µs 593.494µs 0.32% 0.154 -0.051 0.10% 0.042µs 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput 168493.720op/s 169051.845op/s ± 169.985op/s 169039.638op/s ± 126.559op/s 169186.900op/s 169300.131op/s 169423.322op/s 169510.946op/s 0.28% -0.148 -0.055 0.10% 12.020op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time [21.614ms; 21.630ms] or [-0.037%; +0.037%] None None None
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput [115581.084op/s; 115665.495op/s] or [-0.037%; +0.037%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time [97.400ms; 97.480ms] or [-0.041%; +0.041%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput [10258.680op/s; 10266.960op/s] or [-0.040%; +0.040%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time [591.453µs; 591.618µs] or [-0.014%; +0.014%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput [169028.287op/s; 169075.404op/s] or [-0.014%; +0.014%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz d20319f 1786412181 agent/ffe-rejected-flag-errors
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 141.948µs 145.414µs ± 5.875µs 144.225µs ± 0.743µs 145.071µs 150.727µs 168.141µs 207.899µs 44.15% 7.479 68.451 4.03% 0.415µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [144.599µs; 146.228µs] or [-0.560%; +0.560%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz d20319f 1786412181 agent/ffe-rejected-flag-errors
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 155.946ns 159.414ns ± 2.849ns 158.691ns ± 1.718ns 160.804ns 164.643ns 169.031ns 173.743ns 9.49% 1.555 3.604 1.78% 0.201ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [159.019ns; 159.809ns] or [-0.248%; +0.248%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 130ed6b 1786380770 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time 5.408ms 5.429ms ± 0.015ms 5.428ms ± 0.004ms 5.432ms 5.439ms 5.453ms 5.622ms 3.57% 9.917 121.384 0.28% 0.001ms 1 200
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput 444708.740op/s 460506.524op/s ± 1270.877op/s 460573.185op/s ± 340.083op/s 460928.290op/s 461514.449op/s 461841.601op/s 462243.447op/s 0.36% -9.705 117.835 0.28% 89.865op/s 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time 7.970ms 7.985ms ± 0.008ms 7.984ms ± 0.004ms 7.988ms 7.994ms 8.016ms 8.026ms 0.52% 2.293 9.261 0.10% 0.001ms 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput 124598.699op/s 125232.935op/s ± 119.355op/s 125243.531op/s ± 58.932op/s 125300.921op/s 125372.771op/s 125430.079op/s 125469.024op/s 0.18% -2.276 9.165 0.10% 8.440op/s 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time 182.319µs 183.211µs ± 0.621µs 183.085µs ± 0.164µs 183.333µs 184.167µs 184.639µs 189.517µs 3.51% 6.048 53.713 0.34% 0.044µs 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput 527658.542op/s 545824.008op/s ± 1814.184op/s 546193.409op/s ± 488.925op/s 546586.134op/s 547218.937op/s 547749.592op/s 548488.284op/s 0.42% -5.858 50.977 0.33% 128.282op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time [5.427ms; 5.431ms] or [-0.039%; +0.039%] None None None
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput [460330.393op/s; 460682.655op/s] or [-0.038%; +0.038%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time [7.984ms; 7.986ms] or [-0.013%; +0.013%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput [125216.394op/s; 125249.477op/s] or [-0.013%; +0.013%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time [183.125µs; 183.297µs] or [-0.047%; +0.047%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput [545572.580op/s; 546075.437op/s] or [-0.046%; +0.046%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 130ed6b 1786380770 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time 21.845ms 21.913ms ± 0.032ms 21.910ms ± 0.022ms 21.933ms 21.970ms 21.990ms 22.009ms 0.45% 0.442 -0.011 0.15% 0.002ms 1 200
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput 113592.323op/s 114088.268op/s ± 166.621op/s 114105.718op/s ± 112.575op/s 114206.588op/s 114334.590op/s 114417.786op/s 114444.440op/s 0.30% -0.434 -0.022 0.15% 11.782op/s 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time 95.602ms 96.241ms ± 0.289ms 96.250ms ± 0.146ms 96.389ms 96.677ms 97.035ms 98.205ms 2.03% 1.634 10.334 0.30% 0.020ms 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput 10182.829op/s 10390.694op/s ± 31.067op/s 10389.650op/s ± 15.772op/s 10405.836op/s 10443.565op/s 10454.042op/s 10459.989op/s 0.68% -1.548 9.730 0.30% 2.197op/s 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time 610.222µs 612.628µs ± 0.699µs 612.590µs ± 0.486µs 613.084µs 613.731µs 614.618µs 614.719µs 0.35% 0.141 0.579 0.11% 0.049µs 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput 162676.087op/s 163231.425op/s ± 186.165op/s 163241.326op/s ± 129.592op/s 163360.899op/s 163521.955op/s 163618.100op/s 163874.659op/s 0.39% -0.133 0.578 0.11% 13.164op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time [21.908ms; 21.917ms] or [-0.020%; +0.020%] None None None
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput [114065.175op/s; 114111.360op/s] or [-0.020%; +0.020%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time [96.201ms; 96.281ms] or [-0.042%; +0.042%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput [10386.388op/s; 10394.999op/s] or [-0.041%; +0.041%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time [612.531µs; 612.725µs] or [-0.016%; +0.016%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput [163205.624op/s; 163257.226op/s] or [-0.016%; +0.016%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 130ed6b 1786380770 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 124.706µs 127.606µs ± 5.294µs 126.491µs ± 0.695µs 127.503µs 132.045µs 145.099µs 185.126µs 46.35% 7.809 73.935 4.14% 0.374µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [126.872µs; 128.339µs] or [-0.575%; +0.575%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 130ed6b 1786380770 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 156.875ns 160.430ns ± 2.752ns 159.883ns ± 1.506ns 161.617ns 165.936ns 169.821ns 172.632ns 7.97% 1.507 2.902 1.71% 0.195ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [160.049ns; 160.811ns] or [-0.238%; +0.238%] None None None

@leoromanovsky
leoromanovsky marked this pull request as ready for review August 11, 2026 01:52
@leoromanovsky
leoromanovsky requested a review from a team as a code owner August 11, 2026 01:52
@leoromanovsky
leoromanovsky requested review from btthomas and sameerank and removed request for a team August 11, 2026 01:52
@dd-octo-sts

dd-octo-sts Bot commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 89.19 MB 89.18 MB -0% (-7.32 KB) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.20 MB 8.20 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.06 MB 11.06 MB -0% (-152 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 100.36 MB 100.35 MB -0% (-7.37 KB) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.63 MB 26.62 MB --.01% (-4.00 KB) 💪
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 94.56 KB 94.56 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.41 MB 191.39 MB --.01% (-24.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 1001.43 MB 1001.42 MB -0% (-8.96 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.71 MB 8.71 MB --.01% (-1.00 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 94.56 KB 94.56 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.62 MB 25.62 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.01 MB 51.01 MB -0% (-3.59 KB) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.22 MB 23.21 MB --.01% (-3.50 KB) 💪
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 96.04 KB 96.04 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.33 MB 196.33 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 990.81 MB 990.80 MB -0% (-8.92 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.73 MB 6.73 MB --.01% (-1.00 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 96.04 KB 96.04 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.55 MB 27.55 MB --.02% (-8.00 KB) 💪
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.55 MB 48.55 MB -0% (-3.65 KB) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 79.46 MB 79.45 MB --.01% (-9.23 KB) 💪
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.12 MB 9.12 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 95.12 MB 95.11 MB -0% (-9.20 KB) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.14 MB 11.14 MB -0% (-96 B) 👌

@leoromanovsky
leoromanovsky requested a lite review from Copilot August 11, 2026 16:27
Copilot AI reviewed Aug 11, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates libdd-ffe’s Rust and FFI error semantics so that per-flag ingestion rejections (e.g., invalid regex) are surfaced to callers as parse errors, while leaving neighboring valid flags evaluatable. It also aligns SemVer comparisons with SemVer precedence rules (ignoring build metadata) and updates tests and canonical fixtures to match the revised contract.

Changes:

  • Map FlagConfigurationInvalid to FFI Reason::Error with ErrorCode::ParseError (and update canonical fixtures accordingly).
  • Add regression tests ensuring a rejected/invalid flag does not invalidate evaluation of a valid neighboring flag.
  • Compare semantic versions by precedence (cmp_precedence) so build metadata does not affect comparisons; add coverage for +build variants.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
libdd-ffe/src/rules_based/ufc/models.rs Updates ingestion/evaluation comment to reflect regex compilation during ingestion (reject flag on compile failure).
libdd-ffe/src/rules_based/eval/eval_rules.rs Switches SemVer equality/ordering to precedence comparison and adds tests for build metadata behavior.
libdd-ffe/src/rules_based/eval/eval_assignment.rs Adds regression test proving an invalid flag does not poison a valid neighboring flag.
libdd-ffe/src/rules_based/error.rs Clarifies FlagConfigurationInvalid contract: treat as per-flag parse error without invalidating the rest of the config.
libdd-ffe-test-suite/tests/canonical_fixtures.rs Updates canonical adapter mappings so FlagConfigurationInvalid is treated as ERROR with PARSE_ERROR.
libdd-ffe-ffi/src/assignment.rs Fixes FFI mapping: FlagConfigurationInvalid now yields ParseError, and adds a regression unit test.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

This was referenced Aug 11, 2026
Closed
Draft
Draft
Closed
Draft
sameerank
sameerank approved these changes Aug 11, 2026
View reviewed changes
Comment thread libdd-ffe/src/rules_based/error.rs
TargetingKeyMissing,

/// Failed to parse configuration. This should normally never happen and is likely a signal
/// that you should update SDK.

@sameerank sameerank Aug 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this comment could also be updated with something like "Failures encountered while compiling an individual flag are converted to FlagConfigurationInvalid at the per-flag ingestion boundary."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@sameerank sameerank sameerank approved these changes
@btthomas btthomas Awaiting requested review from btthomas btthomas is a code owner automatically assigned from DataDog/feature-flagging-and-experimentation-sdk

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@leoromanovsky @sameerank