feat(NODE-7624): support baseBackoffMS and update client backpressure backoff

[PavelSafronov]

Description

Summary of Changes

Add support for baseBackoffMS to existing client backpressure drivers.
This change also alters how backoffs are calculated and changes the backpressure wire representation.

Release Highlight

Updated Intelligent Workload Management

Improved performance for MongoDB 9.0's Intelligent Workload Management (IWM) by only retrying overload errors when doing so is expected to not worsen server conditions

Double check the following

• Lint is passing (npm run check:lint)
• Self-review completed using the steps outlined here
• PR title follows the correct format: type(NODE-xxxx)[!]: description
  • Example: feat(NODE-1234)!: rewriting everything in coffeescript
• Changes are covered by tests
• New TODOs have a related JIRA ticket

[Copilot]

Pull request overview

Adds support for server-provided baseBackoffMS to the client backpressure retry logic, updates the exponential backoff calculation to match the revised spec behavior, and updates handshake backpressure metadata to the new wire representation.

Changes:

• Introduces calculateBaseBackoffMS() and uses it to compute overload retry backoff durations in executeOperation.
• Updates retry backoff math in transactions and client-backpressure prose tests to reflect the new backoff schedule.
• Changes handshake backpressure metadata from true to the new '2' representation and updates related tests.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/unit/operations/execute_operation.test.ts	Adds unit tests for calculateBaseBackoffMS() behavior.
test/spec/client-backpressure/README.md	Updates prose spec text for revised backoff schedule and adds baseBackoffMS prose steps.
test/integration/transactions-convenient-api/transactions-convenient-api.prose.test.ts	Adjusts expected retry-backoff timing for transaction convenient API prose test.
test/integration/mongodb-handshake/mongodb-handshake.prose.test.ts	Updates handshake backpressure assertions to expect '2'.
test/integration/client-backpressure/client-backpressure.prose.test.ts	Updates backoff timing assertions and adds an integration test for server-provided baseBackoffMS.
src/sessions.ts	Updates transaction retry backoff exponent to match new attempt numbering.
src/operations/execute_operation.ts	Exposes base backoff constant, adds calculateBaseBackoffMS(), and updates overload retry backoff calculation.
src/cmap/connect.ts	Updates handshake document backpressure metadata type/value to '2'.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[PavelSafronov]

Good point, addressed.

[dariakp]

do we actually apply useBigInt user preference to the deserialization of the whole doc? I thought we used our own settings for deserializing non-user-data in responses?

[PavelSafronov]

Nope, in most cases we use the BSON settings on the client, like with find:

  find(
    filter: Filter<TSchema> = {},
    options: FindOptions & Abortable = {}
  ): FindCursor<WithId<TSchema>> {
    return new FindCursor<WithId<TSchema>>(
      this.client,
      this.s.namespace,
      filter,
      resolveOptions(this, options) // <-- here we use parent options
    );
  }

Some paths specifically don't pick up the client options, like db.command(), but in that case the command can use the passed-in options, which can still override the bigint setting:

  async command(command: Document, options?: RunCommandOptions & Abortable): Promise<Document> {
    // Intentionally, we do not inherit options from parent for this operation.
    return await executeOperation(
      this.client,
      new RunCommandOperation(
        this.s.namespace,
        command,
        resolveOptions(undefined, {  // <-- here we ignore parent options
          ...resolveBSONOptions(options),
          timeoutMS: options?.timeoutMS ?? this.timeoutMS,
          session: options?.session,
          readPreference: options?.readPreference,
          signal: options?.signal
        })
      )
    );
  }

[nbbeeken]

You're both correct! Daria's referring to the MongoDBResponse feature, mainly used in the connection.ts file but sometimes propagated to the operation layer. It defers parsing of server responses so we can control how we parse protocol-level BSON separately from user-level.

That's controlled here:

node-mongodb-native/src/cmap/connection.ts

Lines 582 to 586 in dab09b2

	if (responseType == null) {
	yield (object ??= document.toObject(bsonOptions));
	} else {
	yield document;
	}

And is optional because we needed to progressively adopt it rather than impacting every code path in the driver. But ideally we should eventually always use MongoDBResponse, so we can strongly assert the responses we expect and granualarly control the deserialization of server fields (like say: decoding baseBackoffMS as a bigint everytime)

This is the first instance where we care about a field on the error object and may want to consider how we can enhance the deserialization of the error object to control this field's type at parse time:

node-mongodb-native/src/cmap/connection.ts

Line 563 in dab09b2

throw new MongoServerError((object ??= document.toObject(bsonOptions)));

[nbbeeken]

I approved the PR because this type of "handle multiple type possibilities" isn't new to the driver, and its fine to move BP features along with that status quo in mind. (and all the rest LGTM)

The "MongoDBResponse" solution for typing an error would need a small bit of design. Return types is something we can handle by using a generic to decide based on the input argument (responseType) to the function we can pick the correct return type.

Errors are not reflected in the TS API and if we simply passed a flag we would convert all errors to some new typed format or not, and you do not know when an error is going to be BP related or not until you inspect the code. We should pursue this long term, because protocol BSON and user-data BSON should be separated to increase the future flexibility of BSON and the driver.

[PavelSafronov]

Addressed.

[dariakp]

see question above about whether this is a real scenario

[nbbeeken]

I approved the PR because this type of "handle multiple type possibilities" isn't new to the driver, and its fine to move BP features along with that status quo in mind. (and all the rest LGTM)

The "MongoDBResponse" solution for typing an error would need a small bit of design. Return types is something we can handle by using a generic to decide based on the input argument (responseType) to the function we can pick the correct return type.

Errors are not reflected in the TS API and if we simply passed a flag we would convert all errors to some new typed format or not, and you do not know when an error is going to be BP related or not until you inspect the code. We should pursue this long term, because protocol BSON and user-data BSON should be separated to increase the future flexibility of BSON and the driver.