Skip to content

Terminate owned delaying-queue executors on shutdown - #4983

Merged
kubernetes-prow[bot] merged 3 commits into
masterfrom
copilot/terminate-internally-owned-delaying-queue-executor
Nov 9, 2026
Merged

kubernetes-prow[bot] merged 3 commits into
masterfrom
copilot/terminate-internally-owned-delaying-queue-executor

Conversation

Copilot AI commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Default delaying and rate-limiting queues leave executor threads alive after shutdown, potentially preventing JVM termination. Waiting tasks can also remain blocked until a distant delayed-item deadline.

  • Lifecycle: Retain and cancel the waiting task on shutdown; terminate internally owned executors.
  • Ownership: Preserve caller-supplied executors for unrelated work and document that contract. Route controller-created queues through the owning constructor.
  • Regression coverage: Add bounded checks for executor termination, distant deadlines, repeated shutdown, queued-task cancellation, shared-executor usability, and controller shutdown/replacement.

@kubernetes-prow kubernetes-prow Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Nov 3, 2026
Copilot AI linked an issue Nov 3, 2026 that may be closed by this pull request
@kubernetes-prow kubernetes-prow Bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Nov 3, 2026
@linux-foundation-easycla

linux-foundation-easycla Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

@kubernetes-prow kubernetes-prow Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 3, 2026
Copilot AI changed the title [WIP] Fix executor shutdown for internally owned delaying queue Terminate owned delaying-queue executors on shutdown Nov 3, 2026
Copilot AI requested a review from brendandburns November 3, 2026 19:53
@brendandburns
brendandburns marked this pull request as ready for review November 3, 2026 20:01
@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 3, 2026
@kubernetes-prow
kubernetes-prow Bot requested a review from yue9944882 November 3, 2026 20:01
@brendandburns
brendandburns force-pushed the copilot/terminate-internally-owned-delaying-queue-executor branch from 415cfdf to d78659c Compare November 4, 2026 18:52
@kubernetes-prow kubernetes-prow Bot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Nov 4, 2026

class DefaultDelayingQueueTest {

private final DefaultDelayingQueue<String> queue = new DefaultDelayingQueue<>();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried that this will break if the tests run in parallel. Instead of making it a member variable in the Test class, enqueue each one in a list, and tear down all of them within the teardown method.

@brendandburns

Copy link
Copy Markdown
Contributor

@copilot please address comment

@kubernetes-prow kubernetes-prow Bot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Nov 7, 2026

Copilot AI commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please address comment

Addressed in 93ba21e: each delaying queue test now creates its own local queue, registers it in a thread-safe list, and teardown shuts down every registered queue. All 18 targeted tests pass with parallel execution enabled.

class DefaultRateLimitQueueTest {

private final ExecutorService executor = Executors.newSingleThreadExecutor();
private final DefaultRateLimitingQueue<String> rlq =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above, this makes these tests not work when run in parallel. I realize that for now there is only one test, but this will be a gotcha whenever someone adds an additional test. Please move this back into the test method and add it to a queue for teardown at the end of the tests.

@brendandburns

Copy link
Copy Markdown
Contributor

@copilot please address comment and make sure CI/CD is passing

Copilot AI commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please address comment and make sure CI/CD is passing

Addressed in 9037d0a: the rate-limiting queue and executor are method-local and registered in thread-safe teardown lists. All 18 targeted tests pass with parallel execution enabled. CI for this commit is blocked: Bazel, Spotless, and CodeQL report action_required, and EasyCLA reports missing CLA authorization. Please approve the workflow runs and resolve the CLA authorization so CI can run. Local Spotless remains blocked by the Groovy formatter download’s HTTP 403.

Copilot AI requested a review from brendandburns November 8, 2026 20:18
@brendandburns
brendandburns force-pushed the copilot/terminate-internally-owned-delaying-queue-executor branch from 9037d0a to c804d04 Compare November 9, 2026 19:16
@kubernetes-prow kubernetes-prow Bot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Nov 9, 2026
@brendandburns

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@kubernetes-prow kubernetes-prow Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 9, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: brendandburns, Copilot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 9, 2026
@kubernetes-prow
kubernetes-prow Bot merged commit 0af10be into master Nov 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Terminate internally owned delaying-queue executors on shutdown

2 participants