[fix][broker] Prevent partition expansion from inheriting delayed-delivery bucket state#26179
Conversation
…ivery bucket state
3613eb5 to
6fc0a4b
Compare
| Map<String, String> subscriptionProperties = | ||
| filterSubscriptionPropertiesForPartitionExpansion( | ||
| entry.getValue().getSubscriptionProperties()); |
There was a problem hiding this comment.
Filtering here addresses partition expansion, but since #26159, the Admin getSubscriptionProperties endpoint returns raw cursor properties, including #pulsar.internal.*, while updateSubscriptionProperties delegates to ManagedCursorImpl#setCursorProperties, which rejects maps containing that prefix. Consequently, a GET → modify → PUT cycle can fail once bucket state exists. Should we note a follow-up to restore the user/internal property boundary for instance, by exposing filtered user properties through the Admin API while keeping raw cursor properties accessible only to broker internals?
There was a problem hiding this comment.
Thanks for pointing this out. I agree this is a separate issue around the public API boundary. Since it is outside the partition-expansion fix in this PR, I would prefer to track it in a follow-up issue/PR.
Fixes #26178
Motivation
When increasing partitions, Pulsar copies durable subscription properties from
partition-0 to new partitions.
Bucket delayed delivery stores cursor-local snapshot metadata as
#pulsar.internal.delayed.bucket*. Copying that state makes a new partitionrecover delayed-message indexes that belong to partition-0, which can result in
incorrect delayed-delivery state and invalid replay attempts.
Modifications
Filter
#pulsar.internal.*properties before creating subscriptions on newpartitions. User-defined subscription properties are still copied.
Existing cursor-property and stats exposure is unchanged.
Verifying this change
tracker and verifies it recovers zero delayed messages.
./gradlew --no-daemon --console=plain :pulsar-broker:test --tests org.apache.pulsar.broker.admin.IncrementPartitionsTest --tests org.apache.pulsar.broker.service.persistent.BucketDelayedDeliveryTest./gradlew --no-daemon --console=plain :pulsar-broker:checkstyleMain :pulsar-broker:checkstyleTestDoes this pull request potentially affect one of the following parts: