feature: brokerage order polling service - #9696
Draft
Romazes wants to merge 10 commits into
Draft
Conversation
- one core service polls order updates when the stream is missing, lost or silent - two modes by constructor: read one watched brokerage order id, or all orders - brokerages convert their wire model into a shared BrokerOrderState and one core diff emits the events - grounded in a survey of eight brokerage plugins
…ge queue - BrokerageMessageQueue owns the lock, buffer and dispatch, raised through a MessageReceived event - BrokerageConcurrentMessageHandler<T> becomes a thin wrapper with the same public API - add RegisterMessageType<TMessage> and a generic HandleNewMessage<TMessage>, so a second message source can queue behind the same lock instead of a separate, independent one - add BrokerageMessageQueueTests covering the shared lock, an unmatched message type, and a registered second type
- abstract BrokerageOrderPollingService with per-order-id and all-orders modes: watch registry, state compare, watch timeout, repeated-failure warning - plugins seed the registry before Start, so the first sweep repeats nothing the stream already reported - ADR: seed-before-start handover and the survey of streaming plugins that fit it - tests: 25 polling service, 4 message queue
- guard, buffered-message processing, one seed per open Lean order, Start - in the only safe order - both callbacks optional and nullable
- add non-generic BrokerageConcurrentMessageHandler with per-type Register, reusing the generic handler by composition - restore BrokerageConcurrentMessageHandler<T> to master and drop BrokerageMessageQueue - service constructor takes the handler, registers ProcessOrderState and routes the states; SeedAndStart drains the handler internally - update the ADR wiring, seed-before-start and risk sections, and note the CharlesSchwab pilot
- a plain place watches the main id; the first sweep assigns the leg ids by symbol and reports the submit through the diff - the assignment is one shared method with the stream's OrderAccepted and releases the three minute place wait in both modes - the pilot note lists the leg id assignment and the replace reporting as what stays in the plugin
- watch replacement watches the new id of a replace and drops the replaced one - the diff's first state for a marked id reports the update submit - the new id starts with no fill state; carry-over brokers seed instead
- survey of nine sibling plugins' update paths, checked line by line - replacement watch design, the no-wait rule, and the rejected event reuse
- status entries: pilot verified live on real accounts; public.com is the second plugin on the service - pricing text matches what shipped: public keeps its change-of-average recovery inside its mapping - rollout item 3 trimmed to the one intentional behavior change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A reusable order polling service in Lean core, so a brokerage plugin can read the order lifecycle over HTTP when it has no stream, when its stream dies, or when the broker never replied about one order.
Brokerages/Services/: abstractBrokerageOrderPollingService— the loop, the watch registry, the state compare, the watch timeout and the repeated-failure warning — with the two read modes as subclasses:PerOrderIdPollingService(get-order endpoint) andAllOrdersPollingService(bulk endpoint). The plugin converts its wire model intoBrokerOrderStatesnapshots; the shared compare reports submits, fills and closes, each exactly once.BrokerageMessageQueue: the lock and queue behindBrokerageConcurrentMessageHandler<T>, split out so one handler can carry a second message type (RegisterMessageType<TMessage>). Polled snapshots queue behind an order request that holds the stream lock, so a fill can never be reported before its submit. The generic handler keeps its exact public surface — the thirteen plugins using it today do not change.SeedAndStart(drainBufferedMessages, seed): the handover from a dying stream in one call, in the only safe order — process what the stream already delivered, seed one watch per open Lean order, then start — so the first sweep repeats nothing the stream reported.The full design, the alternatives considered and the plugin survey live in
Documentation/ADR/0001-brokerage-order-polling-service.md, included in this PR.Related PR(s)
QuantConnect/Lean.Brokerages.CharlesSchwab#107 — the first adopter (draft): Charles Schwab replaces its own polling fallback with this service.
Related Issue
N/A
Motivation and Context
The same polling loop and state compare is already written three times — Public.com, Tradier and Charles Schwab — and InteractiveBrokers blocks the order thread for up to five minutes waiting for a submission event instead. One core service replaces the copies and gives every plugin the same behavior.
Requires Documentation Change
No — the design document ships in this PR (
Documentation/ADR/0001-brokerage-order-polling-service.md).How Has This Been Tested?
BrokerageOrderPollingServiceTests— 29 tests: submit exactly once, cumulative fill compare (never repeats, never invents a price), combo fills split byGroupOrderManager, terminal exactly once, seeded watches, watch timeout, failure warning once per outage,SeedAndStartordering and guards.BrokerageMessageQueueTests— 4 tests: two message types share one lock and one queue.BrokerageConcurrentMessageHandlerTests— the existing 8 tests pass unchanged; the handler's public surface is untouched.Types of changes
Checklist:
bug-<issue#>-<description>orfeature-<issue#>-<description>