SelectorParams mempool policy#47
Conversation
21d52e8 to
c98521b
Compare
noahjoeris
left a comment
There was a problem hiding this comment.
I think the policy checks should be integrated more directly into the normal transaction-building flow.
Right now the caller has to remember to do a separate post-build check:
let policy = MempoolPolicy { tip_height, tip_mtp };
policy.check_all(&selection, &tx)?;This is easy for callers to forget. It also asks them to provide both a Selection and a Transaction, even though the transaction should be derived from Selection + PsbtParams. A checked path should instead construct the transaction itself and validate exactly the transaction the crate is going to return.
Suggested shape:
-
Output checks in
SelectorParamsBuilderbuild()could validate against a default policy.build_with_policy(policy)could validate against a customMempoolPolicy.- For unchecked, e.g.
build_unchecked()
-
Final transaction checks should happen in
Selection.create_psbt(...)defaults to e.g. Core v30- Add
create_psbt_with_policy(policy)andcreate_psbt_unchecked() - This avoids a public
check_all(&Selection, &Transaction)API and makesSelectionTxMismatchunnecessary
-
So I think we should have some form of abstraction/config over policy.
- No hardcoded policy values but make them part of MempoolPolicy struct.
- Also chain-tip data is an unrelated runtime input that should be passed alongside, not embedded
- Use
MempoolPolicy::bitcoin_core_v30()as the default, for example. - Users should be able to pass a different policy if their node has different relay settings.
- Then this policy can be passed into both
SelectorParamsBuilder::build_with_policy(...)andSelection::create_psbt_with_policy(...).
Let me know what you think.
|
Thank you for the review. I agree with all of this. The two-layer split made sense when I wrote it, but you're right that the manual policy.check_all(&selection, &tx) step is invisible at the type level. Nothing prompts the caller to run it, and accepting both a Selection and a Transaction lets the two drift. I will fold the validation into construction and have create_psbt_with_policy build the tx it validates. |
c98521b to
97c1bfd
Compare
There was a problem hiding this comment.
Nice, thanks for updating 🔥
I’d wait now for a Concept ACK on this direction by @ValuedMammal before spending more time on details.
- One API point: I’d prefer to keep
create_psbt(...). It could either preserve the old unchecked behavior for compatibility, or become checked by default ascreate_psbt(params, tip)usingMempoolPolicy::default(). I think we should discuss that explicitly. Right now most examples moved tocreate_psbt_unchecked(...), which makes the unchecked path look like the normal path. - Also,
MempoolPolicyonly models part of Core v30 policy for now, so we should expect it to grow as more checks are added.
|
I did a first pass. Overall I like the idea and it appears to be well implemented. Biggest concern is the long term API stability as policies change and grow outdated. Pros
Cons
|
| pub fn is_standard_script(script: &Script) -> bool { | ||
| script.is_p2pk() | ||
| || script.is_p2pkh() | ||
| || script.is_p2sh() | ||
| || script.is_p2wpkh() | ||
| || script.is_p2wsh() | ||
| || script.is_p2tr() | ||
| || script.is_op_return() | ||
| } |
There was a problem hiding this comment.
We could collapse the segwit/taproot conditions to script.is_witness_program(), which should also cover pay-to-anchor?
| /// Output-side standardness checks (dust, `OP_RETURN`, standard script | ||
| /// types) are evaluated against `policy`. The dust feerate used by the | ||
| /// change-policy computation also defaults to `policy.dust_relay_feerate` | ||
| /// unless explicitly overridden via [`Self::dust_relay_feerate`]. |
There was a problem hiding this comment.
Confused why the dust-feerate of the params would override the MempoolPolicy when we're supposed to be validating the specified params against a chosen policy.
There was a problem hiding this comment.
This will be removed in the next push, along with MempoolPolicy itself.
Thank you for the review. I overshot the scope. Re-reading the original TODO, this is where I got the misconception - "error on anything that doesn't satisfy mempool policy". I took that literally and tried to build toward it. I can see that this implementation was tending toward tx broadcast preparation, which is outside the scope of |
a40d517 to
a64761b
Compare
Maybe instead of default, we just have constructors which construct MempoolPolicy of certain Bitcoin core versions |
noahjoeris
left a comment
There was a problem hiding this comment.
Thanks for updating again! 🔥 I left some comments.
Could you outline where these checks come from and why we picked this subset? Are there other checks we are intentionally leaving out?
| // The constructed tx is unsigned, so add satisfaction weights and the | ||
| // segwit marker/flag overhead to estimate the signed weight. | ||
| let satisfaction: Weight = self | ||
| .inputs | ||
| .iter() | ||
| .map(|i| Weight::from_wu(i.satisfaction_weight())) | ||
| .sum(); | ||
| let segwit_overhead = if self.inputs.iter().any(|i| i.is_segwit()) { | ||
| Weight::from_wu(2) | ||
| } else { | ||
| Weight::ZERO | ||
| }; | ||
| let weight = psbt.unsigned_tx.weight() + satisfaction + segwit_overhead; | ||
|
|
||
| if weight > Weight::from_wu(MAX_STANDARD_TX_WEIGHT as u64) { | ||
| return Err(CreatePsbtError::MaxStandardTxWeightExceeded { weight }); | ||
| } |
There was a problem hiding this comment.
With this we don't have an unchecked path anymore for non standard txs right?
Also create_psbt is getting quite big. Should we add helper functions for the checks (weight, negativefee)?
There was a problem hiding this comment.
We've only covered weight and output-side checks.
Helper functions have been added.
| script.is_p2pk() | ||
| || script.is_p2pkh() | ||
| || script.is_p2sh() | ||
| || script.is_witness_program() |
There was a problem hiding this comment.
I'm not sure script.is_witness_program() is correct here. It allows scripts in the range of 2-40 bytes length. Which could be non-standard.
Maybe it's better to add them directly.
script.is_p2wpkh()
|| script.is_p2wsh()
|| script.is_p2tr()
|| script.is_p2a()There was a problem hiding this comment.
script.is_p2a() is not available yet, so i used a local helper
| || script.is_p2pkh() | ||
| || script.is_p2sh() | ||
| || script.is_witness_program() | ||
| || script.is_op_return() |
There was a problem hiding this comment.
I think is_op_return() only checks the first byte which might not be enough.
e.g. OP_RETURN OP_CHECKSIG could pass the check
There was a problem hiding this comment.
Thanks for the catch. I've updated
a64761b to
82d236b
Compare
The whole
After @ValuedMammal's review I've narrowed the scope significantly. The checks agreed upon will be construction invariants pre-selection, which are drawn from Bitcoin Core's |
82d236b to
064d9f9
Compare
064d9f9 to
5aaf4ef
Compare
| /// Total output value exceeds total input value. | ||
| NegativeFee, |
There was a problem hiding this comment.
check_negative_fee is unreachable since the coin selector already errors when there is insufficient funds.
This is essentially an internal logic check exposed as an external API. We should instead just have an assert statement.
There was a problem hiding this comment.
Approach NACK
I'd like to push back on the overall direction here. Mempool policy is being baked into SelectorParams (and into module-level constants), but mempool policy is not a constant — it varies by Bitcoin Core version (P2A became standard in v28, OP_RETURN policy changed materially in v30), by node implementation (Knots, btcd), and by node operator tuning (-dustrelayfee, -datacarriersize, -maxmempool). Encoding one frozen-in-time snapshot inside the selector hides that variation and makes it untunable.
Concretely, the current PR scatters policy in several places:
dust_relay_feerateis a field onSelectorParams.MAX_OP_RETURN_BYTESis apub constinselector.rs.MAX_STANDARD_TX_WEIGHTis hardcoded insideSelection::check_max_standard_weight.- The standard-script allowlist is hardcoded in
is_standard_script.
These are all the same kind of thing, presented as if there's one true answer.
I'd prefer a dedicated StandardnessPolicy type that owns this surface:
#[non_exhaustive]
pub struct StandardnessPolicy {
pub dust_relay_feerate: FeeRate,
pub max_standard_tx_weight: Weight,
pub max_op_return_bytes: usize,
// ...standard output kinds, datacarrier flags, etc.
}
impl StandardnessPolicy {
pub fn bitcoin_core_v28() -> Self { ... }
pub fn bitcoin_core_v29() -> Self { ... }
pub fn bitcoin_core_v30() -> Self { ... }
pub fn check_output(&self, _: &Output) -> Result<(), _>;
pub fn check_input(&self, _: &Input) -> Result<(), _>;
pub fn check_tx(&self, _: &TxTemplate) -> Result<(), _>;
}The name maps 1:1 to Bitcoin Core's IsStandardTx — these are exactly the rules that function checks. RelayPolicy or MempoolPolicy would be fuzzier, since "relay" / "mempool" also cover stateful concerns (min-relay-fee, RBF rules, ancestor limits) that explicitly do not belong here (see point 4 below).
SelectorParams would drop dust_relay_feerate and the standardness checks. Callers can validate outputs pre-selection (policy.check_output), the full shape post-template (policy.check_tx), or skip validation entirely if they're deliberately producing non-standard txs. The selector itself stays policy-agnostic.
A few specific things worth pinning down before any rework:
-
Change-output dust threshold.
to_cs_change_policycurrently consumesdust_relay_feeratedirectly. If policy moves out, either pass&StandardnessPolicyat call time or keep just the feerate field onSelectorParams. I'd lean toward passing the policy — keepsSelectorParamsclean. -
check_inputscope. The standard prevout script type is already covered by checking outputs we constructed. The remaining input-side rules are mostly satisfaction-size limits (scriptSig ≤ 1650 B, P2WSH script ≤ 3600 B, taproot stack item sizes). Worth being concrete in the design — otherwise it's a method that's hard to specify. -
Block on #73.
check_tx(&TxTemplate)is the right surface butTxTemplatedoesn't exist onmasteryet. Land #73 first; that also gives a natural place for a fluenttx_template.check_against(&policy)step. -
Scope ceiling. Be explicit that this models static standardness, not full mempool acceptance. Ancestor/descendant counts, RBF rules 2/3/4, and mempool-min-fee are stateful and don't belong here — the name
StandardnessPolicymakes that boundary self-evident.
One note on the existing Selection::create_psbt checks: check_max_standard_weight is pure policy and should move into StandardnessPolicy::check_tx. check_negative_fee is a correctness invariant rather than policy — that one can stay where it is.
|
@evanlinjin Thanks for the review. I want to push back on the architectural direction, though I agree with some of the observations. bdk-tx is a transaction construction library. The checks in this PR are deliberately minimal construction-bug guards. The four locations you flagged ( Following earlier discussion with @ValuedMammal , I think it's okay for configurable standardness/relay validation to belong in a separate library scoped to that concern. For reference, this is the design notes, a tx preparedness library will have a fundamentally different maintenance contract like tracking Core releases, node implementations, which are exactly the burden you identified. Earlier reviews on this PR pushed back on Once Some of the points you raised:
What I suggest we change
|
|
@aagbotemi have you curated this AI generated response before posting it? Standardness policy by definition is a subset of mempool policy. The response is self contradictory. I'm not even sure how to respond to it. |
|
Looking at the
I think it makes sense to turn this repo into a cargo workspace to contain the above. cc. @ValuedMammal @nymius In regards to what remains in this PR (if we are going forward with this proposal):
|
@evanlinjin the summary of the response I gave is trying to explain that the bdk-tx is a transaction building library and trying to stay in that scope. I understand that Standardness Policy is a subset of Mempool Policy, but only pointing out maintenance burden of having v28/v29/v30/v31 and so on, for a tx building library.
I think your proposal to split into a cargo workspace makes sense, such that the logic will be shared into different crates. |
|
I agree the Bitcoin Core version specific policies are overkill for right now, and we should instead focus on structural validity. Not sure what that leaves to be implemented since we've ruled out the negative fee possibility. While some of these technically fall under standardness I'm not against the idea of including basic sanity checks that we define and aren't likely to change very often. Of course these are guardrails and not hard and fast rules.
|
What do you mean by overkill?
Yes,
There are two concerns here. One is
Both of these concerns can be categorized under "basic sanity checks". However, I think it makes sense for the UX checks to exist in this crate (if we decide to have them). The values are ours to define, they don't change with Core releases. Let me clarify my reasoning for the proposed direction. Why did I suggest a separate
|
Description
Add a validated builder interface for
SelectorParamsthat catches dust outputs, zero-value OP_RETURNs, oversized OP_RETURN aggregates, and non-standard script types at construction time. Add a negative-fee guard and a max standard tx weight toSelection::create_psbt.Notes to the reviewers
is_standard_scriptchecks each type explicitly (P2PK, P2PKH, P2SH, P2WPKH, P2WSH, P2TR, OP_RETURN). P2A is matched via a raw byte comparison inis_p2a.change_dust_relay_feeratetodust_relay_feerate. The field applies to all outputs, not just change.Closes #41
Closes #49
Changelog notice
SelectorParamsBuilderwith chainable setters and two methods:build(runscheck_standardness) andbuild_unchecked(skips validation).SelectorParamsErrorfor builder validation failures.MAX_OP_RETURN_BYTESconst (100,000-byte sanity ceiling).SelectorParams::change_dust_relay_feeratetodust_relay_feerate.Selection::create_psbtnow rejects transactions where outputs exceed inputs (CreatePsbtError::NegativeFee) or where the weight exceeds the standard 400,000 WU limit (CreatePsbtError::MaxStandardTxWeightExceeded).Checklists