Skip to content

fix: reject hidden concatenated BIP21 requests - #204

Merged
pwltr merged 4 commits into
masterfrom
fix/bip21-hidden-concatenation
Nov 12, 2026
Merged

pwltr merged 4 commits into
masterfrom
fix/bip21-hidden-concatenation

Conversation

@pwltr

@pwltr pwltr commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Refs synonymdev/bitkit-ios#1272.

Description

A benign bitcoin:donation prefix can hide a later payment URI from the current first-occurrence check. The decoder then accepts the first recipient with fields from the second request. For example, bitcoin:<first>?message=bitcoin:donationbitcoin:<second>?label=merchant&amount=0.00005 returns the first recipient with the second request's 5,000 sats.

Check the full query, including keys, for embedded payment URIs before collecting parameters. Inspect the final scheme before each question mark so a benign earlier mention cannot hide a valid address, without repeatedly scanning the remaining input. Reject embedded addressless requests containing known payment parameters too, including when unknown parameters come first or contain question marks, covering the other shape raised in the original Core #151 review.

Preserve legacy address case and normalize Bech32 addresses for validation. Keep ordinary bitcoin:donation text, unrelated question marks, address-only metadata, and percent-encoded text accepted. This changes private scanner internals only; no public binding API or dependency changes.

Related adoption PRs: synonymdev/bitkit-ios#909 and synonymdev/bitkit-android#1454. They still pin the published rc13 SDK; a fixed SDK and app dependency updates are required before the apps receive this fix.

Preview

N/A — no UI changes.

QA Notes

Initial validation at 9f9a449 (base f69c29861d7ee1f7f9d79cc5f427c60d46a77816). Tested the exact committed tree a4de18cc40bd853169f17e8ea0a2674a7f05cfe2 on macOS arm64, Rust 1.91.1, with CARGO_TARGET_DIR=/Users/agent-runner/projects/bitkit-core/target and protobuf compiler 36.2 on PATH.

  • cargo +1.91.1 test --locked --lib embedded_bip21 before the production fix: four regression tests failed, reproducing the masked occurrence, query-key, address-case and addressless forms on current master.
  • cargo +1.91.1 test --locked --lib modules::scanner: 46 passed, no ignored tests.
  • cargo +1.91.1 test --locked --lib -- --skip modules::blocktank::tests: 763 passed, 31 ignored, 48 Blocktank tests excluded. The ignored and excluded cases are not passing evidence.
  • cargo +1.91.1 build --locked: passed.
  • cargo +1.91.1 clippy --locked --lib --tests: passed with warnings outside the changed code.
  • cargo +1.91.1 fmt --check and git diff --check: passed.
  • Applied the same patch to published rc13's source revision 184ccf5f290e84baad0e0b4f49447af78ac28278; cargo +1.90.0 test --locked --lib modules::scanner: 46 passed, no ignored tests.

Tool prerequisites resolved during validation: the default Rust 1.90 toolchain cannot build current master's dependencies; used the installed 1.91.1 toolchain and installed its missing rustfmt/Clippy components. This did not change the repository's dependencies or the machine's default toolchain.

Native platform bindgen/CI and app journeys against a rebuilt SDK remain pending. These Rust tests do not represent device execution. Release publication, SDK adoption and final app regression checks are separate steps; no release is published by this change.

Parameter-order review follow-up at de6b3c0

The Medium finding was reproduced before this follow-up: an unknown first key allowed an addressless request to add an amount and overwrite Lightning metadata when the outer request had no amount. The new regression failed on 9f9a449 with the exact review payload.

Track an embedded addressless request through the remaining query and reject subsequent known payment fields, regardless of unknown parameter ordering or question marks inside unknown values. Regressions cover all six recognized payment keys, case-insensitive keys, multiple unknown/empty parameters, and the question-mark case. A positive case preserves unknown-only bitcoin:?help=1 metadata.

Verified the exact signed commit tree 9554b9e88a78592c03adfb7d1bdaaa115dd29ca3:

  • cargo +1.91.1 test --locked --lib modules::scanner: 48 passed; none ignored.
  • cargo +1.91.1 test --locked --lib -- --skip modules::blocktank::tests: 765 passed, 31 ignored, 48 Blocktank tests excluded.
  • cargo +1.91.1 build --locked, cargo +1.91.1 clippy --locked --lib --tests, cargo +1.91.1 fmt --check, git diff --check: passed. Build/Clippy retain warnings outside changed code.
  • Same cumulative patch on rc13 revision 184ccf5: cargo +1.90.0 test --locked --lib modules::scanner: 48 passed; none ignored.

Native platform CI is rerunning on this head. No rebuilt-SDK device tests or publication are claimed.

Historical CI mirror attempt at d77e1d7 — reverted in f652de2

The Linux jobs on de6b3c0 failed during system dependency installation with Temporary failure resolving azure.archive.ubuntu.com. Android bindgen, paykit-route and wake repeated this failure on the first retry; USDT was also blocked installing dependencies. The failure logs provide no compilation or test evidence.

That attempt set /etc/apt/apt-mirrors.txt to https://archive.ubuntu.com/ubuntu/ before apt-get update in Android bindgen, paykit-route, USDT and wake workflows. It was an eight-line CI-only change; the scanner implementation and tests are identical to de6b3c0.

  • Parsed all four changed workflows with Ruby YAML and checked their dependency-install shell blocks with bash -n: passed.
  • git diff --check: passed.
  • Verified that Ubuntu's archive serves both Noble and Noble Security release metadata.
  • actionlint is not installed locally; it was not run. APT installation cannot be executed on this macOS host; the new hosted CI run is the remaining execution check.
  • The prior Rust and rc13 test results remain attached to de6b3c0; no application rebuild was needed for this workflow-only follow-up.

Historical CI blocker and revert at f652de2 — validation now passed

The mirror attempt did not help: all four Linux jobs also failed resolving archive.ubuntu.com, before compilation or tests. Similar failures across Ubuntu runner versions are reported in actions/runner-images#15296 and #15297.

Restored the original workflow files rather than retain an ineffective workaround. The full tree at f652de2 is exactly identical to tested revision de6b3c0 (9554b9e88a78592c03adfb7d1bdaaa115dd29ca3). Verified with git diff --exit-code de6b3c0 HEAD; YAML parsing, install-block bash -n, and git diff --check passed. This revert does not introduce a new Rust implementation or binding revision.

The first hosted Linux attempts at this head were blocked by dependency downloads. On 2026-11-12, the reruns completed successfully on unchanged f652de2; the successful runs below establish hosted validation. The earlier infrastructure failures are retained above as history, not counted as passing tests.

Completed hosted validation at f652de2 (2026-11-12)

All seven build/test/manifest checks succeeded; Ovi's reviewer check completed neutral with no findings. No source or workflow changes were needed for these reruns.

Reviewer approval is still required. This validates the Core PR; it does not establish app device journeys against a newly published SDK.

Models used

  • Planning/scoping: gpt-6.1-sol (reasoning: high)
  • Implementation: gpt-6.1-sol (reasoning: high)
  • Review: Not performed

Model metadata source: authoritative T3 Code runtime information for this session.

Check every candidate embedded payment URI in query keys and values, including empty-address requests beginning with a known payment parameter. Preserve benign bitcoin: metadata and cover recipient/amount mixing, legacy address case, uppercase Bech32, Unicode, and escaped text.

Refs synonymdev/bitkit-ios#1272.

@talosmachina talosmachina left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings. Replaces the per-value BIP21 check with one pass over the whole query (keys included) that validates the last bitcoin: before each ?, so a benign bitcoin:donation can no longer mask a second payment URI, and rejects addressless embedded requests that start with a known BIP21 key. Reviewed 9f9a449, full tier, static.

What I checked, and 5 candidates I ruled out

Read in full: src/modules/scanner/implementation.rs (decode, decode_onchain, contains_embedded_bip21_uri, normalize_address_for_validation), the new tests in src/modules/scanner/tests.rs.
Call sites traced: decode_onchain is reached from the bitcoin: branch of decode (query passed through unchanged) and the bare-address branch (no query); contains_embedded_bip21_uri has the one caller.
Build: skipped, talos-pr-run could not read the author's permission on this repo and fails closed, so nothing here was compiled or run. The author's run (46 scanner tests on 1.91.1) is the only execution evidence.
CI: green on manifests, paykit-route, usdt and wake tests; iOS and Android bindgen still pending. No CI job runs the scanner tests.

Ruled out

  • Index mismatch slicing part with an offset from part.to_ascii_lowercase(): ASCII lowercasing keeps byte length and boundaries, so start + 8 is a valid char boundary in the original even with the non-ASCII prefixes the tests use.
  • An earlier bitcoin:<valid> in the same ?-segment hidden by a later benign one: within one segment, any earlier scheme's address runs into the later bitcoin:, so it can never validate; only the last one can form a URI, which is what rfind checks.
  • Addressless embedded request whose first key is not in the list (bitcoin:?foo=1&amount=...): the trailing params then attach to the only recipient there is, and a duplicate amount is still caught by the singleton check, so no second recipient is mixed in.
  • Regression on legitimate URIs carrying ? in values (pj= URLs, free text): rejection needs a bitcoin: immediately followed by a valid address or by a known key, which ordinary values and payjoin endpoints do not contain; the benign-mentions test covers the text cases.
  • Percent-encoded embedded URI (bitcoin%3A...): query values are not percent-decoded anywhere in decode_onchain, so the encoded form never becomes a second request in this parser; the PR keeps it accepted by design and tests that.

Merge confidence: 4/5, no findings, but the scanner tests run in no CI job and I could not run them here.

@pwltr pwltr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 9f9a449.

Medium — Addressless concatenations bypass the guard when an unknown key comes first. In src/modules/scanner/implementation.rs:399, only the first embedded query key is checked. For example, bitcoin:bcrt1qr289x0fhg62672e8urudfnxnsr8tcax64xk2vk?lightning=LNAbitcoin:?custom=x&amount=0.00005&lightning=LNB is accepted with the outer address, 5,000 sats and params["lightning"] = "LNB". There is no duplicate amount, and lightning is not a singleton. Changing custom=x to label=x causes rejection, so optional parameter ordering controls the protection. This behavior predates the PR but remains a gap in its addressless-request guard. Inspect subsequent embedded payment keys and add this ordering variant to the regression cases.

Review details

General review
Model: GPT-6.1 Sol
Reasoning effort: high

Combined review
Model: GPT-6.1 Sol
Reasoning effort: high

Reject known payment fields after an embedded addressless request even when unknown parameters come first or contain question marks. Cover the review payload without an outer amount and preserve unknown-only metadata.

Refs synonymdev/bitkit-ios#1272.
Use the official Ubuntu archive for APT mirror downloads in the four affected Linux workflows. The original run and retry both failed resolving azure.archive.ubuntu.com before building or testing. Retain the existing package selections and repository signature verification.

Refs synonymdev/bitkit-ios#1272.
The new run also fails resolving archive.ubuntu.com. Restore the original CI workflows; the temporary mirror override does not mitigate the broader Ubuntu DNS outage reported in actions/runner-images#15296 and #15297. Keep this PR limited to the scanner fix while hosted Linux checks remain blocked before compilation.

Refs synonymdev/bitkit-ios#1272.
@pwltr
pwltr requested review from a team, ben-kaufman and ovitrif and removed request for a team November 11, 2026 16:10

@pwltr pwltr left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed f652de2: de6b3c0, plus a CI workaround that f652de2 reverts. No findings.

After an addressless bitcoin:, every later ?-separated part is now checked for a payment field, not only the first embedded parameter. Its first segment counts as the previous value's text, so custom=Why?&amount= is rejected while custom=Why?amount= stays inside the unknown value. Key case and empty segments are covered too, and unknown-only metadata stays accepted.

One consequence worth knowing: an outer payment field placed after a value that holds a raw bitcoin:?, as in ?message=bitcoin:?help=1&amount=X, is now rejected as ambiguous. That's the safe side, since such a value should be percent-encoded.

CI at f652de2: the manifest checks and iOS bindgen passed. Android bindgen and the wake, usdt and paykit-route tests failed at apt-get (exit code 100) before running, and no CI job runs the scanner tests.

Review details

General review
Model: Claude Opus 5.5
Reasoning effort: high

@ovi-reviewer ovi-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 💬 Comment — no findings; Linux validation remains unverified.

Findings: None.

QA for f652de2:

Reject concatenated requests and preserve ordinary metadata ✅ The committed Rust scanner suite passed.

cargo +1.91.1 test --locked --lib modules::scanner: 48 passed, none failed or ignored on macOS arm64. This exercises hidden payment URIs in keys and values, legacy and uppercase Bech32 addresses, reordered addressless fields, duplicate singletons, Unicode, question marks and percent-encoded metadata.

The unknown-first-key payload from the earlier review is rejected by the follow-up regression.

Decode through freshly generated native bindings ✅ Exported decoder calls rejected eight ambiguous inputs and preserved six valid controls.

Built the exact head with cargo +1.91.1 build --locked, generated Python bindings with UniFFI, and called the exported async decode. All eight malformed inputs returned InvalidFormat; all six positive controls retained the recipient, 3,500 sats and exact metadata, including a long Unicode value.

To import the generated Python module, the test copy needed one syntax correction: remove the optional default preceding a required argument in the unrelated UsdtWallet constructor. That constructor was never called, and the Rust library and decoder bindings were unchanged. This is native FFI evidence, not mobile app execution.

Check: formatting and whitespace ✅ Rust formatting and whitespace checks passed.

cargo +1.91.1 fmt --check and git diff --check passed at the reviewed head.

Remaining verification: No device run. This PR changes the shared Rust library; the app adoptions below still pin 0.8.0-rc13. Rebuilt-SDK wallet journeys remain separate and are not established by these native checks. I did not rerun the broader non-scanner suites or Clippy.

The four failed Linux jobs stop at APT mirror DNS resolution before compilation or tests; they are not passing evidence:

Full review of f69c298..f652de2, covering both changed files, their callers, address validation and exported decoder error handling. No public API or dependency change. No mobile app flow was driven with this library revision.


Reviewed by gpt-6.1-sol-high

@pwltr
pwltr merged commit 8f01555 into master Nov 12, 2026
8 of 12 checks passed
@pwltr
pwltr deleted the fix/bip21-hidden-concatenation branch November 12, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants