Skip to content

[r8-obfuscation] Add PE metadata rebuild substrate - #12629

Merged
simonrozsival merged 7 commits into
mainfrom
simonrozsival-pe-metadata-rebuild-substrate
Sep 3, 2026
Merged

[r8-obfuscation] Add PE metadata rebuild substrate#12629
simonrozsival merged 7 commits into
mainfrom
simonrozsival-pe-metadata-rebuild-substrate

Conversation

@simonrozsival

@simonrozsival simonrozsival commented Sep 1, 2026

Copy link
Copy Markdown
Member

Summary

This is layer 2 of 6 in the replacement stack for PR #12575. It adds the internal PE and metadata reconstruction substrate required by later JNI name-rewriting layers, without enabling any build behavior.

  • Rebuilds managed PE images while preserving existing metadata tokens and supported table row ordering.
  • Re-emits method bodies, embedded resources, debug-directory data, native resources, and FieldRVA mapped data, including ranges from different PE sections.
  • Preserves overlapping/aliased FieldRVA backing data by relocating rewritten fields instead of modifying shared storage.
  • Validates malformed IL, resource offsets/directories, and strong-name signature directories through controlled rewrite errors.
  • Emits event/property method semantics explicitly in HasSemantics coded-index order.
  • Rejects unsupported field-backed ImplMap rows instead of silently dropping metadata.
  • Adds low-level metadata encoding, raw-column, IL operand-scanning, and rewrite-plan primitives.
  • Covers longer and shorter UTF-8 FieldRVA replacements, aliased mapped fields, event/property accessor preservation, per-use #US splitting, and strong-name signature preservation directly through AssemblyRebuilder.
  • Uses the executing runtime's core assembly version in generated fixtures so rebuilt images load across target frameworks.

Scope

This layer intentionally does not add JNI-specific planning or attribute/ldstr orchestration, the assembly-rewriter facade, the RewriteJniNamesForR8 MSBuild task, typemap rewrite behavior, CoreCLR/NativeAOT target integration, or documentation. Those surfaces belong to later stack layers.

Validation

  • ./dotnet-local.sh build src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Xamarin.Android.Build.Tests.csproj -c Debug -v:minimal --no-restore
  • ./dotnet-local.sh test bin/TestDebug/net10.0/Xamarin.Android.Build.Tests.dll --filter 'FullyQualifiedName~NativeResourceSectionCopierTests|FullyQualifiedName~AssemblyRebuilderTests'
  • Direct layer-2 result: 18 passed, 0 failed.
  • Cumulative layer-3 RewrittenAssemblyLoadsAndRunsInTheRuntime result: 1 passed, 0 failed.

Related to #12535

Depends on #12628

@simonrozsival simonrozsival changed the title Add PE metadata rebuild substrate [r8-obfuscation] Add PE metadata rebuild substrate Sep 1, 2026
@simonrozsival
simonrozsival force-pushed the simonrozsival-pe-metadata-rebuild-substrate branch 3 times, most recently from e83dcaa to 58ac474 Compare September 1, 2026 18:36
@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@simonrozsival
simonrozsival force-pushed the simonrozsival-pe-metadata-rebuild-substrate branch 2 times, most recently from fdb7393 to 311a3bf Compare September 2, 2026 08:52
@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Base automatically changed from simonrozsival-r8-mapping-primitives to main September 2, 2026 11:23
simonrozsival and others added 3 commits September 2, 2026 13:23
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use the executing runtime version in metadata fixtures so rebuilt images load across target frameworks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival
simonrozsival force-pushed the simonrozsival-pe-metadata-rebuild-substrate branch from 311a3bf to cc7f216 Compare September 2, 2026 11:23
@simonrozsival
simonrozsival marked this pull request as ready for review September 2, 2026 11:27
Copilot AI lite review requested due to automatic review settings September 2, 2026 11:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

MetadataEncoding.ReadCompressedInteger does not guard against negative offsets, causing unintended IndexOutOfRangeException instead of a controlled rewrite failure.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1 Medium severity · 1 Low severity

New issues introduced by this change (2)
Severity Finding
Medium severity src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​MetadataEncoding.cs — ❌ error Bug — ReadCompressedInteger allows a negative offset, which will throw…
Low severity src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​AssemblyRebuilder.cs — 💡 suggestion Formatting / Conventions — Prefer [] over Array.Empty&lt;T&gt;() in this repo to…
What changed in this PR

Adds the internal PE + ECMA-335 metadata rebuild substrate under Xamarin.Android.Build.Tasks to support later R8/JNI name-rewriting layers, while keeping this layer behaviorally inert (no build integration yet). The new utilities focus on rebuilding managed assemblies without moving metadata tokens, while re-emitting method bodies, resources, debug directory data, native resources, and FieldRVA mapped data, plus adding targeted validation against malformed inputs.

Changes:

  • Introduces AssemblyRebuilder and supporting primitives for token-preserving PE/metadata reconstruction, IL scanning, and selective heap/value rewrites.
  • Adds low-level helpers for raw metadata column access, compressed integer encoding/decoding, FieldRVA decoding/relayout, and Win32 resource section relocation.
  • Adds focused NUnit coverage with a fixture generator to validate native resources, FieldRVA behaviors, IL scanning validation, and strong-name signature handling.
File Description
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​AssemblyRebuilder.cs Core token-preserving managed PE rebuild implementation (IL, resources, debug/native resources, FieldRVA).
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​FieldRvaTable.cs Reads and interprets FieldRVA rows and mapped data, including typemap-style UTF-8 payload detection.
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​NativeResourceSectionCopier.cs Copies and relocates .rsrc directory data for rebuilt PEs with validation.
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​MetadataRawColumns.cs Reads metadata table columns not exposed by MetadataReader (e.g., ImplMap.MemberForwarded).
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​MetadataEncoding.cs ECMA-335 compressed integer encode/decode helper used for manual blob rewriting.
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​JniRewritePlan.cs Holds per-use-site rewrite decisions for attributes, ldstr, and UTF-8 FieldRVA data.
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​JniRewriteException.cs Dedicated exception type for rewrite failures and validation errors.
src/​Xamarin.Android.Build.Tasks/​Utilities/​JniRemapping/​IlOpcodeTable.cs Minimal operand-size table enabling IL walks without a full IL parser.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​Utilities/​JniRemapping/​AssemblyRebuilderTests.cs End-to-end tests for rebuild invariants (FieldRVA/resources, IL, strong-name space, validation).
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​Utilities/​JniRemapping/​NativeResourceSectionCopierTests.cs Validates .rsrc copying/relocation and malformed directory rejection.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​Utilities/​JniRemapping/​JniFixtureBuilder.cs Produces minimal managed PE fixtures to exercise the rebuilder without external binaries.

Comment thread src/Xamarin.Android.Build.Tasks/Utilities/JniRemapping/MetadataEncoding.cs Outdated
Comment thread src/Xamarin.Android.Build.Tasks/Utilities/JniRemapping/AssemblyRebuilder.cs Outdated
simonrozsival and others added 3 commits September 2, 2026 13:34
Detach rewritten mapped fields from overlapping source storage so shorter replacements cannot corrupt aliased fields.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Validate negative compressed-integer offsets and use the repository empty-array convention when rebuilding method bodies.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Validate malformed resource and strong-name directory values through the rewrite error contract, and emit MethodSemantics rows in HasSemantics coded-index order.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival simonrozsival added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Sep 2, 2026
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12629

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Needs Changes — 1 error, 2 warnings.

The table-by-table token preservation and malformed-metadata coverage are strong, and all 44 CI checks are green. Before merge, the rebuild needs to reject or preserve non-MethodDef entry points; I also called out two validation gaps that can silently produce altered or inconsistent output for malformed metadata.

Generated by Android PR Reviewer for #12629 · gpt56 · 227.7 AIC · ⌖ 9.02 AIC · ⊞ 25.7K
Comment /review to run again

Validate colliding synthetic UTF-8 types before reuse and reject half-populated native resource directories.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@simonrozsival

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@simonrozsival
simonrozsival merged commit 0267ea6 into main Sep 3, 2026
44 checks passed
@simonrozsival
simonrozsival deleted the simonrozsival-pe-metadata-rebuild-substrate branch September 3, 2026 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants