Skip to content

[#2792] fix(spark): Make Uniffle listener events deserializable by newer Jackson versions - #2793

Open
wForget wants to merge 1 commit into
apache:masterfrom
wForget:UNIFFLE-2792
Open

wForget wants to merge 1 commit into
apache:masterfrom
wForget:UNIFFLE-2792

Conversation

@wForget

@wForget wForget commented Sep 18, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Add no-argument constructors and JavaBean setters to Uniffle listener events and their nested metric/timing objects so Spark History Server can deserialize them.

Make the relevant metric fields mutable while preserving existing constructors and JSON property names. Add serialization compatibility tests and align test-scoped Jackson dependencies with the selected Spark profile.

Why are the changes needed?

Opening an application's history page can fail with HTTP 500:

InvalidDefinitionException: Cannot construct instance of
org.apache.spark.shuffle.events.ShuffleAssignmentInfoEvent
(no Creators, like default constructor, exist)

Older Jackson Scala modules inferred constructor parameter names for these Java classes. Newer versions restrict this behavior, leaving Uniffle events without a supported deserialization constructor.

closes #2792

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually verified that the previously failing application history page opens successfully on Spark 4.2.0 History Server after applying this patch.

@github-actions

Copy link
Copy Markdown

Test Results

 3 833 files  +34   3 833 suites  +34   8h 13m 26s ⏱️ + 23m 2s
 1 266 tests ± 0   1 255 ✅ + 1  11 💤 ±0  0 ❌ ±0 
19 156 runs  +64  19 108 ✅ +65  48 💤 ±0  0 ❌ ±0 

Results for commit 1ccd563. ± Comparison against base commit b9b5d6d.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Spark 4.2.0 History Server fails to replay Uniffle events with newer Jackson versions

1 participant