Skip to content

[tiering] State machine race: first tiering round fails with 'Scheduled to Tiered is not a valid state change' #3672

Description

@qzyu999

Summary

The first tiering round for a newly created datalake-enabled table fails with a state machine validation error. Subsequent rounds succeed.

Error

ERROR org.apache.fluss.server.coordinator.LakeTableTieringManager - Fail to change state for table 1 from Scheduled to Tiered as it's not a valid state change. ERROR org.apache.fluss.server.coordinator.LakeTableTieringManager - Fail to change state for table 1 from Scheduled to Scheduled as it's not a valid state change.

Expected Behavior

The state machine should transition Scheduled Pending Tiering Tiered without errors on the first tiering round.

Actual Behavior

The tiering job completes its work (S3 write succeeds, storeLakeTableOffsetsFile() succeeds) but the Coordinator rejects the state transition because the table is still in \Scheduled\ state when the tiering result arrives. The expected intermediate transitions (Scheduled Pending Tiering) haven't occurred yet.

The second tiering round works fine by then the state machine has caught up.

Steps to Reproduce

  1. Create a table with 'table.datalake.enabled' = 'true', 'table.datalake.freshness' = '60s'
  2. Insert a small amount of data (e.g., 3 rows)
  3. Start the tiering service
  4. Observe Coordinator logs within the first freshness interval

Environment

Analysis

Likely a timing issue: with a small dataset and fast S3 writes, the tiering job completes its first round before LakeTableTieringManager has transitioned the table through the intermediate states. The state machine expects the sequence Scheduled Pending Tiering Tiered, but the tiering completion arrives while still in Scheduled.

This may be more likely with:

  • Small datasets (fast tiering)
  • Fast network to S3
  • Short freshness intervals

Impact

  • First tiering round's snapshot is lost (not committed to Iceberg metadata)
  • Subsequent rounds work correctly
  • Data is eventually tiered, but there's a one-round delay on fresh tables

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions