Skip to content

[datafusion-spark] Support 2-argument ceil(value, scale)#21710

Open
diegoQuinas wants to merge 3 commits intoapache:mainfrom
diegoQuinas:feat/ceil-two-args
Open

[datafusion-spark] Support 2-argument ceil(value, scale)#21710
diegoQuinas wants to merge 3 commits intoapache:mainfrom
diegoQuinas:feat/ceil-two-args

Conversation

@diegoQuinas
Copy link
Copy Markdown

Which issue does this PR close?

Closes #21560

Rationale for this change

The Spark ceil function supports an optional scale parameter that controls
the decimal position to round up to. This was not yet implemented in
datafusion-spark.

What changes are included in this PR?

  • Updated Signature to accept 1 or 2 arguments, following the same pattern as SparkRound
  • Updated return_type: floats preserve their type when a scale is provided (instead of returning Int64); scale=0 preserves the original behavior
  • Added get_scale() helper to extract the optional scale argument, returning None for NULL scale (which produces a NULL result)
  • Added ceil_float() helper for ceiling floats at arbitrary decimal positions
  • Updated spark_ceil_scalar and spark_ceil_array to apply the scale

Are these changes tested?

Yes, unit tests covering:

  • Float64/Float32 scalar with positive scale, negative scale, zero scale, and NULL scale
  • Float64 array with scale
  • Existing 1-argument tests continue to pass

Are there any user-facing changes?

Yes — ceil(expr, scale) is now supported in addition to the existing ceil(expr).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support 2-argument ceil(value, scale) for Spark

1 participant