Skip to content

fix(spark): accept STRING and TIMESTAMP start dates in next_day - #26127

Open
harsh-ande wants to merge 2 commits into
apache:mainfrom
harsh-ande:fix-next-day-coercion
Open

harsh-ande wants to merge 2 commits into
apache:mainfrom
harsh-ande:fix-next-day-coercion

Conversation

@harsh-ande

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

Spark's NextDay casts its first argument to DATE, so next_day('2015-07-23', 'Mon') and next_day(TIMESTAMP '2015-07-23 12:12:12', 'Mon') both work. datafusion-spark declared Exact(Date32, Utf8) and failed at planning for STRING and TIMESTAMP inputs.

What changes are included in this PR?

  • next_day now uses a coercible signature: the first argument is implicitly cast to DATE from STRING or TIMESTAMP, and the day-of-week is any string type (numeric is still coerced to string).
  • The kernel is unchanged.
  • In next_day.slt, three existing query error expectations changed to the new planner messages (arity and non-date first argument), and the unparseable-string case now returns a cast error. They still fail at planning or execution as before.

What is the testing strategy for this PR?

New cases in spark/datetime/next_day.slt: string, string with a time part, timestamp literal, string column with NULL, timestamp column, and LargeUtf8/Utf8View day-of-week. I compared the expected results with pyspark==4.0.0, ANSI on and off.

Are there any user-facing changes?

next_day accepts more argument types. No API changes.

Known limitations / unknowns

Signed-off-by: Harsh Ande <harsh.vardhan.ande@gmail.com>
@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) spark labels Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spark sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] next_day rejects STRING and TIMESTAMP start dates that Spark accepts

1 participant