[Python] pyarrow.compute.strftime silently echoes unsupported format specifiers (%s returns the literal '%s')

[ptimizeroracle]

Describe the bug

pyarrow.compute.strftime passes unsupported format specifiers through literally instead of either formatting them or raising. %s (epoch seconds, a widely used strftime extension) returns the format string itself as the data:

>>> import pyarrow as pa, pyarrow.compute as pc
>>> from datetime import datetime
>>> d = pa.array([datetime(2021, 1, 1, 12, 30, 45)])
>>> pc.strftime(d, format="%s").to_pylist()
['%s']
>>> int(datetime(2021, 1, 1, 12, 30, 45).timestamp())  # expected
1609500645

polars and duckdb both return the epoch value for %s, and raising would also be acceptable, but silently returning '%s' as column data means downstream code consumes the format string as if it were real data with no error anywhere.

(side observation on the same kernel: %S returns '45.000000' rather than the two-digit '45', which also diverges from strftime conventions, but the %s case is the silent-data-corruption one)

Component(s)

Python

[kita-renji]

I can reproduce this on Linux with pyarrow 23.0.1 and 25.0.1, and the code path is unchanged on current main (51955ec).

It isn't specific to %s. Any specifier the vendored date library doesn't know is copied to the output as-is, for timestamp, date32 and time64: %s, %k, %l, %P, flags like %-d, most %E?/%O? combinations, and undefined ones like %i or a trailing %.

Strftime::Make only checks for %c and %z/%Z (

arrow/cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

Lines 1207 to 1225 in 51955ec

	static Result<Strftime> Make(KernelContext* ctx, const DataType& type) {
	const StrftimeOptions& options = StrftimeState::Get(ctx);
	// This check is due to surprising %c behavior.
	// See https://github.com/HowardHinnant/date/issues/704
	if ((options.format.find("%c") != std::string::npos) && (options.locale != "C")) {
	return Status::Invalid("%c flag is not supported in non-C locales.");
	}
	const auto& timezone = GetInputTimezone(type);
	if (timezone.empty()) {
	if ((options.format.find("%z") != std::string::npos) ||
	(options.format.find("%Z") != std::string::npos)) {
	return Status::Invalid(
	"Timezone not present, cannot convert to string with timezone: ",
	options.format);
	}
	}
	ARROW_ASSIGN_OR_RAISE(auto tz, LocateZone(timezone.empty() ? "UTC" : timezone));

), and everything else goes to date.h's to_stream, whose default: case writes unknown specifiers back out (

arrow/cpp/src/arrow/vendored/datetime/date.h

Lines 6104 to 6116 in 51955ec

	default:
	if (command)
	{
	os << CharT{'%'};
	command = nullptr;
	}
	if (modified != CharT{})
	{
	os << modified;
	modified = CharT{};
	}
	os << *fmt;
	break;

). On Windows the std::format backend should throw for these instead, so the same format likely errors there and silently echoes elsewhere. Those checks are also plain substring searches, so %Ez on naive input prints +00:00 and %%z wrongly raises "Timezone not present".

I'd suggest having Make tokenize the format and return Invalid for any specifier outside the supported set, with the %c and %z/%Z checks in the same pass. That covers every specifier and both substring cases. The catch is that formats which currently "work" by echoing would start raising.

Implementing %s could be a follow-up, since its semantics need a decision: glibc and Python use the local timezone, which is why the expected value in the report (1609500645) is an hour off the UTC epoch (1609504245). Until then, pc.cast(pc.cast(arr, pa.timestamp("s")), pa.int64()) gives epoch seconds. (%S printing 45.000000 is documented: the array is timestamp[us].)

I have the validation change and a C++ test working locally. Happy to open a PR with a Python test if this direction sounds right.

Investigated with help from Claude Code; I verified the repro and results.