Skip to content

msgpack and cbor2 serialize naive datetimes using the encoding machine's local timezone #774

Description

@onk3sh

The msgpack and cbor2 preconfigured converters unstructure datetime with datetime.timestamp(). On a naive datetime that method interprets the value as local time, so the serialized number — and therefore the bytes on the wire — depend on the timezone of the machine doing the unstructuring.

The same object serialized on two machines in different timezones produces two different payloads, and neither round-trips back to the input.

Reproduction

from datetime import datetime, timezone

from cattrs.preconf.msgpack import make_converter

converter = make_converter()
naive = datetime(2026, 8, 25, 12, 30)

print(converter.dumps(naive).hex())
print(converter.loads(converter.dumps(naive), datetime))

Running that same script under three timezones, on cattrs d4ff793:

TZ dumps output loads returns
UTC cb41daa362b2000000 2026-08-25 12:30:00+00:00
Asia/Tokyo cb41daa3430e000000 2026-08-25 03:30:00+00:00
America/Toronto cb41daa370c2000000 2026-08-25 16:30:00+00:00

cbor2 behaves identically (fb… instead of cb…).

An aware datetime is the control — it produces cb41daa362b2000000 in all three timezones and round-trips exactly. So the payload is deterministic for aware input and machine-dependent for naive input, through the same converter and the same field.

The other preconfigured converters that handle datetimes themselves — json, ujson, pyyaml, msgspec, orjson — round-trip naive datetimes exactly, in every timezone.

Cause

src/cattrs/preconf/msgpack.py:

converter.register_unstructure_hook(datetime, lambda v: v.timestamp())
converter.register_structure_hook(
    datetime, lambda v, _: datetime.fromtimestamp(v, timezone.utc)
)

src/cattrs/preconf/cbor2.py has the same pair.

The structure hook states the wire contract: timestamps are UTC. The unstructure hook does not honour it for naive input, because datetime.timestamp() falls back to local time when tzinfo is None. The two hooks disagree.

Worth noting the date hook immediately below it in msgpack.py already does this correctly:

converter.register_unstructure_hook(
    date, lambda v: datetime.combine(v, time(tzinfo=timezone.utc)).timestamp()
)

It pins tzinfo=timezone.utc before calling .timestamp(). The datetime hook is missing that step.

Why it has not been caught

tests/test_preconf.py never generates a naive datetime:

  • L155-158 pins the shared strategy to timezones=just(timezone.utc).
  • The one strategy that would produce naive datetimes, native_unions at L235-239, is passed include_datetimes=False at all seven of its call sites (L320, 414, 502, 582, 774, 878, 945).

Unpinning L158 to generate naive datetimes fails test_msgpack, test_msgpack_converter, test_cbor2 and test_cbor2_converter, with hypothesis minimising to a_datetime=datetime.datetime(2000, 1, 1, 0, 0).

(It also fails the two bson tests, but for an unrelated reason: those pass CodecOptions(tz_aware=True) at L631/L652, so naive input legitimately comes back aware. bson itself is not affected — its output is identical in all three timezones. I mention it only so the failure list is not misleading.)

Suggested fix

Treat naive datetimes as UTC in the unstructure hook, so it agrees with the structure hook and the output stops depending on the host:

if v.tzinfo is None:
    v = v.replace(tzinfo=timezone.utc)
return v.timestamp()

A naive input still returns aware on the way back, since a bare float has nowhere to record awareness — but the value is preserved and the payload becomes machine-independent.

The alternative would be to raise on naive input rather than assume UTC. That is stricter and arguably more honest, at the cost of breaking anyone currently passing naive datetimes. Happy to go either way — I have the first version written with regression tests and can open a PR, or switch it to raising if you prefer that.

I could not find this reported previously, and HISTORY.md and docs/preconf.md do not mention naive datetimes, so I do not think it was a deliberate decision.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions