Handle zero-sized literal output buffers

[Sn0wyDay]

Fixes #2851.

The shared writer behind WriteFloatHex() and WriteDoubleHex(), along with WriteUint128(), underflows size - 1 when called with a zero-sized output buffer. That produces an invalid copy length or array index.

Return immediately for size == 0 in both implementations and add regressions covering all three public functions.

Validation:

• Focused zero-size regressions: 2/2 pass.
• Three-function ASan/UBSan zero-size harness: pass; all output sentinels preserved.
• WABT unit tests: 137/137 pass.
• Full WERROR=ON CMake build and check target: pass on macOS with AppleClang.

[zherczeg]

Are these user functions? Or just internal functions?

[Sn0wyDay]

Those are public libwabt C++ functions: all three are declared in the installed include/wabt/literal.h, so an embedding application can call them. They are not directly exposed through the WABT command-line tools, though. All current in-tree production call sites pass nonzero fixed-size buffers, and I found no .wat/.wasm input path that can make size zero. So the affected case is a library caller explicitly passing size == 0, rather than an input-triggerable issue in the bundled tools.

[sbc100]

If these function are not used we should just delete them instead of fixing them.

[Sn0wyDay]

Thanks. I rechecked the usage on current main.

WriteFloatHex() and WriteDoubleHex() are each used at five non-test call sites in wat-writer.cc, binary-reader-objdump.cc, and binary-reader-stats.cc. Removing them causes the build to fail.

WriteUint128() has no current non-test in-tree caller. However, all three functions are declared in the installed include/wabt/literal.h and present in the installed static library, so removing them could break external source or link compatibility. I found no public downstream caller for WriteUint128(), but I cannot rule out private or unindexed users.

My preference is therefore to keep the small zero-size guards. If you prefer removing the unused exposed function despite that compatibility risk, I can delete only WriteUint128() and its tests while retaining the Float/Double guards.

[zherczeg]

The include/wabt expose the entire library, and that includes many helper functions. You can misuse at least half of them to create crashes.
Is there a rule for what is public API, and what is internal?