Skip to content

fix(z-image): dequantize GGUF pad tokens stored as a 2-D row - #9942

Open
Pfannkuchensack wants to merge 1 commit into
invoke-ai:mainfrom
Pfannkuchensack:fix/z-image-gguf-pad-tokens
Open

Pfannkuchensack wants to merge 1 commit into
invoke-ai:mainfrom
Pfannkuchensack:fix/z-image-gguf-pad-tokens

Conversation

@Pfannkuchensack

@Pfannkuchensack Pfannkuchensack commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Summary

Z-Image GGUF transformers whose pad tokens are stored as a 2-D row (x_pad_token/cap_pad_token with GGUF shape [3840, 1], e.g. z-image-turbo-Q4_K_S.gguf) could not generate at all: every run failed in diffusers' _prepare_sequence with TypeError: Multiple dispatch failed for 'torch._ops.aten.where.self' … GGMLTensor. The GGUF key converter only dequantized pad tokens that were 1-D, so the 2-D ones stayed packed. Pad tokens of any single-row shape are now dequantized and reshaped to (1, dim); a pad token that is not a single row is refused with a clear error instead of being reshaped.

Related Issues / Discussions

Found while testing Z-Image ControlNets on a GGUF base.

QA Instructions

RTX 4090, z-image-turbo-Q4_K_S.gguf + Qwen3-4B + FLUX VAE, 8 steps: before, every generation failed with the error above (768×1344 and 1024², with and without ControlNet); after, both sizes generate normally.

Tests: a CPU end-to-end test writes a tiny GGUF with the released file's layout (Q8_0 layers, BF16 refiners/embedders, F32 norms, pad tokens F32 [dim, 1]), loads it through the real GGUF loader and compares the forward with a dense reference (fails with the same TypeError before the fix; zeroed or scrambled pad tokens also fail it); unit tests for 1-D, (dim,) and (1, dim) pad tokens and the refusal. pytest tests/backend/model_manager/load -k "z_image or gguf" and tests/backend/quantization/gguf: passed. Ruff clean.

Review

No material findings. Noted for follow-up, unchanged here: the Z-Image GGUF loader does not unpack small GGML tensors at load (other GGUF loaders do), and splitting the fused qkv keeps those weights dequantized in memory.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

GGUF files that store x_pad_token/cap_pad_token as [dim, 1] kept them packed, so every generation failed in torch.where.
@github-actions github-actions Bot added python PRs that change python files backend PRs that change backend files python-tests PRs that change python tests labels Oct 19, 2026
@lstein lstein added the 7.0.0 label Oct 19, 2026
@lstein lstein moved this to 7.0 Theme: Modular Design in Invoke - Community Roadmap Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

7.0.0 backend PRs that change backend files python PRs that change python files python-tests PRs that change python tests

Projects

Status: 7.0 Theme: Modular Design

Development

Successfully merging this pull request may close these issues.

3 participants