Repository navigation
fix(flux2,z-image): run Diffusers schedulers on the node's sigma schedule - #9851
Open
Pfannkuchensack wants to merge 4 commits into
Open
Pfannkuchensack wants to merge 4 commits into
Pfannkuchensack wants to merge 4 commits into
Conversation
…dule FLUX.2 shifts before clipping on every path; Euler/LCM keep the terminal sigma and Heun runs the shifted schedule. Z-Image Heun runs the clipped sigmas from the img2img start; Heun progress reaches n/n on both. Upscale offers Heun for FLUX.1 and Z-Image, which now start at the blend. Closes invoke-ai#9792
Member
Author
joshistoast
reviewed
Oct 19, 2026
| if is_heun: | ||
| # Heun runs the shifted, clipped schedule above, so it starts at the sigma the img2img | ||
| # preblend used and stops at the requested end. | ||
| set_heun_timesteps_from_sigmas(scheduler, sigmas[:-1], sigmas[-1], device) |
Collaborator
There was a problem hiding this comment.
P2 Handle zero-shift schedules before passing them to Heun
shift=0 is accepted by this invocation, and _get_sigmas(0, 8) returns [1, 0, 0, …, 0]. Passing that schedule to Heun now evaluates the model at zero sigma: Diffusers 0.41.0 divides by sigma_next during its second-order correction, producing NaN latents on the second evaluation even with a finite constant prediction. I reproduced this using the pinned schedule builder and helper; the base revision's scheduler setup returns finite latents for the same configuration. Handle the degenerate schedule before configuring Heun so an accepted shift override cannot corrupt the generated result.
Heun runs only the schedule's nonzero intervals; a zero-length interval at sigma 0 divided by zero. Its progress counts the intervals it runs; with none left the pass returns the blended latents.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
#9724 fixed FLUX.1's Diffusers scheduler paths. FLUX.2 and Z-Image still had the same problems:
denoising_end < 1finished the image instead of stopping. txt2img also clipped the unshifted schedule, while img2img clipped the shifted one, so a pass ending atdenoising_end = xand a pass continuing fromdenoising_start = xdid not meet at the same sigma.muanddenoising_end.order=2is halved bycalc_percentage).Now:
time_shift_flux2(mu)once, before clipping, on every path. Its Diffusers schedulers are built without a shift of their own. Euler and LCM run those sigmas plus the requested terminal sigma via a newset_timesteps_from_sigmas()helper (shared with FLUX.1's Euler); Heun runs them viaset_heun_timesteps_from_sigmas()from Fix FLUX scheduler integration #9724.order=1on both; FLUX.2 previews are taken from the sample and sigma of the model evaluation, as on FLUX.1.Same seed on an RTX 4090, mean absolute pixel difference (0–255) against the Euler result of the same graph, before → after:
add_noise=false) vs single Euler runRelated Issues / Discussions
Closes #9792
Follows #9724, which added the Heun helper for FLUX.1.
QA Instructions
Automated:
uv run --no-sync pytest tests/app/invocations/test_denoise_noise_inputs.py tests/app/invocations/test_flux2_denoise_wiring.py tests/app/invocations/test_flux2_denoise_img2img_normalization.py tests/app/invocations/test_flux2_working_memory.py tests/app/invocations/test_z_image_schedule_shift.py tests/app/invocations/test_z_image_denoise_working_memory.py tests/app/invocations/test_z_image_denoise_schedulers.py tests/app/invocations/test_upscale_index_clipped_starts.py tests/app/invocations/test_module_compatibility.py tests/backend/flux tests/backend/flux2 tests/backend/architectures/test_scheduler_set.py→ 525 passed, 4 xfailed (the known LCM preblend cases).main: terminal sigma, Heun schedule and progress, previews from the evaluation sample, shift-before-clip on txt2img, Z-Image Heun schedule/img2img start/progress, and a node built with a scheduler that shifts again (shift=3.0).uv tool run ruff@0.11.2 check/format --checkon the changed files: clean.pnpm lintpassed;vitest run src/features/upscale97 passed;vitest run --config vitest.browser.config.mts src/app/UpscaleCapabilityGate.browser.test.tsxpassed (fails when Z-Image's list is reverted to Euler only).Manual (RTX 4090, FLUX.2 Klein 4B fp8 and Z-Image Turbo int8, 1024², 8 steps, seed 42): the graphs in the table above, run against
mainand this branch on the same data root. In the Upscale widget (Vite dev server against this branch), Z-Image's Scheduler field lists "Euler" and "Heun (2nd order)"; a 2× Z-Image upscale with Heun ran end to end and matched the Euler upscale visually.Not run: the full Python suite and
pnpm check:release; no GPU check on AMD/ROCm or macOS.Review
Resolved findings:
shift=0(allowed by the field'sge=0) evaluated the model at sigma 0 and returned NaN latents. Heun now skips the schedule's zero-length intervals, so it takes the single 1 → 0 step Euler takes and its progress counts the steps it runs. The field keeps its range, so saved workflows still load.Remaining limitations:
timesteps[0]).timestep=int(t_curr), i.e. 0, in its progress state (pre-existing).Compatibility / Rollout
denoising_end < 1, which now returns latents at sigma1 − denoising_endon the shifted schedule instead of a finished image. Generate, Canvas and Upscale always senddenoising_end = 1; only workflow-editor graphs use it.invokeai.backend.flux2.denoise.denoise()no longer takesmu, andFLUX2_TXT2IMG_SCHEDULER_KWARGSis gone frominvokeai.backend.flux2.sampling_utils. Neither is part ofinvokeai.invocation_apiand nothing in the repo uses them; out-of-tree code that forked_run_diffusionfails with aTypeError/ImportErrorinstead of shifting twice.scheduler.step(...)line ininvokeai/backend/flux2/denoise.pyand the removedtest_flux2_lcm_scheduler_setup_passes_mu; whichever merges second keepsgenerator=generatoron the step call and drops that test. There is no semantic conflict.Checklist
What's Newcopy (if doing a release after this PR)Need help on this PR? Tag
@codesmith-botwith what you need.