Skip to content

[bug]: ERNIE-Image Heun scheduler refuses every partial denoise (img2img, inpaint, Canvas) #10003

Closed

Description

@Pfannkuchensack

Is there an existing issue for this problem?

  • I have searched the existing issues

Install method

Manual (dev checkout)

Operating system

Windows

GPU vendor

Not GPU-specific (reproduced on CPU)

GPU model

No response

GPU VRAM

No response

Version number

main at cbc5ab0595

What happened

ERNIE-Image with the Heun scheduler refuses every partial denoise. Any img2img, inpaint or Canvas pass with a denoising strength below 1 fails before the first step:

ValueError: FlowMatchHeunDiscreteScheduler does not accept an explicit sigma schedule, so it cannot honor denoising_start/denoising_end. Use the euler or lcm scheduler for partial denoising.

The check is in invokeai/backend/ernie_image/denoise.py (around line 88). Euler and LCM pass the clipped sigmas through set_timesteps(sigmas=...). FlowMatchHeunDiscreteScheduler.set_timesteps takes only a step count, so the node refuses any schedule that does not run from 1 to 0.

Generate offers Heun for ERNIE-Image (scheduler_set="flow" in invokeai/backend/architectures/defs/ernie_image.py), so a user can select it and only finds out when the Canvas pass fails.

What you expected to happen

Heun starts at denoising_start and stops at denoising_end, as it does for FLUX.1 and Z-Image since #9851.

How to reproduce the problem

  1. Select an ERNIE-Image model and the Heun scheduler.
  2. Run img2img or a Canvas pass with denoising strength below 1.
  3. The queue item fails with the ValueError above.

Without a model, the backend call reproduces it directly:

import torch
from invokeai.backend.ernie_image.denoise import denoise
from invokeai.backend.flux.schedulers import ERNIE_IMAGE_SCHEDULER_MAP

img = torch.zeros(1, 4, 4, 4)
denoise(
    model=None, img=img, text_bth=None, text_lens=None,
    timesteps=[0.6, 0.4, 0.2, 0.0], step_callback=lambda s: None, cfg_scale=[1.0] * 3,
    scheduler=ERNIE_IMAGE_SCHEDULER_MAP["heun"](), init_latents=img,
)

Additional context

FLUX.1 and FLUX.2 already drive Heun on a clipped schedule with set_heun_timesteps_from_sigmas in invokeai/backend/flux/schedulers.py. ERNIE-Image could use the same helper. One difference to account for: ERNIE passes raw timesteps and lets the scheduler apply its own shift inside set_timesteps, while the helper expects the sigmas the model runs on.

Discord username

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions