Cosmos Transfer2.5 Auto-Regressive Inference Pipeline

[miguelmartin75]

What does this PR do?

This builds off #13066 by adding auto-regressive inference for Cosmos Transfer2.5. This pipeline does not require the controlnet or controls to be input. From the documentation:

The call function can be used in two modes: with or without controls.
When controls are not provided (controls is None), inference works in the same manner as predict2.5 (see
Cosmos2_5_PredictPipeline). This mode strictly uses the base transformer (self.transformer) to perform
inference and accepts as input an optional image or video along with a prompt / negative_prompt, and
can be used in the following ways:
- Text2World: image=None, video=None, prompt provided.
- Image2World: image provided, video=None, prompt provided.
- Video2World: video provided, image=None, prompt provided.
When controls are provided and a ControlNet is attached, controls drive the conditioning and video &
image is ignored. Controls are assumed to be pre-processed, e.g. edge maps are pre-computed.
Setting num_frames will restrict the total number of frames output, if not provided or assigned to None
(default) then the number of output frames will match the input video, image or controls respectively.
Auto-regressive inference is supported and thus a sliding window of num_frames_per_chunk frames are used per
denoising loop. In addition, when auto-regressive inference is performed, the previous
num_latent_conditional_frames or num_conditional_frames are used to condition the following denoising
inference loops.

Who can review?

• Pipelines: @yiyixuxu
• Docs: @stevhliu and @sayakpaul

[yiyixuxu]

thanks for the PR!

my main question is would it make sense to make this pipeline strictly ControlNet-focused? looking at the pipeline code, this would simplify the pipeline quite a bit

[yiyixuxu]

thanks!
i left a few more questions/feedbacks, let me know what you think
we can merge this soon!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yiyixuxu]

thanks!
let's fix the CI too

[miguelmartin75]

CI should be fixed now