[REQ] Better VIDEO super resolution models

[MarcoRavich]

Hi there,
just discovered this interesting HW-optimized library for IMAGE super resolution.

As well explained in this demo video for VideoGigaGAN (paper here) by University of Maryland/Adobe Research, there are multiple factors to consider for "correct" VIDEO super resolution inferencing:

https://github.com/videogigagan/videogigagan.github.io/raw/refs/heads/main/assets/videos/demo.mp4

Anyway, since VideoGigaGAN sources aren't available, I recommend you to "port" alternative (more contemporary, if possible) VIDEO processing-oriented algos for better/consistent results...
...here are some interesting projects - you can already exploit - to carefully evaluate:

• StableVSR: Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models (paper);
• FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring (paper)
• EvTexture: Event-driven Texture Enhancement for Video Super-Resolution (paper)
• MIA-VSR: Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attention (paper)
• STVSR-NO: Space-Time Video Super-resolution with Neural Operator (paper);
• STINet: Enhancing Space-time Video Super-resolution via Spatial-temporal Feature Interaction (paper);
• GIRNet: Global Spatial-Temporal Information-based Residual ConvLSTM for Video Space-Time Super-Resolution (paper);
  ...

Resources collections:

• HyMPS \ VIDEO \ AI-based \ Upscalers;
• Awesome Video Enhancement \ Video Super Resolution;
• upcoming CVPR 2025, ICME 2025, ICML 2025 and ICCV 2025 results, of course.

Last but not least, I strongly suggest you to enable the Discussion section for this repository in order to let 3rd-party devs/users to exchange ideas & knowledge to push the evolution.

Hope that inspires.

[djie1]

Thank you @MarcoRavich

To enable DL VSR models, we also have the library and FFmpeg plugin: https://github.com/OpenVisualCloud/iVSR, it's based on OpenVINO. For example, it supports the Enhanced BasicVSR model which was optimized based on the public BasicVSR model.

[MarcoRavich]

Just discovered @ArcticHare105's S3Diff (Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors paper implementation) that looks VERY promising too (for image upscale/enhance)...

Hope that inspires !