More Works from Our Lab

Paper Title 1

Brief description of the work and its main contribution.

Conference/Journal 2024
Paper Title 2

Brief description of the work and its main contribution.

Conference/Journal 2023
Paper Title 3

Brief description of the work and its main contribution.

Conference/Journal 2023

Bi-directional Autoregressive Diffusion for Large Complex Motion Interpolation

Yongrui Ma1,2, Shijie Zhao1*, Mingde Yao2,4, Junlin Li1, Li Zhang1, Xiaohong Liu5, Qi Dou3, Jinwei Gu3, Tianfan Xue2,4*
1ByteDance Inc., 2CUHK MMLab, 3CUHK, 4CPII under InnoHK, 5Shanghai Jiao Tong University
CVPR 2026
*Indicates Corresponding Authors
Paper Supplementary Code arXiv
Teaser image showing realtime video frame interpolation

The proposed ARVFI interpolates a sequence frame based on previous predictions, while the current full-sequence interpolation method, such as Wan, generates all intermediate frames simultaneously. This bidirectional autoregressive interpolation scheme mitigates increasing FID errors as frames move away from the input frames and generates more continuous and consistent results, as shown in the bottom left figure. Additionally, because a frame is predicted based on all previous interpolation results, the diffusion network can interpolate with fewer diffusion sampling steps and superior efficiency. Our RDVFI accelerates Wan by 3x with higher interpolation accuracy (FID score in the bottom left figure) and visual quality (see yellow boxes).

Abstract

Despite recent progress, diffusion-based video frame interpolation methods still struggle with large, complex motions, resulting in discontinuous motions and inconsistent object appearances across frames. We observe that these limitations arise from both the current full-sequence interpolation strategy and the pixel reconstruction training objective. To solve these challenges, we propose ARVFI, a novel video diffusion-based interpolation method for large complex motion interpolation. Instead of generating all intermediate frames simultaneously, ARVFI interpolates in an autoregressive manner from two input frames to the middle ones. Thus, ARVFI interpolates a frame that is further away from the inputs based on all previous interpolation results, resulting in smoother motion transitions and better temporal consistency. Additionally, ARVFI further utilizes DINOv3 features as motion representations, which provide high-level semantics for accurate motion estimation, compared with a simple pixel-level loss. With all these designs, ARVFI generates the intermediate DINOv3 features first and then the frames with an effective conditional generation method for frames. Our ARVFI consistently outperforms existing methods with superior interpolation accuracy and visual quality.