MagicDistillation

Weak-to-Strong Distribution Matching Distillation for Efficient Large-Scale Video Synthesis

Shitong Shao1,2 Hongwei Yi2 Hanzhong Guo3 Tian Ye1,2 Daquan Zhou4
Michael Lingelbach2 Zhiqiang Xu5 Zeke Xie1
1HKUST (GZ) 2Hedra Inc. 3HKU 4Peking University 5MBZUAI

Recently, open-source video diffusion models (VDMs), such as WanX, Magic141 and HunyuanVideo, have been scaled to over 10 billion parameters. These large-scale VDMs have demonstrated significant improvements over smaller-scale VDMs across multiple dimensions, including enhanced visual quality and more natural motion dynamics. However, these models face two major limitations: (1) High inference overhead: Large-scale VDMs require approximately 10 minutes to synthesize a 28-step video on a single H100 GPU. (2) Limited in portrait video synthesis: Models like WanX-I2V and HunyuanVideo-I2V often produce unnatural facial expressions and movements in portrait videos. To address these challenges, we propose MagicDistillation, a novel framework designed to reduce inference overhead while ensuring the generalization of VDMs for portrait video synthesis. Specifically, we primarily use sufficiently high-quality talking video to fine-tune Magic141, which is dedicated to portrait video synthesis. We then employ LoRA to effectively and efficiently fine-tune the fake DiT within the step distillation framework known as distribution matching distillation (DMD). Following this, we apply weak-to-strong (W2S) distribution matching and minimize the discrepancy between the fake data distribution and the ground truth distribution, thereby improving the visual fidelity and motion dynamics of the synthesized videos. Experimental results on portrait video synthesis demonstrate the effectiveness of MagicDistillation, as our method surpasses Euler, LCM, and DMD baselines in both FID/FVD metrics and VBench. Moreover, MagicDistillation, requiring only 4 steps, also outperforms WanX-I2V (14B) and HunyuanVideo-I2V (13B) on visualization and VBench.

Method

fig_pipeline

Experimental Results (General Video Synthesis)

Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕

Experimental Results (Portrait Video Synthesis)

Click on the video and a prompt will appear

Compared Method: Euler (28-step)

A captivating artwork of a fantasy character against a dreamy sky. The character has flowing, wavy pink hair and blue eyes, adorned in an elegant pink outfit with blue and gold accents. She appears serene amidst a backdrop of magical clouds, with streaks of purples, blues, and stars in the sky, creating a mystical and tranquil atmosphere.
A man sits at a desk set against a whiteboard filled with written content in a classroom, wearing a black shirt and an orange-striped tie. He is bald, with intense eyes, and his hands are clasped in front of him on papers. The room has rows of empty desks, framed pictures on the walls, and a clock above a door in the background.
A young man with curly hair and a slight smile stands on a city street wearing a dark blue hoodie. He has a small black backpack and is looking at the camera. Behind him, the blurred background shows city buildings and other passersby, adding to the urban ambiance.
A woman with long, straight, chestnut-brown hair, fair skin, and a subtle smile, wearing a dark outfit. She sits in front of a dual-toned background split vertically - one side is purple and the other side is pinkish-lavender. Her hair flows down her shoulders, and the background colors contrast with her hair and outfit, creating a visually appealing scene.
A woman with long, flowing auburn hair is sitting in an office. She wears a sophisticated dark blazer. The background is a dual-toned, purple and blue gradient. Her warm smile showcases fair skin and flawless makeup.
A young woman with neatly styled dark hair stands by a window, smiling at the camera. She wears a blue, button-down shirt and the natural light from the window highlights her joyful expression. The window behind her displays blurred figures and greenery, creating a softly focused backdrop.

Compared Method: Euler (4-step)

A captivating artwork of a fantasy character against a dreamy sky. The character has flowing, wavy pink hair and blue eyes, adorned in an elegant pink outfit with blue and gold accents. She appears serene amidst a backdrop of magical clouds, with streaks of purples, blues, and stars in the sky, creating a mystical and tranquil atmosphere.
A man sits at a desk set against a whiteboard filled with written content in a classroom, wearing a black shirt and an orange-striped tie. He is bald, with intense eyes, and his hands are clasped in front of him on papers. The room has rows of empty desks, framed pictures on the walls, and a clock above a door in the background.
A young man with curly hair and a slight smile stands on a city street wearing a dark blue hoodie. He has a small black backpack and is looking at the camera. Behind him, the blurred background shows city buildings and other passersby, adding to the urban ambiance.
A woman with long, straight, chestnut-brown hair, fair skin, and a subtle smile, wearing a dark outfit. She sits in front of a dual-toned background split vertically - one side is purple and the other side is pinkish-lavender. Her hair flows down her shoulders, and the background colors contrast with her hair and outfit, creating a visually appealing scene.
A woman with long, flowing auburn hair is sitting in an office. She wears a sophisticated dark blazer. The background is a dual-toned, purple and blue gradient. Her warm smile showcases fair skin and flawless makeup.
A young woman with neatly styled dark hair stands by a window, smiling at the camera. She wears a blue, button-down shirt and the natural light from the window highlights her joyful expression. The window behind her displays blurred figures and greenery, creating a softly focused backdrop.

Compared Method: LCM (4-step)

A captivating artwork of a fantasy character against a dreamy sky. The character has flowing, wavy pink hair and blue eyes, adorned in an elegant pink outfit with blue and gold accents. She appears serene amidst a backdrop of magical clouds, with streaks of purples, blues, and stars in the sky, creating a mystical and tranquil atmosphere.
A man sits at a desk set against a whiteboard filled with written content in a classroom, wearing a black shirt and an orange-striped tie. He is bald, with intense eyes, and his hands are clasped in front of him on papers. The room has rows of empty desks, framed pictures on the walls, and a clock above a door in the background.
A young man with curly hair and a slight smile stands on a city street wearing a dark blue hoodie. He has a small black backpack and is looking at the camera. Behind him, the blurred background shows city buildings and other passersby, adding to the urban ambiance.
A woman with long, straight, chestnut-brown hair, fair skin, and a subtle smile, wearing a dark outfit. She sits in front of a dual-toned background split vertically - one side is purple and the other side is pinkish-lavender. Her hair flows down her shoulders, and the background colors contrast with her hair and outfit, creating a visually appealing scene.
A woman with long, flowing auburn hair is sitting in an office. She wears a sophisticated dark blazer. The background is a dual-toned, purple and blue gradient. Her warm smile showcases fair skin and flawless makeup.
A young woman with neatly styled dark hair stands by a window, smiling at the camera. She wears a blue, button-down shirt and the natural light from the window highlights her joyful expression. The window behind her displays blurred figures and greenery, creating a softly focused backdrop.

Compared Method: Vanilla DMD2 (4-step)

A captivating artwork of a fantasy character against a dreamy sky. The character has flowing, wavy pink hair and blue eyes, adorned in an elegant pink outfit with blue and gold accents. She appears serene amidst a backdrop of magical clouds, with streaks of purples, blues, and stars in the sky, creating a mystical and tranquil atmosphere.
A man sits at a desk set against a whiteboard filled with written content in a classroom, wearing a black shirt and an orange-striped tie. He is bald, with intense eyes, and his hands are clasped in front of him on papers. The room has rows of empty desks, framed pictures on the walls, and a clock above a door in the background.
A young man with curly hair and a slight smile stands on a city street wearing a dark blue hoodie. He has a small black backpack and is looking at the camera. Behind him, the blurred background shows city buildings and other passersby, adding to the urban ambiance.
A woman with long, straight, chestnut-brown hair, fair skin, and a subtle smile, wearing a dark outfit. She sits in front of a dual-toned background split vertically - one side is purple and the other side is pinkish-lavender. Her hair flows down her shoulders, and the background colors contrast with her hair and outfit, creating a visually appealing scene.
A woman with long, flowing auburn hair is sitting in an office. She wears a sophisticated dark blazer. The background is a dual-toned, purple and blue gradient. Her warm smile showcases fair skin and flawless makeup.
A young woman with neatly styled dark hair stands by a window, smiling at the camera. She wears a blue, button-down shirt and the natural light from the window highlights her joyful expression. The window behind her displays blurred figures and greenery, creating a softly focused backdrop.

MagicDistillation (4-step)

A captivating artwork of a fantasy character against a dreamy sky. The character has flowing, wavy pink hair and blue eyes, adorned in an elegant pink outfit with blue and gold accents. She appears serene amidst a backdrop of magical clouds, with streaks of purples, blues, and stars in the sky, creating a mystical and tranquil atmosphere.
A man sits at a desk set against a whiteboard filled with written content in a classroom, wearing a black shirt and an orange-striped tie. He is bald, with intense eyes, and his hands are clasped in front of him on papers. The room has rows of empty desks, framed pictures on the walls, and a clock above a door in the background.
A young man with curly hair and a slight smile stands on a city street wearing a dark blue hoodie. He has a small black backpack and is looking at the camera. Behind him, the blurred background shows city buildings and other passersby, adding to the urban ambiance.
A woman with long, straight, chestnut-brown hair, fair skin, and a subtle smile, wearing a dark outfit. She sits in front of a dual-toned background split vertically - one side is purple and the other side is pinkish-lavender. Her hair flows down her shoulders, and the background colors contrast with her hair and outfit, creating a visually appealing scene.
A woman with long, flowing auburn hair is sitting in an office. She wears a sophisticated dark blazer. The background is a dual-toned, purple and blue gradient. Her warm smile showcases fair skin and flawless makeup.
A young woman with neatly styled dark hair stands by a window, smiling at the camera. She wears a blue, button-down shirt and the natural light from the window highlights her joyful expression. The window behind her displays blurred figures and greenery, creating a softly focused backdrop.

Comparison between Magic141, MagicDistillation, HunyuanVideo-I2V (13B) and WanX-I2V (14B)

Loading...
☕
Loading...
☕
Loading...
☕
Loading...
☕

Quantitive Evaluations

BibTeX

@article{shao2025magicdistillation,
      title={MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesis},
      author={Shao, Shitong and Yi, Hongwei and Guo, Hanzhong and Ye, Tian and Zhou, Daquan and Lingelbach, Michael and Xu, Zhiqiang and Xie, Zeke},
      journal={arXiv preprint arXiv:2503.13319},
      year={2025}
    }