Wan-Animate - A New Era in AI-Powered Character Animation

A new AI framework called Wan-Animate is changing the landscape of character animation and video editing. This innovative tool offers a unified system for both animating characters and seamlessly replacing them in existing videos.

Wan-Animate operates in two primary modes: "Animation" (or "Move") mode and "Replacement" (or "Mix") mode. In Animation mode, a user can take a still image of a character and a reference video, and the AI will generate a high-fidelity animation of the character, precisely mimicking the facial expressions and body movements from the video. The Replacement mode takes this a step further by integrating the animated character into the original reference video, effectively replacing the person in the video with the new character. This mode is sophisticated enough to replicate the scene's original lighting and color tones, ensuring the new character blends in naturally with the environment.

The technology behind Wan-Animate is built upon the Wan AI model. It uses spatially aligned skeletal signals to replicate body movements and extracts implicit facial features from the source image to reproduce expressions with high controllabilty. To achieve realistic environmental blending in the Replacement mode, Wan-Animate utilizes an auxiliary "Relighting LoRA" module. This component adjusts the lighting and color on the animated character to match the surrounding scene while maintaining the character's appearance.

According to reports, Wan-Animate's performance surpasses existing open-source and closed-source models. In a significant move for the AI community, the model weights and inference code for Wan-Animate have been made open source, allowing for wider access and development.