AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance
Paper • 2311.12886 • Published
How to use zyand/animate-anything-v1.02 with Diffusers:
pip install -U diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("zyand/animate-anything-v1.02", dtype=torch.bfloat16, device_map="cuda")
pipe.to("cuda")
prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)
output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")Given an input image, this model can animate the image under the guidance of text prompt, motion brush, and motion strength.
Given a random size and random aspect ratio image, this model generate 2s videos in 8fps. The resolution of generated video is equivalent to 512x512.