Extending Wan 2.1 generated video - First 14b 720p text to video, then using last frame automatically to to generate a video with 14b 720p image to video - with RIFE 32 FPS 10 second 1280x720p video - Our app has this fully automated
Our Wan 2.1 app has this fully automated : https://www.patreon.com/posts/123105403
Here how it works image : https://ibb.co/b582z3R6
Then use MMAudio to add sound
Here initial video
Prompt: Close-up shot of a Roman gladiator, wearing a leather loincloth and armored gloves, standing confidently with a determined expression, holding a sword and shield. The lighting highlights his muscular build and the textures of his worn armor.
Negative Prompt: Overexposure, static, blurred details, subtitles, paintings, pictures, still, overall gray, worst quality, low quality, JPEG compression residue, ugly, mutilated, redundant fingers, poorly painted hands, poorly painted faces, deformed, disfigured, deformed limbs, fused fingers, cluttered background, three legs, a lot of people in the background, upside down
Used Model: WAN 2.1 14B Text-to-Video
Number of Inference Steps: 20
CFG Scale: 6
Sigma Shift: 10
Seed: 224866642
Number of Frames: 81
Denoising Strength: N/A
LoRA Model: None
TeaCache Enabled: True
TeaCache L1 Threshold: 0.15
TeaCache Model ID: Wan2.1-T2V-14B
Precision: BF16
Auto Crop: Enabled
Final Resolution: 1280x720
Generation Duration: 770.66 seconds
And here video extension
Prompt: Close-up shot of a Roman gladiator, wearing a leather loincloth and armored gloves, standing confidently with a determined expression, holding a sword and shield. The lighting highlights his muscular build and the textures of his worn armor.
Negative Prompt: Overexposure, static, blurred details, subtitles, paintings, pictures, still, overall gray, worst quality, low quality, JPEG compression residue, ugly, mutilated, redundant fingers, poorly painted hands, poorly painted faces, deformed, disfigured, deformed limbs, fused fingers, cluttered background, three legs, a lot of people in the background, upside down
Used Model: WAN 2.1 14B Image-to-Video 720P
Number of Inference Steps: 20
CFG Scale: 6
Sigma Shift: 10
Seed: 1311387356
Number of Frames: 81
Denoising Strength: N/A
LoRA Model: None
TeaCache Enabled: True
TeaCache L1 Threshold: 0.15
TeaCache Model ID: Wan2.1-I2V-14B-720P
Precision: BF16
Auto Crop: Enabled
Final Resolution: 1280x720
Generation Duration: 1054.83 seconds
Comments (10)
When wan2.1 first came out i was using vlc to take a snapshot of the last frame, using that for the same prompt and then using avidmux to join all the videos together, it did have slight issues with keeping the same brightness and colour saturation between the videos, hopefully this way it maintains them for a more clear unsegmentated video
yes this works perfect you can try
Thanks for extending it to 10s. Is it possible to make it Image to Video -> last frame -> next video -> last frame -> next video (and So on) with a continuous single prompt / prompt separately for each 5s? Or am I asking too much? :D
it is exactly working that way. you can set extending count to 4 to get 20 sec :D
On Runpod, the installation of the WAN UI went very smooth, also the model downloader does it's thing. The only issue I have is that no LoRa is listed in the Lora-dropdown, even tho I downloaded some LoRA in the LoRas folder.
linux is case sensitive is your folder name exactly as LoRAs
got this error: TypeError: object of type 'NoneType' has no len()
well give me more info. did you download models?
Thanks for the update. For the "extension video factor (1=no extension)" it might be simpler if 0=no extensions, 1=1 extension). If you set it up so before you've generated the video at all that you set it to generate the video and extend it once, then you won't know if the initial video will come out well enough and whether the last frame of the video is good quality enough to continue with (by the last frame of the original video it might be too messed up and not worth extending). Also if it uses the same prompt for the extension wouldn't that make it likely to try to repeat the actions that were done in the original video even if it wasn't what was wanted (eg. prompt was "person does x and then person does y". Each extension would try to make them do x and then y again if they use the same prompt). It might be that half-way through the video things looked okay but by the end frame the video didn't look good enough. For extending the video you could wait until you've generated the first section and then the user could be allowed to pick the last good frame that they want to continue from (eg. half way through the video) and have the ability to enter a different prompt (so it wouldn't try to repeat what was done originally if that wasn't what was wanted).
agreed would be better to choose after initial first generation