[Breakdown] "Talk" [ft. Jibaro's "Sara Silkin"] - AI Motion Capture Experiment Nº3
motion_ctrl / experiment nº3
choreography: Sara Silkin
dancers: Coco Williams & Joey Vice
vfx: myself
In collaboration with Sara, Joey, and Coco, I transformed a low-resolution recording of this beautiful performance, into this multi-angle audiovisual piece.
Want the exact breakdown of how this was made without ultra-expensive gear or a full production budget?
Breakdown:
I started from the original dance/performance video and split it into 14.9s clips.
Then I used Uisato Studio’s Seedance 2 Video mode, with the "Intelligent" setup and the "Subject Swap / Character Transfer" prompt recipe.
Inputs were:
1. the original performance video as the reference video
2. a target image with the robot / bio-tech aesthetic as the reference image
3. a short director’s intent describing the look, tone, and what I wanted beyond the character swap
From there, Uisato Studio generated the prompt, direction, and optimal setup. I reviewed it, made small adjustments, generated the clips, and then assembled the final piece in editing.
Voilá, multi-character subject replacement on-a-budget.
Comments (3)
beautiful.. love how we really keep the dancers soul via the animation of the performance. Was the change in his feet position (when it flashed to the original plate) an artefact of the model interpretation? Does seed dance come with native swap capabilities via only prompting or were controlNet/skeletons or whatever needed? thanks
Can it remember the same character from video to video if you make a series
Not necessarily, but you can "trick" the model into. What I did to revolve that issue was to generate the first clip with the target image, then for the rest you switch to using a frame from the first generated clip [picking one in which all characters were clearly visible]. I'm sure there's a much more optimal way [further angle generation from selected frame/video to catch all front, sides, and behind of all characters, etc.] My minimal approach isn't bad, but it definitely has room for improvement.