What do you wanna see next? 馃
I'm currently working on a few videos but not sure on what I should focus on.
Comments (58)
I'm after Vid2Vid workflows that consistent but aren't too crazy. Civitai have some great looking results from theirs but I can't keep up with them in their tutorials and get errors all the time, probably because I'm not knowledgeable enough to understand what to do when a model link give me a 404 and even then I'm not sure where to place it within the ComfyUI folder structure. Cheers.
Is this a video creator that might be good for a local install option? https://huggingface.co/spaces/maxin-cn/Cinemo It seems a bit like EasyAnimate but a bit different. Maybe there's less motion. It can add distortion but sometimes not as much as EasyAnimate. Maybe I'm not using enough steps on the demo though. I assume with a local install it could have higher quality due to higher resolutions and it setting the right resolution based on the source image. It might allow other parameters to be set too locally (like maybe how many frames to generate).
I see nothing exceptional here, looks like any other open source img to video tools. Same bad quality, only 2s of video and it probably also takes a lot of resources and time to generate those 2s
AFTER THE LAST 1 CLICK INSTALLER, LIVE PORTRAIT, I CANCLED MY MEMBERSHIP, TO DAMN COMPLICATED AND IN THE END IT DIDNT EVEN WORK
Dm me man, already answered in another post
HERES WHAT ID LIKE TO SEE 1 CLICK INSTALLERS THAT ARE ACTUALLY 1 CLICK AND IT WOULD BE GREAT IF IN THE END THEY WORKED
This new version of EasyAnimate was released recently. There's demo clips in a video on the link: https://www.reddit.com/r/StableDiffusion/comments/1dzjxov/an_opensourced_textimage2video_model_supports_up/ It's not perfect and the online one isn't, but the online one is more restricted in the number of frames it can output and the resolution. So it would be good if there could be installation info/video for this if it's okay (if it's open source I assume there's nothing unsafe), since the demo clips are quite good.
I tried it...yeah...still not good, those are some very heavily cherry picked results...I'll soon have an "animated" video coming soon though
Will there be videos about installing Stable Diffusion 3 Medium (and other versions when it comes out) for use with Automatic1111 please? I read that Automatic1111 has recently been updated to support SD3. Would it be a couple of clicks thing? I know SD3 Medium isn't perfect right now but since the license that they started off with has got better it's probably worth using now, eg. if the T5 text encoder is enabled it might produce output that better fits what the prompt asks for including text (though I tried the online demo and that keeps getting text that's a bit long wrong, but I'm not sure if they/re using the T5 text encoder in that demo). Maybe there could also be installation guides for other free image generators if they're similar but might be better in some ways, as well free video ones.
I won't be making an install guide for SD3 for now, since we'll get a new SD3 model anyway "soon", so i'll wait until it comes out. SD3 can already be used inside auto1111 though, just update the webui. And since the license for SD3 has been changed, once the new SD3 model drops, If it is done correctly, people will want to start using it, for now after seeing what other models made by other companies can do, I think our best bet is to wait for the release of the ("SD 3.1")
An updated LORA video for the new version of the UI would be very helpful.
Hi, have you thought about doing image generation for sillytavern?
yes I might soon do an update
Hi :-) is there a local install for animating a person to create a video with the TTS-Voice creator?
there might be one soon actually, I'm waiting for the source code to be released
The big A has made an installer for oobabooga. So I have that installed. I'll give it a go.
Ok, thanks for the reply.
Do you have any suggestions about the best software to use for generating stories locally? I have read that some people use Kobold. Any comments on that? I'm also interested in the possibility of being able to train large language models, similar to what we do when making LoRas, I suppose. Any advice/comments would be welcome. If you make a video discussing this subject that would be great. Many thanks.
Hi, depends what exactly you mean by generating stories, kobold is a way to run LLM like text gen webui that you can also use for training LLM or some other webui. Now this really depends on your GPU but personally I haven't done it myself yet as the uses cases for it is definitely much more narrow than the normal "LORA" SD training, I might do a video on it one day maybe but not right now
Secondly, I have recently been reading about AI story generation and seen that this can be done online using ChatGPT. However, I feel uncomfortable about big brother watching, so would prefer to do this locally.
Hi Aitrepreneur. Firstly, thanks for all your great videos on the latest AI tech and how to use it locally; especially the "one click" installers which take the pain out of installing these programs for us computer-unsavy folks.
Things I care for: - Image2Image generation (free or very well made options) - Drawing with AI like Krita and Photoshop with SD support. - AI voices - AI text2speech - AI Lyrics to music - AI spelling support, text style, text rephrases, text generation from notes. - AI image2video - AI video2video _ AI Text2Video - Ai Animations - Ai Text, Image to 3D - AI Image, Video lip sync from audio (no video reference). - Search on windows (Better local search options with AI) - AI search engines for the web. - AI Assistance/Agents - AI Editors - AI Translators - AI text generators for things like factual text with images, (Papper assignments for students with questions and all). - AI on mobile - AI on PC
well that's pretty much what I like as well 馃槃 so whenever I find stuff I really like I'll do a video on it
Will there be video creation related videos in future please? eg. is AnimateDiff the best of the current ones (for most frames at 1024x1024) - it seems better at generating a bit longer videos than HotShotXL that doesn't seem to work properly at 1024x1024. Though AnimateDiff at that res is very slow (especially when using FreeInit to make it more temporally consistent. I know there's Stable Video Diffsuion but I haven't tried that and it might be better to wait for SD3 for something like that (eg. more rights). Are there better video things that could be used (eg. with automatic1111)? There was a recent demo of StoryDiffsuion that's supposed to create videos up to 30 secs long according to one video and the site says "compatible with all SD1.5 and SDXL-based image diffusion models". There's a github page for it. Is that easy to install and use? Would it work through Automatic1111 or do you need to do something else?
unfortunately everything that has to do with video generation is still really bad, it's basically unusable so I won't be making videos about that until we have something good
Please figure out how to use AutoGPT with self hosted (LM Studio) LLM. It's GOT to be possible.
I want video generation software that does more than 2 seconds of video.
I'd love to see furry anthro or feral artstyles :3
I would love to see (not sure if possible) the online ai where they take one signer's voice signing another signger music (like bing crosby signing baby got back). What is out there is limited and this would be cool, i think.
no person in the discord, very inactive
Can you do a video on voicecraft? there's no easy way to install it or videos that show how to do it on windows that I know of. https://github.com/jasonppy/VoiceCraft
how to use 1 cartoon image to create 1 model instead of several images to train a lora
I want something related with uncensored models
Also I'd like to see a really quick and easy way of rendering characters or objects over transparent backgrounds in stable diffusion (png+alpha). I think it was shown as a controlnet thing somewhere but showing the easiest ways would be good
Chatbots, roleplay, NSFW
Who doesn't like the idea of tasteful NSFW rp chat? Ideally having the chat bot use a voice from a 10sec clip of another person or char from a game.
I can easily train faces into Lora now, but poses with a given environment are still difficult because Lora takes over the style of the images. So I'd be curious to know how to train poses with environments. (For example, holding a ball in her hand in a split image or petting a puppy in lap but the person's head is not visible.) Unfortunately, Controlnet doesn't always give good results, or in Lora's case, other environments (such as location) would be more flexible, and if the person is not facing or their head is not visible, OpenPose is not usable.
I have the same challenge.
Ai agents. I鈥檓 working on autogen but autogpt or other options I鈥檓 too busy to figure out. Opinions on favorites would be cool
The ability to add Instant Style into Automatic like you did with Instant ID.
What about train an Slider LORA? Using LECO and so on. Or... Dora?
I'd like "make your own Lora and models" videos
I would love to see an update to the Silly Tavern where we fully integrate the ability for the character to generate images through Stable Diffusion. I'm using Forge UI, and wouldn't mind seeing an update on how to use Instant ID with that UI as well - seems a slight bit different from A1111.
Voted Something Else, but relates to AI Agents. Remember Silly Tavern and what makes it possible to use Oobabooga as a backend? API Calls. I don't just want to look at AI Agents, but ones that can develop API calls to have things like Silly Tavern running in Unreal Engine for example. I've seen some really cool demos with LLMs being used to power fully 3D characters in NSFW software like VAM, also see plugins like "Voxta" which looks pretty cool, but it would be better if we could just have our own custom tailored tools with an AI agent to add new functionality? There's some AI Agents that can fix bugs on GitHub projects, but that's a bit limited. There's a lot of AI Agents out there, but some are pretty limited.
My evergreen problem: upscalers
It would be awesome to show easiest tool/way to make moving pictures or deep fakes. A lot of tools I have used are very inconsistent.
I too would love to know if there is a local pic to vid
AI agents sounds good if it's all with local language models (not the paid type) and can do useful things. And the video could show other useful things that can be done with large language models and showing which are the best for various things. Maybe you could show training a Lora in a large language model using automatic1111 and show the best way of that so it would have custom knowledge. You could also talk a bit about what stable diffusion 3 is probably going to be like in a video and when it might be released, what things have been improved, the new features (eg. if video is an option). Maybe you could also talk about other video options/extensions and which are currently the best (or maybe it's just best to wait for SD3 for that). Also maybe show good ways that the API could be used in oobabooga so it could be used from other apps (python ones and if possible other ones) to do clever things.
a little greedy but maybe a video on compfyUI
I'd still love to see how to do either wav2lip or talking heads in silly tavern
I have a very special person who currently has just a single image, presenting with a single emotion. I would LOVE to be able to make it so that they can have different emotions with AI art generation!
Which I would integrate into Silly Tavern.
Would love to find a great prompt generator or trainer
As mentioned before, a tutorial on slider training would be AWESOME! It's kind of tricky, but the applications are WILD.
Something with video in Stable Diffusion wouldn't be bad: Cause I can't deal with it at all ... and help for the totally unskilled is most welcome :)
A floor plan generator be nice, all those are locked behind paywalls like 119 usd per project pay walls
Dumb question: what are Ai agents?
I wanna know too haha
