StreamDiffusion v0.1.2 TOX 🚀🖼️
Prompt Blocks added for mixing multiple prompts with per prompt weights
- 'Normalize Prompt Weights' toggle parameter
- 'Total Weight Slider' parameter to set amount to normalize prompt weights.
Seeds Blocks added for mixing multiple seeds with per seed weights
- 'Total Seed Noise Multiplier' parameter to boost/reduce overall noise
- NOTE: Mixing the same seed isn't recommended, but mixing different seeds should blend.
Improved Installation Process (all moved to the "Install" parameter page)
- Checks for correct python version (3.10) and cuda version (11.8 or 12.1)
- Paired down to essential imports + preps for tensorrt installation.
- Additional Installation Pulse for tensorrt. Need to run after main install. But should be working.
- if you had issues before. delete the venv folder and try with this operator.
- IMPORTANT NOTE [ for TensorRT ]: it will take around 10 minutes or so for the model to build an engine the first time you launch with tensorrt. BE PATIENT. there will be minutes where it seems like nothing is happening. This process only needs to happen once per step count / model id. And the engines that are made are cached inside StreamDiffusion/engines in case you want to delete them later.
Other fixes/improvements !
- Fixed the issue that requires the enable/disable after stream starts
- Fixed Spout BGR color issue.
- Improved handling of NDI 'stream in name' to ensure unique name from the StreamDiffusion server
Parameter Menu Pages Cleaned.
- Settings/setup changed to Settings 1/2 Page
- Settings 1 page is now dedicated to settings that can be changed when the stream is active.
- Install page added for dedicated installation/Basefolder parameters
Comments (29)
Really a wonderfull tool! Love your work, hope our path's with cross somday!
not working with cuda 12.1 @dotsimulate :(
@dot thers a way in the development of your code in TD to check different version of cuda or installing parallel cuda (like pyenv for python for example) to perform the last cuda version or the best for the machin?
Hi there! Thanks a lot for this! Unfortunately after installing everything and press "Start stream" a cmd window appears and then goes away without giving any error message. Stream Info says "Not Streaming". Any idea of what could be the problem? I've also tried with the first version and I have the same problem... Thanks for your help and work!
Ignore me, I forgot to install the NDI... Once I did that and reinstalled everything again it worked perfectly. Thanks so much! This is so much fun :)
I am unable to load sd-turbo by downloading the safetensors file from hugging face (same ID as the default you have). In the load it says path is not a valid local file (and for sure path is valid). I would ideally like to be able to use this disconnected from internet. But its super amazing otherwise thank you so much for making this.
Yep! I looked into this today for a good bit. Going to push an update that supports fully local. I wasn't aware that it was checking with huggingfacehub for models details each time the stream starts.
This is so amazing thank you so much!!! I'm having difficulty getting a lora working with this (using SDXL Loras), do you have to write something in the prompt to activate it similar to automatic1111? Thanks :)
sdxl isn't supported by streamdiffusion ! sd-turbo is 2.1 based model and doesn't have many or any loras that work I believe. 1.5 models use the LCM lora and you can load loras with them.
I downloaded the python 3.10.9 version but I get the error on the TD window "Python 3.10 executable not found. Please ensure Python 3.10 is installed and accessible" now I was wondering how to make the executable accessible?
Sorry for not seeing this ! hopefully you made some progress .... but ! if python is installed the operator should be able to find it. Try reinstalling or modifying and fixing the install.
This is an amazing tool, thank you so much. Is there a way to use other checkpoints I want with Stream Diffusion, or can I use other checkpoints as I do in Comfy UI? I saw some other turbo models on the Civit AI website, but I don't have much of an idea on how to use them with Stream Diffusion. Thanks in advance. Best regards.
i'm spending a lot of time trying to make it work, but won't work... my last issue is: ERROR: Package 'streamdiffusion' requires a different Python: 3.9.13 not in '>=3.10.0' as you suggest, but uninstalling 3.10 and installing 3.9 doesn't affect the result, they're added to path by installation checkbox. Then when i try to start streaming it say: Error caught was: No module named 'triton' that i'm not able to solve. ModuleNotFoundError: No module named 'pytorch_lightning' any suggestion? please
about Triton , it s a "normal" bug we all have, but doesn"t affect the program as far as it s been said ^^ for python 3.9, tryc to check your env variable PATH and if the 3.10 isn't still there or placed above/before your 3.9 version.. after that i d suggest deleting the 'venv' folder inside your streamdiffusion folder, and from TD2023 opening streamdiffusion toe, and add the main folder path, and update (first button) (read the yellow message to see about update state), reinstall venv with second, and install tensorRT with third .. save your project and restart ... just advices, i m far from being an expert but if it can help a bit .. :)
please update to use with conda
Hey there, thank you for your awesome work! Im itching to try out stream diffusion and i think i did all the steps to install all the librarys etc but it doesnt seem to work at all. All i get out when i connect a TOP to the Stream Diffusion Base is the exact same image as the input. Do you have a suggestion what i did wrong? Thanks!
So genius! It's great that TensorRT was added to the installation flow! Thank you for your great work!
Hello! Thank you very much for this! I bought the .tox and when it's installing TensorRT, it's not progressing while creating the wheel for TensorRT (pyproject.toml).
Hello, can I have the discord please, so I can discuss some issues I had with the tox
Hey man! Amazing work! Was just wonedering how can I know for sure that TensorRT is installed correctly and activated? Also, what degrees of speedups could one expect with it? :)
When it is running there will be a clear printout that says 'TensorRT acceleration enabled.' You need to set the acceleration mode under the model parameter to tensorrt, and first time loading it will take a few minutes to build the tensorrt engine. Engine is built based on the model name + step count, and if those settings are the same it won't rebuild when relaunching.
Perhaps I'm missing something here, but I'm only able to get TensorRT to work with the default 512x512 resolution. When I try to rebuild at a different resolution, it doesn't seem to make much of a difference. Any tips?
Yes. looked into this for a bit today and wasn't able to get things working. Not too sure on that front currently
What a cool tool bro thanks!!!! I'm having an issue: Started with ndi input using CMD. Streming info - not streaming. I've installed apart: NDI SDK, python3.10, linked the model v1-5-pruned-emaonly.ckpt, also ndi/spout installed through the tox 1.1, and the three buttons completed without errors. The error says: unable to create ndi sender for specific name.. but changing name won't affect. Any suggestion? thanks a lot do you prefer sd-turbo instead of sd1.5?
Works amazing! Thank you for your hard work!
Stoked!!! Finally got a chance to try it out today for my first time. This is just beyond rad! Now another update so fast. Hats off man!!!
Cant wait to try this!
Mate, what a release! (easily v0.2 material)
Thanks !! and true. it is a bit more than a patch...