StreamDiffusionTD 0.1.0 🚀🖼️ (NEW)
✨ Hey friends! I'm excited to share this install/start-up guide for StreamDiffusionTD.
( TOX available now in separate post )
This operator is built in TouchDesigner 2023 and won't be compatible with earlier versions! Also, projects saved in 2023 cannot be opened in 2022. See the release post for more - Touchdesigner 2023 Official Update
🛠 Installation Tips!! (working on a readme + smoother op install process)
Python v 3.10.9 💎is tested and working! Other versions work, but may run into issues with the installation process.
+ NDI-SDK, Python (3.10.9) and GIT are required for everything to install smoothly.
Installing NDI-SDK (needed for ndi-python) - https://ndi.video/download-ndi-sdk/
Adding GIT to PATH on windows - download / install & check add to PATH when asked if already installed and not in PATH - https://www.delftstack.com/howto/git/add-git-to-path-on-windows/
👋 Let me know if you run into issues! This is v1 ! Reach out ( Discord is best ) if you have any questions or want to share what you've created with StreamDiffusionTD. Always great to connect with everyone and see how you all use these tools.
StreamDiffusion github - https://github.com/cumulo-autumn/StreamDiffusion
Happy creating. More coming soon.
- Lyell
Real-time diffusion in TouchDesigner - StreamdiffusionTD Setup + Install + Settings (YouTube)
Comments (44)
I get this error Warning: No CUDA devices available. Defaulting to CPU. Traceback (most recent call last): File "C:\Users\ricar\Downloads\streamv2v-main\streamv2v-main\StreamDiffusion\streamdiffusionTD\main_sdtd.py", line 84, in import cv2 ModuleNotFoundError: No module named 'cv2' [proceso terminado con el código 1 (0x00000001)] Ahora puede cerrar este terminal con Ctrl+D o presionar Entrar para reiniciar.
I get this error
I get this error
I am having problems not wanting to work when I start streaming. The CMD page always closes the window automatically, and the information is Launching StreamdiffusionTD. What's wrong with this? Please help.
Hey ! there is a DEBUG command toggle in the settings 2 page that will keep the command window open when there is an error.
also I was wondering how to export single images from TD using the above method also can we use safetensors and checkpoints within? I'm sorry for the waterfall of questions but im really new to this
Hi where is the link to discord?
I actually have one more question: Did anybody figure out how to run SDXL with streamdiffusion? I tried to wade through the code but I'm really green in terms of the diffusers library. It seems like replacing some StableDiffusionPipeline calls with StableDiffusionXLPipeline would do the trick but some quick tests showed me that it didn't :D I now see that I have another question, is there some "low vram" mode in streamdiffusion so that I can load larger models (like in comfy or automatic webui)? (I now realize that these are actually streamdiffusion questions, I hope I'm not being irrelevant here)
I'm trying to find a way of using a server-client setup and to connect to SD on my RTX pc, while using TD from my macbook. I started the server (pulsed the stream) from the pc, then shut down TD. on the mac, with NDI tools, i can see the output stream. however: a) StreamDiffusion-TD doesn't see the remote NDI stream b) can't figure out how to send the out stream to the remote NDI server. any ideas?
Great work man -like all your works!- really appreciate this. I run everything just fine. I installed all the missing -but not required- stuff that couldn't be installed automatically, and I'm getting about 5-6 fps on RTX3060 mobile with 6gb vram. Something that I could not figure out though: I used "stabilityai/sd-turbo" as model in the operator parameters. The scripts downloaded the model, but i can't figure out where? not in streamdiffusion/models, there's nothing called sd-turbo anywhere in the directory. where does this one put the downloaded model? :D
It is stored in huggingface cache. and links with the model based on the Huggingface id. C:\Users\\.cache\huggingface\hub\models--stabilityai--sd-turbo
Im getting this error when i try to run `Install ndi/spout req only. The system cannot find the path specified. Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases. Do you know what can resolve that problem ?
hey ! try with the 0.1.2 operator. also the python not found, might be python not installed or in your user path. https://realpython.com/add-python-to-path/
Hello, I am Mexican and I am having problems installing what is necessary for it to work correctly, I don't know if it is the language but I am a little frustrated. I have already installed everything necessary but when I install SD environment it sends me errors and does not allow me to continue
hey. if you are able to shot a message via discord with screenshots of the error, I am happy to look at it. Patreon is not ideal for the troubleshooting stuffs.
hi i’m getting a weird error saying "streamdiffusion" requires a different python 3.9.13 not in >=3.10.0. i tried to install the 3.10.9 as you mentioned above but it seems like it wants 9?
Hi, I'm getting an error when starting the stream for the first time. imoprt NDIlib as ndi ModuleNotFoundError: No module named 'NDILib' I also noticed that I get an error when running the ndi-python install from the settings: error: legacy-install-failure Encountered error while trying to install package. -->ndi-python So I'm guessing I've got something wrong when I install this package. I've done all the installing through the steps inside the tox file. So I'm not sure if it's possible to have a conflict with Python or any of the other modules. Any thoughts? Thanks!
Hey! What CUDA version are you running at? 11.8?
Yes. cuda 11.8 and python 3.10.9 I should have noted or had an option for that. if using 12.1, you could manually install via the github, or edit the installStreamDiffusion.bat file to reflect the correct Cuda for your system,
Hello there, Thank you for sharing this file! When I start streaming it gets stuck at "Loading pipeline components...: 60%" What could be the reason?
Not sure why it would get stuck there. I would try shutting down by closing the cmd window, and then relaunching.
This is incredible! I only hit 4 FPS however at max, on an RTX3070-Ti. Incredible nonetheless!
anyone have some tips on how to get better FPS? or it depends mainly on the hardware? (i'm using a NVIDIA GTX 3000 ADA GENERATION, w/ 8GB vRAM)
Yes on GPU as well as CPU, since the VAE model is run with CPU. tensorrt seems to be the largest improvement in speed, but is a pain to set up. Hoping to get enough feedback on setup from people with that though in order to put together a more useful installation cheat sheet or something for it.
for those wondering: I had python 3.12.something and things didn't work, uninstalled and re installed the 3.10.9 and everything works great.
Hi Thank you for the cool project. I followed every step (and have python 3.10.9 installed) but when I click on Start Stream Palse, I get this error in CMD: File "C:\StreamDiffusion\StreamDiffusion\venv\lib\site-packages\requests\adapters.py", line 501, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: (ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)), '(Request ID: 9e4fb7bc-2849-4433-a8ad-ff96428da757)') Model load has failed. Doesn't exist. I have also tried to manually download the repo for stabilityai/sd-turbo on my machine and set the Model ID / Path to the local directory or the safetensor. Neither work. I have also tried your v 0.1.1
A few other people have run into that. A solve I believe has been downloading the model locally, but seems like you tried that as well. Have you tried setting the model parameter to a local 1.5 model.safetensors file?
is that work with non commercial version ?
I have not specifically tested but it should.
You freaking did it man!!!! Eeeeeek!!! I’m so god damn excited to try it!!!! Absolutely blown away!!!!!
THIS RULES
hi hi :) is there a solution for M2 mac user?
Not currently. But is something I am thinking about + looking into.
Another awesome integration, thanks to you. What a time to be alive!
It's very ingenious and greatly stimulates my creativity. Thank you for sharing this! Do you support TensorRT?
Yep! Tensorrt will definitely work. You can set the acceleration mode under the model parameter. Installation is a bit trickier for that but it’s working for several people. Gives a good 5-10 fps boost.
Got it working with python 3.10.11 -- this is siiiick! Thx
hey i seem to have a problem when i push the ndi install on the SDTD1 comp it open the CMD window but says : Encountered error while trying to install package. ╰─> ndi-python any help?
hey ! just edited the post and added the links for additional needed install stuff. Also seems like python version 3.10.9 is working best.
Thanks Legend.
crazy, thank you !
yessss oh my, so excited about this!!
Thanks a lots !!
Thank you for this marvellous tool you gave us dude !!!
