StreamDiffusionTD 0.1.11 🚀🎮🖼️ [ Changelog]

help me solve this problem, I'm quite frustrated looking for this problem : from huggingface_hub import HfFolder, cached_download, hf_hub_download, model_info ImportError: cannot import name 'cached_download' from 'huggingface_hub' (D:\StreamDiffusion\venv\lib\site-packages\huggingface_hub\__init__.py) [process exited with code 1 (0x00000001)] You can now close this terminal with Ctrl+D, or press Enter to restart.

Hey! Couldnt get the TensorRT to work, and can't find the Discord for support :)

Hi, I've been experimenting with this this Tox for StreamDiffusion and really like it; except that the images that I'm getting are far from great (txt to img), no matter how well defined and detailed is the prompt that I am using. SD Turbo model works best that others but is still far from good images. Any advice?

Its running perfectly, can anyone guide me how to use it on GPU. thank you

absolutely STOKED to have controlnet added thank you so much!

Hello DotSimulate. Thanks for the GREAT tool. Sadly I cannot run it. I have a Razer laptop with an RTX3080 Ti . Do not know if it's powerfull enough . Anyways the error I get when pulsing the "Start Stream" is ModuleNotFoundError: No module named 'diffuser' in the cmd window. (ps: it opens and closes quickly so I did a screen recording to read the error. ) Any clue on what this is ? I've searched this thread but not found any info. Thanks in advance

hey! I have a trouble running it. I followed the step-by-step tutorial, but it doesn't seem to work. Where can I tell you about the error? First it's "A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton'" and then while running the model ESRGAN (that work normally fine in automatic 1111), it shows me this exception 'OSError: Error no file named model_index.json found in directory D:\ARTISTIC PROJECTS\2024\Stable Diffusion AI\stable-diffusion-webui\models\ESRGAN.' & "UnboundLocalError: local variable 'pipe' referenced before assignment" Do you have any idea how to fix it? Thank you

Hi, your works are awesome! Can I get the discord link? Thank you very much

I need a tutorial on how to change the model ID using the Hugging Face or Civitai website

Such a great tool. Are there any resources on how to run completely local, no internet? I have everything running ok I think, max 16fps on 3090 with a single Step Schedule. Should be more, ya? Very much appreciate any leads on these.

Hey loving this tool thankyou for making. I'm having trouble loading a controlnet model. I get the following error \Users\markw\OneDrive\Documents\Touch\StreamDiffusion\streamdiffusionTD\pipeline_td.py", line 499, in predict_x0_batch control_image = torch.cat((control_image, self.control_image_buffer), dim=0) TypeError: expected Tensor as element 1 in argument 0, but got NoneType

你好，我点击start stream pulse之后，一直显示Loading pipeline components...: 20%|██████████▍ | 1/5 [00:00<00:02, 1.75it/s]和RuntimeError: Numpy is not available和UnboundLocalError: local variable 'pipe' referenced before assignment，请问怎么解决？

Hello, Thank you for making this tool <3 In terms of processing power and hardware, do you know what hardware we should buy to make this more efficient? Does a gaming GPU outperform a Quadro? (OpenCL vs OpenGL) Or would an AI card be better? Would having more CPU power and ram vs GPU help? We're getting about 15-20 fps on a 4090 and I'd love to buy something that would give us better fps, but we don't know WHAT we need to buy for better performance? Any insight would be greatly appreciated. :)

Dot it could be that the Guidance Scale parameter of Stream Setting 1, when it exceeds the value 1 in this version, it breaks?

我在TD里运行 view installlation guide ,出现黑屏。

a video on correct tensort implementation would be amazing!

Hi, I am getting this error. Any fix? ...Loading ControlNet model: controlnet_sd15_depth_f16.safetensors Failed to load ControlNet model E:/Stable Diffusion/stable-diffusion-webui/extensions/sd-webui-controlnet/models/controlnet_sd15_depth_f16.safetensors using from Huggingface (from_pretrained) due to: It looks like the config file at 'E:/Stable Diffusion/stable-diffusion-webui/extensions/sd-webui-controlnet/models/controlnet_sd15_depth_f16.safetensors' is not a valid JSON file. Failed to load ControlNet model from local directory (from_single_file) E:/Stable Diffusion/stable-diffusion-webui/extensions/sd-webui-controlnet/models/controlnet_sd15_depth_f16.safetensors due to: 'time_embed.0.weight' Thank you

how would you go about applying motion loras to this ?

Hello there! I tried installing/reinstalling/deleting venv folder/ installing different version of python/... but I keep getting the following error: Failed to load SD model from both local and Hugging Face due to: No module named 'pytorch_lightning' Any idea on how to resolve this? Thanks a lot!

hey i need help i can only use openai/turbo-sd model any other 1.5 or xl turbo xl etc. cant work? also non of my control net models are being accepted

Hello! Is there a way to see the result of the control net? as an output

heyy - could you please reshare the discord link? thanks

hey! I´m havin issues running it, i know CUDA is necessary but I have a nvidia 330 so i don´t know if the issues come from it

"Failed to build tensorrt ERROR: Could not build wheels for tensorrt, which is required to install pyproject.toml-based projects", seems like a common error, would you have any workaround?

Hey man, having issues with tensorrt...

Hello, Thank you for providing amazing TOX! I would like to know how to change the output resolution? I try to get into [StreamDiffusion TD component] and change the [ndiout] but still doesn't work.

Thank you!

Hello and thank you! I've installed everything successfully and get a stream info output, but my StreamDiffusionTD is blank. The only error I see is: StreamDiffusion\streamdiffusionTD\main_sdtd.py:393: RuntimeWarning: invalid value encountered in cast processed_np = (processed_np * 255).astype(np.uint8)

Hello Would you help me install LORA on your latest Tox from streamdiffusion? I get the following error " ERROR loading LoRA Models: Cannot load because down.weight expected shape tensor(..., device='meta', size=(32, 320)), but got torch.Size([32, 320, 1, 1]). If you want to instead overwrite randomly initialized weights, please make sure to pass both `low_cpu_mem_usage=False` and `ignore_mismatched_sizes=True`. For more information, see also: https://github.com/huggingface/diffusers/issues/1619#issuecomment-1345604389 as an example. "

thanks amazing work

I just downloaded the tox and followed the installation but when I try to start the stream the console opens and closes immediately. I see an error "no module named torchvision"

Any ideas on when are we getting NDI support back?

silly question - where is the link for discord? Am i blind?

I often touch 40fps but it doesn't seem to be using the gpu a 20 percent is using it parsec to remote the machine. when i install tensorcore it gives error but then if out of venv i install the correct version it seems to go. it gives me an error if i turn on cfg type it doesn't seem to work any controlnet so I wonder, did you put the page in without explaining which ones you can use because they don't work yet? I would need help but I didn't understand if I have to send a private message to Dotsimulate, if I have to go to discord, if I have to post every question publicly, I would really need feedback but I am not familiar with the rules. https://imgur.com/a/dotsimulate-streamdiffusion-krd-test-v1-1-4xw3Xvr

I thought when I subscribed to your Patreon I was going to be in and out just to download the operator, but I've stayed for the constant updates. You really work hard on this and I love the new features in each update!

Hi! I have this issue when i try to install tensorT: RuntimeError: ########################################################################################### The package you are trying to install is only a placeholder project on PyPI.org repository. This package is hosted on NVIDIA Python Package Index. This package can be installed as: ``` $ pip install nvidia-pyindex $ pip install nvidia-cudnn-cu116 ``` ###########################################################################################

Hi! First of all, thanks for your work!! I have a few questions. The first one: When I press the "Install (venv + all req)" button, I encounter a series of errors throughout the installation: Building wheel for n-vidia-pyindex (project.toml) ... error error: subprocess-exited-with-error Building wheel for n-vidia-pyindex (project.toml) did not run successfully. exit code: 1 /88 lines of output RuntimeError Traceback (most recent call last): File "" , line 1, in ModuleNotFoundError: No module named 'pip' ModuleNotFoundError: No module named 'setuptools' note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for nvidia-pyindex Failed to build nvidia-pyindex ERROR: Could not build wheels for nvidia-pyindex, which is required to install pyproject.toml-based projects Installing dependencies with pip from the activated virtual environment... Requirement already satisfied: pip in c:\users\flore\downloads\fran\x\streamdiffusion\venv\lib\site-packages (24.0) Looking in indexes: https://download.pytorch.org/whl/cu118 ERROR: Could not find a version that satisfies the requirement torch==2.1.0 (from versions: 2.2.0+cu118, 2.2.1+cu118, 2.2.2+cu118, 2.3.0+cu118) ERROR: No matching distribution found for torch==2.1.0 The second is that (despite the errors I get at the beginning) when I click on "Start Stream", a window opens with some text that I can't read because it closes immediately. In "Log Message" it says "started streaming using CMD" and in "stream Info" it says "not streaming". I hope you can help me. best regards!

Hi, I managed to load a Controlnet model: "Successfully loaded ControlNet model from local directory". But creating a model dropdown does not work - only a single file. When starting I get many error messages about specific files, under the title Process-3. And ultimately also a RuntimeError: "mat1 and mat2 shapes cannot by multiplied (154x2048 and 768x320)". I don't know exactly what to do with input 2 either. Can you show a working example? Kind regards, Pieter

This is fantastic! Where can I learn about installing Controlnet specifically for StreamDiffusionTD?

Ouuu, this is a big one here! Appreciate your hard work my dude 🙏🏽

Everything works fine, the fps count its around 0,5 and by pc specs are a Intel Core i5-10300H, have 8g of ram and have a rtx 1650, but i have a problem in wich the screen becomes black or rainbow color and the model does not load up, Does someone knows how to fix it?

I am using cuda 12.1 . I had a cuda version error which is fixed with dotsimulate's pull request on the streamdiffusion GitHub page. I also had a no module found error for cuda and onnxruntime , which can be fixed by activating the venv ( pulse in streamdiffusiontd didn't work for me ) and pip install cuda-python and onnxruntime. I don't think I had other errors on rttensors install. Thanks dotsimulate btw , nice work.

Has anyone else had any issues with building TensorRT? It keeps giving me errors and failing to build.