Node packVerifiedExternal

ComfyUI-HunyuanVideoWrapper

Sourcegithub.com/kijai/ComfyUI-HunyuanVideoWrapper

Maintainer@kijaiS · adoptiongrade

Package profile

Status: Verified
Publisher: @kijai
Source: GitHub

README

ComfyUI wrapper nodes for HunyuanVideo

Update 5

So I know I said I'd stop working on this, but with all the new stuff out I wanted to work on those and have included the official I2V, it's "fixed" version 2 and the LoRAs they included in the release

https://github.com/user-attachments/assets/8ce4b1ee-fb63-49a2-83b4-ba8ef1a8b842

and the dashtoon keyframe LoRA.

https://github.com/user-attachments/assets/2b6e32e4-470f-4feb-b299-5a453e2b4fa1

Also because there's been so much trouble in using the transformer model for text encoding, I figured a way to use the text embeds from native ComfyUI text encoding, like this:

image

Not that it does give somewhat different results and using these nodes like that can't be considered as original implementation wrapper anymore.

Update 4, the non-update:

As the native implementation exists, and has support for most features by now, I will mostly stop working on these nodes for anything but it's main purpose: early access and testing of potential new features that are difficult (at least for me) to implement natively.

Some resources for native workflows:

Flowedit and enhance-a-video can be found from these nodes: https://github.com/logtd/ComfyUI-HunyuanLoom

TeaCache equilevant FirstBlockCache, as well as torch.compile with LoRA support: https://github.com/chengzeyi/Comfy-WaveSpeed

Sageattention can be enabled by --use-sage-attention startup argument for ComfyUI, or with a patcher node found in KJNodes as well as some other node packs.

Leapfusion I2V can be used with my patcher node found in the KJNodes as well, example workflow: https://github.com/kijai/ComfyUI-KJNodes/blob/main/example_workflows/leapfusion_hunyuuanvideo_i2v_native_testing.json

What remains missing from native implementation currently:

context windowing
direct image embed support through IP2V
manual memory management

Update 3:

It's been hectic couple of weeks with this model, I've lost track of what has happened since the start, but I'll try to present some of the more important updates:

Official scaled fp8 weights were released:

https://huggingface.co/tencent/HunyuanVideo/blob/main/hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states_fp8.pt

Even if this file is .pt it's completely safe and it is loaded with weights_only, the scale map is included with the nodes. To use this model you have to use the fp8_scaled -quantization option in the model loader. The quality of these weights is much closer to the original bf16, downside is that they do not currently support fp8 fast mode, or LoRAs.

Almost free quality increase with Enhance-A-Video:

This has a very slight hit on inference speed and zero hit on memory use, initial tests indicate it's absolutely worth using.

image

https://github.com/user-attachments/assets/e19b30e1-5f67-4e75-9c73-716d4569c319

https://github.com/user-attachments/assets/083353a2-e9aa-43e9-a916-ff3af1d581c1

Update 2: Experimental IP2V - Image Prompting to Video via VLM by @Dango233

WORK IN PROGRESS - But it should work now!

Now you can feed image to the VLM as condition of generations! This is different from image2video where the image become the first frame of the video. IP2V uses image as a part of the prompt, to extract the concept and style of the image. So - very much like IPAdapter - but VLM will do the heavy lifting for you!

Now this is a tuning free approach but with further task specific tuning we can expand the use scenarios.