工作流 /
视频 /
AI Audio Maker Workflow
AI Audio Maker Workflow
AI Audio Maker Workflow
Have you eever wanted to add ai generated audio to your wan2.2 generations make them as immersive as LTX2.3 well here is the workflow.Download:https://github.com/kijai/ComfyUI-MMAudiodownload the models put them in the f…
视频
7 个节点 206 次使用 The_frizzy1 16 KB
需要的权重
这个工作流引用了 1 个权重文件，其中 0 个能在本站模型库里找到。
The_frizzy1_ai-audio-maker_v1.0.1.json
模型库暂未收录
节点构成
MarkdownNote
× 2
GetVideoComponents
× 1
CreateVideo
× 1
SaveVideo
× 1
LoadVideo
× 1
9c211236-0538-4e8d-a969-ea42af6f0327
× 1
官方标注的模型
apple_DFN5B-CLIP-ViT-H-14-384_fp16.safetensorsmmaudio_large_44k_v2_fp16.safetensorsmmaudio_synchformer_fp16.safetensorsmmaudio_vae_44k_fp16.safetensors
工作流 JSON
和上面「下载工作流 JSON」是同一份文件。
1{
2 "id": "d8ce3e4a-8b0e-48f8-849a-4bb63237923f",
3 "revision": 0,
4 "last_node_id": 506,
5 "last_link_id": 1078,
6 "nodes": [
7 {
8 "id": 502,
9 "type": "GetVideoComponents",
10 "pos": [
11 343.238622652859,
12 45.89585500348741
13 ],
14 "size": [
15 193.530859375,
16 66
17 ],
18 "flags": [],
19 "order": 1,
20 "mode": 0,
21 "inputs": [
22 {
23 "name": "video",
24 "type": "VIDEO",
25 "link": 1073
26 }
27 ],
28 "outputs": [
29 {
30 "name": "images",
31 "type": "IMAGE",
32 "links": [
33 1074,
34 1075
35 ]
36 },
37 {
38 "name": "audio",
39 "type": "AUDIO",
40 "links": null
41 },
42 {
43 "name": "fps",
44 "type": "FLOAT",
45 "links": [
46 1076
47 ]
48 }
49 ],
50 "properties": {
51 "Node name for S&R": "GetVideoComponents"
52 },
53 "widgets_values": []
54 },
55 {
56 "id": 503,
57 "type": "CreateVideo",
58 "pos": [
59 1307.8348150734957,
60 -142.57931805195176
61 ],
62 "size": [
63 270,
64 78
65 ],
66 "flags": [],
67 "order": 3,
68 "mode": 0,
69 "inputs": [
70 {
71 "name": "images",
72 "type": "IMAGE",
73 "link": 1075
74 },
75 {
76 "name": "audio",
77 "shape": 7,
78 "type": "AUDIO",
79 "link": 1077
80 },
81 {
82 "name": "fps",
83 "type": "FLOAT",
84 "widget": {
85 "name": "fps"
86 },
87 "link": 1076
88 }
89 ],
90 "outputs": [
91 {
92 "name": "VIDEO",
93 "type": "VIDEO",
94 "links": [
95 1078
96 ]
97 }
98 ],
99 "properties": {
100 "Node name for S&R": "CreateVideo"
101 },
102 "widgets_values": [
103 30
104 ]
105 },
106 {
107 "id": 504,
108 "type": "SaveVideo",
109 "pos": [
110 1674.5492121641328,
111 -166.9110789963542
112 ],
113 "size": [
114 270,
115 251
116 ],
117 "flags": [],
118 "order": 4,
119 "mode": 0,
120 "inputs": [
121 {
122 "name": "video",
123 "type": "VIDEO",
124 "link": 1078
125 }
126 ],
127 "outputs": [],
128 "properties": [],
129 "widgets_values": [
130 "video/ComfyUI",
131 "auto",
132 "auto"
133 ]
134 },
135 {
136 "id": 501,
137 "type": "LoadVideo",
138 "pos": [
139 -5.402580103149309,
140 52.424716478319056
141 ],
142 "size": [
143 282.798828125,
144 234.25266927083334
145 ],
146 "flags": [],
147 "order": 0,
148 "mode": 0,
149 "inputs": [],
150 "outputs": [
151 {
152 "name": "VIDEO",
153 "type": "VIDEO",
154 "links": [
155 1073
156 ]
157 }
158 ],
159 "properties": {
160 "Node name for S&R": "LoadVideo"
161 },
162 "widgets_values": [
163 "Wan2.2_I2V_00010_.mp4",
164 "image"
165 ]
166 },
167 {
168 "id": 500,
169 "type": "9c211236-0538-4e8d-a969-ea42af6f0327",
170 "pos": [
171 667.8834912515958,
172 -92.87322838967663
173 ],
174 "size": [
175 473.4242930755099,
176 456.9763632220595
177 ],
178 "flags": {
179 "collapsed": false
180 },
181 "order": 2,
182 "mode": 0,
183 "inputs": [
184 {
185 "name": "images",
186 "type": "IMAGE",
187 "link": 1074
188 }
189 ],
190 "outputs": [
191 {
192 "name": "audio",
193 "type": "AUDIO",
194 "links": [
195 1077
196 ]
197 }
198 ],
199 "properties": {
200 "proxyWidgets": [
201 [
202 "448",
203 "prompt"
204 ],
205 [
206 "448",
207 "negative_prompt"
208 ],
209 [
210 "448",
211 "duration"
212 ],
213 [
214 "451",
215 "mmaudio_model"
216 ],
217 [
218 "450",
219 "vae_model"
220 ],
221 [
222 "450",
223 "clip_model"
224 ]
225 ]
226 },
227 "widgets_values": []
228 },
229 {
230 "id": 505,
231 "type": "MarkdownNote",
232 "pos": [
233 -700,
234 -170
235 ],
236 "size": [
237 640,
238 900
239 ],
240 "flags": [],
241 "order": 5,
242 "mode": 0,
243 "inputs": [],
244 "outputs": [],
245 "title": "How to use",
246 "properties": [],
247 "widgets_values": [
248 "## How to use\n\n**What it does:** loads a video, MMAudio listens to the frames and writes a matching sound track (the Synchformer lines the sounds up with the motion), then saves the same video with the new audio. MMAudio 44 kHz.\n\n**Inputs**\n1. **Load Video**: your clip (for example a Wan or LTX render). Its frames and fps are reused, the picture is not changed.\n2. **Prompt**: open the **Audio (optional)** node (double-click) and type it on the **MMAudio Sampler**. Describe the sounds you want (wind, engine hum, footsteps), not the picture.\n\n**Settings (MMAudio Sampler)**\n| Setting | Default | What it does |\n|---|---|---|\n| duration | 8 | seconds of audio. Set it to your clip's length. |\n| steps | 25 | sampling steps |\n| cfg | 4.5 | how strongly the prompt is followed |\n| seed | randomize | set to fixed to repeat a result |\n| negative_prompt | empty | sounds to keep out |\n| mask_away_clip | off | on = the CLIP image features of the video are not used |\n| force_offload | on | moves the model off the GPU after the run |\n\nLoaders: model `mmaudio_large_44k_v2_fp16`, base precision fp16; feature utils in 44k mode, fp16.\n\n**Output:** Save Video writes the video with the new audio track to `ComfyUI/output/video/` (prefix `video/ComfyUI`).\n\n**Tips**\n- The images input of the sampler is optional: without a video MMAudio makes audio from the prompt alone (save it with a Save Audio node instead of Create Video).\n- MMAudio is light on VRAM.\n- Attention: --use-sage-attention used less VRAM at the same speed on the 24 GB bench server (measured on Z-Image, not on MMAudio). --use-ck-attention gave a different output for the same seed there.\n\n**Tested on:** no tested-on record yet. Checked on ComfyUI 0.45.3: the graph converts, but the ComfyUI-MMAudio pack did not import on that server (missing pip package `accelerate`), so it was not run there."
249 ],
250 "color": "#432",
251 "bgcolor": "#653"
252 },
253 {
254 "id": 506,
255 "type": "MarkdownNote",
256 "pos": [
257 -700,
258 770
259 ],
260 "size": [
261 640,
262 420
263 ],
264 "flags": [],
265 "order": 6,
266 "mode": 0,
267 "inputs": [],
268 "outputs": [],
269 "title": "Downloads",
270 "properties": [],
271 "widgets_values": [
272 "## Downloads\n\nPut all four files in `ComfyUI/models/mmaudio/`. Source: [Kijai/MMAudio_safetensors](https://huggingface.co/Kijai/MMAudio_safetensors)\n\n| Role | File (click = download) |\n|---|---|\n| MMAudio model | [mmaudio_large_44k_v2_fp16.safetensors](https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_large_44k_v2_fp16.safetensors) |\n| Synchformer | [mmaudio_synchformer_fp16.safetensors](https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_synchformer_fp16.safetensors) |\n| MMAudio VAE | [mmaudio_vae_44k_fp16.safetensors](https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_vae_44k_fp16.safetensors) |\n| CLIP (feature) | [apple_DFN5B-CLIP-ViT-H-14-384_fp16.safetensors](https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/apple_DFN5B-CLIP-ViT-H-14-384_fp16.safetensors) |\n\n**Custom node packs:** [ComfyUI-MMAudio](https://github.com/kijai/ComfyUI-MMAudio) (its requirements.txt includes `accelerate`), [ComfyUI-VideoHelperSuite](https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite). Load / Save Video in this graph are core nodes.\n\nOne command from the repo root: `python scripts/frizzy.py doctor audio/ai-audio-maker`"
273 ],
274 "color": "#432",
275 "bgcolor": "#653"
276 }
277 ],
278 "links": [
279 [
280 1073,
281 501,
282 0,
283 502,
284 0,
285 "VIDEO"
286 ],
287 [
288 1074,
289 502,
290 0,
291 500,
292 0,
293 "IMAGE"
294 ],
295 [
296 1075,
297 502,
298 0,
299 503,
300 0,
301 "IMAGE"
302 ],
303 [
304 1076,
305 502,
306 2,
307 503,
308 2,
309 "FLOAT"
310 ],
311 [
312 1077,
313 500,
314 0,
315 503,
316 1,
317 "AUDIO"
318 ],
319 [
320 1078,
321 503,
322 0,
323 504,
324 0,
325 "VIDEO"
326 ]
327 ],
328 "groups": [],
329 "definitions": {
330 "subgraphs": [
331 {
332 "id": "9c211236-0538-4e8d-a969-ea42af6f0327",
333 "version": 1,
334 "state": {
335 "lastGroupId": 6,
336 "lastNodeId": 506,
337 "lastLinkId": 1078,
338 "lastRerouteId": 0
339 },
340 "revision": 0,
341 "config": [],
342 "name": "Audio (optional)",
343 "inputNode": {
344 "id": -10,
345 "bounding": [
346 467.1715358627323,
347 1531.6939776127624,
348 120,
349 60
350 ]
351 },
352 "outputNode": {
353 "id": -20,
354 "bounding": [
355 2739.828555514975,
356 1531.6939776127624,
357 120,
358 60
359 ]
360 },
361 "inputs": [
362 {
363 "id": "40239410-00a8-4912-9589-0a99ebb7fb20",
364 "name": "images",
365 "type": "IMAGE",
366 "linkIds": [
367 1063
368 ],
369 "localized_name": "images",
370 "shape": 7,
371 "pos": [
372 567.1715358627323,
373 1551.6939776127624
374 ]
375 }
376 ],
377 "outputs": [
378 {
379 "id": "394e2c5f-1b0d-41ad-bafe-e0024470c105",
380 "name": "audio",
381 "type": "AUDIO",
382 "linkIds": [
383 953
384 ],
385 "localized_name": "audio",
386 "pos": [
387 2759.828555514975,
388 1551.6939776127624
389 ]
390 }
391 ],
392 "widgets": [],
393 "nodes": [
394 {
395 "id": 448,
396 "type": "MMAudioSampler",
397 "pos": [
398 1937.3985555149752,
399 825.4869498256517
400 ],
401 "size": [
402 742.4299999999994,
403 628.3000000000002
404 ],
405 "flags": [],
406 "order": 2,
407 "mode": 0,
408 "inputs": [
409 {
410 "localized_name": "mmaudio_model",
411 "name": "mmaudio_model",
412 "type": "MMAUDIO_MODEL",
413 "link": 949
414 },
415 {
416 "localized_name": "feature_utils",
417 "name": "feature_utils",
418 "type": "MMAUDIO_FEATUREUTILS",
419 "link": 950
420 },
421 {
422 "localized_name": "images",
423 "name": "images",
424 "shape": 7,
425 "type": "IMAGE",
426 "link": 1063
427 }
428 ],
429 "outputs": [
430 {
431 "localized_name": "audio",
432 "name": "audio",
433 "type": "AUDIO",
434 "links": [
435 953
436 ]
437 }
438 ],
439 "properties": {
440 "Node name for S&R": "MMAudioSampler"
441 },
442 "widgets_values": [
443 8,
444 25,
445 4.5,
446 413475454084225,
447 "randomize",
448 "locked camera shot through airplane window glass, model seated motionless on aircraft wing at cruising altitude, thirty thousand feet, dense cloud layer far below drifting slowly, golden hour light shifting almost imperceptibly across model's cheekbone and coat shoulder, wing edge catching light as aircraft banks one or two degrees very slowly, condensation on window glass catches light, model completely still, expression neutral and distant, hair moves a single strand in invisible airstream, clouds below drift at speed, god rays through upper atmosphere, cold blue altitude colour temperature, fashion editorial cinematography, Helmut Newton, wax sculpture stillness on the figure, all motion is environment not subject, film grain, cinematic",
449 "",
450 false,
451 true
452 ]
453 },
454 {
455 "id": 451,
456 "type": "MMAudioModelLoader",
457 "pos": [
458 736.8736821289174,
459 1939.3942079926117
460 ],
461 "size": [
462 508.70000000000005,
463 86.40000000000009
464 ],
465 "flags": [],
466 "order": 0,
467 "mode": 0,
468 "inputs": [],
469 "outputs": [
470 {
471 "localized_name": "mmaudio_model",
472 "name": "mmaudio_model",
473 "type": "MMAUDIO_MODEL",
474 "links": [
475 949
476 ]
477 }
478 ],
479 "properties": {
480 "Node name for S&R": "MMAudioModelLoader"
481 },
482 "widgets_values": [
483 "mmaudio_large_44k_v2_fp16.safetensors",
484 "fp16"
485 ]
486 },
487 {
488 "id": 450,
489 "type": "MMAudioFeatureUtilsLoader",
490 "pos": [
491 647.1715358627323,
492 2082.1654682097906
493 ],
494 "size": [
495 605.6585259555786,
496 245.7355371900826
497 ],
498 "flags": {
499 "collapsed": false
500 },
501 "order": 1,
502 "mode": 0,
503 "inputs": [
504 {
505 "localized_name": "bigvgan_vocoder_model",
506 "name": "bigvgan_vocoder_model",
507 "shape": 7,
508 "type": "VOCODER_MODEL",
509 "link": null
510 }
511 ],
512 "outputs": [
513 {
514 "localized_name": "mmaudio_featureutils",
515 "name": "mmaudio_featureutils",
516 "type": "MMAUDIO_FEATUREUTILS",
517 "links": [
518 950
519 ]
520 }
521 ],
522 "properties": {
523 "Node name for S&R": "MMAudioFeatureUtilsLoader"
524 },
525 "widgets_values": [
526 "mmaudio_vae_44k_fp16.safetensors",
527 "mmaudio_synchformer_fp16.safetensors",
528 "apple_DFN5B-CLIP-ViT-H-14-384_fp16.safetensors",
529 "44k",
530 "fp16"
531 ]
532 }
533 ],
534 "groups": [],
535 "links": [
536 {
537 "id": 949,
538 "origin_id": 451,
539 "origin_slot": 0,
540 "target_id": 448,
541 "target_slot": 0,
542 "type": "MMAUDIO_MODEL"
543 },
544 {
545 "id": 950,
546 "origin_id": 450,
547 "origin_slot": 0,
548 "target_id": 448,
549 "target_slot": 1,
550 "type": "MMAUDIO_FEATUREUTILS"
551 },
552 {
553 "id": 1063,
554 "origin_id": -10,
555 "origin_slot": 0,
556 "target_id": 448,
557 "target_slot": 2,
558 "type": "IMAGE"
559 },
560 {
561 "id": 953,
562 "origin_id": 448,
563 "origin_slot": 0,
564 "target_id": -20,
565 "target_slot": 0,
566 "type": "AUDIO"
567 }
568 ],
569 "extra": []
570 }
571 ]
572 },
573 "config": [],
574 "extra": {
575 "ds": {
576 "scale": 0.5753796460515007,
577 "offset": [
578 368.9190128199193,
579 796.7518399607604
580 ]
581 },
582 "frontendVersion": "1.41.21",
583 "VHS_latentpreview": false,
584 "VHS_latentpreviewrate": 0,
585 "VHS_MetadataImage": true,
586 "VHS_KeepIntermediate": true
587 },
588 "version": 0.4
589}