name: image-with-comfyui description: 'Call a local ComfyUI instance for text-to-image (T2I), image-to-image/edit (I2I), and image-to-video (I2V) generation. Supports Z-Image, SD3.5 Medium, Qwen Image Edit, and Wan2.2 models with automatic prompt formatting and VRAM purge after run.' metadata: openclaw: emoji: "🎨" version: "1.5.0" requires: anyBins: ["python3"] config: path: "config.json"

Image with ComfyUI

Call a local ComfyUI server to generate or edit images and videos. Four modes:

• T2I (Text → Image) → Z-Image or SD3.5 Medium model
• I2I (Image → Image / Edit) → Qwen Image Edit model
• I2V (Image → Video) → Wan2.2 model

When to Use

• User asks to generate images from text
• User asks to edit an image
• User asks to generate a video from an image + text
• User provides a description and wants visual output

Image-First Conversational Pattern (Image-First Mode)

Detection rules: 1. User sends only an image (no text, no other message in the same turn) 2. Within 2 minutes, the user sends a text message that looks like an edit or video request (Chinese keywords like: 修一下/換背景/加個特效/變成動畫/把顏色改藍的) 3. The text intent is I2I (edit the image) or I2V (animate the image)

Action: - Route the remembered image + the new text to image_with_comfyui.py i2i or wan2.2 accordingly - Use the latest image received as the --image input - Use the text as the --prompt - If unsure whether I2I or I2V, default to I2I (edit) unless the text clearly says video/animation - Do NOT ask the user for the image again — the agent already has the image from the previous turn

Context tracking: - Store the latest image media path (or URL) in a variable when no text is received - Clear the stored image after it's used (or after 2 minutes of no new text) - Only apply this to the immediately preceding message — don't look back further than 2 minutes

Examples: - User: [image: a photo of a dog] → Agent: (wait) - User: [text: change the background to a beach] → Agent: calls i2i --image <path> --prompt "change the background to a beach" - User: [image: a cat sitting on a chair] → Agent: (wait) - User: [text: make it stand up and walk] → Agent: calls wan2.2 --image <path> --prompt "the cat stands up and walks"

Configuration

Read config.json relative to this SKILL's directory.

The following environment variables are optional — they override config.json values when set, but the skill works perfectly fine without them (defaults are baked into the script):

Priority: Environment variables > config.json > script defaults

Workflow Files

Error Handling

The system automatically handles two types of errors:

1. Missing Node Detection

When a workflow references a custom node that isn't installed, the system detects it and reports: - Which node is missing (class type name) - Which package provides it - GitHub URL for manual download - Manual install instruction (the script does NOT execute git clone; ComfyUI may be on a remote server)

Example: If ImpactKSamplerBasicPipe is missing:

⚠️ Missing node: `ImpactKSamplerBasicPipe`
📦 Package: ComfyUI-Impact-Pack
🔗 GitHub: https://github.com/ltdrdata/ComfyUI-Impact-Pack
ℹ️ Install manually: cd ComfyUI/custom_nodes && git clone https://github.com/ltdrdata/ComfyUI-Impact-Pack

2. Missing Model Substitution

When a workflow references a model file that doesn't exist, the system attempts to find a compatible substitute:

Example: If my_custom_sd3_medium_v2.safetensors is missing:

⚠️ Model missing: `my_custom_sd3_medium_v2.safetensors`
🔄 Substituted: `sd3.5_large.safetensors`
📦 Loader: CheckpointLoaderSimple.ckpt_name

After substitution, the workflow is retried automatically with the substitute model.

3. Missing Utility Node Bypass (UnloadAllModels)

When the workflow references UnloadAllModels (a memory cleanup node) which isn't available, the system automatically bypasses it by rerouting the signal path: - Removes the missing UnloadAllModels node - Redirects the upstream processing node directly to the downstream output node - Generation continues without interruption - User receives a warning about the bypass

Example:

⚠️ Workflow missing node: `UnloadAllModels` (memory cleanup, non-critical)
🔄 Auto-bypassed — generation continues

Core Rules

1. Send media attachment: After generating an image or video, you must send it via the message tool using media or filePath.
2. Send in original session: Always deliver generated media in the user's original request session/thread — do not send to a separate topic, group, or thread unless explicitly told otherwise.
3. Don't include paths: Unless the user explicitly asks, never send local file paths, ComfyUI URLs, or other address info.

Prompt Formatting

Z-Image (T2I)

Z-Image works best with structured natural language prompts, not keyword spam.

6-part formula:

Subject + Scene + Composition + Lighting + Style + Constraints

Rules: - ✅ Use natural language sentences (not comma-separated tags) - ✅ Be specific about subject, camera, lighting, style - ❌ NO negative prompts — Z-Image Turbo ignores them completely - ❌ No weighted tags like (word:1.2)

Example:

A young woman with long wavy blonde hair sits at a wooden café table,
steam rising from a ceramic cup. Shot from a 3/4 angle, close-up framing.
Soft morning light filters through sheer curtains, casting warm golden tones.
Cinematic photography, shallow depth of field, Kodak Portra 400 aesthetic.
No text, no logos, photorealistic skin texture.

Aspect ratios: 1:1, 4:3, 3:4, 16:9, 9:16, 3:2, 2:3

SD3.5 Medium (T2I)

SD3.5 Medium uses natural language prompts with optional negative prompts.

Prompt formula:

[Composition/Angle] + [Subject] + [Scene/Environment] + [Lighting/Color] + [Style/Texture] + [Details]

Rules: - ✅ Complete natural language sentences — describe like telling a human what to see - ✅ Subject first (model prioritizes early text) - ✅ Be specific about colors, materials, mood, atmosphere - ✅ Mixed CN/EN is fine (Chinese works better for Chinese scenes) - ✅ Use --negative for elements to exclude - ✅ Default 1:1 (1024×1024), use --aspect to change - ✅ Default 20 steps, CFG 4.01 (higher = stronger control) - ✅ Seed defaults to random; specify --seed for reproducibility - ❌ No comma-separated keyword spam (beautiful, amazing, 4k) - ❌ No weighted tags (word:1.2) — SD3.5 doesn't recognize them

Parameter recommendations: - CFG: 4-7 (4.01 = softer, 5-7 = stronger control) - Steps: 20-25 (below 20 may lack detail) - Negative prompt: Highly effective in SD3.5

Common negative prompt words:

blurry, low quality, pixelated, grainy,
overexposed, underexposed, flat lighting,
text, watermark, logo, signature, caption,
poorly drawn face, deformed, mutated, disfigured, extra limbs,
cartoonish (when realism is wanted)
本技能來自小蔥技能站7w4.net。

Chinese example:

上海魔都春日花海 — 黃浦江畔，大片鬱金香、櫻花、油菜花盛開，繁花似錦，
春日和煦陽光，遠景陸家嘴三件套天際線，溼潤的濱江步道倒映花影，
低飽和膠片色調，文藝清新，廣角視野

English example:

Cinematic photography, wide-angle shot of a bustling Tokyo street at night,
neon signs reflecting on wet pavement, people with transparent umbrellas,
moody atmospheric lighting, deep blues and vibrant reds, street photography,
shallow depth of field with bokeh background

Qwen Image Edit (I2I) — Concise Prompts

I2I prompts must be concise and direct. Keep the user's original language.

Rules: - ✅ Positive prompt only — no negative prompts - ✅ Use user's exact words (don't translate or expand) - ✅ Concise: "換件紅色外套", "把背景換成藍天白雲", "將女孩換成男孩" - ❌ Don't translate between languages - ❌ Don't over-explain or add details

Prompt routing fix (2026-04-22): - The Qwen workflow has TWO TextEncodeQwenImageEditPlus nodes: - 115:110 — empty negative prompt node - 115:111 — positive prompt node (contains default text like "the girl") - Script must route prompts to node 111 (positive), not node 110 - The prepare_i2i_workflow() function auto-detects by scanning for existing default text

Wan2.2 I2V (Image → Video)

Wan2.2 generates short videos (~5 seconds) from a static image + motion description.

Rules: - ✅ Prompt describes actions/movement (not scene description) - ✅ Write motion description in English for best results - ✅ Focus on "who does what" and "how the camera moves" - ❌ Don't describe static scene elements in motion prompt

• Default: 81 frames (~5s @ 16fps), 4 steps, CFG 4.5
• Base resolution: 560×720 (3:4, fast and OK quality)
• Auto-detect input image aspect ratio and select reference resolution:

Resolution Reference

Other available resolutions: - 3:4: 416×544, 672×864, 784×1008 - 2:3: 384×576, 624×912, 736×1072 - 9:16: 368×624, 576×1008, 672×1184

Examples:

prompt: "the cat walks forward and looks at the camera, tail wagging"
prompt: "the girl smiles and turns her head, wind blowing her hair"
prompt: "the person stands in a busy street, camera pans left and slowly zooms in, cars driving, red flag fluttering"

CLI Usage

T2I (Text → Image)

# Z-Image (default model)
python3 image_with_comfyui.py t2i \
  --prompt "Your detailed image description" \
  --aspect 16:9 \
  --steps 9

# SD3.5 Medium
python3 image_with_comfyui.py sd35 \
  --prompt "A beautiful sunset over mountains" \
  --aspect 16:9 \
  --negative "text, watermark, blurry" \
  --steps 20 \
  --cfg 5.5

I2I (Edit Image)

python3 image_with_comfyui.py i2i \
  --prompt "Change background to a beach" \
  --image /path/to/source.jpg \
  --steps 4

I2V (Image → Video)

python3 image_with_comfyui.py wan2.2 \
  --prompt "the person walks forward and smiles" \
  --image /path/to/source.jpg \
  --length 81 --steps 4

Health Check

python3 image_with_comfyui.py test

Output Delivery

Send the media attachment directly. Be minimal.

🚨 Delivery Rules

⚠️ Absolutely forbidden: Only writing text descriptions without actually sending files!

✅ Correct approach: Send MEDIA path with the appropriate prefix for the current Channel

| Channel | Format | Example | |---------|--------|---------|| | WhatsApp | MEDIA:./image.jpg | MEDIA:./angel_video.mp4 | | Telegram | MEDIA: or filePath: | Varies by implementation | | Discord | Direct attachment | Varies by implementation |

Rules: 1. When sending images/videos/audio on WhatsApp, must add MEDIA: prefix + relative path 2. Place files in ./media/outbound/ directory to ensure accessibility 3. Don't just write text descriptions like "video attachment" — that does not equal sending a file 4. First cp the file to ~/.openclaw/media/outbound/, then send via MEDIA: 5. Other Channels use their corresponding formats (see respective Channel docs) 6. Default to sending to the requesting session: Generated images/videos must be sent back to the session that initiated the request, unless the user explicitly says "don't send" or "only save locally" 7. Pure MEDIA line for WhatsApp — no text prefix: The MEDIA: line must be the sole content of the message, with no [[reply_to_current]], text, or anything else before it. Otherwise WhatsApp splits the text and attachment into two separate messages, making it look like "sent twice". If caption text is needed, use MEDIA:./file.ext caption=description format.

Example delivery

WhatsApp:

MEDIA:./output_image.png

Universal：

[📎 Image attachment via MEDIA prefix]

Never replace actual file sending with text descriptions!

Timeout Reference

Additional Notes

• ComfyUI unreachable → report error with URL tried
• Generation fails (empty output) → report prompt_id for debugging
• Missing required node → report which node was not found
• Timeout → report elapsed time and prompt_id

這個工具能幫你用文字生成圖片、編輯照片、甚至把圖片變成短影片，效果不錯，操作也比較智慧。但它本質上是呼叫遠端的ComfyUI服務，你得先自己裝好ComfyUI和各種模型節點，配置起來比較麻煩。圖片會發送到伺服器處理，可能有隱私方面的考慮。生成靜態圖片質量較好，影片效果就一般般了。適合有一定技術基礎、願意折騰環境的使用者。