feat(gamedev): ronda 2b — 5 builders de workflow 2D (pixelart/seamless/iso/sprite/vfx)
Cinco builders puros que devuelven dict API format, cada uno componiendo funciones existentes del registry (comfyui_build_txt2img_workflow, comfyui_inject_*, comfyui_build_ipadapter_workflow). class_types verificados contra /object_info. Probados e2e en GPU (8GB lowvram): pixelart (pixel-perfect), seamless (sin costura), vfx (AnimateDiff loop -> luma-alpha -> spritesheet RGBA). 30 tests offline verdes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -14,7 +14,7 @@ Indice de grupos de capacidades del registry. Cada grupo agrupa >=3 funciones qu
|
||||
|
||||
| Grupo | N | Que cubre |
|
||||
|---|---|---|
|
||||
| [gamedev](gamedev-2d.md) | 5 | Assets 2D para Godot: post-proceso (pixelize, luma->alpha) + puente de assets a proyectos Godot 4 (carpetas + .import + reimport headless) |
|
||||
| [gamedev](gamedev-2d.md) | 10 | Assets 2D para Godot: builders de workflow ComfyUI (pixelart/seamless/iso/sprite/VFX, tag `gamedev-2d`) + post-proceso (pixelize, luma->alpha) + puente de assets a Godot 4 (.import + reimport headless) |
|
||||
| [registry](registry.md) | 17 | Auditoria y monitorizacion del propio registry: copied-code, uses-functions, unused, proposals, telemetria |
|
||||
| [systemd](systemd.md) | 14 | Generar, instalar, restart y status de unit files systemd via SSH (deploys a VPS) |
|
||||
| [ssh](ssh.md) | 19 | Operar hosts remotos via SSH: config, conn, ejecutar comandos, port-forward, deploys con SCP/rsync |
|
||||
|
||||
@@ -1,19 +1,41 @@
|
||||
# Capability group: `gamedev` — assets 2D para Godot (post-proceso + puente)
|
||||
# Capability group: `gamedev` / `gamedev-2d` — assets 2D para Godot (generación + post-proceso + puente)
|
||||
|
||||
Cluster de funciones para producir y mover assets 2D de juego entre **ComfyUI**
|
||||
(generación) y **Godot 4** (consumo). Cubre el **post-proceso determinista** de los
|
||||
crudos generados (pixelizar, recortar a alpha) y el **puente de assets** que los
|
||||
coloca en un proyecto Godot con sus import settings correctos. Todas son CPU-only:
|
||||
ninguna toca la GPU ni descarga modelos.
|
||||
(generación) y **Godot 4** (consumo). Tres capas:
|
||||
|
||||
Tag plano del grupo: `gamedev`. Filtro: `mcp__registry__fn_search query="" tag="gamedev"`.
|
||||
1. **Builders de workflow 2D** (`gamedev-2d`, GPU): construyen el dict (API format)
|
||||
de los workflows ComfyUI para pixel-art, tiles seamless, isométrico, sprites de
|
||||
personaje y VFX en bucle. Son **puros** (no tocan GPU al construir); el coste GPU
|
||||
está al enviar con `comfyui_submit_workflow`.
|
||||
2. **Post-proceso determinista** (`gamedev`, CPU): pixelizar, recortar a alpha.
|
||||
3. **Puente de assets** (`gamedev`, CPU): coloca el resultado en un proyecto Godot
|
||||
con sus import settings.
|
||||
|
||||
Documento hermano del grupo `comfyui` (generación de imágenes/video/3D): este grupo
|
||||
empieza donde el crudo ya existe en `~/ComfyUI/output/`. Diseño del puente:
|
||||
`docs/comfyui-godot-integration.md`. Planes origen: `reports/0135` (pixelart),
|
||||
`reports/0140` (VFX), `reports/0137`/`0138` (puente Godot).
|
||||
Tags: `gamedev` (post-proceso + puente) y `gamedev-2d` (builders de workflow).
|
||||
Filtro: `mcp__registry__fn_search query="" tag="gamedev-2d"`.
|
||||
|
||||
## Funciones del grupo
|
||||
Documento hermano del grupo `comfyui` (generación genérica de imágenes/video/3D).
|
||||
Diseño del puente: `docs/comfyui-godot-integration.md`. Planes origen: `reports/0135`
|
||||
(pixelart), `reports/0139` (entornos/tiles/iso), `reports/0137` (personajes/sprites),
|
||||
`reports/0140` (VFX), `reports/0143` (ronda 2b: builders).
|
||||
|
||||
## Builders de workflow 2D (`gamedev-2d`, puros — generación)
|
||||
|
||||
Construyen el dict API format listo para `comfyui_submit_workflow`. Cada uno compone
|
||||
funciones existentes del registry (`comfyui_build_txt2img_workflow`, `comfyui_inject_*`,
|
||||
`comfyui_build_ipadapter_workflow`) — no reinventan el grafo. class_types verificados
|
||||
contra `/object_info` del server (8GB lowvram). Probados e2e en GPU: pixelart, seamless,
|
||||
VFX (ver `reports/0143`).
|
||||
|
||||
| ID | Firma corta | Qué hace |
|
||||
|---|---|---|
|
||||
| `comfyui_build_pixelart_workflow_py_ml` | `(positive, negative=…, *, ckpt_name="juggernaut_xl_v11…", pixel_lora="pixel-art-xl…", use_lcm=True, …) -> dict` | Fase 1 pixel-art: SDXL + LoRA pixel-art-xl (+ LCM 8 steps). El pixel-perfect es post (`comfyui_pixelize_image`). |
|
||||
| `comfyui_build_seamless_tile_workflow_py_ml` | `(positive, negative="", *, tiling="enable", copy_model="Make a copy", circular_vae=True, material_lora=None, …) -> dict` | Textura tileable: `SeamlessTile` (Conv2d circular) + `CircularVAEDecode`. Coste VRAM ≈0. |
|
||||
| `comfyui_build_isometric_workflow_py_ml` | `(positive, negative=…, *, iso_lora="isometric_game_assets_sd15…", grid_image=None, …) -> dict` | Asset iso 2:1: LoRA iso + ControlNet grid opcional. |
|
||||
| `comfyui_build_sprite_sheet_workflow_py_ml` | `(subject, *, ref_image=None, pose_skeleton=None, char_lora=None, transparent=True, …) -> dict` | UN sprite de personaje: IPAdapter-FaceID + LoRA + ControlNet OpenPose (Advanced, end<1) + Rembg. Varias poses → sheet. SD1.5. |
|
||||
| `comfyui_build_vfx_spritesheet_workflow_py_ml` | `(prompt, *, motion_model="mm_sd_v15_v2.ckpt", num_frames=16, closed_loop=True, lora=None, …) -> dict` | N frames AnimateDiff loop sobre negro (insumo de luma→alpha). 8GB: 16f@512² revienta, usar ≤8f@512² o bajar resolución. |
|
||||
|
||||
## Funciones de post-proceso y puente (`gamedev`, CPU)
|
||||
|
||||
| ID | Firma corta | Qué hace |
|
||||
|---|---|---|
|
||||
@@ -23,7 +45,34 @@ empieza donde el crudo ya existe en `~/ComfyUI/output/`. Diseño del puente:
|
||||
| `godot_map_asset_dir_py_core` | `(kind) -> str` | Mapea `kind` -> subcarpeta de `res://assets/`. Pura. |
|
||||
| `godot_clean_asset_name_py_core` | `(filename, *, override=None) -> str` | Normaliza el nombre `<prefijo>_NNNNN_.<ext>` a snake_case seguro para `res://`. Pura. |
|
||||
|
||||
## Ejemplo canónico end-to-end
|
||||
## Ejemplo end-to-end con builder (Fase 1 GPU → Fase 2 CPU → Godot)
|
||||
|
||||
Flujo completo pixel-art: construir workflow → generar en ComfyUI → pixel-perfect → Godot.
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from ml.comfyui_build_pixelart_workflow import comfyui_build_pixelart_workflow
|
||||
from ml.comfyui_submit_workflow import comfyui_submit_workflow
|
||||
from ml.comfyui_wait_result import comfyui_wait_result
|
||||
from ml.comfyui_fetch_output_image import comfyui_fetch_output_image
|
||||
from ml.comfyui_pixelize_image import comfyui_pixelize_image
|
||||
|
||||
# 1. Construir (puro) + 2. generar (GPU)
|
||||
wf = comfyui_build_pixelart_workflow("isometric tiny house, pixel, 32x32 style", use_lcm=True, seed=42)
|
||||
pid = comfyui_submit_workflow(wf)["prompt_id"]
|
||||
outs = comfyui_wait_result(pid, timeout=300)
|
||||
fn = next(img["filename"] for o in outs.values() for img in o.get("images", []))
|
||||
raw = comfyui_fetch_output_image(fn, dest_dir="/tmp")["out_path"]
|
||||
# 3. pixel-perfect (CPU) -> 4. export Godot (ver ejemplo de abajo)
|
||||
px = comfyui_pixelize_image(raw, "/tmp/house_pixel.png", downscale=8, colors=16)
|
||||
```
|
||||
|
||||
VFX: `comfyui_build_vfx_spritesheet_workflow(prompt, num_frames=8)` → submit → fetch N frames
|
||||
→ `comfyui_matting_luma_to_alpha` por frame → montar sheet RGBA con `Image.alpha_composite`
|
||||
(NO `comfyui_build_grid`, que aplana el alpha).
|
||||
|
||||
## Ejemplo canónico de post-proceso
|
||||
|
||||
Flujo: crudo generado en ComfyUI -> pixelizar -> exportar a Godot con Nearest.
|
||||
|
||||
@@ -52,11 +101,16 @@ comfyui_export_asset_to_godot(rgba["out_path"], "vfx", PROJ)
|
||||
|
||||
## Fronteras (qué NO cubre)
|
||||
|
||||
- **Generación**: este grupo no genera imágenes. La Fase 1 (SDXL + LoRA
|
||||
`pixel-art-xl`, AnimateDiff loop, etc.) vive en el grupo `comfyui` y necesita GPU.
|
||||
- **Montaje de spritesheet** (grid RGBA + JSON sidecar) y **builders de workflow**
|
||||
(pixelart/VFX-loop): pendientes de la ronda siguiente (planes `reports/0135` F3/F4
|
||||
y `reports/0140` F2/F3). Cuando se añadan, van a este mismo grupo.
|
||||
- **Montaje de spritesheet dedicado** (grid RGBA + JSON sidecar para Godot/Unity):
|
||||
no hay función propia todavía — el ejemplo VFX monta con `Image.alpha_composite`
|
||||
inline. `comfyui_build_grid` NO sirve (aplana el alpha sobre fondo oscuro). Pendiente
|
||||
de R4 (plan `reports/0140` F2).
|
||||
- **Pipelines one-shot** (build → submit → wait → fetch → post en una call) para
|
||||
pixelart/sprite/VFX: pendientes. Hoy se encadena a mano (ver ejemplos). Candidatos a
|
||||
promoción a pipeline (issue 0087) cuando el patrón se repita.
|
||||
- **Sprite turnaround multi-vista** (orquestar N poses con identidad fija + juez):
|
||||
el builder `comfyui_build_sprite_sheet_workflow` produce UN frame; la orquestación
|
||||
multi-pose es pipeline pendiente (plan `reports/0137` T2).
|
||||
- **Paletas lospec por red** (`load_lospec_palette`): no incluido. `pixelize` usa
|
||||
paletas fijas embebidas (game-boy/pico-8/nes) o lista de hex, sin HTTP.
|
||||
- **TileSet / SpriteFrames `.tres`**: Godot no los deriva solos; `export_asset_to_godot`
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
name: comfyui_build_isometric_workflow
|
||||
kind: function
|
||||
lang: py
|
||||
domain: ml
|
||||
version: "1.0.0"
|
||||
purity: pure
|
||||
signature: "def comfyui_build_isometric_workflow(positive: str, negative: str = \"perspective, vanishing point, blurry, low quality\", *, ckpt_name: str = \"dreamshaper_8.safetensors\", iso_lora: str = \"isometric_game_assets_sd15.safetensors\", lora_strength: float = 0.9, grid_image: str | None = None, controlnet_name: str = \"control_v11p_sd15_canny_fp16.safetensors\", controlnet_strength: float = 0.6, steps: int = 28, cfg: float = 6.0, width: int = 1024, height: int = 1024, seed: int = 0, sampler_name: str = \"dpmpp_2m\", scheduler: str = \"karras\", filename_prefix: str = \"isometric\") -> dict"
|
||||
description: "Construye el dict (API format) de un workflow ComfyUI isometrico: txt2img + LoRA isometrica (isometric_game_assets_sd15) que impone el angulo 2:1, con ControlNet grid opcional (plantilla de rejilla iso preprocesada) para reforzar el layout. Compone comfyui_build_txt2img_workflow + comfyui_inject_lora (+ comfyui_inject_controlnet si grid_image). Pura, sin red ni I/O. class_types verificados contra /object_info."
|
||||
tags: [comfyui, ml, gamedev, gamedev-2d, isometric, tile, workflow, stable-diffusion]
|
||||
uses_functions: [comfyui_build_txt2img_workflow_py_ml, comfyui_inject_lora_py_ml, comfyui_inject_controlnet_py_ml]
|
||||
uses_types: []
|
||||
returns: []
|
||||
returns_optional: false
|
||||
error_type: ""
|
||||
imports: []
|
||||
params:
|
||||
- name: positive
|
||||
desc: "Prompt del asset iso (ej. 'isometric medieval house, game asset, 2:1 projection'). No puede estar vacio."
|
||||
- name: negative
|
||||
desc: "Prompt negativo. Por defecto evita perspectiva de fuga que rompe el angulo iso."
|
||||
- name: ckpt_name
|
||||
desc: "Checkpoint base. Default 'dreamshaper_8.safetensors' (SD1.5 holgado, casa con la LoRA iso SD1.5). keyword-only."
|
||||
- name: iso_lora
|
||||
desc: "LoRA isometrica en models/loras. Default 'isometric_game_assets_sd15.safetensors'. keyword-only."
|
||||
- name: lora_strength
|
||||
desc: "Fuerza de la LoRA iso sobre model y clip (recomendado 0.9). keyword-only."
|
||||
- name: grid_image
|
||||
desc: "Nombre de una plantilla de rejilla iso ya preprocesada (line-art/canny) en input/ del servidor. Si se pasa, inyecta un ControlNet que fuerza el grid 2:1. None = solo LoRA. keyword-only."
|
||||
- name: controlnet_name
|
||||
desc: "Modelo ControlNet para el grid (default canny SD1.5; debe casar con la familia del checkpoint). keyword-only."
|
||||
- name: controlnet_strength
|
||||
desc: "Fuerza del ControlNet del grid (0..1). keyword-only."
|
||||
- name: steps
|
||||
desc: "Pasos del KSampler. keyword-only."
|
||||
- name: cfg
|
||||
desc: "CFG del KSampler. keyword-only."
|
||||
- name: width
|
||||
desc: "Ancho en px. keyword-only."
|
||||
- name: height
|
||||
desc: "Alto en px. keyword-only."
|
||||
- name: seed
|
||||
desc: "Semilla del KSampler. keyword-only."
|
||||
- name: sampler_name
|
||||
desc: "Sampler del KSampler. keyword-only."
|
||||
- name: scheduler
|
||||
desc: "Scheduler del KSampler. keyword-only."
|
||||
- name: filename_prefix
|
||||
desc: "Prefijo del PNG en output/. keyword-only."
|
||||
output: "dict en API format listo para comfyui_submit_workflow: txt2img base + LoraLoader (iso) y, si grid_image, una rama ControlNet (LoadImage + ControlNetLoader + ControlNetApply) que condiciona el positivo."
|
||||
tested: true
|
||||
tests: ["golden: 1 LoraLoader iso@0.9, sin ControlNet, KSampler dpmpp_2m/karras", "edge grid_image: ControlNetApply + ControlNetLoader (canny) + LoadImage con la plantilla", "error positive vacio -> ValueError", "determinismo"]
|
||||
test_file_path: "python/functions/ml/comfyui_build_isometric_workflow_test.py"
|
||||
file_path: "python/functions/ml/comfyui_build_isometric_workflow.py"
|
||||
---
|
||||
|
||||
## Ejemplo
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from ml.comfyui_build_isometric_workflow import comfyui_build_isometric_workflow
|
||||
|
||||
# Solo LoRA iso (via principal):
|
||||
wf = comfyui_build_isometric_workflow(
|
||||
"isometric medieval blacksmith building, game asset, vibrant",
|
||||
seed=11,
|
||||
)
|
||||
# Con rejilla guia (refuerza el angulo 2:1):
|
||||
wf_grid = comfyui_build_isometric_workflow(
|
||||
"isometric grass tile, game asset",
|
||||
grid_image="iso_grid_template.png", # plantilla canny/line-art en input/
|
||||
seed=11,
|
||||
)
|
||||
# -> comfyui_submit_workflow + comfyui_wait_result + comfyui_fetch_output_image
|
||||
```
|
||||
|
||||
O lanzable directo con: `./fn run comfyui_build_isometric_workflow` (imprime los node_ids de ambos ejemplos).
|
||||
|
||||
## Cuando usarla
|
||||
|
||||
Cuando necesites assets con proyección isométrica 2:1 (edificios, tiles, props
|
||||
para un juego iso/top-down). La LoRA sola suele bastar; añade `grid_image` cuando
|
||||
el prompt se desvíe del ángulo y quieras forzar la rejilla. Para una textura iso
|
||||
*tileable*, combínala con `comfyui_build_seamless_tile_workflow`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Requiere la LoRA `isometric_game_assets_sd15.safetensors` en models/loras**
|
||||
(ya presente). Es SD1.5: usa un checkpoint SD1.5 (`dreamshaper_8`).
|
||||
- **`comfyui_inject_controlnet` NO preprocesa**: `grid_image` debe ser una imagen
|
||||
de control ya en line-art/canny limpio. Si partes de un render, preprocésala
|
||||
antes con `CannyEdgePreprocessor`/`LineArtPreprocessor` (ya instalados).
|
||||
- El ControlNet del grid usa el legacy `ControlNetApply` (solo condiciona el
|
||||
positivo) — suficiente para una rejilla guía.
|
||||
- Función pura: no valida contra el server. LoRA/ControlNet ausente -> HTTP 400 al
|
||||
enviar.
|
||||
@@ -0,0 +1,127 @@
|
||||
"""Construye un workflow ComfyUI isometrico (LoRA iso + ControlNet grid) en API format.
|
||||
|
||||
La proyeccion isometrica 2:1 no necesita node nuevo: es composicion pura (report
|
||||
0139). Dos piezas combinables:
|
||||
|
||||
1. LoRA isometrica (via principal): impone el angulo 2:1 y el look de tile. Se
|
||||
inyecta con comfyui_inject_lora sobre la base txt2img. LoRA instalada:
|
||||
isometric_game_assets_sd15.safetensors (SD1.5, holgada en 8GB).
|
||||
2. ControlNet grid (refuerzo opcional): una plantilla de rejilla isometrica
|
||||
(rombos 2:1) ya preprocesada (canny/lineart) fuerza el layout. Se inyecta con
|
||||
comfyui_inject_controlnet (legacy ControlNetApply; basta para un grid guia).
|
||||
|
||||
Compone comfyui_build_txt2img_workflow + comfyui_inject_lora (+ comfyui_inject_controlnet
|
||||
si grid_image). class_types verificados contra /object_info: CheckpointLoaderSimple,
|
||||
LoraLoader, ControlNetLoader (control_v11p_sd15_canny_fp16.safetensors presente),
|
||||
ControlNetApply, LoadImage.
|
||||
|
||||
Gotcha: comfyui_inject_controlnet asume el control_image YA preprocesado (no incluye
|
||||
el preprocessor). Para una plantilla de grid que ya es line-art limpio, pasarla
|
||||
directa es trivial; si se parte de un render, preprocesar antes
|
||||
(CannyEdgePreprocessor/LineArtPreprocessor, ya instalados).
|
||||
|
||||
Funcion pura: sin red, sin I/O. Determinista para los mismos argumentos.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
|
||||
def comfyui_build_isometric_workflow(
|
||||
positive: str,
|
||||
negative: str = "perspective, vanishing point, blurry, low quality",
|
||||
*,
|
||||
ckpt_name: str = "dreamshaper_8.safetensors",
|
||||
iso_lora: str = "isometric_game_assets_sd15.safetensors",
|
||||
lora_strength: float = 0.9,
|
||||
grid_image: str | None = None,
|
||||
controlnet_name: str = "control_v11p_sd15_canny_fp16.safetensors",
|
||||
controlnet_strength: float = 0.6,
|
||||
steps: int = 28,
|
||||
cfg: float = 6.0,
|
||||
width: int = 1024,
|
||||
height: int = 1024,
|
||||
seed: int = 0,
|
||||
sampler_name: str = "dpmpp_2m",
|
||||
scheduler: str = "karras",
|
||||
filename_prefix: str = "isometric",
|
||||
) -> dict:
|
||||
"""Construye el dict (API format) de un workflow isometrico.
|
||||
|
||||
Args:
|
||||
positive: prompt del asset isometrico (ej. "isometric medieval house,
|
||||
game asset, 2:1 projection"). No puede estar vacio.
|
||||
negative: prompt negativo (por defecto evita perspectiva de fuga que
|
||||
rompe el angulo iso).
|
||||
ckpt_name: checkpoint base (default 'dreamshaper_8.safetensors', SD1.5
|
||||
holgado y casa con la LoRA iso SD1.5). keyword-only.
|
||||
iso_lora: LoRA isometrica en models/loras
|
||||
(default 'isometric_game_assets_sd15.safetensors').
|
||||
lora_strength: fuerza de la LoRA iso sobre model y clip (recomendado 0.9).
|
||||
grid_image: nombre de una plantilla de rejilla isometrica ya preprocesada
|
||||
(line-art/canny) en input/ del servidor. Si se pasa, inyecta un
|
||||
ControlNet que fuerza el grid 2:1. None = solo LoRA. keyword-only.
|
||||
controlnet_name: modelo ControlNet para el grid
|
||||
(default canny SD1.5; debe casar con la familia del checkpoint).
|
||||
controlnet_strength: fuerza del ControlNet del grid (0..1).
|
||||
steps, cfg, width, height, seed, sampler_name, scheduler, filename_prefix:
|
||||
parametros de generacion pasados a comfyui_build_txt2img_workflow.
|
||||
|
||||
Returns:
|
||||
dict en API format listo para comfyui_submit_workflow: txt2img base +
|
||||
LoraLoader (iso) y, si grid_image, una rama ControlNet (LoadImage +
|
||||
ControlNetLoader + ControlNetApply) que condiciona el positivo.
|
||||
|
||||
Raises:
|
||||
ValueError: si positive esta vacio (o el inject propaga su error).
|
||||
"""
|
||||
from ml.comfyui_build_txt2img_workflow import comfyui_build_txt2img_workflow
|
||||
from ml.comfyui_inject_lora import comfyui_inject_lora
|
||||
|
||||
if not positive or not positive.strip():
|
||||
raise ValueError("comfyui_build_isometric_workflow: 'positive' no puede estar vacio")
|
||||
|
||||
base = comfyui_build_txt2img_workflow(
|
||||
ckpt_name,
|
||||
positive,
|
||||
negative,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
width=width,
|
||||
height=height,
|
||||
seed=seed,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
filename_prefix=filename_prefix,
|
||||
)
|
||||
|
||||
wf = comfyui_inject_lora(
|
||||
base, iso_lora, strength_model=lora_strength, strength_clip=lora_strength
|
||||
)
|
||||
|
||||
if grid_image:
|
||||
from ml.comfyui_inject_controlnet import comfyui_inject_controlnet
|
||||
|
||||
wf = comfyui_inject_controlnet(
|
||||
wf, grid_image, controlnet_name, strength=controlnet_strength
|
||||
)
|
||||
|
||||
return wf
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
|
||||
wf = comfyui_build_isometric_workflow(
|
||||
"isometric medieval blacksmith building, game asset, vibrant",
|
||||
seed=11,
|
||||
)
|
||||
wf_grid = comfyui_build_isometric_workflow(
|
||||
"isometric grass tile, game asset",
|
||||
grid_image="iso_grid_template.png",
|
||||
seed=11,
|
||||
)
|
||||
print(json.dumps({"plain_nodes": list(wf), "grid_nodes": list(wf_grid)}, indent=2))
|
||||
@@ -0,0 +1,54 @@
|
||||
"""Tests offline de comfyui_build_isometric_workflow (estructura del dict, sin GPU)."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from ml.comfyui_build_isometric_workflow import comfyui_build_isometric_workflow # noqa: E402
|
||||
|
||||
|
||||
def _by_class(wf, cls):
|
||||
return [n for n in wf.values() if n["class_type"] == cls]
|
||||
|
||||
|
||||
def test_golden_iso_lora_no_controlnet():
|
||||
wf = comfyui_build_isometric_workflow("isometric house, game asset")
|
||||
loras = _by_class(wf, "LoraLoader")
|
||||
assert len(loras) == 1
|
||||
assert loras[0]["inputs"]["lora_name"] == "isometric_game_assets_sd15.safetensors"
|
||||
assert loras[0]["inputs"]["strength_model"] == 0.9
|
||||
# Sin grid_image -> sin ControlNet.
|
||||
assert len(_by_class(wf, "ControlNetApply")) == 0
|
||||
assert len(_by_class(wf, "ControlNetLoader")) == 0
|
||||
ks = next(n for n in wf.values() if n["class_type"] == "KSampler")
|
||||
assert ks["inputs"]["sampler_name"] == "dpmpp_2m"
|
||||
assert ks["inputs"]["scheduler"] == "karras"
|
||||
|
||||
|
||||
def test_edge_with_grid_controlnet():
|
||||
wf = comfyui_build_isometric_workflow(
|
||||
"isometric grass tile", grid_image="iso_grid.png", controlnet_strength=0.7
|
||||
)
|
||||
assert len(_by_class(wf, "ControlNetApply")) == 1
|
||||
assert len(_by_class(wf, "ControlNetLoader")) == 1
|
||||
cn = _by_class(wf, "ControlNetLoader")[0]
|
||||
assert cn["inputs"]["control_net_name"] == "control_v11p_sd15_canny_fp16.safetensors"
|
||||
apply = _by_class(wf, "ControlNetApply")[0]
|
||||
assert apply["inputs"]["strength"] == 0.7
|
||||
# LoadImage del grid presente.
|
||||
loads = [n for n in wf.values() if n["class_type"] == "LoadImage"]
|
||||
assert any(n["inputs"]["image"] == "iso_grid.png" for n in loads)
|
||||
|
||||
|
||||
def test_error_empty_prompt():
|
||||
try:
|
||||
comfyui_build_isometric_workflow("")
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "positive" in str(e)
|
||||
|
||||
|
||||
def test_determinism():
|
||||
a = comfyui_build_isometric_workflow("iso tower", seed=2)
|
||||
b = comfyui_build_isometric_workflow("iso tower", seed=2)
|
||||
assert a == b
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
name: comfyui_build_pixelart_workflow
|
||||
kind: function
|
||||
lang: py
|
||||
domain: ml
|
||||
version: "1.0.0"
|
||||
purity: pure
|
||||
signature: "def comfyui_build_pixelart_workflow(positive: str, negative: str = \"blurry, jpeg artifacts, gradient, smooth shading, anti-aliasing\", *, ckpt_name: str = \"juggernaut_xl_v11.safetensors\", pixel_lora: str = \"pixel-art-xl.safetensors\", lora_strength: float = 1.2, use_lcm: bool = True, lcm_lora: str = \"lcm-lora-sdxl.safetensors\", lcm_strength: float = 1.0, steps: int | None = None, cfg: float | None = None, width: int = 1024, height: int = 1024, seed: int = 0, sampler_name: str | None = None, scheduler: str | None = None, filename_prefix: str = \"pixelart\") -> dict"
|
||||
description: "Construye el dict (API format) del workflow ComfyUI de pixel-art Fase 1: SDXL base + LoRA pixel-art-xl (nerijs), opcionalmente con LCM-LoRA para 8 steps. Compone comfyui_build_txt2img_workflow + comfyui_inject_multi_lora. El pixel-perfect (Fase 2) lo hace comfyui_pixelize_image, no este workflow. Pura, sin red ni I/O. class_types verificados contra /object_info (8GB lowvram)."
|
||||
tags: [comfyui, ml, gamedev, gamedev-2d, pixelart, workflow, stable-diffusion, sdxl]
|
||||
uses_functions: [comfyui_build_txt2img_workflow_py_ml, comfyui_inject_multi_lora_py_ml]
|
||||
uses_types: []
|
||||
returns: []
|
||||
returns_optional: false
|
||||
error_type: ""
|
||||
imports: []
|
||||
params:
|
||||
- name: positive
|
||||
desc: "Prompt positivo. Trucos para empujar el grid: incluir 'pixel', 'isometric view', '32x32 style'. No puede estar vacio."
|
||||
- name: negative
|
||||
desc: "Prompt negativo. Por defecto evita blur/gradientes/anti-alias que estropean el look pixel."
|
||||
- name: ckpt_name
|
||||
desc: "Checkpoint SDXL base. Default 'juggernaut_xl_v11.safetensors' (el SDXL instalado; pixel-art-xl es LoRA SDXL). keyword-only."
|
||||
- name: pixel_lora
|
||||
desc: "Archivo de la LoRA de estilo pixel-art en models/loras. Default 'pixel-art-xl.safetensors'. keyword-only."
|
||||
- name: lora_strength
|
||||
desc: "Fuerza de pixel-art-xl sobre model y clip (recomendado 1.2). Se clampa a [0.0, 2.0]. keyword-only."
|
||||
- name: use_lcm
|
||||
desc: "Si True encadena LCM-LoRA SDXL y usa defaults rapidos (8 steps, cfg 1.5, sampler 'lcm', scheduler 'sgm_uniform'); si False usa defaults SDXL normales (25 steps, cfg 7, 'euler'/'normal'). keyword-only."
|
||||
- name: lcm_lora
|
||||
desc: "Archivo de la LCM-LoRA SDXL en models/loras (solo si use_lcm). Default 'lcm-lora-sdxl.safetensors'. keyword-only."
|
||||
- name: lcm_strength
|
||||
desc: "Fuerza de la LCM-LoRA sobre model y clip (recomendado 1.0). keyword-only."
|
||||
- name: steps
|
||||
desc: "Pasos del KSampler. None = usar el default del modo (8 con LCM, 25 sin). keyword-only."
|
||||
- name: cfg
|
||||
desc: "CFG del KSampler. None = default del modo (1.5 con LCM, 7 sin). keyword-only."
|
||||
- name: width
|
||||
desc: "Ancho base en px (1024 SDXL; luego downscale x8 -> 128 en Fase 2). keyword-only."
|
||||
- name: height
|
||||
desc: "Alto base en px (1024 SDXL). keyword-only."
|
||||
- name: seed
|
||||
desc: "Semilla del KSampler. keyword-only."
|
||||
- name: sampler_name
|
||||
desc: "Sampler del KSampler. None = default del modo ('lcm' con LCM, 'euler' sin). keyword-only."
|
||||
- name: scheduler
|
||||
desc: "Scheduler del KSampler. None = default del modo ('sgm_uniform' con LCM, 'normal' sin). keyword-only."
|
||||
- name: filename_prefix
|
||||
desc: "Prefijo del PNG que SaveImage escribe en output/. keyword-only."
|
||||
output: "dict en API format listo para comfyui_submit_workflow: CheckpointLoaderSimple + 1 LoraLoader (pixel-art-xl) o 2 (+ lcm-lora-sdxl si use_lcm) + KSampler con params del modo + SaveImage."
|
||||
tested: true
|
||||
tests: ["golden use_lcm=True: 2 LoraLoader (pixel-art-xl@1.2, lcm@1.0) + KSampler steps 8/cfg 1.5/sampler lcm/sgm_uniform", "edge use_lcm=False: 1 LoraLoader + KSampler steps 25/cfg 7/euler/normal", "edge overrides steps/cfg + clamp lora_strength a 2.0", "error positive vacio -> ValueError", "determinismo"]
|
||||
test_file_path: "python/functions/ml/comfyui_build_pixelart_workflow_test.py"
|
||||
file_path: "python/functions/ml/comfyui_build_pixelart_workflow.py"
|
||||
---
|
||||
|
||||
## Ejemplo
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from ml.comfyui_build_pixelart_workflow import comfyui_build_pixelart_workflow
|
||||
|
||||
# Fase 1: generar el crudo pixel-art (SDXL + pixel-art-xl + LCM, 8 steps).
|
||||
wf = comfyui_build_pixelart_workflow(
|
||||
"isometric tiny house, pixel, 32x32 style, vibrant colors",
|
||||
use_lcm=True,
|
||||
seed=42,
|
||||
)
|
||||
# -> pasar a comfyui_submit_workflow + comfyui_wait_result + comfyui_fetch_output_image
|
||||
# Fase 2 (pixel-perfect): comfyui_pixelize_image(raw_png, out_png, downscale=8, colors=16)
|
||||
```
|
||||
|
||||
O lanzable directo con: `./fn run comfyui_build_pixelart_workflow` (imprime el JSON del workflow de ejemplo).
|
||||
|
||||
## Cuando usarla
|
||||
|
||||
Cuando quieras generar arte pixel-art para un juego 2D: tiles, sprites, props.
|
||||
Es la Fase 1 (genera el *look*). SIEMPRE encadénala con la Fase 2
|
||||
`comfyui_pixelize_image` (downscale nearest + cuantización) para obtener pixeles
|
||||
duros y paleta limitada — sin la Fase 2 el resultado es "pixelart borroso de IA".
|
||||
Para tilesets, genera cada tile por separado y ensambla con `comfyui_build_grid`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Es API format**, no el formato de la UI. Pásalo a `comfyui_submit_workflow`,
|
||||
no lo pegues en la UI.
|
||||
- **No produce pixel-perfect por sí solo**: deja pixeles irregulares y cientos de
|
||||
colores. El pixel-perfect es post-proceso (`comfyui_pixelize_image`, CPU/PIL).
|
||||
- `use_lcm=True` requiere `lcm-lora-sdxl.safetensors` en models/loras y el sampler
|
||||
`lcm`; ambos verificados presentes. Da iteración rápida (8 steps) en 8GB.
|
||||
- `ckpt_name` debe ser un checkpoint SDXL (pixel-art-xl es LoRA SDXL). Default
|
||||
`juggernaut_xl_v11` (no existe `sd_xl_base_1.0` instalado). SDXL en 8GB corre con
|
||||
`--lowvram`; la Fase 2 es CPU y no toca VRAM.
|
||||
- Función pura: no valida contra el server. Si una LoRA/checkpoint falta, el HTTP
|
||||
400 salta al enviar con `comfyui_submit_workflow`.
|
||||
@@ -0,0 +1,131 @@
|
||||
"""Construye un workflow ComfyUI de pixel-art (SDXL + LoRA pixel-art-xl) en API format.
|
||||
|
||||
Fase 1 del pipeline pixel-art (ver report 0135): genera el *look* pixel-art con
|
||||
SDXL base + la LoRA `pixel-art-xl` (nerijs), opcionalmente acelerada con la
|
||||
LCM-LoRA SDXL para iterar en 8 steps. El resultado todavia tiene pixeles de
|
||||
tamano irregular y cientos de colores: el pixel-perfect (Fase 2) lo hace
|
||||
`comfyui_pixelize_image` (downscale nearest + cuantizacion), NO este workflow.
|
||||
|
||||
Compone funciones existentes del registry, no reescribe el grafo:
|
||||
- comfyui_build_txt2img_workflow -> base SDXL txt2img
|
||||
- comfyui_inject_multi_lora -> encadena pixel-art-xl (+ lcm-lora-sdxl)
|
||||
|
||||
class_types/params verificados contra /object_info del servidor (8GB lowvram):
|
||||
CheckpointLoaderSimple, LoraLoader, CLIPTextEncode, EmptyLatentImage,
|
||||
KSampler (sampler 'lcm', scheduler 'sgm_uniform' presentes), VAEDecode, SaveImage.
|
||||
LoRAs presentes en models/loras: pixel-art-xl.safetensors, lcm-lora-sdxl.safetensors.
|
||||
|
||||
Funcion pura: sin red, sin I/O. Determinista para los mismos argumentos.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
# Defaults canonicos de la ficha HF nerijs/pixel-art-xl (report 0135).
|
||||
_LCM_DEFAULTS = {"steps": 8, "cfg": 1.5, "sampler_name": "lcm", "scheduler": "sgm_uniform"}
|
||||
_PLAIN_DEFAULTS = {"steps": 25, "cfg": 7.0, "sampler_name": "euler", "scheduler": "normal"}
|
||||
|
||||
|
||||
def comfyui_build_pixelart_workflow(
|
||||
positive: str,
|
||||
negative: str = "blurry, jpeg artifacts, gradient, smooth shading, anti-aliasing",
|
||||
*,
|
||||
ckpt_name: str = "juggernaut_xl_v11.safetensors",
|
||||
pixel_lora: str = "pixel-art-xl.safetensors",
|
||||
lora_strength: float = 1.2,
|
||||
use_lcm: bool = True,
|
||||
lcm_lora: str = "lcm-lora-sdxl.safetensors",
|
||||
lcm_strength: float = 1.0,
|
||||
steps: int | None = None,
|
||||
cfg: float | None = None,
|
||||
width: int = 1024,
|
||||
height: int = 1024,
|
||||
seed: int = 0,
|
||||
sampler_name: str | None = None,
|
||||
scheduler: str | None = None,
|
||||
filename_prefix: str = "pixelart",
|
||||
) -> dict:
|
||||
"""Construye el dict (API format) del workflow pixel-art SDXL + LoRA.
|
||||
|
||||
Args:
|
||||
positive: prompt positivo. Trucos para empujar el grid: incluir 'pixel',
|
||||
'isometric view', '32x32 style'. No puede estar vacio.
|
||||
negative: prompt negativo (por defecto evita blur/gradientes/anti-alias,
|
||||
que estropean el look pixel).
|
||||
ckpt_name: checkpoint SDXL base (default 'juggernaut_xl_v11.safetensors',
|
||||
el SDXL instalado; pixel-art-xl es una LoRA SDXL). keyword-only.
|
||||
pixel_lora: archivo de la LoRA de estilo pixel-art en models/loras.
|
||||
lora_strength: fuerza de pixel-art-xl sobre model y clip (recomendado 1.2).
|
||||
Se clampa a [0.0, 2.0].
|
||||
use_lcm: si True encadena la LCM-LoRA SDXL y usa los defaults rapidos
|
||||
(8 steps, cfg 1.5, sampler 'lcm', scheduler 'sgm_uniform'); si False
|
||||
usa los defaults SDXL normales (25 steps, cfg 7, 'euler'/'normal').
|
||||
lcm_lora: archivo de la LCM-LoRA SDXL en models/loras (solo si use_lcm).
|
||||
lcm_strength: fuerza de la LCM-LoRA sobre model y clip (recomendado 1.0).
|
||||
steps, cfg, sampler_name, scheduler: si se pasan, sobreescriben el default
|
||||
del modo (LCM vs normal). None = usar el default del modo.
|
||||
width, height: resolucion base (1024x1024 SDXL; luego downscale x8 -> 128
|
||||
en la Fase 2 con comfyui_pixelize_image).
|
||||
seed: semilla del KSampler.
|
||||
filename_prefix: prefijo del PNG en output/.
|
||||
|
||||
Returns:
|
||||
dict en API format listo para comfyui_submit_workflow, con el
|
||||
CheckpointLoaderSimple, 1 LoraLoader (pixel-art-xl) o 2 (pixel-art-xl +
|
||||
lcm-lora-sdxl si use_lcm), KSampler con los params del modo y SaveImage.
|
||||
|
||||
Raises:
|
||||
ValueError: si positive esta vacio.
|
||||
"""
|
||||
from ml.comfyui_build_txt2img_workflow import comfyui_build_txt2img_workflow
|
||||
from ml.comfyui_inject_multi_lora import comfyui_inject_multi_lora
|
||||
|
||||
if not positive or not positive.strip():
|
||||
raise ValueError("comfyui_build_pixelart_workflow: 'positive' no puede estar vacio")
|
||||
|
||||
lora_strength = max(0.0, min(2.0, float(lora_strength)))
|
||||
lcm_strength = max(0.0, min(2.0, float(lcm_strength)))
|
||||
|
||||
defaults = _LCM_DEFAULTS if use_lcm else _PLAIN_DEFAULTS
|
||||
eff_steps = defaults["steps"] if steps is None else int(steps)
|
||||
eff_cfg = defaults["cfg"] if cfg is None else float(cfg)
|
||||
eff_sampler = defaults["sampler_name"] if sampler_name is None else sampler_name
|
||||
eff_scheduler = defaults["scheduler"] if scheduler is None else scheduler
|
||||
|
||||
base = comfyui_build_txt2img_workflow(
|
||||
ckpt_name,
|
||||
positive,
|
||||
negative,
|
||||
steps=eff_steps,
|
||||
cfg=eff_cfg,
|
||||
width=width,
|
||||
height=height,
|
||||
seed=seed,
|
||||
sampler_name=eff_sampler,
|
||||
scheduler=eff_scheduler,
|
||||
filename_prefix=filename_prefix,
|
||||
)
|
||||
|
||||
loras = [
|
||||
{"name": pixel_lora, "strength_model": lora_strength, "strength_clip": lora_strength},
|
||||
]
|
||||
if use_lcm:
|
||||
loras.append(
|
||||
{"name": lcm_lora, "strength_model": lcm_strength, "strength_clip": lcm_strength}
|
||||
)
|
||||
|
||||
return comfyui_inject_multi_lora(base, loras)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
|
||||
wf = comfyui_build_pixelart_workflow(
|
||||
"isometric tiny house, pixel, 32x32 style, vibrant colors",
|
||||
use_lcm=True,
|
||||
seed=42,
|
||||
)
|
||||
print(json.dumps(wf, indent=2))
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Tests offline de comfyui_build_pixelart_workflow (sin red ni GPU; estructura del dict)."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from ml.comfyui_build_pixelart_workflow import comfyui_build_pixelart_workflow # noqa: E402
|
||||
|
||||
|
||||
def _classes(wf):
|
||||
return [n["class_type"] for n in wf.values()]
|
||||
|
||||
|
||||
def _ksampler(wf):
|
||||
return next(n for n in wf.values() if n["class_type"] == "KSampler")
|
||||
|
||||
|
||||
def test_golden_lcm_two_loras():
|
||||
wf = comfyui_build_pixelart_workflow("isometric house, pixel, 32x32 style", use_lcm=True)
|
||||
cls = _classes(wf)
|
||||
# Dos LoraLoader: pixel-art-xl + lcm-lora-sdxl.
|
||||
loras = [n for n in wf.values() if n["class_type"] == "LoraLoader"]
|
||||
assert len(loras) == 2
|
||||
names = {n["inputs"]["lora_name"] for n in loras}
|
||||
assert names == {"pixel-art-xl.safetensors", "lcm-lora-sdxl.safetensors"}
|
||||
px = next(n for n in loras if n["inputs"]["lora_name"] == "pixel-art-xl.safetensors")
|
||||
assert px["inputs"]["strength_model"] == 1.2
|
||||
# KSampler con defaults LCM.
|
||||
ks = _ksampler(wf)["inputs"]
|
||||
assert ks["steps"] == 8 and ks["cfg"] == 1.5
|
||||
assert ks["sampler_name"] == "lcm" and ks["scheduler"] == "sgm_uniform"
|
||||
assert "CheckpointLoaderSimple" in cls and "SaveImage" in cls
|
||||
|
||||
|
||||
def test_edge_no_lcm_single_lora():
|
||||
wf = comfyui_build_pixelart_workflow("a pixel sword", use_lcm=False)
|
||||
loras = [n for n in wf.values() if n["class_type"] == "LoraLoader"]
|
||||
assert len(loras) == 1
|
||||
assert loras[0]["inputs"]["lora_name"] == "pixel-art-xl.safetensors"
|
||||
ks = _ksampler(wf)["inputs"]
|
||||
assert ks["steps"] == 25 and ks["cfg"] == 7.0
|
||||
assert ks["sampler_name"] == "euler" and ks["scheduler"] == "normal"
|
||||
|
||||
|
||||
def test_edge_overrides_and_clamp():
|
||||
wf = comfyui_build_pixelart_workflow(
|
||||
"pixel knight", use_lcm=True, steps=12, cfg=2.0, lora_strength=5.0
|
||||
)
|
||||
ks = _ksampler(wf)["inputs"]
|
||||
assert ks["steps"] == 12 and ks["cfg"] == 2.0
|
||||
px = next(
|
||||
n for n in wf.values()
|
||||
if n["class_type"] == "LoraLoader" and n["inputs"]["lora_name"] == "pixel-art-xl.safetensors"
|
||||
)
|
||||
assert px["inputs"]["strength_model"] == 2.0 # clamp a [0,2]
|
||||
|
||||
|
||||
def test_error_empty_prompt():
|
||||
try:
|
||||
comfyui_build_pixelart_workflow(" ")
|
||||
assert False, "deberia lanzar ValueError"
|
||||
except ValueError as e:
|
||||
assert "positive" in str(e)
|
||||
|
||||
|
||||
def test_determinism():
|
||||
a = comfyui_build_pixelart_workflow("pixel cat", seed=3)
|
||||
b = comfyui_build_pixelart_workflow("pixel cat", seed=3)
|
||||
assert a == b
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
name: comfyui_build_seamless_tile_workflow
|
||||
kind: function
|
||||
lang: py
|
||||
domain: ml
|
||||
version: "1.0.0"
|
||||
purity: pure
|
||||
signature: "def comfyui_build_seamless_tile_workflow(positive: str, negative: str = \"\", *, ckpt_name: str = \"dreamshaper_8.safetensors\", tiling: str = \"enable\", copy_model: str = \"Make a copy\", circular_vae: bool = True, material_lora: str | None = None, lora_strength: float = 1.0, steps: int = 20, cfg: float = 7.0, width: int = 512, height: int = 512, seed: int = 0, sampler_name: str = \"euler\", scheduler: str = \"normal\", filename_prefix: str = \"seamless\") -> dict"
|
||||
description: "Construye el dict (API format) de un workflow ComfyUI de textura SEAMLESS (tileable) usando el custom node spinagon/ComfyUI-seamless-tiling: inserta SeamlessTile (Conv2d circular) entre la fuente MODEL y el KSampler, y CircularVAEDecode en lugar de VAEDecode. Compone comfyui_build_txt2img_workflow (+ comfyui_inject_lora opcional). Pura, sin red ni I/O. class_types verificados contra /object_info."
|
||||
tags: [comfyui, ml, gamedev, gamedev-2d, seamless, tile, texture, workflow, stable-diffusion]
|
||||
uses_functions: [comfyui_build_txt2img_workflow_py_ml, comfyui_inject_lora_py_ml]
|
||||
uses_types: []
|
||||
returns: []
|
||||
returns_optional: false
|
||||
error_type: ""
|
||||
imports: []
|
||||
params:
|
||||
- name: positive
|
||||
desc: "Prompt de la textura (ej. 'seamless grass texture, top down'). No puede estar vacio."
|
||||
- name: negative
|
||||
desc: "Prompt negativo (ej. 'seam, border, frame')."
|
||||
- name: ckpt_name
|
||||
desc: "Checkpoint. Default 'dreamshaper_8.safetensors' (SD1.5 holgado en 8GB; el seamless cuesta ~0 VRAM). keyword-only."
|
||||
- name: tiling
|
||||
desc: "Eje de tileado: 'enable' (ambos), 'x_only' (horizontal, muros), 'y_only' (vertical), 'disable'. Un solo widget. keyword-only."
|
||||
- name: copy_model
|
||||
desc: "'Make a copy' (recomendado, no contamina el modelo cacheado del server compartido) o 'Modify in place'. keyword-only."
|
||||
- name: circular_vae
|
||||
desc: "Si True reemplaza VAEDecode por CircularVAEDecode (decode tambien circular, sin color-bleeding). keyword-only."
|
||||
- name: material_lora
|
||||
desc: "LoRA de material/estilo opcional a inyectar antes del SeamlessTile (ej. piedra/madera). None = sin LoRA. keyword-only."
|
||||
- name: lora_strength
|
||||
desc: "Fuerza del material_lora sobre model y clip. keyword-only."
|
||||
- name: steps
|
||||
desc: "Pasos del KSampler. keyword-only."
|
||||
- name: cfg
|
||||
desc: "CFG del KSampler. keyword-only."
|
||||
- name: width
|
||||
desc: "Ancho en px (multiplo de 8). keyword-only."
|
||||
- name: height
|
||||
desc: "Alto en px (multiplo de 8). keyword-only."
|
||||
- name: seed
|
||||
desc: "Semilla del KSampler. keyword-only."
|
||||
- name: sampler_name
|
||||
desc: "Sampler del KSampler. keyword-only."
|
||||
- name: scheduler
|
||||
desc: "Scheduler del KSampler. keyword-only."
|
||||
- name: filename_prefix
|
||||
desc: "Prefijo del PNG en output/. keyword-only."
|
||||
output: "dict en API format listo para comfyui_submit_workflow: txt2img base (+ LoRA opcional) con un SeamlessTile entre la fuente MODEL y el KSampler, y CircularVAEDecode (si circular_vae) en lugar de VAEDecode."
|
||||
tested: true
|
||||
tests: ["golden: SeamlessTile insertado, KSampler.model -> SeamlessTile, CircularVAEDecode reemplaza VAEDecode, SeamlessTile.model -> checkpoint", "edge tiling='x_only' propagado a SeamlessTile y CircularVAEDecode", "edge circular_vae=False conserva VAEDecode", "edge material_lora: SeamlessTile.model -> LoRA", "error positive vacio / tiling invalido / copy_model invalido -> ValueError", "determinismo"]
|
||||
test_file_path: "python/functions/ml/comfyui_build_seamless_tile_workflow_test.py"
|
||||
file_path: "python/functions/ml/comfyui_build_seamless_tile_workflow.py"
|
||||
---
|
||||
|
||||
## Ejemplo
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from ml.comfyui_build_seamless_tile_workflow import comfyui_build_seamless_tile_workflow
|
||||
|
||||
wf = comfyui_build_seamless_tile_workflow(
|
||||
"seamless mossy stone wall texture, top down, game tile",
|
||||
negative="seam, border, frame, watermark",
|
||||
tiling="enable", # 'x_only' para muros/cielos
|
||||
seed=7,
|
||||
)
|
||||
# -> comfyui_submit_workflow + comfyui_wait_result + comfyui_fetch_output_image
|
||||
# Verificar costura: generar y montar 3x3 (o nodo OffsetImage 50/50); si aparece
|
||||
# costura en el centro, no es seamless.
|
||||
```
|
||||
|
||||
O lanzable directo con: `./fn run comfyui_build_seamless_tile_workflow` (imprime el JSON del workflow de ejemplo).
|
||||
|
||||
## Cuando usarla
|
||||
|
||||
Cuando necesites una textura tileable para el motor (suelos, muros, fondos
|
||||
repetibles, terrenos). El modo `x_only` para superficies que solo tilean en
|
||||
horizontal (muros, cielos). Combínala con un `material_lora` para texturas de
|
||||
material concreto (piedra, madera, metal). Para verificar que tilea de verdad,
|
||||
usa el check objetivo (montaje 3×3 / `OffsetImage` 50/50), no "se ve bien".
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Requiere el custom node `spinagon/ComfyUI-seamless-tiling` instalado** (ya lo
|
||||
está: `SeamlessTile`, `CircularVAEDecode`, `MakeCircularVAE`, `OffsetImage` en
|
||||
/object_info). Sin él, HTTP 400 al enviar.
|
||||
- **class_type `SeamlessTile` (sin espacio)**, NO el display name `"Seamless Tile"`.
|
||||
- **`SeamlessTile` va SIEMPRE entre el checkpoint/LoRAs y el KSampler**. Si fuese
|
||||
después, no tilearía (parchea las Conv2d del UNet antes del sampling).
|
||||
- **`copy_model="Make a copy"` por defecto** porque el server es compartido
|
||||
(vídeo/3D): `Modify in place` contaminaría el checkpoint cacheado en memoria.
|
||||
- Coste VRAM del seamless ≈ 0 (model-level): cabe sobre cualquier base en 8GB.
|
||||
- Función pura: no valida contra el server.
|
||||
@@ -0,0 +1,179 @@
|
||||
"""Construye un workflow ComfyUI de textura SEAMLESS (tileable) en API format.
|
||||
|
||||
Usa el custom node spinagon/ComfyUI-seamless-tiling: parchea las Conv2d del UNet
|
||||
a modo circular para que la imagen tilee sin costura. Cableado (report 0139):
|
||||
|
||||
CheckpointLoaderSimple ─MODEL─► [LoRA opcional] ─► SeamlessTile ─► KSampler
|
||||
─VAE──────────────────────────────────────► CircularVAEDecode ─► SaveImage
|
||||
|
||||
Claves verificadas contra /object_info del servidor (custom node ya instalado):
|
||||
- class_type EXACTO 'SeamlessTile' (sin espacio; el display 'Seamless Tile' NO sirve).
|
||||
inputs: model(MODEL), tiling(enum enable|x_only|y_only|disable), copy_model
|
||||
(enum 'Make a copy'|'Modify in place'). RETURN: (MODEL,).
|
||||
- SeamlessTile va SIEMPRE entre la fuente MODEL (tras las LoRAs) y el KSampler;
|
||||
si va despues, no tilea.
|
||||
- CircularVAEDecode reemplaza al VAEDecode estandar: samples(LATENT), vae(VAE),
|
||||
tiling(mismo enum). RETURN: (IMAGE,). Evita color-bleeding en bordes.
|
||||
- copy_model='Make a copy' por defecto: el server es compartido (video/3D);
|
||||
modificar el modelo in-place contaminaria el checkpoint cacheado.
|
||||
|
||||
Compone comfyui_build_txt2img_workflow (+ comfyui_inject_lora si hay material_lora)
|
||||
e inserta los nodos seamless. Verificacion de costura: OffsetImage(50,50) o montaje
|
||||
3x3 (no se hace aqui; es check e2e post-generacion).
|
||||
|
||||
Funcion pura: sin red, sin I/O. No muta el dict de entrada (copia profunda).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
_TILING = ("enable", "x_only", "y_only", "disable")
|
||||
_COPY_MODEL = ("Make a copy", "Modify in place")
|
||||
|
||||
|
||||
def _is_link(v) -> bool:
|
||||
return isinstance(v, list) and len(v) == 2 and isinstance(v[0], str) and isinstance(v[1], int)
|
||||
|
||||
|
||||
def comfyui_build_seamless_tile_workflow(
|
||||
positive: str,
|
||||
negative: str = "",
|
||||
*,
|
||||
ckpt_name: str = "dreamshaper_8.safetensors",
|
||||
tiling: str = "enable",
|
||||
copy_model: str = "Make a copy",
|
||||
circular_vae: bool = True,
|
||||
material_lora: str | None = None,
|
||||
lora_strength: float = 1.0,
|
||||
steps: int = 20,
|
||||
cfg: float = 7.0,
|
||||
width: int = 512,
|
||||
height: int = 512,
|
||||
seed: int = 0,
|
||||
sampler_name: str = "euler",
|
||||
scheduler: str = "normal",
|
||||
filename_prefix: str = "seamless",
|
||||
) -> dict:
|
||||
"""Construye el dict (API format) de un workflow de textura seamless.
|
||||
|
||||
Args:
|
||||
positive: prompt de la textura (ej. "seamless grass texture, top down").
|
||||
No puede estar vacio.
|
||||
negative: prompt negativo.
|
||||
ckpt_name: checkpoint (default 'dreamshaper_8.safetensors', SD1.5 holgado
|
||||
en 8GB; el coste VRAM del seamless es ~0). keyword-only.
|
||||
tiling: eje de tileado. 'enable' (ambos), 'x_only' (horizontal, muros),
|
||||
'y_only' (vertical), 'disable'. Un solo widget, no dos bools.
|
||||
copy_model: 'Make a copy' (recomendado, no contamina el modelo cacheado)
|
||||
o 'Modify in place'.
|
||||
circular_vae: si True reemplaza el VAEDecode por CircularVAEDecode (decode
|
||||
tambien circular, sin color-bleeding). Si False conserva VAEDecode.
|
||||
material_lora: LoRA de material/estilo opcional a inyectar antes del
|
||||
SeamlessTile (ej. una LoRA de piedra/madera). None = sin LoRA.
|
||||
lora_strength: fuerza del material_lora sobre model y clip.
|
||||
steps, cfg, width, height, seed, sampler_name, scheduler, filename_prefix:
|
||||
parametros de generacion pasados a comfyui_build_txt2img_workflow.
|
||||
|
||||
Returns:
|
||||
dict en API format listo para comfyui_submit_workflow: txt2img base
|
||||
(+ LoRA opcional) con un nodo SeamlessTile entre la fuente MODEL y el
|
||||
KSampler, y CircularVAEDecode (si circular_vae) en lugar de VAEDecode.
|
||||
|
||||
Raises:
|
||||
ValueError: si positive esta vacio, tiling/copy_model invalidos, o la base
|
||||
no tiene KSampler/VAEDecode.
|
||||
"""
|
||||
from ml.comfyui_build_txt2img_workflow import comfyui_build_txt2img_workflow
|
||||
|
||||
if not positive or not positive.strip():
|
||||
raise ValueError("comfyui_build_seamless_tile_workflow: 'positive' no puede estar vacio")
|
||||
if tiling not in _TILING:
|
||||
raise ValueError(
|
||||
f"comfyui_build_seamless_tile_workflow: tiling debe ser uno de {_TILING}, no {tiling!r}"
|
||||
)
|
||||
if copy_model not in _COPY_MODEL:
|
||||
raise ValueError(
|
||||
f"comfyui_build_seamless_tile_workflow: copy_model debe ser uno de {_COPY_MODEL}, "
|
||||
f"no {copy_model!r}"
|
||||
)
|
||||
|
||||
wf = comfyui_build_txt2img_workflow(
|
||||
ckpt_name,
|
||||
positive,
|
||||
negative,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
width=width,
|
||||
height=height,
|
||||
seed=seed,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
filename_prefix=filename_prefix,
|
||||
)
|
||||
|
||||
if material_lora:
|
||||
from ml.comfyui_inject_lora import comfyui_inject_lora
|
||||
|
||||
wf = comfyui_inject_lora(
|
||||
wf, material_lora, strength_model=lora_strength, strength_clip=lora_strength
|
||||
)
|
||||
|
||||
wf = copy.deepcopy(wf)
|
||||
|
||||
ksampler_id = next(
|
||||
(nid for nid, n in wf.items() if str(n.get("class_type", "")).endswith("KSampler")), None
|
||||
)
|
||||
if ksampler_id is None:
|
||||
raise ValueError("comfyui_build_seamless_tile_workflow: no se encontro KSampler en la base")
|
||||
ks_inputs = wf[ksampler_id]["inputs"]
|
||||
if not _is_link(ks_inputs.get("model")):
|
||||
raise ValueError(
|
||||
"comfyui_build_seamless_tile_workflow: el KSampler no tiene una fuente MODEL valida"
|
||||
)
|
||||
model_src = list(ks_inputs["model"])
|
||||
|
||||
numeric = [int(k) for k in wf.keys() if str(k).isdigit()]
|
||||
tile_id = str((max(numeric) + 1) if numeric else len(wf) + 1)
|
||||
|
||||
# SeamlessTile entre la fuente MODEL actual y el KSampler.
|
||||
wf[tile_id] = {
|
||||
"class_type": "SeamlessTile",
|
||||
"inputs": {"model": model_src, "tiling": tiling, "copy_model": copy_model},
|
||||
}
|
||||
wf[ksampler_id]["inputs"]["model"] = [tile_id, 0]
|
||||
|
||||
if circular_vae:
|
||||
vaedecode_id = next(
|
||||
(nid for nid, n in wf.items() if n.get("class_type") == "VAEDecode"), None
|
||||
)
|
||||
if vaedecode_id is None:
|
||||
raise ValueError(
|
||||
"comfyui_build_seamless_tile_workflow: no se encontro VAEDecode para reemplazar"
|
||||
)
|
||||
old = wf[vaedecode_id]["inputs"]
|
||||
wf[vaedecode_id] = {
|
||||
"class_type": "CircularVAEDecode",
|
||||
"inputs": {
|
||||
"samples": old.get("samples"),
|
||||
"vae": old.get("vae"),
|
||||
"tiling": tiling,
|
||||
},
|
||||
}
|
||||
|
||||
return wf
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
|
||||
wf = comfyui_build_seamless_tile_workflow(
|
||||
"seamless mossy stone wall texture, top down, game tile",
|
||||
negative="seam, border, frame, watermark",
|
||||
tiling="enable",
|
||||
seed=7,
|
||||
)
|
||||
print(json.dumps(wf, indent=2))
|
||||
@@ -0,0 +1,101 @@
|
||||
"""Tests offline de comfyui_build_seamless_tile_workflow (estructura del dict, sin GPU)."""
|
||||
|
||||
import copy
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from ml.comfyui_build_seamless_tile_workflow import ( # noqa: E402
|
||||
comfyui_build_seamless_tile_workflow,
|
||||
)
|
||||
|
||||
|
||||
def _by_class(wf, cls):
|
||||
return [n for n in wf.values() if n["class_type"] == cls]
|
||||
|
||||
|
||||
def _id_of(wf, cls):
|
||||
return next(nid for nid, n in wf.items() if n["class_type"] == cls)
|
||||
|
||||
|
||||
def test_golden_seamless_inserted():
|
||||
wf = comfyui_build_seamless_tile_workflow("seamless grass texture")
|
||||
tiles = _by_class(wf, "SeamlessTile")
|
||||
assert len(tiles) == 1
|
||||
st = tiles[0]
|
||||
assert st["inputs"]["tiling"] == "enable"
|
||||
assert st["inputs"]["copy_model"] == "Make a copy"
|
||||
# KSampler toma el MODEL del SeamlessTile.
|
||||
tile_id = _id_of(wf, "SeamlessTile")
|
||||
ks = next(n for n in wf.values() if n["class_type"] == "KSampler")
|
||||
assert ks["inputs"]["model"] == [tile_id, 0]
|
||||
# SeamlessTile.model apunta al checkpoint (no hay LoRA).
|
||||
ckpt_id = _id_of(wf, "CheckpointLoaderSimple")
|
||||
assert st["inputs"]["model"] == [ckpt_id, 0]
|
||||
# CircularVAEDecode reemplaza VAEDecode.
|
||||
assert len(_by_class(wf, "CircularVAEDecode")) == 1
|
||||
assert len(_by_class(wf, "VAEDecode")) == 0
|
||||
cvd = _by_class(wf, "CircularVAEDecode")[0]
|
||||
assert cvd["inputs"]["tiling"] == "enable"
|
||||
|
||||
|
||||
def test_edge_x_only():
|
||||
wf = comfyui_build_seamless_tile_workflow("brick wall", tiling="x_only")
|
||||
assert _by_class(wf, "SeamlessTile")[0]["inputs"]["tiling"] == "x_only"
|
||||
assert _by_class(wf, "CircularVAEDecode")[0]["inputs"]["tiling"] == "x_only"
|
||||
|
||||
|
||||
def test_edge_no_circular_vae():
|
||||
wf = comfyui_build_seamless_tile_workflow("grass", circular_vae=False)
|
||||
assert len(_by_class(wf, "CircularVAEDecode")) == 0
|
||||
assert len(_by_class(wf, "VAEDecode")) == 1
|
||||
assert len(_by_class(wf, "SeamlessTile")) == 1
|
||||
|
||||
|
||||
def test_edge_material_lora_before_tile():
|
||||
wf = comfyui_build_seamless_tile_workflow(
|
||||
"stone", material_lora="detail_tweaker_sd15.safetensors", lora_strength=0.7
|
||||
)
|
||||
loras = _by_class(wf, "LoraLoader")
|
||||
assert len(loras) == 1
|
||||
lora_id = _id_of(wf, "LoraLoader")
|
||||
# SeamlessTile toma el MODEL de la LoRA (no del checkpoint crudo).
|
||||
st = _by_class(wf, "SeamlessTile")[0]
|
||||
assert st["inputs"]["model"] == [lora_id, 0]
|
||||
|
||||
|
||||
def test_error_empty_prompt():
|
||||
try:
|
||||
comfyui_build_seamless_tile_workflow("")
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "positive" in str(e)
|
||||
|
||||
|
||||
def test_error_bad_tiling():
|
||||
try:
|
||||
comfyui_build_seamless_tile_workflow("x", tiling="diagonal")
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "tiling" in str(e)
|
||||
|
||||
|
||||
def test_error_bad_copy_model():
|
||||
try:
|
||||
comfyui_build_seamless_tile_workflow("x", copy_model="overwrite")
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "copy_model" in str(e)
|
||||
|
||||
|
||||
def test_purity_input_not_mutated():
|
||||
from ml.comfyui_build_txt2img_workflow import comfyui_build_txt2img_workflow
|
||||
|
||||
base = comfyui_build_txt2img_workflow("dreamshaper_8.safetensors", "x")
|
||||
snapshot = copy.deepcopy(base)
|
||||
# No es la entrada directa, pero verificamos que el builder no muta el resultado de txt2img
|
||||
# llamandolo dos veces sin estado compartido.
|
||||
a = comfyui_build_seamless_tile_workflow("grass", seed=1)
|
||||
b = comfyui_build_seamless_tile_workflow("grass", seed=1)
|
||||
assert a == b
|
||||
assert base == snapshot
|
||||
@@ -0,0 +1,112 @@
|
||||
---
|
||||
name: comfyui_build_sprite_sheet_workflow
|
||||
kind: function
|
||||
lang: py
|
||||
domain: ml
|
||||
version: "1.0.0"
|
||||
purity: pure
|
||||
signature: "def comfyui_build_sprite_sheet_workflow(subject: str, *, ref_image: str | None = None, pose_skeleton: str | None = None, ckpt_name: str = \"dreamshaper_8.safetensors\", char_lora: str | None = None, lora_strength: float = 1.0, controlnet_name: str = \"control_v11p_sd15_openpose_fp16.safetensors\", controlnet_strength: float = 0.55, controlnet_start: float = 0.0, controlnet_end: float = 0.8, transparent: bool = True, rembg_model: str = \"u2net\", weight: float = 0.75, negative: str = \"blurry, lowres, extra limbs, deformed\", width: int = 512, height: int = 768, steps: int = 24, cfg: float = 7.0, seed: int = 0, sampler_name: str = \"dpmpp_2m\", scheduler: str = \"karras\", filename_prefix: str = \"sprite\") -> dict"
|
||||
description: "Construye el dict (API format) del workflow de UN sprite de personaje 2D: identidad (IPAdapter-FaceID si ref_image) + LoRA opcional + pose (ControlNet OpenPose con ControlNetApplyAdvanced end<1.0) + transparencia (Rembg). Compone comfyui_build_ipadapter_workflow / comfyui_build_txt2img_workflow + comfyui_inject_lora. Es UN frame; varias poses (misma seed) -> sprite sheet montado con comfyui_build_grid. Pura, sin red ni I/O. class_types verificados contra /object_info."
|
||||
tags: [comfyui, ml, gamedev, gamedev-2d, sprite, character, faceid, openpose, workflow]
|
||||
uses_functions: [comfyui_build_txt2img_workflow_py_ml, comfyui_build_ipadapter_workflow_py_ml, comfyui_inject_lora_py_ml]
|
||||
uses_types: []
|
||||
returns: []
|
||||
returns_optional: false
|
||||
error_type: ""
|
||||
imports: []
|
||||
params:
|
||||
- name: subject
|
||||
desc: "Descripcion del personaje (ej. 'a knight in silver armor'). Se completa con ', full body, game sprite, simple background'. No puede estar vacio."
|
||||
- name: ref_image
|
||||
desc: "Imagen de referencia de rostro en input/ del servidor. Si se pasa, usa IPAdapter-FaceID (identidad consistente). None = identidad solo por prompt + seed. keyword-only."
|
||||
- name: pose_skeleton
|
||||
desc: "Imagen de esqueleto OpenPose en input/ para fijar la pose via ControlNet. None = pose libre. keyword-only."
|
||||
- name: ckpt_name
|
||||
desc: "Checkpoint SD1.5 (FaceID + OpenPose solo instalados en SD1.5; default 'dreamshaper_8.safetensors'). keyword-only."
|
||||
- name: char_lora
|
||||
desc: "LoRA de personaje/estilo opcional en models/loras. keyword-only."
|
||||
- name: lora_strength
|
||||
desc: "Fuerza del char_lora sobre model y clip. keyword-only."
|
||||
- name: controlnet_name
|
||||
desc: "ControlNet OpenPose (default SD1.5). keyword-only."
|
||||
- name: controlnet_strength
|
||||
desc: "Fuerza del OpenPose (recomendado 0.55). keyword-only."
|
||||
- name: controlnet_start
|
||||
desc: "Inicio de aplicacion del OpenPose (fraccion 0..1). keyword-only."
|
||||
- name: controlnet_end
|
||||
desc: "Fin de aplicacion del OpenPose (end<1.0 deja libres los ultimos pasos para pelo/ropa). keyword-only."
|
||||
- name: transparent
|
||||
desc: "Si True inyecta Rembg para PNG con alpha. False = opaco. keyword-only."
|
||||
- name: rembg_model
|
||||
desc: "Modelo Rembg ('u2net' general, 'isnet-anime' para anime). keyword-only."
|
||||
- name: weight
|
||||
desc: "Peso del IPAdapter-FaceID (solo si ref_image). keyword-only."
|
||||
- name: negative
|
||||
desc: "Prompt negativo. keyword-only."
|
||||
- name: width
|
||||
desc: "Ancho en px (512). keyword-only."
|
||||
- name: height
|
||||
desc: "Alto en px (768, vertical, encuadra cuerpo entero). keyword-only."
|
||||
- name: steps
|
||||
desc: "Pasos del KSampler. keyword-only."
|
||||
- name: cfg
|
||||
desc: "CFG del KSampler. keyword-only."
|
||||
- name: seed
|
||||
desc: "Semilla del KSampler (fijar igual entre poses para identidad estable). keyword-only."
|
||||
- name: sampler_name
|
||||
desc: "Sampler del KSampler. keyword-only."
|
||||
- name: scheduler
|
||||
desc: "Scheduler del KSampler. keyword-only."
|
||||
- name: filename_prefix
|
||||
desc: "Prefijo del PNG en output/. keyword-only."
|
||||
output: "dict en API format listo para comfyui_submit_workflow: base txt2img (o IPAdapter-FaceID si ref_image) + LoRA opcional + ControlNet OpenPose (si pose_skeleton, via ControlNetApplyAdvanced) + Rembg (si transparent). UN sprite; varias poses -> sprite sheet."
|
||||
tested: true
|
||||
tests: ["golden ref+pose+transparent: clases IPAdapterUnifiedLoaderFaceID/IPAdapterFaceID/ControlNetApplyAdvanced/Image Rembg; KSampler.positive/negative <- ControlNetApplyAdvanced; SaveImage <- Rembg; end_percent 0.8", "edge sin ref: sin IPAdapter, base txt2img", "edge transparent=False: sin Rembg, SaveImage <- VAEDecode", "edge char_lora: LoraLoader presente", "error subject vacio -> ValueError", "determinismo"]
|
||||
test_file_path: "python/functions/ml/comfyui_build_sprite_sheet_workflow_test.py"
|
||||
file_path: "python/functions/ml/comfyui_build_sprite_sheet_workflow.py"
|
||||
---
|
||||
|
||||
## Ejemplo
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from ml.comfyui_build_sprite_sheet_workflow import comfyui_build_sprite_sheet_workflow
|
||||
|
||||
# Un frame del turnaround: identidad (FaceID) + pose frontal + alpha.
|
||||
wf = comfyui_build_sprite_sheet_workflow(
|
||||
"a knight in silver armor",
|
||||
ref_image="faceref.png", # cara de referencia en input/
|
||||
pose_skeleton="pose_front.png", # esqueleto OpenPose en input/
|
||||
transparent=True,
|
||||
seed=5,
|
||||
)
|
||||
# Sprite sheet completo: repetir cambiando pose_skeleton (front/side/back),
|
||||
# MISMA seed para identidad estable, y montar los PNG con alpha:
|
||||
# comfyui_submit_workflow x N -> comfyui_build_grid(paths)
|
||||
```
|
||||
|
||||
O lanzable directo con: `./fn run comfyui_build_sprite_sheet_workflow` (imprime nodos + class_types del ejemplo).
|
||||
|
||||
## Cuando usarla
|
||||
|
||||
Cuando generes personajes 2D para un juego con identidad consistente entre poses
|
||||
(turnaround front/side/back, set de acciones). Fija `seed` y `ref_image` iguales
|
||||
entre llamadas y varía `pose_skeleton` para mantener la misma cara. `transparent`
|
||||
deja el sprite recortado (alpha) listo para el motor. Para el contact-sheet final,
|
||||
monta los PNG resultantes con `comfyui_build_grid`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Solo SD1.5 hoy**: IPAdapter-FaceID y ControlNet-OpenPose están instalados solo
|
||||
en SD1.5. Usa `dreamshaper_8` u otro checkpoint SD1.5.
|
||||
- **`ref_image`/`pose_skeleton` son nombres de archivos en el dir `input/` del
|
||||
servidor**, no rutas locales. Súbelas antes (LoadImage las lee de ahí).
|
||||
- **Usa `ControlNetApplyAdvanced`** (no el legacy `ControlNetApply`): `end_percent`
|
||||
0.8 deja los últimos pasos libres para que pelo/ropa no queden aplastados contra
|
||||
el esqueleto.
|
||||
- `Image Rembg` da matting binario (silueta sólida) — perfecto para personajes,
|
||||
pero NO para efectos translúcidos (humo/fuego): para eso, luma-as-alpha.
|
||||
- Sin `char_lora` la consistencia de ropa/cuerpo entre vistas depende de
|
||||
prompt+seed (FaceID fija sobre todo la cara). Una LoRA de personaje la mejora.
|
||||
- Función pura: no valida contra el server.
|
||||
@@ -0,0 +1,259 @@
|
||||
"""Construye el workflow ComfyUI de UN sprite de personaje (identidad + pose + alpha).
|
||||
|
||||
Receta de personaje del report 0137, construible HOY en SD1.5 (todo verificado en
|
||||
/object_info): IPAdapter-FaceID (identidad de rostro) + LoRA opcional (estilo/char)
|
||||
+ ControlNet OpenPose (pose) + Rembg (transparencia). Produce el workflow de UNA
|
||||
pose/vista; un sprite sheet completo se obtiene generando varias poses (cambiando
|
||||
`pose_skeleton` + prompt, misma `seed` para identidad estable) y montandolas con
|
||||
`comfyui_build_grid` / `comfyui_assemble_sprite_sheet` en un pipeline.
|
||||
|
||||
Compone funciones existentes del registry:
|
||||
- comfyui_build_ipadapter_workflow (mode='faceid') -> base + identidad de cara
|
||||
(o comfyui_build_txt2img_workflow si no hay ref_image)
|
||||
- comfyui_inject_lora -> LoRA de personaje/estilo
|
||||
- ControlNetApplyAdvanced (helper local) -> pose OpenPose con end<1.0
|
||||
- Image Rembg (Remove Background) (helper local) -> PNG con alpha
|
||||
|
||||
class_types/inputs verificados contra /object_info (8GB lowvram):
|
||||
IPAdapterUnifiedLoaderFaceID/IPAdapterFaceID (FaceID solo SD1.5 instalado),
|
||||
control_v11p_sd15_openpose_fp16.safetensors, ControlNetApplyAdvanced
|
||||
(positive,negative,control_net,image,strength,start_percent,end_percent ->
|
||||
positive,negative), 'Image Rembg (Remove Background)' (transparency BOOLEAN).
|
||||
|
||||
Por que ControlNetApplyAdvanced y no el legacy ControlNetApply (comfyui_inject_controlnet):
|
||||
end_percent<1.0 baja la fuerza del esqueleto en los ultimos pasos para que el pelo
|
||||
y la ropa no queden aplastados contra el OpenPose (report 0137).
|
||||
|
||||
Funcion pura: sin red, sin I/O. No muta dicts (copia profunda en los helpers).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
|
||||
def _is_link(v) -> bool:
|
||||
return isinstance(v, list) and len(v) == 2 and isinstance(v[0], str) and isinstance(v[1], int)
|
||||
|
||||
|
||||
def _next_id(wf: dict) -> int:
|
||||
numeric = [int(k) for k in wf.keys() if str(k).isdigit()]
|
||||
return (max(numeric) + 1) if numeric else len(wf) + 1
|
||||
|
||||
|
||||
def _inject_openpose_advanced(
|
||||
workflow: dict,
|
||||
skeleton_image: str,
|
||||
cn_name: str,
|
||||
strength: float,
|
||||
start_percent: float,
|
||||
end_percent: float,
|
||||
) -> dict:
|
||||
"""Inserta LoadImage + ControlNetLoader + ControlNetApplyAdvanced (positive+negative).
|
||||
|
||||
Repunta KSampler.positive y KSampler.negative a las salidas del
|
||||
ControlNetApplyAdvanced. Mas fino que el legacy ControlNetApply: permite
|
||||
start/end percent (bajar la pose en los ultimos pasos).
|
||||
"""
|
||||
wf = copy.deepcopy(workflow)
|
||||
ks_id = next(
|
||||
(nid for nid, n in wf.items() if str(n.get("class_type", "")).endswith("KSampler")), None
|
||||
)
|
||||
if ks_id is None:
|
||||
raise ValueError("comfyui_build_sprite_sheet_workflow: no se encontro KSampler")
|
||||
ks = wf[ks_id]["inputs"]
|
||||
if not (_is_link(ks.get("positive")) and _is_link(ks.get("negative"))):
|
||||
raise ValueError(
|
||||
"comfyui_build_sprite_sheet_workflow: el KSampler necesita positive y negative validos"
|
||||
)
|
||||
pos_src, neg_src = list(ks["positive"]), list(ks["negative"])
|
||||
|
||||
base = _next_id(wf)
|
||||
load_id, loader_id, apply_id = str(base), str(base + 1), str(base + 2)
|
||||
wf[load_id] = {"class_type": "LoadImage", "inputs": {"image": skeleton_image}}
|
||||
wf[loader_id] = {"class_type": "ControlNetLoader", "inputs": {"control_net_name": cn_name}}
|
||||
wf[apply_id] = {
|
||||
"class_type": "ControlNetApplyAdvanced",
|
||||
"inputs": {
|
||||
"positive": pos_src,
|
||||
"negative": neg_src,
|
||||
"control_net": [loader_id, 0],
|
||||
"image": [load_id, 0],
|
||||
"strength": strength,
|
||||
"start_percent": start_percent,
|
||||
"end_percent": end_percent,
|
||||
},
|
||||
}
|
||||
wf[ks_id]["inputs"]["positive"] = [apply_id, 0]
|
||||
wf[ks_id]["inputs"]["negative"] = [apply_id, 1]
|
||||
return wf
|
||||
|
||||
|
||||
def _inject_rembg(workflow: dict, model: str) -> dict:
|
||||
"""Inserta 'Image Rembg (Remove Background)' (transparency=True) entre VAEDecode y SaveImage."""
|
||||
wf = copy.deepcopy(workflow)
|
||||
vaedecode_id = next(
|
||||
(nid for nid, n in wf.items() if n.get("class_type") == "VAEDecode"), None
|
||||
)
|
||||
save_id = next((nid for nid, n in wf.items() if n.get("class_type") == "SaveImage"), None)
|
||||
if vaedecode_id is None or save_id is None:
|
||||
raise ValueError(
|
||||
"comfyui_build_sprite_sheet_workflow: no se encontro VAEDecode/SaveImage para Rembg"
|
||||
)
|
||||
rembg_id = str(_next_id(wf))
|
||||
wf[rembg_id] = {
|
||||
"class_type": "Image Rembg (Remove Background)",
|
||||
"inputs": {
|
||||
"images": [vaedecode_id, 0],
|
||||
"transparency": True,
|
||||
"model": model,
|
||||
"post_processing": False,
|
||||
"only_mask": False,
|
||||
"alpha_matting": False,
|
||||
"alpha_matting_foreground_threshold": 240,
|
||||
"alpha_matting_background_threshold": 10,
|
||||
"alpha_matting_erode_size": 10,
|
||||
"background_color": "none",
|
||||
},
|
||||
}
|
||||
wf[save_id]["inputs"]["images"] = [rembg_id, 0]
|
||||
return wf
|
||||
|
||||
|
||||
def comfyui_build_sprite_sheet_workflow(
|
||||
subject: str,
|
||||
*,
|
||||
ref_image: str | None = None,
|
||||
pose_skeleton: str | None = None,
|
||||
ckpt_name: str = "dreamshaper_8.safetensors",
|
||||
char_lora: str | None = None,
|
||||
lora_strength: float = 1.0,
|
||||
controlnet_name: str = "control_v11p_sd15_openpose_fp16.safetensors",
|
||||
controlnet_strength: float = 0.55,
|
||||
controlnet_start: float = 0.0,
|
||||
controlnet_end: float = 0.8,
|
||||
transparent: bool = True,
|
||||
rembg_model: str = "u2net",
|
||||
weight: float = 0.75,
|
||||
negative: str = "blurry, lowres, extra limbs, deformed",
|
||||
width: int = 512,
|
||||
height: int = 768,
|
||||
steps: int = 24,
|
||||
cfg: float = 7.0,
|
||||
seed: int = 0,
|
||||
sampler_name: str = "dpmpp_2m",
|
||||
scheduler: str = "karras",
|
||||
filename_prefix: str = "sprite",
|
||||
) -> dict:
|
||||
"""Construye el dict (API format) del workflow de un sprite de personaje.
|
||||
|
||||
Args:
|
||||
subject: descripcion del personaje (ej. "a knight in silver armor"). Se
|
||||
completa con ", full body, game sprite, simple background". No vacio.
|
||||
ref_image: imagen de referencia de rostro en input/ del servidor. Si se
|
||||
pasa, usa IPAdapter-FaceID (identidad consistente). None = identidad
|
||||
solo por prompt + seed. keyword-only.
|
||||
pose_skeleton: imagen de esqueleto OpenPose en input/ para fijar la pose
|
||||
via ControlNet. None = pose libre. keyword-only.
|
||||
ckpt_name: checkpoint SD1.5 (FaceID + OpenPose solo instalados en SD1.5;
|
||||
default 'dreamshaper_8.safetensors'). keyword-only.
|
||||
char_lora: LoRA de personaje/estilo opcional en models/loras.
|
||||
lora_strength: fuerza del char_lora sobre model y clip.
|
||||
controlnet_name: ControlNet OpenPose (default SD1.5).
|
||||
controlnet_strength: fuerza del OpenPose (recomendado 0.55).
|
||||
controlnet_start, controlnet_end: rango de aplicacion del OpenPose
|
||||
(end<1.0 deja libres los ultimos pasos para pelo/ropa).
|
||||
transparent: si True inyecta Rembg para PNG con alpha. False = opaco.
|
||||
rembg_model: modelo Rembg ('u2net' general, 'isnet-anime' para anime).
|
||||
weight: peso del IPAdapter-FaceID (solo si ref_image).
|
||||
negative, width, height, steps, cfg, seed, sampler_name, scheduler,
|
||||
filename_prefix: parametros de generacion. width 512 x height 768
|
||||
(vertical, encuadra cuerpo entero).
|
||||
|
||||
Returns:
|
||||
dict en API format listo para comfyui_submit_workflow: base txt2img (o
|
||||
IPAdapter-FaceID) + LoRA opcional + ControlNet OpenPose (si pose_skeleton)
|
||||
+ Rembg (si transparent). Es UN sprite; varias poses -> sprite sheet.
|
||||
|
||||
Raises:
|
||||
ValueError: si subject esta vacio, o si la base no tiene KSampler/
|
||||
VAEDecode/SaveImage donde inyectar (propagado por los helpers).
|
||||
"""
|
||||
from ml.comfyui_build_txt2img_workflow import comfyui_build_txt2img_workflow
|
||||
|
||||
if not subject or not subject.strip():
|
||||
raise ValueError("comfyui_build_sprite_sheet_workflow: 'subject' no puede estar vacio")
|
||||
|
||||
positive = f"{subject}, full body, game sprite, simple background"
|
||||
|
||||
if ref_image:
|
||||
from ml.comfyui_build_ipadapter_workflow import comfyui_build_ipadapter_workflow
|
||||
|
||||
wf = comfyui_build_ipadapter_workflow(
|
||||
positive,
|
||||
ref_image,
|
||||
base_checkpoint=ckpt_name,
|
||||
mode="faceid",
|
||||
weight=weight,
|
||||
negative=negative,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
width=width,
|
||||
height=height,
|
||||
seed=seed,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
filename_prefix=filename_prefix,
|
||||
)
|
||||
else:
|
||||
wf = comfyui_build_txt2img_workflow(
|
||||
ckpt_name,
|
||||
positive,
|
||||
negative,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
width=width,
|
||||
height=height,
|
||||
seed=seed,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
filename_prefix=filename_prefix,
|
||||
)
|
||||
|
||||
if char_lora:
|
||||
from ml.comfyui_inject_lora import comfyui_inject_lora
|
||||
|
||||
wf = comfyui_inject_lora(
|
||||
wf, char_lora, strength_model=lora_strength, strength_clip=lora_strength
|
||||
)
|
||||
|
||||
if pose_skeleton:
|
||||
wf = _inject_openpose_advanced(
|
||||
wf,
|
||||
pose_skeleton,
|
||||
controlnet_name,
|
||||
controlnet_strength,
|
||||
controlnet_start,
|
||||
controlnet_end,
|
||||
)
|
||||
|
||||
if transparent:
|
||||
wf = _inject_rembg(wf, rembg_model)
|
||||
|
||||
return wf
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
|
||||
wf = comfyui_build_sprite_sheet_workflow(
|
||||
"a knight in silver armor",
|
||||
ref_image="faceref.png",
|
||||
pose_skeleton="pose_front.png",
|
||||
transparent=True,
|
||||
seed=5,
|
||||
)
|
||||
print(json.dumps({"nodes": list(wf), "classes": sorted({n["class_type"] for n in wf.values()})}, indent=2))
|
||||
@@ -0,0 +1,87 @@
|
||||
"""Tests offline de comfyui_build_sprite_sheet_workflow (estructura del dict, sin GPU)."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from ml.comfyui_build_sprite_sheet_workflow import ( # noqa: E402
|
||||
comfyui_build_sprite_sheet_workflow,
|
||||
)
|
||||
|
||||
|
||||
def _classes(wf):
|
||||
return sorted({n["class_type"] for n in wf.values()})
|
||||
|
||||
|
||||
def _by_class(wf, cls):
|
||||
return [n for n in wf.values() if n["class_type"] == cls]
|
||||
|
||||
|
||||
def _id_of(wf, cls):
|
||||
return next(nid for nid, n in wf.items() if n["class_type"] == cls)
|
||||
|
||||
|
||||
def test_golden_full_recipe():
|
||||
wf = comfyui_build_sprite_sheet_workflow(
|
||||
"a knight in silver armor",
|
||||
ref_image="faceref.png",
|
||||
pose_skeleton="pose_front.png",
|
||||
transparent=True,
|
||||
)
|
||||
cls = _classes(wf)
|
||||
assert "IPAdapterUnifiedLoaderFaceID" in cls
|
||||
assert "IPAdapterFaceID" in cls
|
||||
assert "ControlNetApplyAdvanced" in cls
|
||||
assert "Image Rembg (Remove Background)" in cls
|
||||
# KSampler.positive/negative vienen del ControlNetApplyAdvanced.
|
||||
cna_id = _id_of(wf, "ControlNetApplyAdvanced")
|
||||
ks = next(n for n in wf.values() if n["class_type"] == "KSampler")
|
||||
assert ks["inputs"]["positive"] == [cna_id, 0]
|
||||
assert ks["inputs"]["negative"] == [cna_id, 1]
|
||||
# SaveImage toma la imagen del Rembg.
|
||||
rembg_id = _id_of(wf, "Image Rembg (Remove Background)")
|
||||
save = next(n for n in wf.values() if n["class_type"] == "SaveImage")
|
||||
assert save["inputs"]["images"] == [rembg_id, 0]
|
||||
# ControlNetApplyAdvanced con end_percent<1.0.
|
||||
cna = _by_class(wf, "ControlNetApplyAdvanced")[0]
|
||||
assert cna["inputs"]["end_percent"] == 0.8
|
||||
|
||||
|
||||
def test_edge_no_ref_image():
|
||||
wf = comfyui_build_sprite_sheet_workflow("a goblin", pose_skeleton="pose.png")
|
||||
cls = _classes(wf)
|
||||
assert "IPAdapterFaceID" not in cls
|
||||
assert "CheckpointLoaderSimple" in cls
|
||||
assert "ControlNetApplyAdvanced" in cls
|
||||
|
||||
|
||||
def test_edge_opaque_no_rembg():
|
||||
wf = comfyui_build_sprite_sheet_workflow("a mage", transparent=False)
|
||||
assert "Image Rembg (Remove Background)" not in _classes(wf)
|
||||
# SaveImage toma del VAEDecode directamente.
|
||||
vd_id = _id_of(wf, "VAEDecode")
|
||||
save = next(n for n in wf.values() if n["class_type"] == "SaveImage")
|
||||
assert save["inputs"]["images"] == [vd_id, 0]
|
||||
|
||||
|
||||
def test_edge_char_lora():
|
||||
wf = comfyui_build_sprite_sheet_workflow(
|
||||
"a hero", char_lora="anime_style_box_sd15.safetensors", lora_strength=0.8
|
||||
)
|
||||
loras = _by_class(wf, "LoraLoader")
|
||||
assert len(loras) == 1
|
||||
assert loras[0]["inputs"]["lora_name"] == "anime_style_box_sd15.safetensors"
|
||||
|
||||
|
||||
def test_error_empty_subject():
|
||||
try:
|
||||
comfyui_build_sprite_sheet_workflow(" ")
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "subject" in str(e)
|
||||
|
||||
|
||||
def test_determinism():
|
||||
a = comfyui_build_sprite_sheet_workflow("a knight", pose_skeleton="p.png", seed=4)
|
||||
b = comfyui_build_sprite_sheet_workflow("a knight", pose_skeleton="p.png", seed=4)
|
||||
assert a == b
|
||||
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: comfyui_build_vfx_spritesheet_workflow
|
||||
kind: function
|
||||
lang: py
|
||||
domain: ml
|
||||
version: "1.0.0"
|
||||
purity: pure
|
||||
signature: "def comfyui_build_vfx_spritesheet_workflow(prompt: str, *, checkpoint: str = \"dreamshaper_8.safetensors\", motion_model: str = \"mm_sd_v15_v2.ckpt\", beta_schedule: str = \"sqrt_linear (AnimateDiff)\", lora: str | None = None, lora_strength: float = 1.1, negative: str = \"low quality, watermark, text, background detail\", width: int = 512, height: int = 512, num_frames: int = 16, context_length: int = 16, context_stride: int = 1, context_overlap: int = 4, closed_loop: bool = True, steps: int = 20, cfg: float = 7.5, sampler_name: str = \"euler\", scheduler: str = \"normal\", seed: int = 0, filename_prefix: str = \"vfx_loop\") -> dict"
|
||||
description: "Construye el dict (API format) del workflow ComfyUI AnimateDiff loop para generar N frames de un efecto VFX 2D (humo/fuego/magia/portal) en bucle seamless sobre fondo NEGRO. Inserta ADE_LoopedUniformContextOptions + ADE_AnimateDiffLoaderGen1 (motion mm_sd_v15_v2.ckpt) sobre un txt2img base y pone batch_size = num_frames. Los frames son insumo de comfyui_matting_luma_to_alpha + montaje del spritesheet (pipeline, no este builder). Compone comfyui_build_txt2img_workflow (+ comfyui_inject_lora). Pura. class_types verificados contra /object_info."
|
||||
tags: [comfyui, ml, gamedev, gamedev-vfx, gamedev-2d, animatediff, vfx, spritesheet, workflow]
|
||||
uses_functions: [comfyui_build_txt2img_workflow_py_ml, comfyui_inject_lora_py_ml]
|
||||
uses_types: []
|
||||
returns: []
|
||||
returns_optional: false
|
||||
error_type: ""
|
||||
imports: []
|
||||
params:
|
||||
- name: prompt
|
||||
desc: "Prompt del efecto. Deberia incluir 'on pure black background' (insumo de luma-as-alpha). No puede estar vacio."
|
||||
- name: checkpoint
|
||||
desc: "Checkpoint SD1.5 (AnimateDiff SD1.5 cabe en 8GB, SDXL-video no; default 'dreamshaper_8.safetensors'). keyword-only."
|
||||
- name: motion_model
|
||||
desc: "Motion module en models/animatediff_models. Default 'mm_sd_v15_v2.ckpt'. Se asigna al input 'model_name' del loader (OJO: no 'motion_model'). keyword-only."
|
||||
- name: beta_schedule
|
||||
desc: "Schedule del motion. Default 'sqrt_linear (AnimateDiff)'. keyword-only."
|
||||
- name: lora
|
||||
desc: "LoRA de FX opcional (humo/fuego/explosion) en models/loras. None = sin LoRA. keyword-only."
|
||||
- name: lora_strength
|
||||
desc: "Fuerza del LoRA FX sobre model y clip (recomendado 1.1). keyword-only."
|
||||
- name: negative
|
||||
desc: "Prompt negativo. keyword-only."
|
||||
- name: width
|
||||
desc: "Ancho por frame en px (512 cabe en 8GB). keyword-only."
|
||||
- name: height
|
||||
desc: "Alto por frame en px (512). keyword-only."
|
||||
- name: num_frames
|
||||
desc: "Nº de frames del batch (EmptyLatentImage.batch_size). Debe ser >= context_length. keyword-only."
|
||||
- name: context_length
|
||||
desc: "Tamano de la ventana de contexto temporal (<= num_frames). keyword-only."
|
||||
- name: context_stride
|
||||
desc: "Paso de las ventanas de contexto. keyword-only."
|
||||
- name: context_overlap
|
||||
desc: "Solape de las ventanas (alto = transicion mas suave a mas coste). keyword-only."
|
||||
- name: closed_loop
|
||||
desc: "Si True (recomendado) el ultimo frame enlaza con el primero (loop seamless). False = secuencia abierta. keyword-only."
|
||||
- name: steps
|
||||
desc: "Pasos del KSampler (AnimateLCM: 6). keyword-only."
|
||||
- name: cfg
|
||||
desc: "CFG del KSampler (AnimateLCM: 2.0). keyword-only."
|
||||
- name: sampler_name
|
||||
desc: "Sampler del KSampler (AnimateLCM: 'lcm'). keyword-only."
|
||||
- name: scheduler
|
||||
desc: "Scheduler del KSampler. keyword-only."
|
||||
- name: seed
|
||||
desc: "Semilla del KSampler. keyword-only."
|
||||
- name: filename_prefix
|
||||
desc: "Prefijo de los PNG del batch en output/. keyword-only."
|
||||
output: "dict en API format listo para comfyui_submit_workflow: txt2img base (+ LoRA FX opcional) con ADE_LoopedUniformContextOptions + ADE_AnimateDiffLoaderGen1 inyectados, KSampler repuntado al MODEL con motion y EmptyLatentImage.batch_size = num_frames. SaveImage escribe los N PNG del batch."
|
||||
tested: true
|
||||
tests: ["golden: ADE_AnimateDiffLoaderGen1 + ADE_LoopedUniformContextOptions, model_name mm_sd_v15_v2.ckpt, closed_loop True, batch_size=num_frames, KSampler.model <- ADE loader, context_options <- ctx node", "edge closed_loop=False", "edge lora FX: ADE loader toma MODEL de la LoRA", "edge AnimateLCM (sampler lcm/steps 6/cfg 2.0)", "error prompt vacio / num_frames<context_length -> ValueError", "determinismo"]
|
||||
test_file_path: "python/functions/ml/comfyui_build_vfx_spritesheet_workflow_test.py"
|
||||
file_path: "python/functions/ml/comfyui_build_vfx_spritesheet_workflow.py"
|
||||
---
|
||||
|
||||
## Ejemplo
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from ml.comfyui_build_vfx_spritesheet_workflow import comfyui_build_vfx_spritesheet_workflow
|
||||
|
||||
# Frames de un efecto en bucle sobre fondo negro:
|
||||
wf = comfyui_build_vfx_spritesheet_workflow(
|
||||
"burning campfire flame, glowing, vfx, on pure black background",
|
||||
num_frames=16,
|
||||
closed_loop=True,
|
||||
seed=0,
|
||||
)
|
||||
# Pipeline completo (los frames son el insumo):
|
||||
# comfyui_submit_workflow -> comfyui_wait_result -> comfyui_fetch_output_image (N PNG)
|
||||
# -> [por frame] comfyui_matting_luma_to_alpha (luma -> alpha)
|
||||
# -> comfyui_build_grid (spritesheet RGBA)
|
||||
```
|
||||
|
||||
O lanzable directo con: `./fn run comfyui_build_vfx_spritesheet_workflow` (imprime nodos + class_types del ejemplo).
|
||||
|
||||
## Cuando usarla
|
||||
|
||||
Cuando necesites un spritesheet de un efecto 2D animado en bucle (humo, fuego,
|
||||
magia, portal, explosión) para `GPUParticles2D`/`AnimatedSprite2D` de Godot o el
|
||||
VFX Graph de Unity. SIEMPRE genera sobre fondo negro (`on pure black background`
|
||||
en el prompt) porque los frames se recortan con luma-as-alpha
|
||||
(`comfyui_matting_luma_to_alpha`), no con rembg (que rompe los translúcidos).
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Requiere `ComfyUI-AnimateDiff-Evolved` + el motion module `mm_sd_v15_v2.ckpt`**
|
||||
(ya instalados: `ADE_AnimateDiffLoaderGen1`, `ADE_LoopedUniformContextOptions` en
|
||||
/object_info). Sin ellos, HTTP 400 al enviar.
|
||||
- **El input del motion es `model_name`, NO `motion_model`** (el plan inicial lo
|
||||
nombraba mal; verificado contra /object_info).
|
||||
- **`num_frames >= context_length`** o lanza ValueError (la ventana no puede ser
|
||||
mayor que el batch).
|
||||
- **Genera sobre fondo NEGRO**: es deliberado, el negro se vuelve transparente con
|
||||
luma-as-alpha. No usar fondo de color ni escena.
|
||||
- **8GB**: SD1.5 + motion module + 16 frames @ 512² cabe con `--lowvram`.
|
||||
AnimateLCM (`sampler 'lcm', steps 6, cfg 2.0`) baja el pico y el tiempo. Con
|
||||
ControlNet/IPAdapter encima ya no cabe (bajar a ~49f/512×288 o quitar adapters).
|
||||
SDXL-video NO cabe.
|
||||
- Función pura: no valida contra el server.
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Construye el workflow ComfyUI AnimateDiff loop para frames de VFX en API format.
|
||||
|
||||
Genera N frames de un efecto 2D (humo, fuego, magia, portal) en bucle seamless
|
||||
sobre fondo NEGRO (report 0140). Los frames son el insumo de:
|
||||
- comfyui_matting_luma_to_alpha -> luminance-as-alpha (brillante=opaco, negro=transparente)
|
||||
- comfyui_build_grid / spritesheet -> montaje del spritesheet RGBA + JSON sidecar
|
||||
Esos dos pasos van en un pipeline, NO en este workflow (el builder solo arma el grafo).
|
||||
|
||||
Vía recomendada: AnimateDiff SD1.5 con closed_loop (unica que cierra el ciclo
|
||||
frame N -> 0 sin costura). Custom node ComfyUI-AnimateDiff-Evolved + motion module
|
||||
mm_sd_v15_v2.ckpt YA instalados (verificado en /object_info).
|
||||
|
||||
Compone comfyui_build_txt2img_workflow (base) + comfyui_inject_lora (LoRA FX opcional)
|
||||
e inserta la rama AnimateDiff. class_types/inputs verificados contra /object_info:
|
||||
- ADE_AnimateDiffLoaderGen1: model(MODEL), model_name(enum ['mm_sd_v15_v2.ckpt']),
|
||||
beta_schedule(enum, 'sqrt_linear (AnimateDiff)'), context_options(opcional). -> MODEL.
|
||||
OJO: el input es 'model_name', NO 'motion_model' (el plan 0140 lo nombraba mal).
|
||||
- ADE_LoopedUniformContextOptions: context_length, context_stride, context_overlap,
|
||||
closed_loop(BOOLEAN). -> CONTEXT_OPTS.
|
||||
- El KSampler se repunta al MODEL con motion; EmptyLatentImage.batch_size = num_frames.
|
||||
|
||||
Generar sobre fondo negro es deliberado (insumo de luma-as-alpha): el prompt deberia
|
||||
incluir "on pure black background". context_length <= num_frames (ventana <= batch).
|
||||
|
||||
Funcion pura: sin red, sin I/O. Determinista. No valida contra el server (si el
|
||||
node ADE_* o el motion module faltaran, comfyui_submit_workflow daria HTTP 400).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
|
||||
def _is_link(v) -> bool:
|
||||
return isinstance(v, list) and len(v) == 2 and isinstance(v[0], str) and isinstance(v[1], int)
|
||||
|
||||
|
||||
def comfyui_build_vfx_spritesheet_workflow(
|
||||
prompt: str,
|
||||
*,
|
||||
checkpoint: str = "dreamshaper_8.safetensors",
|
||||
motion_model: str = "mm_sd_v15_v2.ckpt",
|
||||
beta_schedule: str = "sqrt_linear (AnimateDiff)",
|
||||
lora: str | None = None,
|
||||
lora_strength: float = 1.1,
|
||||
negative: str = "low quality, watermark, text, background detail",
|
||||
width: int = 512,
|
||||
height: int = 512,
|
||||
num_frames: int = 16,
|
||||
context_length: int = 16,
|
||||
context_stride: int = 1,
|
||||
context_overlap: int = 4,
|
||||
closed_loop: bool = True,
|
||||
steps: int = 20,
|
||||
cfg: float = 7.5,
|
||||
sampler_name: str = "euler",
|
||||
scheduler: str = "normal",
|
||||
seed: int = 0,
|
||||
filename_prefix: str = "vfx_loop",
|
||||
) -> dict:
|
||||
"""Construye el dict (API format) del workflow AnimateDiff loop de VFX.
|
||||
|
||||
Args:
|
||||
prompt: prompt del efecto. Deberia incluir "on pure black background"
|
||||
(insumo de luma-as-alpha). No puede estar vacio.
|
||||
checkpoint: checkpoint SD1.5 (default 'dreamshaper_8.safetensors'; AnimateDiff
|
||||
SD1.5 cabe en 8GB, SDXL-video no). keyword-only.
|
||||
motion_model: motion module en models/animatediff_models
|
||||
(default 'mm_sd_v15_v2.ckpt'). Se asigna al input 'model_name' del loader.
|
||||
beta_schedule: schedule del motion (default 'sqrt_linear (AnimateDiff)').
|
||||
lora: LoRA de FX opcional (humo/fuego/explosion) en models/loras. None = sin LoRA.
|
||||
lora_strength: fuerza del LoRA FX sobre model y clip (recomendado 1.1).
|
||||
negative: prompt negativo.
|
||||
width, height: resolucion por frame (512x512 cabe en 8GB; bajar si OOM).
|
||||
num_frames: nº de frames del batch (EmptyLatentImage.batch_size). Debe ser
|
||||
>= context_length.
|
||||
context_length: tamano de la ventana de contexto temporal (<= num_frames).
|
||||
context_stride, context_overlap: paso y solape de las ventanas (overlap
|
||||
alto = transicion mas suave a mas coste).
|
||||
closed_loop: si True (recomendado) el ultimo frame enlaza con el primero
|
||||
(loop seamless). False = secuencia abierta.
|
||||
steps, cfg, sampler_name, scheduler, seed, filename_prefix: parametros de
|
||||
generacion. Para AnimateLCM usar (sampler 'lcm', steps 6, cfg 2.0).
|
||||
|
||||
Returns:
|
||||
dict en API format listo para comfyui_submit_workflow: txt2img base
|
||||
(+ LoRA FX opcional) con ADE_LoopedUniformContextOptions +
|
||||
ADE_AnimateDiffLoaderGen1 inyectados, el KSampler repuntado al MODEL con
|
||||
motion y EmptyLatentImage.batch_size = num_frames. SaveImage escribe los
|
||||
N PNG del batch a output/.
|
||||
|
||||
Raises:
|
||||
ValueError: si prompt esta vacio o num_frames < context_length (la ventana
|
||||
no puede ser mayor que el batch).
|
||||
"""
|
||||
from ml.comfyui_build_txt2img_workflow import comfyui_build_txt2img_workflow
|
||||
|
||||
if not prompt or not prompt.strip():
|
||||
raise ValueError("comfyui_build_vfx_spritesheet_workflow: 'prompt' no puede estar vacio")
|
||||
if num_frames < context_length:
|
||||
raise ValueError(
|
||||
"comfyui_build_vfx_spritesheet_workflow: num_frames "
|
||||
f"({num_frames}) debe ser >= context_length ({context_length})"
|
||||
)
|
||||
|
||||
wf = comfyui_build_txt2img_workflow(
|
||||
checkpoint,
|
||||
prompt,
|
||||
negative,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
width=width,
|
||||
height=height,
|
||||
seed=seed,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
filename_prefix=filename_prefix,
|
||||
)
|
||||
|
||||
if lora:
|
||||
from ml.comfyui_inject_lora import comfyui_inject_lora
|
||||
|
||||
wf = comfyui_inject_lora(
|
||||
wf, lora, strength_model=lora_strength, strength_clip=lora_strength
|
||||
)
|
||||
|
||||
wf = copy.deepcopy(wf)
|
||||
|
||||
# batch_size del latente = nº de frames del loop.
|
||||
latent_id = next(
|
||||
(nid for nid, n in wf.items() if n.get("class_type") == "EmptyLatentImage"), None
|
||||
)
|
||||
if latent_id is None:
|
||||
raise ValueError(
|
||||
"comfyui_build_vfx_spritesheet_workflow: no se encontro EmptyLatentImage en la base"
|
||||
)
|
||||
wf[latent_id]["inputs"]["batch_size"] = num_frames
|
||||
|
||||
ksampler_id = next(
|
||||
(nid for nid, n in wf.items() if str(n.get("class_type", "")).endswith("KSampler")), None
|
||||
)
|
||||
if ksampler_id is None:
|
||||
raise ValueError("comfyui_build_vfx_spritesheet_workflow: no se encontro KSampler en la base")
|
||||
model_src = list(wf[ksampler_id]["inputs"]["model"])
|
||||
|
||||
numeric = [int(k) for k in wf.keys() if str(k).isdigit()]
|
||||
base = (max(numeric) + 1) if numeric else len(wf) + 1
|
||||
ctx_id, ade_id = str(base), str(base + 1)
|
||||
|
||||
wf[ctx_id] = {
|
||||
"class_type": "ADE_LoopedUniformContextOptions",
|
||||
"inputs": {
|
||||
"context_length": context_length,
|
||||
"context_stride": context_stride,
|
||||
"context_overlap": context_overlap,
|
||||
"closed_loop": closed_loop,
|
||||
},
|
||||
}
|
||||
wf[ade_id] = {
|
||||
"class_type": "ADE_AnimateDiffLoaderGen1",
|
||||
"inputs": {
|
||||
"model": model_src,
|
||||
"model_name": motion_model,
|
||||
"beta_schedule": beta_schedule,
|
||||
"context_options": [ctx_id, 0],
|
||||
},
|
||||
}
|
||||
wf[ksampler_id]["inputs"]["model"] = [ade_id, 0]
|
||||
|
||||
return wf
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
|
||||
wf = comfyui_build_vfx_spritesheet_workflow(
|
||||
"burning campfire flame, glowing, vfx, on pure black background",
|
||||
lora=None,
|
||||
num_frames=16,
|
||||
closed_loop=True,
|
||||
seed=0,
|
||||
)
|
||||
print(json.dumps({"nodes": list(wf), "classes": sorted({n["class_type"] for n in wf.values()})}, indent=2))
|
||||
@@ -0,0 +1,88 @@
|
||||
"""Tests offline de comfyui_build_vfx_spritesheet_workflow (estructura del dict, sin GPU)."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from ml.comfyui_build_vfx_spritesheet_workflow import ( # noqa: E402
|
||||
comfyui_build_vfx_spritesheet_workflow,
|
||||
)
|
||||
|
||||
|
||||
def _by_class(wf, cls):
|
||||
return [n for n in wf.values() if n["class_type"] == cls]
|
||||
|
||||
|
||||
def _id_of(wf, cls):
|
||||
return next(nid for nid, n in wf.items() if n["class_type"] == cls)
|
||||
|
||||
|
||||
def test_golden_animatediff_loop():
|
||||
wf = comfyui_build_vfx_spritesheet_workflow(
|
||||
"burning flame, on pure black background", num_frames=16
|
||||
)
|
||||
assert len(_by_class(wf, "ADE_AnimateDiffLoaderGen1")) == 1
|
||||
assert len(_by_class(wf, "ADE_LoopedUniformContextOptions")) == 1
|
||||
ade = _by_class(wf, "ADE_AnimateDiffLoaderGen1")[0]
|
||||
assert ade["inputs"]["model_name"] == "mm_sd_v15_v2.ckpt"
|
||||
ctx = _by_class(wf, "ADE_LoopedUniformContextOptions")[0]
|
||||
assert ctx["inputs"]["closed_loop"] is True
|
||||
# batch_size = num_frames.
|
||||
latent = _by_class(wf, "EmptyLatentImage")[0]
|
||||
assert latent["inputs"]["batch_size"] == 16
|
||||
# KSampler toma el MODEL con motion.
|
||||
ade_id = _id_of(wf, "ADE_AnimateDiffLoaderGen1")
|
||||
ks = next(n for n in wf.values() if n["class_type"] == "KSampler")
|
||||
assert ks["inputs"]["model"] == [ade_id, 0]
|
||||
# context_options del loader apunta al nodo de contexto.
|
||||
ctx_id = _id_of(wf, "ADE_LoopedUniformContextOptions")
|
||||
assert ade["inputs"]["context_options"] == [ctx_id, 0]
|
||||
|
||||
|
||||
def test_edge_open_loop():
|
||||
wf = comfyui_build_vfx_spritesheet_workflow("smoke", closed_loop=False)
|
||||
ctx = _by_class(wf, "ADE_LoopedUniformContextOptions")[0]
|
||||
assert ctx["inputs"]["closed_loop"] is False
|
||||
|
||||
|
||||
def test_edge_lora_fx():
|
||||
wf = comfyui_build_vfx_spritesheet_workflow(
|
||||
"explosion", lora="detail_tweaker_sd15.safetensors", lora_strength=1.0
|
||||
)
|
||||
loras = _by_class(wf, "LoraLoader")
|
||||
assert len(loras) == 1
|
||||
# El ADE loader toma el MODEL de la LoRA (no del checkpoint crudo).
|
||||
lora_id = _id_of(wf, "LoraLoader")
|
||||
ade = _by_class(wf, "ADE_AnimateDiffLoaderGen1")[0]
|
||||
assert ade["inputs"]["model"] == [lora_id, 0]
|
||||
|
||||
|
||||
def test_edge_animatelcm_params():
|
||||
wf = comfyui_build_vfx_spritesheet_workflow(
|
||||
"portal", sampler_name="lcm", steps=6, cfg=2.0
|
||||
)
|
||||
ks = next(n for n in wf.values() if n["class_type"] == "KSampler")
|
||||
assert ks["inputs"]["sampler_name"] == "lcm"
|
||||
assert ks["inputs"]["steps"] == 6 and ks["inputs"]["cfg"] == 2.0
|
||||
|
||||
|
||||
def test_error_empty_prompt():
|
||||
try:
|
||||
comfyui_build_vfx_spritesheet_workflow("")
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "prompt" in str(e)
|
||||
|
||||
|
||||
def test_error_frames_lt_context():
|
||||
try:
|
||||
comfyui_build_vfx_spritesheet_workflow("flame", num_frames=8, context_length=16)
|
||||
assert False
|
||||
except ValueError as e:
|
||||
assert "num_frames" in str(e)
|
||||
|
||||
|
||||
def test_determinism():
|
||||
a = comfyui_build_vfx_spritesheet_workflow("flame", seed=0)
|
||||
b = comfyui_build_vfx_spritesheet_workflow("flame", seed=0)
|
||||
assert a == b
|
||||
Reference in New Issue
Block a user