feat(ml): cierre del bucle de mejora comfyui-skill (genera→juzga→bump)
Tres funciones nuevas que cierran el lazo skill→generación→juicio→promoción del grupo comfyui-skill (issue 0087): - comfyui_bump_skill_version (impura): promueve una versión nueva SOLO si el score del panel-juez sube (gate objetivo). Snapshot versions/vN.json pre-mutación, deep-merge de recipe_patch, semver↑, línea en growth_log.jsonl. force=True salta el gate. No usa datetime.now(). - comfyui_update_skill_score (impura): media incremental de score_mean/score_n reescribiendo recipe.json in-place (sin snapshot ni growth_log). - comfyui_generate_with_skill_oneshot (pipeline): one-shot load→build→submit→ wait→fetch→judge→score_mean. recipe_patch prueba variantes sin guardar score. Compone 7 funciones del registry. Tests offline: 11 passed (gate, semver, deep-merge, media incremental, errores). Página madre docs/capabilities/comfyui-skill.md: +3 funciones, sección "Bucle de mejora" con diagrama, fronteras de scoring actualizadas. Demo real verificada: skill seed portrait_cinematic_sd15 (SD1.5) generó imagen SFW real, el panel la juzgó, una variante puntuó más alto (4.787 > 4.7276) y el gate promovió v1.0.0→v1.1.0 con el judge_run_id como evidencia. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,93 @@
|
||||
---
|
||||
name: comfyui_generate_with_skill_oneshot
|
||||
kind: pipeline
|
||||
lang: py
|
||||
domain: pipelines
|
||||
version: "1.0.0"
|
||||
purity: impure
|
||||
signature: "def comfyui_generate_with_skill_oneshot(slug: str, subject: str, *, server: str = \"127.0.0.1:8188\", dest: str | None = None, seed: int = 0, judge: bool = True, recipe_patch: dict | None = None, library_dir: str | None = None, wait_timeout: float = 600.0) -> dict"
|
||||
description: "Pipeline skill + subject -> PNG juzgado en una sola llamada. Carga la receta de la skill, la compila para el subject (build_skill_workflow), encola, espera, descarga el PNG y si judge=True lo puntua con el panel comfyui-judge, acumulando el score en la media de la skill (salvo que se pase recipe_patch, que prueba una variante en memoria sin guardar score). Cierra el bucle de generacion->juicio del grupo comfyui-skill (issue 0087). Compone comfyui_load_skill + comfyui_build_skill_workflow + comfyui_submit_workflow + comfyui_wait_result + comfyui_fetch_output_image + comfyui_judge_image + comfyui_update_skill_score. Impuro: HTTP + disco + API Anthropic."
|
||||
tags: [comfyui, comfyui-skill, pipelines, txt2img, judge, launcher]
|
||||
uses_functions: [comfyui_load_skill_py_ml, comfyui_build_skill_workflow_py_ml, comfyui_submit_workflow_py_ml, comfyui_wait_result_py_ml, comfyui_fetch_output_image_py_ml, comfyui_judge_image_py_ml, comfyui_update_skill_score_py_ml]
|
||||
uses_types: []
|
||||
returns: []
|
||||
returns_optional: false
|
||||
error_type: error_py_core
|
||||
imports: [comfyui_load_skill_py_ml, comfyui_build_skill_workflow_py_ml, comfyui_submit_workflow_py_ml, comfyui_wait_result_py_ml, comfyui_fetch_output_image_py_ml, comfyui_judge_image_py_ml, comfyui_update_skill_score_py_ml]
|
||||
params:
|
||||
- name: slug
|
||||
desc: "Slug de la skill a usar (su carpeta en la libreria)."
|
||||
- name: subject
|
||||
desc: "Sujeto concreto que sustituye {subject} en el scaffold del prompt (p.ej. 'a woman with red hair')."
|
||||
- name: server
|
||||
desc: "host:port del servidor ComfyUI (sin esquema). keyword-only."
|
||||
- name: dest
|
||||
desc: "Directorio local donde guardar el PNG (None = cwd). keyword-only."
|
||||
- name: seed
|
||||
desc: "Semilla de generacion. keyword-only."
|
||||
- name: judge
|
||||
desc: "Si True, puntua el PNG con el panel comfyui-judge y acumula el score en la skill (salvo recipe_patch). keyword-only."
|
||||
- name: recipe_patch
|
||||
desc: "Dict de cambios aplicados a la receta EN MEMORIA antes de compilar (deep-merge), para probar una variante sin guardarla; con patch el score NO se acumula. keyword-only."
|
||||
- name: library_dir
|
||||
desc: "Raiz de la libreria. Default ~/ComfyUI/skills_library. keyword-only."
|
||||
- name: wait_timeout
|
||||
desc: "Segundos maximos esperando al servidor. keyword-only."
|
||||
output: "dict {ok, slug, prompt_id, image_path, prompt_resolved, judge, judge_run_id, score_mean, score_n, error}. judge = {verdict, score, votes} o None. judge_run_id correlaciona el juicio (evidencia para comfyui_bump_skill_version). Si falla, ok=False y error explica el paso."
|
||||
tested: false
|
||||
tests: []
|
||||
test_file_path: ""
|
||||
file_path: "python/functions/pipelines/comfyui_generate_with_skill_oneshot.py"
|
||||
---
|
||||
|
||||
# comfyui_generate_with_skill_oneshot
|
||||
|
||||
One-shot del bucle de mejora del grupo [`comfyui-skill`](../../../docs/capabilities/comfyui-skill.md):
|
||||
de una skill guardada a un PNG **ya puntuado** por el panel
|
||||
[`comfyui-judge`](../../../docs/capabilities/comfyui-judge.md), en una llamada. Promoción de la
|
||||
secuencia repetida `load → build → submit → wait → fetch → judge` (issue 0087).
|
||||
|
||||
## Ejemplo
|
||||
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.join(os.environ["HOME"], "fn_registry", "python", "functions"))
|
||||
from pipelines.comfyui_generate_with_skill_oneshot import comfyui_generate_with_skill_oneshot
|
||||
|
||||
# Genera + juzga con la receta canónica (acumula score_mean):
|
||||
res = comfyui_generate_with_skill_oneshot(
|
||||
"portrait_cinematic_sd15", "a woman with red hair",
|
||||
dest="/tmp/comfy_skill", seed=42, judge=True,
|
||||
)
|
||||
print(res["image_path"], res["judge"]["verdict"], res["judge"]["score"], res["score_mean"])
|
||||
|
||||
# Probar una variante sin guardar (no toca score_mean); su judge_run_id sirve de evidencia
|
||||
# para promoverla con comfyui_bump_skill_version si supera a la canónica:
|
||||
var = comfyui_generate_with_skill_oneshot(
|
||||
"portrait_cinematic_sd15", "a woman with red hair",
|
||||
seed=42, recipe_patch={"params": {"steps": 32}},
|
||||
)
|
||||
print(var["judge"]["score"], var["judge_run_id"])
|
||||
```
|
||||
|
||||
## Cuando usarla
|
||||
|
||||
Cuando quieras generar una imagen a partir de una skill ya guardada y saber al instante si es
|
||||
buena (veredicto del panel). Para el **bucle de mejora**: genera la canónica, luego genera una
|
||||
variante con `recipe_patch`, compara `judge.score`, y si la variante gana llama a
|
||||
`comfyui_bump_skill_version` con su `judge_run_id`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Cuesta API**: con `judge=True` invoca el panel multi-juez (incluye 1 llamada LLM-vision a
|
||||
Anthropic). Pásalo a `judge=False` para generar sin puntuar.
|
||||
- **`recipe_patch` no persiste**: aplica la variante solo en memoria y NO acumula su score en la
|
||||
skill (es una prueba). Promueve la variante explícitamente con `comfyui_bump_skill_version`.
|
||||
- **Server vivo requerido**: necesita ComfyUI en `server` con el checkpoint/LoRAs/detector de la
|
||||
receta instalados. `build_skill_workflow` es puro y no valida contra el servidor.
|
||||
- **`wait_timeout` amplio (600s)**: cubre jobs con facedetailer/hires. Imágenes simples retornan en
|
||||
cuanto los outputs están listos.
|
||||
- **Juez caído ≠ fallo fatal**: si el panel entero falla, la imagen ya está en disco; el resultado
|
||||
vuelve con `ok=True`, `judge=None` y `error` anotado.
|
||||
- **OOM en 8GB**: si el server va corto de VRAM, usa una skill SD1.5 (`dreamshaper_8`) a resolución
|
||||
modesta; el error de OOM se propaga en `{ok:False, error}` sin reiniciar el server.
|
||||
@@ -0,0 +1,207 @@
|
||||
"""comfyui_generate_with_skill_oneshot — skill + subject → PNG juzgado en una llamada.
|
||||
|
||||
Cierra el bucle "habilidades de generación cada vez mejores" del grupo `comfyui-skill`:
|
||||
carga una receta de skill, la compila para un *subject* concreto, encola, espera, descarga
|
||||
el PNG y (si ``judge=True``) lo puntúa con el panel `comfyui-judge`, acumulando el score en
|
||||
la media de la skill. Es la promoción a one-shot de la secuencia repetida
|
||||
load → build → submit → wait → fetch → judge (issue 0087).
|
||||
|
||||
Compone funciones del registry:
|
||||
|
||||
comfyui_load_skill_py_ml (lee recipe.json)
|
||||
comfyui_build_skill_workflow_py_ml (receta + subject → workflow API format, PURA)
|
||||
comfyui_submit_workflow_py_ml (POST /prompt)
|
||||
comfyui_wait_result_py_ml (poll /history)
|
||||
comfyui_fetch_output_image_py_ml (GET /view → disco)
|
||||
comfyui_judge_image_py_ml (panel multi-juez: estético + CLIP + LLM-vision)
|
||||
comfyui_update_skill_score_py_ml (media incremental de score_mean/score_n)
|
||||
|
||||
Pipeline impuro: red (HTTP) + escritura en disco + (si juzga) API Anthropic.
|
||||
|
||||
`recipe_patch` permite probar una **variante** en memoria sin guardarla: se aplica a la
|
||||
receta cargada antes de compilar. Cuando se pasa un patch, el score juzgado NO se acumula en
|
||||
``score_mean`` (es una prueba, no la receta canónica): el caller decide si promover la
|
||||
variante con `comfyui_bump_skill_version`. Sin patch, el score sí alimenta la media de la skill.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Importa las funciones del registry (mismo árbol python/functions).
|
||||
_FUNCTIONS_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
if _FUNCTIONS_ROOT not in sys.path:
|
||||
sys.path.insert(0, _FUNCTIONS_ROOT)
|
||||
|
||||
from ml.comfyui_build_skill_workflow import build_skill_workflow, SkillWorkflowError
|
||||
from ml.comfyui_fetch_output_image import comfyui_fetch_output_image
|
||||
from ml.comfyui_judge_image import comfyui_judge_image
|
||||
from ml.comfyui_load_skill import comfyui_load_skill
|
||||
from ml.comfyui_submit_workflow import comfyui_submit_workflow
|
||||
from ml.comfyui_update_skill_score import comfyui_update_skill_score
|
||||
from ml.comfyui_wait_result import comfyui_wait_result
|
||||
|
||||
|
||||
def _deep_merge(base: dict, patch: dict) -> dict:
|
||||
"""Merge recursivo de ``patch`` sobre ``base`` sin mutar los originales."""
|
||||
out = dict(base)
|
||||
for key, val in (patch or {}).items():
|
||||
if isinstance(val, dict) and isinstance(out.get(key), dict):
|
||||
out[key] = _deep_merge(out[key], val)
|
||||
else:
|
||||
out[key] = val
|
||||
return out
|
||||
|
||||
|
||||
def _resolve_prompt(recipe: dict, subject: str) -> str:
|
||||
"""Reconstruye el prompt positivo resuelto (para pasárselo al juez de fidelidad)."""
|
||||
scaffold = recipe.get("prompt_scaffold") or {}
|
||||
positive = str(scaffold.get("positive", "") or "")
|
||||
if "{subject}" in positive:
|
||||
positive = positive.replace("{subject}", subject)
|
||||
elif not positive:
|
||||
positive = subject
|
||||
else:
|
||||
positive = f"{subject}, {positive}"
|
||||
triggers = scaffold.get("trigger_words") or []
|
||||
if triggers:
|
||||
positive = ", ".join(list(triggers) + [positive]) if positive else ", ".join(triggers)
|
||||
return positive
|
||||
|
||||
|
||||
def comfyui_generate_with_skill_oneshot(
|
||||
slug: str,
|
||||
subject: str,
|
||||
*,
|
||||
server: str = "127.0.0.1:8188",
|
||||
dest: str | None = None,
|
||||
seed: int = 0,
|
||||
judge: bool = True,
|
||||
recipe_patch: dict | None = None,
|
||||
library_dir: str | None = None,
|
||||
wait_timeout: float = 600.0,
|
||||
) -> dict:
|
||||
"""Genera (y opcionalmente juzga) una imagen a partir de una skill, end-to-end.
|
||||
|
||||
Args:
|
||||
slug: slug de la skill a usar (su carpeta en la librería).
|
||||
subject: sujeto concreto que sustituye ``{subject}`` en el scaffold del prompt
|
||||
(p.ej. "a woman with red hair").
|
||||
server: host:port del servidor ComfyUI (sin esquema). keyword-only.
|
||||
dest: directorio local donde guardar el PNG (None = cwd). keyword-only.
|
||||
seed: semilla de generación. keyword-only.
|
||||
judge: si True, puntúa el PNG con el panel `comfyui-judge` y acumula el score en la
|
||||
skill (salvo que se pase ``recipe_patch``). keyword-only.
|
||||
recipe_patch: dict de cambios aplicados a la receta EN MEMORIA antes de compilar
|
||||
(deep-merge), para probar una variante sin guardarla. Con patch, el score NO se
|
||||
acumula en ``score_mean``. keyword-only.
|
||||
library_dir: raíz de la librería. Default ``~/ComfyUI/skills_library``. keyword-only.
|
||||
wait_timeout: segundos máximos esperando al servidor. keyword-only.
|
||||
|
||||
Returns:
|
||||
dict ``{ok, slug, prompt_id, image_path, prompt_resolved, judge, judge_run_id,
|
||||
score_mean, score_n, error}``. ``judge`` = ``{verdict, score, votes}`` (o None si
|
||||
``judge=False`` o el panel falla). ``judge_run_id`` correlaciona el juicio (sirve de
|
||||
evidencia para `comfyui_bump_skill_version`). Si falla, ``ok=False`` y ``error``
|
||||
explica en qué paso.
|
||||
"""
|
||||
base = {"ok": False, "slug": slug, "prompt_id": "", "image_path": "",
|
||||
"prompt_resolved": "", "judge": None, "judge_run_id": "",
|
||||
"score_mean": None, "score_n": None, "error": ""}
|
||||
|
||||
# 1. Cargar la receta de la skill.
|
||||
loaded = comfyui_load_skill(slug, library_dir=library_dir)
|
||||
if not loaded.get("ok"):
|
||||
return {**base, "error": f"load_skill falló: {loaded.get('error')}"}
|
||||
recipe = loaded["recipe"]
|
||||
if recipe_patch:
|
||||
recipe = _deep_merge(recipe, recipe_patch)
|
||||
|
||||
prompt_resolved = _resolve_prompt(recipe, subject)
|
||||
|
||||
# 2. Compilar la receta a un workflow (función pura del registry).
|
||||
try:
|
||||
workflow = build_skill_workflow(recipe, subject, seed=seed)
|
||||
except SkillWorkflowError as exc:
|
||||
return {**base, "prompt_resolved": prompt_resolved,
|
||||
"error": f"build_skill_workflow falló: {exc}"}
|
||||
|
||||
# 3. Encolar.
|
||||
try:
|
||||
sub = comfyui_submit_workflow(workflow, server=server)
|
||||
prompt_id = sub["prompt_id"]
|
||||
except (RuntimeError, KeyError) as exc:
|
||||
return {**base, "prompt_resolved": prompt_resolved,
|
||||
"error": f"submit falló: {exc}"}
|
||||
|
||||
# 4. Esperar a que termine.
|
||||
try:
|
||||
outputs = comfyui_wait_result(prompt_id, server=server, timeout=wait_timeout)
|
||||
except (TimeoutError, RuntimeError) as exc:
|
||||
return {**base, "prompt_id": prompt_id, "prompt_resolved": prompt_resolved,
|
||||
"error": f"wait falló: {exc}"}
|
||||
|
||||
# 5. Localizar el primer PNG en los outputs (nodo SaveImage → images).
|
||||
img = None
|
||||
for node_out in outputs.values():
|
||||
images = node_out.get("images") if isinstance(node_out, dict) else None
|
||||
if images:
|
||||
img = images[0]
|
||||
break
|
||||
if img is None:
|
||||
return {**base, "prompt_id": prompt_id, "prompt_resolved": prompt_resolved,
|
||||
"error": f"el workflow no produjo imágenes (outputs={list(outputs)})"}
|
||||
|
||||
# 6. Descargar la imagen a disco.
|
||||
fetched = comfyui_fetch_output_image(
|
||||
img["filename"], subfolder=img.get("subfolder", ""),
|
||||
type_=img.get("type", "output"), server=server, dest_dir=dest or ".",
|
||||
)
|
||||
if not fetched.get("ok"):
|
||||
return {**base, "prompt_id": prompt_id, "prompt_resolved": prompt_resolved,
|
||||
"error": f"fetch de imagen falló: {fetched.get('error')}"}
|
||||
image_path = fetched["path"]
|
||||
|
||||
result = {**base, "ok": True, "prompt_id": prompt_id, "image_path": image_path,
|
||||
"prompt_resolved": prompt_resolved}
|
||||
|
||||
if not judge:
|
||||
return result
|
||||
|
||||
# 7. Juzgar el resultado con el panel multi-juez.
|
||||
judge_run_id = f"judge_{prompt_id}"
|
||||
verdict = comfyui_judge_image(image_path, prompt_resolved, server=server)
|
||||
if not verdict.get("ok"):
|
||||
# El panel falló entero (los 3 jueces caídos): la imagen ya está, no es error fatal.
|
||||
result["judge_run_id"] = judge_run_id
|
||||
result["error"] = f"juez falló (imagen generada igualmente): {verdict.get('error')}"
|
||||
return result
|
||||
|
||||
result["judge"] = {"verdict": verdict["verdict"], "score": verdict["score"],
|
||||
"votes": verdict["votes"]}
|
||||
result["judge_run_id"] = judge_run_id
|
||||
|
||||
# 8. Acumular el score en la media de la skill — solo para la receta canónica
|
||||
# (una variante con recipe_patch es una prueba; el caller decide si la promueve).
|
||||
if not recipe_patch:
|
||||
upd = comfyui_update_skill_score(slug, verdict["score"], library_dir=library_dir)
|
||||
if upd.get("ok"):
|
||||
result["score_mean"] = upd["score_mean"]
|
||||
result["score_n"] = upd["score_n"]
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# Alias con el nombre completo del ID para descubrimiento por convención.
|
||||
generate_with_skill_oneshot = comfyui_generate_with_skill_oneshot
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
|
||||
res = comfyui_generate_with_skill_oneshot(
|
||||
"portrait_cinematic_sd15", "a woman with red hair",
|
||||
dest="/tmp/comfy_skill", seed=42, judge=True,
|
||||
)
|
||||
print(json.dumps(res, indent=2, ensure_ascii=False))
|
||||
Reference in New Issue
Block a user