feat(issues): 0128 agents_and_robots HTTP API + 0129 agents_dashboard C++ ImGui

0128 (backend, blocks 0129): HTTP daemon en cmd/launcher con apikey Bearer auth, SSE pubsub in-memory para status+logs, TLS via Traefik en agents.organic-machine.com, systemd Restart=always. Scope v0.1 lean: list/start/stop/restart/logs SSE. Send-message + config-edit en v0.2. 0129 (frontend): C++ ImGui agents_dashboard en projects/element_agents/apps/. Panels Connection/Agents/Logs/Status. Persistencia local cifrada DPAPI. Depende de 0128. Ambos issues con dod_evidence_schema completo (9 + 9 items). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 21:11:30 +02:00
parent dce725e69f
commit 50a0875c6b
2 changed files with 332 additions and 0 deletions
@@ -0,0 +1,152 @@
+---
+id: "0128"
+title: "agents_and_robots: HTTP API + SSE + apikey + TLS subdominio"
+status: pendiente
+type: feature
+domain:
+  - agents
+  - infra
+  - deploy
+scope: app
+priority: alta
+depends: []
+blocks:
+  - "0129"
+related: []
+created: 2026-05-22
+updated: 2026-05-22
+tags: [agents_and_robots, http, sse, apikey, traefik, systemd]
+dod_evidence_schema:
+  - id: build_ok
+    kind: cmd
+    expected: "cd projects/element_agents/apps/agents_and_robots && go build -tags goolm ./cmd/launcher → exit 0"
+    required: true
+  - id: api_list_authorized
+    kind: cmd
+    expected: "curl -fsS -H 'Authorization: Bearer $AGENTS_API_KEY' https://agents.organic-machine.com/agents devuelve JSON con N>=7 agentes"
+    required: true
+  - id: api_list_unauthorized_401
+    kind: cmd
+    expected: "curl -s -o /dev/null -w '%{http_code}' https://agents.organic-machine.com/agents == 401"
+    required: true
+  - id: api_start_stop_roundtrip
+    kind: cmd
+    expected: "POST /agents/test-bot/stop → POST /agents/test-bot/start: status running confirmado via GET /agents/test-bot tras 2s"
+    required: true
+  - id: sse_logs_streaming
+    kind: cmd
+    expected: "curl -N -H 'Authorization: Bearer $KEY' https://agents.organic-machine.com/sse/agents/assistant-bot/logs entrega >=1 line en 5s con agente activo"
+    required: true
+  - id: sse_status_broadcast
+    kind: cmd
+    expected: "curl -N /sse/status recibe evento {agent_id, old_status, new_status} tras stop/start manual"
+    required: true
+  - id: systemd_active
+    kind: cmd
+    expected: "ssh organic-machine.com 'systemctl is-active agents_and_robots.service' == active"
+    required: true
+  - id: traefik_route
+    kind: url
+    expected: "agents.organic-machine.com resuelve y devuelve cert LE valido (curl -vI muestra subject CN=agents.organic-machine.com)"
+    required: true
+  - id: app_md_drift_fixed
+    kind: cmd
+    expected: "fn doctor services-spec apps/element_agents/apps/agents_and_robots reporta OK (sin drift runtime/systemd)"
+    required: true
+---
+
+# 0128 — agents_and_robots HTTP API + SSE + apikey + TLS
+
+## Contexto
+
+Hoy `agents_and_robots` solo expone control via `agentctl` CLI local (filesystem-based, `shell/process.Manager`). No hay forma remota de gestionar agentes.
+
+Necesitamos backend HTTP seguro para que un frontend local C++ (issue 0129) pueda listar, start/stop/restart agentes, y streamear logs/status en vivo.
+
+## Decision
+
+**Integrar daemon HTTP DENTRO de `cmd/launcher`** como goroutine. Comparte `process.Manager` + acceso a `shell/memory/*.db` + Matrix clients. Un solo proceso, sin drift entre daemon y supervisor.
+
+**Auth:** `Authorization: Bearer <AGENTS_API_KEY>` con `subtle.ConstantTimeCompare`. Clave 32 bytes hex en `.env` (`AGENTS_API_KEY`). 401 sin header o key invalida.
+
+**TLS:** Traefik en VPS organic-machine.com con LE cert auto. Subdominio `agents.organic-machine.com` (DNS A record nuevo → IP del VPS). Ruta Traefik `agents.organic-machine.com → 127.0.0.1:8487`.
+
+**SSE in-memory pubsub.** NATS OFF de momento (1 cliente local, broker = overhead). Documentar TODO en app.md para anadir bus si llega 2do consumidor.
+
+## Scope v0.1 (lean)
+
+| Verbo | Path | Wrap |
+|---|---|---|
+| GET | `/health` | 200 OK sin auth (liveness) |
+| GET | `/agents` | `Scan` + `StatusAll` + `msg_count_24h` (query `shell/memory/*.db`) |
+| GET | `/agents/{id}` | detail + config + `LogTail(200)` |
+| POST | `/agents/{id}/start` | `Manager.Start` |
+| POST | `/agents/{id}/stop` | `Manager.Stop` |
+| POST | `/agents/{id}/restart` | Stop+Start con espera health |
+| GET | `/agents/{id}/logs?n=200` | `LogTail` snapshot |
+
+**SSE:**
+- `GET /sse/status` — broadcast cambios de status (poll cada 2s + diff)
+- `GET /sse/agents/{id}/logs` — tail -f del logfile, emite line events
+
+**Fuera de scope v0.1** (queda v0.2):
+- POST `/agents/{id}/message` (send Matrix message)
+- PUT `/agents/{id}/config` (config edit)
+- SSE messages stream
+
+## Tareas
+
+1. **Nuevo paquete `internal/api`** con server HTTP (stdlib `net/http`, sin gin/echo).
+   - `api.New(mgr *process.Manager, apiKey string, port int) *Server`
+   - `Server.Run(ctx) error` arranca y bloquea hasta ctx done.
+   - Middleware: log + auth + recover.
+2. **Handlers REST** sobre `process.Manager`. Tests unitarios con mock manager.
+3. **SSE pubsub in-memory** (`internal/api/pubsub.go`):
+   - `Bus` con `Subscribe(topic) <-chan event` + `Publish(topic, event)`.
+   - Poller goroutine que llama `StatusAll` cada 2s y publica diffs.
+   - Tail goroutine por logfile (`file_tail_follow` — buscar en registry o crear).
+4. **Integrar en launcher** — `cmd/launcher/main.go` arranca `api.Server` en goroutine si `--api-port > 0`.
+5. **Crear systemd unit** `/etc/systemd/system/agents_and_robots.service` con `Restart=always`, `EnvironmentFile=.env`, `ExecStart=.../bin/launcher --log-level info --api-port 8487`.
+6. **Traefik route + DNS:**
+   - Anadir `agents.organic-machine.com` en DNS (A record).
+   - Anadir config Traefik (label en docker-compose del stack o file provider) apuntando a `127.0.0.1:8487`.
+7. **Fix drift app.md** — `runtime: systemd-system` ahora es verdad. Verificar con `fn doctor services-spec`.
+8. **Tests:**
+   - Go: pkg `internal/api` con httptest.
+   - e2e: `e2e_checks` en `app.md` con curl smoke.
+9. **Deploy:**
+   - `rsync_deploy_bash_infra` o `deploy_server` target nuevo.
+   - Generar `AGENTS_API_KEY` con `openssl rand -hex 32` y escribir `.env` remoto.
+   - `systemctl enable --now agents_and_robots.service`.
+
+## Funciones del registry a usar / proponer
+
+Buscar antes de codear:
+
+- `mcp__registry__fn_search query="tail follow file" lang="go"` — ¿existe `file_tail_follow_go_infra`? Si no, delegar a fn-constructor.
+- `mcp__registry__fn_search query="http auth bearer" lang="go"` — middleware auth.
+- `mcp__registry__fn_search query="sse server" lang="go"` — helper SSE.
+- `systemd_generate_unit_go_infra` + `systemd_install_go_infra` — generar/instalar unit.
+
+## Acceptance
+
+- [ ] `curl -fsS -H 'Authorization: Bearer $KEY' https://agents.organic-machine.com/agents` devuelve lista correcta.
+- [ ] Sin header → 401. Con key invalida → 401. Key valida → 200.
+- [ ] Start/Stop/Restart cambian estado real del proceso (verificable con `ps`).
+- [ ] SSE logs entrega lineas en menos de 1s de aparecer en el archivo.
+- [ ] SSE status broadcast tras stop/start manual.
+- [ ] systemd unit activo y reinicia tras kill -9.
+- [ ] `fn doctor services-spec` reporta OK.
+- [ ] Tests Go pasan.
+
+## DoD humano
+
+- **Donde:** terminal local → `curl https://agents.organic-machine.com/agents`. SSE verificable con `curl -N`.
+- **Latencia:** SSE log lag < 1s. REST list < 200ms.
+- **Onboarding:** README de agents_and_robots actualizado con seccion "HTTP API" + ejemplos curl.
+
+## Riesgos
+
+- DNS propagation puede tardar (configurar con TTL bajo).
+- Traefik en este VPS: verificar si esta gestionado por Coolify o standalone — anadir ruta donde corresponda.
+- `LogTail` actual solo lee snapshot — necesitamos `tail -f` real para SSE. Si no existe en el registry, ronda previa.
@@ -0,0 +1,180 @@
+---
+id: "0129"
+title: "agents_dashboard: C++ ImGui frontend para gestionar agentes Matrix"
+status: pendiente
+type: feature
+domain:
+  - agents
+  - tui
+scope: app
+priority: alta
+depends:
+  - "0128"
+blocks: []
+related: []
+created: 2026-05-22
+updated: 2026-05-22
+tags: [cpp, imgui, agents, dashboard, sse, http-client]
+dod_evidence_schema:
+  - id: scaffold_ok
+    kind: cmd
+    expected: "ls projects/element_agents/apps/agents_dashboard/{app.md,main.cpp,CMakeLists.txt,.git} todos existen"
+    required: true
+  - id: build_windows
+    kind: cmd
+    expected: "cmake --build cpp/build/windows --target agents_dashboard -j → exit 0"
+    required: true
+  - id: appicon_embedded
+    kind: cmd
+    expected: "x86_64-w64-mingw32-objdump -h cpp/build/windows/apps/agents_dashboard/agents_dashboard.exe | grep .rsrc"
+    required: true
+  - id: hub_card_visible
+    kind: screenshot
+    expected: "App Hub muestra tarjeta agents_dashboard con icono robot violeta + description correcta"
+    required: true
+  - id: connection_flow
+    kind: screenshot
+    expected: "Panel Connection con base_url + apikey input, LED verde tras handshake exitoso con backend"
+    required: true
+  - id: agents_table_populated
+    kind: screenshot
+    expected: "Tabla Agents muestra >=7 filas con id/status/uptime/msg_24h + botones accion"
+    required: true
+  - id: start_stop_works
+    kind: screenshot
+    expected: "Click stop sobre test-bot lo apaga (status cambia a stopped en menos de 2s); click start lo reinicia"
+    required: true
+  - id: logs_sse_streaming
+    kind: screenshot
+    expected: "Panel Logs streamea lineas en vivo de assistant-bot (lineas nuevas aparecen sin pulsar refresh)"
+    required: true
+  - id: apikey_encrypted_local
+    kind: cmd
+    expected: "strings cpp/build/windows/apps/agents_dashboard/local_files/agents_dashboard.db | grep -v '<plaintext apikey>' (apikey no aparece en claro)"
+    required: true
+  - id: e2e_self_test
+    kind: cmd
+    expected: "agents_dashboard.exe --self-test exit 0 (verifica subsistemas: GL loader, http client, SSE client, DB local)"
+    required: true
+---
+
+# 0129 — agents_dashboard C++ ImGui frontend
+
+## Contexto
+
+Cuando 0128 cierre, el backend `agents_and_robots` expondra HTTPS API + SSE en `agents.organic-machine.com` con apikey. Necesitamos frontend local C++ ImGui que consuma esa API y permita gestionar agentes sin SSH ni terminal.
+
+## Decision
+
+C++ ImGui app en `projects/element_agents/apps/agents_dashboard/`. Sub-repo Gitea `dataforge/agents_dashboard`. Integrada en App Hub con icono propio.
+
+Scope v0.1 = lo que 0128 expone: list + start/stop/restart + logs SSE. v0.2 anade send-message + config-edit cuando backend los exponga.
+
+## Tareas
+
+### 1. Scaffold (REGLA: scaffolder canonico, NUNCA a mano)
+
+```bash
+./fn run init_cpp_app agents_dashboard \
+  --project element_agents \
+  --desc "Frontend C++ ImGui para gestionar agentes Matrix de agents_and_robots via HTTPS+apikey, SSE para logs/status en vivo"
+```
+
+Tras scaffold:
+- `git init` dentro de `projects/element_agents/apps/agents_dashboard/` (regla `apps_subrepo.md`).
+- Trio `app.md`: `description` + `icon.phosphor: "robot"` + `icon.accent: "#8b5cf6"`.
+- `./fn run regenerate_app_icons agents_dashboard`.
+- `./fn run refresh_app_hub` para que aparezca en el hub.
+
+### 2. Funciones del registry — buscar primero
+
+| Necesidad | Buscar en registry | Si falta |
+|---|---|---|
+| HTTP client C++ (sync GET/POST + Bearer + JSON body) | `mcp__registry__fn_search query="http client" lang="cpp"` | Delegar `fn-constructor`: `http_client_cpp_infra` con libcurl |
+| SSE client C++ | `sse_client_cpp_core` (FRESH 7d) | ✓ reuso directo |
+| JSON parse/serialize C++ | buscar nlohmann wrapper | Si falta, vendoring `cpp/vendor/json.hpp` (single-header) |
+| Data table | `data_table_cpp_viz` | ✓ reuso |
+| Secret store local (DPAPI Windows) | buscar | Si falta: `secret_store_cpp_infra` (DPAPI wrap, base64 fallback Linux) |
+| Ring buffer C++ | buscar | Si falta: `ring_buffer_cpp_core` |
+
+Delegacion paralela: **una sola llamada Agent con N tool_use blocks paralelos** para las que falten (regla `delegation.md`).
+
+### 3. Paneles UI
+
+- **Connection** — `base_url` input + apikey input (mask) + boton "Test" → GET /health + GET /agents. LED estado SSE (gris/amarillo/verde/rojo). Save credentials en `local_files/agents_dashboard.db` encriptadas via secret_store.
+- **Agents** — `data_table_cpp_viz` con cols:
+  - id (texto)
+  - status (icono colored: running=green, stopped=gray, crashed=red)
+  - uptime (humanized)
+  - msg_24h (numero)
+  - actions (botones `▶ ⏹ ↻` por fila)
+  - Filtro por substring + sort por col.
+- **Logs** — selector agente (combo) + tail viewport (ring buffer 5000 lineas) + autoscroll toggle + boton "Pause". Stream via `/sse/agents/{id}/logs`.
+- **Status feed** — panel collapsible con eventos del `/sse/status` (timeline reciente).
+
+### 4. Persistencia local
+
+- `<exe_dir>/local_files/agents_dashboard.db` (SQLite via funciones del registry o sqlite3 directo).
+- Schema migraciones en `migrations/001_init.sql`:
+  ```sql
+  CREATE TABLE connections (
+      id INTEGER PRIMARY KEY,
+      name TEXT NOT NULL,
+      base_url TEXT NOT NULL,
+      apikey_encrypted BLOB NOT NULL,
+      last_used DATETIME DEFAULT CURRENT_TIMESTAMP
+  );
+  CREATE TABLE app_state (
+      key TEXT PRIMARY KEY,
+      value TEXT
+  );
+  ```
+- `app_settings.ini` via `fn_ui::settings_*` (theme, layout).
+- apikey cifrada con DPAPI Windows (clave nunca abandona la maquina).
+
+### 5. Build + deploy local
+
+- CMake target `agents_dashboard` en `cpp/CMakeLists.txt` (auto via scaffolder).
+- Build Windows: `cmake --build cpp/build/windows --target agents_dashboard -j`.
+- Deploy local: `./fn run redeploy_cpp_app_windows agents_dashboard projects/element_agents/apps/agents_dashboard --build`.
+- Icono via windres (gestionado por `add_imgui_app`).
+
+### 6. Tests + e2e_checks
+
+```yaml
+e2e_checks:
+  - id: build
+    cmd: "cmake --build cpp/build/windows --target agents_dashboard -j"
+    timeout_s: 180
+  - id: self_test
+    cmd: "./cpp/build/windows/apps/agents_dashboard/agents_dashboard.exe --self-test"
+    timeout_s: 30
+  - id: pytest_mock
+    cmd: "cd projects/element_agents/apps/agents_dashboard/tests && python3 -m pytest -x -q"
+    timeout_s: 60
+```
+
+Mock server pytest emula 0128 (list/start/stop + SSE) y verifica que la app C++ conecta + popula tabla + start/stop funciona en headless con `--capture` mode.
+
+## Acceptance
+
+- [ ] App arranca, muestra Connection panel.
+- [ ] Tras meter apikey valida → tabla Agents populated con datos reales de VPS.
+- [ ] Stop/Start desde UI cambia estado real del agente en VPS.
+- [ ] Logs streamea lineas nuevas sin polling.
+- [ ] Cerrar y reabrir app → credentials persisten (cifradas).
+- [ ] Sin red / apikey invalida → error visible, app no crashea.
+- [ ] `--self-test` exit 0.
+- [ ] Visible en App Hub con icono + description correctos.
+
+## DoD humano
+
+- **Donde:** Windows Desktop → App Hub → Click "agents_dashboard".
+- **Latencia:** logs SSE < 1s lag. Lista agents < 200ms tras handshake.
+- **Onboarding:** First-run wizard pide base_url + apikey; tooltip explica donde obtener la key (gestor de secretos del VPS).
+
+## Riesgos
+
+- libcurl en Windows mingw-w64: cross-compile setup. Si `http_client_cpp_infra` no existe, dedicar tiempo al wrapper antes de UI.
+- DPAPI solo Windows: fallback Linux puede ser texto plano con permisos 0600 + warning visible en UI.
+- SSE reconnect logic: backoff exponencial + indicador de estado claro.