142 lines
5.6 KiB
Markdown
142 lines
5.6 KiB
Markdown
---
|
|
id: "0071b"
|
|
title: "Extraer `jobs_queue_panel` a cpp/functions/core/ (sub-issue de 0071)"
|
|
status: pendiente
|
|
type: feature
|
|
domain:
|
|
- registry-quality
|
|
scope: registry-only
|
|
priority: media
|
|
depends:
|
|
- "0071f"
|
|
blocks: []
|
|
related: []
|
|
created: 2026-05-10
|
|
updated: 2026-05-17
|
|
tags: []
|
|
---
|
|
|
|
## Contexto
|
|
|
|
Sub-issue derivado de 0071. Patron jobs queue (cola de trabajos asincronos con progreso, stdout/stderr capture, cancel) duplicado entre apps:
|
|
|
|
| Consumidor | Archivos | LoC | Notas |
|
|
|---|---|---|---|
|
|
| `projects/osint_graph/apps/graph_explorer/jobs.cpp` + `views_jobs.cpp` | 2 | 1366 + ~300 | Implementacion completa con persistencia DuckDB, agentes, retries |
|
|
| `projects/osint_graph/apps/odr_console/` (panel Jobs) | 1 | TBD | Por auditar — issue 0066 lo introduce |
|
|
|
|
Consumidor potencial inmediato: `navegator_dashboard` para scraping jobs (un crawl = un job).
|
|
|
|
Issue **0065** ya existe y propone exactamente esto: "Extraer jobs system de graph_explorer al registry". Este 0071b lo ABSORBE — es la misma extraccion, ahora encadrada bajo el paraguas 0071.
|
|
|
|
## Pre-requisito: auditoria de duplicacion
|
|
|
|
**ANTES** de extraer, ejecutar:
|
|
|
|
```bash
|
|
diff -u <(grep -c "^" projects/osint_graph/apps/graph_explorer/jobs.cpp) \
|
|
<(grep -c "^" projects/osint_graph/apps/odr_console/<jobs_file>.cpp)
|
|
```
|
|
|
|
Y comparar manualmente:
|
|
- Schema de la tabla `jobs` en cada SQLite.
|
|
- Estados (queued|running|done|failed|canceled).
|
|
- API publica (submit, cancel, list, on_complete callback).
|
|
|
|
Si la divergencia es < 30%, extraer. Si es > 50%, abrir issue de unificacion previa antes de extraer.
|
|
|
|
## Objetivo
|
|
|
|
Funcion `jobs_queue_panel` en `cpp/functions/core/` con:
|
|
- Submit jobs con `argv` + `cwd` + `env` + `tags`.
|
|
- Persistencia automatica en SQLite local (cada app pasa su path).
|
|
- UI ImGui: lista filtrable por estado, barra progreso, click → ver stdout/stderr.
|
|
- Cancel/retry.
|
|
- Worker pool configurable (N workers concurrentes).
|
|
- Callback global `on_complete(job_id, exit_code)` para que la app reaccione.
|
|
|
|
## Dependencia previa
|
|
|
|
**0071f** (`subprocess_streamer`) DEBE estar mergeado. El worker pool del jobs_panel usa subprocess_streamer para spawn + capture.
|
|
|
|
## API propuesta
|
|
|
|
```cpp
|
|
namespace fn_core {
|
|
|
|
struct Job {
|
|
std::string id; // generado o pasado por la app
|
|
std::string title; // mostrado en la UI
|
|
std::vector<std::string> argv; // comando a ejecutar
|
|
std::string cwd;
|
|
std::map<std::string, std::string> env;
|
|
std::vector<std::string> tags; // filtros UI
|
|
int priority = 0; // mayor = antes
|
|
};
|
|
|
|
struct JobStatus {
|
|
enum State { Queued, Running, Done, Failed, Canceled } state;
|
|
int exit_code = -1;
|
|
std::string stdout_buf; // capturado completo
|
|
std::string stderr_buf;
|
|
int64_t queued_at_ms = 0;
|
|
int64_t started_at_ms = 0;
|
|
int64_t finished_at_ms = 0;
|
|
float progress = 0.0f; // 0..1, opcional (parseado de stdout patrones)
|
|
};
|
|
|
|
struct JobsQueueConfig {
|
|
std::string db_path; // SQLite con tabla jobs (auto-migrada)
|
|
int max_workers = 2;
|
|
std::function<void(const std::string& job_id, int exit_code)> on_complete;
|
|
// Parser opcional para extraer progreso de stdout (ej. regex "(\\d+)%").
|
|
std::function<float(const std::string& line)> progress_parser;
|
|
};
|
|
|
|
struct JobsHandle;
|
|
JobsHandle* jobs_create(const JobsQueueConfig& cfg);
|
|
std::string jobs_submit(JobsHandle*, const Job& j); // returns job_id
|
|
void jobs_cancel(JobsHandle*, const std::string& job_id);
|
|
void jobs_retry(JobsHandle*, const std::string& job_id);
|
|
JobStatus jobs_status(JobsHandle*, const std::string& job_id);
|
|
std::vector<std::pair<Job, JobStatus>> jobs_list(JobsHandle*, JobStatus::State filter);
|
|
void jobs_render(JobsHandle*, bool* p_open);
|
|
void jobs_destroy(JobsHandle*);
|
|
|
|
} // namespace fn_core
|
|
```
|
|
|
|
## Migracion
|
|
|
|
1. **Auditoria** (ver pre-requisito) — documentar diff entre graph_explorer y odr_console.
|
|
2. **Schema canonico** — tabla `jobs` en `cpp/functions/core/jobs_queue_panel/migrations/001_init.sql`.
|
|
3. **Crear** `jobs_queue_panel.{h,cpp,md}` + tests.
|
|
4. **Tests** en primitives_gallery:
|
|
- submit `echo hello` → estado Done, exit 0, stdout "hello".
|
|
- submit `false` → estado Failed, exit 1.
|
|
- submit + cancel mientras Queued → estado Canceled.
|
|
- 2 workers, submit 4 jobs → max 2 Running simultaneos.
|
|
5. **Migrar `graph_explorer`** — `jobs.cpp` reducido a wrapper de ~50 LoC. Migracion de DB existente: copiar filas a la nueva schema si difiere.
|
|
6. **Migrar `odr_console`** despues.
|
|
7. **Cerrar issue 0065** apuntando a este 0071b como reemplazo.
|
|
|
|
## Definicion de hecho
|
|
|
|
- `jobs_queue_panel` registrado, tests pasan.
|
|
- graph_explorer/jobs.cpp reducido a ~50 LoC (wrapper + config).
|
|
- odr_console migrado.
|
|
- Issue 0065 cerrado (movido a `completed/` con nota "absorbido por 0071b").
|
|
- DB de jobs existente migrada sin perdida de historico.
|
|
|
|
## Riesgos
|
|
|
|
- **Migracion de schema** de `jobs.duckdb` (si lo es) o `operations.db` puede ser destructiva. Mitigacion: backup de `apps/<app>/operations.db` antes de migrar; script de migracion idempotente con `IF NOT EXISTS`.
|
|
- **Hilos** — el worker pool corre en background. Cuidado con teardown: `jobs_destroy` debe joinear todos los threads o detach + cancel. Tests con valgrind/asan recomendados.
|
|
- **odr_console aun no MVP-listo** (issue 0066 pending). Si su jobs no es estable, esperar.
|
|
|
|
## Anti-patrones
|
|
|
|
- Generalizacion de "task scheduler" (cron, retries exponenciales, prioridades complejas). KISS — FIFO + priority int + cancel basta.
|
|
- Acoplar al esquema graph (`entities`, `relations`). El panel solo conoce comandos shell.
|
|
- Workers via `std::async` — usar `std::thread` + `std::condition_variable` simple.
|