diff --git a/.claude/rules/INDEX.md b/.claude/rules/INDEX.md index bde8f0ed..3182630a 100644 --- a/.claude/rules/INDEX.md +++ b/.claude/rules/INDEX.md @@ -17,3 +17,4 @@ Reglas operativas del proyecto. Cada archivo es una regla independiente. | 11 | [sources.md](sources.md) | Extraccion de funciones desde repos externos | | 12 | [notebook_collaboration.md](notebook_collaboration.md) | Colaboración en notebooks Jupyter via funciones del registry | | 13 | [frontend_theming.md](frontend_theming.md) | Componentes propios y sistema de temas en frontends | +| 14 | [deploy.md](deploy.md) | Deploy de apps a VPS remotos via SSH + systemd + rsync | diff --git a/.claude/rules/deploy.md b/.claude/rules/deploy.md new file mode 100644 index 00000000..55bb1827 --- /dev/null +++ b/.claude/rules/deploy.md @@ -0,0 +1,134 @@ +## Deploy de apps a VPS remotos + +### Arquitectura + +El sistema de deploy usa SSH + systemd + rsync. No Docker, no Kubernetes. + +- **Conexiones SSH** → `~/.ssh/config` (alias, IP, user, key). Ya hay funciones CRUD: `ssh_config_read`, `ssh_config_find`, `ssh_config_parse`. +- **Config de deploy** → `apps/deploy_server/operations.db` tabla `deploy_targets` (app, host, remote_dir, build_cmd, port, health_path, env). +- **Logs de deploy** → misma BD, tabla `deploy_logs` (app, host, status, trigger, duration_ms, error). + +### App: `deploy_server` (`apps/deploy_server/`) + +CLI + servidor HTTP. Binario: `deploy_server`. Build: `CGO_ENABLED=1 go build -o deploy_server .` + +```bash +cd apps/deploy_server + +# Gestionar targets +./deploy_server target add --app --host --port --health /path --build "comando" [--user deploy] [--env '{"K":"V"}'] +./deploy_server target list +./deploy_server target remove + +# Setup inicial (primera vez, crea dirs + systemd unit) +./deploy_server setup --host + +# Deploy continuo (build local → rsync → restart → health check) +./deploy_server deploy [--host ] + +# Estado del servicio remoto +./deploy_server status +./deploy_server status --all + +# Servidor webhook (auto-deploy en cada push a Gitea) +./deploy_server serve --port 9090 +``` + +### Funciones del registry involucradas + +| Función | Qué hace | Purity | +|---|---|---| +| `rsync_deploy_bash_infra` | rsync local→remoto con exclusiones | impure | +| `systemd_generate_unit_go_infra` | Genera texto .service | **pure** | +| `systemd_install_go_infra` | Sube unit + daemon-reload + enable + start | impure | +| `systemd_restart_go_infra` | Reinicia servicio remoto | impure | +| `systemd_status_go_infra` | Estado + logs de servicio remoto | impure | +| `vps_setup_app_go_infra` | Crea dirs + usuario en VPS | impure | +| `gitea_create_webhook_bash_infra` | Crea webhook push en Gitea | impure | +| `setup_vps_app_go_infra` | Pipeline: setup completo primera vez | impure | +| `deploy_app_remote_go_infra` | Pipeline: deploy continuo | impure | + +Tipo: `DeployConfig_go_infra` — struct con toda la config de deploy. + +### Workflow para un agente + +Cuando el usuario diga **"sube esta app a este VPS"** o **"deploya X en Y"**: + +#### 1. Verificar que el host SSH existe + +```bash +grep "^Host " ~/.ssh/config +# Si no existe el alias, añadirlo: +# Usar ssh_config_add_entry o editar ~/.ssh/config directamente +``` + +#### 2. Verificar conectividad + +```bash +ssh -o BatchMode=yes -o ConnectTimeout=5 true +``` + +#### 3. Registrar el target en deploy_server + +```bash +cd apps/deploy_server +# Build deploy_server si no existe el binario +CGO_ENABLED=1 go build -o deploy_server . + +./deploy_server target add \ + --app \ + --host \ + --port \ + --health \ + --build "CGO_ENABLED=0 GOOS=linux go build -o ." \ + --user deploy +``` + +#### 4. Setup inicial + +```bash +./deploy_server setup --host +``` + +Esto crea dirs en `/opt/apps//`, sube el código, genera el unit systemd e instala el servicio. + +#### 5. Deploys posteriores + +```bash +./deploy_server deploy +``` + +Build local → rsync → restart systemd → health check. + +#### 6. Auto-deploy con webhook (opcional) + +```bash +# Lanzar servidor +./deploy_server serve --port 9090 + +# Crear webhook en Gitea +source bash/functions/infra/gitea_create_webhook.sh +gitea_create_webhook "" "" "http://:9090/webhook/push" "" +``` + +### Requisitos en el VPS + +- SSH accesible con key auth (configurado en `~/.ssh/config` local) +- El usuario SSH debe tener **sudo sin password** para: `systemctl`, `mv` a `/etc/systemd/system/`, `mkdir` en `/opt/apps/`, `useradd`, `chown` +- `rsync` instalado en el VPS +- Puerto del servicio abierto en el firewall del VPS + +### Builds por lenguaje + +| Lenguaje | Build command típico | +|---|---| +| Go | `CGO_ENABLED=0 GOOS=linux go build -o .` | +| Go + SQLite | `CGO_ENABLED=1 GOOS=linux go build -tags fts5 -o .` | +| Python | No build — rsync sube los .py, systemd ejecuta `python3 main.py` | +| Bash | No build — rsync sube los .sh, systemd ejecuta `bash main.sh` | + +Para Go con CGO (SQLite), el VPS debe tener `gcc` y `libc-dev`, o cross-compilar con `CGO_ENABLED=0` si la app no usa SQLite. + +### Exclusiones de rsync + +El deploy excluye automáticamente: `.git`, `operations.db*`, `*.exe`, `node_modules`, `.venv`, `__pycache__`, `build/`, `*.db-shm`, `*.db-wal`, `registry.db`. diff --git a/.gitignore b/.gitignore index 755e00c7..8a8bcc66 100644 --- a/.gitignore +++ b/.gitignore @@ -37,6 +37,13 @@ python/.venv/ apps/*/ analysis/*/ +# Projects (each is its own git repo, only project.md templates are versioned) +projects/*/ + +# Vaults — data stores (symlinks, dirs, files); only vault.yaml manifest is versioned +vaults/*/ +!vaults/vault.yaml + # Node / pnpm **/node_modules/ diff --git a/bash/functions/infra/gitea_create_webhook.md b/bash/functions/infra/gitea_create_webhook.md new file mode 100644 index 00000000..f519e42e --- /dev/null +++ b/bash/functions/infra/gitea_create_webhook.md @@ -0,0 +1,50 @@ +--- +name: gitea_create_webhook +kind: function +lang: bash +domain: infra +version: "1.0.0" +purity: impure +signature: "gitea_create_webhook(owner: string, repo: string, target_url: string, secret?: string) -> json" +description: "Crea un webhook de push en un repositorio Gitea. El webhook notifica a target_url en cada push." +tags: [gitea, webhook, push, deploy, ci, infra] +uses_functions: [] +uses_types: [] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [] +params: + - name: owner + desc: "usuario u organización propietaria del repositorio" + - name: repo + desc: "nombre del repositorio" + - name: target_url + desc: "URL que recibirá el POST del webhook en cada push" + - name: secret + desc: "secreto compartido para firmar el payload (opcional)" +output: "JSON con webhook_id, owner, repo, target_url" +tested: false +tests: [] +test_file_path: "" +file_path: "bash/functions/infra/gitea_create_webhook.sh" +--- + +## Ejemplo + +```bash +source bash/functions/infra/gitea_create_webhook.sh + +export GITEA_URL="https://git.example.com" +export GITEA_TOKEN="$(pass agentes/dataforge-token)" + +# Crear webhook para auto-deploy +gitea_create_webhook "myorg" "dag_engine" "http://vps:9090/webhook/push" "mi_secreto" +# {"webhook_id":42,"owner":"myorg","repo":"dag_engine","target_url":"http://vps:9090/webhook/push"} +``` + +## Notas + +- Requiere `GITEA_URL` y `GITEA_TOKEN` como variables de entorno. +- Solo escucha eventos `push`. Para otros eventos, modificar el array `events` en el payload. +- Si el webhook ya existe para la misma URL, Gitea crea uno duplicado (no es idempotente). diff --git a/bash/functions/infra/gitea_create_webhook.sh b/bash/functions/infra/gitea_create_webhook.sh new file mode 100644 index 00000000..89abd53f --- /dev/null +++ b/bash/functions/infra/gitea_create_webhook.sh @@ -0,0 +1,60 @@ +#!/usr/bin/env bash +# gitea_create_webhook — Crea un webhook de push en un repositorio Gitea +set -euo pipefail + +gitea_create_webhook() { + local owner="$1" + local repo="$2" + local target_url="$3" + local secret="${4:-}" + + if [[ -z "$owner" || -z "$repo" || -z "$target_url" ]]; then + echo "usage: gitea_create_webhook [secret]" >&2 + return 1 + fi + + local gitea_url="${GITEA_URL:?GITEA_URL no seteada}" + local gitea_token="${GITEA_TOKEN:?GITEA_TOKEN no seteada}" + + # Payload JSON para el webhook + local payload + payload=$(cat <&2 + return 1 + fi +} + +if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then + gitea_create_webhook "$@" +fi diff --git a/bash/functions/infra/rsync_deploy.md b/bash/functions/infra/rsync_deploy.md new file mode 100644 index 00000000..6d88fa46 --- /dev/null +++ b/bash/functions/infra/rsync_deploy.md @@ -0,0 +1,52 @@ +--- +name: rsync_deploy +kind: function +lang: bash +domain: infra +version: "1.0.0" +purity: impure +signature: "rsync_deploy(local_dir: string, ssh_alias: string, remote_dir: string) -> json" +description: "Sincroniza un directorio local a un host remoto via rsync+SSH. Excluye archivos de desarrollo y bases de datos locales. Crea el directorio remoto si no existe." +tags: [rsync, deploy, sync, ssh, remote, infra] +uses_functions: [] +uses_types: [] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [] +params: + - name: local_dir + desc: "ruta al directorio local a sincronizar (ej: apps/dag_engine/)" + - name: ssh_alias + desc: "alias SSH del host destino definido en ~/.ssh/config (ej: myserver)" + - name: remote_dir + desc: "ruta absoluta del directorio destino en el host remoto (ej: /opt/apps/dag_engine)" +output: "JSON con files_transferred (int), total_size (string), ssh_alias (string), remote_dir (string)" +tested: false +tests: [] +test_file_path: "" +file_path: "bash/functions/infra/rsync_deploy.sh" +--- + +## Ejemplo + +```bash +source bash/functions/infra/rsync_deploy.sh + +# Deploy de una app al servidor de producción +result=$(rsync_deploy "apps/dag_engine/" "prod-server" "/opt/apps/dag_engine") +echo "$result" +# {"files_transferred": 12, "total_size": "1.23 MB", "ssh_alias": "prod-server", "remote_dir": "/opt/apps/dag_engine"} + +# Deploy con ruta absoluta local +rsync_deploy "/home/lucas/fn_registry/apps/myapp/" "myserver" "/opt/myapp" +``` + +## Notas + +- Usa `rsync -avz --delete`: archivos borrados localmente se borran también en el remoto. +- Antes del rsync crea el directorio remoto con `ssh mkdir -p` para evitar errores si no existe. +- Archivos excluidos: `.git`, `operations.db*`, `*.exe`, `node_modules`, `.venv`, `__pycache__`, `build/`, `*.db-shm`, `*.db-wal`. +- El JSON de salida va a stdout; los mensajes de progreso y errores van a stderr. +- Exit code 1 si rsync falla o si el directorio local no existe. +- El `ssh_alias` se resuelve con la configuración de `~/.ssh/config`, incluyendo host, user, identityfile y puerto. diff --git a/bash/functions/infra/rsync_deploy.sh b/bash/functions/infra/rsync_deploy.sh new file mode 100644 index 00000000..70bbe40d --- /dev/null +++ b/bash/functions/infra/rsync_deploy.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# rsync_deploy — Sincroniza un directorio local a un host remoto via rsync+SSH +set -euo pipefail + +rsync_deploy() { + local local_dir="$1" + local ssh_alias="$2" + local remote_dir="$3" + + if [[ -z "$local_dir" || -z "$ssh_alias" || -z "$remote_dir" ]]; then + echo "rsync_deploy: se requieren local_dir, ssh_alias y remote_dir" >&2 + return 1 + fi + + if [[ ! -d "$local_dir" ]]; then + echo "rsync_deploy: directorio local '$local_dir' no existe" >&2 + return 1 + fi + + # Crear directorio remoto si no existe + echo "rsync_deploy: verificando directorio remoto '$remote_dir' en '$ssh_alias'..." >&2 + if ! ssh "$ssh_alias" "mkdir -p '$remote_dir'" 2>&1; then + echo "rsync_deploy: no se pudo crear el directorio remoto '$remote_dir' en '$ssh_alias'" >&2 + return 1 + fi + + # Ejecutar rsync y capturar salida para parsear estadísticas + local rsync_output + rsync_output=$(rsync -avz --delete \ + --exclude='.git' \ + --exclude='operations.db*' \ + --exclude='*.exe' \ + --exclude='node_modules' \ + --exclude='.venv' \ + --exclude='__pycache__' \ + --exclude='build/' \ + --exclude='*.db-shm' \ + --exclude='*.db-wal' \ + -e ssh \ + "$local_dir" \ + "${ssh_alias}:${remote_dir}" 2>&1) || { + echo "rsync_deploy: rsync falló al sincronizar '$local_dir' → '${ssh_alias}:${remote_dir}'" >&2 + echo "$rsync_output" >&2 + return 1 + } + + echo "$rsync_output" >&2 + + # Parsear número de archivos transferidos + local files_transferred + files_transferred=$(echo "$rsync_output" | grep -oP 'Number of regular files transferred: \K[0-9,]+' | tr -d ',' || echo "0") + if [[ -z "$files_transferred" ]]; then + # Intentar formato alternativo de rsync + files_transferred=$(echo "$rsync_output" | grep -oP 'Number of files transferred: \K[0-9,]+' | tr -d ',' || echo "0") + fi + if [[ -z "$files_transferred" ]]; then + files_transferred="0" + fi + + # Parsear tamaño total transferido + local total_size + total_size=$(echo "$rsync_output" | grep -oP 'Total transferred file size: \K[0-9,.]+ \w+' || echo "0 bytes") + if [[ -z "$total_size" ]]; then + total_size="0 bytes" + fi + + # Emitir JSON a stdout + printf '{"files_transferred": %s, "total_size": "%s", "ssh_alias": "%s", "remote_dir": "%s"}\n' \ + "$files_transferred" \ + "$total_size" \ + "$ssh_alias" \ + "$remote_dir" +} + +if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then + rsync_deploy "$@" +fi diff --git a/cmd/fn/main.go b/cmd/fn/main.go index 9ad95e89..84a2db7d 100644 --- a/cmd/fn/main.go +++ b/cmd/fn/main.go @@ -37,6 +37,8 @@ func main() { cmdRun(os.Args[2:]) case "check": cmdCheck(os.Args[2:]) + case "project": + cmdProject(os.Args[2:]) case "app": cmdApp(os.Args[2:]) case "analysis": @@ -63,6 +65,7 @@ Usage: fn check params Lista funciones sin params_schema fn ops Gestiona operations.db (fn ops help) fn proposal Gestiona proposals + fn project Gestiona proyectos fn app Gestiona apps externas (Gitea) fn analysis Gestiona analyses externas (Gitea)`) } @@ -126,7 +129,8 @@ func cmdIndex() { } } - fmt.Printf("Indexed %d functions, %d types, %d apps, %d analysis, %d unit_tests\n", result.Functions, result.Types, result.Apps, result.Analysis, result.UnitTests) + fmt.Printf("Indexed %d functions, %d types, %d apps, %d analysis, %d projects, %d vaults, %d unit_tests\n", + result.Functions, result.Types, result.Apps, result.Analysis, result.Projects, result.Vaults, result.UnitTests) for _, e := range result.ValidationErrors { fmt.Fprintf(os.Stderr, " INVALID: %s\n", e) } @@ -190,7 +194,13 @@ func cmdSearch(args []string) { os.Exit(1) } - if len(fns) == 0 && len(types) == 0 && len(apps) == 0 && len(analyses) == 0 { + projects, err := db.SearchProjects(query) + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + + if len(fns) == 0 && len(types) == 0 && len(apps) == 0 && len(analyses) == 0 && len(projects) == 0 { fmt.Println("No results.") return } @@ -233,6 +243,16 @@ func cmdSearch(args []string) { fmt.Fprintf(w, "analysis\t%s\t%s\t%s\n", a.ID, a.Lang, desc) } } + if len(projects) > 0 { + if len(fns) > 0 || len(types) > 0 || len(apps) > 0 || len(analyses) > 0 { + fmt.Fprintln(w) + } + fmt.Fprintln(w, "PROJECT\tID\tDESCRIPTION") + for _, p := range projects { + desc := truncate(p.Description, 60) + fmt.Fprintf(w, "project\t%s\t%s\n", p.ID, desc) + } + } w.Flush() } @@ -317,7 +337,22 @@ func cmdList(args []string) { fmt.Fprintf(w, "analysis\t%s\t%s\t%s\n", a.ID, a.Lang, a.Domain) } } - if len(fns) == 0 && len(types) == 0 && len(apps) == 0 && len(analyses) == 0 { + projects, err := db.ListAllProjects() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + + if len(projects) > 0 { + if len(fns) > 0 || len(types) > 0 || len(apps) > 0 || len(analyses) > 0 { + fmt.Fprintln(w) + } + fmt.Fprintln(w, "PROJECT\tID\tDESCRIPTION") + for _, p := range projects { + fmt.Fprintf(w, "project\t%s\t%s\n", p.ID, truncate(p.Description, 60)) + } + } + if len(fns) == 0 && len(types) == 0 && len(apps) == 0 && len(analyses) == 0 && len(projects) == 0 { fmt.Println("Registry is empty. Run 'fn index' first.") } w.Flush() @@ -359,6 +394,18 @@ func cmdShow(args []string) { return } + p, errP := db.GetProject(id) + if errP == nil { + printProjectEntry(p) + return + } + + v, errV := db.GetVault(id) + if errV == nil { + printVaultEntry(v) + return + } + fmt.Fprintf(os.Stderr, "not found: %s\n", id) os.Exit(1) } @@ -518,6 +565,37 @@ func printAnalysisEntry(a *registry.Analysis) { } } +func printProjectEntry(p *registry.Project) { + fmt.Printf("ID: %s\n", p.ID) + fmt.Printf("Name: %s\n", p.Name) + fmt.Printf("Description: %s\n", p.Description) + fmt.Printf("Tags: %s\n", strings.Join(p.Tags, ", ")) + fmt.Printf("Dir: %s\n", p.DirPath) + if p.RepoURL != "" { + fmt.Printf("Repo URL: %s\n", p.RepoURL) + } + if p.Notes != "" { + fmt.Printf("\nNotes:\n%s\n", p.Notes) + } + if p.Documentation != "" { + fmt.Printf("\nDocumentation:\n%s\n", p.Documentation) + } +} + +func printVaultEntry(v *registry.Vault) { + fmt.Printf("ID: %s\n", v.ID) + fmt.Printf("Name: %s\n", v.Name) + if v.ProjectID != "" { + fmt.Printf("Project: %s\n", v.ProjectID) + } + fmt.Printf("Description: %s\n", v.Description) + if v.Path != "" { + fmt.Printf("Path: %s\n", v.Path) + } + fmt.Printf("Symlink: %v\n", v.Symlink) + fmt.Printf("Tags: %s\n", strings.Join(v.Tags, ", ")) +} + // --- check --- func cmdCheck(args []string) { @@ -594,8 +672,10 @@ func cmdAdd(args []string) { templatePath = filepath.Join(r, "docs", "templates", "app.md") case "analysis": templatePath = filepath.Join(r, "docs", "templates", "analysis.md") + case "project": + templatePath = filepath.Join(r, "docs", "templates", "project.md") default: - fmt.Fprintf(os.Stderr, "unknown kind: %s (use function, pipeline, component, app, or analysis)\n", kind) + fmt.Fprintf(os.Stderr, "unknown kind: %s (use function, pipeline, component, app, analysis, or project)\n", kind) os.Exit(1) } diff --git a/cmd/fn/project.go b/cmd/fn/project.go new file mode 100644 index 00000000..635240bd --- /dev/null +++ b/cmd/fn/project.go @@ -0,0 +1,247 @@ +package main + +import ( + "fmt" + "os" + "path/filepath" + "strings" + "text/tabwriter" +) + +func cmdProject(args []string) { + if len(args) < 1 { + printProjectUsage() + os.Exit(1) + } + + switch args[0] { + case "init": + cmdProjectInit(args[1:]) + case "list": + cmdProjectList() + case "show": + cmdProjectShow(args[1:]) + case "status": + cmdProjectStatus(args[1:]) + case "help", "-h", "--help": + printProjectUsage() + default: + fmt.Fprintf(os.Stderr, "unknown project subcommand: %s\n", args[0]) + printProjectUsage() + os.Exit(1) + } +} + +func printProjectUsage() { + fmt.Println(`fn project — manage project workspaces + +Usage: + fn project init Crea scaffold de proyecto + fn project list Lista proyectos del registry + fn project show Muestra proyecto con apps, analysis y vaults + fn project status [] Estado resumido de un proyecto`) +} + +func cmdProjectInit(args []string) { + if len(args) < 1 { + fmt.Fprintln(os.Stderr, "usage: fn project init ") + os.Exit(1) + } + name := args[0] + r := root() + projDir := filepath.Join(r, "projects", name) + + if _, err := os.Stat(projDir); err == nil { + fmt.Fprintf(os.Stderr, "project %q already exists at %s\n", name, projDir) + os.Exit(1) + } + + // Create directory structure + dirs := []string{ + projDir, + filepath.Join(projDir, "apps"), + filepath.Join(projDir, "analysis"), + filepath.Join(projDir, "vaults"), + } + for _, d := range dirs { + if err := os.MkdirAll(d, 0o755); err != nil { + fmt.Fprintf(os.Stderr, "error creating %s: %v\n", d, err) + os.Exit(1) + } + } + + // Create project.md + projectMD := fmt.Sprintf(`--- +name: %s +description: "" +tags: [] +repo_url: "" +--- + +## Notas + +`, name) + if err := os.WriteFile(filepath.Join(projDir, "project.md"), []byte(projectMD), 0o644); err != nil { + fmt.Fprintf(os.Stderr, "error writing project.md: %v\n", err) + os.Exit(1) + } + + // Create vault.yaml + vaultYAML := `vaults: [] +` + if err := os.WriteFile(filepath.Join(projDir, "vaults", "vault.yaml"), []byte(vaultYAML), 0o644); err != nil { + fmt.Fprintf(os.Stderr, "error writing vault.yaml: %v\n", err) + os.Exit(1) + } + + fmt.Printf("Project %q created at %s\n", name, projDir) + fmt.Println("\nStructure:") + fmt.Printf(" %s/\n", filepath.Join("projects", name)) + fmt.Printf(" project.md\n") + fmt.Printf(" apps/\n") + fmt.Printf(" analysis/\n") + fmt.Printf(" vaults/\n") + fmt.Printf(" vault.yaml\n") + fmt.Println("\nNext steps:") + fmt.Printf(" 1. Edit projects/%s/project.md (add description and tags)\n", name) + fmt.Printf(" 2. Create apps or analysis inside the project\n") + fmt.Printf(" 3. Run 'fn index' to register the project\n") +} + +func cmdProjectList() { + db := openDB() + defer db.Close() + + projects, err := db.ListAllProjects() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + + if len(projects) == 0 { + fmt.Println("No projects found. Create one with 'fn project init '.") + return + } + + w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) + fmt.Fprintln(w, "ID\tDESCRIPTION\tTAGS") + for _, p := range projects { + desc := truncate(p.Description, 50) + tags := strings.Join(p.Tags, ", ") + fmt.Fprintf(w, "%s\t%s\t%s\n", p.ID, desc, tags) + } + w.Flush() +} + +func cmdProjectShow(args []string) { + if len(args) < 1 { + fmt.Fprintln(os.Stderr, "usage: fn project show ") + os.Exit(1) + } + id := args[0] + + db := openDB() + defer db.Close() + + p, err := db.GetProject(id) + if err != nil { + fmt.Fprintf(os.Stderr, "project not found: %s\n", id) + os.Exit(1) + } + + fmt.Printf("ID: %s\n", p.ID) + fmt.Printf("Name: %s\n", p.Name) + fmt.Printf("Description: %s\n", p.Description) + fmt.Printf("Tags: %s\n", strings.Join(p.Tags, ", ")) + fmt.Printf("Dir: %s\n", p.DirPath) + if p.RepoURL != "" { + fmt.Printf("Repo URL: %s\n", p.RepoURL) + } + if p.Notes != "" { + fmt.Printf("\nNotes:\n%s\n", p.Notes) + } + if p.Documentation != "" { + fmt.Printf("\nDocumentation:\n%s\n", p.Documentation) + } + + // Show project apps + apps, _ := db.GetProjectApps(id) + if len(apps) > 0 { + fmt.Printf("\nApps (%d):\n", len(apps)) + for _, a := range apps { + fmt.Printf(" - %s (%s) — %s\n", a.ID, a.Lang, truncate(a.Description, 60)) + } + } + + // Show project analysis + analyses, _ := db.GetProjectAnalysis(id) + if len(analyses) > 0 { + fmt.Printf("\nAnalysis (%d):\n", len(analyses)) + for _, an := range analyses { + fmt.Printf(" - %s (%s) — %s\n", an.ID, an.Lang, truncate(an.Description, 60)) + } + } + + // Show project vaults + vaults, _ := db.GetProjectVaults(id) + if len(vaults) > 0 { + fmt.Printf("\nVaults (%d):\n", len(vaults)) + for _, v := range vaults { + sym := "" + if v.Symlink { + sym = fmt.Sprintf(" -> %s", v.Path) + } + fmt.Printf(" - %s — %s%s\n", v.Name, v.Description, sym) + } + } +} + +func cmdProjectStatus(args []string) { + db := openDB() + defer db.Close() + + if len(args) > 0 { + // Status for a specific project + id := args[0] + p, err := db.GetProject(id) + if err != nil { + fmt.Fprintf(os.Stderr, "project not found: %s\n", id) + os.Exit(1) + } + + apps, _ := db.GetProjectApps(id) + analyses, _ := db.GetProjectAnalysis(id) + vaults, _ := db.GetProjectVaults(id) + + fmt.Printf("%s — %s\n", p.Name, p.Description) + fmt.Printf(" Apps: %d\n", len(apps)) + fmt.Printf(" Analysis: %d\n", len(analyses)) + fmt.Printf(" Vaults: %d\n", len(vaults)) + fmt.Printf(" Updated: %s\n", p.UpdatedAt.Format("2006-01-02 15:04")) + return + } + + // Status for all projects + projects, err := db.ListAllProjects() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + + if len(projects) == 0 { + fmt.Println("No projects found.") + return + } + + w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) + fmt.Fprintln(w, "PROJECT\tAPPS\tANALYSIS\tVAULTS\tUPDATED") + for _, p := range projects { + apps, _ := db.GetProjectApps(p.ID) + analyses, _ := db.GetProjectAnalysis(p.ID) + vaults, _ := db.GetProjectVaults(p.ID) + fmt.Fprintf(w, "%s\t%d\t%d\t%d\t%s\n", + p.Name, len(apps), len(analyses), len(vaults), + p.UpdatedAt.Format("2006-01-02")) + } + w.Flush() +} diff --git a/docs/templates/project.md b/docs/templates/project.md new file mode 100644 index 00000000..ecddc590 --- /dev/null +++ b/docs/templates/project.md @@ -0,0 +1,10 @@ +--- +name: my_project +description: "Descripcion breve del proyecto." +tags: [] +repo_url: "" +--- + +## Notas + +Notas adicionales sobre el proyecto. diff --git a/functions/infra/deploy_app_remote.go b/functions/infra/deploy_app_remote.go new file mode 100644 index 00000000..7dcc8eb5 --- /dev/null +++ b/functions/infra/deploy_app_remote.go @@ -0,0 +1,46 @@ +package infra + +import ( + "fmt" + "os/exec" + "strings" +) + +// DeployAppRemote orquesta el deploy continuo de una app a un VPS remoto. +// Pasos: verificar SSH → build local → rsync → restart systemd → health check. +func DeployAppRemote(conn SSHConn, cfg DeployConfig) error { + // 1. Verificar conectividad SSH + if err := SSHCheck(conn); err != nil { + return fmt.Errorf("deploy_app_remote: ssh check: %w", err) + } + + // 2. Build local (si hay comando de build) + if cfg.BuildCmd != "" { + cmd := exec.Command("bash", "-c", cfg.BuildCmd) + cmd.Dir = cfg.LocalDir + out, err := cmd.CombinedOutput() + if err != nil { + return fmt.Errorf("deploy_app_remote: build failed: %s\n%s", err, strings.TrimSpace(string(out))) + } + } + + // 3. Subir binario compilado + if err := uploadAppFiles(conn, cfg); err != nil { + return fmt.Errorf("deploy_app_remote: upload: %w", err) + } + + // 4. Restart systemd service + if err := SystemdRestart(conn, cfg.AppName); err != nil { + return fmt.Errorf("deploy_app_remote: restart: %w", err) + } + + // 5. Health check (si está configurado) + if cfg.HealthPath != "" && cfg.Port > 0 { + url := fmt.Sprintf("http://%s:%d%s", conn.Host, cfg.Port, cfg.HealthPath) + if err := HealthCheckHTTP(url, 30, 2000); err != nil { + return fmt.Errorf("deploy_app_remote: health check: %w", err) + } + } + + return nil +} diff --git a/functions/infra/deploy_app_remote.md b/functions/infra/deploy_app_remote.md new file mode 100644 index 00000000..f49fdfde --- /dev/null +++ b/functions/infra/deploy_app_remote.md @@ -0,0 +1,47 @@ +--- +name: deploy_app_remote +kind: pipeline +lang: go +domain: infra +version: "1.0.0" +purity: impure +signature: "func DeployAppRemote(conn SSHConn, cfg DeployConfig) error" +description: "Orquesta el deploy continuo de una app a un VPS: verifica SSH, compila localmente, sube binario, reinicia systemd y hace health check." +tags: [deploy, vps, remote, ci, cd, pipeline, infra] +uses_functions: [ssh_check_go_infra, ssh_upload_go_infra, ssh_exec_go_infra, systemd_restart_go_infra, health_check_http_go_infra] +uses_types: [ssh_conn_go_infra, DeployConfig_go_infra] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [fmt, os/exec, strings] +params: + - name: conn + desc: "conexión SSH al VPS destino" + - name: cfg + desc: "configuración de deploy con nombre, rutas, build command, puerto y health path" +output: "nil si el deploy completo fue exitoso" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/deploy_app_remote.go" +--- + +## Ejemplo + +```go +conn := SSHConn{Host: "185.x.x.x", User: "root"} +cfg := DeployConfig{ + AppName: "dag_engine", + LocalDir: "apps/dag_engine", + RemoteDir: "/opt/apps/dag_engine", + BinaryName: "dag_engine", + BuildCmd: "CGO_ENABLED=0 GOOS=linux go build -o dag_engine .", + Port: 8080, + HealthPath: "/api/health", +} +err := DeployAppRemote(conn, cfg) +``` + +## Notas + +Pipeline de 5 pasos para deploy continuo (asume que el setup inicial ya se hizo con `setup_vps_app`). El build corre localmente en `LocalDir` con `bash -c BuildCmd`. Si `BuildCmd` está vacío, se salta el build y sube directamente el binario existente. diff --git a/functions/infra/deploy_config.go b/functions/infra/deploy_config.go new file mode 100644 index 00000000..d826dfc1 --- /dev/null +++ b/functions/infra/deploy_config.go @@ -0,0 +1,14 @@ +package infra + +// DeployConfig parametriza un deploy de app a un VPS remoto. +type DeployConfig struct { + AppName string // nombre de la app (usado para systemd unit y logging) + LocalDir string // directorio local de la app (ej: apps/dag_engine) + RemoteDir string // directorio destino en el VPS (ej: /opt/apps/dag_engine) + BinaryName string // nombre del binario compilado (ej: dag_engine) + BuildCmd string // comando de build (ej: CGO_ENABLED=0 GOOS=linux go build -o dag_engine .) + ServiceUser string // usuario del sistema para el servicio (vacío = sin crear) + Port int // puerto del servicio (0 si no expone HTTP) + HealthPath string // path del health check (ej: /api/health, vacío = sin check) + Env map[string]string // variables de entorno para el servicio +} diff --git a/functions/infra/setup_vps_app.go b/functions/infra/setup_vps_app.go new file mode 100644 index 00000000..3372b0f6 --- /dev/null +++ b/functions/infra/setup_vps_app.go @@ -0,0 +1,61 @@ +package infra + +import "fmt" + +// SetupVPSApp orquesta el setup inicial de una app en un VPS remoto. +// Pasos: verificar SSH → crear dirs/usuario → rsync código → generar unit → instalar systemd → health check. +func SetupVPSApp(conn SSHConn, cfg DeployConfig) error { + // 1. Verificar conectividad SSH + if err := SSHCheck(conn); err != nil { + return fmt.Errorf("setup_vps_app: ssh check: %w", err) + } + + // 2. Preparar directorios y usuario en el VPS + if err := VPSSetupApp(conn, cfg.AppName, cfg.RemoteDir, cfg.ServiceUser); err != nil { + return fmt.Errorf("setup_vps_app: vps setup: %w", err) + } + + // 3. Subir binario o archivos de la app + if err := uploadAppFiles(conn, cfg); err != nil { + return fmt.Errorf("setup_vps_app: upload: %w", err) + } + + // 4. Generar unit de systemd + execStart := fmt.Sprintf("%s/%s", cfg.RemoteDir, cfg.BinaryName) + unit := SystemdGenerateUnit(cfg.AppName, execStart, cfg.RemoteDir, cfg.ServiceUser, cfg.Env) + + // 5. Instalar y arrancar servicio + if err := SystemdInstall(conn, cfg.AppName, unit); err != nil { + return fmt.Errorf("setup_vps_app: systemd install: %w", err) + } + + // 6. Health check (si hay endpoint configurado) + if cfg.HealthPath != "" && cfg.Port > 0 { + url := fmt.Sprintf("http://%s:%d%s", conn.Host, cfg.Port, cfg.HealthPath) + if err := HealthCheckHTTP(url, 30, 2000); err != nil { + return fmt.Errorf("setup_vps_app: health check: %w", err) + } + } + + return nil +} + +// uploadAppFiles sube el binario compilado al VPS via SCP. +func uploadAppFiles(conn SSHConn, cfg DeployConfig) error { + localBinary := fmt.Sprintf("%s/%s", cfg.LocalDir, cfg.BinaryName) + remoteBinary := fmt.Sprintf("%s/%s", cfg.RemoteDir, cfg.BinaryName) + + if err := SSHUpload(conn, localBinary, remoteBinary); err != nil { + return err + } + + // Hacer ejecutable + _, stderr, code, err := SSHExec(conn, fmt.Sprintf("chmod +x %s", remoteBinary)) + if err != nil { + return err + } + if code != 0 { + return fmt.Errorf("chmod: %s", stderr) + } + return nil +} diff --git a/functions/infra/setup_vps_app.md b/functions/infra/setup_vps_app.md new file mode 100644 index 00000000..c8fe9f44 --- /dev/null +++ b/functions/infra/setup_vps_app.md @@ -0,0 +1,48 @@ +--- +name: setup_vps_app +kind: pipeline +lang: go +domain: infra +version: "1.0.0" +purity: impure +signature: "func SetupVPSApp(conn SSHConn, cfg DeployConfig) error" +description: "Orquesta el setup inicial de una app en un VPS remoto: verifica SSH, crea dirs y usuario, sube binario, instala systemd unit y hace health check." +tags: [deploy, vps, setup, systemd, ssh, pipeline, infra] +uses_functions: [ssh_check_go_infra, vps_setup_app_go_infra, ssh_upload_go_infra, ssh_exec_go_infra, systemd_generate_unit_go_infra, systemd_install_go_infra, health_check_http_go_infra] +uses_types: [ssh_conn_go_infra, DeployConfig_go_infra] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [fmt] +params: + - name: conn + desc: "conexión SSH al VPS destino" + - name: cfg + desc: "configuración de deploy con nombre, rutas, build, puerto, env vars" +output: "nil si el setup completo fue exitoso" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/setup_vps_app.go" +--- + +## Ejemplo + +```go +conn := SSHConn{Host: "185.x.x.x", User: "root"} +cfg := DeployConfig{ + AppName: "dag_engine", + LocalDir: "apps/dag_engine", + RemoteDir: "/opt/apps/dag_engine", + BinaryName: "dag_engine", + ServiceUser: "deploy", + Port: 8080, + HealthPath: "/api/health", + Env: map[string]string{"PORT": "8080"}, +} +err := SetupVPSApp(conn, cfg) +``` + +## Notas + +Pipeline de 6 pasos para primera instalación. Después del setup inicial, usar `deploy_app_remote` para deploys continuos (no regenera dirs ni systemd unit). El health check espera hasta 30 segundos con polling cada 2s. diff --git a/functions/infra/systemd_generate_unit.go b/functions/infra/systemd_generate_unit.go new file mode 100644 index 00000000..f07d3807 --- /dev/null +++ b/functions/infra/systemd_generate_unit.go @@ -0,0 +1,46 @@ +package infra + +import ( + "fmt" + "sort" + "strings" +) + +// SystemdGenerateUnit genera el texto de un archivo .service de systemd. +func SystemdGenerateUnit(name, execStart, workDir, user string, env map[string]string) string { + var b strings.Builder + + b.WriteString("[Unit]\n") + b.WriteString(fmt.Sprintf("Description=%s\n", name)) + b.WriteString("After=network.target\n\n") + + b.WriteString("[Service]\n") + b.WriteString("Type=simple\n") + b.WriteString(fmt.Sprintf("ExecStart=%s\n", execStart)) + if workDir != "" { + b.WriteString(fmt.Sprintf("WorkingDirectory=%s\n", workDir)) + } + if user != "" { + b.WriteString(fmt.Sprintf("User=%s\n", user)) + } + + // Environment vars en orden determinista + if len(env) > 0 { + keys := make([]string, 0, len(env)) + for k := range env { + keys = append(keys, k) + } + sort.Strings(keys) + for _, k := range keys { + b.WriteString(fmt.Sprintf("Environment=%s=%s\n", k, env[k])) + } + } + + b.WriteString("Restart=on-failure\n") + b.WriteString("RestartSec=5\n\n") + + b.WriteString("[Install]\n") + b.WriteString("WantedBy=multi-user.target\n") + + return b.String() +} diff --git a/functions/infra/systemd_generate_unit.md b/functions/infra/systemd_generate_unit.md new file mode 100644 index 00000000..ae847379 --- /dev/null +++ b/functions/infra/systemd_generate_unit.md @@ -0,0 +1,50 @@ +--- +name: systemd_generate_unit +kind: function +lang: go +domain: infra +version: "1.0.0" +purity: pure +signature: "func SystemdGenerateUnit(name, execStart, workDir, user string, env map[string]string) string" +description: "Genera el texto de un archivo .service de systemd para una app. Incluye restart automático y env vars en orden determinista." +tags: [systemd, unit, service, generate, deploy] +uses_functions: [] +uses_types: [] +returns: [] +returns_optional: false +error_type: "" +imports: [fmt, sort, strings] +params: + - name: name + desc: "nombre del servicio (aparece en Description)" + - name: execStart + desc: "comando completo para arrancar la app (ruta absoluta al binario + args)" + - name: workDir + desc: "directorio de trabajo del servicio (vacío para omitir)" + - name: user + desc: "usuario del sistema bajo el que corre el servicio (vacío para omitir)" + - name: env + desc: "variables de entorno key=value para el servicio" +output: "texto completo del archivo .service listo para escribir a /etc/systemd/system/" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/systemd_generate_unit.go" +--- + +## Ejemplo + +```go +unit := SystemdGenerateUnit( + "dag_engine", + "/opt/apps/dag_engine/dag_engine", + "/opt/apps/dag_engine", + "deploy", + map[string]string{"PORT": "8080", "DB_PATH": "/opt/apps/dag_engine/data/ops.db"}, +) +fmt.Println(unit) +``` + +## Notas + +Función pura sin I/O. Las env vars se ordenan alfabéticamente para output determinista. Genera un unit con Restart=on-failure y RestartSec=5. diff --git a/functions/infra/systemd_install.go b/functions/infra/systemd_install.go new file mode 100644 index 00000000..bee3b6d0 --- /dev/null +++ b/functions/infra/systemd_install.go @@ -0,0 +1,34 @@ +package infra + +import "fmt" + +// SystemdInstall sube un unit file al host remoto, hace daemon-reload, enable y restart. +// Idempotente: si el unit ya existe, lo reemplaza. +func SystemdInstall(conn SSHConn, unitName, unitContent string) error { + // Escribir a archivo temporal y mover a /etc/systemd/system/ + tmpPath := fmt.Sprintf("/tmp/%s.service", unitName) + destPath := fmt.Sprintf("/etc/systemd/system/%s.service", unitName) + + writeCmd := fmt.Sprintf("cat > %s << 'UNIT_EOF'\n%sUNIT_EOF", tmpPath, unitContent) + _, stderr, code, err := SSHExec(conn, writeCmd) + if err != nil { + return fmt.Errorf("systemd_install: ssh write: %w", err) + } + if code != 0 { + return fmt.Errorf("systemd_install: write unit file: %s", stderr) + } + + // Mover a systemd y aplicar + cmds := fmt.Sprintf("sudo mv %s %s && sudo systemctl daemon-reload && sudo systemctl enable %s && sudo systemctl restart %s", + tmpPath, destPath, unitName, unitName) + + _, stderr, code, err = SSHExec(conn, cmds) + if err != nil { + return fmt.Errorf("systemd_install: ssh exec: %w", err) + } + if code != 0 { + return fmt.Errorf("systemd_install: %s", stderr) + } + + return nil +} diff --git a/functions/infra/systemd_install.md b/functions/infra/systemd_install.md new file mode 100644 index 00000000..bdfec4cf --- /dev/null +++ b/functions/infra/systemd_install.md @@ -0,0 +1,41 @@ +--- +name: systemd_install +kind: function +lang: go +domain: infra +version: "1.0.0" +purity: impure +signature: "func SystemdInstall(conn SSHConn, unitName, unitContent string) error" +description: "Sube un unit file al host remoto, hace daemon-reload, enable y restart. Idempotente: reemplaza si el unit ya existe." +tags: [systemd, install, deploy, service, remote] +uses_functions: [ssh_exec_go_infra] +uses_types: [ssh_conn_go_infra] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [fmt] +params: + - name: conn + desc: "conexión SSH al host remoto" + - name: unitName + desc: "nombre del unit sin extensión (ej: dag_engine)" + - name: unitContent + desc: "contenido completo del archivo .service (generado por SystemdGenerateUnit)" +output: "nil si el unit se instaló y arrancó correctamente" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/systemd_install.go" +--- + +## Ejemplo + +```go +conn := SSHConn{Host: "192.168.1.100", User: "deploy"} +unit := SystemdGenerateUnit("dag_engine", "/opt/apps/dag_engine/dag_engine", "/opt/apps/dag_engine", "deploy", nil) +err := SystemdInstall(conn, "dag_engine", unit) +``` + +## Notas + +Escribe el unit a un archivo temporal en /tmp y lo mueve con sudo a /etc/systemd/system/. Requiere que el usuario SSH tenga permisos sudo sin password para systemctl y mv a /etc/systemd/system/. diff --git a/functions/infra/systemd_restart.go b/functions/infra/systemd_restart.go new file mode 100644 index 00000000..acc7d9de --- /dev/null +++ b/functions/infra/systemd_restart.go @@ -0,0 +1,15 @@ +package infra + +import "fmt" + +// SystemdRestart reinicia un servicio systemd en un host remoto. +func SystemdRestart(conn SSHConn, unitName string) error { + _, stderr, code, err := SSHExec(conn, fmt.Sprintf("sudo systemctl restart %s", unitName)) + if err != nil { + return fmt.Errorf("systemd_restart: ssh exec: %w", err) + } + if code != 0 { + return fmt.Errorf("systemd_restart %s: %s", unitName, stderr) + } + return nil +} diff --git a/functions/infra/systemd_restart.md b/functions/infra/systemd_restart.md new file mode 100644 index 00000000..6ce30e5d --- /dev/null +++ b/functions/infra/systemd_restart.md @@ -0,0 +1,40 @@ +--- +name: systemd_restart +kind: function +lang: go +domain: infra +version: "1.0.0" +purity: impure +signature: "func SystemdRestart(conn SSHConn, unitName string) error" +description: "Reinicia un servicio systemd en un host remoto via SSH." +tags: [systemd, restart, service, remote] +uses_functions: [ssh_exec_go_infra] +uses_types: [ssh_conn_go_infra] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [fmt] +params: + - name: conn + desc: "conexión SSH al host remoto" + - name: unitName + desc: "nombre del unit systemd a reiniciar (sin .service)" +output: "nil si el reinicio fue exitoso" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/systemd_restart.go" +--- + +## Ejemplo + +```go +conn := SSHConn{Host: "192.168.1.100", User: "deploy"} +if err := SystemdRestart(conn, "dag_engine"); err != nil { + log.Fatal(err) +} +``` + +## Notas + +Usa `sudo systemctl restart`. Requiere que el usuario SSH tenga permisos sudo. Si el servicio no existe, systemctl retorna error. diff --git a/functions/infra/systemd_status.go b/functions/infra/systemd_status.go new file mode 100644 index 00000000..9fcf2c3c --- /dev/null +++ b/functions/infra/systemd_status.go @@ -0,0 +1,57 @@ +package infra + +import ( + "fmt" + "strings" +) + +// SystemdServiceStatus resultado de consultar el estado de un servicio systemd. +type SystemdServiceStatus struct { + Unit string + Active string // "active", "inactive", "failed" + SubState string // "running", "dead", "failed" + MainPID string + Logs string +} + +// SystemdStatus consulta el estado de un servicio systemd en un host remoto. +func SystemdStatus(conn SSHConn, unitName string, logLines int) (SystemdServiceStatus, error) { + status := SystemdServiceStatus{Unit: unitName} + + // Obtener propiedades del servicio + cmd := fmt.Sprintf("systemctl show %s --property=ActiveState,SubState,MainPID --no-pager", unitName) + stdout, stderr, code, err := SSHExec(conn, cmd) + if err != nil { + return status, fmt.Errorf("systemd_status: ssh exec: %w", err) + } + if code != 0 { + return status, fmt.Errorf("systemd_status: systemctl show: %s", stderr) + } + + // Parsear propiedades key=value + for _, line := range strings.Split(stdout, "\n") { + parts := strings.SplitN(line, "=", 2) + if len(parts) != 2 { + continue + } + switch parts[0] { + case "ActiveState": + status.Active = parts[1] + case "SubState": + status.SubState = parts[1] + case "MainPID": + status.MainPID = parts[1] + } + } + + // Obtener logs recientes + if logLines > 0 { + logCmd := fmt.Sprintf("journalctl -u %s -n %d --no-pager 2>/dev/null || true", unitName, logLines) + logOut, _, _, logErr := SSHExec(conn, logCmd) + if logErr == nil { + status.Logs = strings.TrimSpace(logOut) + } + } + + return status, nil +} diff --git a/functions/infra/systemd_status.md b/functions/infra/systemd_status.md new file mode 100644 index 00000000..5078196a --- /dev/null +++ b/functions/infra/systemd_status.md @@ -0,0 +1,45 @@ +--- +name: systemd_status +kind: function +lang: go +domain: infra +version: "1.0.0" +purity: impure +signature: "func SystemdStatus(conn SSHConn, unitName string, logLines int) (SystemdServiceStatus, error)" +description: "Consulta el estado de un servicio systemd en un host remoto. Retorna estado activo, sub-estado, PID y logs recientes." +tags: [systemd, status, monitor, service, remote] +uses_functions: [ssh_exec_go_infra] +uses_types: [ssh_conn_go_infra] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [fmt, strings] +params: + - name: conn + desc: "conexión SSH al host remoto" + - name: unitName + desc: "nombre del unit systemd a consultar (sin .service)" + - name: logLines + desc: "número de líneas de journalctl a incluir (0 para no incluir logs)" +output: "SystemdServiceStatus con Active, SubState, MainPID y Logs del servicio" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/systemd_status.go" +--- + +## Ejemplo + +```go +conn := SSHConn{Host: "192.168.1.100", User: "deploy"} +s, err := SystemdStatus(conn, "dag_engine", 20) +if err != nil { + log.Fatal(err) +} +fmt.Printf("State: %s/%s PID: %s\n", s.Active, s.SubState, s.MainPID) +fmt.Println(s.Logs) +``` + +## Notas + +Usa `systemctl show` para obtener propiedades sin formato humano. Los logs se obtienen con journalctl y son opcionales (logLines=0 los omite). Si journalctl falla (ej: permisos), los logs quedan vacíos sin error. diff --git a/functions/infra/vps_setup_app.go b/functions/infra/vps_setup_app.go new file mode 100644 index 00000000..f3caead2 --- /dev/null +++ b/functions/infra/vps_setup_app.go @@ -0,0 +1,42 @@ +package infra + +import "fmt" + +// VPSSetupApp prepara un host remoto para recibir una app: +// crea directorios, usuario de servicio si no existe, y directorio de datos. +func VPSSetupApp(conn SSHConn, appName, remoteDir, serviceUser string) error { + // Crear directorio de la app y subdirectorios comunes + mkdirCmd := fmt.Sprintf("sudo mkdir -p %s/data %s/logs", remoteDir, remoteDir) + _, stderr, code, err := SSHExec(conn, mkdirCmd) + if err != nil { + return fmt.Errorf("vps_setup_app: ssh exec: %w", err) + } + if code != 0 { + return fmt.Errorf("vps_setup_app: mkdir: %s", stderr) + } + + // Crear usuario de servicio si se especificó y no existe + if serviceUser != "" { + userCmd := fmt.Sprintf("id %s >/dev/null 2>&1 || sudo useradd -r -s /usr/sbin/nologin -d %s %s", + serviceUser, remoteDir, serviceUser) + _, stderr, code, err = SSHExec(conn, userCmd) + if err != nil { + return fmt.Errorf("vps_setup_app: ssh exec: %w", err) + } + if code != 0 { + return fmt.Errorf("vps_setup_app: create user: %s", stderr) + } + + // Asignar ownership al usuario de servicio + chownCmd := fmt.Sprintf("sudo chown -R %s:%s %s", serviceUser, serviceUser, remoteDir) + _, stderr, code, err = SSHExec(conn, chownCmd) + if err != nil { + return fmt.Errorf("vps_setup_app: ssh exec: %w", err) + } + if code != 0 { + return fmt.Errorf("vps_setup_app: chown: %s", stderr) + } + } + + return nil +} diff --git a/functions/infra/vps_setup_app.md b/functions/infra/vps_setup_app.md new file mode 100644 index 00000000..d0a040e5 --- /dev/null +++ b/functions/infra/vps_setup_app.md @@ -0,0 +1,42 @@ +--- +name: vps_setup_app +kind: function +lang: go +domain: infra +version: "1.0.0" +purity: impure +signature: "func VPSSetupApp(conn SSHConn, appName, remoteDir, serviceUser string) error" +description: "Prepara un host remoto para recibir una app: crea directorios, usuario de servicio y asigna ownership." +tags: [vps, setup, deploy, remote, infra] +uses_functions: [ssh_exec_go_infra] +uses_types: [ssh_conn_go_infra] +returns: [] +returns_optional: false +error_type: "error_go_core" +imports: [fmt] +params: + - name: conn + desc: "conexión SSH al host remoto" + - name: appName + desc: "nombre de la app (para logging)" + - name: remoteDir + desc: "ruta absoluta donde vivirá la app en el remoto (ej: /opt/apps/dag_engine)" + - name: serviceUser + desc: "usuario del sistema para correr el servicio (vacío para omitir creación de usuario)" +output: "nil si el setup fue exitoso" +tested: false +tests: [] +test_file_path: "" +file_path: "functions/infra/vps_setup_app.go" +--- + +## Ejemplo + +```go +conn := SSHConn{Host: "192.168.1.100", User: "deploy"} +err := VPSSetupApp(conn, "dag_engine", "/opt/apps/dag_engine", "deploy") +``` + +## Notas + +Idempotente: mkdir -p no falla si el directorio existe, useradd se salta si el usuario existe. Crea subdirectorios `data/` y `logs/` dentro del remoteDir. Requiere sudo. diff --git a/registry/hash.go b/registry/hash.go index 83b28102..0d2afea8 100644 --- a/registry/hash.go +++ b/registry/hash.go @@ -77,9 +77,28 @@ func ComputeAnalysisHash(a *Analysis) string { return fmt.Sprintf("%x", h.Sum(nil)) } +// ComputeProjectHash computes a deterministic hash of all content fields of a Project. +func ComputeProjectHash(p *Project) string { + h := sha256.New() + fmt.Fprintf(h, "%s|%s|%s", + p.ID, p.Name, p.Description) + fmt.Fprintf(h, "|%s", marshalStrings(p.Tags)) + fmt.Fprintf(h, "|%s|%s|%s|%s", p.RepoURL, p.DirPath, p.Documentation, p.Notes) + return fmt.Sprintf("%x", h.Sum(nil)) +} + +// ComputeVaultHash computes a deterministic hash of all content fields of a Vault. +func ComputeVaultHash(v *Vault) string { + h := sha256.New() + fmt.Fprintf(h, "%s|%s|%s|%s|%s|%t", + v.ID, v.Name, v.ProjectID, v.Description, v.Path, v.Symlink) + fmt.Fprintf(h, "|%s", marshalStrings(v.Tags)) + return fmt.Sprintf("%x", h.Sum(nil)) +} + // LoadTimestamps reads existing id → {created_at, updated_at, content_hash} from all tables. // Called before Purge so we can preserve dates across reindexing. -func (db *DB) LoadTimestamps() (funcs, types, apps, analysis map[string]timestampRecord, err error) { +func (db *DB) LoadTimestamps() (funcs, types, apps, analysis, projects, vaults map[string]timestampRecord, err error) { funcs, err = loadTable(db, "functions") if err != nil { return @@ -93,6 +112,14 @@ func (db *DB) LoadTimestamps() (funcs, types, apps, analysis map[string]timestam return } analysis, err = loadTable(db, "analysis") + if err != nil { + return + } + projects, err = loadTable(db, "projects") + if err != nil { + return + } + vaults, err = loadTable(db, "vaults") return } diff --git a/registry/indexer.go b/registry/indexer.go index 6970a098..f0b13374 100644 --- a/registry/indexer.go +++ b/registry/indexer.go @@ -14,6 +14,8 @@ type IndexResult struct { Types int Apps int Analysis int + Projects int + Vaults int UnitTests int ValidationErrors []string Warnings []string @@ -29,7 +31,7 @@ type IndexResult struct { // directories (e.g. python/functions/, python/types/). func Index(db *DB, root string) (*IndexResult, error) { // Load existing timestamps before purging so we can preserve created_at - oldFuncs, oldTypes, oldApps, oldAnalysis, err := db.LoadTimestamps() + oldFuncs, oldTypes, oldApps, oldAnalysis, oldProjects, oldVaults, err := db.LoadTimestamps() if err != nil { return nil, fmt.Errorf("loading timestamps: %w", err) } @@ -82,7 +84,7 @@ func Index(db *DB, root string) (*IndexResult, error) { }) } - // Parse apps from apps/*/app.md + // Parse apps from apps/*/app.md (standalone apps, no project) var apps []*App localAppIDs := make(map[string]bool) appsDir := filepath.Join(root, "apps") @@ -106,7 +108,7 @@ func Index(db *DB, root string) (*IndexResult, error) { } } - // Parse analysis from analysis/*/analysis.md + // Parse analysis from analysis/*/analysis.md (standalone, no project) var analyses []*Analysis localAnalysisIDs := make(map[string]bool) analysisDir := filepath.Join(root, "analysis") @@ -130,8 +132,111 @@ func Index(db *DB, root string) (*IndexResult, error) { } } - // Selective purge: preserve remote-only apps/analysis (have repo_url, not cloned locally) - if err := db.PurgeLocalOnly(localAppIDs, localAnalysisIDs); err != nil { + // Parse projects from projects/*/project.md + var projects []*Project + var vaults []*Vault + localProjectIDs := make(map[string]bool) + projectsDir := filepath.Join(root, "projects") + if fi, err := os.Stat(projectsDir); err == nil && fi.IsDir() { + projEntries, _ := os.ReadDir(projectsDir) + for _, pe := range projEntries { + if !pe.IsDir() { + continue + } + projName := pe.Name() + projDir := filepath.Join(projectsDir, projName) + + // Parse project.md + projMD := filepath.Join(projDir, "project.md") + if _, err := os.Stat(projMD); err != nil { + continue + } + p, err := ParseProjectMD(projMD, root) + if err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("parse %s: %v", projMD, err)) + continue + } + projects = append(projects, p) + localProjectIDs[p.ID] = true + + // Parse project apps from projects/{name}/apps/*/app.md + projAppsDir := filepath.Join(projDir, "apps") + if fi, err := os.Stat(projAppsDir); err == nil && fi.IsDir() { + appEntries, _ := os.ReadDir(projAppsDir) + for _, ae := range appEntries { + if !ae.IsDir() { + continue + } + appMD := filepath.Join(projAppsDir, ae.Name(), "app.md") + if _, err := os.Stat(appMD); err != nil { + continue + } + a, err := ParseAppMD(appMD, root) + if err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("parse %s: %v", appMD, err)) + continue + } + a.ProjectID = p.ID + if a.DirPath == "" { + a.DirPath = filepath.Join("projects", projName, "apps", ae.Name()) + } + apps = append(apps, a) + localAppIDs[a.ID] = true + } + } + + // Parse project analysis from projects/{name}/analysis/*/analysis.md + projAnalysisDir := filepath.Join(projDir, "analysis") + if fi, err := os.Stat(projAnalysisDir); err == nil && fi.IsDir() { + anEntries, _ := os.ReadDir(projAnalysisDir) + for _, ane := range anEntries { + if !ane.IsDir() { + continue + } + anMD := filepath.Join(projAnalysisDir, ane.Name(), "analysis.md") + if _, err := os.Stat(anMD); err != nil { + continue + } + an, err := ParseAnalysisMD(anMD, root) + if err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("parse %s: %v", anMD, err)) + continue + } + an.ProjectID = p.ID + if an.DirPath == "" { + an.DirPath = filepath.Join("projects", projName, "analysis", ane.Name()) + } + analyses = append(analyses, an) + localAnalysisIDs[an.ID] = true + } + } + + // Parse project vaults from projects/{name}/vaults/vault.yaml + projVaultYAML := filepath.Join(projDir, "vaults", "vault.yaml") + if _, err := os.Stat(projVaultYAML); err == nil { + vs, err := ParseVaultYAML(projVaultYAML, p.ID, filepath.Join(projDir, "vaults")) + if err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("parse %s: %v", projVaultYAML, err)) + } else { + vaults = append(vaults, vs...) + } + } + } + } + + // Parse registry-level vaults from vaults/vault.yaml + registryVaultYAML := filepath.Join(root, "vaults", "vault.yaml") + if _, err := os.Stat(registryVaultYAML); err == nil { + vs, err := ParseVaultYAML(registryVaultYAML, "", filepath.Join(root, "vaults")) + if err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("parse %s: %v", registryVaultYAML, err)) + } else { + vaults = append(vaults, vs...) + } + } + + // Selective purge: preserve remote-only apps/analysis/projects (have repo_url, not cloned locally) + if err := db.PurgeLocalOnly(localAppIDs, localAnalysisIDs, localProjectIDs); err != nil { return nil, fmt.Errorf("purging database: %w", err) } @@ -204,6 +309,30 @@ func Index(db *DB, root string) (*IndexResult, error) { result.Analysis++ } + for _, p := range projects { + if verr := ValidateProject(p); verr != nil { + result.ValidationErrors = append(result.ValidationErrors, verr.Error()) + continue + } + p.ContentHash = ComputeProjectHash(p) + applyTimestamps(&p.CreatedAt, &p.UpdatedAt, p.ContentHash, oldProjects[p.ID], now) + if err := db.InsertProject(p); err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("insert project %s: %v", p.ID, err)) + continue + } + result.Projects++ + } + + for _, v := range vaults { + v.ContentHash = ComputeVaultHash(v) + applyTimestamps(&v.CreatedAt, &v.UpdatedAt, v.ContentHash, oldVaults[v.ID], now) + if err := db.InsertVault(v); err != nil { + result.Errors = append(result.Errors, fmt.Sprintf("insert vault %s: %v", v.ID, err)) + continue + } + result.Vaults++ + } + // Extract unit tests from test files of tested functions if err := db.PurgeUnitTests(); err != nil { result.Warnings = append(result.Warnings, fmt.Sprintf("purging unit_tests: %v", err)) diff --git a/registry/migrations/010_projects.sql b/registry/migrations/010_projects.sql new file mode 100644 index 00000000..bcb51eb1 --- /dev/null +++ b/registry/migrations/010_projects.sql @@ -0,0 +1,62 @@ +-- Projects: agrupan apps, analysis y vaults bajo un tema comun. +-- Vaults: almacenes de datos (symlinks o directorios) asociados a proyectos o al registry. + +CREATE TABLE IF NOT EXISTS projects ( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + description TEXT NOT NULL DEFAULT '', + tags TEXT NOT NULL DEFAULT '[]', + repo_url TEXT NOT NULL DEFAULT '', + dir_path TEXT NOT NULL DEFAULT '', + documentation TEXT NOT NULL DEFAULT '', + notes TEXT NOT NULL DEFAULT '', + content_hash TEXT NOT NULL DEFAULT '', + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); + +CREATE VIRTUAL TABLE IF NOT EXISTS projects_fts USING fts5( + id, + name, + description, + tags, + documentation, + notes, + content='projects', + content_rowid='rowid' +); + +CREATE TRIGGER IF NOT EXISTS projects_ai AFTER INSERT ON projects BEGIN + INSERT INTO projects_fts(rowid, id, name, description, tags, documentation, notes) + VALUES (new.rowid, new.id, new.name, new.description, new.tags, new.documentation, new.notes); +END; + +CREATE TRIGGER IF NOT EXISTS projects_ad AFTER DELETE ON projects BEGIN + INSERT INTO projects_fts(projects_fts, rowid, id, name, description, tags, documentation, notes) + VALUES ('delete', old.rowid, old.id, old.name, old.description, old.tags, old.documentation, old.notes); +END; + +CREATE TRIGGER IF NOT EXISTS projects_au AFTER UPDATE ON projects BEGIN + INSERT INTO projects_fts(projects_fts, rowid, id, name, description, tags, documentation, notes) + VALUES ('delete', old.rowid, old.id, old.name, old.description, old.tags, old.documentation, old.notes); + INSERT INTO projects_fts(rowid, id, name, description, tags, documentation, notes) + VALUES (new.rowid, new.id, new.name, new.description, new.tags, new.documentation, new.notes); +END; + +-- Vaults: almacenes de datos trackados en el registry. +CREATE TABLE IF NOT EXISTS vaults ( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + project_id TEXT NOT NULL DEFAULT '', + description TEXT NOT NULL DEFAULT '', + path TEXT NOT NULL DEFAULT '', + symlink INTEGER NOT NULL DEFAULT 0, + tags TEXT NOT NULL DEFAULT '[]', + content_hash TEXT NOT NULL DEFAULT '', + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); + +-- Columna project_id en apps y analysis para vincular a un proyecto. +ALTER TABLE apps ADD COLUMN project_id TEXT NOT NULL DEFAULT ''; +ALTER TABLE analysis ADD COLUMN project_id TEXT NOT NULL DEFAULT ''; diff --git a/registry/models.go b/registry/models.go index a6c4c038..165eb76e 100644 --- a/registry/models.go +++ b/registry/models.go @@ -120,6 +120,7 @@ type App struct { DirPath string `json:"dir_path"` ContentHash string `json:"content_hash"` RepoURL string `json:"repo_url"` + ProjectID string `json:"project_id"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } @@ -141,6 +142,7 @@ type Analysis struct { RepoURL string `json:"repo_url"` DirPath string `json:"dir_path"` ContentHash string `json:"content_hash"` + ProjectID string `json:"project_id"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } @@ -193,6 +195,35 @@ type UnitTest struct { UpdatedAt time.Time `json:"updated_at"` } +// Project groups apps, analysis and vaults under a common theme. +type Project struct { + ID string `json:"id"` + Name string `json:"name"` + Description string `json:"description"` + Tags []string `json:"tags"` + RepoURL string `json:"repo_url"` + DirPath string `json:"dir_path"` + Documentation string `json:"documentation"` + Notes string `json:"notes"` + ContentHash string `json:"content_hash"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} + +// Vault is a data store (symlink or directory) associated with a project or the registry. +type Vault struct { + ID string `json:"id"` + Name string `json:"name"` + ProjectID string `json:"project_id"` + Description string `json:"description"` + Path string `json:"path"` + Symlink bool `json:"symlink"` + Tags []string `json:"tags"` + ContentHash string `json:"content_hash"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} + // GenerateID builds the canonical ID: {name}_{lang}_{domain} func GenerateID(name, lang, domain string) string { return name + "_" + lang + "_" + domain diff --git a/registry/parser.go b/registry/parser.go index 8f46526e..5e7afb57 100644 --- a/registry/parser.go +++ b/registry/parser.go @@ -103,6 +103,27 @@ type rawAnalysis struct { RepoURL string `yaml:"repo_url"` } +// rawProject mirrors the YAML frontmatter of a project .md file. +type rawProject struct { + Name string `yaml:"name"` + Description string `yaml:"description"` + Tags []string `yaml:"tags"` + RepoURL string `yaml:"repo_url"` +} + +// rawVaultFile mirrors the YAML of a vault.yaml manifest file. +type rawVaultFile struct { + Vaults []rawVaultEntry `yaml:"vaults"` +} + +// rawVaultEntry describes a single vault in vault.yaml. +type rawVaultEntry struct { + Name string `yaml:"name"` + Description string `yaml:"description"` + Path string `yaml:"path"` + Tags []string `yaml:"tags"` +} + // extractFrontmatter splits a .md file into YAML frontmatter and body. func extractFrontmatter(data []byte) ([]byte, []byte, error) { content := data @@ -356,6 +377,99 @@ func ParseAnalysisMD(path string, root string) (*Analysis, error) { return an, nil } +// ParseProjectMD parses a project .md file into a Project. +func ParseProjectMD(path string, root string) (*Project, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("reading %s: %w", path, err) + } + + fm, body, err := extractFrontmatter(data) + if err != nil { + return nil, fmt.Errorf("parsing %s: %w", path, err) + } + + var raw rawProject + if err := yaml.Unmarshal(fm, &raw); err != nil { + return nil, fmt.Errorf("parsing YAML in %s: %w", path, err) + } + + if raw.Name == "" { + return nil, fmt.Errorf("%s: name is required", path) + } + if raw.Description == "" { + return nil, fmt.Errorf("%s: description is required", path) + } + + sections := extractSections(body) + + p := &Project{ + ID: raw.Name, + Name: raw.Name, + Description: raw.Description, + Tags: raw.Tags, + RepoURL: raw.RepoURL, + DirPath: filepath.Join("projects", raw.Name), + Documentation: sections.documentation, + Notes: sections.notes, + } + + return p, nil +} + +// ParseVaultYAML parses a vault.yaml manifest into a slice of Vaults. +// projectID is the owning project ID, or "" for registry-level vaults. +// vaultsDir is the directory containing vault.yaml (used to detect symlinks). +func ParseVaultYAML(path string, projectID string, vaultsDir string) ([]*Vault, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("reading %s: %w", path, err) + } + + var raw rawVaultFile + if err := yaml.Unmarshal(data, &raw); err != nil { + return nil, fmt.Errorf("parsing YAML in %s: %w", path, err) + } + + var vaults []*Vault + for _, rv := range raw.Vaults { + if rv.Name == "" { + continue + } + + suffix := projectID + if suffix == "" { + suffix = "registry" + } + id := rv.Name + "_" + suffix + + // Detect if the vault entry on disk is a symlink + isSymlink := false + vaultPath := rv.Path + entryPath := filepath.Join(vaultsDir, rv.Name) + if fi, err := os.Lstat(entryPath); err == nil { + if fi.Mode()&os.ModeSymlink != 0 { + isSymlink = true + if target, err := os.Readlink(entryPath); err == nil && vaultPath == "" { + vaultPath = target + } + } + } + + vaults = append(vaults, &Vault{ + ID: id, + Name: rv.Name, + ProjectID: projectID, + Description: rv.Description, + Path: vaultPath, + Symlink: isSymlink, + Tags: rv.Tags, + }) + } + + return vaults, nil +} + // bodySections holds the extracted sections from a .md body. type bodySections struct { example string // content under ## Ejemplo diff --git a/registry/store.go b/registry/store.go index 246097ba..9bd7d3d5 100644 --- a/registry/store.go +++ b/registry/store.go @@ -291,12 +291,12 @@ func (db *DB) InsertApp(a *App) error { INSERT OR REPLACE INTO apps ( id, name, lang, domain, description, tags, uses_functions, uses_types, framework, entry_point, - documentation, notes, dir_path, content_hash, created_at, updated_at, repo_url - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + documentation, notes, dir_path, content_hash, created_at, updated_at, repo_url, project_id + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, a.ID, a.Name, a.Lang, a.Domain, a.Description, marshalStrings(a.Tags), marshalStrings(a.UsesFunctions), marshalStrings(a.UsesTypes), a.Framework, a.EntryPoint, a.Documentation, a.Notes, a.DirPath, a.ContentHash, a.CreatedAt.Format(time.RFC3339), a.UpdatedAt.Format(time.RFC3339), - a.RepoURL, + a.RepoURL, a.ProjectID, ) return err } @@ -363,7 +363,7 @@ func scanApps(rows interface{ Next() bool; Scan(...any) error }) ([]App, error) &a.ID, &a.Name, &a.Lang, &a.Domain, &a.Description, &tagsJSON, &usesFnJSON, &usesTypJSON, &a.Framework, &a.EntryPoint, &a.Documentation, &a.Notes, &a.DirPath, &createdAt, &updatedAt, &a.ContentHash, - &a.RepoURL, + &a.RepoURL, &a.ProjectID, ) if err != nil { return nil, fmt.Errorf("scanning app: %w", err) @@ -380,7 +380,7 @@ func scanApps(rows interface{ Next() bool; Scan(...any) error }) ([]App, error) return result, nil } -// Purge deletes all data from functions, types, apps and analysis. Used before re-indexing. +// Purge deletes all data from functions, types, apps, analysis, projects and vaults. Used before re-indexing. func (db *DB) Purge() error { if _, err := db.conn.Exec("DELETE FROM functions"); err != nil { return err @@ -391,13 +391,19 @@ func (db *DB) Purge() error { if _, err := db.conn.Exec("DELETE FROM apps"); err != nil { return err } - _, err := db.conn.Exec("DELETE FROM analysis") + if _, err := db.conn.Exec("DELETE FROM analysis"); err != nil { + return err + } + if _, err := db.conn.Exec("DELETE FROM projects"); err != nil { + return err + } + _, err := db.conn.Exec("DELETE FROM vaults") return err } -// PurgeLocalOnly deletes functions, types, and only locally-present apps/analysis. -// Remote-only records (repo_url set, not in localAppIDs/localAnalysisIDs) are preserved. -func (db *DB) PurgeLocalOnly(localAppIDs, localAnalysisIDs map[string]bool) error { +// PurgeLocalOnly deletes functions, types, and only locally-present apps/analysis/projects/vaults. +// Remote-only records (repo_url set, not in local ID maps) are preserved. +func (db *DB) PurgeLocalOnly(localAppIDs, localAnalysisIDs, localProjectIDs map[string]bool) error { if _, err := db.conn.Exec("DELETE FROM functions"); err != nil { return err } @@ -423,6 +429,19 @@ func (db *DB) PurgeLocalOnly(localAppIDs, localAnalysisIDs map[string]bool) erro if _, err := db.conn.Exec("DELETE FROM analysis WHERE repo_url = '' OR repo_url IS NULL"); err != nil { return err } + // Projects: delete locally-scanned, preserve remote-only + for id := range localProjectIDs { + if _, err := db.conn.Exec("DELETE FROM projects WHERE id = ?", id); err != nil { + return err + } + } + if _, err := db.conn.Exec("DELETE FROM projects WHERE repo_url = '' OR repo_url IS NULL"); err != nil { + return err + } + // Vaults: always purge and re-insert from vault.yaml + if _, err := db.conn.Exec("DELETE FROM vaults"); err != nil { + return err + } return nil } @@ -446,12 +465,12 @@ func (db *DB) InsertAnalysis(a *Analysis) error { INSERT OR REPLACE INTO analysis ( id, name, lang, domain, description, tags, uses_functions, uses_types, framework, entry_point, - documentation, notes, repo_url, dir_path, content_hash, created_at, updated_at - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + documentation, notes, repo_url, dir_path, content_hash, created_at, updated_at, project_id + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, a.ID, a.Name, a.Lang, a.Domain, a.Description, marshalStrings(a.Tags), marshalStrings(a.UsesFunctions), marshalStrings(a.UsesTypes), a.Framework, a.EntryPoint, a.Documentation, a.Notes, a.RepoURL, a.DirPath, a.ContentHash, - a.CreatedAt.Format(time.RFC3339), a.UpdatedAt.Format(time.RFC3339), + a.CreatedAt.Format(time.RFC3339), a.UpdatedAt.Format(time.RFC3339), a.ProjectID, ) return err } @@ -528,7 +547,7 @@ func scanAnalysis(rows interface{ Next() bool; Scan(...any) error }) ([]Analysis &a.ID, &a.Name, &a.Lang, &a.Domain, &a.Description, &tagsJSON, &usesFnJSON, &usesTypJSON, &a.Framework, &a.EntryPoint, &a.Documentation, &a.Notes, &a.RepoURL, &a.DirPath, &a.ContentHash, - &createdAt, &updatedAt, + &createdAt, &updatedAt, &a.ProjectID, ) if err != nil { return nil, fmt.Errorf("scanning analysis: %w", err) @@ -545,6 +564,229 @@ func scanAnalysis(rows interface{ Next() bool; Scan(...any) error }) ([]Analysis return result, nil } +// --- Project CRUD --- + +// InsertProject inserts or replaces a project entry. +func (db *DB) InsertProject(p *Project) error { + now := time.Now().UTC() + if p.CreatedAt.IsZero() { + p.CreatedAt = now + } + if p.UpdatedAt.IsZero() { + p.UpdatedAt = now + } + + _, err := db.conn.Exec(` + INSERT OR REPLACE INTO projects ( + id, name, description, tags, repo_url, dir_path, + documentation, notes, content_hash, created_at, updated_at + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + p.ID, p.Name, p.Description, marshalStrings(p.Tags), p.RepoURL, p.DirPath, + p.Documentation, p.Notes, p.ContentHash, + p.CreatedAt.Format(time.RFC3339), p.UpdatedAt.Format(time.RFC3339), + ) + return err +} + +// GetProject returns a single project by ID. +func (db *DB) GetProject(id string) (*Project, error) { + rows, err := db.conn.Query("SELECT * FROM projects WHERE id = ?", id) + if err != nil { + return nil, err + } + defer rows.Close() + + ps, err := scanProjects(rows) + if err != nil { + return nil, err + } + if len(ps) == 0 { + return nil, fmt.Errorf("project %q not found", id) + } + return &ps[0], nil +} + +// SearchProjects performs FTS search on projects. +func (db *DB) SearchProjects(query string) ([]Project, error) { + where := []string{} + args := []any{} + + if query != "" { + where = append(where, "p.id IN (SELECT id FROM projects_fts WHERE projects_fts MATCH ?)") + args = append(args, query) + } + + sql := "SELECT * FROM projects p" + if len(where) > 0 { + sql += " WHERE " + strings.Join(where, " AND ") + } + sql += " ORDER BY p.name" + + rows, err := db.conn.Query(sql, args...) + if err != nil { + return nil, fmt.Errorf("search projects: %w", err) + } + defer rows.Close() + + return scanProjects(rows) +} + +// ListAllProjects returns all project entries. +func (db *DB) ListAllProjects() ([]Project, error) { + return db.SearchProjects("") +} + +// GetProjectApps returns all apps belonging to a project. +func (db *DB) GetProjectApps(projectID string) ([]App, error) { + rows, err := db.conn.Query("SELECT * FROM apps WHERE project_id = ? ORDER BY name", projectID) + if err != nil { + return nil, err + } + defer rows.Close() + return scanApps(rows) +} + +// GetProjectAnalysis returns all analysis entries belonging to a project. +func (db *DB) GetProjectAnalysis(projectID string) ([]Analysis, error) { + rows, err := db.conn.Query("SELECT * FROM analysis WHERE project_id = ? ORDER BY name", projectID) + if err != nil { + return nil, err + } + defer rows.Close() + return scanAnalysis(rows) +} + +func scanProjects(rows interface{ Next() bool; Scan(...any) error }) ([]Project, error) { + var result []Project + for rows.Next() { + var p Project + var tagsJSON string + var createdAt, updatedAt string + + err := rows.Scan( + &p.ID, &p.Name, &p.Description, &tagsJSON, &p.RepoURL, &p.DirPath, + &p.Documentation, &p.Notes, &p.ContentHash, &createdAt, &updatedAt, + ) + if err != nil { + return nil, fmt.Errorf("scanning project: %w", err) + } + + p.Tags = unmarshalStrings(tagsJSON) + p.CreatedAt, _ = time.Parse(time.RFC3339, createdAt) + p.UpdatedAt, _ = time.Parse(time.RFC3339, updatedAt) + + result = append(result, p) + } + return result, nil +} + +// --- Vault CRUD --- + +// InsertVault inserts or replaces a vault entry. +func (db *DB) InsertVault(v *Vault) error { + now := time.Now().UTC() + if v.CreatedAt.IsZero() { + v.CreatedAt = now + } + if v.UpdatedAt.IsZero() { + v.UpdatedAt = now + } + + sym := 0 + if v.Symlink { + sym = 1 + } + + _, err := db.conn.Exec(` + INSERT OR REPLACE INTO vaults ( + id, name, project_id, description, path, symlink, tags, + content_hash, created_at, updated_at + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + v.ID, v.Name, v.ProjectID, v.Description, v.Path, sym, marshalStrings(v.Tags), + v.ContentHash, v.CreatedAt.Format(time.RFC3339), v.UpdatedAt.Format(time.RFC3339), + ) + return err +} + +// GetVault returns a single vault by ID. +func (db *DB) GetVault(id string) (*Vault, error) { + rows, err := db.conn.Query("SELECT * FROM vaults WHERE id = ?", id) + if err != nil { + return nil, err + } + defer rows.Close() + + vs, err := scanVaults(rows) + if err != nil { + return nil, err + } + if len(vs) == 0 { + return nil, fmt.Errorf("vault %q not found", id) + } + return &vs[0], nil +} + +// SearchVaults performs search on vaults with optional project filter. +func (db *DB) SearchVaults(query, projectID string) ([]Vault, error) { + where := []string{} + args := []any{} + + if query != "" { + where = append(where, "name LIKE ? OR description LIKE ?") + q := "%" + query + "%" + args = append(args, q, q) + } + if projectID != "" { + where = append(where, "project_id = ?") + args = append(args, projectID) + } + + sql := "SELECT * FROM vaults" + if len(where) > 0 { + sql += " WHERE " + strings.Join(where, " AND ") + } + sql += " ORDER BY name" + + rows, err := db.conn.Query(sql, args...) + if err != nil { + return nil, fmt.Errorf("search vaults: %w", err) + } + defer rows.Close() + + return scanVaults(rows) +} + +// GetProjectVaults returns all vaults belonging to a project. +func (db *DB) GetProjectVaults(projectID string) ([]Vault, error) { + return db.SearchVaults("", projectID) +} + +func scanVaults(rows interface{ Next() bool; Scan(...any) error }) ([]Vault, error) { + var result []Vault + for rows.Next() { + var v Vault + var tagsJSON string + var createdAt, updatedAt string + var sym int + + err := rows.Scan( + &v.ID, &v.Name, &v.ProjectID, &v.Description, &v.Path, &sym, &tagsJSON, + &v.ContentHash, &createdAt, &updatedAt, + ) + if err != nil { + return nil, fmt.Errorf("scanning vault: %w", err) + } + + v.Symlink = sym == 1 + v.Tags = unmarshalStrings(tagsJSON) + v.CreatedAt, _ = time.Parse(time.RFC3339, createdAt) + v.UpdatedAt, _ = time.Parse(time.RFC3339, updatedAt) + + result = append(result, v) + } + return result, nil +} + func scanFunctions(rows interface{ Next() bool; Scan(...any) error }) ([]Function, error) { var result []Function for rows.Next() { diff --git a/registry/validate.go b/registry/validate.go index 1115efa4..8bb34d1d 100644 --- a/registry/validate.go +++ b/registry/validate.go @@ -237,6 +237,29 @@ func ValidateAnalysis(a *Analysis, knownFunctions, knownTypes map[string]bool) * return nil } +// ValidateProject checks integrity rules for projects. +func ValidateProject(p *Project) *ValidationError { + var errs []string + + if p.ID == "" { + errs = append(errs, "id is required") + } + if p.Name == "" { + errs = append(errs, "name is required") + } + if p.Description == "" { + errs = append(errs, "description is required") + } + if p.DirPath != "" && strings.HasPrefix(p.DirPath, "/") { + errs = append(errs, "dir_path must be relative to registry root") + } + + if len(errs) > 0 { + return &ValidationError{ID: p.ID, Errors: errs} + } + return nil +} + // ValidateType checks integrity rules for types. func ValidateType(t *Type, knownTypes map[string]bool) *ValidationError { var errs []string diff --git a/types/infra/deploy_config.md b/types/infra/deploy_config.md new file mode 100644 index 00000000..9b73b0ac --- /dev/null +++ b/types/infra/deploy_config.md @@ -0,0 +1,32 @@ +--- +name: DeployConfig +lang: go +domain: infra +version: "1.0.0" +algebraic: product +definition: "type DeployConfig struct { AppName, LocalDir, RemoteDir, BinaryName, BuildCmd, ServiceUser string; Port int; HealthPath string; Env map[string]string }" +description: "Parametriza un deploy de app a un VPS remoto. Agrupa nombre, rutas, build, servicio, puerto, health check y env vars." +tags: [deploy, config, vps, remote, infra] +uses_types: [] +file_path: "functions/infra/deploy_config.go" +--- + +## Ejemplo + +```go +cfg := DeployConfig{ + AppName: "dag_engine", + LocalDir: "apps/dag_engine", + RemoteDir: "/opt/apps/dag_engine", + BinaryName: "dag_engine", + BuildCmd: "CGO_ENABLED=0 GOOS=linux go build -o dag_engine .", + ServiceUser: "deploy", + Port: 8080, + HealthPath: "/api/health", + Env: map[string]string{"DB_PATH": "/opt/apps/dag_engine/data/ops.db"}, +} +``` + +## Notas + +Usado por los pipelines `setup_vps_app` y `deploy_app_remote`. El campo `BuildCmd` se ejecuta localmente con `bash -c` en el directorio `LocalDir`. Si `HealthPath` está vacío o `Port` es 0, se omite el health check.