dataforge/fn_registry
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(doctor): add fn doctor CLI + 14 functions for system management

Browse Source 
Adds `fn doctor` read-only diagnostic command with subcommands artefacts,
services, sync, uses-functions, unused, and --json flag for agents.
Each subcommand wraps a registry function in functions/infra/.

New functions:
- artefact_doctor, services_status, pc_locations_drift,
  audit_uses_functions, find_unused_functions (Go diagnostics)
- backup_sqlite_db, rotate_backups, wait_for_http, wait_for_port,
  port_kill, tail_journal, pre_commit_hook_install (bash utilities)
- notify_telegram (Go HTTP)
- backup_all pipeline (tag launcher)

Plus prior session leftovers (scan_secrets_in_dirty, append_diary_entry,
git utilities, http_session_cookie_middleware, compile/full-git pipelines).

Fixes pc_locations_drift filepath.Join bug with absolute dir_path.
Documents fn doctor in CLAUDE.md, .claude/rules/fn_doctor.md (rule 23),
docs/architecture.md, CHANGELOG.md (2026-05-07), and diary entry.

First fn doctor uses-functions run found drift in 7/12 apps (deuda
para sincronizar app.md con imports reales).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Egutierrez
2026-05-07 01:42:10 +02:00
parent fd9e9135a9
commit 2a3d780347
77 changed files with 6511 additions and 534 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bash/functions/pipelines/backup_all.md
+73
View File
@@ -0,0 +1,73 @@
---
name: backup_all
kind: pipeline
lang: bash
domain: pipelines
version: "1.0.0"
purity: impure
signature: "backup_all(backup_root: string) -> void"
description: "Backup completo del estado del registry: snapshot atomico de registry.db, snapshot de cada operations.db de cada app, y rsync de todos los vaults declarados en vault.yaml. Aplica retention 7/4/12 (daily/weekly/monthly) con rotate_backups. Idempotente, llamable a diario desde cron o systemd-timer."
tags: ["backup", "launcher", "pipeline", "retention"]
uses_functions:
- backup_sqlite_db_bash_infra
- rotate_backups_bash_infra
uses_types: []
returns: []
returns_optional: false
error_type: "error_go_core"
imports: []
params:
- name: backup_root
desc: "Directorio raiz donde se guardan todos los backups (ej. ~/backups/fn_registry). Se crea si no existe."
output: "Linea de resumen a stdout: ISO_timestamp, bytes de registry.db, conteo de ops backupeadas, conteo de vaults sincronizados, errores parciales y segundos transcurridos. Misma linea se hace append en backup_root/backup_log.txt."
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/pipelines/backup_all.sh"
---
## Ejemplo
```bash
# Backup manual a ~/backups/fn_registry
export FN_REGISTRY_ROOT=/home/lucas/fn_registry
backup_all ~/backups/fn_registry
# Salida esperada:
# 2026-05-07T10:30:00+02:00 registry=4194304B ops=3 vaults=2 partial_errors=0 elapsed=12s
# Entrada en crontab (diario a las 02:00)
# 0 2 * * * FN_REGISTRY_ROOT=/home/lucas/fn_registry bash /home/lucas/fn_registry/bash/functions/pipelines/backup_all.sh ~/backups/fn_registry
```
## Estructura de backup_root/
```
registry/
daily.0 daily.1 ... daily.6
weekly.0 ... weekly.3
monthly.0 ... monthly.11
operations/
<app_name>/
daily.0 ... (misma retencion)
vaults/
<vault_name>/
daily.0/ (directorio rsync con hard-links)
daily.1/ ...
backup_log.txt
```
## Codigos de salida
| Codigo | Significado |
|--------|-------------|
| 0 | Exito completo |
| 1 | FN_REGISTRY_ROOT no localizable |
| 2 | backup_root no se puede crear/escribir |
| 3 | Fallo critico en backup de registry.db |
| 4 | Errores parciales en ops o vaults (no critico, continua) |
| 5 | Herramientas del sistema faltantes (sqlite3, rsync, find) |
## Notas
Idempotente: llamar multiples veces el mismo dia solo rota si hay cambios de fecha. Los vaults deducan bloques iguales con rsync --link-dest, por lo que el primer run ocupa el espacio real y los sucesivos solo almacenan los deltas. Operations.db de apps sin dicho archivo se ignoran silenciosamente. Requiere FN_REGISTRY_ROOT seteado o ejecutar desde la raiz del registry.
bash/functions/pipelines/backup_all.sh
+165
View File
@@ -0,0 +1,165 @@
#!/usr/bin/env bash
# backup_all — Backup completo del estado del registry: registry.db, operations.db de cada app, y vaults declarados.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../infra/backup_sqlite_db.sh"
source "$SCRIPT_DIR/../infra/rotate_backups.sh"
backup_all() {
local backup_root="${1:?Arg 1 requerido: backup_root}"
local start_ts
start_ts=$(date +%s)
# --- 1. Localizar FN_REGISTRY_ROOT ---
local registry_root
if [[ -n "${FN_REGISTRY_ROOT:-}" && -f "$FN_REGISTRY_ROOT/registry.db" ]]; then
registry_root="$FN_REGISTRY_ROOT"
elif [[ -f "$(pwd)/registry.db" ]]; then
registry_root="$(pwd)"
else
echo "ERROR: No se puede localizar registry.db. Setea FN_REGISTRY_ROOT o ejecuta desde la raiz del registry." >&2
return 1
fi
# --- 2. Verificar herramientas del sistema ---
local missing_tools=()
for tool in sqlite3 rsync find; do
command -v "$tool" &>/dev/null || missing_tools+=("$tool")
done
if [[ ${#missing_tools[@]} -gt 0 ]]; then
echo "ERROR: Herramientas faltantes: ${missing_tools[*]}" >&2
return 5
fi
# --- 3. Crear backup_root ---
if ! mkdir -p "$backup_root/registry" "$backup_root/operations" "$backup_root/vaults"; then
echo "ERROR: No se puede crear/escribir en $backup_root" >&2
return 2
fi
local log_file="$backup_root/backup_log.txt"
local iso_ts
iso_ts=$(date --iso-8601=seconds)
local ops_count=0
local vaults_count=0
local registry_bytes=0
local partial_errors=0
# --- 4. Backup registry.db ---
local snap_registry="/tmp/registry-snap-$$.db"
if ! backup_sqlite_db "$registry_root/registry.db" "$snap_registry"; then
echo "ERROR critico: Fallo snapshot de registry.db" >&2
rm -f "$snap_registry"
return 3
fi
registry_bytes=$(stat -c%s "$snap_registry" 2>/dev/null || echo 0)
rotate_backups "$backup_root/registry" "$snap_registry" 7 4 12
rm -f "$snap_registry"
# --- 5. Backup operations.db de cada app ---
while IFS= read -r ops_db; do
local app_dir
app_dir="$(dirname "$ops_db")"
local app_name
app_name="$(basename "$app_dir")"
local snap_ops="/tmp/ops-snap-$$-${app_name}.db"
if backup_sqlite_db "$ops_db" "$snap_ops"; then
rotate_backups "$backup_root/operations/$app_name" "$snap_ops" 7 4 12 || ((partial_errors++))
rm -f "$snap_ops"
((ops_count++))
else
echo "WARN: Fallo snapshot de $ops_db — skipped" >&2
rm -f "$snap_ops"
((partial_errors++))
fi
done < <(find "$registry_root/apps" "$registry_root/projects" -name "operations.db" -maxdepth 4 2>/dev/null || true)
# --- 6. Backup vaults via rsync + link-dest ---
local vault_yaml
while IFS= read -r vault_yaml; do
if [[ ! -f "$vault_yaml" ]]; then continue; fi
# Parsear entradas de vault.yaml: buscar pares name/path
local current_name=""
while IFS= read -r line; do
if [[ "$line" =~ ^[[:space:]]*-[[:space:]]*name:[[:space:]]*(.+)$ ]]; then
current_name="${BASH_REMATCH[1]}"
elif [[ "$line" =~ ^[[:space:]]*path:[[:space:]]*(.+)$ && -n "$current_name" ]]; then
local vault_path="${BASH_REMATCH[1]}"
# Expandir ~ si fuera necesario
vault_path="${vault_path/#\~/$HOME}"
if [[ ! -d "$vault_path" ]]; then
echo "WARN: Vault '$current_name' path '$vault_path' no existe — skipped" >&2
((partial_errors++))
current_name=""
continue
fi
local vault_dest="$backup_root/vaults/$current_name"
mkdir -p "$vault_dest"
local link_dest="$vault_dest/daily.1"
local tmp_dest="$vault_dest/daily.0.tmp"
rm -rf "$tmp_dest"
if [[ -d "$link_dest" ]]; then
rsync -a --link-dest="$link_dest" "$vault_path/" "$tmp_dest/"
else
rsync -a "$vault_path/" "$tmp_dest/"
fi
# Rotacion manual de directorios (7 daily, 4 weekly, 12 monthly)
_rotate_vault_dirs "$vault_dest" 7 4 12
mv "$tmp_dest" "$vault_dest/daily.0"
((vaults_count++))
current_name=""
fi
done < "$vault_yaml"
done < <(find "$registry_root/projects" -name "vault.yaml" -maxdepth 4 2>/dev/null || true)
# --- 7. Log y stdout ---
local end_ts elapsed
end_ts=$(date +%s)
elapsed=$(( end_ts - start_ts ))
local summary="${iso_ts} registry=${registry_bytes}B ops=${ops_count} vaults=${vaults_count} partial_errors=${partial_errors} elapsed=${elapsed}s"
echo "$summary" >> "$log_file"
echo "$summary"
if [[ $partial_errors -gt 0 ]]; then
echo "WARN: $partial_errors errores parciales (no criticos). Ver $log_file" >&2
return 4
fi
return 0
}
# Rotacion manual de directorios vault (mismo algoritmo que rotate_backups pero sobre dirs)
_rotate_vault_dirs() {
local dir="$1"
local daily="${2:-7}"
local weekly="${3:-4}"
local monthly="${4:-12}"
# Promover a monthly (del weekly.0 al ultimo monthly)
local week_day
week_day=$(date +%u) # 1=lunes..7=domingo
local month_day
month_day=$(date +%d)
if [[ "$month_day" == "01" && -d "$dir/weekly.0" ]]; then
for ((i=monthly-1; i>=1; i--)); do
[[ -d "$dir/monthly.$((i-1))" ]] && mv "$dir/monthly.$((i-1))" "$dir/monthly.$i"
done
[[ -d "$dir/weekly.0" ]] && cp -al "$dir/weekly.0" "$dir/monthly.0"
fi
if [[ "$week_day" == "7" && -d "$dir/daily.0" ]]; then
for ((i=weekly-1; i>=1; i--)); do
[[ -d "$dir/weekly.$((i-1))" ]] && mv "$dir/weekly.$((i-1))" "$dir/weekly.$i"
done
[[ -d "$dir/daily.0" ]] && cp -al "$dir/daily.0" "$dir/weekly.0"
fi
# Rotar daily
[[ -d "$dir/daily.$((daily-1))" ]] && rm -rf "$dir/daily.$((daily-1))"
for ((i=daily-1; i>=1; i--)); do
[[ -d "$dir/daily.$((i-1))" ]] && mv "$dir/daily.$((i-1))" "$dir/daily.$i"
done
}
backup_all "$@"
bash/functions/pipelines/compile_cpp_app.md
+60
View File
@@ -0,0 +1,60 @@
---
name: compile_cpp_app
kind: pipeline
lang: bash
domain: pipelines
version: "1.0.0"
purity: impure
signature: "compile_cpp_app(app_name?: string) -> void"
description: "Pipeline que resuelve la app C++ desde el nombre o CWD, la cross-compila para Windows con mingw-w64, y despliega el .exe al escritorio de Windows. Composicion de resolve_cpp_app_dir + build_cpp_windows + deploy_cpp_exe_to_windows."
tags: [cpp, compile, windows, mingw, cross-compile, deploy, pipeline]
uses_functions:
- resolve_cpp_app_dir_bash_infra
- build_cpp_windows_bash_infra
- deploy_cpp_exe_to_windows_bash_infra
uses_types: []
returns: []
returns_optional: false
error_type: "error_go_core"
imports: []
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/pipelines/compile_cpp_app.sh"
params:
- name: app_name
desc: "Nombre de la app a compilar (opcional). Sin arg se deduce desde el directorio actual si estamos dentro de cpp/apps/<X>/ o projects/*/apps/<X>/."
output: "Compila el .exe y lo despliega al escritorio de Windows. Imprime progreso por steps a stderr y resumen final con ls -lh del .exe resultante."
---
## Ejemplo
```bash
# Desde dentro del directorio de la app (sin arg)
cd /home/lucas/fn_registry/cpp/apps/chart_demo
fn run compile_cpp_app
# Con nombre explicito desde cualquier directorio
fn run compile_cpp_app registry_dashboard
# Directo
bash bash/functions/pipelines/compile_cpp_app.sh graph_explorer
```
## Flujo
1. `resolve_cpp_app_dir` — deduce nombre y directorio absoluto de la app (desde CWD o arg)
2. Verifica que existe `CMakeLists.txt` en el directorio de la app
3. `build_cpp_windows` — cross-compila con mingw-w64 solo el target de la app
4. `deploy_cpp_exe_to_windows` — copia exe, DLLs, assets, enrichers y runtime al escritorio de Windows
5. Imprime `ls -lh` del exe final en Desktop/apps/<APP>/
## Variables de entorno
- `FN_REGISTRY_ROOT` — raiz del registry; default `/home/lucas/fn_registry`
- `BUILD_WIN` — directorio de build Windows; default `$FN_REGISTRY_ROOT/cpp/build/windows`
- `WIN_DESKTOP_APPS` — directorio destino; default `/mnt/c/Users/lucas/Desktop/apps`
## Notas
Reemplaza la logica del slash command `/compile`. No lleva tag `launcher` porque no es un pipeline TUI-lanzable (tarda minutos en compilar).
bash/functions/pipelines/compile_cpp_app.sh
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Pipeline: compile_cpp_app — Resuelve la app, la cross-compila para Windows y despliega al escritorio.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INFRA_DIR="$SCRIPT_DIR/../infra"
source "$INFRA_DIR/resolve_cpp_app_dir.sh"
source "$INFRA_DIR/build_cpp_windows.sh"
source "$INFRA_DIR/deploy_cpp_exe_to_windows.sh"
compile_cpp_app() {
local app_arg="${1:-}"
# --- Paso 1: Resolver nombre y directorio de la app ---
echo "[1/3] Resolviendo app..." >&2
local resolved
resolved=$(resolve_cpp_app_dir "$app_arg")
local APP APP_DIR
APP="$(echo "$resolved" | cut -f1)"
APP_DIR="$(echo "$resolved" | cut -f2)"
echo " App: $APP" >&2
echo " Dir: $APP_DIR" >&2
# --- Verificar que tiene CMakeLists.txt ---
if [ ! -f "$APP_DIR/CMakeLists.txt" ]; then
echo "ERROR: $APP_DIR/CMakeLists.txt no encontrado." >&2
echo "La app '$APP' no esta registrada con CMake. Ver cpp_apps.md §5." >&2
return 1
fi
# --- Paso 2: Cross-compilar para Windows ---
echo "" >&2
echo "[2/3] Compilando '$APP' para Windows (mingw-w64)..." >&2
build_cpp_windows "$APP"
# --- Paso 3: Desplegar al escritorio de Windows ---
echo "" >&2
echo "[3/3] Desplegando '$APP' al escritorio de Windows..." >&2
deploy_cpp_exe_to_windows "$APP" "$APP_DIR"
# --- Resumen final ---
local root="${FN_REGISTRY_ROOT:-/home/lucas/fn_registry}"
local build_win="${BUILD_WIN:-$root/cpp/build/windows}"
local win_desktop_apps="${WIN_DESKTOP_APPS:-/mnt/c/Users/lucas/Desktop/apps}"
local final_exe="$win_desktop_apps/$APP/$APP.exe"
echo "" >&2
if [ -f "$final_exe" ]; then
echo "===== compile_cpp_app: OK =====" >&2
ls -lh "$final_exe" >&2
else
echo "WARN: no se encuentra $final_exe" >&2
fi
}
compile_cpp_app "$@"
bash/functions/pipelines/full_git_pull.md
+54
View File
@@ -0,0 +1,54 @@
---
name: full_git_pull
kind: pipeline
lang: bash
domain: pipelines
version: "1.0.0"
purity: impure
signature: "full_git_pull() -> stdout: tabla resumen"
description: "Pull automatico de fn_registry + todos los sub-repos locales + submodules + fn sync. Descubre repos locales, stashea dirty trees antes de pullear, hace pull --ff-only, actualiza submodulos del repo principal, pulla ~/.password-store, regenera registry.db con fn index y ejecuta fn sync."
tags: [git, pull, sync, registry, pipeline]
uses_functions:
- discover_git_repos_bash_infra
- git_pull_with_stash_bash_infra
- pass_get_bash_infra
uses_types: []
returns: []
returns_optional: false
error_type: "error_go_core"
imports: []
params: []
output: "tabla resumen por stdout: pull status de cada repo, estado de pass-secrets, submodulos actualizados, resultado de fn index, resultado de fn sync; lista de repos con divergencia o conflicto de stash al final"
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/pipelines/full_git_pull.sh"
---
## Ejemplo
```bash
# Pull completo
fn run full_git_pull
# Directo
bash bash/functions/pipelines/full_git_pull.sh
```
## Flujo
1. `discover_git_repos` — lista repos git locales bajo `$FN_REGISTRY_ROOT`
2. `git_pull_with_stash` — para cada repo: stash si dirty, fetch, pull --ff-only, pop stash
3. `git submodule update --init --recursive` — actualiza submodulos del repo principal
4. `git_pull_with_stash` sobre `~/.password-store` (si existe)
5. `CGO_ENABLED=1 ./fn index` — regenera registry.db
6. `fn sync` — sincroniza proposals, apps, projects, analysis, vaults, pc_locations
## Variables de entorno
- `FN_REGISTRY_ROOT` — raiz del registry; default `/home/lucas/fn_registry`
- `FN_REGISTRY_API`, `REGISTRY_API_TOKEN` — se cargan de `pass registry/*`
## Notas
Solo hace pull fast-forward — nunca rebase ni merge automatico. Los repos con divergencia o conflicto de stash se listan al final del resumen para intervencion manual, pero el pipeline no aborta por ellos. No clona repos faltantes: cada PC tiene el subset que le interesa (clonar manualmente si se necesita uno nuevo). Modo completamente no-interactivo.
bash/functions/pipelines/full_git_pull.sh
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env bash
# Pipeline: full_git_pull — Pull automatico de fn_registry + sub-repos + submodules + fn sync
# Descubre repos locales, stashea dirty trees, hace pull --ff-only, actualiza submodules,
# regenera registry.db y ejecuta fn sync.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INFRA_DIR="$SCRIPT_DIR/../infra"
source "$INFRA_DIR/discover_git_repos.sh"
source "$INFRA_DIR/git_pull_with_stash.sh"
source "$INFRA_DIR/pass_get.sh"
full_git_pull() {
# Resolver raiz del registry
local registry_root="${FN_REGISTRY_ROOT:-/home/lucas/fn_registry}"
cd "$registry_root"
echo "=== full_git_pull: inicio ===" >&2
echo "Registry root: $registry_root" >&2
# --- Paso 1: Descubrir repos ---
echo "" >&2
echo "[1/5] Descubriendo repos git..." >&2
local repos
repos=$(discover_git_repos "$registry_root")
local n_repos
n_repos=$(echo "$repos" | grep -c . || true)
echo " Encontrados: $n_repos repos" >&2
# --- Paso 2: Pull de cada repo ---
echo "" >&2
echo "[2/5] Pullando repos..." >&2
local pull_summary=""
local diverged=()
local conflicts=()
while IFS= read -r repo; do
[[ -z "$repo" ]] && continue
local result
result=$(git_pull_with_stash "$repo" 2>/dev/null || true)
if [[ -n "$result" ]]; then
echo " $result" >&2
pull_summary="$pull_summary"$'\n'" $result"
if [[ "$result" == "[diverged]"* ]]; then
diverged+=("$repo")
elif [[ "$result" == "[stash-conflict]"* ]]; then
conflicts+=("$repo")
fi
fi
done <<< "$repos"
# --- Paso 3: Submodules del repo principal ---
echo "" >&2
echo "[3/5] Actualizando submodulos del repo principal..." >&2
local submodule_summary=" [skip] sin submodulos"
if [[ -f "$registry_root/.gitmodules" ]]; then
local sub_out
sub_out=$(git -C "$registry_root" submodule update --init --recursive 2>&1 | tail -10 || true)
echo "$sub_out" >&2
submodule_summary=" OK: $sub_out"
fi
# --- Paso 3b: Pull de ~/.password-store ---
echo "" >&2
echo "[3b] Pullando ~/.password-store..." >&2
local pass_dir="${PASSWORD_STORE_DIR:-$HOME/.password-store}"
local pass_summary=" [skip] password-store: no encontrado"
if [[ -d "$pass_dir/.git" ]]; then
local pass_result
pass_result=$(git_pull_with_stash "$pass_dir" 2>/dev/null || true)
echo " $pass_result" >&2
pass_summary=" $pass_result"
if [[ "$pass_result" == "[diverged]"* ]]; then
diverged+=("$pass_dir")
elif [[ "$pass_result" == "[stash-conflict]"* ]]; then
conflicts+=("$pass_dir")
fi
fi
# --- Paso 4: Regenerar registry.db ---
echo "" >&2
echo "[4/5] Regenerando registry.db..." >&2
local index_summary=" [skip] fn no encontrado"
local fn_bin="$registry_root/fn"
if [[ -x "$fn_bin" ]]; then
local index_out
index_out=$(CGO_ENABLED=1 "$fn_bin" index 2>&1 | tail -3 || true)
echo "$index_out" >&2
index_summary=" OK: $index_out"
else
echo " [warn] $fn_bin no encontrado — intentando build..." >&2
if command -v go >/dev/null 2>&1; then
CGO_ENABLED=1 go build -tags fts5 -o "$fn_bin" "$registry_root/cmd/fn/" 2>&1 >&2 || true
if [[ -x "$fn_bin" ]]; then
local index_out
index_out=$(CGO_ENABLED=1 "$fn_bin" index 2>&1 | tail -3 || true)
echo "$index_out" >&2
index_summary=" OK (post-build): $index_out"
fi
fi
fi
# --- Paso 5: fn sync ---
echo "" >&2
echo "[5/5] Ejecutando fn sync..." >&2
local sync_summary=" [skip] fn sync: credenciales no disponibles"
if [[ -x "$fn_bin" ]]; then
local api_user api_pass api_token
api_user=$(pass_get registry/basicauth-user | head -n1 2>/dev/null || true)
api_pass=$(pass_get registry/basicauth-pass | head -n1 2>/dev/null || true)
api_token=$(pass_get registry/api-token | head -n1 2>/dev/null || true)
if [[ -n "$api_user" && -n "$api_pass" && -n "$api_token" ]]; then
export FN_REGISTRY_API="https://${api_user}:${api_pass}@registry.organic-machine.com"
export REGISTRY_API_TOKEN="$api_token"
local sync_out
sync_out=$("$fn_bin" sync 2>&1) && {
sync_summary=" OK: $sync_out"
} || {
sync_summary=" [error] fn sync: $sync_out"
}
echo " $sync_summary" >&2
else
echo " [warn] Credenciales registry no disponibles — omitiendo fn sync" >&2
fi
fi
# --- Resumen ---
echo ""
echo "===== RESUMEN full_git_pull ====="
echo ""
echo "Pull status por repo:"
if [[ -n "$pull_summary" ]]; then
echo "$pull_summary"
else
echo " (ninguno)"
fi
echo ""
echo "pass-secrets:"
echo "$pass_summary"
echo ""
echo "Submodulos:"
echo "$submodule_summary"
echo ""
echo "fn index:"
echo "$index_summary"
echo ""
echo "fn sync:"
echo "$sync_summary"
if [[ ${#diverged[@]} -gt 0 || ${#conflicts[@]} -gt 0 ]]; then
echo ""
echo "ATENCION — Repos que requieren intervencion manual:"
for r in "${diverged[@]+"${diverged[@]}"}"; do
echo " [diverged] $r → git rebase o git merge manual"
done
for r in "${conflicts[@]+"${conflicts[@]}"}"; do
echo " [stash-conflict] $r → resolver conflicto y git stash drop"
done
fi
echo ""
echo "================================="
}
full_git_pull "$@"
bash/functions/pipelines/full_git_push.md
+64
View File
@@ -0,0 +1,64 @@
---
name: full_git_push
kind: pipeline
lang: bash
domain: pipelines
version: "1.0.0"
purity: impure
signature: "full_git_push(commit_message?: string) -> stdout: tabla resumen"
description: "Push automatico de fn_registry + todos los sub-repos + fn sync. Descubre repos, escanea secrets (aborta si detecta), auto-inicializa apps/analyses sin .git via ensure_repo_synced, auto-commitea dirty trees, pushea solo repos adelantados, pushea ~/.password-store sin commitear, y ejecuta fn sync."
tags: [git, push, sync, registry, pipeline]
uses_functions:
- discover_git_repos_bash_infra
- scan_secrets_in_dirty_bash_cybersecurity
- git_auto_commit_dirty_bash_infra
- git_push_if_ahead_bash_infra
- ensure_repo_synced_bash_infra
- pass_get_bash_infra
uses_types: []
returns: []
returns_optional: false
error_type: "error_go_core"
imports: []
params:
- name: commit_message
desc: "mensaje de commit fijo para todos los repos (opcional); si se omite, cada repo recibe un mensaje generado automaticamente segun sus cambios"
output: "tabla resumen por stdout: commits creados por repo, push status de cada repo, estado de pass-secrets, resultado de fn sync"
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/pipelines/full_git_push.sh"
---
## Ejemplo
```bash
# Push con mensaje automatico
fn run full_git_push
# Push con mensaje fijo para todos los repos
fn run full_git_push "chore: sync desde home-wsl"
# Directo
bash bash/functions/pipelines/full_git_push.sh "feat: nueva funcion"
```
## Flujo
1. `discover_git_repos` — lista todos los repos bajo `$FN_REGISTRY_ROOT`
2. Auto-init — para cada app/analysis sin `.git`, llama `ensure_repo_synced` (requiere `GITEA_URL`/`GITEA_TOKEN` via `pass_get`)
3. `scan_secrets_in_dirty` — escanea cada repo; si hay matches **aborta todo** y lista los archivos
4. `git_auto_commit_dirty` — commitea dirty trees con mensaje fijo o generado
5. `git_push_if_ahead` — pushea solo repos con commits locales (sin tocar la red para los up-to-date)
6. Push de `~/.password-store` — solo push (sin commit; pass se autocommitea)
7. `fn sync` — sincroniza proposals, apps, projects, analysis, vaults, pc_locations con registry_api
## Variables de entorno
- `FN_REGISTRY_ROOT` — raiz del registry; default `/home/lucas/fn_registry`
- `GITEA_URL`, `GITEA_TOKEN` — se cargan de `pass agentes/gitea-url` y `pass gitea/dataforge-git-token`
- `FN_REGISTRY_API`, `REGISTRY_API_TOKEN` — se cargan de `pass registry/*`
## Notas
El unico motivo para abortar antes de commitear es la deteccion de secrets. Cualquier otro error (push rechazado por non-fast-forward, fn sync no disponible) se reporta en el resumen y el pipeline continua con el resto de repos. Modo completamente no-interactivo.
bash/functions/pipelines/full_git_push.sh
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/env bash
# Pipeline: full_git_push — Push automatico de fn_registry + todos los sub-repos + fn sync
# Descubre repos, escanea secrets, auto-commitea dirty trees, pushea solo los ahead,
# pushea ~/.password-store, y ejecuta fn sync para sincronizar metadata no regenerable.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INFRA_DIR="$SCRIPT_DIR/../infra"
CYBERSEC_DIR="$SCRIPT_DIR/../cybersecurity"
source "$INFRA_DIR/discover_git_repos.sh"
source "$INFRA_DIR/git_auto_commit_dirty.sh"
source "$INFRA_DIR/git_push_if_ahead.sh"
source "$INFRA_DIR/pass_get.sh"
source "$INFRA_DIR/ensure_repo_synced.sh"
source "$CYBERSEC_DIR/scan_secrets_in_dirty.sh"
full_git_push() {
local commit_message="${1:-}"
# Resolver raiz del registry
local registry_root="${FN_REGISTRY_ROOT:-/home/lucas/fn_registry}"
cd "$registry_root"
echo "=== full_git_push: inicio ===" >&2
echo "Registry root: $registry_root" >&2
# --- Paso 1: Descubrir repos ---
echo "" >&2
echo "[1/6] Descubriendo repos git..." >&2
local repos
repos=$(discover_git_repos "$registry_root")
# --- Paso 1b: Auto-inicializar apps/analyses sin .git ---
echo "" >&2
echo "[1b] Verificando apps/analyses sin git..." >&2
local gitea_url gitea_token
gitea_url=$(pass_get agentes/gitea-url | head -n1 2>/dev/null || true)
gitea_token=$(pass_get gitea/dataforge-git-token | head -n1 2>/dev/null || true)
if [[ -n "$gitea_url" && -n "$gitea_token" ]]; then
export GITEA_URL="$gitea_url"
export GITEA_TOKEN="$gitea_token"
export FN_REGISTRY_INFRA_DIR="$INFRA_DIR"
local missing_dirs=()
for pattern in "apps/*/" "analysis/*/" "projects/*/apps/*/" "projects/*/analysis/*/"; do
while IFS= read -r d; do
d="${d%/}"
if [[ -d "$d" && ! -d "$d/.git" ]]; then
missing_dirs+=("$d")
fi
done < <(find "$registry_root" -maxdepth 4 -type d -name "$(basename "$pattern")" 2>/dev/null | grep -E "$pattern" || true)
done
# Forma mas directa: iterar directorios conocidos
for pattern in apps analysis; do
if [[ -d "$registry_root/$pattern" ]]; then
for d in "$registry_root/$pattern"/*/; do
d="${d%/}"
[[ -d "$d" ]] || continue
[[ -d "$d/.git" ]] && continue
echo " auto-init: $d" >&2
ensure_repo_synced "$d" dataforge "$(basename "$d")" master "chore: initial sync" || \
echo " [warn] fallo inicializando $d" >&2
done
fi
done
for proj in "$registry_root"/projects/*/; do
for subdir in apps analysis; do
[[ -d "$proj$subdir" ]] || continue
for d in "$proj$subdir"/*/; do
d="${d%/}"
[[ -d "$d" ]] || continue
[[ -d "$d/.git" ]] && continue
echo " auto-init: $d" >&2
ensure_repo_synced "$d" dataforge "$(basename "$d")" master "chore: initial sync" || \
echo " [warn] fallo inicializando $d" >&2
done
done
done
else
echo " [skip] GITEA_URL/GITEA_TOKEN no disponibles — omitiendo auto-init" >&2
fi
# Redescubrir repos tras posibles inicializaciones
repos=$(discover_git_repos "$registry_root")
# --- Paso 2: Escanear secrets ---
echo "" >&2
echo "[2/6] Escaneando secrets en dirty trees..." >&2
local secret_matches=""
while IFS= read -r repo; do
[[ -z "$repo" ]] && continue
local matches
matches=$(scan_secrets_in_dirty "$repo" 2>/dev/null || true)
if [[ -n "$matches" ]]; then
secret_matches="$secret_matches"$'\n'"--- $repo ---"$'\n'"$matches"
fi
done <<< "$repos"
if [[ -n "$secret_matches" ]]; then
echo "" >&2
echo "ABORTANDO: archivos sospechosos detectados antes de commitear:" >&2
echo "$secret_matches" >&2
echo "" >&2
echo "Gestiona esos archivos (.gitignore, mover, o decidir si entran) y reintenta." >&2
return 1
fi
echo " OK: sin archivos sospechosos" >&2
# --- Paso 3: Auto-commitear dirty trees ---
echo "" >&2
echo "[3/6] Auto-commiteando dirty trees..." >&2
local commits_summary=""
while IFS= read -r repo; do
[[ -z "$repo" ]] && continue
local subject
subject=$(git_auto_commit_dirty "$repo" "$commit_message" 2>/dev/null || true)
if [[ -n "$subject" ]]; then
local repo_name
repo_name="$(basename "$repo")"
echo " commit: $repo_name — $subject" >&2
commits_summary="$commits_summary"$'\n'" $repo_name: $subject"
fi
done <<< "$repos"
# --- Paso 4: Push de repos con commits locales ---
echo "" >&2
echo "[4/6] Pusheando repos adelantados..." >&2
local push_summary=""
while IFS= read -r repo; do
[[ -z "$repo" ]] && continue
local status_line
status_line=$(git_push_if_ahead "$repo" 2>/dev/null || true)
if [[ -n "$status_line" ]]; then
echo " $status_line" >&2
push_summary="$push_summary"$'\n'" $status_line"
fi
done <<< "$repos"
# --- Paso 5: Push de ~/.password-store (sin commitear) ---
echo "" >&2
echo "[5/6] Verificando ~/.password-store..." >&2
local pass_dir="${PASSWORD_STORE_DIR:-$HOME/.password-store}"
local pass_summary=" [skip] password-store: no encontrado"
if [[ -d "$pass_dir/.git" ]]; then
local pass_dirty
pass_dirty=$(git -C "$pass_dir" status --porcelain | wc -l)
if [[ "$pass_dirty" -gt 0 ]]; then
echo " [warn] ~/.password-store tiene cambios sin commitear; pass debe commitear solo. Saltando push." >&2
pass_summary=" [warn] password-store: dirty (pass no commiteo)"
else
local pass_status
pass_status=$(git_push_if_ahead "$pass_dir" 2>/dev/null || true)
echo " $pass_status" >&2
pass_summary=" $pass_status"
fi
fi
# --- Paso 6: fn sync ---
echo "" >&2
echo "[6/6] Ejecutando fn sync..." >&2
local sync_summary=" [skip] fn sync: credenciales no disponibles"
local fn_bin="$registry_root/fn"
if [[ -x "$fn_bin" ]]; then
local api_user api_pass api_token
api_user=$(pass_get registry/basicauth-user | head -n1 2>/dev/null || true)
api_pass=$(pass_get registry/basicauth-pass | head -n1 2>/dev/null || true)
api_token=$(pass_get registry/api-token | head -n1 2>/dev/null || true)
if [[ -n "$api_user" && -n "$api_pass" && -n "$api_token" ]]; then
export FN_REGISTRY_API="https://${api_user}:${api_pass}@registry.organic-machine.com"
export REGISTRY_API_TOKEN="$api_token"
local sync_out
sync_out=$("$fn_bin" sync 2>&1) && {
sync_summary=" OK: $sync_out"
} || {
sync_summary=" [error] fn sync: $sync_out"
}
echo " $sync_summary" >&2
else
echo " [warn] Credenciales registry no disponibles — omitiendo fn sync" >&2
fi
else
echo " [warn] $fn_bin no encontrado — omitiendo fn sync" >&2
fi
# --- Resumen ---
echo ""
echo "===== RESUMEN full_git_push ====="
echo ""
echo "Commits creados:"
if [[ -n "$commits_summary" ]]; then
echo "$commits_summary"
else
echo " (ninguno)"
fi
echo ""
echo "Push status:"
if [[ -n "$push_summary" ]]; then
echo "$push_summary"
else
echo " (ninguno)"
fi
echo ""
echo "pass-secrets:"
echo "$pass_summary"
echo ""
echo "fn sync:"
echo "$sync_summary"
echo ""
echo "================================="
}
full_git_push "$@"