feat: externalize apps/analysis to Gitea repos, add analysis table

- Migration 007: repo_url on apps table + analysis table with FTS5
- Analysis struct, parser, CRUD, validation, hash computation
- Selective purge: remote-only apps/analysis preserved across fn index
- CLI: fn app list/clone/pull, fn analysis list/clone/pull
- search/show/list now include analysis results
- Apps removed from git tracking (content lives in Gitea repos)
- .gitkeep for apps/ and analysis/ dirs
- Bash functions: jupyter analysis pipeline, shell utilities
- Browser domain: CDP functions moved from infra to browser

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-01 04:23:51 +02:00
parent 722f29b71b
commit bf1efb2099
111 changed files with 2766 additions and 5043 deletions
+35
View File
@@ -0,0 +1,35 @@
---
name: exit_with_status
kind: function
lang: bash
domain: shell
version: "1.0.0"
purity: pure
signature: "exit_with_status(total_steps: int, ok_steps: int, failed_steps: int) -> int"
description: "Calcula el exit code estandar (0=success, 1=failure, 2=partial) a partir de contadores de pasos. Si failed_steps=0 imprime 0 y sale con 0. Si ok_steps=0 imprime 1 y sale con 1. Si hay ambos imprime 2 y sale con 2."
tags: [execution, status, exit-code, standard]
uses_functions: []
uses_types: []
returns: []
returns_optional: false
error_type: ""
imports: []
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/shell/exit_with_status.sh"
---
## Ejemplo
```bash
source bash/functions/shell/exit_with_status.sh
exit_with_status 5 5 0 # stdout: 0, exit code: 0
exit_with_status 5 0 5 # stdout: 1, exit code: 1
exit_with_status 5 3 2 # stdout: 2, exit code: 2
```
## Notas
Funcion pura: no realiza I/O de sistema, no modifica estado global, no lee variables de entorno. El argumento `total_steps` se recibe para completitud semantica pero la logica solo depende de `ok_steps` y `failed_steps`. El valor se imprime a stdout ademas de usarse como exit code, de modo que el caller puede capturarlo con `$(exit_with_status ...)` o evaluar directamente con `exit_with_status ... ; echo $?`.
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# exit_with_status — calcula el exit code estandar a partir de contadores de pasos
# exit_with_status <total_steps> <ok_steps> <failed_steps>
#
# Calcula el exit code estandar:
# 0 — todos los pasos exitosos (failed_steps == 0)
# 1 — todos los pasos fallaron (ok_steps == 0)
# 2 — resultado parcial (hay ok y failed)
#
# Imprime el codigo a stdout y sale con ese codigo.
exit_with_status() {
local total_steps="$1"
local ok_steps="$2"
local failed_steps="$3"
local code
if [[ "$failed_steps" -eq 0 ]]; then
code=0
elif [[ "$ok_steps" -eq 0 ]]; then
code=1
else
code=2
fi
echo "$code"
return "$code"
}
+36
View File
@@ -0,0 +1,36 @@
---
name: find_free_port
kind: function
lang: bash
domain: shell
version: "1.0.0"
purity: impure
signature: "find_free_port([start_port: int], [end_port: int]) -> int"
description: "Busca el primer puerto TCP libre en un rango dado usando ss y lsof. Retorna el numero de puerto a stdout."
tags: [network, port, shell, utility]
uses_functions: []
uses_types: []
returns: []
returns_optional: false
error_type: "error_go_core"
imports: []
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/shell/find_free_port.sh"
---
## Ejemplo
```bash
source find_free_port.sh
port=$(find_free_port 8888 8899)
echo "Puerto libre: $port"
# Con defaults (8888-8899)
port=$(find_free_port)
```
## Notas
Usa `ss -tln` como primer intento y `lsof` como fallback. Ambos deben confirmar que el puerto no esta en uso. Si ningun puerto esta libre en el rango, sale con exit code 1.
+25
View File
@@ -0,0 +1,25 @@
# find_free_port
# ---------------
# Busca el primer puerto libre en un rango dado.
# Imprime el puerto encontrado a stdout.
# Sale con exit code 1 si no encuentra ninguno.
#
# USO (sourced):
# source find_free_port.sh
# port=$(find_free_port 8888 8899)
find_free_port() {
local start="${1:-8888}"
local end="${2:-8899}"
for ((port=start; port<=end; port++)); do
if ! ss -tln 2>/dev/null | grep -q ":${port} " && \
! lsof -i:"$port" >/dev/null 2>&1; then
echo "$port"
return 0
fi
done
echo "find_free_port: no se encontro puerto libre en rango ${start}-${end}" >&2
return 1
}
@@ -0,0 +1,57 @@
---
name: report_execution_json
kind: function
lang: bash
domain: shell
version: "2.0.0"
purity: pure
signature: "report_execution_json(flow_name: string, status: string, exit_code: int, started_at: string, ended_at: string, duration_ms: int, steps_file: string) -> string"
description: "Genera un JSON de reporte de ejecucion siguiendo el estandar fn-registry (docs/execution_standard.md). Recibe los metadatos del flujo y un archivo TSV con resultados de pasos (columnas: name, action, status, elapsed_ms, output, error). Imprime el JSON completo a stdout. Usa jq si esta disponible, con fallback a printf. Funcion pura: sin efectos secundarios ni I/O adicional."
tags: [execution, json, report, standard, shell, pure]
uses_functions: []
uses_types: []
returns: []
returns_optional: false
error_type: ""
imports: []
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/shell/report_execution_json.sh"
---
## Ejemplo
```bash
source bash/functions/shell/report_execution_json.sh
# Crear archivo de pasos (TSV sin cabecera, 6 columnas)
cat > /tmp/steps.tsv <<'EOF'
check_db command ok 10 exists
backup command ok 450 done
push command error 2540 remote rejected
EOF
report_execution_json \
"backup_db" "partial" 2 \
"2026-04-01T02:00:00Z" "2026-04-01T02:00:03Z" 3000 \
/tmp/steps.tsv
```
Output esperado:
```json
{"name":"backup_db","status":"partial","exit_code":2,"started_at":"2026-04-01T02:00:00Z","ended_at":"2026-04-01T02:00:03Z","duration_ms":3000,"steps_total":3,"steps_ok":2,"steps_failed":1,"steps":[{"name":"check_db","action":"command","status":"ok","elapsed_ms":10,"output":"exists"},{"name":"backup","action":"command","status":"ok","elapsed_ms":450,"output":"done"},{"name":"push","action":"command","status":"error","elapsed_ms":2540,"error":"remote rejected"}]}
```
## Notas
Funcion pura: solo lee el archivo de pasos y escribe JSON a stdout, sin efectos secundarios ni I/O adicional mas alla de leer el steps_file.
Formato TSV: 6 columnas separadas por tabulador real (sin cabecera): name, action, status (ok|error), elapsed_ms, output, error. Los campos output y error pueden estar vacios; se omiten del JSON si no tienen valor, siguiendo el estandar de output estructurado.
El caller es responsable de calcular status (success|failure|partial) y exit_code (0|1|2) segun las reglas del estandar. Ver exit_with_status_bash_shell para esa logica.
Compatibilidad dual: con jq construye el JSON de forma robusta (maneja caracteres especiales y saltos de linea en output/error). Sin jq, el fallback con printf escapa backslash, comillas dobles y caracteres de control basicos.
Puede ejecutarse directamente: `bash report_execution_json.sh <args>`.
@@ -0,0 +1,210 @@
#!/usr/bin/env bash
# report_execution_json — Genera un JSON de reporte de ejecucion siguiendo el estandar fn-registry.
#
# USO (sourced):
# source report_execution_json.sh
# report_execution_json "backup_db" "partial" 2 \
# "2026-04-01T02:00:00Z" "2026-04-01T02:00:03Z" 3000 /tmp/steps.tsv
#
# USO (ejecutado directamente):
# bash report_execution_json.sh "backup_db" "partial" 2 \
# "2026-04-01T02:00:00Z" "2026-04-01T02:00:03Z" 3000 /tmp/steps.tsv
#
# FORMATO steps_file (TSV sin cabecera, 6 columnas):
# name<TAB>action<TAB>status<TAB>elapsed_ms<TAB>output<TAB>error
# Los campos output y error pueden estar vacios.
# status valido: ok | error
#
# NOTA sobre IFS y tabs: bash trata tab como whitespace en IFS y colapsa
# campos vacios consecutivos. Se usa 'cut -f N' para parsear cada columna
# de forma correcta cuando hay campos vacios entre tabs.
report_execution_json() {
local flow_name="$1"
local status="$2"
local exit_code="$3"
local started_at="$4"
local ended_at="$5"
local duration_ms="$6"
local steps_file="$7"
if [[ -z "$flow_name" || -z "$status" || -z "$exit_code" || \
-z "$started_at" || -z "$ended_at" || -z "$duration_ms" || \
-z "$steps_file" ]]; then
echo "report_execution_json: uso: report_execution_json <flow_name> <status> <exit_code> <started_at> <ended_at> <duration_ms> <steps_file>" >&2
return 1
fi
if [[ ! -f "$steps_file" ]]; then
echo "report_execution_json: archivo de pasos no encontrado: $steps_file" >&2
return 1
fi
if command -v jq >/dev/null 2>&1; then
_report_execution_json_jq \
"$flow_name" "$status" "$exit_code" \
"$started_at" "$ended_at" "$duration_ms" "$steps_file"
else
_report_execution_json_printf \
"$flow_name" "$status" "$exit_code" \
"$started_at" "$ended_at" "$duration_ms" "$steps_file"
fi
}
# --- Implementacion con jq ---
_report_execution_json_jq() {
local flow_name="$1" status="$2" exit_code="$3"
local started_at="$4" ended_at="$5" duration_ms="$6" steps_file="$7"
local steps_ok=0 steps_failed=0
local steps_json="[]"
while IFS= read -r line; do
[[ -z "$line" ]] && continue
local s_name s_action s_status s_elapsed s_output s_error
s_name=$(printf '%s' "$line" | cut -f1)
s_action=$(printf '%s' "$line" | cut -f2)
s_status=$(printf '%s' "$line" | cut -f3)
s_elapsed=$(printf '%s' "$line" | cut -f4)
s_output=$(printf '%s' "$line" | cut -f5)
s_error=$(printf '%s' "$line" | cut -f6)
[[ -z "$s_name" ]] && continue
local step_obj
step_obj=$(jq -n \
--arg name "$s_name" \
--arg action "$s_action" \
--arg st "$s_status" \
--argjson ms "${s_elapsed:-0}" \
--arg output "$s_output" \
--arg error "$s_error" \
'{name: $name, action: $action, status: $st, elapsed_ms: $ms}
+ (if $output != "" then {output: $output} else {} end)
+ (if $error != "" then {error: $error} else {} end)')
steps_json=$(printf '%s' "$steps_json" | jq --argjson step "$step_obj" '. + [$step]')
if [[ "$s_status" == "ok" ]]; then
((steps_ok++))
else
((steps_failed++))
fi
done < "$steps_file"
local steps_total=$(( steps_ok + steps_failed ))
jq -n \
--arg name "$flow_name" \
--arg st "$status" \
--argjson exit_code "$exit_code" \
--arg started_at "$started_at" \
--arg ended_at "$ended_at" \
--argjson duration_ms "$duration_ms" \
--argjson total "$steps_total" \
--argjson ok "$steps_ok" \
--argjson failed "$steps_failed" \
--argjson steps "$steps_json" \
'{
name: $name,
status: $st,
exit_code: $exit_code,
started_at: $started_at,
ended_at: $ended_at,
duration_ms: $duration_ms,
steps_total: $total,
steps_ok: $ok,
steps_failed: $failed,
steps: $steps
}'
}
# --- Implementacion con printf (fallback sin jq) ---
_json_escape_rj() {
local s="$1"
s="${s//\\/\\\\}"
s="${s//\"/\\\"}"
s="${s//$'\n'/\\n}"
s="${s//$'\r'/\\r}"
s="${s//$'\t'/\\t}"
printf '%s' "$s"
}
_report_execution_json_printf() {
local flow_name="$1" status="$2" exit_code="$3"
local started_at="$4" ended_at="$5" duration_ms="$6" steps_file="$7"
local steps_ok=0 steps_failed=0
local steps_parts=()
while IFS= read -r line; do
[[ -z "$line" ]] && continue
local s_name s_action s_status s_elapsed s_output s_error
s_name=$(printf '%s' "$line" | cut -f1)
s_action=$(printf '%s' "$line" | cut -f2)
s_status=$(printf '%s' "$line" | cut -f3)
s_elapsed=$(printf '%s' "$line" | cut -f4)
s_output=$(printf '%s' "$line" | cut -f5)
s_error=$(printf '%s' "$line" | cut -f6)
[[ -z "$s_name" ]] && continue
local part
part=$(printf '{"name":"%s","action":"%s","status":"%s","elapsed_ms":%s' \
"$(_json_escape_rj "$s_name")" \
"$(_json_escape_rj "$s_action")" \
"$(_json_escape_rj "$s_status")" \
"${s_elapsed:-0}")
if [[ -n "$s_output" ]]; then
part+=",\"output\":\"$(_json_escape_rj "$s_output")\""
fi
if [[ -n "$s_error" ]]; then
part+=",\"error\":\"$(_json_escape_rj "$s_error")\""
fi
part+="}"
steps_parts+=("$part")
if [[ "$s_status" == "ok" ]]; then
((steps_ok++))
else
((steps_failed++))
fi
done < "$steps_file"
local steps_total=$(( steps_ok + steps_failed ))
local steps_array="["
local first=1
for part in "${steps_parts[@]}"; do
if [[ $first -eq 1 ]]; then
steps_array+="$part"
first=0
else
steps_array+=",$part"
fi
done
steps_array+="]"
printf '{"name":"%s","status":"%s","exit_code":%s,"started_at":"%s","ended_at":"%s","duration_ms":%s,"steps_total":%s,"steps_ok":%s,"steps_failed":%s,"steps":%s}\n' \
"$(_json_escape_rj "$flow_name")" \
"$(_json_escape_rj "$status")" \
"$exit_code" \
"$(_json_escape_rj "$started_at")" \
"$(_json_escape_rj "$ended_at")" \
"$duration_ms" \
"$steps_total" \
"$steps_ok" \
"$steps_failed" \
"$steps_array"
}
# Permitir ejecucion directa (no solo sourced)
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
report_execution_json "$@"
fi
+72
View File
@@ -0,0 +1,72 @@
---
name: run_steps
kind: function
lang: bash
domain: shell
version: "1.0.0"
purity: impure
signature: "run_steps(yaml_file: string, [--strict]) -> string"
description: "Ejecuta pasos de un YAML generico donde cada step tiene action=command. Lee el YAML con yq, ejecuta cada paso secuencialmente con timeout configurable, captura exit code y output, respeta continue_on_error, y al final reporta JSON estandar a stdout via report_execution_json. Sale con exit code 0 (success), 1 (failure) o 2 (partial). Con --strict mapea partial a failure."
tags: [execution, yaml, runner, standard, pipeline]
uses_functions: [exit_with_status_bash_shell, report_execution_json_bash_shell]
uses_types: []
returns: []
returns_optional: false
error_type: "error_go_core"
imports: []
tested: false
tests: []
test_file_path: ""
file_path: "bash/functions/shell/run_steps.sh"
---
## Ejemplo
```yaml
# /tmp/test_flow.yaml
name: test_flow
steps:
- name: check
action: command
command: "echo hello"
- name: list
action: command
command: "ls /tmp"
continue_on_error: true
- name: slow_step
action: command
command: "sleep 2"
timeout_ms: 1000
continue_on_error: true
```
```bash
source bash/functions/shell/run_steps.sh
run_steps /tmp/test_flow.yaml
# Imprime JSON a stdout:
# {
# "name": "test_flow",
# "status": "partial",
# "exit_code": 2,
# "started_at": "2026-04-01T10:00:00Z",
# "ended_at": "2026-04-01T10:00:01Z",
# "duration_ms": 1250,
# "steps_total": 3,
# "steps_ok": 2,
# "steps_failed": 1,
# "steps": [
# {"name": "check", "action": "command", "status": "ok", "elapsed_ms": 10, "output": "hello"},
# {"name": "list", "action": "command", "status": "ok", "elapsed_ms": 15},
# {"name": "slow_step", "action": "command", "status": "error", "elapsed_ms": 1001, "error": "timeout: comando excedio 1000ms"}
# ]
# }
# Sale con exit code 2 (partial)
run_steps /tmp/test_flow.yaml --strict
# Igual pero status="failure" y exit code=1
```
## Notas
Requiere `yq` (v4+, modo expression `-e`) y `jq` (o la implementacion printf de report_execution_json). Solo soporta `action: command` — otros actions se marcan como error del paso. El campo `timeout_ms` por defecto es 30000 (30s); si el comando excede el timeout, `timeout(1)` sale con exit code 124 y se registra el error correspondiente. El output del comando (stdout+stderr combinados) se sanitiza reemplazando tabuladores y saltos de linea por espacios antes de escribir al TSV temporal. Las funciones `exit_with_status` y `report_execution_json` se cargan via `source` desde el mismo directorio que `run_steps.sh`.
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env bash
# run_steps — ejecuta pasos de un YAML generico (action=command)
# run_steps <yaml_file> [--strict]
#
# Lee un YAML con la estructura:
# name: <run_name>
# steps:
# - name: <step_name>
# action: command
# command: "<shell_command>"
# continue_on_error: true|false # opcional, default false
# timeout_ms: 30000 # opcional, default 30000
#
# Para cada paso de action=command:
# - Ejecuta el command con timeout (timeout_ms ms)
# - Captura exit code, stdout+stderr y elapsed time
# - Si falla y continue_on_error=false → aborta
# - Acumula resultados en TSV temporal
#
# Al final:
# - Llama a report_execution_json para generar JSON a stdout
# - Llama a exit_with_status para determinar el exit code
#
# --strict: mapea partial (2) a failure (1) en status y exit code
#
# Requiere: yq, jq (jq opcional si report_execution_json tiene fallback printf)
run_steps() {
local yaml_file="$1"
local strict=0
if [[ "${2:-}" == "--strict" ]]; then
strict=1
fi
# --- validaciones previas ---
if [[ -z "$yaml_file" ]]; then
echo "run_steps: yaml_file requerido" >&2
return 1
fi
if [[ ! -f "$yaml_file" ]]; then
echo "run_steps: archivo '$yaml_file' no existe" >&2
return 1
fi
if ! command -v yq &>/dev/null; then
echo "run_steps: yq no encontrado en PATH" >&2
return 1
fi
# --- cargar funciones del estandar ---
local script_dir
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=bash/functions/shell/exit_with_status.sh
source "$script_dir/exit_with_status.sh"
# shellcheck source=bash/functions/shell/report_execution_json.sh
source "$script_dir/report_execution_json.sh"
# --- leer nombre del run ---
local run_name
run_name=$(yq e '.name // "unnamed"' "$yaml_file")
# --- contar pasos ---
local step_count
step_count=$(yq e '.steps | length' "$yaml_file")
if [[ -z "$step_count" || "$step_count" -eq 0 ]]; then
echo "run_steps: el YAML no tiene steps" >&2
return 1
fi
# --- archivo TSV temporal para acumular resultados ---
# columnas: name<TAB>action<TAB>status<TAB>elapsed_ms<TAB>output<TAB>error
local tsv_file
tsv_file=$(mktemp /tmp/run_steps_XXXXXX.tsv)
# shellcheck disable=SC2064
trap "rm -f '$tsv_file'" EXIT
local ok_steps=0
local failed_steps=0
local started_at
started_at=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
local t_run_start
t_run_start=$(date +%s%3N)
# --- iterar sobre cada paso ---
local i
for (( i=0; i<step_count; i++ )); do
local step_name step_action step_command step_continue step_timeout_ms
step_name=$(yq e ".steps[$i].name // \"step_$i\"" "$yaml_file")
step_action=$(yq e ".steps[$i].action // \"command\"" "$yaml_file")
step_command=$(yq e ".steps[$i].command // \"\"" "$yaml_file")
step_continue=$(yq e ".steps[$i].continue_on_error // false" "$yaml_file")
step_timeout_ms=$(yq e ".steps[$i].timeout_ms // 30000" "$yaml_file")
# convertir timeout de ms a segundos enteros (timeout(1) usa segundos)
local timeout_sec
timeout_sec=$(( (step_timeout_ms + 999) / 1000 ))
# solo soportamos action=command
if [[ "$step_action" != "command" ]]; then
local err_msg="action '$step_action' no soportada (solo command)"
printf '%s\t%s\t%s\t%s\t%s\t%s\n' \
"$step_name" "$step_action" "error" "0" "" "$err_msg" >> "$tsv_file"
failed_steps=$((failed_steps + 1))
if [[ "$step_continue" != "true" ]]; then
echo "run_steps: paso '$step_name' — $err_msg — abortando" >&2
break
fi
continue
fi
if [[ -z "$step_command" ]]; then
local err_msg="campo 'command' vacio en paso '$step_name'"
printf '%s\t%s\t%s\t%s\t%s\t%s\n' \
"$step_name" "command" "error" "0" "" "$err_msg" >> "$tsv_file"
failed_steps=$((failed_steps + 1))
if [[ "$step_continue" != "true" ]]; then
echo "run_steps: $err_msg — abortando" >&2
break
fi
continue
fi
# ejecutar con timeout y capturar output + tiempos
local t_start t_end elapsed_ms step_output step_exit
t_start=$(date +%s%3N)
step_output=$(timeout "$timeout_sec" bash -c "$step_command" 2>&1)
step_exit=$?
t_end=$(date +%s%3N)
elapsed_ms=$(( t_end - t_start ))
# timeout(1) sale con 124 cuando agota el tiempo
local step_error=""
if [[ "$step_exit" -eq 124 ]]; then
step_error="timeout: comando excedio ${step_timeout_ms}ms"
fi
# escapar tabuladores y saltos de linea en el output para el TSV
local safe_output
safe_output=$(printf '%s' "$step_output" | tr '\t' ' ' | tr '\n' ' ')
if [[ "$step_exit" -eq 0 ]]; then
printf '%s\t%s\t%s\t%s\t%s\t%s\n' \
"$step_name" "command" "ok" "$elapsed_ms" "$safe_output" "" >> "$tsv_file"
ok_steps=$((ok_steps + 1))
else
printf '%s\t%s\t%s\t%s\t%s\t%s\n' \
"$step_name" "command" "error" "$elapsed_ms" "$safe_output" "$step_error" >> "$tsv_file"
failed_steps=$((failed_steps + 1))
if [[ "$step_continue" != "true" ]]; then
echo "run_steps: paso '$step_name' fallo (exit $step_exit) — abortando" >&2
break
fi
fi
done
local t_run_end
t_run_end=$(date +%s%3N)
local total_duration_ms=$(( t_run_end - t_run_start ))
local ended_at
ended_at=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
local total_steps=$(( ok_steps + failed_steps ))
# --- calcular status y exit code ---
local status_code
status_code=$(exit_with_status "$total_steps" "$ok_steps" "$failed_steps")
local run_status
case "$status_code" in
0) run_status="success" ;;
1) run_status="failure" ;;
2)
if [[ "$strict" -eq 1 ]]; then
run_status="failure"
status_code=1
else
run_status="partial"
fi
;;
*) run_status="failure"; status_code=1 ;;
esac
# --- generar JSON a stdout ---
report_execution_json \
"$run_name" \
"$run_status" \
"$status_code" \
"$started_at" \
"$ended_at" \
"$total_duration_ms" \
"$tsv_file"
return "$status_code"
}