chore: sync from fn-registry agent
This commit is contained in:
fn-registry agent
@@ -0,0 +1,123 @@
|
||||
---
|
||||
name: call_monitor
|
||||
lang: go
|
||||
domain: infra
|
||||
description: "Telemetria de invocaciones del agente al fn_registry. Persiste eventos (calls, code_writes, test_runs, e2e_runs_fn, violations, patterns, sessions) en su propia operations.db. Vista agregada function_stats por function_id alimenta el bucle reactivo (proposals automaticas). Issue 0085."
|
||||
tags: [service, telemetry, monitoring, registry, sqlite]
|
||||
uses_functions:
|
||||
- sqlite_open_go_infra
|
||||
- sqlite_apply_migrations_go_infra
|
||||
- audit_copied_code_go_infra
|
||||
- generate_proposals_from_telemetry_go_infra
|
||||
uses_types: []
|
||||
framework: "stdlib"
|
||||
entry_point: "main.go"
|
||||
dir_path: "projects/fn_monitoring/apps/call_monitor"
|
||||
repo_url: ""
|
||||
e2e_checks:
|
||||
- id: build
|
||||
cmd: "CGO_ENABLED=1 go build -tags fts5 -o call_monitor ."
|
||||
timeout_s: 60
|
||||
- id: init
|
||||
cmd: "./call_monitor init --db /tmp/call_monitor_e2e.db"
|
||||
expect_stdout_contains: "ready"
|
||||
- id: status_empty
|
||||
cmd: "./call_monitor status --db /tmp/call_monitor_e2e.db"
|
||||
expect_stdout_contains: "no calls recorded yet"
|
||||
- id: schema_view
|
||||
cmd: "sqlite3 /tmp/call_monitor_e2e.db 'SELECT COUNT(*) FROM function_stats;'"
|
||||
expect_stdout_contains: "0"
|
||||
---
|
||||
|
||||
## Arquitectura
|
||||
|
||||
App Go bajo modulo `fn-registry` (sin `go.mod` propio). Estructura:
|
||||
|
||||
```
|
||||
projects/fn_monitoring/apps/call_monitor/
|
||||
app.md
|
||||
main.go # CLI: init, status
|
||||
db.go # openDB + apply migrations + queries
|
||||
migrations/
|
||||
001_init.sql # 7 tablas event-log
|
||||
002_function_stats_view.sql # vista agregada por function_id
|
||||
operations.db # creada al primer init (gitignored)
|
||||
```
|
||||
|
||||
Reusa `infra.SQLiteOpen` + `infra.ApplyMigrations` del registry (mismo patron que `apps/kanban/backend/db.go`).
|
||||
|
||||
## Tablas event-log (append-only)
|
||||
|
||||
| Tabla | Captura |
|
||||
|---|---|
|
||||
| `sessions` | Sesion Claude Code: session_id, cwd, started_at, ended_at, health_score, mcp_ratio |
|
||||
| `calls` | Cada invocacion al registry (heredoc/mcp/fn_run): function_id, tool_used, duration_ms, success, error_class, args_hash |
|
||||
| `code_writes` | Edit/Write sobre archivo del registry: function_id, file_path, lines_added/removed |
|
||||
| `test_runs` | Unit tests: function_id, test_id, passed, duration_ms, output_snippet |
|
||||
| `e2e_runs_fn` | E2E checks de apps que dependen: function_id, app_id, check_id, passed |
|
||||
| `violations` | Antipatrones (sqlite3 inline, import *, heredoc reinvento): rule_id, function_id, severity |
|
||||
| `patterns` | Heredocs clusterizados por similitud: pattern_hash, occurrences, session_ids[] |
|
||||
| `function_versions` | Historial de versiones por function_id. source = `index` (poblado por `call_monitor snapshot` tras `fn index`), `edit_hook` (poblado por hook PostToolUse), `copy_detected` (futura fase 0085k) |
|
||||
|
||||
Datos sensibles: solo `args_hash`, NUNCA argumentos concretos.
|
||||
|
||||
## Vista `function_stats`
|
||||
|
||||
Rollup por `function_id` con:
|
||||
|
||||
- **Uso**: calls_total, calls_24h/7d/30d/90d, last_used_at
|
||||
- **Errores**: errors_total, error_rate, last_error_ts
|
||||
- **Performance**: mean_duration_ms (p95 pendiente — requires window functions o sub-query)
|
||||
- **Codigo**: writes_count, last_write_at
|
||||
- **Tests**: tests_total, tests_failed, test_fail_rate, last_test_failed_at
|
||||
- **E2E**: e2e_total, e2e_failed, e2e_fail_rate, consumer_apps_count
|
||||
- **Salud**: violations_caused
|
||||
|
||||
Vista O(N) sobre tablas event-log. Si performance degrada en >100k filas, materializar como TABLE refrescada por cron.
|
||||
|
||||
## Uso
|
||||
|
||||
```bash
|
||||
# Build
|
||||
cd projects/fn_monitoring/apps/call_monitor
|
||||
CGO_ENABLED=1 go build -tags fts5 -o call_monitor .
|
||||
|
||||
# Crear/abrir BD (aplica migraciones idempotentemente)
|
||||
./call_monitor init
|
||||
|
||||
# Resumen actual
|
||||
./call_monitor status --top 20
|
||||
|
||||
# Snapshot versions desde registry.db (idempotente, ejecutar tras cada fn index)
|
||||
./call_monitor snapshot
|
||||
./call_monitor snapshot --registry /home/lucas/fn_registry/registry.db
|
||||
|
||||
# BD personalizada
|
||||
./call_monitor init --db /tmp/test.db
|
||||
./call_monitor status --db /tmp/test.db
|
||||
```
|
||||
|
||||
## Integracion con el resto
|
||||
|
||||
| Componente | Como interactua |
|
||||
|---|---|
|
||||
| Hook `PostToolUse` (0085b) | Parsea cada Bash + cada mcp__registry__* y hace `INSERT INTO calls/...` directo sobre operations.db |
|
||||
| Wrapper Python `registry_telemetry` (0085c) | Patcha imports al activar `FN_TELEMETRY=1`, registra calls del heredoc |
|
||||
| `registry_dashboard` (UI) | Lee via `sqlite_api`: nuevo datasource `ops:call_monitor`. Tab "Claude usage" con top funciones, huerfanas, patrones |
|
||||
| `fn-mejorador` (fase 5 bucle reactivo) | Consulta `function_stats` + `patterns` + `violations` para generar proposals con evidencia trazable |
|
||||
| `fn-orquestador` (issue 0069) | Usa `sessions.health_score` como criterio de exito adicional |
|
||||
|
||||
## Roadmap
|
||||
|
||||
- 0085a (este paso): schema + skeleton ✓
|
||||
- 0085b: hook PostToolUse Bash que insert en `calls`/`code_writes`/`violations`
|
||||
- 0085c: wrapper Python con `FN_TELEMETRY=1`
|
||||
- 0085d: anadir datasource a `sqlite_api` + tabs en `registry_dashboard`
|
||||
- 0085e..h: clusterizacion, proposals automaticas, gating
|
||||
- p95 en mean_duration_ms via percentile calc o ext.
|
||||
|
||||
## Notas
|
||||
|
||||
- BD vive **junto al binario** (`<exe_dir>/operations.db`) por defecto, no en el cwd del agente. Hook puede pasar `--db` explicito si conviene.
|
||||
- `operations.db` gitignored — telemetria es local por PC, no se sincroniza.
|
||||
- Sin `repo_url`: aun no se ha hecho `gitea_create_repo`. Se inicializara con `/full-git-push` cuando este la fase 0085b lista para evitar repos vacios.
|
||||
Executable
BIN
Binary file not shown.
@@ -0,0 +1,45 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"fn-registry/functions/infra"
|
||||
)
|
||||
|
||||
// persistCopiedCode runs infra.AuditCopiedCode and INSERT OR IGNOREs results
|
||||
// into the copied_code table. Returns (newly_inserted, total_detected, error).
|
||||
func persistCopiedCode(callDB *DB, registryRoot string) (int, int, error) {
|
||||
entries, err := infra.AuditCopiedCode(registryRoot)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("audit: %w", err)
|
||||
}
|
||||
now := time.Now().UTC().Unix()
|
||||
tx, err := callDB.conn.Begin()
|
||||
if err != nil {
|
||||
return 0, 0, err
|
||||
}
|
||||
stmt, err := tx.Prepare(`INSERT OR IGNORE INTO copied_code
|
||||
(app_file, app_function, registry_id, body_hash, similarity, kind, detected_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`)
|
||||
if err != nil {
|
||||
_ = tx.Rollback()
|
||||
return 0, 0, err
|
||||
}
|
||||
defer stmt.Close()
|
||||
inserted := 0
|
||||
for _, e := range entries {
|
||||
res, err := stmt.Exec(e.AppFile, e.AppFunction, e.RegistryID, e.BodyHash, e.Similarity, e.Kind, now)
|
||||
if err != nil {
|
||||
_ = tx.Rollback()
|
||||
return 0, 0, err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n > 0 {
|
||||
inserted++
|
||||
}
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return 0, 0, err
|
||||
}
|
||||
return inserted, len(entries), nil
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"embed"
|
||||
"fmt"
|
||||
|
||||
"fn-registry/functions/infra"
|
||||
)
|
||||
|
||||
//go:embed migrations/*.sql
|
||||
var migrationsFS embed.FS
|
||||
|
||||
type DB struct{ conn *sql.DB }
|
||||
|
||||
func openDB(path string) (*DB, error) {
|
||||
conn, err := infra.SQLiteOpen(path, "")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := infra.ApplyMigrations(conn, migrationsFS, "migrations/*.sql"); err != nil {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("migrate: %w", err)
|
||||
}
|
||||
return &DB{conn: conn}, nil
|
||||
}
|
||||
|
||||
func (d *DB) Close() error { return d.conn.Close() }
|
||||
|
||||
type tableCount struct {
|
||||
Name string
|
||||
Rows int64
|
||||
}
|
||||
|
||||
func (d *DB) tableCounts() ([]tableCount, error) {
|
||||
tables := []string{"sessions", "calls", "code_writes", "test_runs", "e2e_runs_fn", "violations", "patterns", "function_versions", "copied_code"}
|
||||
out := make([]tableCount, 0, len(tables))
|
||||
for _, t := range tables {
|
||||
var n int64
|
||||
if err := d.conn.QueryRow("SELECT COUNT(*) FROM " + t).Scan(&n); err != nil {
|
||||
return nil, fmt.Errorf("count %s: %w", t, err)
|
||||
}
|
||||
out = append(out, tableCount{Name: t, Rows: n})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
type funcStat struct {
|
||||
FunctionID string
|
||||
CallsTotal int64
|
||||
Calls7d int64
|
||||
ErrorsTotal int64
|
||||
ErrorRate float64
|
||||
MeanDurationMs float64
|
||||
LastUsedAt sql.NullInt64
|
||||
}
|
||||
|
||||
func (d *DB) topFunctions(limit int) ([]funcStat, error) {
|
||||
rows, err := d.conn.Query(`
|
||||
SELECT function_id, calls_total, calls_7d, errors_total, error_rate, mean_duration_ms, last_used_at
|
||||
FROM function_stats
|
||||
ORDER BY calls_total DESC
|
||||
LIMIT ?`, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []funcStat
|
||||
for rows.Next() {
|
||||
var s funcStat
|
||||
if err := rows.Scan(&s.FunctionID, &s.CallsTotal, &s.Calls7d, &s.ErrorsTotal, &s.ErrorRate, &s.MeanDurationMs, &s.LastUsedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, s)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,235 @@
|
||||
// call_monitor: telemetria de invocaciones del agente al fn_registry.
|
||||
// Issue 0085. Persiste eventos en operations.db local. Hook PostToolUse (0085b)
|
||||
// y wrapper Python (0085c) escriben aqui. registry_dashboard lee via sqlite_api.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"text/tabwriter"
|
||||
)
|
||||
|
||||
const defaultDBName = "operations.db"
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
usage()
|
||||
os.Exit(2)
|
||||
}
|
||||
sub := os.Args[1]
|
||||
|
||||
fs := flag.NewFlagSet(sub, flag.ExitOnError)
|
||||
dbPath := fs.String("db", "", "Path to operations.db (default: ./operations.db relative to this binary's directory).")
|
||||
|
||||
switch sub {
|
||||
case "init":
|
||||
fs.Parse(os.Args[2:])
|
||||
runInit(resolveDB(*dbPath))
|
||||
case "status":
|
||||
topN := fs.Int("top", 10, "Top N functions to list.")
|
||||
fs.Parse(os.Args[2:])
|
||||
runStatus(resolveDB(*dbPath), *topN)
|
||||
case "snapshot":
|
||||
registry := fs.String("registry", "", "Path to registry.db (default: walk up from cwd until found).")
|
||||
fs.Parse(os.Args[2:])
|
||||
runSnapshot(resolveDB(*dbPath), *registry)
|
||||
case "copied-code":
|
||||
root := fs.String("root", "", "Path to fn_registry root (default: walk up from cwd until registry.db found).")
|
||||
fs.Parse(os.Args[2:])
|
||||
runCopiedCode(resolveDB(*dbPath), *root)
|
||||
case "propose":
|
||||
root := fs.String("root", "", "Path to fn_registry root (default: walk up from cwd).")
|
||||
dry := fs.Bool("dry-run", false, "Generate drafts without persisting to registry.db.proposals.")
|
||||
fs.Parse(os.Args[2:])
|
||||
runPropose(*root, *dry)
|
||||
case "-h", "--help", "help":
|
||||
usage()
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown subcommand: %s\n\n", sub)
|
||||
usage()
|
||||
os.Exit(2)
|
||||
}
|
||||
}
|
||||
|
||||
func usage() {
|
||||
fmt.Fprintln(os.Stderr, `call_monitor — telemetria de invocaciones del agente al fn_registry
|
||||
|
||||
USO:
|
||||
call_monitor <subcomando> [flags]
|
||||
|
||||
SUBCOMANDOS:
|
||||
init Crea/abre operations.db y aplica migraciones (idempotente).
|
||||
status Resumen: conteo de filas por tabla + top funciones por calls_total.
|
||||
snapshot Lee registry.db.functions y snapshotea (function_id, content_hash) en
|
||||
function_versions con source='index'. Idempotente: solo inserta nuevas tuplas.
|
||||
|
||||
FLAGS GLOBALES:
|
||||
--db PATH Ruta a operations.db (default: ./operations.db junto al binario).
|
||||
--registry PATH (subcomando snapshot) Ruta a registry.db. Default: walk up.
|
||||
|
||||
EJEMPLOS:
|
||||
call_monitor init
|
||||
call_monitor status --top 20
|
||||
call_monitor snapshot
|
||||
call_monitor snapshot --registry /home/lucas/fn_registry/registry.db`)
|
||||
}
|
||||
|
||||
func resolveRegistryDB(override string) string {
|
||||
if override != "" {
|
||||
return override
|
||||
}
|
||||
exe, err := os.Executable()
|
||||
if err != nil {
|
||||
return "registry.db"
|
||||
}
|
||||
dir := filepath.Dir(exe)
|
||||
for {
|
||||
candidate := filepath.Join(dir, "registry.db")
|
||||
if _, err := os.Stat(candidate); err == nil {
|
||||
return candidate
|
||||
}
|
||||
parent := filepath.Dir(dir)
|
||||
if parent == dir {
|
||||
break
|
||||
}
|
||||
dir = parent
|
||||
}
|
||||
return "registry.db"
|
||||
}
|
||||
|
||||
func resolveRegistryRoot(override string) string {
|
||||
if override != "" {
|
||||
return override
|
||||
}
|
||||
exe, err := os.Executable()
|
||||
if err != nil {
|
||||
return "."
|
||||
}
|
||||
dir := filepath.Dir(exe)
|
||||
for {
|
||||
if _, err := os.Stat(filepath.Join(dir, "registry.db")); err == nil {
|
||||
return dir
|
||||
}
|
||||
parent := filepath.Dir(dir)
|
||||
if parent == dir {
|
||||
break
|
||||
}
|
||||
dir = parent
|
||||
}
|
||||
return "."
|
||||
}
|
||||
|
||||
func runCopiedCode(callDBPath, rootOverride string) {
|
||||
db, err := openDB(callDBPath)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: open call_monitor db: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
root := resolveRegistryRoot(rootOverride)
|
||||
if _, err := os.Stat(filepath.Join(root, "registry.db")); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: registry.db not found under %s\n", root)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
inserted, total, err := persistCopiedCode(db, root)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: copied-code: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("copied-code: %d total match(es) detected, %d newly inserted into copied_code.\n", total, inserted)
|
||||
}
|
||||
|
||||
func runSnapshot(callDBPath, registryOverride string) {
|
||||
db, err := openDB(callDBPath)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: open call_monitor db: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
registryPath := resolveRegistryDB(registryOverride)
|
||||
if _, err := os.Stat(registryPath); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: registry.db not found at %s\n", registryPath)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
inserted, seen, err := snapshotFromRegistry(db, registryPath)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: snapshot: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("snapshot: %d new versions inserted (out of %d functions seen with content_hash)\n", inserted, seen)
|
||||
}
|
||||
|
||||
func resolveDB(override string) string {
|
||||
if override != "" {
|
||||
return override
|
||||
}
|
||||
exe, err := os.Executable()
|
||||
if err == nil {
|
||||
return filepath.Join(filepath.Dir(exe), defaultDBName)
|
||||
}
|
||||
return defaultDBName
|
||||
}
|
||||
|
||||
func runInit(path string) {
|
||||
db, err := openDB(path)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer db.Close()
|
||||
abs, _ := filepath.Abs(path)
|
||||
fmt.Printf("call_monitor.operations.db ready: %s\n", abs)
|
||||
}
|
||||
|
||||
func runStatus(path string, topN int) {
|
||||
db, err := openDB(path)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
counts, err := db.tableCounts()
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "table counts: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0)
|
||||
fmt.Fprintln(tw, "=== Tables ===")
|
||||
fmt.Fprintln(tw, "TABLE\tROWS")
|
||||
for _, c := range counts {
|
||||
fmt.Fprintf(tw, "%s\t%d\n", c.Name, c.Rows)
|
||||
}
|
||||
tw.Flush()
|
||||
|
||||
top, err := db.topFunctions(topN)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "top functions: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Println()
|
||||
tw = tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0)
|
||||
fmt.Fprintf(tw, "=== Top %d functions (by calls_total) ===\n", topN)
|
||||
if len(top) == 0 {
|
||||
fmt.Fprintln(tw, "(no calls recorded yet)")
|
||||
tw.Flush()
|
||||
return
|
||||
}
|
||||
fmt.Fprintln(tw, "FUNCTION_ID\tCALLS\tCALLS_7D\tERRORS\tERROR_RATE\tMEAN_MS\tLAST_USED_TS")
|
||||
for _, s := range top {
|
||||
last := ""
|
||||
if s.LastUsedAt.Valid {
|
||||
last = fmt.Sprintf("%d", s.LastUsedAt.Int64)
|
||||
}
|
||||
fmt.Fprintf(tw, "%s\t%d\t%d\t%d\t%.2f\t%.0f\t%s\n",
|
||||
s.FunctionID, s.CallsTotal, s.Calls7d, s.ErrorsTotal, s.ErrorRate, s.MeanDurationMs, last)
|
||||
}
|
||||
tw.Flush()
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
-- call_monitor schema v1.0.0
|
||||
-- Event-log de invocaciones del agente al registry + telemetria asociada.
|
||||
-- Issue 0085. Aditivo. Aplicado via embed.FS al abrir operations.db.
|
||||
|
||||
PRAGMA journal_mode=WAL;
|
||||
PRAGMA foreign_keys=ON;
|
||||
|
||||
-- Sesiones Claude Code. Una por arranque de claude-code.
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
session_id TEXT PRIMARY KEY,
|
||||
cwd TEXT NOT NULL DEFAULT '',
|
||||
started_at INTEGER NOT NULL,
|
||||
ended_at INTEGER,
|
||||
health_score REAL,
|
||||
mcp_ratio REAL,
|
||||
notes TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_started ON sessions(started_at);
|
||||
|
||||
-- Cada invocacion del agente sobre funcion del registry (heredoc/mcp/fn_run).
|
||||
CREATE TABLE IF NOT EXISTS calls (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL DEFAULT '',
|
||||
function_id TEXT NOT NULL DEFAULT '',
|
||||
tool_used TEXT NOT NULL,
|
||||
args_hash TEXT NOT NULL DEFAULT '',
|
||||
duration_ms INTEGER NOT NULL DEFAULT 0,
|
||||
success INTEGER NOT NULL DEFAULT 1,
|
||||
error_class TEXT NOT NULL DEFAULT '',
|
||||
error_snippet TEXT NOT NULL DEFAULT '',
|
||||
ts INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_calls_function ON calls(function_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_calls_session ON calls(session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_calls_ts ON calls(ts);
|
||||
CREATE INDEX IF NOT EXISTS idx_calls_tool ON calls(tool_used);
|
||||
CREATE INDEX IF NOT EXISTS idx_calls_success ON calls(success);
|
||||
|
||||
-- Edit/Write del agente sobre archivos del registry.
|
||||
CREATE TABLE IF NOT EXISTS code_writes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL DEFAULT '',
|
||||
function_id TEXT NOT NULL DEFAULT '',
|
||||
file_path TEXT NOT NULL,
|
||||
lines_added INTEGER NOT NULL DEFAULT 0,
|
||||
lines_removed INTEGER NOT NULL DEFAULT 0,
|
||||
ts INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_code_writes_function ON code_writes(function_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_code_writes_ts ON code_writes(ts);
|
||||
|
||||
-- Test run de unit tests del registry (go test / pytest / etc.).
|
||||
CREATE TABLE IF NOT EXISTS test_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL DEFAULT '',
|
||||
function_id TEXT NOT NULL DEFAULT '',
|
||||
test_id TEXT NOT NULL DEFAULT '',
|
||||
passed INTEGER NOT NULL DEFAULT 1,
|
||||
duration_ms INTEGER NOT NULL DEFAULT 0,
|
||||
output_snippet TEXT NOT NULL DEFAULT '',
|
||||
ts INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_test_runs_function ON test_runs(function_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_test_runs_passed ON test_runs(passed);
|
||||
CREATE INDEX IF NOT EXISTS idx_test_runs_ts ON test_runs(ts);
|
||||
|
||||
-- E2E checks de apps que dependen de una funcion del registry.
|
||||
CREATE TABLE IF NOT EXISTS e2e_runs_fn (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL DEFAULT '',
|
||||
function_id TEXT NOT NULL,
|
||||
app_id TEXT NOT NULL,
|
||||
check_id TEXT NOT NULL,
|
||||
passed INTEGER NOT NULL DEFAULT 1,
|
||||
ts INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_e2e_function ON e2e_runs_fn(function_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_e2e_app ON e2e_runs_fn(app_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_e2e_passed ON e2e_runs_fn(passed);
|
||||
|
||||
-- Antipatrones detectados (sqlite3 inline, import *, heredoc reescribiendo, etc.).
|
||||
CREATE TABLE IF NOT EXISTS violations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL DEFAULT '',
|
||||
rule_id TEXT NOT NULL,
|
||||
function_id TEXT NOT NULL DEFAULT '',
|
||||
command_snippet TEXT NOT NULL DEFAULT '',
|
||||
severity TEXT NOT NULL DEFAULT 'warning',
|
||||
ts INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_violations_rule ON violations(rule_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_violations_function ON violations(function_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_violations_severity ON violations(severity);
|
||||
|
||||
-- Heredocs/snippets clusterizados por similitud (alimenta proposals new_function).
|
||||
CREATE TABLE IF NOT EXISTS patterns (
|
||||
pattern_hash TEXT PRIMARY KEY,
|
||||
representative_snippet TEXT NOT NULL,
|
||||
occurrences INTEGER NOT NULL DEFAULT 1,
|
||||
session_ids_json TEXT NOT NULL DEFAULT '[]',
|
||||
first_seen INTEGER NOT NULL,
|
||||
last_seen INTEGER NOT NULL,
|
||||
proposal_id TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_patterns_occurrences ON patterns(occurrences);
|
||||
CREATE INDEX IF NOT EXISTS idx_patterns_last_seen ON patterns(last_seen);
|
||||
@@ -0,0 +1,100 @@
|
||||
-- Vista agregada por funcion del registry. Lectura O(N) sobre tablas event-log.
|
||||
-- Si performance degrada, materializar como TABLE refrescada por cron.
|
||||
|
||||
DROP VIEW IF EXISTS function_stats;
|
||||
|
||||
CREATE VIEW function_stats AS
|
||||
WITH
|
||||
call_agg AS (
|
||||
SELECT
|
||||
function_id,
|
||||
COUNT(*) AS calls_total,
|
||||
SUM(CASE WHEN ts >= CAST(strftime('%s','now','-1 day') AS INTEGER) THEN 1 ELSE 0 END) AS calls_24h,
|
||||
SUM(CASE WHEN ts >= CAST(strftime('%s','now','-7 days') AS INTEGER) THEN 1 ELSE 0 END) AS calls_7d,
|
||||
SUM(CASE WHEN ts >= CAST(strftime('%s','now','-30 days') AS INTEGER) THEN 1 ELSE 0 END) AS calls_30d,
|
||||
SUM(CASE WHEN ts >= CAST(strftime('%s','now','-90 days') AS INTEGER) THEN 1 ELSE 0 END) AS calls_90d,
|
||||
SUM(CASE WHEN success = 0 THEN 1 ELSE 0 END) AS errors_total,
|
||||
AVG(duration_ms) AS mean_duration_ms,
|
||||
MAX(ts) AS last_used_at,
|
||||
MAX(CASE WHEN success = 0 THEN ts ELSE 0 END) AS last_error_ts
|
||||
FROM calls
|
||||
WHERE function_id != ''
|
||||
GROUP BY function_id
|
||||
),
|
||||
write_agg AS (
|
||||
SELECT function_id, COUNT(*) AS writes_count, MAX(ts) AS last_write_at
|
||||
FROM code_writes
|
||||
WHERE function_id != ''
|
||||
GROUP BY function_id
|
||||
),
|
||||
test_agg AS (
|
||||
SELECT
|
||||
function_id,
|
||||
COUNT(*) AS tests_total,
|
||||
SUM(CASE WHEN passed = 0 THEN 1 ELSE 0 END) AS tests_failed,
|
||||
MAX(CASE WHEN passed = 0 THEN ts ELSE 0 END) AS last_test_failed_at
|
||||
FROM test_runs
|
||||
WHERE function_id != ''
|
||||
GROUP BY function_id
|
||||
),
|
||||
e2e_agg AS (
|
||||
SELECT
|
||||
function_id,
|
||||
COUNT(*) AS e2e_total,
|
||||
SUM(CASE WHEN passed = 0 THEN 1 ELSE 0 END) AS e2e_failed,
|
||||
COUNT(DISTINCT app_id) AS consumer_apps_count
|
||||
FROM e2e_runs_fn
|
||||
GROUP BY function_id
|
||||
),
|
||||
viol_agg AS (
|
||||
SELECT function_id, COUNT(*) AS violations_caused
|
||||
FROM violations
|
||||
WHERE function_id != ''
|
||||
GROUP BY function_id
|
||||
),
|
||||
all_fns AS (
|
||||
SELECT function_id FROM call_agg
|
||||
UNION
|
||||
SELECT function_id FROM write_agg
|
||||
UNION
|
||||
SELECT function_id FROM test_agg
|
||||
UNION
|
||||
SELECT function_id FROM e2e_agg
|
||||
UNION
|
||||
SELECT function_id FROM viol_agg
|
||||
)
|
||||
SELECT
|
||||
f.function_id,
|
||||
COALESCE(c.calls_total, 0) AS calls_total,
|
||||
COALESCE(c.calls_24h, 0) AS calls_24h,
|
||||
COALESCE(c.calls_7d, 0) AS calls_7d,
|
||||
COALESCE(c.calls_30d, 0) AS calls_30d,
|
||||
COALESCE(c.calls_90d, 0) AS calls_90d,
|
||||
COALESCE(c.errors_total, 0) AS errors_total,
|
||||
CASE WHEN COALESCE(c.calls_total, 0) > 0
|
||||
THEN CAST(c.errors_total AS REAL) / c.calls_total
|
||||
ELSE 0 END AS error_rate,
|
||||
COALESCE(c.mean_duration_ms, 0) AS mean_duration_ms,
|
||||
c.last_used_at,
|
||||
CASE WHEN c.last_error_ts > 0 THEN c.last_error_ts END AS last_error_ts,
|
||||
COALESCE(w.writes_count, 0) AS writes_count,
|
||||
w.last_write_at,
|
||||
COALESCE(t.tests_total, 0) AS tests_total,
|
||||
COALESCE(t.tests_failed, 0) AS tests_failed,
|
||||
CASE WHEN COALESCE(t.tests_total, 0) > 0
|
||||
THEN CAST(t.tests_failed AS REAL) / t.tests_total
|
||||
ELSE 0 END AS test_fail_rate,
|
||||
CASE WHEN t.last_test_failed_at > 0 THEN t.last_test_failed_at END AS last_test_failed_at,
|
||||
COALESCE(e.e2e_total, 0) AS e2e_total,
|
||||
COALESCE(e.e2e_failed, 0) AS e2e_failed,
|
||||
CASE WHEN COALESCE(e.e2e_total, 0) > 0
|
||||
THEN CAST(e.e2e_failed AS REAL) / e.e2e_total
|
||||
ELSE 0 END AS e2e_fail_rate,
|
||||
COALESCE(e.consumer_apps_count, 0) AS consumer_apps_count,
|
||||
COALESCE(v.violations_caused, 0) AS violations_caused
|
||||
FROM all_fns f
|
||||
LEFT JOIN call_agg c ON c.function_id = f.function_id
|
||||
LEFT JOIN write_agg w ON w.function_id = f.function_id
|
||||
LEFT JOIN test_agg t ON t.function_id = f.function_id
|
||||
LEFT JOIN e2e_agg e ON e.function_id = f.function_id
|
||||
LEFT JOIN viol_agg v ON v.function_id = f.function_id;
|
||||
@@ -0,0 +1,23 @@
|
||||
-- function_versions: historial de versiones por function_id.
|
||||
-- Fuente principal = `call_monitor snapshot` que lee registry.db.functions.content_hash
|
||||
-- tras cada `fn index`. Edit-hook tambien anota cambios con sha256 del archivo.
|
||||
-- copy_detected = source de la fase 0085k (fn doctor copied-code).
|
||||
--
|
||||
-- PK incluye source para permitir que cada source mantenga su propio espacio de
|
||||
-- hashes (el sha256 del archivo NO coincide con el content_hash canonical del
|
||||
-- registry.db, que incluye metadata adicional).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS function_versions (
|
||||
function_id TEXT NOT NULL,
|
||||
content_hash TEXT NOT NULL,
|
||||
version TEXT NOT NULL DEFAULT '',
|
||||
snapped_at INTEGER NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
lines_added INTEGER NOT NULL DEFAULT 0,
|
||||
lines_removed INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (function_id, content_hash, source)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_fn_versions_function ON function_versions(function_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_fn_versions_snapped_at ON function_versions(snapped_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_fn_versions_source ON function_versions(source);
|
||||
@@ -0,0 +1,20 @@
|
||||
-- copied_code: matches de cuerpos de funcion del registry encontrados en apps.
|
||||
-- Poblado por `call_monitor copied-code` que invoca infra.AuditCopiedCode.
|
||||
-- Cada fila representa un (app_file, app_function, registry_id) sospechoso.
|
||||
-- UNIQUE incluye body_hash para que solo se inserte una vez por (path,fn,id,hash).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS copied_code (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
app_file TEXT NOT NULL,
|
||||
app_function TEXT NOT NULL,
|
||||
registry_id TEXT NOT NULL,
|
||||
body_hash TEXT NOT NULL,
|
||||
similarity REAL NOT NULL DEFAULT 1.0,
|
||||
kind TEXT NOT NULL,
|
||||
detected_at INTEGER NOT NULL,
|
||||
UNIQUE(app_file, app_function, registry_id, body_hash)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_copied_registry ON copied_code(registry_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_copied_kind ON copied_code(kind);
|
||||
CREATE INDEX IF NOT EXISTS idx_copied_detected ON copied_code(detected_at);
|
||||
@@ -0,0 +1,37 @@
|
||||
-- session_capability_growth: vista que mide el ciclo "crear+usar" por sesion.
|
||||
-- Una funcion se considera "creada en sesion X" si el primer code_writes
|
||||
-- registrado para ese function_id (a nivel global) ocurrio dentro de la sesion X.
|
||||
-- Una funcion "usada en sesion X" = >=1 fila en calls con misma sesion y ts >= created_at.
|
||||
--
|
||||
-- Lectura: created_this_session = COUNT(*) WHERE session_id = ?
|
||||
-- used = COUNT(*) WHERE session_id = ? AND calls_in_session > 0
|
||||
-- orphan = COUNT(*) WHERE session_id = ? AND calls_in_session = 0
|
||||
--
|
||||
-- Issue 0086. Aditivo. No reescribe schema previo.
|
||||
|
||||
CREATE VIEW IF NOT EXISTS session_capability_growth AS
|
||||
WITH first_write AS (
|
||||
SELECT
|
||||
cw.function_id,
|
||||
cw.session_id,
|
||||
cw.ts AS created_at
|
||||
FROM code_writes cw
|
||||
WHERE cw.function_id != ''
|
||||
AND cw.ts = (
|
||||
SELECT MIN(ts) FROM code_writes
|
||||
WHERE function_id = cw.function_id
|
||||
)
|
||||
)
|
||||
SELECT
|
||||
fw.session_id,
|
||||
fw.function_id,
|
||||
fw.created_at,
|
||||
(SELECT MIN(c.ts) FROM calls c
|
||||
WHERE c.function_id = fw.function_id
|
||||
AND c.session_id = fw.session_id
|
||||
AND c.ts >= fw.created_at) AS first_call_at,
|
||||
(SELECT COUNT(*) FROM calls c
|
||||
WHERE c.function_id = fw.function_id
|
||||
AND c.session_id = fw.session_id
|
||||
AND c.ts >= fw.created_at) AS calls_in_session
|
||||
FROM first_write fw;
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
+62
@@ -0,0 +1,62 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"fn-registry/functions/infra"
|
||||
)
|
||||
|
||||
// runPropose invokes the proposal generator pipeline and persists drafts
|
||||
// into registry.db.proposals (INSERT OR IGNORE — never overwrites reviewed).
|
||||
func runPropose(rootOverride string, dryRun bool) {
|
||||
root := resolveRegistryRoot(rootOverride)
|
||||
if _, err := os.Stat(rootOverride + "/registry.db"); err != nil && rootOverride != "" {
|
||||
// fall back to walk-up
|
||||
root = resolveRegistryRoot("")
|
||||
}
|
||||
if _, err := os.Stat(root + "/registry.db"); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: registry.db not found under %s\n", root)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
drafts, err := infra.GenerateProposalsFromTelemetry(root)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: generate: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
if len(drafts) == 0 {
|
||||
fmt.Println("propose: no proposal drafts generated (insufficient telemetry or all rules pass).")
|
||||
return
|
||||
}
|
||||
|
||||
// Group by rule for the summary
|
||||
byRule := map[string]int{}
|
||||
for _, d := range drafts {
|
||||
byRule[d.RuleID]++
|
||||
}
|
||||
|
||||
fmt.Printf("propose: %d draft(s) generated\n", len(drafts))
|
||||
for rule, n := range byRule {
|
||||
fmt.Printf(" %s: %d\n", rule, n)
|
||||
}
|
||||
|
||||
if dryRun {
|
||||
fmt.Println("\nDRY RUN — drafts NOT persisted. Sample (first 5):")
|
||||
for i, d := range drafts {
|
||||
if i >= 5 {
|
||||
break
|
||||
}
|
||||
fmt.Printf(" [%s] %s → %s\n", d.RuleID, d.ID, d.Title)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
inserted, total, err := infra.PersistProposalDrafts(root, drafts)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "error: persist: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("\npersisted: %d new, %d total (idempotent — existing IDs skipped)\n", inserted, total)
|
||||
}
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"fn-registry/functions/infra"
|
||||
)
|
||||
|
||||
// snapshotFromRegistry reads registry.db.functions and inserts one row in
|
||||
// function_versions per (function_id, content_hash) tuple with source='index'.
|
||||
// Duplicate rows (same hash for same function from the same source) are
|
||||
// silently ignored — so this can be re-run after every `fn index` to capture
|
||||
// only NEW versions.
|
||||
//
|
||||
// Returns (inserted_rows, total_seen, error).
|
||||
func snapshotFromRegistry(callDB *DB, registryPath string) (int, int, error) {
|
||||
rconn, err := infra.SQLiteOpen(registryPath, "")
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("open registry: %w", err)
|
||||
}
|
||||
defer rconn.Close()
|
||||
|
||||
rows, err := rconn.Query("SELECT id, content_hash, version FROM functions WHERE content_hash != ''")
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("query functions: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
now := time.Now().UTC().Unix()
|
||||
tx, err := callDB.conn.Begin()
|
||||
if err != nil {
|
||||
return 0, 0, err
|
||||
}
|
||||
stmt, err := tx.Prepare(`INSERT OR IGNORE INTO function_versions
|
||||
(function_id, content_hash, version, snapped_at, source, lines_added, lines_removed)
|
||||
VALUES (?, ?, ?, ?, 'index', 0, 0)`)
|
||||
if err != nil {
|
||||
_ = tx.Rollback()
|
||||
return 0, 0, err
|
||||
}
|
||||
defer stmt.Close()
|
||||
|
||||
inserted, seen := 0, 0
|
||||
for rows.Next() {
|
||||
var id, hash, version string
|
||||
if err := rows.Scan(&id, &hash, &version); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return 0, 0, err
|
||||
}
|
||||
seen++
|
||||
res, err := stmt.Exec(id, hash, version, now)
|
||||
if err != nil {
|
||||
_ = tx.Rollback()
|
||||
return 0, 0, err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n > 0 {
|
||||
inserted++
|
||||
}
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return 0, 0, err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return 0, 0, err
|
||||
}
|
||||
return inserted, seen, nil
|
||||
}
|
||||
|
||||
// versionsSummary returns aggregate stats per function_id from function_versions.
|
||||
func (d *DB) versionsSummary(limit int) ([]funcVersionSummary, error) {
|
||||
rows, err := d.conn.Query(`
|
||||
SELECT function_id,
|
||||
COUNT(DISTINCT content_hash) AS versions,
|
||||
MAX(snapped_at) AS last_snapped_at
|
||||
FROM function_versions
|
||||
GROUP BY function_id
|
||||
ORDER BY versions DESC, last_snapped_at DESC
|
||||
LIMIT ?`, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []funcVersionSummary
|
||||
for rows.Next() {
|
||||
var s funcVersionSummary
|
||||
if err := rows.Scan(&s.FunctionID, &s.Versions, &s.LastSnappedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, s)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
type funcVersionSummary struct {
|
||||
FunctionID string
|
||||
Versions int64
|
||||
LastSnappedAt int64
|
||||
}
|
||||
Reference in New Issue
Block a user