Commit Graph

70 Commits

Author SHA1 Message Date
egutierrez c29428a187 feat(viz): graph_types modelo extendido + EntityType/RelationType + flags (issue 0049e)
Extiende el modelo agnostico de graph_types.h para soportar shapes/iconos/
filtros/labels/streaming sin acoplar a backend. Migra el unico consumer
(demos_graph) en el mismo cambio.

- GraphNode v2: type_id + shape_override/color_override/size_override +
  flags (NF_PINNED/VISIBLE/SELECTED/HOVERED) + label_idx + user_data.
- GraphEdge v2: type_id + style_override + flags (EF_DIRECTED/VISIBLE).
- EntityType / RelationType: tablas en GraphData (types, rel_types).
- Helpers de resolucion (resolve_node_color/shape/size, resolve_edge_*)
  y constructores ergonomicos (graph_node, graph_edge, entity_type,
  relation_type) — sentinel-based para herencia automatica del tipo.
- graph_renderer v1.4: lee NF_VISIBLE / EF_VISIBLE, resuelve apariencia
  via override → EntityType → fallback indexado por type_id. Skipea
  aristas con endpoints invisibles. Shapes siguen pintandose como
  circulo (0049f cableara el dispatch real).
- graph_force_layout v1.2: pinned ahora vive en flags & NF_PINNED.
- graph_viewport v1.1: hover/seleccion publican NF_HOVERED/SELECTED en
  el grafo (clear-then-set). Drag usa NF_PINNED. Tooltip muestra Type/
  user_data en lugar de community/value/label.
- demos_graph: 8 EntityType (paleta antigua) + 1 RelationType. type_id
  por cluster. user_data = indice numerico del nodo. Apariencia visual
  identica al pre-cambio.
- test_graph_types.cpp: 12 casos cubriendo helpers, defaults, bitmask
  manipulation y resoluciones override-vs-EntityType. test_graph_edge_
  static actualizado al nuevo modelo (ya no tiene .color directo).
- 4 .md de tipos nuevos (graph_node, graph_edge, entity_type,
  relation_type) + GraphData v2.0 actualizado.

Tests: 31/31 ctest verdes (incluye test_visual golden).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 22:44:40 +02:00
egutierrez daf491cd99 perf(viz): graph_renderer edges via TBO + vertex pulling (issue 0049d)
El buffer de aristas pasa a estatico (16B/arista: source, target, color,
flags) y solo se reupload cuando cambia el grafo. Las posiciones de los
nodos viven en un Texture Buffer Object (RG32F) actualizado por frame; el
vertex shader hace texelFetch con gl_VertexID & 1 para elegir endpoint.
Draw call: glDrawArraysInstanced(GL_LINES, 0, 2, edge_count) con divisor=1.

Para 100k aristas: el upload de 4.8 MB/frame baja a 0 en regimen estable.
edge_alpha pasa a uniform; la pre-multiplicacion en CPU desaparece. GLSL
sigue en 330 core (samplerBuffer/texelFetch estan en 1.40+).

gl_loader gana glBufferSubData, glVertexAttribIPointer y glTexBuffer (en
Linux ya estaban via GL_GLEXT_PROTOTYPES; ahora estan disponibles tambien
en MinGW/Windows).

Tests: nuevo test_graph_edge_static valida el layout de 16B y el packing
RGBA8 del fallback. test_visual sigue verde — render visualmente identico.

Bump graph_renderer 1.2.0 -> 1.3.0.
2026-04-29 22:32:38 +02:00
egutierrez 02b4141cc1 perf(viz): graph_renderer Tier 1 (RGBA8 + orphan + frustum cull) + force_layout auto-pause helper
Issue 0049c. Tres optimizaciones internas en graph_renderer.cpp + un
helper puro en graph_force_layout para detectar convergencia. API publica
intacta — solo cambian el layout interno de los buffers, el shader y
los costes por frame.

1. RGBA8 color packing
   - El instance buffer de nodos pasa de (x,y,size,r,g,b,a) 28B a
     (x,y,size,color_u32) 16B (-43%). Aristas: 24B → 12B/vertex (-50%).
   - Shaders desempaquetan con bit shifts (compatible GL 3.30+, no
     necesita unpackUnorm4x8 que es 4.20+).
   - Helpers expuestos: pack_rgba8 / unpack_rgba8 / modulate_alpha_rgba8
     en graph_renderer.h. Los GraphNode.color y la paleta ya tenian el
     layout correcto (R en LSB), asi que CPU ahora pasa el uint32 directo
     sin convertir a 4 floats por nodo y por frame.

2. Capacity-tracked streaming buffers
   - Sustituye el doble glBufferData de antes por:
       glBufferData(NULL, capacity, STREAM_DRAW)   // orphan + reserva
       glBufferSubData(0, used_bytes, data)        // solo lo usado
   - capacity crece x2 cuando hace falta (inicial 4096 nodos /
     8192 vertices de aristas) → reallocaciones en O(log N).
   - Staging CPU (NodeInstance* / EdgeVertex*) reusado entre frames con
     realloc, no malloc/free per frame.

3. Frustum cull (CPU-side)
   - AABB del viewport en world coords con margen 10%.
   - Aristas: skip si AABB del segmento no intersecta el viewport.
   - Nodos: solo los visibles entran al instance buffer; visible_count
     es el N que pasa a glDrawArraysInstanced. Pop-in de borde mitigado
     por el margen.

4. graph_force_layout_should_pause(low_frames, min_consecutive)
   - Helper puro: el caller mantiene el contador, la funcion solo
     decide si parar. Reemplaza la rama inline en demos_graph.cpp.
   - Test Catch2 con secuencias artificiales.

Tests: test_graph_pack_rgba8 (16401 asserts, 4 cases — roundtrip exhaustivo
+ alpha modulation + clamp). test_graph_should_pause (3 cases, 14 asserts).
Los 29 tests del cpp/tests/ siguen verdes (incluido test_visual con goldens).

Bump versiones:
- graph_renderer 1.1.0 → 1.2.0
- graph_force_layout 1.0.0 → 1.1.0  (tested: true via should_pause test)
2026-04-29 22:17:13 +02:00
egutierrez 492e6b59cd feat(framework): bump OpenGL 3.3 → 4.3 core context
Cierra 0049b. El context de fn::run_app pide ahora GL 4.3 core con
forward-compat global, habilitando compute shaders, SSBOs, image
load/store y atomic counters — bloques esenciales del graph_renderer GPU
del proyecto osint_graph (issues 0049f y 0049h).

Cambios:

- cpp/framework/app_base.cpp: 4.3 core + forward-compat. Comentario
  marcando que es backward-compatible con shaders #version 330.
- cpp/apps/primitives_gallery/capture.cpp: deja explicitamente 3.3 core
  porque WSL Mesa no entrega 4.3 offscreen (GLXBadFBConfig); ImGui +
  ImPlot funcionan igual en 3.3 para los goldens.
- primitives_gallery: nuevo demo Gfx > gl_info que muestra
  Vendor/Renderer/Version/GLSL en runtime + status 4.3 (verde) +
  limites (MAX_TEXTURE_SIZE, MAX_VERTEX_ATTRIBS, MAX_UNIFORM_BLOCK_SIZE
  y, si 4.3+, MAX_SHADER_STORAGE_BUFFER_BINDINGS y compute shared mem).
  Solo glGetString/glGetIntegerv — sin loader extra.
- About bumped a 0.4.0 con la nota del nuevo demo y de GL 4.3.
- cpp/tests/test_visual.cpp: usa LIBGL_ALWAYS_SOFTWARE=1 al lanzar el
  capture para alinear el driver con update_goldens.sh; sin esto las
  diferencias de strings (llvmpipe vs d3d12) hacen que gl_info supere
  el 1% de tolerancia.
- cpp/tests/golden/gl_info.png: nuevo golden.

Build verificado en Linux (cmake build OK) + Windows cross-compile
(cmake build OK). Las 27 pruebas pasan (incluida test_visual con 42
demos comparadas).
2026-04-29 21:23:15 +02:00
egutierrez 4f25564132 feat(projects): osint_graph project + graph_explorer sub-repo bootstrap
Cierra 0049a. Estructura local en projects/osint_graph/ (gitignored):

- project.md con frontmatter (name, description, tags).
- vaults/vault.yaml + symlink osint_data → ~/vaults/osint_graph/{raw,
  processed,exports}.
- apps/graph_explorer/ inicializado como sub-repo Gitea
  (dataforge/graph_explorer, branch master) con commit vacio.

Indexado verificado: 4 projects, 2 vaults; fn show osint_graph OK.
2026-04-29 21:08:47 +02:00
egutierrez f8f72e4bf7 chore(issues): plan 0049 OSINT graph viewer multi-issue
Aggregates the planning artifacts for the 0049 series (umbrella + 0049a..0049k):

- New rule cpp_apps.md (registered in INDEX) — standardize structure, CMake
  patterns, app.md frontmatter and sub-repo for C++ apps; points to the
  authoritative cpp/PATTERNS.md and cpp/DESIGN_SYSTEM.md.
- Feature flag osint_graph_v1 (disabled until 0049k closes).
- Issue 0049 (umbrella) and sub-issues 0049b..0049k describing the GPU
  rendering system, force-layout, types, sources, labels and the final
  graph_explorer app integration.
- README updated with the new rows (all pending; 0049a will flip to
  completed in the next commit).
2026-04-29 21:08:36 +02:00
egutierrez 5851ac97cd docs(issues): cerrar 0046 — actualizar README 2026-04-29 00:31:10 +02:00
egutierrez d3cf6d2774 docs: cerrar issue 0046 2026-04-29 00:29:54 +02:00
egutierrez 32927c5e9f docs(issues): cerrar 0048 — actualizar README 2026-04-29 00:20:13 +02:00
egutierrez 2ba71c6d79 docs: cerrar issue 0048 2026-04-29 00:18:58 +02:00
egutierrez 98e134c935 docs(issues): cerrar 0043 — actualizar README 2026-04-29 00:10:25 +02:00
egutierrez ab3f1d442a docs: cerrar issue 0043 2026-04-29 00:08:56 +02:00
egutierrez 98b7b460b8 docs(issues): cerrar 0045 — actualizar README 2026-04-29 00:00:56 +02:00
egutierrez 5a25095a54 docs: cerrar issue 0045 2026-04-28 23:59:08 +02:00
egutierrez 6430e993c6 docs(issues): cerrar 0041, 0042, 0044, 0047 + actualizar README 2026-04-28 23:45:40 +02:00
egutierrez 08e2b1280f merge: issue/0042-cpp-layout-storage-public — implementación paralela 2026-04-28 23:44:44 +02:00
egutierrez cdba3869b2 docs: cerrar issue 0042 2026-04-28 23:41:19 +02:00
egutierrez a0c0aa4f18 docs(issues): añadir 0041-0048 — refactor C++ apps, tests, primitives standarization 2026-04-28 23:38:56 +02:00
egutierrez fa1050841d docs(issues): añadir 0037-0040 — extraccion de entidades y relaciones
- 0037: IoC regex extractor (IP, email, dominio, hash, wallet, CVE, MAC)
- 0038: GLiNER entity extractor (zero-shot NER multilingue)
- 0039: GLiREL relation extractor (zero-shot triplets)
- 0040: pipeline hibrido extraccion grafos (regex + GLiNER + GLiREL + LLM fallback)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 18:41:44 +02:00
egutierrez bd95dc5f99 docs(issues): marcar 0028, 0029, 0031, 0032, 0034 completados
Wave 2 de parallel-fix-issues integrada a master:
- 0028: ImPlot3D vendored como submodule + surface_plot_3d real + scatter_3d
- 0029: mesh_viewer + mesh_obj_load + mesh_gpu + orbit_camera
- 0031: tween_curves + bezier_editor + timeline (animation core)
- 0032: sql_workbench (text_editor SQL + table_view + sqlite3)
- 0034: scientific viz (treemap squarified, sankey BFS, chord, contour
  marching squares, voronoi raster)

primitives_gallery linux + windows cross-compile OK. fn index registra los
nuevos primitivos. exe Windows actualizado en Desktop/apps/primitives_gallery/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 21:59:54 +02:00
egutierrez dda03b14cc merge: issue/0034 — scientific viz (treemap, sankey, chord, contour, voronoi)
# Conflicts:
#	cpp/apps/primitives_gallery/CMakeLists.txt
#	cpp/apps/primitives_gallery/demos.h
#	cpp/apps/primitives_gallery/main.cpp
2026-04-25 21:55:49 +02:00
egutierrez 2f6dea6ebd merge: issue/0032 — sql_workbench
# Conflicts:
#	cpp/apps/primitives_gallery/CMakeLists.txt
#	cpp/apps/primitives_gallery/demos.h
#	cpp/apps/primitives_gallery/main.cpp
2026-04-25 21:55:17 +02:00
egutierrez 25936dff79 merge: issue/0031 — animation curves (timeline + bezier_editor + tween_curves)
# Conflicts:
#	cpp/apps/primitives_gallery/CMakeLists.txt
2026-04-25 21:54:48 +02:00
egutierrez 55939cb60b merge: issue/0029 — mesh_viewer + obj loader + orbit_camera
# Conflicts:
#	cpp/apps/primitives_gallery/demos.h
#	cpp/apps/primitives_gallery/main.cpp
2026-04-25 21:54:27 +02:00
egutierrez ca7ec360aa chore(issues): close 0034 — C++ scientific viz 2026-04-25 21:53:05 +02:00
egutierrez d42a385eb8 chore(issues): cerrar 0029 — mesh_viewer + obj loader + orbit_camera 2026-04-25 21:51:32 +02:00
egutierrez cf3936314b chore(issues): close 0031 — animation curves shipped
Three primitives delivered + gallery demos + builds cleanly:
- tween_curves_cpp_core (16 Penner curves, header-mostly)
- bezier_editor_cpp_core (visual cubic Bezier editor)
- timeline_cpp_core (DAW-style keyframe widget)
2026-04-25 21:50:50 +02:00
egutierrez a757de0f05 docs: cerrar issue 0028 2026-04-25 21:48:55 +02:00
egutierrez e24d16db59 feat(core): sql_workbench component (issue 0032)
ImGui SQL workbench: editor (text_editor + CodeLang::SQL), schema sidebar
desde sqlite_master, tabla resultado (table_view), historial. Backend
sql_workbench_run_query separado de UI; readonly opt-in (rechaza non-SELECT/
PRAGMA/EXPLAIN/WITH); cap de filas (truncated_at, default 10000); last_ms
medido con steady_clock.

API: sql_workbench(id, sqlite3*, SqlWorkbenchState&, size). DB caller-owned.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 21:48:50 +02:00
egutierrez 53402d84d5 docs(issues): marcar 0025 y 0026 como completados + WIP master
Wave 1 de parallel-fix-issues integrada a master:
- 0025: text_editor_cpp_core + file_watcher_cpp_core
- 0026: gl_texture_load_cpp_gfx (vendor: stb_image v2.30)

Ademas se commitea WIP previo de master que estaba sin commitear (cambios
en shaders_lab, dag_*, framework, tokens, kpi_card, gl_loader.md, etc.)
para dejar HEAD buildable.

Notas:
- Algunos deps del gallery (button.cpp, toolbar.cpp, modal_dialog.cpp...)
  siguen UNTRACKED — gating con FN_BUILD_GALLERY=ON (default OFF) para
  que master build (sin flag) no los necesite.
- Build OK con y sin flag. fn index registra 904 functions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 21:14:15 +02:00
egutierrez 5e94dd5087 merge: issue/0026 — gl_texture_load (stb_image)
# Conflicts:
#	cpp/CMakeLists.txt
2026-04-25 21:05:11 +02:00
egutierrez 9454194de0 docs: cerrar issue 0025
Implementado text_editor_cpp_core (PIMPL sobre ImGuiColorTextEdit MIT) y
file_watcher_cpp_core (inotify Linux / ReadDirectoryChangesW Win) con demo
combinada en primitives_gallery + smoke test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 21:01:32 +02:00
egutierrez a27884ed0a docs: cerrar issue 0026
gl_texture_load_cpp_gfx implementado, vendoreado stb_image v2.30,
gl_loader extendido con glActiveTexture y glGenerateMipmap. Demo
demos_gl_texture.cpp + asset PNG damero 256x256 listos para la
primitives_gallery cuando se integre en master.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 20:59:03 +02:00
egutierrez 461bb77298 docs(issues): añadir 0025-0036 — features C++ para registry y primitives_gallery
12 issues nuevos para implementacion paralela: text_editor, file_watcher,
gl_texture_load, gl_compute+pingpong+DAG Compute, ImPlot3D, mesh_viewer,
audio reactivo, animation curves, sql_workbench, http+ws inspector,
scientific viz (5 charts), map_tiles, image_canvas + webcam_texture.

Cada issue añade funciones al registry y un demo propio en
primitives_gallery/demos_<feature>.cpp para minimizar conflictos en paralelo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 20:48:18 +02:00
egutierrez 16140eda60 docs: actualizar README de issues — marcar 8 issues como completados (wave 1-3)
Issues integrados a master en esta serie:
- 0010 Auth System (JWT, passwords, OAuth2, RBAC, sessions)
- 0011 WebSocket & SSE Server
- 0014 File Upload & Storage (+ S3 stubs)
- 0016 Rate Limiting (token bucket)
- 0019 Structured Logging (slog-based)
- 0021 CRUD Generator generico
- 0022 Init Pipelines (scaffolding api/web/desktop/cli)
- 0024 Dashboard YAML split por tab

Total aprox: 54 funciones y 23 tipos nuevos en functions/infra/
+ 4 pipelines bash en bash/functions/pipelines/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 18:00:57 +02:00
egutierrez 7cddf872e5 docs: cerrar issue 0022
Los 4 init pipelines (init_api_app, init_web_app, init_desktop_app,
init_cli_app) estan implementados, verificados end-to-end con `fn run`, e
indexados en registry.db como kind=pipeline + tag=launcher. Guia consolidada
en docs/init-pipelines.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:59:04 +02:00
egutierrez 2a6e26b6ad docs: cerrar issue 0010 2026-04-18 17:44:41 +02:00
egutierrez a5d532c001 merge: issue/0011-websocket-sse — WebSocket + SSE (8 fns, 4 tipos)
# Conflicts:
#	registry.db
2026-04-18 17:33:32 +02:00
egutierrez d4fd36bb96 merge: issue/0014-file-upload — file upload/storage + S3 stubs (11 fns, 3 tipos)
# Conflicts:
#	registry.db
2026-04-18 17:33:28 +02:00
egutierrez 8175bf82c4 merge: issue/0021-crud-generator — CRUD generator genérico (9 fns, 4 tipos)
# Conflicts:
#	registry.db
2026-04-18 17:33:23 +02:00
egutierrez 0ac5a6c78b merge: issue/0019-structured-logging — slog-based structured logging (7 fns, 3 tipos)
# Conflicts:
#	registry.db
2026-04-18 17:33:18 +02:00
egutierrez be41c928af merge: issue/0016-rate-limiting — token bucket rate limiter (6 fns, 3 tipos) 2026-04-18 17:33:07 +02:00
egutierrez 27ee150dcd docs: cerrar issue 0011 2026-04-18 17:30:28 +02:00
egutierrez f4f68c3de5 docs: cerrar issue 0021 2026-04-18 17:18:11 +02:00
egutierrez db8188cad3 docs: cerrar issue 0014 2026-04-18 17:17:49 +02:00
egutierrez 8c489f9a5f docs: cerrar issue 0019 structured logging
Funciones, tipos y tests implementados. Registry actualizado.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:15:45 +02:00
egutierrez 893b98f5b9 docs: cerrar issue 0016 (rate limiting) 2026-04-18 17:14:50 +02:00
egutierrez f242441bb2 docs: cerrar issue 0024
El split de dashboards YAML por tab ya esta implementado y committeado en
apps/auto_metabase (repo dataforge/auto_metabase, commit 47b5f89):

- dashboard_split.py: helpers puros split_dashboard_payload() y merge_dashboard_parts()
- dashboard_split_test.py: 23 tests passing (round-trip, edge cases, real aurgi YAML)
- sync_pull.py: escribe directorio {slug}/_dashboard.yaml + tab_*.yaml
- sync_push.py: lee directorio o legacy monolitico, reconstruye payload unificado
- sync_validate.py: valida parent_slug, tab_ids y detecta archivos huerfanos
- payload.py: resolve dashboards/{slug}/_dashboard.yaml preferentemente
- app.md: documenta nuevo layout con seccion "Dashboard split por tab"
- Backward-compat con legacy dashboards/{slug}.yaml
- Aplicado a aurgi (dashboard id=734) con 9 tabs separados

Las apps viven en repos Gitea independientes (dataforge/auto_metabase), por
lo que los cambios de codigo no quedan reflejados en el historial de
fn_registry, solo el cierre del issue.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:10:36 +02:00
egutierrez 9d9193a8cc docs(issues): nuevos issues de backlog y limpieza de tracker
- Añade issues nuevos al backlog: 0010 auth-system, 0011 websocket-sse, 0014 file-upload, 0016 rate-limiting, 0019 structured-logging, 0021 crud-generator, 0022 init-pipelines, 0024 dashboard-yaml-split-por-tab.
- Elimina 0008-sqlite-api-web.md (ya no aplica).
- Actualiza dev/issues/README.md con el estado del tracker.
2026-04-13 23:31:47 +02:00
egutierrez 386a5471e0 docs: actualizar README de issues — marcar 8 issues de Wave 1 como completados
Wave 1 completada e integrada a master:
- 0009 HTTP Server Foundation
- 0012 Email & SMTP
- 0013 Background Job Queue
- 0015 Database Migrations
- 0017 Frontend Data Hooks
- 0018 Config & Env Management
- 0020 PDF Generation
- 0023 Testing Utilities

Pendientes (Wave 2 descartada, Wave 3 sin iniciar):
0010, 0011, 0014, 0016, 0019, 0021, 0022

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 11:26:05 +02:00