dataforge/fn_registry
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity
Files
01bc2aeb1493d73aecc83a5ae9a04e4222dc42ef
fn_registry/modules/data_table/data_table.cpp
T
egutierrez 01bc2aeb14 feat(0133-3): wire filter/sort readers to columnar snapshot 
Change 3 of issue 0133 — rewire compute_visible_rows, filter eval,
and sort comparators to read from the SnapshotCache when available.

Hot paths rewired:
- compute_visible_rows (overload with snap): filter eval uses
  compare_snap (fast i64/f64 numeric compare for Int/Float cols;
  id-compare for low-cardinality string Eq/Neq; raw cells fallback
  for Contains/StartsWith/EndsWith).
- Sort comparators: direct i64/f64 array compare for Int/Float cols
  (goto sort_done skips string fallback); string sort uses uint32_t
  id compare with pool lookup only on mismatch.
- Stage>0 filter/sort: same snapshot overload.

Materialization paths (build_so, s0_backing, mat_backing, config popup)
kept on raw cells — they copy into std::string anyway, no benefit from
snapshot and snprintf-per-cell was 2M extra calls per frame.

Bug fixes (required for correctness):
1. StringPool::intern() realloc safety: force reserve before
   emplace_back so string_view keys in the map never go dangling.
2. SnapshotCache::pool_size_built sentinel: detects when a new State
   is created with an empty pool but same cells pointer (begin_scenario
   pattern). Prevents str_ids from indexing into an empty pool (SIGSEGV).
3. Cardinality cap (2048 uniques / 25% sample): high-cardinality string
   cols (timestamps-as-strings, UUIDs, names) skip interning — str_ids
   stays empty and compare_snap falls back to raw cells. Prevents 30MB+
   pool bloat that hurt cache for filter/sort on other cols.

Bench delta vs baseline (100k rows, LIBGL_ALWAYS_SOFTWARE=1):
  linear_scroll: 16.0 -> 15.5 fps p50  (-3%, baseline already FAIL)
  filter_like:   59.7 -> 56.0 fps p50  (-6%, still PASS at 56fps)
  sort_numeric:   3.9 ->  9.0 fps p50 (+131%, snapshot i64 sort)
  color_rule:    15.2 -> 14.8 fps p50  (-3%, baseline already FAIL)

Build: green for all 10 available Linux consumers (text_editor_smoke
linker failure is preexisting, not caused by this change).

API public intact. TableEvent.row indexing TableInput preserved.
Pointer-identity invalidation preserved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 00:21:09 +02:00

2127 lines
95 KiB
C++
Raw Blame History

// data_table — render UI completa de tabla TQL.
// Entry-point publica del stack data_table del registry.
// Issue 0081-H. Promovido desde cpp/apps/primitives_gallery/playground/tables/data_table.cpp
//
// Dependencias del registry:
// - core/data_table_types.h (tipos compartidos: State, TableInput, Stage, ...)
// - core/compute_stage.h (compute_stage_cpp_core)
// - core/compute_pipeline.h (compute_pipeline_cpp_core)
// - core/compute_column_stats.h (compute_column_stats_cpp_core)
// - core/auto_detect_type.h (auto_detect_type_cpp_core)
// - core/tql_emit.h (tql_emit_cpp_core)
// - core/tql_apply.h (tql_apply_cpp_core)
// - core/lua_engine.h (lua_engine_cpp_core)
// - core/join_tables.h (join_tables_cpp_core)
// - viz/viz_render.h (viz_render_cpp_viz)
//
// Notas de deuda tecnica:
// - tql_apply_cpp_core expone firma reducida; el playground usaba tql::apply
// con cells/rows/orig_cols. Las llamadas internas de este archivo usan el
// namespace tql:: del playground via include del tql_apply_cpp_core header.
// Pendiente: ampliar tql_apply_cpp_core a la firma extendida (Wave 4/5 proposal).
// - llm_anthropic (Ask AI modal, fase 11): incluido desde el playground (no en registry).
// Pendiente: promover a cpp/functions/infra/llm_anthropic — deuda Wave 4.
// - tql_to_sql (SQL transpile): incluido desde el playground. Pendiente: registry Wave 4.
// - tql_duckdb (FN_TQL_DUCKDB): opcional, sin wrapper en registry todavia.
#include "data_table/data_table.h"
// Sub-funciones extraidas (issue 0107c).
#include "viz/data_table_drill.h"
#include "viz/data_table_color_rules.h"
#include "viz/data_table_ai_panel.h"
#include "viz/data_table_chips.h"
#include "viz/data_table_grid.h"
#include "viz/data_table_viz_panels.h"
// Contrato interno del modulo: UiState + inline helpers compartidos.
#include "data_table/data_table_internal.h"
// Framework ImGui (via fn_framework)
#include "imgui.h"
// Registry Wave 1+2 includes (all resolved via fn_table_viz include path).
#include "core/lua_engine.h"
#include "core/tql_apply.h"
#include "core/tql_emit.h"
#include "core/tql_helpers.h"
#include "core/compute_stage.h"
#include "core/compute_pipeline.h"
#include "core/compute_column_stats.h"
#include "core/auto_detect_type.h"
#include "core/join_tables.h"
#include "core/tql_to_sql.h"
#include "viz/viz_render.h"
// llm_anthropic — Ask AI modal. Promoted to registry (cpp/functions/core/) in
// Wave 3.5. Real implementation linked by fn_table_viz; stub kept under
// !FN_LLM_ANTHROPIC for environments that build without the lib.
#ifdef FN_LLM_ANTHROPIC
# include "core/llm_anthropic.h"
#endif
#ifdef FN_TQL_DUCKDB
# include "tql_duckdb.h"
#endif
// fn::local_path — from fn_framework (framework/app_base.h).
// Required by the Ask AI modal and TQL save/load paths.
#include "app_base.h"
#include <algorithm>
#include <cfloat>
#include <cinttypes>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <ctime>
#include <fstream>
#include <limits>
#include <string>
#include <unordered_map>
// icons_tabler.h: needed by draw_cell_custom icon renderer (issue 0081-N).
#include "core/icons_tabler.h"
// ---------------------------------------------------------------------------
// llm_anthropic stub (Wave 4 TODO: replace with infra/llm_anthropic.h)
// Provides no-op types/functions so fn_table_viz links without the playground.
// When FN_LLM_ANTHROPIC is defined the real header is included above instead.
// ---------------------------------------------------------------------------
#ifndef FN_LLM_ANTHROPIC
namespace llm_anthropic {
enum class OutputMode { TQL, SQL };
struct AskInput {
std::string question;
std::string tql_current;
std::vector<std::string> col_names;
std::vector<data_table::ColumnType> col_types;
std::vector<std::string> joinable_names;
OutputMode mode = OutputMode::TQL;
std::string model;
int max_tokens = 8192;
};
struct AskResult {
std::string code;
std::string raw;
std::string error;
int tokens_in = 0;
int tokens_out = 0;
};
inline AskResult ask(const AskInput&, const std::string& = "") {
AskResult r;
r.error = "llm_anthropic not available (stub). Build with FN_LLM_ANTHROPIC=1.";
return r;
}
} // namespace llm_anthropic
#endif // FN_LLM_ANTHROPIC
namespace data_table {
// ---------------------------------------------------------------------------
// Helpers from playground data_table_logic — declared static so they do not
// leak into the data_table namespace beyond this translation unit.
// Promoted inline to remove dependency on playground headers. Issue 0081-I.
// ---------------------------------------------------------------------------
// column_type_icon: returns a Tabler icon UTF-8 sequence for each ColumnType.
static const char* column_type_icon(ColumnType t) {
switch (t) {
case ColumnType::Auto: return "\xef\xa4\x9d"; // TI_HELP_CIRCLE
case ColumnType::String: return "\xef\x95\xa7"; // TI_ABC
case ColumnType::Int: return "\xef\x95\x94"; // TI_123
case ColumnType::Float: return "\xef\xa8\xa6"; // TI_DECIMAL
case ColumnType::Bool: return "\xee\xae\xa6"; // TI_CHECKBOX
case ColumnType::Date: return "\xee\xa9\x93"; // TI_CALENDAR
case ColumnType::Json: return "\xee\xaf\x8c"; // TI_BRACES
}
return "?";
}
// ---------------------------------------------------------------------------
// hex_to_imcolor: parses "#rrggbb" or "rrggbb" -> ImVec4 (alpha=1).
// Returns {-1,-1,-1,-1} on failure. Used only by cell renderers in this TU.
// Static: not part of the module public API.
// ---------------------------------------------------------------------------
static ImVec4 hex_to_imcolor(const std::string& hex) {
const char* p = hex.c_str();
if (*p == '#') ++p;
unsigned int r = 0, g = 0, b = 0;
if (std::sscanf(p, "%02x%02x%02x", &r, &g, &b) != 3)
return ImVec4(-1.f, -1.f, -1.f, -1.f);
return ImVec4(r / 255.f, g / 255.f, b / 255.f, 1.f);
}
// lerp_color_along_stops: LERP between N color stops based on t in [0,1].
// Default gradient: green -> amber -> red. Used by ColorScale renderer.
// Static: calls parse_hex_color (from data_table_color_rules.cpp via header).
// ---------------------------------------------------------------------------
static ImU32 lerp_color_along_stops(
const std::vector<ColorStop>& stops, float t, float alpha)
{
static const std::vector<ColorStop> kDefault = {
{0.0f, "#22c55e"},
{0.5f, "#f59e0b"},
{1.0f, "#ef4444"},
};
const auto& sv = stops.empty() ? kDefault : stops;
std::vector<ColorStop> sorted_sv = sv;
std::sort(sorted_sv.begin(), sorted_sv.end(),
[](const ColorStop& a, const ColorStop& b){
return a.position < b.position;
});
t = t < 0.f ? 0.f : (t > 1.f ? 1.f : t);
if (t <= sorted_sv.front().position)
return parse_hex_color(sorted_sv.front().color, alpha);
if (t >= sorted_sv.back().position)
return parse_hex_color(sorted_sv.back().color, alpha);
for (size_t i = 0; i + 1 < sorted_sv.size(); ++i) {
const auto& lo = sorted_sv[i];
const auto& hi = sorted_sv[i + 1];
if (t >= lo.position && t <= hi.position) {
float span = hi.position - lo.position;
float f = (span > 1e-6f) ? (t - lo.position) / span : 0.f;
ImVec4 ca = hex_to_imcolor(lo.color);
ImVec4 cb = hex_to_imcolor(hi.color);
if (ca.x < 0.f) ca = ImVec4(0.5f, 0.5f, 0.5f, 1.f);
if (cb.x < 0.f) cb = ImVec4(0.5f, 0.5f, 0.5f, 1.f);
float r = ca.x + f * (cb.x - ca.x);
float g = ca.y + f * (cb.y - ca.y);
float b = ca.z + f * (cb.z - ca.z);
unsigned int ri = (unsigned int)(r * 255.f + 0.5f);
unsigned int gi = (unsigned int)(g * 255.f + 0.5f);
unsigned int bi = (unsigned int)(b * 255.f + 0.5f);
unsigned int ai = (unsigned int)(alpha * 255.f + 0.5f);
return IM_COL32(ri, gi, bi, ai);
}
}
return parse_hex_color(sorted_sv.back().color, alpha);
}
// parse_hex_color, auto_categorical_color, resolve_categorical_dot_color,
// parse_cell_number, apply_color_rules_for_cell: defined in
// viz/data_table_color_rules.cpp (issue 0107c).
// Declared via #include "viz/data_table_color_rules.h" above.
// Covers the ~30 most-used icons. Returns nullptr if not found.
// ---------------------------------------------------------------------------
static const char* icon_name_to_glyph(const std::string& name) {
static const std::unordered_map<std::string, const char*> kMap = {
{"TI_CHECK", TI_CHECK},
{"TI_X", TI_X},
{"TI_ALERT_CIRCLE", TI_ALERT_CIRCLE},
{"TI_CIRCLE_DOT", TI_CIRCLE_DOT},
{"TI_CLOCK", TI_CLOCK},
{"TI_LOADER", TI_LOADER},
{"TI_BAN", TI_BAN},
{"TI_PLAYER_PLAY", TI_PLAYER_PLAY},
{"TI_PLAYER_PAUSE", TI_PLAYER_PAUSE},
{"TI_PLAYER_STOP", TI_PLAYER_STOP},
{"TI_DATABASE", TI_DATABASE},
{"TI_SETTINGS", TI_SETTINGS},
{"TI_USER", TI_USER},
{"TI_USERS", TI_USERS},
{"TI_FILE", TI_FILE},
{"TI_FOLDER", TI_FOLDER},
{"TI_REFRESH", TI_REFRESH},
{"TI_BOLT", TI_BOLT},
{"TI_INFO_CIRCLE", TI_INFO_CIRCLE},
{"TI_ARROW_UP", TI_ARROW_UP},
{"TI_ARROW_DOWN", TI_ARROW_DOWN},
{"TI_ARROW_RIGHT", TI_ARROW_RIGHT},
{"TI_ARROW_LEFT", TI_ARROW_LEFT},
{"TI_DOTS", TI_DOTS},
{"TI_EYE", TI_EYE},
{"TI_EYE_OFF", TI_EYE_OFF},
{"TI_EDIT", TI_EDIT},
{"TI_TRASH", TI_TRASH},
{"TI_COPY", TI_COPY},
{"TI_EXTERNAL_LINK", TI_EXTERNAL_LINK},
};
auto it = kMap.find(name);
return it != kMap.end() ? it->second : nullptr;
}
// draw_cell_custom: moved to cpp/functions/viz/data_table_grid.cpp (issue 0107c).
// Public declaration in viz/data_table_grid.h — callers within this TU resolve
// directly to that symbol (no forwarder needed; same namespace, same lib).
// compare: cell-level comparison supporting all Op variants.
// Uses parse_number (from auto_detect_type.h) for numeric comparisons.
static bool compare(const char* a, const char* b, Op op) {
if (!a) a = "";
if (!b) b = "";
switch (op) {
case Op::Contains: return std::strstr(a, b) != nullptr;
case Op::NotContains: return std::strstr(a, b) == nullptr;
case Op::StartsWith: {
size_t lb = std::strlen(b);
return std::strncmp(a, b, lb) == 0;
}
case Op::EndsWith: {
size_t la = std::strlen(a), lb = std::strlen(b);
return lb <= la && std::strcmp(a + la - lb, b) == 0;
}
default: break;
}
double na, nb;
bool numeric = parse_number(a, na) && parse_number(b, nb);
if (numeric) {
switch (op) {
case Op::Eq: return na == nb;
case Op::Neq: return na != nb;
case Op::Gt: return na > nb;
case Op::Gte: return na >= nb;
case Op::Lt: return na < nb;
case Op::Lte: return na <= nb;
default: break;
}
}
int c = std::strcmp(a, b);
switch (op) {
case Op::Eq: return c == 0;
case Op::Neq: return c != 0;
case Op::Gt: return c > 0;
case Op::Gte: return c >= 0;
case Op::Lt: return c < 0;
case Op::Lte: return c <= 0;
default: break;
}
return false;
}
// make_drill_filter, apply_drill_step, undo_drill_step, drill_up:
// Moved to cpp/functions/viz/data_table_drill.cpp (issue 0107c).
// Declared in viz/data_table_drill.h, included above.
// row_to_tsv: serializes a single row to a two-line TSV (header + values).
static std::string row_to_tsv(const char* const* cells, int rows, int cols,
int row_idx, const std::vector<std::string>& headers) {
if (row_idx < 0 || row_idx >= rows || cols <= 0) return "";
std::string out;
for (int c = 0; c < cols; ++c) {
if (c > 0) out += '\t';
if (c < (int)headers.size()) out += headers[c];
}
out += "\r\n";
for (int c = 0; c < cols; ++c) {
if (c > 0) out += '\t';
const char* v = cells[row_idx * cols + c];
if (v) out += v;
}
out += "\r\n";
return out;
}
// ---------------------------------------------------------------------------
// Issue 0133 — Change 3: Reader rewire helpers.
//
// snap_cell: devuelve el string de una celda desde el snapshot columnar cuando
// la columna esta en rango, con fallback al raw cells array.
// Para columnas Int/Float usa un buffer thread_local de 32 bytes (evita alloc).
// ---------------------------------------------------------------------------
static inline const char* snap_cell(int r, int c,
const SnapshotCache& snap,
const StringPool& pool,
char (&tmp)[32])
{
if (c >= 0 && c < (int)snap.cols.size()) {
const ColumnSnapshot& cs = snap.cols[(size_t)c];
if (cs.type == ColumnType::Int) {
std::snprintf(tmp, sizeof(tmp), "%" PRId64, cs.i64[(size_t)r]);
return tmp;
} else if (cs.type == ColumnType::Float) {
std::snprintf(tmp, sizeof(tmp), "%.17g", cs.f64[(size_t)r]);
return tmp;
} else if (!cs.str_ids.empty()) {
return pool.at(cs.str_ids[(size_t)r]).c_str();
}
}
(void)tmp;
return nullptr; // caller must fallback to cells[r*cols+c]
}
// compare_snap: evaluates filter f against row r using snapshot when available.
// Falls back to raw cells if column not in snapshot.
static inline bool compare_snap(int r, int f_col,
const char* f_val, Op f_op,
const char* const* cells, int cols,
const SnapshotCache& snap,
const StringPool& pool)
{
// Fast numeric path: avoid string conversion for numeric comparisons.
if (f_col >= 0 && f_col < (int)snap.cols.size()) {
const ColumnSnapshot& cs = snap.cols[(size_t)f_col];
if (cs.type == ColumnType::Int &&
(f_op == Op::Eq || f_op == Op::Neq || f_op == Op::Gt ||
f_op == Op::Gte || f_op == Op::Lt || f_op == Op::Lte)) {
double fv;
if (parse_number(f_val, fv)) {
int64_t av = cs.i64[(size_t)r];
int64_t bv = (int64_t)fv;
switch (f_op) {
case Op::Eq: return av == bv;
case Op::Neq: return av != bv;
case Op::Gt: return av > bv;
case Op::Gte: return av >= bv;
case Op::Lt: return av < bv;
case Op::Lte: return av <= bv;
default: break;
}
}
}
if (cs.type == ColumnType::Float &&
(f_op == Op::Eq || f_op == Op::Neq || f_op == Op::Gt ||
f_op == Op::Gte || f_op == Op::Lt || f_op == Op::Lte)) {
double fv;
if (parse_number(f_val, fv)) {
double av = cs.f64[(size_t)r];
switch (f_op) {
case Op::Eq: return av == fv;
case Op::Neq: return av != fv;
case Op::Gt: return av > fv;
case Op::Gte: return av >= fv;
case Op::Lt: return av < fv;
case Op::Lte: return av <= fv;
default: break;
}
}
}
// String column: snapshot offers no speed advantage for substring ops
// (Contains/NotContains/StartsWith/EndsWith need full string scan regardless).
// Only use intern path for equality (id compare avoids strcmp).
if (!cs.str_ids.empty()) {
if (f_op == Op::Eq || f_op == Op::Neq) {
// Find the interned id of f_val (if not found, no row can match Eq,
// and all rows match Neq).
std::string_view fv_sv(f_val);
auto fv_it = pool.index.find(fv_sv);
if (f_op == Op::Eq) {
if (fv_it == pool.index.end()) return false; // f_val not interned => no match
return cs.str_ids[(size_t)r] == fv_it->second;
} else { // Op::Neq
if (fv_it == pool.index.end()) return true; // f_val not interned => all differ
return cs.str_ids[(size_t)r] != fv_it->second;
}
}
// For substring / prefix / suffix ops: fall through to raw cells (no snapshot benefit).
}
}
// Fallback: raw cells (e.g. derived column not in snapshot, or string substring op).
const char* cell = (f_col >= 0 && f_col < cols) ? cells[r * cols + f_col] : nullptr;
return compare(cell, f_val, f_op);
}
// compute_visible_rows: applies stage-0 filters + optional sort, returns matching row indices.
// Issue 0133 — Change 3: overload with snapshot for columnar reads.
static std::vector<int> compute_visible_rows(const char* const* cells,
int rows, int cols,
const State& st,
const SnapshotCache& snap,
const StringPool& pool)
{
std::vector<int> out;
out.reserve(rows);
const Stage& s = st.raw();
for (int r = 0; r < rows; ++r) {
bool keep = true;
for (const auto& f : s.filters) {
if (f.col < 0 || f.col >= cols) continue;
if (!compare_snap(r, f.col, f.value.c_str(), f.op,
cells, cols, snap, pool)) {
keep = false; break;
}
}
if (keep) out.push_back(r);
}
if (!s.sorts.empty()) {
const SortClause& sc0 = s.sorts.front();
int sc = -1;
if (!sc0.col.empty() && sc0.col[0] == '@') {
sc = std::atoi(sc0.col.c_str() + 1);
}
bool desc = sc0.desc;
if (sc >= 0 && sc < cols) {
// Fast numeric sort via snapshot.
if (sc < (int)snap.cols.size()) {
const ColumnSnapshot& cs = snap.cols[(size_t)sc];
if (cs.type == ColumnType::Int) {
std::sort(out.begin(), out.end(), [&](int a, int b) {
int64_t va = cs.i64[(size_t)a];
int64_t vb = cs.i64[(size_t)b];
return desc ? (va > vb) : (va < vb);
});
goto sort_done;
} else if (cs.type == ColumnType::Float) {
std::sort(out.begin(), out.end(), [&](int a, int b) {
double va = cs.f64[(size_t)a];
double vb = cs.f64[(size_t)b];
return desc ? (va > vb) : (va < vb);
});
goto sort_done;
} else if (!cs.str_ids.empty()) {
// String sort: compare uint32_t ids first (if equal -> same string).
std::sort(out.begin(), out.end(), [&](int a, int b) {
uint32_t ia = cs.str_ids[(size_t)a];
uint32_t ib = cs.str_ids[(size_t)b];
if (ia == ib) return false; // equal
int cmp = std::strcmp(pool.at(ia).c_str(), pool.at(ib).c_str());
return desc ? (cmp > 0) : (cmp < 0);
});
goto sort_done;
}
}
// Fallback sort via raw cells.
std::sort(out.begin(), out.end(), [&](int a, int b) {
const char* ca = cells[a * cols + sc];
const char* cb = cells[b * cols + sc];
if (!ca) ca = "";
if (!cb) cb = "";
double na, nb;
bool num = parse_number(ca, na) && parse_number(cb, nb);
int cmp;
if (num) cmp = (na < nb) ? -1 : (na > nb ? 1 : 0);
else cmp = std::strcmp(ca, cb);
return desc ? (cmp > 0) : (cmp < 0);
});
sort_done:;
}
}
return out;
}
// compute_visible_rows: legacy overload without snapshot (used by stage>0 path
// which operates on materialized StageOutput — not the raw cells snapshot).
static std::vector<int> compute_visible_rows(const char* const* cells,
int rows, int cols,
const State& st)
{
std::vector<int> out;
out.reserve(rows);
const Stage& s = st.raw();
for (int r = 0; r < rows; ++r) {
bool keep = true;
for (const auto& f : s.filters) {
if (f.col < 0 || f.col >= cols) continue;
const char* cell = cells[r * cols + f.col];
if (!compare(cell, f.value.c_str(), f.op)) { keep = false; break; }
}
if (keep) out.push_back(r);
}
if (!s.sorts.empty()) {
const SortClause& sc0 = s.sorts.front();
int sc = -1;
if (!sc0.col.empty() && sc0.col[0] == '@') {
sc = std::atoi(sc0.col.c_str() + 1);
}
bool desc = sc0.desc;
if (sc >= 0 && sc < cols) {
std::sort(out.begin(), out.end(), [&](int a, int b) {
const char* ca = cells[a * cols + sc];
const char* cb = cells[b * cols + sc];
if (!ca) ca = "";
if (!cb) cb = "";
double na, nb;
bool num = parse_number(ca, na) && parse_number(cb, nb);
int cmp;
if (num) cmp = (na < nb) ? -1 : (na > nb ? 1 : 0);
else cmp = std::strcmp(ca, cb);
return desc ? (cmp > 0) : (cmp < 0);
});
}
}
return out;
}
// csv_escape: wraps s in double-quotes if it contains commas, quotes, or newlines.
static std::string csv_escape(const char* s) {
if (!s) return "";
bool needs = false;
for (const char* p = s; *p; ++p) {
if (*p == ',' || *p == '"' || *p == '\n' || *p == '\r') { needs = true; break; }
}
if (!needs) return std::string(s);
std::string out; out.reserve(std::strlen(s) + 4);
out += '"';
for (const char* p = s; *p; ++p) {
if (*p == '"') out += '"';
out += *p;
}
out += '"';
return out;
}
// reorder_column: moves col src to position of col dst in st.col_order.
static void reorder_column(State& st, int src, int dst) {
if (src == dst) return;
auto it_s = std::find(st.col_order.begin(), st.col_order.end(), src);
auto it_d = std::find(st.col_order.begin(), st.col_order.end(), dst);
if (it_s == st.col_order.end() || it_d == st.col_order.end()) return;
int si = (int)(it_s - st.col_order.begin());
int di = (int)(it_d - st.col_order.begin());
int v = st.col_order[si];
st.col_order.erase(st.col_order.begin() + si);
if (di > (int)st.col_order.size()) di = (int)st.col_order.size();
st.col_order.insert(st.col_order.begin() + di, v);
}
// find_open_bracket: scans buf[0..cursor) backwards for an unmatched '['.
// Returns index of '[' and fills filter_text with content after it, or -1 if none.
static int find_open_bracket(const char* buf, int len, int cursor, std::string& filter_text) {
filter_text.clear();
if (!buf || cursor <= 0 || cursor > len) return -1;
for (int i = cursor - 1; i >= 0; --i) {
char c = buf[i];
if (c == ']' || c == '\n') return -1;
if (c == '[') {
filter_text.assign(buf + i + 1, cursor - i - 1);
return i;
}
}
return -1;
}
// insert_column_ref: replaces src[start..cursor) with "[name]", updating new_cursor.
static std::string insert_column_ref(const std::string& src, int start, int cursor,
const std::string& name, int& new_cursor) {
if (start < 0 || start > (int)src.size() || cursor < start || cursor > (int)src.size()) {
new_cursor = cursor;
return src;
}
std::string replacement = "[" + name + "]";
std::string out;
out.reserve(src.size() - (cursor - start) + replacement.size());
out.append(src, 0, start);
out += replacement;
out.append(src, cursor, std::string::npos);
new_cursor = start + (int)replacement.size();
return out;
}
// ---------------------------------------------------------------------------
// Additional helpers from playground data_table_logic — view_mode, joins,
// filter presets, date helpers, effective_type, etc.
// All declared static to stay internal to this translation unit.
// ---------------------------------------------------------------------------
// effective_type, ops_for_type, resolve_main_idx, join_strategy_label:
// ahora inline en data_table_internal.h (compartidos con sub-funciones).
// ViewModeInfo, kViewModes, kViewModesN, view_mode_label, view_mode_needs_aggregation,
// all_view_modes: moved to data_table_internal.h (shared with data_table_viz_panels.cpp).
// Date helpers (for filter presets and breakout auto-granularity).
namespace {
static bool parse_ymd_local(const std::string& s, int& y, int& m, int& d) {
if (s.size() < 10) return false;
for (int i : {0,1,2,3,5,6,8,9}) {
if (s[(size_t)i] < '0' || s[(size_t)i] > '9') return false;
}
if (s[4] != '-' || s[7] != '-') return false;
y = (s[0]-'0')*1000 + (s[1]-'0')*100 + (s[2]-'0')*10 + (s[3]-'0');
m = (s[5]-'0')*10 + (s[6]-'0');
d = (s[8]-'0')*10 + (s[9]-'0');
if (m < 1 || m > 12 || d < 1 || d > 31) return false;
return true;
}
static long ymd_to_days_local(int y, int m, int d) {
if (m <= 2) { y -= 1; m += 12; }
long era = (y >= 0 ? y : y - 399) / 400;
unsigned yoe = (unsigned)(y - era * 400);
unsigned doy = (unsigned)((153 * (m - 3) + 2) / 5 + d - 1);
unsigned doe = yoe * 365 + yoe/4 - yoe/100 + doy;
return era * 146097 + (long)doe;
}
static void days_to_ymd_local(long days, int& y, int& m, int& d) {
long era = (days >= 0 ? days : days - 146096) / 146097;
unsigned doe = (unsigned)(days - era * 146097);
unsigned yoe = (doe - doe/1460 + doe/36524 - doe/146096) / 365;
int yr = (int)yoe + (int)era * 400;
unsigned doy = doe - (365*yoe + yoe/4 - yoe/100);
unsigned mp = (5*doy + 2)/153;
unsigned day = doy - (153*mp + 2)/5 + 1;
unsigned mon = mp < 10 ? mp + 3 : mp - 9;
if (mon <= 2) yr += 1;
y = yr; m = (int)mon; d = (int)day;
}
} // anon
static void column_min_max(const char* const* cells, int rows, int cols, int col_idx,
std::string& min_out, std::string& max_out) {
min_out.clear(); max_out.clear();
if (col_idx < 0 || col_idx >= cols) return;
bool first = true;
for (int r = 0; r < rows; ++r) {
const char* v = cells[r * cols + col_idx];
if (!v || !*v) continue;
std::string s(v);
if (first) { min_out = s; max_out = s; first = false; }
else { if (s < min_out) min_out = s; if (s > max_out) max_out = s; }
}
}
static DateGranularity auto_date_granularity(const std::string& min_ymd,
const std::string& max_ymd) {
int y1,m1,d1, y2,m2,d2;
if (!parse_ymd_local(min_ymd, y1,m1,d1)) return DateGranularity::Day;
if (!parse_ymd_local(max_ymd, y2,m2,d2)) return DateGranularity::Day;
long span = ymd_to_days_local(y2,m2,d2) - ymd_to_days_local(y1,m1,d1);
if (span < 0) span = -span;
if (span > 730) return DateGranularity::Year;
if (span > 60) return DateGranularity::Month;
if (span > 14) return DateGranularity::Week;
return DateGranularity::Day;
}
static std::string compose_breakout(const std::string& col, DateGranularity g) {
if (g == DateGranularity::None) return col;
return col + ":" + date_granularity_token(g);
}
static const char* filter_preset_label(FilterPreset p) {
switch (p) {
case FilterPreset::Last7d: return "Last 7 days";
case FilterPreset::Last30d: return "Last 30 days";
case FilterPreset::Last90d: return "Last 90 days";
case FilterPreset::ExcludeNulls: return "Exclude nulls";
case FilterPreset::NonZero: return "Non-zero only";
}
return "?";
}
static std::vector<Filter> build_preset_filters(FilterPreset preset, int col,
const std::string& today_ymd) {
std::vector<Filter> out;
auto last_n = [&](int n) {
int y, m, d;
if (!parse_ymd_local(today_ymd, y, m, d)) return;
long days = ymd_to_days_local(y, m, d) - n;
int yy, mm, dd;
days_to_ymd_local(days, yy, mm, dd);
char buf[16];
std::snprintf(buf, sizeof(buf), "%04d-%02d-%02d", yy, mm, dd);
Filter f; f.col = col; f.op = Op::Gte; f.value = buf;
out.push_back(f);
};
switch (preset) {
case FilterPreset::Last7d: last_n(7); break;
case FilterPreset::Last30d: last_n(30); break;
case FilterPreset::Last90d: last_n(90); break;
case FilterPreset::ExcludeNulls: {
Filter f; f.col = col; f.op = Op::Neq; f.value = "";
out.push_back(f); break;
}
case FilterPreset::NonZero: {
Filter f1; f1.col = col; f1.op = Op::Neq; f1.value = "";
Filter f2; f2.col = col; f2.op = Op::Neq; f2.value = "0";
out.push_back(f1); out.push_back(f2); break;
}
}
return out;
}
// agg_fn_name, op_is_string_only — small helpers not in tql_helpers.h.
// op_label and aggregation_alias are already provided by tql_helpers.h.
static const char* agg_fn_name(AggFn f) {
switch (f) {
case AggFn::Count: return "count";
case AggFn::Sum: return "sum";
case AggFn::Avg: return "avg";
case AggFn::Min: return "min";
case AggFn::Max: return "max";
case AggFn::Distinct: return "distinct";
case AggFn::Stddev: return "stddev";
case AggFn::Median: return "median";
case AggFn::P25: return "p25";
case AggFn::P75: return "p75";
case AggFn::P90: return "p90";
case AggFn::P99: return "p99";
case AggFn::Percentile: return "percentile";
}
return "?";
}
static bool op_is_string_only(Op o) {
return o == Op::Contains || o == Op::NotContains ||
o == Op::StartsWith || o == Op::EndsWith;
}
// UTC date today as ISO YYYY-MM-DD. Para preset filtros Last7/30/90d.
static std::string today_iso() {
std::time_t t = std::time(nullptr);
std::tm tm = *std::gmtime(&t);
char buf[16];
std::snprintf(buf, sizeof(buf), "%04d-%02d-%02d",
tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday);
return buf;
}
// UiState definido en data_table_internal.h (issue 0107c).
// Singleton ui() definido en namespace data_table a continuacion del bloque anonimo.
namespace {
// draw_row_inspector_modal: moved to cpp/functions/viz/data_table_grid.cpp (issue 0107c).
// Called from render_grid_stage_n (also in data_table_grid.cpp).
int autocomplete_cb(ImGuiInputTextCallbackData* data) {
UiState* U = (UiState*)data->UserData;
if (data->EventFlag == ImGuiInputTextFlags_CallbackAlways) {
if (U->cf_force_cursor) {
data->CursorPos = U->cf_target_cursor;
U->cf_force_cursor = false;
}
}
if (data->EventFlag == ImGuiInputTextFlags_CallbackEdit) {
std::string filter;
int idx = find_open_bracket(data->Buf, data->BufTextLen,
data->CursorPos, filter);
if (idx >= 0) {
U->cf_ac_open = true;
U->cf_ac_start = idx;
U->cf_ac_cursor = data->CursorPos;
U->cf_ac_filter = filter;
} else {
U->cf_ac_open = false;
}
}
return 0;
}
// filters_hash: moved to data_table_internal.h (shared with data_table_viz_panels.cpp).
void ensure_init(State& st, int eff_cols) {
if ((int)st.col_visible.size() < eff_cols) st.col_visible.resize(eff_cols, true);
if ((int)st.col_order.size() != eff_cols) {
std::vector<int> next;
next.reserve(eff_cols);
for (int x : st.col_order) if (x >= 0 && x < eff_cols) next.push_back(x);
std::vector<bool> seen(eff_cols, false);
for (int x : next) seen[x] = true;
for (int i = 0; i < eff_cols; ++i) if (!seen[i]) next.push_back(i);
st.col_order = std::move(next);
}
if ((int)st.col_visible.size() < (int)st.col_order.size())
st.col_visible.resize(st.col_order.size(), true);
}
// draw_stage_breadcrumb: Moved to cpp/functions/viz/data_table_drill.cpp (issue 0107c).
// Declared in viz/data_table_drill.h, included above.
// ColInfo, collect_active_col_info, auto_promote_aggregated:
// moved to data_table_internal.h (shared with data_table_viz_panels.cpp).
// draw_joins_chips, draw_filter_chips, draw_breakout_chips, draw_aggregation_chips,
// draw_sort_chips, apply_header_sort_click, draw_edit_*_popup, draw_add_*_popup,
// draw_header_menu, draw_tql_bar:
// Extraidos a cpp/functions/viz/data_table_chips.cpp (issue 0107c).
// Declarados en viz/data_table_chips.h, incluido arriba.
// draw_table_toggle, draw_extra_panel, draw_viz_config_popup, draw_viz_selector,
// maybe_recompute_stats:
// Extraidos a cpp/functions/viz/data_table_viz_panels.cpp (issue 0107c).
// Declarados en viz/data_table_viz_panels.h, incluido arriba.
// drill_into: Moved to cpp/functions/viz/data_table_drill.cpp (issue 0107c).
// Declared in viz/data_table_drill.h, included above.
} // anon namespace
// UiState singleton — definido aqui (entrypoint del modulo).
// Las sub-funciones en sus .cpp llaman ui() via declaracion en data_table_internal.h.
UiState& ui() { static UiState s; return s; }
void render(const char* id,
const std::vector<TableInput>& tables,
State& st,
std::vector<TableEvent>* events_out,
bool show_chrome)
{
if (tables.empty()) return;
int main_idx = resolve_main_idx(tables, st.main_source);
if (main_idx < 0) return;
// Merge aux_column_specs from State into TableInput when the caller passed
// empty column_specs. Caller-provided specs always take precedence.
// We keep a local copy to avoid mutating the caller's const tables.
static thread_local TableInput main_t_merged;
{
const TableInput& src = tables[(size_t)main_idx];
if (src.column_specs.empty() &&
main_idx < (int)st.aux_column_specs.size() &&
!st.aux_column_specs[(size_t)main_idx].empty())
{
main_t_merged = src;
main_t_merged.column_specs = st.aux_column_specs[(size_t)main_idx];
} else {
main_t_merged = src;
}
}
const TableInput& main_t = main_t_merged;
static thread_local std::vector<const char*> main_hdr_ptrs;
main_hdr_ptrs.clear();
main_hdr_ptrs.reserve(main_t.cols);
for (int c = 0; c < main_t.cols; ++c) main_hdr_ptrs.push_back(main_t.headers[c].c_str());
const char* const* headers_in = main_hdr_ptrs.data();
int col_count = main_t.cols;
const char* const* cells_in = main_t.cells;
int row_count_in = main_t.rows;
const ColumnType* declared_types_in = main_t.types.data();
// Joinables = todas las demas tablas.
static thread_local std::vector<TableInput> joinables_v;
joinables_v.clear();
for (int i = 0; i < (int)tables.size(); ++i) {
if (i != main_idx) joinables_v.push_back(tables[(size_t)i]);
}
const std::vector<TableInput>* joinables = joinables_v.empty() ? nullptr : &joinables_v;
// Per-table chrome visibility (issue: previously global in UiCache → flipping
// one table's "Show UI" affected all tables on screen). Now lives in State.
bool chrome_visible = st.chrome_user_set ? st.chrome_user_visible : show_chrome;
// Toggle Hide/Show UI siempre visible (botoncito arriba a la derecha).
{
float right = ImGui::GetWindowContentRegionMax().x;
ImGui::SetCursorPosX(right - 90.0f);
if (ImGui::SmallButton(chrome_visible ? "Hide UI##chrome" : "Show UI##chrome")) {
st.chrome_user_set = true;
st.chrome_user_visible = !chrome_visible;
}
}
// Main source dropdown — solo si > 1 tabla disponibles.
if (chrome_visible && tables.size() > 1) {
ImGui::SameLine();
float right = ImGui::GetWindowContentRegionMax().x;
ImGui::SetCursorPosX(right - 90.0f - 280.0f);
ImGui::TextDisabled("Main table:");
ImGui::SameLine();
ImGui::SetNextItemWidth(180);
const char* cur_main = main_t.name.c_str();
if (ImGui::BeginCombo("##main_table", cur_main)) {
for (const auto& t : tables) {
bool sel = (t.name == cur_main);
if (ImGui::Selectable(t.name.c_str(), sel)) {
st.main_source = t.name;
}
}
ImGui::EndCombo();
}
}
st.ensure_stage0();
// -------- Pre-pipeline: materialize joins --------
// Si state.joins no vacio + joinables provistos, ejecutar chain de join_tables.
// El resultado reemplaza headers/cells/declared_types para el resto del render.
static thread_local std::vector<std::string> joined_headers_store;
static thread_local std::vector<ColumnType> joined_types_store;
static thread_local std::vector<const char*> joined_headers_ptrs;
static thread_local std::vector<const char*> joined_cells_ptrs;
static thread_local std::vector<ColumnType> joined_declared_types;
static thread_local StageOutput joined_so;
const char* const* headers = headers_in;
const char* const* cells = cells_in;
int row_count = row_count_in;
int orig_cols = col_count;
const ColumnType* declared_types = declared_types_in;
bool joined = false;
if (!st.joins.empty() && joinables && !joinables->empty()) {
joined_so = StageOutput{};
// Build initial left from main.
std::vector<std::string> cur_h(orig_cols);
std::vector<ColumnType> cur_t(orig_cols);
for (int c = 0; c < orig_cols; ++c) {
cur_h[c] = headers_in[c];
cur_t[c] = declared_types_in ? declared_types_in[c] : ColumnType::Auto;
}
const char* const* cur_cells = cells_in;
int cur_rows = row_count_in;
int cur_cols = orig_cols;
// Chain join por cada joins[i].
std::vector<StageOutput> chain;
chain.reserve(st.joins.size());
for (const auto& jn : st.joins) {
const TableInput* match = nullptr;
for (const auto& ti : *joinables) {
if (ti.name == jn.source) { match = &ti; break; }
}
if (!match) continue;
StageOutput so = join_tables(cur_cells, cur_rows, cur_cols,
cur_h, cur_t, *match, jn);
chain.push_back(std::move(so));
const StageOutput& last = chain.back();
cur_cells = last.cells.data();
cur_rows = last.rows;
cur_cols = last.cols;
cur_h = last.headers;
cur_t = last.types;
}
if (!chain.empty()) {
joined = true;
joined_so = std::move(chain.back());
joined_headers_store = joined_so.headers;
joined_types_store = joined_so.types;
joined_headers_ptrs.clear();
joined_cells_ptrs.clear();
for (const auto& s : joined_headers_store) joined_headers_ptrs.push_back(s.c_str());
for (const auto& s : joined_so.cell_backing) joined_cells_ptrs.push_back(s.c_str());
joined_declared_types = joined_types_store;
headers = joined_headers_ptrs.data();
cells = joined_cells_ptrs.data();
row_count = joined_so.rows;
orig_cols = joined_so.cols;
declared_types = joined_declared_types.data();
}
}
Stage& stage0 = st.stages[0];
int eff_cols = orig_cols + (int)stage0.derived.size();
ensure_init(st, eff_cols);
auto& U = ui();
// -------------------------------------------------------------------------
// Issue 0133 — Change 1+2: Columnar snapshot + string interning.
//
// Se reconstruye si:
// - Es el primer frame (last_cells_ptr == nullptr), o
// - El puntero de `cells` cambio (caller reemplazo el buffer).
//
// Snapshot cubre las columnas ORIGINALES (pre-derived) del stage-0 input.
// Las derived columns no se incluyen en el snapshot — se calculan en
// compute_stage y el snapshot solo optimiza el acceso a datos crudos.
//
// StringPool.clear() + rebuild siempre que el snapshot se reconstruya,
// para mantener coherencia de indices entre pool y snapshot.
// -------------------------------------------------------------------------
// Snapshot invalido si:
// 1. El puntero de cells cambio (nuevos datos).
// 2. El pool fue limpiado despues del build (st.string_pool es un nuevo State
// o fue cleared externamente): pool_size_built != strings.size().
// Esto cubre el caso "begin_scenario crea nuevo State con pool vacio pero
// same cells pointer" — sin este check los str_ids apuntarian a un pool
// vacio y se crashearia en pool.at(str_ids[r]).
const bool snap_stale = (U.snapshot.last_cells_ptr != cells) ||
(U.snapshot.pool_size_built !=
(uint32_t)st.string_pool.strings.size() &&
!U.snapshot.cols.empty());
if (snap_stale) {
// Invalidar y reconstruir.
U.snapshot.last_cells_ptr = cells;
U.snapshot.cols.clear();
U.snapshot.cols.resize((size_t)orig_cols);
// Limpiar el StringPool del State para este rebuild.
st.string_pool.clear();
// Reservar capacidad estimada para evitar reallocs que invalidarian
// los string_view del mapa interno del pool.
// Estimamos hasta row_count valores unicos por columna string (worst case).
// En practica muchos menos; reserve no aloca el doble automatico.
st.string_pool.strings.reserve((size_t)(row_count < 65536 ? row_count : 65536));
for (int c = 0; c < orig_cols; ++c) {
ColumnSnapshot& cs = U.snapshot.cols[(size_t)c];
// Detectar tipo efectivo para esta columna.
ColumnType d = declared_types ? declared_types[c] : ColumnType::Auto;
ColumnType ct = effective_type(d, cells, row_count, orig_cols, c);
cs.type = ct;
if (ct == ColumnType::Int) {
cs.i64.resize((size_t)row_count);
for (int r = 0; r < row_count; ++r) {
const char* sv = cells[(size_t)(r * orig_cols + c)];
double tmp = 0.0;
if (sv && parse_number(sv, tmp)) {
cs.i64[(size_t)r] = (int64_t)tmp;
} else {
cs.i64[(size_t)r] = 0;
}
}
} else if (ct == ColumnType::Float) {
cs.f64.resize((size_t)row_count);
for (int r = 0; r < row_count; ++r) {
const char* sv = cells[(size_t)(r * orig_cols + c)];
double tmp = 0.0;
if (sv && parse_number(sv, tmp)) {
cs.f64[(size_t)r] = tmp;
} else {
cs.f64[(size_t)r] = 0.0;
}
}
} else {
// String, Bool, Date, Json, Auto → intern as string.
// Cardinality cap: if >2048 unique values seen in first 25% of rows,
// skip interning this column (high-cardinality cols like timestamps
// offer no compression benefit and hurt cache). str_ids stays empty;
// compare_snap falls back to raw cells for this column.
static const int kCardinalityCap = 2048;
const int sample_n = (row_count < 4) ? row_count : (row_count / 4);
uint32_t pool_before = (uint32_t)st.string_pool.strings.size();
bool skip_intern = false;
for (int r = 0; r < sample_n; ++r) {
const char* sv = cells[(size_t)(r * orig_cols + c)];
std::string_view svv = sv ? std::string_view(sv) : std::string_view("");
st.string_pool.intern(svv);
if ((int)st.string_pool.strings.size() - (int)pool_before > kCardinalityCap) {
skip_intern = true;
break;
}
}
if (skip_intern) {
// Rollback pool entries added during sample (remove tail entries).
// Simpler: just leave pool with sample entries and mark col as no-intern
// by keeping str_ids empty. Pool entries are harmless (amortized).
// cs.str_ids stays empty → compare_snap falls through to raw cells.
cs.str_ids.clear();
} else {
// Low cardinality: intern all rows.
cs.str_ids.resize((size_t)row_count);
// Fill already-sampled rows from pool (intern is idempotent).
for (int r = 0; r < sample_n; ++r) {
const char* sv = cells[(size_t)(r * orig_cols + c)];
std::string_view svv = sv ? std::string_view(sv) : std::string_view("");
cs.str_ids[(size_t)r] = st.string_pool.intern(svv);
}
// Intern remaining rows.
for (int r = sample_n; r < row_count; ++r) {
const char* sv = cells[(size_t)(r * orig_cols + c)];
std::string_view svv = sv ? std::string_view(sv) : std::string_view("");
cs.str_ids[(size_t)r] = st.string_pool.intern(svv);
}
}
}
}
// Record pool size at end of build so validity check is accurate.
U.snapshot.pool_size_built = (uint32_t)st.string_pool.strings.size();
}
// Build eff_headers / src_for_eff / eff_types para STAGE 0.
std::vector<const char*> eff_headers(eff_cols);
std::vector<int> src_for_eff(eff_cols);
std::vector<ColumnType> eff_types(eff_cols);
for (int c = 0; c < eff_cols; ++c) {
if (c < orig_cols) {
eff_headers[c] = headers[c];
src_for_eff[c] = c;
ColumnType d = declared_types ? declared_types[c] : ColumnType::Auto;
eff_types[c] = effective_type(d, cells, row_count, orig_cols, c);
} else {
const DerivedColumn& d = stage0.derived[c - orig_cols];
eff_headers[c] = d.name.c_str();
src_for_eff[c] = d.source_col;
eff_types[c] = d.type;
}
}
static thread_local std::vector<std::string> hn_storage;
static thread_local std::unordered_map<std::string, int> name_to_col;
static thread_local std::unordered_map<std::string, int> derived_n2i;
hn_storage.clear();
name_to_col.clear();
derived_n2i.clear();
hn_storage.reserve(orig_cols);
for (int c = 0; c < orig_cols; ++c) {
hn_storage.emplace_back(headers[c]);
name_to_col[hn_storage.back()] = c;
}
for (int i = 0; i < (int)stage0.derived.size(); ++i) {
derived_n2i[stage0.derived[i].name] = i;
}
// Re-fit auto en cambio de display, stage o config.
auto hash_cfg = [](const ViewConfig& c) -> size_t {
std::string s = c.x_col + "|" + c.cat_col + "|" + c.size_col;
for (auto& y : c.y_cols) { s += "|"; s += y; }
s += "|"; s += std::to_string(c.primary_color);
s += "|"; s += std::to_string(c.hist_bins);
s += "|"; s += std::to_string(c.pie_radius);
s += "|"; s += c.show_legend ? "1" : "0";
s += "|"; s += c.show_markers ? "1" : "0";
return std::hash<std::string>{}(s);
};
size_t cur_cfg_h = hash_cfg(st.viz_config);
if (U.prev_viz_display != st.display || U.prev_viz_stage != st.active_stage ||
U.prev_viz_cfg_h != cur_cfg_h) {
st.viz_config.fit_request = true;
U.prev_viz_display = st.display;
U.prev_viz_stage = st.active_stage;
U.prev_viz_cfg_h = cur_cfg_h;
}
// ----- Breadcrumb + viz selector (chrome) -----
if (chrome_visible) {
draw_stage_breadcrumb(st);
draw_viz_selector(st);
}
int active = st.active_stage;
bool is_raw = (active == 0);
// ----- Chips del stage activo -----
Stage& act = st.stages[active];
if (is_raw && chrome_visible) {
ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(8, 2));
// Joins chip row — solo si hay joinables disponibles.
if (joinables && !joinables->empty()) {
// Nueva firma (issue 0107c): const char* const* eff_headers, int eff_cols, types.
draw_joins_chips(st, *joinables, headers, orig_cols, eff_types);
}
draw_filter_chips(act, eff_headers.data(), eff_cols, eff_types);
draw_add_filter_popup(act, eff_headers.data(), eff_cols, eff_types);
draw_edit_filter_popup(act, eff_headers.data(), eff_cols, eff_types);
// Custom columns chips (solo stage 0)
{
ImGui::PushStyleColor(ImGuiCol_Button, IM_COL32(110, 110, 110, 200));
ImGui::PushStyleColor(ImGuiCol_ButtonHovered, IM_COL32(140, 140, 140, 230));
ImGui::PushStyleColor(ImGuiCol_ButtonActive, IM_COL32( 85, 85, 85, 230));
if (ImGui::SmallButton("+##addcustomcol")) {
U.cf_open = true;
U.cf_editing = false;
U.cf_edit_idx = -1;
U.cf_target_stage = 0;
U.cf_formula.clear();
U.cf_name.clear();
U.cf_type = ColumnType::String;
U.cf_error.clear();
}
ImGui::PopStyleColor(3);
ImGui::SameLine();
bool any = false;
for (size_t i = 0; i < stage0.derived.size(); ++i) {
if (stage0.derived[i].formula.empty()) continue;
any = true;
const auto& d = stage0.derived[i];
char buf[256];
std::snprintf(buf, sizeof(buf), "%s %s x##custom%zu",
column_type_icon(d.type), d.name.c_str(), i);
ImGui::PushStyleColor(ImGuiCol_Button, IM_COL32(140, 140, 140, 220));
ImGui::PushStyleColor(ImGuiCol_ButtonHovered, IM_COL32(170, 170, 170, 240));
ImGui::PushStyleColor(ImGuiCol_ButtonActive, IM_COL32(110, 110, 110, 240));
bool clicked = ImGui::SmallButton(buf);
ImGui::PopStyleColor(3);
if (ImGui::IsItemClicked(ImGuiMouseButton_Right)) {
U.cf_open = true;
U.cf_editing = true;
U.cf_edit_idx = (int)i;
U.cf_target_stage = 0;
U.cf_formula = d.formula;
U.cf_name = d.name;
U.cf_type = d.type;
U.cf_error.clear();
}
if (clicked) {
if (d.lua_id >= 0) lua_engine::release(lua_engine::get(), d.lua_id);
stage0.derived.erase(stage0.derived.begin() + i);
break;
}
ImGui::SameLine();
}
if (!any) ImGui::TextDisabled("Custom columns: + para anadir.");
else ImGui::NewLine();
}
// Sort chips para stage 0 (input headers para popup).
draw_sort_chips(act);
draw_add_sort_popup(act, eff_headers.data(), eff_cols, eff_types);
draw_edit_sort_popup(act, eff_headers.data(), eff_cols);
ImGui::PopStyleVar(); // ItemSpacing
}
// Para stages 1+, compute input headers/types del stage previo.
// Esto requiere compute_stage chain. Lo haremos abajo.
// ---------- Compute view: chain compute_stage 0..active ----------
// Stage 0 expressions: derived cols. Pero compute_stage no sabe de Lua.
// Estrategia: stage 0 lo aplicamos a mano (orig cells + filter + sort)
// y exponemos un eff_cells "virtual" donde derived cols se llenan via Lua
// en el render. Esto preserva el path actual.
//
// Para stages 1+, compute_stage opera sobre cells materializadas. Hay que
// materializar el stage 0 output como cells reales (con derived evaluadas).
// Simpler: si active == 0, mantener el path actual (orig cells + Lua).
// Si active > 0, materializar stage 0 + chain compute_stage(stage 1..active).
if (is_raw) {
// ----- Path stage 0: orig cells + filters/sort manuales + Lua per cell.
// compute_visible_rows opera sobre orig cells. filter.col es eff col,
// hay que traducir a src col (igual que codigo anterior).
State st_tmp = st;
st_tmp.ensure_stage0();
for (auto& f : st_tmp.stages[0].filters) {
if (f.col >= 0 && f.col < eff_cols) f.col = src_for_eff[f.col];
}
// Sort: la pasamos por @idx convention.
st_tmp.stages[0].sorts.clear();
if (!stage0.sorts.empty()) {
// resolve col name -> col idx (de eff_cols) -> src
const SortClause& sc0 = stage0.sorts.front();
int sc_eff = -1;
for (int c = 0; c < eff_cols; ++c) {
if (std::strcmp(eff_headers[c], sc0.col.c_str()) == 0) { sc_eff = c; break; }
}
if (sc_eff >= 0) {
int sc_src = src_for_eff[sc_eff];
char tmp[16]; std::snprintf(tmp, sizeof(tmp), "@%d", sc_src);
st_tmp.stages[0].sorts.push_back({tmp, sc0.desc});
}
}
// Issue 0133 — Change 3: use snapshot-aware overload when snapshot is valid.
auto visible_rows = (U.snapshot.last_cells_ptr == cells && !U.snapshot.cols.empty())
? compute_visible_rows(cells, row_count, orig_cols, st_tmp, U.snapshot, st.string_pool)
: compute_visible_rows(cells, row_count, orig_cols, st_tmp);
int visible_cols = 0;
for (int k = 0; k < eff_cols; ++k) if (st.col_visible[k]) ++visible_cols;
// Snapshot del active output (stage 0) para el config popup.
U.active_headers.clear();
U.active_types.clear();
for (int k = 0; k < eff_cols; ++k) {
if (!st.col_visible[k]) continue;
U.active_headers.emplace_back(eff_headers[k]);
U.active_types.push_back(eff_types[k]);
}
// Input == orig + derived (stage 0 no tiene upstream que agrupe).
U.input_headers_active = U.active_headers;
U.input_types_active = U.active_types;
if (chrome_visible)
{
ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(8, 2));
ImGui::Text("Filas: %d / %d Columnas: %d / %d",
(int)visible_rows.size(), row_count, visible_cols, eff_cols);
ImGui::SameLine();
if (ImGui::SmallButton(st.stats_mode ? "Hide stats" : "Show stats")) {
st.stats_mode = !st.stats_mode;
}
ImGui::SameLine();
if (ImGui::SmallButton("Export CSV")) {
std::string out;
bool first = true;
for (int oc = 0; oc < (int)st.col_order.size(); ++oc) {
int c = st.col_order[oc];
if (c < 0 || c >= eff_cols) continue;
if (!st.col_visible[c]) continue;
if (!first) out += ',';
out += csv_escape(eff_headers[c]);
first = false;
}
out += '\n';
for (int r : visible_rows) {
first = true;
for (int oc = 0; oc < (int)st.col_order.size(); ++oc) {
int c = st.col_order[oc];
if (c < 0 || c >= eff_cols) continue;
if (!st.col_visible[c]) continue;
int src = src_for_eff[c];
if (!first) out += ',';
// Issue 0133 — Change 3: use snapshot for orig cols.
char tmp32[32];
const char* cv = (src < orig_cols && src < (int)U.snapshot.cols.size())
? snap_cell(r, src, U.snapshot, st.string_pool, tmp32)
: nullptr;
if (!cv) cv = cells[r * orig_cols + src];
out += csv_escape(cv);
first = false;
}
out += '\n';
}
const char* p = fn::local_path("export_table.csv");
std::ofstream f(p, std::ios::binary | std::ios::trunc);
if (f) { f << out; U.last_export_path = p; }
}
if (!U.last_export_path.empty()) {
ImGui::SameLine();
ImGui::TextDisabled("-> %s", U.last_export_path.c_str());
}
// TQL bar: Show TQL / Apply TQL / Save .tql / Load .tql (issue 0107c).
{
std::vector<std::string> orig_h(orig_cols);
std::vector<ColumnType> orig_t(orig_cols);
for (int c = 0; c < orig_cols; ++c) { orig_h[c] = headers[c]; orig_t[c] = eff_types[c]; }
draw_tql_bar(U.tql_bar, st, orig_h, orig_t, cells, row_count, orig_cols);
}
ImGui::PopStyleVar();
} // chrome_visible
maybe_recompute_stats(st, cells, row_count, orig_cols, eff_cols,
st_tmp.stages[0].filters,
visible_rows, src_for_eff);
// Toggle Table <-> View: solo visible cuando NO estamos en Table.
// Desde la tabla no se ofrece volver a chart (la tabla es estado
// canonico final). Cambia display via menu/chips si quieres ver chart.
if (st.display != ViewMode::Table) {
draw_table_toggle(st.display, U.last_non_table_main, "main", &st);
}
// SO compartido: main viz + extras. Construido on-demand.
StageOutput so_main;
bool so_built = false;
auto build_so = [&]() -> StageOutput& {
if (so_built) return so_main;
so_built = true;
std::vector<int> vcols;
for (int c = 0; c < eff_cols; ++c) if (st.col_visible[c]) vcols.push_back(c);
so_main.cols = (int)vcols.size();
so_main.rows = (int)visible_rows.size();
so_main.headers.reserve(so_main.cols);
so_main.types.reserve(so_main.cols);
for (int c : vcols) {
so_main.headers.emplace_back(eff_headers[c]);
so_main.types.push_back(eff_types[c]);
}
so_main.cell_backing.reserve((size_t)so_main.rows * so_main.cols);
for (int r : visible_rows) {
for (int c : vcols) {
if (c < orig_cols) {
// Raw pointer: materialization copies to string anyway — snapshot
// path offers no benefit here and adds snprintf overhead for Int/Float.
const char* p = cells[r * orig_cols + c];
so_main.cell_backing.emplace_back(p ? p : "");
} else {
const DerivedColumn& d = stage0.derived[c - orig_cols];
if (!d.formula.empty() && d.lua_id >= 0) {
lua_engine::RowCtx ctx;
ctx.cells = cells;
ctx.orig_cols = orig_cols;
ctx.row = r;
ctx.header_names = &hn_storage;
ctx.name_to_col = &name_to_col;
ctx.types_orig = eff_types.data();
ctx.n_types_orig = orig_cols;
ctx.derived = &stage0.derived;
ctx.derived_name_to_idx = &derived_n2i;
std::string err;
so_main.cell_backing.emplace_back(
lua_engine::eval(lua_engine::get(), d.lua_id, ctx, &err));
} else {
int src = d.source_col;
const char* sp = (src >= 0 && src < orig_cols) ? cells[r * orig_cols + src] : "";
so_main.cell_backing.emplace_back(sp ? sp : "");
}
}
}
}
so_main.cells.reserve(so_main.cell_backing.size());
for (auto& s : so_main.cell_backing) so_main.cells.push_back(s.c_str());
return so_main;
};
if (visible_cols == 0) {
ImGui::TextDisabled("(todas las columnas ocultas)");
// Modales fuera del table block.
} else if (st.display != ViewMode::Table) {
viz::render(build_so(), st.display, st.viz_config, ImVec2(-1, -1));
} else {
// ----- Grid stage 0: pre-materialise all effective cells (raw + Lua derived)
// into a contiguous buffer, then delegate rendering to render_grid_stage0.
// This keeps Lua evaluation in the entrypoint and the ImGui loop in the TU.
// (issue 0107c)
int n_vrows = (int)visible_rows.size();
std::vector<std::string> s0_backing;
std::vector<const char*> s0_cells;
s0_backing.reserve((size_t)n_vrows * eff_cols);
s0_cells.reserve((size_t)n_vrows * eff_cols);
for (int ri = 0; ri < n_vrows; ++ri) {
int r = visible_rows[ri];
for (int c = 0; c < eff_cols; ++c) {
if (c >= orig_cols && !stage0.derived[c - orig_cols].formula.empty()) {
const auto& d = stage0.derived[c - orig_cols];
if (d.lua_id < 0) {
s0_backing.emplace_back("?");
} else {
lua_engine::RowCtx ctx;
ctx.cells = cells;
ctx.orig_cols = orig_cols;
ctx.row = r;
ctx.header_names = &hn_storage;
ctx.name_to_col = &name_to_col;
ctx.types_orig = eff_types.data();
ctx.n_types_orig = orig_cols;
ctx.derived = &stage0.derived;
ctx.derived_name_to_idx = &derived_n2i;
std::string err;
s0_backing.emplace_back(
lua_engine::eval(lua_engine::get(), d.lua_id, ctx, &err));
}
} else {
int src = src_for_eff[c];
// Raw pointer: materialization copies to string anyway.
const char* p = cells[r * orig_cols + src];
s0_backing.emplace_back(p ? p : "");
}
}
}
// Build pointer array after backing is stable (no realloc).
for (auto& s : s0_backing) s0_cells.push_back(s.c_str());
// Identity src map: pre-materialised array has eff_cols per row.
std::vector<int> s0_src(eff_cols);
for (int c = 0; c < eff_cols; ++c) s0_src[c] = c;
// Build sequential visible_rows for the materialised buffer.
std::vector<int> s0_vrows(n_vrows);
for (int i = 0; i < n_vrows; ++i) s0_vrows[i] = i;
render_grid_stage0(id, st,
s0_cells.data(),
n_vrows, eff_cols, eff_cols,
eff_headers.data(), eff_types.data(), s0_src.data(),
s0_vrows, main_t, events_out);
}
// Render extras panels (stage 0 path). Solo cuando display != Table —
// desde la tabla no se muestran chart panels adicionales.
if (st.display != ViewMode::Table && !st.extra_panels.empty() && visible_cols > 0) {
int close_idx = -1;
const std::vector<ColumnSpec>* ep_specs =
main_t.column_specs.empty() ? nullptr : &main_t.column_specs;
for (int i = 0; i < (int)st.extra_panels.size(); ++i) {
if (draw_extra_panel(st, st.extra_panels[i], i, build_so(), ep_specs)) close_idx = i;
}
if (close_idx >= 0) st.extra_panels.erase(st.extra_panels.begin() + close_idx);
}
} else {
// ----- Path stage > 0: materializar stage 0 con cells reales + chain.
// Materializar stage 0: aplicar filters/sort sobre orig + evaluar derived.
State st_tmp = st;
for (auto& f : st_tmp.stages[0].filters) {
if (f.col >= 0 && f.col < eff_cols) f.col = src_for_eff[f.col];
}
st_tmp.stages[0].sorts.clear();
if (!stage0.sorts.empty()) {
const SortClause& sc0 = stage0.sorts.front();
int sc_eff = -1;
for (int c = 0; c < eff_cols; ++c) {
if (std::strcmp(eff_headers[c], sc0.col.c_str()) == 0) { sc_eff = c; break; }
}
if (sc_eff >= 0) {
int sc_src = src_for_eff[sc_eff];
char tmp[16]; std::snprintf(tmp, sizeof(tmp), "@%d", sc_src);
st_tmp.stages[0].sorts.push_back({tmp, sc0.desc});
}
}
// Issue 0133 — Change 3: use snapshot-aware filter/sort when available.
auto vrows = (U.snapshot.last_cells_ptr == cells && !U.snapshot.cols.empty())
? compute_visible_rows(cells, row_count, orig_cols, st_tmp, U.snapshot, st.string_pool)
: compute_visible_rows(cells, row_count, orig_cols, st_tmp);
// Materializar stage0 output: cells (eff_cols) con derived evaluadas.
std::vector<std::string> mat_backing;
std::vector<const char*> mat_cells;
mat_backing.reserve((size_t)vrows.size() * eff_cols);
mat_cells.reserve((size_t)vrows.size() * eff_cols);
for (int r : vrows) {
for (int c = 0; c < eff_cols; ++c) {
if (c < orig_cols) {
// Raw pointer: materialization copies to string anyway.
const char* p = cells[r * orig_cols + c];
mat_backing.emplace_back(p ? p : "");
} else {
const DerivedColumn& d = stage0.derived[c - orig_cols];
if (!d.formula.empty()) {
if (d.lua_id < 0) {
mat_backing.emplace_back("");
} else {
lua_engine::RowCtx ctx;
ctx.cells = cells;
ctx.orig_cols = orig_cols;
ctx.row = r;
ctx.header_names = &hn_storage;
ctx.name_to_col = &name_to_col;
ctx.types_orig = eff_types.data();
ctx.n_types_orig = orig_cols;
ctx.derived = &stage0.derived;
ctx.derived_name_to_idx = &derived_n2i;
std::string err;
mat_backing.emplace_back(
lua_engine::eval(lua_engine::get(), d.lua_id, ctx, &err));
}
} else {
// retipo puro — raw pointer from orig cells.
int src = d.source_col;
const char* sp = (src >= 0 && src < orig_cols) ? cells[r * orig_cols + src] : "";
mat_backing.emplace_back(sp ? sp : "");
}
}
}
}
// Punteros tras llenar backing (reserve garantiza no realloc).
for (auto& s : mat_backing) mat_cells.push_back(s.c_str());
std::vector<std::string> cur_headers(eff_cols);
std::vector<ColumnType> cur_types(eff_cols);
for (int c = 0; c < eff_cols; ++c) {
cur_headers[c] = eff_headers[c];
cur_types[c] = eff_types[c];
}
// Chain compute_stage 1..active.
// Para encadenar, mantenemos vectores por iteracion. cur_cells apunta al
// ultimo output.
const char* const* cur_cells = mat_cells.data();
int cur_rows = (int)vrows.size();
int cur_cols_n = eff_cols;
std::vector<StageOutput> outs;
outs.reserve(st.stages.size());
// Headers del INPUT del active (= output del active-1)
std::vector<std::string> input_headers_active = cur_headers;
std::vector<ColumnType> input_types_active = cur_types;
for (int si = 1; si <= active; ++si) {
const Stage& sN = st.stages[si];
// Antes de computar: si es el active stage, los input_headers son cur_*.
if (si == active) {
input_headers_active = cur_headers;
input_types_active = cur_types;
}
StageOutput so = compute_stage(cur_cells, cur_rows, cur_cols_n,
cur_headers, cur_types, sN);
outs.push_back(std::move(so));
const StageOutput& last = outs.back();
cur_cells = last.cells.data();
cur_rows = last.rows;
cur_cols_n = last.cols;
cur_headers = last.headers;
cur_types = last.types;
}
// ----- Chips del active stage (uses input_headers_active) -----
std::vector<const char*> ih_ptrs(input_headers_active.size());
for (size_t i = 0; i < input_headers_active.size(); ++i)
ih_ptrs[i] = input_headers_active[i].c_str();
int in_cols_n = (int)input_headers_active.size();
if (chrome_visible) {
ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(8, 2));
draw_filter_chips(act, ih_ptrs.data(), in_cols_n, input_types_active);
draw_add_filter_popup(act, ih_ptrs.data(), in_cols_n, input_types_active);
draw_edit_filter_popup(act, ih_ptrs.data(), in_cols_n, input_types_active);
draw_breakout_chips(act, ih_ptrs.data(), in_cols_n, input_types_active);
draw_add_breakout_popup(act, ih_ptrs.data(), in_cols_n, input_types_active,
cur_cells, cur_rows);
draw_edit_breakout_popup(act, ih_ptrs.data(), in_cols_n);
draw_aggregation_chips(act, ih_ptrs.data(), in_cols_n);
draw_add_aggregation_popup(act, ih_ptrs.data(), in_cols_n, input_types_active);
draw_edit_agg_popup(act, ih_ptrs.data(), in_cols_n);
// ----- Custom column chips (stages 1+, target = active stage) -----
{
ImGui::PushStyleColor(ImGuiCol_Button, IM_COL32(110, 110, 110, 200));
ImGui::PushStyleColor(ImGuiCol_ButtonHovered, IM_COL32(140, 140, 140, 230));
ImGui::PushStyleColor(ImGuiCol_ButtonActive, IM_COL32( 85, 85, 85, 230));
if (ImGui::SmallButton("+##addcustomcol_stage")) {
U.cf_open = true;
U.cf_editing = false;
U.cf_edit_idx = -1;
U.cf_target_stage = active;
U.cf_formula.clear();
U.cf_name.clear();
U.cf_type = ColumnType::String;
U.cf_error.clear();
}
ImGui::PopStyleColor(3);
ImGui::SameLine();
bool any = false;
for (size_t i = 0; i < act.derived.size(); ++i) {
if (act.derived[i].formula.empty()) continue;
any = true;
const auto& d = act.derived[i];
char buf[256];
std::snprintf(buf, sizeof(buf), "%s %s x##custom_st_%zu",
column_type_icon(d.type), d.name.c_str(), i);
ImGui::PushStyleColor(ImGuiCol_Button, IM_COL32(140, 140, 140, 220));
ImGui::PushStyleColor(ImGuiCol_ButtonHovered, IM_COL32(170, 170, 170, 240));
ImGui::PushStyleColor(ImGuiCol_ButtonActive, IM_COL32(110, 110, 110, 240));
bool clicked = ImGui::SmallButton(buf);
ImGui::PopStyleColor(3);
if (ImGui::IsItemClicked(ImGuiMouseButton_Right)) {
U.cf_open = true;
U.cf_editing = true;
U.cf_edit_idx = (int)i;
U.cf_target_stage = active;
U.cf_formula = d.formula;
U.cf_name = d.name;
U.cf_type = d.type;
U.cf_error.clear();
}
if (clicked) {
if (d.lua_id >= 0) lua_engine::release(lua_engine::get(), d.lua_id);
act.derived.erase(act.derived.begin() + i);
break;
}
ImGui::SameLine();
}
if (!any) ImGui::TextDisabled("Custom columns (stage %d): + para anadir.", active);
else ImGui::NewLine();
}
draw_sort_chips(act);
// Sort col options son los headers del OUTPUT del stage activo.
std::vector<const char*> out_h_ptrs(cur_headers.size());
for (size_t i = 0; i < cur_headers.size(); ++i) out_h_ptrs[i] = cur_headers[i].c_str();
draw_add_sort_popup(act, out_h_ptrs.data(), (int)cur_headers.size(), cur_types);
draw_edit_sort_popup(act, out_h_ptrs.data(), (int)cur_headers.size());
ImGui::PopStyleVar();
} // chrome_visible
// ----- Materializar act.derived sobre cur_cells -----
// Para cada derived col formula del active stage, eval per output row.
std::vector<std::string> ext_backing;
std::vector<const char*> ext_cells;
std::vector<std::string> ext_headers;
std::vector<ColumnType> ext_types;
if (!act.derived.empty()) {
int orig_out_cols = cur_cols_n;
std::vector<std::string> out_hn = cur_headers;
std::unordered_map<std::string, int> out_n2c;
for (size_t i = 0; i < out_hn.size(); ++i) out_n2c[out_hn[i]] = (int)i;
int n_derived = (int)act.derived.size();
int new_cols = orig_out_cols + n_derived;
ext_backing.reserve((size_t)cur_rows * n_derived);
ext_cells.reserve((size_t)cur_rows * new_cols);
for (int r = 0; r < cur_rows; ++r) {
// copia cols originales del output
for (int c = 0; c < orig_out_cols; ++c) {
ext_cells.push_back(cur_cells[r * orig_out_cols + c]);
}
// anade derived eval
for (int k = 0; k < n_derived; ++k) {
const DerivedColumn& d = act.derived[k];
if (d.formula.empty() || d.lua_id < 0) {
ext_backing.emplace_back("");
} else {
lua_engine::RowCtx ctx;
ctx.cells = cur_cells;
ctx.orig_cols = orig_out_cols;
ctx.row = r;
ctx.header_names = &out_hn;
ctx.name_to_col = &out_n2c;
ctx.types_orig = cur_types.data();
ctx.n_types_orig = orig_out_cols;
std::string e;
ext_backing.emplace_back(
lua_engine::eval(lua_engine::get(), d.lua_id, ctx, &e));
}
// marker placeholder; sera replaced abajo tras backing estable
ext_cells.push_back(nullptr);
}
}
// Construir ext_cells reemplazando placeholders por punteros estables.
size_t bi = 0;
for (int r = 0; r < cur_rows; ++r) {
for (int k = 0; k < n_derived; ++k) {
int idx = r * new_cols + orig_out_cols + k;
ext_cells[idx] = ext_backing[bi++].c_str();
}
}
ext_headers = cur_headers;
ext_types = cur_types;
for (int k = 0; k < n_derived; ++k) {
ext_headers.push_back(act.derived[k].name);
ext_types.push_back(act.derived[k].type);
}
cur_cells = ext_cells.data();
cur_cols_n = new_cols;
cur_headers = ext_headers;
cur_types = ext_types;
}
// Header row + cells render simple (sin clipper porque outputs son
// pequenos tipicamente).
// Snapshot del active output (stage>0) para config popup.
U.active_headers = cur_headers;
U.active_types = cur_types;
// Input del active stage = output del previo. Disponible en
// input_headers_active/input_types_active.
U.input_headers_active = input_headers_active;
U.input_types_active = input_types_active;
if (chrome_visible) {
ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(8, 2));
ImGui::Text("Filas: %d Columnas: %d", cur_rows, cur_cols_n);
ImGui::SameLine();
if (ImGui::SmallButton(st.stats_mode ? "Hide stats" : "Show stats")) {
st.stats_mode = !st.stats_mode;
}
// Recompute stats sobre cur_cells del stage activo.
if (st.stats_mode && cur_cols_n > 0) {
st.stats_cache.resize(cur_cols_n);
st.stats_last_cells = cur_cells;
for (int c = 0; c < cur_cols_n; ++c) {
st.stats_cache[c] = compute_column_stats(cur_cells, cur_rows, cur_cols_n, c);
}
}
// TQL bar: Show TQL / Apply TQL / Save .tql / Load .tql (issue 0107c).
{
std::vector<std::string> orig_h(orig_cols);
std::vector<ColumnType> orig_t(orig_cols);
for (int c = 0; c < orig_cols; ++c) { orig_h[c] = headers[c]; orig_t[c] = eff_types[c]; }
draw_tql_bar(U.tql_bar, st, orig_h, orig_t, cells, row_count, orig_cols);
}
ImGui::PopStyleVar();
} // chrome_visible
// Toggle Table <-> View: solo visible cuando NO estamos en Table.
if (st.display != ViewMode::Table) {
draw_table_toggle(st.display, U.last_non_table_main, "main2", &st);
}
if (st.display != ViewMode::Table && cur_cols_n > 0) {
// outs.back() es el StageOutput del active. Si active no tiene outs
// (cur_rows poblado pero outs vacio cuando active>0 y chain corta),
// construir uno on-the-fly desde cur_cells.
StageOutput so_local;
const StageOutput* so_ptr = nullptr;
if (!outs.empty()) {
so_ptr = &outs.back();
} else {
so_local.cols = cur_cols_n;
so_local.rows = cur_rows;
so_local.headers = cur_headers;
so_local.types = cur_types;
so_local.cells.reserve((size_t)cur_rows * cur_cols_n);
for (int i = 0; i < cur_rows * cur_cols_n; ++i)
so_local.cells.push_back(cur_cells[i]);
so_ptr = &so_local;
}
int clicked_row = -1;
viz::render(*so_ptr, st.display, st.viz_config, ImVec2(-1, -1), &clicked_row);
// Fase 10: click sobre chart -> drill al stage previo usando
// breakout col[0] como filtro Op::Eq sobre cells[clicked_row].
if (clicked_row >= 0 && active > 0 &&
so_ptr->cols > 0 && clicked_row < so_ptr->rows) {
int n_brk = (int)st.stages[active].breakouts.size();
if (n_brk > 0) {
const char* v = so_ptr->cells[clicked_row * so_ptr->cols + 0];
std::string col_clean;
parse_breakout_granularity(so_ptr->headers[0], col_clean);
drill_into(st, active, col_clean,
v ? std::string(v) : "",
input_headers_active);
}
}
goto stage_n_table_end;
}
// Grid stage>0: delegate to render_grid_stage_n (issue 0107c).
{
int n_brk = (int)st.stages[active].breakouts.size();
render_grid_stage_n(id, st, cur_cells, cur_rows, cur_cols_n,
cur_headers, cur_types, input_headers_active,
n_brk, main_t, events_out);
}
stage_n_table_end:;
// Render extras (stage>0 path). Solo cuando display != Table.
if (st.display != ViewMode::Table && !st.extra_panels.empty() && cur_cols_n > 0) {
StageOutput so_local;
const StageOutput* so_ptr = nullptr;
if (!outs.empty()) {
so_ptr = &outs.back();
} else {
so_local.cols = cur_cols_n;
so_local.rows = cur_rows;
so_local.headers = cur_headers;
so_local.types = cur_types;
so_local.cells.reserve((size_t)cur_rows * cur_cols_n);
for (int i = 0; i < cur_rows * cur_cols_n; ++i)
so_local.cells.push_back(cur_cells[i]);
so_ptr = &so_local;
}
int close_idx = -1;
const std::vector<ColumnSpec>* ep_specs2 =
main_t.column_specs.empty() ? nullptr : &main_t.column_specs;
for (int i = 0; i < (int)st.extra_panels.size(); ++i) {
if (draw_extra_panel(st, st.extra_panels[i], i, *so_ptr, ep_specs2)) close_idx = i;
}
if (close_idx >= 0) st.extra_panels.erase(st.extra_panels.begin() + close_idx);
}
}
// ---------- Modales (comunes a ambos paths) ----------
if (U.cf_open) ImGui::OpenPopup("Custom column");
if (ImGui::BeginPopupModal("Custom column", &U.cf_open,
ImGuiWindowFlags_AlwaysAutoResize))
{
ImGui::Text("Nombre:");
char name_buf[128] = {0};
std::snprintf(name_buf, sizeof(name_buf), "%s", U.cf_name.c_str());
ImGui::SetNextItemWidth(520);
if (ImGui::InputText("##cfname", name_buf, sizeof(name_buf))) U.cf_name = name_buf;
ImGui::Spacing();
ImGui::Text("Formula (Lua). Acceso celdas via row.<col_name> o row[idx].");
ImGui::TextDisabled("Ejemplo: return row.size_kb * 1024");
static char formula_buf[4096] = {0};
if (U.cf_force_cursor || std::strcmp(formula_buf, U.cf_formula.c_str()) != 0) {
std::snprintf(formula_buf, sizeof(formula_buf), "%s", U.cf_formula.c_str());
}
ImGuiInputTextFlags flags =
ImGuiInputTextFlags_CallbackEdit | ImGuiInputTextFlags_CallbackAlways;
if (ImGui::InputTextMultiline("##cfformula", formula_buf, sizeof(formula_buf),
ImVec2(520, 200), flags, autocomplete_cb, &U)) {
U.cf_formula = formula_buf;
}
if (U.cf_ac_open) {
ImVec2 box_min = ImGui::GetItemRectMin();
ImVec2 box_max = ImGui::GetItemRectMax();
ImGui::SetNextWindowPos(ImVec2(box_min.x + 20, box_max.y + 4));
ImGui::SetNextWindowSize(ImVec2(280, 0));
ImGuiWindowFlags wf =
ImGuiWindowFlags_NoTitleBar | ImGuiWindowFlags_NoResize |
ImGuiWindowFlags_NoMove | ImGuiWindowFlags_NoSavedSettings |
ImGuiWindowFlags_NoFocusOnAppearing | ImGuiWindowFlags_AlwaysAutoResize;
if (ImGui::Begin("##colpicker", nullptr, wf)) {
ImGui::TextDisabled("Pick column:");
ImGui::Separator();
auto ci_contains = [](const std::string& hay, const std::string& nd) {
if (nd.empty()) return true;
std::string a = hay, b = nd;
for (char& c : a) if (c >= 'A' && c <= 'Z') c += 32;
for (char& c : b) if (c >= 'A' && c <= 'Z') c += 32;
return a.find(b) != std::string::npos;
};
int shown = 0;
for (int c = 0; c < eff_cols && shown < 12; ++c) {
std::string nm = eff_headers[c];
if (!ci_contains(nm, U.cf_ac_filter)) continue;
char lbl[200];
std::snprintf(lbl, sizeof(lbl), "%s %s",
column_type_icon(eff_types[c]), nm.c_str());
if (ImGui::Selectable(lbl)) {
int new_cursor = 0;
std::string updated = insert_column_ref(
U.cf_formula, U.cf_ac_start, U.cf_ac_cursor, nm, new_cursor);
U.cf_formula = updated;
U.cf_target_cursor= new_cursor;
U.cf_force_cursor = true;
U.cf_ac_open = false;
}
++shown;
}
if (shown == 0) ImGui::TextDisabled("(sin matches)");
}
ImGui::End();
}
if (!U.cf_error.empty()) {
ImGui::PushStyleColor(ImGuiCol_Text, IM_COL32(230, 100, 100, 255));
ImGui::TextWrapped("Error: %s", U.cf_error.c_str());
ImGui::PopStyleColor();
}
if (ImGui::Button("Compile & save")) {
std::string err;
int lid = lua_engine::compile(lua_engine::get(), U.cf_formula, &err);
if (lid < 0) {
U.cf_error = err;
} else {
// Build sample context segun cf_target_stage.
// target == 0: usa orig cells + stage 0 derived.
// target > 0: recomputa chain hasta el target (excluyendo
// derived del target) y sample sobre ese output.
int ts = U.cf_target_stage;
if (ts < 0 || ts >= (int)st.stages.size()) ts = 0;
int sample = 0;
std::vector<std::string> samples_str;
if (ts == 0) {
sample = std::min(64, row_count);
for (int r = 0; r < sample; ++r) {
lua_engine::RowCtx ctx;
ctx.cells = cells;
ctx.orig_cols = orig_cols;
ctx.row = r;
ctx.header_names = &hn_storage;
ctx.name_to_col = &name_to_col;
ctx.types_orig = eff_types.data();
ctx.n_types_orig = orig_cols;
ctx.derived = &stage0.derived;
ctx.derived_name_to_idx = &derived_n2i;
std::string e;
samples_str.emplace_back(
lua_engine::eval(lua_engine::get(), lid, ctx, &e));
}
} else {
// Recompute chain hasta stage ts output (sin aplicar derived
// del propio ts).
State st_sample = st;
// Limpia derived del target stage para que el sample no
// se referencie a si mismo.
if (ts < (int)st_sample.stages.size())
st_sample.stages[ts].derived.clear();
// Reusa la logica de materializacion: simple recompute manual.
// Aplica stage 0 (orig + derived) materializado.
State stmp = st;
Stage& s0 = stmp.stages[0];
for (auto& f : s0.filters) {
if (f.col >= 0 && f.col < eff_cols) f.col = src_for_eff[f.col];
}
s0.sorts.clear();
auto v0 = compute_visible_rows(cells, row_count, orig_cols, stmp);
std::vector<std::string> mb;
std::vector<const char*> mc;
mb.reserve((size_t)v0.size() * eff_cols);
mc.reserve((size_t)v0.size() * eff_cols);
for (int r : v0) {
for (int c = 0; c < eff_cols; ++c) {
if (c < orig_cols) {
const char* p = cells[r * orig_cols + c];
mb.emplace_back(p ? p : "");
} else {
const DerivedColumn& d = stage0.derived[c - orig_cols];
if (!d.formula.empty() && d.lua_id >= 0) {
lua_engine::RowCtx ctx;
ctx.cells = cells; ctx.orig_cols = orig_cols;
ctx.row = r;
ctx.header_names = &hn_storage;
ctx.name_to_col = &name_to_col;
ctx.types_orig = eff_types.data();
ctx.n_types_orig = orig_cols;
ctx.derived = &stage0.derived;
ctx.derived_name_to_idx = &derived_n2i;
std::string e;
mb.emplace_back(lua_engine::eval(lua_engine::get(), d.lua_id, ctx, &e));
} else if (d.source_col >= 0) {
const char* p = cells[r * orig_cols + d.source_col];
mb.emplace_back(p ? p : "");
} else mb.emplace_back("");
}
}
}
for (auto& s : mb) mc.push_back(s.c_str());
std::vector<std::string> ch(eff_cols);
std::vector<ColumnType> ct(eff_cols);
for (int c = 0; c < eff_cols; ++c) { ch[c] = eff_headers[c]; ct[c] = eff_types[c]; }
const char* const* cc = mc.data();
int cr = (int)v0.size();
int cn = eff_cols;
std::vector<StageOutput> tmps;
for (int si = 1; si <= ts; ++si) {
Stage stage_sn = st.stages[si];
// En el target stage NO apliques sus propias derived.
if (si == ts) stage_sn.derived.clear();
tmps.push_back(compute_stage(cc, cr, cn, ch, ct, stage_sn));
const StageOutput& l = tmps.back();
cc = l.cells.data(); cr = l.rows; cn = l.cols;
ch = l.headers; ct = l.types;
}
// Build name_to_col map for the target stage output.
std::vector<std::string> hn_t = ch;
std::unordered_map<std::string, int> n2c_t;
for (size_t i = 0; i < hn_t.size(); ++i) n2c_t[hn_t[i]] = (int)i;
sample = std::min(64, cr);
for (int r = 0; r < sample; ++r) {
lua_engine::RowCtx ctx;
ctx.cells = cc;
ctx.orig_cols = cn;
ctx.row = r;
ctx.header_names = &hn_t;
ctx.name_to_col = &n2c_t;
ctx.types_orig = ct.data();
ctx.n_types_orig = cn;
std::string e;
samples_str.emplace_back(
lua_engine::eval(lua_engine::get(), lid, ctx, &e));
}
}
std::vector<const char*> samples_ptr;
samples_ptr.reserve(samples_str.size());
for (auto& s : samples_str) samples_ptr.push_back(s.c_str());
ColumnType auto_t = auto_detect_type(samples_ptr.data(),
(int)samples_ptr.size(), 1, 0);
// Save to target stage.
if (ts < 0 || ts >= (int)st.stages.size()) ts = 0;
auto& target_derived = st.stages[ts].derived;
if (U.cf_editing && U.cf_edit_idx >= 0 &&
U.cf_edit_idx < (int)target_derived.size())
{
auto& d = target_derived[U.cf_edit_idx];
if (d.lua_id >= 0) lua_engine::release(lua_engine::get(), d.lua_id);
d.formula = U.cf_formula;
d.name = U.cf_name.empty() ? "custom" : U.cf_name;
d.type = auto_t;
d.lua_id = lid;
d.compile_error.clear();
} else {
DerivedColumn d;
d.source_col = -1;
d.type = auto_t;
d.name = U.cf_name.empty() ? "custom" : U.cf_name;
d.formula = U.cf_formula;
d.lua_id = lid;
target_derived.push_back(d);
}
U.cf_open = false;
U.cf_error.clear();
}
}
ImGui::SameLine();
if (ImGui::Button("Cancel")) {
U.cf_open = false;
U.cf_error.clear();
}
ImGui::EndPopup();
}
// TQL Show / Apply modals: ahora gestionados dentro de draw_tql_bar (issue 0107c).
// draw_tql_bar se llama dos veces: stage 0 y stage N chrome.
// Los modales se abren cuando tql_bar.show_open / tql_bar.apply_open son true.
// No hace falta codigo adicional aqui.
// Ask AI modal (fase 11 — issue 0080). Extraido a data_table_ai_panel (issue 0107c).
draw_ask_ai_modal(U.ask_ai, st, U.active_headers, U.active_types,
(int)U.active_headers.size());
if (U.open_cell_popup) { ImGui::OpenPopup("##cell_op"); U.open_cell_popup = false; }
if (ImGui::BeginPopup("##cell_op")) {
ColumnType t = (U.pending_col >= 0 && U.pending_col < eff_cols)
? eff_types[U.pending_col] : ColumnType::String;
const char* hdr = (U.pending_col >= 0 && U.pending_col < eff_cols)
? eff_headers[U.pending_col] : "?";
ImGui::TextDisabled("%s %s ?? \"%s\"",
column_type_icon(t), hdr, U.pending_value.c_str());
ImGui::Separator();
auto ops = ops_for_type(t);
for (Op o : ops) {
if (ImGui::MenuItem(op_label(o))) {
st.stages[0].filters.push_back({U.pending_col, o, U.pending_value});
ImGui::CloseCurrentPopup();
}
}
ImGui::EndPopup();
}
}
} // namespace data_table
Reference in New Issue View Git Blame Copy Permalink