feat(datascience): rigor experimental para papers — effect size, IC, Holm + preregistro inmutable

Subsistema de papers reproducibles (grupo de capacidad `papers`). Añade las
funciones estadísticas que un paper honesto necesita y la función que congela la
hipótesis antes de mirar los datos (anti-HARKing).

Nuevas funciones (puras salvo la última):
- effect_size_cohens_d: Cohen's d + Hedges' g (corrección de sesgo para N
  pequeño) + interpretación cualitativa (negligible/small/medium/large por los
  umbrales de Cohen). Dict-no-throw ante varianza cero / N insuficiente.
- confidence_interval_mean: intervalo de confianza de una media (t de Student) o
  de la diferencia de medias con Welch (df de Welch–Satterthwaite, sin asumir
  varianzas iguales). Dict-no-throw; el IC colapsa al punto cuando la varianza es
  cero.
- preregister_hypothesis (impura): congela hipótesis + plan de análisis en
  papers/<slug>/preregistration.md con frozen_at (UTC) y content_hash (sha256 del
  cuerpo normalizado, no del frontmatter). Inmutabilidad: una vez frozen, un
  contenido distinto se RECHAZA sin sobrescribir (mata el HARKing); idempotente si
  el contenido es idéntico. Siempre dict-no-throw.

Extensión:
- fdr_correction 1.0.0 -> 1.1.0: añade method="holm" (Holm-Bonferroni step-down,
  controla FWER, más potente que Bonferroni simple). Reúsa la maquinaria de
  alineación 1:1 con None/inválidos; no rompe los métodos bh/bonferroni.

Reutiliza del registry: fdr_correction (BH + Bonferroni ya existían) como base
para Holm. pearson y spearman_corr ya cubrían correlación.

Tests: 36 pytest verdes (cohen/hedges 8, confidence/welch 8, fdr/holm/bonferroni
12, preregister 4 + extras), golden contra valores conocidos y validados con
scipy. Golden manual del preregistro: congela, idempotente, rechaza edición
(bytes en disco idénticos al congelado).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

python/functions/datascience/__init__.py

@@ -59,11 +59,15 @@ from .acf_pacf import acf_pacf
 from .stl_decompose import stl_decompose
 from .to_returns import to_returns
 from .fdr_correction import fdr_correction
+from .effect_size_cohens_d import effect_size_cohens_d
+from .confidence_interval_mean import confidence_interval_mean
+from .preregister_hypothesis import preregister_hypothesis
 from .suggest_reexpression import suggest_reexpression
 from .exploratory_caveats import exploratory_caveats
 from .render_eda_pdf import render_eda_pdf, render_eda_pdf_relational
 from .render_automatic_eda_pdf import render_automatic_eda_pdf
 from .render_automatic_eda_pptx import render_automatic_eda_pptx
+from .render_automatic_eda_markdown import render_automatic_eda_markdown
 from .detect_time_column import detect_time_column
 from .extract_timeseries_raw import extract_timeseries_raw
 from .build_eda_render_ctx import build_eda_render_ctx
@@ -82,12 +86,16 @@ __all__ = [
     "resample_timeseries",
     "render_automatic_eda_pdf",
     "render_automatic_eda_pptx",
+    "render_automatic_eda_markdown",
     "decode_qr_image",
     "adf_kpss_stationarity",
     "acf_pacf",
     "stl_decompose",
     "to_returns",
     "fdr_correction",
+    "effect_size_cohens_d",
+    "confidence_interval_mean",
+    "preregister_hypothesis",
     "suggest_reexpression",
     "exploratory_caveats",
     "render_eda_pdf",

python/functions/datascience/automatic_eda/__init__.py

@@ -36,6 +36,7 @@ from .model import (  # noqa: F401
 from .chapters_registry import CHAPTER_ORDER, build_chapter, build_document  # noqa: F401
 from .render_pdf_impl import render_pdf  # noqa: F401
 from .render_pptx_impl import render_pptx  # noqa: F401
+from .render_md_impl import render_md  # noqa: F401
 
 __all__ = [
     "ENGINE_NAME",
@@ -60,4 +61,5 @@ __all__ = [
     "build_document",
     "render_pdf",
     "render_pptx",
+    "render_md",
 ]

python/functions/datascience/automatic_eda/chapters/cat_distr.py

@@ -1,19 +1,25 @@
 """Categorical distributions chapter (CAT DISTR).
 
-Third reference chapter for AutomaticEDA. For every categorical column it shows,
-fulfilling the user's request:
+Third reference chapter for AutomaticEDA. Each categorical column gets **its own
+page (PDF) / slide (PPTX)**: every column is wrapped in a keep-together
+``model.Group`` with ``page_break_before=True`` (except the first, which may share
+the intro's page), so its chart sits next to its tables and no column is split.
 
-1. A short opening explanation of **Shannon entropy** (what it measures, its 0
-   and log2(k) bounds, the normalized 0–1 version) and the dataset row total used
-   as a comparison baseline.
-2. Per column, a cardinality key/value table: distinct values, ``% distinct``
-   (distinct / total rows), total dataset rows, singleton values (frequency 1),
-   entropy with its theoretical maximum and the normalized ratio, mode, imbalance
-   and string-length stats.
-3. A short note flagging problematic cardinality (id-like ≈100% distinct, or a
+A short intro names the clickable **[[term:entropia]]entropía[[/term]]** term —
+the full definition lives in the GLOSARIO chapter, so it is NOT repeated inline
+here (one click jumps to the glossary entry). The intro also carries the dataset
+row total used as a comparison baseline.
+
+Per column the Group contains, in order:
+
+1. A cardinality key/value table: distinct values, ``% distinct`` (distinct /
+   total rows), total dataset rows, singleton values (frequency 1), entropy with
+   its theoretical maximum and the normalized ratio, mode, imbalance and
+   string-length stats.
+2. A short note flagging problematic cardinality (id-like ≈100% distinct, or a
    single dominating category).
-4. A ``top-k`` table (value / count / %).
-5. A **donut pie chart** of the most common categories (top-k + an "Otros"
+3. A ``top-k`` table (value / count / %).
+4. A **donut pie chart** of the most common categories (top-k + an "Otros"
    bucket), drawn lazily so the renderers scale it to fit entirely.
 
 Data comes from the ``eda`` group: each ``columns[i]['categorical']`` is the
@@ -33,7 +39,7 @@ import math
 
 from .. import model
 
-CHAPTER_VERSION = "1.1.0"
+CHAPTER_VERSION = "1.2.0"
 CHAPTER_ID = "cat_distr"
 CHAPTER_TITLE = "Distribuciones categóricas"
 
@@ -53,11 +59,17 @@ _TERM_ENTROPIA_DEF = (
 # Cap the number of categorical columns rendered to keep the document bounded;
 # the rest are summarized in a closing note (no silent truncation).
 MAX_COLS = 40
-# Rows shown in each top-k table and explicit slices in the pie.
-TOP_TABLE_ROWS = 15
+# Rows shown in each top-k table and explicit slices in the pie. Kept moderate so
+# the whole column — cardinality table + top-k table + donut — fits on ONE
+# page/slide with the chart next to its tables; the table note still reports
+# "top N of M" so nothing is silently hidden. For id-like columns (≈100%
+# distinct) the top-k table is dropped entirely (it would be a list of unique
+# values — pure noise), which also frees the room the donut needs (see build).
+TOP_TABLE_ROWS = 8
 PIE_TOP_K = 6
-# Truncate very long category labels in tables (the renderer also wraps).
-LABEL_MAX = 48
+# Truncate very long category labels in tables (the renderer also wraps). Kept
+# tight so a column with long id-like values (names, tickets) still fits its page.
+LABEL_MAX = 28
 
 
 def _fmt_int(value) -> str:
@@ -267,45 +279,55 @@ def _normalize_card(card: dict) -> dict:
 
 
 def _cardinality_block(card: dict):
-    """KVTable with the cardinality / entropy metrics for one column."""
+    """KVTable with the cardinality / entropy metrics for one column.
+
+    Related metrics are grouped onto a single row each (distinct/%/unique;
+    entropy bits/max/normalized; length min/mean/max) so the whole column —
+    table + chart — fits one page/slide without dropping any datum; the short
+    16:9 PPTX slide does not fit one metric per row plus a chart otherwise."""
     n_singletons = card.get("n_singletons")
     if n_singletons is not None and card.get("n_singletons_partial"):
-        singletons = f"≥{_fmt_int(n_singletons)} (en top mostrado)"
+        singletons = f"≥{_fmt_int(n_singletons)}"
     elif n_singletons is not None:
         singletons = _fmt_int(n_singletons)
     else:
         singletons = "—"
 
-    entropy_ref = _fmt_num(card.get("entropy"))
-    emax = card.get("entropy_max")
-    if emax is not None:
-        entropy_ref = f"{entropy_ref} (máx {_fmt_num(emax)})"
+    # Distinct count · % distinct · unique (frequency 1) on one row.
+    distinct_combo = (f"{_fmt_int(card.get('n_distinct'))} · "
+                      f"{_fmt_pct_value(card.get('pct_distinct'))} · "
+                      f"{singletons} únicos")
+
+    # Entropy bits · theoretical max · normalized 0–1 on one row.
+    entropy_combo = (f"{_fmt_num(card.get('entropy'))} bits · "
+                     f"máx {_fmt_num(card.get('entropy_max'))} · "
+                     f"norm {_fmt_num(card.get('entropy_norm'))}")
 
     mode = card.get("mode")
     mode_pct = card.get("mode_pct")
-    mode_str = "—" if mode is None else model._safe_str(mode)
+    mode_str = "—" if mode is None else _truncate(mode, 32)
     if mode is not None and mode_pct is not None:
         mode_str = f"{mode_str} ({_fmt_pct_value(mode_pct)})"
 
     rows = [
-        ("Valores distintos", _fmt_int(card.get("n_distinct"))),
-        ("% distintos", _fmt_pct_value(card.get("pct_distinct"))),
+        ("Distintos · % · únicos", distinct_combo),
         ("Total filas (dataset)", _fmt_int(card.get("n_rows"))),
-        ("Valores únicos (frecuencia 1)", singletons),
-        ("Entropía (bits)", entropy_ref),
-        ("Entropía normalizada (0–1)", _fmt_num(card.get("entropy_norm"))),
+        ("Entropía (bits · máx · norm)", entropy_combo),
         ("Moda", mode_str),
     ]
     imbalance = card.get("imbalance")
-    if imbalance is not None:
-        rows.append(("Desbalance", _fmt_num(imbalance)))
     lm = card.get("len_min")
     lmean = card.get("len_mean")
     lmax = card.get("len_max")
+    # Imbalance and string length (both secondary) share one closing row.
+    extras = []
+    if imbalance is not None:
+        extras.append(f"desbalance {_fmt_num(imbalance)}")
     if any(v is not None for v in (lm, lmean, lmax)):
-        rows.append((
-            "Longitud (mín/media/máx)",
-            f"{_fmt_num(lm)} / {_fmt_num(lmean)} / {_fmt_num(lmax)}"))
+        extras.append(
+            f"long. {_fmt_num(lm)}/{_fmt_num(lmean)}/{_fmt_num(lmax)}")
+    if extras:
+        rows.append(("Desbalance · longitud", " · ".join(extras)))
     return model.KVTable(rows=rows, title="Cardinalidad")
 
 
@@ -315,7 +337,8 @@ def _flag_note(card: dict):
         return model.Note(
             "Casi todos los valores son distintos (≈100% distintos): la columna "
             "se comporta como un identificador y aporta poco para agrupar o "
-            "comparar categorías.")
+            "comparar categorías. No se lista el top de categorías (serían "
+            "valores casi todos únicos).")
     if card.get("dominated"):
         mp = card.get("mode_pct")
         mp_str = _fmt_pct_value(mp) if mp is not None else "muy alta"
@@ -335,7 +358,7 @@ def _topk_table(cat: dict):
         if not isinstance(t, dict):
             continue
         rows.append([
-            model._safe_str(t.get("value")),
+            _truncate(t.get("value")),
             _fmt_int(t.get("count")),
             _pct_from_maybe_fraction(t.get("pct")),
         ])
@@ -353,20 +376,16 @@ def _topk_table(cat: dict):
 def _intro_blocks(n_rows, mark_term: bool = False):
     total = _fmt_int(n_rows)
     # Mark the first appearance of the term as a clickable glossary jump when the
-    # term was registered (mark_term). The visible text is identical either way.
-    entropia = ("[[term:entropia]]**entropía de Shannon**[[/term]]" if mark_term
-                else "**entropía de Shannon**")
+    # term was registered (mark_term). The full definition of entropy lives in the
+    # GLOSARIO chapter, so the intro only names the clickable term here instead of
+    # repeating the long explanation (avoids the redundancy with the glossary).
+    entropia = ("[[term:entropia]]entropía[[/term]]" if mark_term
+                else "entropía")
     text = (
-        f"La {entropia} mide cómo de repartidos están los valores de "
-        "una columna categórica, en bits. Vale 0 cuando una sola categoría "
-        "concentra todas las filas (máxima previsibilidad) y alcanza su máximo, "
-        "log2(k) para k categorías distintas, cuando todas aparecen por igual "
-        "(máxima diversidad). La **entropía normalizada** (entropía dividida por "
-        "su máximo) la lleva al rango 0–1 para comparar columnas con distinto "
-        "número de categorías. Para cada columna se muestran los valores "
-        "distintos, el porcentaje que representan sobre el total de filas, los "
-        "valores únicos (que aparecen una sola vez), la tabla de las categorías "
-        "más frecuentes y un gráfico de tarta (donut) de las más comunes."
+        f"Cada columna categórica ocupa su propia página: sus métricas de "
+        f"cardinalidad —incluida la {entropia}—, una nota que señala cardinalidad "
+        "problemática, la tabla de las categorías más frecuentes y un gráfico de "
+        "tarta (donut) de las más comunes, todo junto."
     )
     if n_rows is not None:
         text += f" El dataset tiene {total} filas en total como referencia."
@@ -398,24 +417,37 @@ def build_cat_distr(profile: dict, ctx: dict):
     blocks = list(_intro_blocks(n_rows, mark_term=mark_term))
 
     rendered = cat_cols[:MAX_COLS]
-    for col in rendered:
+    for idx, col in enumerate(rendered):
         name = col.get("name") or "(columna)"
         cat = col.get("categorical") or {}
         card = _normalize_card(_cardinality(cat, n_rows))
 
-        blocks.append(model.Heading(text=str(name), level=2))
-        blocks.append(_cardinality_block(card))
+        # One Group per categorical column: heading + cardinality table + flag
+        # note + top-k table + donut figure are kept together and the renderer
+        # starts each on a fresh page/slide (page_break_before) so every column
+        # gets its own page with its chart next to its tables. The first column
+        # may share the intro's page (no forced break) to avoid a near-empty page.
+        col_blocks = [
+            model.Heading(text=str(name), level=2),
+            _cardinality_block(card),
+        ]
         note = _flag_note(card)
         if note is not None:
-            blocks.append(note)
-        topk = _topk_table(cat)
-        if topk is not None:
-            blocks.append(topk)
-        blocks.append(model.Figure(
+            col_blocks.append(note)
+        # For id-like columns (≈100% distinct) the top-k is a list of unique
+        # values — pure noise; skip it (the flag note already explains why) and
+        # let the donut take that room so the whole column fits one page/slide.
+        if not card.get("id_like"):
+            topk = _topk_table(cat)
+            if topk is not None:
+                col_blocks.append(topk)
+        col_blocks.append(model.Figure(
             make=_pie_make(cat.get("top") or [], card.get("n_distinct"),
                            str(name), n_rows),
             caption=(f"Categorías más comunes de «{_truncate(name, 32)}» "
                      "(donut: top-k + «Otros»)")))
+        blocks.append(model.Group(blocks=col_blocks,
+                                  page_break_before=(idx > 0)))
 
     if len(cat_cols) > len(rendered):
         omitted = len(cat_cols) - len(rendered)

python/functions/datascience/automatic_eda/chapters/cat_distr_test.py

@@ -2,11 +2,14 @@
 
 Self-contained: builds synthetic TableProfiles (no DuckDB) so the suite is fast
 and deterministic. Verifies that ``build_cat_distr`` emits the blocks the user
-asked for (entropy intro, distinct/total/%-distinct/unique metrics, top-k table
-and a donut figure), that the chapter renders inside the full document to both
-PDF and PPTX showing that content, that a profile with no categorical columns
-yields ``None`` without raising, and that long labels / many columns are never
-cut in either output.
+asked for (distinct/total/%-distinct/unique metrics, top-k table and a donut
+figure), that EACH categorical column is wrapped in its own keep-together
+``Group`` that starts on a fresh page/slide (one column per page, chart next to
+its tables), that the long entropy explanation is NOT repeated inline (it lives
+in the glossary — only the clickable term is kept), that the chapter renders
+inside the full document to both PDF and PPTX showing that content, that a
+profile with no categorical columns yields ``None`` without raising, and that
+long labels / many columns are never cut in either output.
 """
 
 import os
@@ -17,7 +20,8 @@ from pypdf import PdfReader
 from pptx import Presentation
 
 from datascience.automatic_eda.model import (
-    DataTable, Figure, Heading, KVTable, Note,
+    DataTable, Figure, GlossaryCollector, Group, Heading, KVTable, Markdown,
+    Note,
 )
 from datascience.automatic_eda.chapters.cat_distr import (
     CHAPTER_ID, CHAPTER_VERSION, build_cat_distr,
@@ -81,8 +85,20 @@ def _pptx_text(path: str) -> str:
     return re.sub(r"\s+", " ", " ".join(parts))
 
 
-def _kinds(chapter):
-    return [b.kind for b in chapter.blocks]
+def _flatten(blocks):
+    """Expand keep-together Groups so the per-column heading/table/figure are
+    inspectable as a flat block list (the chapter wraps each column in a Group)."""
+    out = []
+    for b in blocks:
+        if getattr(b, "kind", "") == "group":
+            out.extend(_flatten(getattr(b, "blocks", []) or []))
+        else:
+            out.append(b)
+    return out
+
+
+def _column_groups(chapter):
+    return [b for b in chapter.blocks if isinstance(b, Group)]
 
 
 def test_golden_build_cat_distr_emite_bloques_pedidos():
@@ -90,36 +106,101 @@ def test_golden_build_cat_distr_emite_bloques_pedidos():
     assert ch is not None
     assert ch.id == CHAPTER_ID
     assert ch.version == CHAPTER_VERSION
-    kinds = _kinds(ch)
-    # Entropy intro present.
+
+    # Entropy intro present, but the long explanation is gone (it lives in the
+    # glossary now): only the term is named, no log2/normalizada walkthrough.
     headings = [b.text for b in ch.blocks if isinstance(b, Heading)]
     assert any("Entrop" in h for h in headings)
-    md = next(b for b in ch.blocks if b.kind == "markdown")
-    assert "entropía" in md.text.lower() and "log2" in md.text
-    # Cardinality metrics: distinct, total rows, %-distinct, unique values.
-    kv = next(b for b in ch.blocks if isinstance(b, KVTable))
+    md = next(b for b in ch.blocks if isinstance(b, Markdown))
+    assert "entropía" in md.text.lower()
+    assert "log2" not in md.text          # redundant explanation removed.
+    assert "máxima diversidad" not in md.text
+
+    # Per-column blocks are wrapped in keep-together Groups: flatten to inspect.
+    flat = _flatten(ch.blocks)
+    kv = next(b for b in flat if isinstance(b, KVTable))
     labels = [r[0] for r in kv.rows]
-    assert "Valores distintos" in labels
-    assert "% distintos" in labels
+    values = " ".join(str(r[1]) for r in kv.rows)
+    # Cardinality metrics: distinct count, %-distinct, unique values and total
+    # rows are present (grouped onto compact rows so the chart fits the page).
+    assert "Distintos · % · únicos" in labels
     assert "Total filas (dataset)" in labels
-    assert "Valores únicos (frecuencia 1)" in labels
     assert any("Entropía" in lbl for lbl in labels)
+    assert "únicos" in values and "%" in values
+    assert "bits" in values and "norm" in values   # entropy + max + normalized.
     # Top-k table + pie figure.
-    dt = next(b for b in ch.blocks if isinstance(b, DataTable))
+    dt = next(b for b in flat if isinstance(b, DataTable))
     assert dt.header == ["Valor", "Conteo", "%"]
     assert any("neumaticos" in str(cell) for row in dt.rows for cell in row)
-    assert any(isinstance(b, Figure) for b in ch.blocks)
-    # id-like column flagged with a Note.
-    assert any(isinstance(b, Note) and "identificador" in b.text
-               for b in ch.blocks)
+    assert any(isinstance(b, Figure) for b in flat)
+    # id-like column flagged with a Note that also explains the top-k is dropped.
+    idnote = next((b for b in flat
+                   if isinstance(b, Note) and "identificador" in b.text), None)
+    assert idnote is not None
+    assert "No se lista el top" in idnote.text
 
 
-def test_golden_render_pdf_muestra_categoricas():
+def test_golden_idlike_omite_topk_y_conserva_donut():
+    # The id-like column (uuid, 100% distinct) must NOT carry a top-k DataTable
+    # (it would be a list of unique values), but must still keep its donut Figure
+    # and its cardinality table so it stays a full per-column page.
+    ch = build_cat_distr(_profile(), {})
+    groups = _column_groups(ch)
+    uuid_group = next(g for g in groups
+                      if any(getattr(b, "text", "") == "uuid" for b in g.blocks))
+    kinds = [b.kind for b in uuid_group.blocks]
+    assert "data_table" not in kinds      # top-k of unique values dropped.
+    assert "kv_table" in kinds            # cardinality kept.
+    assert "figure" in kinds              # donut kept (chart per column).
+    # A non-id-like column keeps its top-k table.
+    cat_group = next(g for g in groups
+                     if any(getattr(b, "text", "") == "categoria"
+                            for b in g.blocks))
+    assert "data_table" in [b.kind for b in cat_group.blocks]
+
+
+def test_golden_una_pagina_por_columna_groups():
+    ch = build_cat_distr(_profile(), {})
+    groups = _column_groups(ch)
+    # Two categorical columns -> two column Groups (numeric column excluded).
+    assert len(groups) == 2
+    # Each Group carries one column: a heading + its cardinality table + figure.
+    for g in groups:
+        kinds = [b.kind for b in g.blocks]
+        assert kinds[0] == "heading"
+        assert "kv_table" in kinds
+        assert "figure" in kinds
+    # The first column may share the intro page (no forced break); every later
+    # column starts on a fresh page/slide so each column gets its own page.
+    assert groups[0].page_break_before is False
+    assert all(g.page_break_before is True for g in groups[1:])
+
+
+def test_golden_entropia_clicable_y_definicion_en_glosario():
+    # With a glossary collector the intro marks the clickable term and the FULL
+    # definition (the long explanation removed from the intro) lands in the
+    # glossary, not inline — no data lost, just relocated.
+    gc = GlossaryCollector()
+    ch = build_cat_distr(_profile(), {"glossary": gc})
+    md = next(b for b in ch.blocks if isinstance(b, Markdown))
+    assert "[[term:entropia]]entropía[[/term]]" in md.text
+    assert gc.has("entropia")
+    entry = gc.get("entropia")
+    assert entry is not None
+    # The definition kept in the glossary still carries the detail removed inline.
+    assert "log2" in entry["definition"]
+    assert "normalizada" in entry["definition"].lower()
+
+
+def test_golden_render_pdf_una_pagina_por_columna():
     with tempfile.TemporaryDirectory() as d:
         out = os.path.join(d, "eda.pdf")
         res = render_automatic_eda_pdf(_profile(), out, {"title": "EDA"})
         assert res["path"] == out and os.path.exists(out)
-        assert CHAPTER_ID in [c["id"] for c in res["chapters"]]
+        cat_meta = next(c for c in res["chapters"] if c["id"] == CHAPTER_ID)
+        # Two categorical columns, each on its own page -> >= 2 pages for the
+        # chapter (intro shares the first column's page).
+        assert cat_meta["n_pages"] >= 2
         txt = _pdf_text(out)
         assert "Entrop" in txt
         assert "distintos" in txt
@@ -133,13 +214,91 @@ def test_golden_render_pptx_muestra_categoricas():
         out = os.path.join(d, "eda.pptx")
         res = render_automatic_eda_pptx(_profile(), out, {"title": "EDA"})
         assert res["path"] == out and os.path.exists(out)
-        assert CHAPTER_ID in [c["id"] for c in res["chapters"]]
+        cat_meta = next(c for c in res["chapters"] if c["id"] == CHAPTER_ID)
+        assert cat_meta["n_slides"] >= 2  # one slide per categorical column.
         txt = _pptx_text(out)
         assert "Entrop" in txt
         assert "categoria" in txt and "neumaticos" in txt
         assert "distintos" in txt
 
 
+def _profile_high_card() -> dict:
+    """Profile with a high-cardinality NON-id-like categorical column whose top-k
+    of long values would split from its donut on a short 16:9 slide unless the
+    renderer trims the table — the exact case the adversarial check flagged
+    (Ticket / Cabin)."""
+    long_vals = [f"Valor largo de categoria numero {i:02d} con texto extra"
+                 for i in range(40)]
+    top = [{"value": v, "count": 60 - i, "pct": (60 - i) / 5000.0}
+           for i, v in enumerate(long_vals)]
+    return {
+        "table": "t", "source": "t.csv", "n_rows": 5000, "n_cols": 3,
+        "quality_score": 80.0,
+        "columns": [
+            {"name": "precio", "inferred_type": "numeric", "null_pct": 0.0,
+             "numeric": {"mean": 1.0, "median": 1.0, "min": 0.0, "max": 2.0,
+                         "std": 0.5}},
+            # 40 distinct over 5000 rows = 0.8% distinct -> NOT id-like, keeps
+            # its (long) top-k table; the tall table must not push the donut off.
+            {"name": "alta_card_col", "inferred_type": "categorical",
+             "null_pct": 0.0, "distinct_count": 40,
+             "categorical": {"top": top, "mode": long_vals[0], "n_distinct": 40,
+                             "entropy": 5.2, "imbalance": 1.2, "len_min": 40,
+                             "len_mean": 45, "len_max": 50}},
+            {"name": "baja_card_col", "inferred_type": "categorical",
+             "null_pct": 0.0, "distinct_count": 4,
+             "categorical": {
+                 "top": [{"value": "norte", "count": 2000, "pct": 0.4},
+                         {"value": "sur", "count": 1500, "pct": 0.3},
+                         {"value": "este", "count": 1000, "pct": 0.2},
+                         {"value": "oeste", "count": 500, "pct": 0.1}],
+                 "mode": "norte", "n_distinct": 4, "entropy": 1.8}},
+        ],
+    }
+
+
+def test_golden_pptx_una_slide_por_columna_con_su_grafico():
+    """Each categorical column occupies EXACTLY ONE cat_distr slide that carries
+    BOTH its cardinality table and its donut figure (picture) — i.e. the chart is
+    never separated from its table, even for a high-cardinality column."""
+    from pptx.enum.shapes import MSO_SHAPE_TYPE
+
+    prof = _profile_high_card()
+    cat_names = ["alta_card_col", "baja_card_col"]
+    with tempfile.TemporaryDirectory() as d:
+        out = os.path.join(d, "eda.pptx")
+        res = render_automatic_eda_pptx(prof, out, {"title": "EDA"})
+        assert res["path"] == out and os.path.exists(out)
+        prs = Presentation(out)
+
+        # Per column: the cat_distr slides whose text mentions it, and whether the
+        # owning slide also has the donut caption + an actual picture shape.
+        slides_with_col = {n: [] for n in cat_names}
+        owner_has_chart = {n: False for n in cat_names}
+        for i, sl in enumerate(prs.slides):
+            texts, has_pic = [], False
+            for sh in sl.shapes:
+                if sh.has_text_frame:
+                    texts.append(sh.text_frame.text)
+                if sh.shape_type == MSO_SHAPE_TYPE.PICTURE:
+                    has_pic = True
+            txt = re.sub(r"\s+", " ", " ".join(texts))
+            if "Distribuciones categ" not in txt:   # footer stamp of the chapter.
+                continue
+            for n in cat_names:
+                if n in txt:
+                    slides_with_col[n].append(i)
+                    has_table = "Cardinalidad" in txt or "distintos" in txt
+                    if has_pic and "donut" in txt and has_table:
+                        owner_has_chart[n] = True
+
+        for n in cat_names:
+            # Exactly one slide carries the column (not split across slides).
+            assert len(slides_with_col[n]) == 1, (n, slides_with_col[n])
+            # That single slide also holds its table AND its donut picture.
+            assert owner_has_chart[n], (n, "tabla y donut no están en el mismo slide")
+
+
 def test_edge_sin_categoricas_devuelve_none():
     only_numeric = {
         "n_rows": 10, "columns": [
@@ -170,11 +329,15 @@ def test_anti_corte_label_largo_y_muchas_columnas():
 
     ch = build_cat_distr(profile, {})
     assert ch is not None
+    # One Group per column, each forcing its own page (except the first).
+    groups = _column_groups(ch)
+    assert len(groups) == 30
+    assert sum(1 for g in groups if g.page_break_before) == 29
     with tempfile.TemporaryDirectory() as d:
         pdf = os.path.join(d, "anti.pdf")
         res = render_automatic_eda_pdf(profile, pdf, {"write_manifest": False})
         assert res["path"] == pdf
-        assert res["n_pages"] > 1       # many columns spilled across pages, OK.
+        assert res["n_pages"] > 1       # one page per column, OK.
         txt = _pdf_text(pdf)
         # Long label wrapped (not truncated): every word survives.
         for word in ("Lorem", "incididunt", "reprehenderit", "voluptate"):

python/functions/datascience/automatic_eda/model.py

@@ -139,10 +139,17 @@ class Group:
     it starts on a fresh page and flows (honest degradation, never cut). Use it to
     bind ``Heading`` + ``Markdown`` + ``Figure`` of one idea together (see the
     DISTR NUM / AGREGACION chapters).
+
+    When ``page_break_before`` is True the renderer additionally forces the group
+    to *start* on a fresh page/slide (unless the current one is already empty), so
+    a chapter can give each unit its own page — e.g. one categorical column per
+    page (see CAT DISTR). It is purely additive: the default False keeps the plain
+    keep-together behaviour for every existing chapter.
     """
 
     blocks: list = field(default_factory=list)
     title: Optional[str] = None
+    page_break_before: bool = False
     kind: str = field(default="group", init=False)
 
 
@@ -228,7 +235,9 @@ def as_block(obj: Any):
                 return Note(text=_safe_str(obj.get("text")))
             if cls is Group:
                 return Group(blocks=as_blocks(obj.get("blocks")),
-                             title=obj.get("title"))
+                             title=obj.get("title"),
+                             page_break_before=bool(
+                                 obj.get("page_break_before", False)))
             if cls is GlossaryEntry:
                 return GlossaryEntry(key=_safe_str(obj.get("key")),
                                      label=_safe_str(obj.get("label")),

python/functions/datascience/automatic_eda/render_md_impl.py

@@ -0,0 +1,458 @@
+"""AutomaticEDA Markdown serializer — one self-contained file to paste to an LLM.
+
+Same document model as the PDF/PPTX renderers (an ordered list of
+:class:`Chapter`, each a list of format-independent blocks) but emitted as plain
+**Markdown** instead of a binary. The goal is different from the other two
+renderers: a Markdown EDA is meant to be *pasted into an LLM*, so it prioritises
+TEXT and DATA over visuals. Tables become Markdown tables (every row dumped, no
+pagination — nothing is cut because there are no pages); a ``Figure`` becomes its
+caption plus, when possible, the underlying bar/histogram data as a Markdown
+table (an LLM cannot see the image); glossary term markers are stripped while
+``**bold**`` is kept (it is valid Markdown).
+
+dict-no-throw (the ``eda`` group style): :func:`render_md` never raises. On a
+fatal error it returns ``{path: None, ...}`` with a ``note`` explaining why; a
+malformed block degrades to a readable note rather than crashing the document.
+"""
+
+from __future__ import annotations
+
+import os
+import re
+
+from . import model
+
+# Glossary span markers (kept text, dropped markers). We intentionally do NOT use
+# ``text_layout.strip_inline_md`` for Markdown blocks because that also removes
+# ``**bold**`` — valid Markdown we want to preserve when pasting to an LLM.
+_TERM_OPEN_RE = re.compile(r"\[\[term:[A-Za-z0-9_]+\]\]")
+_MAX_BAR_ROWS = 100
+
+
+# --------------------------------------------------------------------------- #
+# Small helpers.
+# --------------------------------------------------------------------------- #
+def _clean_terms(s) -> str:
+    """Drop glossary term markers, keeping the visible text (and any **bold**)."""
+    s = model._safe_str(s)
+    s = _TERM_OPEN_RE.sub("", s)
+    return s.replace("[[/term]]", "")
+
+
+def _cell(v) -> str:
+    """Render a value as a safe Markdown table cell.
+
+    Escapes pipes (``|`` -> ``\\|``) so they do not break the column layout and
+    folds newlines to ``<br>`` so a multi-line value stays inside one cell. None
+    becomes an empty string.
+    """
+    s = model._safe_str(v)
+    s = s.replace("|", "\\|")
+    s = s.replace("\r\n", "\n").replace("\r", "\n").replace("\n", "<br>")
+    return s
+
+
+def _slug(text: str) -> str:
+    """GitHub-style heading anchor: lowercase, spaces->'-', drop other symbols."""
+    s = model._safe_str(text).strip().lower()
+    out = []
+    for ch in s:
+        if ch.isalnum():
+            out.append(ch)
+        elif ch in " -":
+            out.append("-")
+        # any other symbol is dropped.
+    slug = "".join(out)
+    while "--" in slug:
+        slug = slug.replace("--", "-")
+    return slug.strip("-")
+
+
+def _fmt_num(v) -> str:
+    """Compact number for the figure data tables (ints as ints, else 4 sig figs)."""
+    try:
+        f = float(v)
+    except Exception:  # noqa: BLE001
+        return model._safe_str(v)
+    if f != f:  # NaN
+        return "NaN"
+    if f == int(f) and abs(f) < 1e15:
+        return str(int(f))
+    return f"{f:.4g}"
+
+
+def _fmt_int(v) -> str:
+    try:
+        return str(int(v))
+    except Exception:  # noqa: BLE001
+        return model._safe_str(v)
+
+
+def _now_iso() -> str:
+    from datetime import datetime, timezone
+    return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
+
+
+# --------------------------------------------------------------------------- #
+# Document header (title + metadata blockquote + numbered index).
+# --------------------------------------------------------------------------- #
+def _meta_block(meta: dict) -> list:
+    """Build the metadata lines for the header blockquote (omitting absentees)."""
+    ctx = meta.get("ctx") if isinstance(meta.get("ctx"), dict) else {}
+    lines: list = []
+
+    def add(label, value) -> None:
+        if value is None:
+            return
+        s = model._safe_str(value).strip()
+        if s and s.lower() != "none":
+            lines.append(f"**{label}:** {s}")
+
+    add("Dataset", ctx.get("dataset_name") or meta.get("dataset_name"))
+    add("Fuente", ctx.get("source_origin") or meta.get("source_origin"))
+    add("Almacenamiento", ctx.get("storage") or meta.get("storage"))
+    n_rows = ctx.get("n_rows", meta.get("n_rows"))
+    n_cols = ctx.get("n_cols", meta.get("n_cols"))
+    if n_rows is not None and n_cols is not None:
+        lines.append(
+            f"**Dimensiones:** {_fmt_int(n_rows)} filas × {_fmt_int(n_cols)} columnas")
+    add("Generado", meta.get("generated_at") or _now_iso())
+    lines.append(f"**Motor:** {model.ENGINE_NAME} v{model.ENGINE_VERSION}")
+    return lines
+
+
+# --------------------------------------------------------------------------- #
+# Per-block serializers. Each returns a Markdown string (no surrounding blanks;
+# the caller separates blocks with a blank line).
+# --------------------------------------------------------------------------- #
+def _md_heading(block) -> str:
+    level = int(getattr(block, "level", 1) or 1)
+    hashes = "#" * min(level + 2, 6)  # level1 -> ###; '#'/'##' reserved for doc/chapter.
+    text = _clean_terms(getattr(block, "text", "")).strip()
+    return f"{hashes} {text}"
+
+
+def _md_markdown(block) -> str:
+    # Keep the text verbatim, dropping only glossary markers (keep **bold**).
+    return _clean_terms(getattr(block, "text", "")).rstrip("\n")
+
+
+def _md_kv_table(block) -> str:
+    lines: list = []
+    title = getattr(block, "title", None)
+    if title:
+        lines.append(f"**{_clean_terms(title).strip()}**")
+        lines.append("")
+    lines.append("| Campo | Valor |")
+    lines.append("| --- | --- |")
+    for row in (getattr(block, "rows", []) or []):
+        try:
+            label, value = row[0], row[1]
+        except Exception:  # noqa: BLE001
+            label, value = row, ""
+        lines.append(f"| {_cell(label)} | {_cell(value)} |")
+    return "\n".join(lines)
+
+
+def _md_data_table(block) -> str:
+    lines: list = []
+    title = getattr(block, "title", None)
+    if title:
+        lines.append(f"**{_clean_terms(title).strip()}**")
+        lines.append("")
+    header = list(getattr(block, "header", []) or [])
+    rows = list(getattr(block, "rows", []) or [])
+    if not header:
+        ncol = max((len(r) for r in rows), default=1)
+        header = [f"col{i + 1}" for i in range(ncol)]
+    ncol = len(header)
+    lines.append("| " + " | ".join(_cell(h) for h in header) + " |")
+    lines.append("| " + " | ".join(["---"] * ncol) + " |")
+    for r in rows:  # dump every row — no pagination, nothing cut.
+        cells = [_cell(r[c]) if c < len(r) else "" for c in range(ncol)]
+        lines.append("| " + " | ".join(cells) + " |")
+    note = getattr(block, "note", None)
+    if note:
+        lines.append("")
+        lines.append(f"*{_clean_terms(note).strip()}*")
+    return "\n".join(lines)
+
+
+def _bars_table(bars: list) -> str:
+    """Render extracted bar/histogram data as a Markdown table (Desde/Hasta/Frec)."""
+    lines = ["| Desde | Hasta | Frecuencia |", "| --- | --- | --- |"]
+    shown = bars[:_MAX_BAR_ROWS]
+    for x0, x1, h in shown:
+        lines.append(f"| {_fmt_num(x0)} | {_fmt_num(x1)} | {_fmt_num(h)} |")
+    out = "\n".join(lines)
+    extra = len(bars) - len(shown)
+    if extra > 0:
+        out += f"\n\n*… ({extra} filas más)*"
+    return out
+
+
+def _extract_bars(fig) -> list:
+    """Collect (x_from, x_to, height) of the rectangular bars of a matplotlib fig.
+
+    Histogram / bar-chart bars are ``matplotlib.patches.Rectangle`` with positive
+    width and height; spines, legends and zero-area artists are skipped. Never
+    raises — returns ``[]`` on any problem.
+    """
+    bars: list = []
+    try:
+        for ax in fig.get_axes():
+            # Collect this axes' positive-area rectangles, then keep only the ones
+            # that look like actual histogram/bar bins. Reference shapes that
+            # matplotlib also stores in ``ax.patches`` — most notably the ``±1σ``
+            # band drawn by ``axvspan`` (a single rectangle far wider than a bin)
+            # and a lone Tukey boxplot box — would otherwise show up as fake
+            # "bins". A histogram axes has several near-equal-width bars, so we
+            # drop any rectangle whose width is more than twice the median width
+            # of that axes' rectangles (the σ-band spans many bins; uniform bins
+            # all sit at the median width and stay).
+            ax_bars: list = []
+            for patch in list(getattr(ax, "patches", []) or []):
+                try:
+                    w = patch.get_width()
+                    h = patch.get_height()
+                    x = patch.get_x()
+                except Exception:  # noqa: BLE001 — not a Rectangle-like patch.
+                    continue
+                if w and w > 0 and h and h > 0:
+                    ax_bars.append((x, x + w, h))
+            if len(ax_bars) >= 3:
+                widths = sorted(b[1] - b[0] for b in ax_bars)
+                median_w = widths[len(widths) // 2]
+                if median_w > 0:
+                    ax_bars = [b for b in ax_bars
+                               if (b[1] - b[0]) <= 2.0 * median_w]
+            bars.extend(ax_bars)
+    except Exception:  # noqa: BLE001
+        return []
+    return bars
+
+
+def _md_figure(block, meta: dict, out_path: str, counter: list) -> str:
+    """Serialize a Figure prioritising TEXT + DATA (an LLM cannot see the image).
+
+    Emits the caption, then — if the matplotlib figure has bars — a Markdown table
+    of the underlying (Desde, Hasta, Frecuencia) values. Optionally (when
+    ``meta['embed_figures']`` is True) also exports a PNG beside the .md and adds
+    an image link; off by default so the Markdown stays self-contained.
+    """
+    caption = model._safe_str(getattr(block, "caption", "")).strip()
+    parts = [f"*Figura: {caption}*" if caption else "*Figura*"]
+    fig = None
+    try:
+        import matplotlib
+        matplotlib.use("Agg")  # defensive: headless rasterization backend.
+        fig = getattr(block, "fig", None)
+        make = getattr(block, "make", None)
+        if fig is None and callable(make):
+            fig = make()
+        if fig is not None:
+            bars = _extract_bars(fig)
+            if bars:
+                parts.append(_bars_table(bars))
+            if meta.get("embed_figures"):
+                png = _embed_png(fig, out_path, counter)
+                if png:
+                    parts.append(f"![{caption}]({png})")
+    except Exception:  # noqa: BLE001 — a bad figure degrades to just its caption.
+        pass
+    finally:
+        if fig is not None:
+            try:
+                import matplotlib.pyplot as plt
+                plt.close(fig)
+            except Exception:  # noqa: BLE001
+                pass
+    return "\n\n".join(parts)
+
+
+def _embed_png(fig, out_path: str, counter: list) -> str:
+    """Export the figure to ``<basename>_figN.png`` beside the .md; return its name."""
+    try:
+        counter[0] += 1
+        base = os.path.splitext(os.path.basename(out_path))[0] or "figura"
+        name = f"{base}_fig{counter[0]}.png"
+        path = os.path.join(os.path.dirname(os.path.abspath(out_path)), name)
+        fig.savefig(path, format="png", dpi=120, bbox_inches="tight")
+        return name
+    except Exception:  # noqa: BLE001
+        return ""
+
+
+def _md_image(block) -> str:
+    path = model._safe_str(getattr(block, "path", ""))
+    caption = model._safe_str(getattr(block, "caption", "")).strip()
+    out = f"![{caption}]({path})"
+    if caption:
+        out += f"\n\n*{caption}*"
+    return out
+
+
+def _md_caption(block) -> str:
+    return f"*{_clean_terms(getattr(block, 'text', '')).strip()}*"
+
+
+def _md_note(block) -> str:
+    text = _clean_terms(getattr(block, "text", "")).strip()
+    lines = text.split("\n")
+    return "\n".join((f"> {ln}" if ln.strip() else ">") for ln in lines)
+
+
+def _md_group(block, meta: dict, out_path: str, counter: list) -> str:
+    parts: list = []
+    title = getattr(block, "title", None)
+    if title:
+        parts.append(f"### {_clean_terms(title).strip()}")
+    for b in (getattr(block, "blocks", []) or []):
+        try:
+            seg = _serialize_block(b, meta, out_path, counter)
+        except Exception:  # noqa: BLE001
+            seg = ""
+        if seg:
+            parts.append(seg)
+    return "\n\n".join(parts)
+
+
+def _md_glossary_entry(block) -> str:
+    label = (model._safe_str(getattr(block, "label", "")).strip()
+             or model._safe_str(getattr(block, "key", "")).strip())
+    definition = _clean_terms(getattr(block, "definition", "")).strip()
+    out = f"### {label}"
+    if definition:
+        out += f"\n\n{definition}"
+    return out
+
+
+def _serialize_block(block, meta: dict, out_path: str, counter: list) -> str:
+    """Dispatch a single block to its Markdown serializer. Unknown -> note."""
+    kind = getattr(block, "kind", "")
+    if kind == "heading":
+        return _md_heading(block)
+    if kind == "markdown":
+        return _md_markdown(block)
+    if kind == "kv_table":
+        return _md_kv_table(block)
+    if kind == "data_table":
+        return _md_data_table(block)
+    if kind == "figure":
+        return _md_figure(block, meta, out_path, counter)
+    if kind == "image":
+        return _md_image(block)
+    if kind == "caption":
+        return _md_caption(block)
+    if kind == "note":
+        return _md_note(block)
+    if kind == "group":
+        return _md_group(block, meta, out_path, counter)
+    if kind == "glossary_entry":
+        return _md_glossary_entry(block)
+    # Unknown content -> readable note (mirrors the model's defensive coercion).
+    return _md_note(model.Note(text=model._safe_str(block)))
+
+
+# --------------------------------------------------------------------------- #
+# Entry point.
+# --------------------------------------------------------------------------- #
+def render_md(chapters: list, out_path: str, meta: dict = None) -> dict:
+    """Serialize a list of Chapters into a single self-contained Markdown file.
+
+    The output leads with ``# <title>``, a metadata blockquote and a numbered
+    ``## Índice`` linking each chapter, then one ``## N. <title>`` section per
+    chapter with its blocks. Tables become Markdown tables (every row dumped),
+    figures become caption + underlying data table, glossary markers are stripped
+    while ``**bold**`` is kept. Designed to be pasted into an LLM.
+
+    Args:
+        chapters: a list of ``Chapter`` (dataclasses or dicts); normalized
+            defensively with ``model.as_chapters``.
+        out_path: filesystem path for the ``.md`` (parent dirs are created).
+        meta: optional dict. Recognised keys: ``title``, ``ctx`` (dict with
+            ``dataset_name``/``source_origin``/``storage``/``n_rows``/``n_cols``),
+            ``generated_at``, ``embed_figures`` (export PNGs beside the .md,
+            default False).
+
+    Returns:
+        dict (never raises): ``{path: str|None, n_chars: int,
+        chapters: list[{id, version}], note: str}``. On a fatal error ``path`` is
+        None and ``note`` explains why.
+    """
+    meta = meta or {}
+    chapters = model.as_chapters(chapters)
+    title = model._safe_str(meta.get("title")) or model.ENGINE_NAME
+
+    # Edge: nothing to render -> a minimal but valid Markdown document.
+    if not chapters:
+        content = (f"# {title}\n\n"
+                   "*(documento vacío — sin capítulos aplicables)*\n")
+        return _write(out_path, content, [], "documento vacío")
+
+    counter = [0]  # document-wide figure counter for unique PNG names.
+    notes: list = []
+    segments: list = [f"# {title}"]
+
+    meta_lines = _meta_block(meta)
+    if meta_lines:
+        segments.append("\n".join(f"> {ln}" for ln in meta_lines))
+
+    # Numbered index. The anchor matches the chapter heading emitted below
+    # (``## N. <title>``) in GitHub slug style.
+    chap_heads = []
+    idx_lines = ["## Índice"]
+    for i, ch in enumerate(chapters, 1):
+        head_text = f"{i}. {model._safe_str(ch.title)}"
+        anchor = _slug(head_text)
+        chap_heads.append((head_text, anchor))
+        idx_lines.append(f"{i}. [{model._safe_str(ch.title)}](#{anchor})")
+    segments.append("\n".join(idx_lines))
+
+    chapters_meta = []
+    for i, ch in enumerate(chapters, 1):
+        segments.append("---")
+        head_text, _anchor = chap_heads[i - 1]
+        segments.append(f"## {head_text}")
+
+        blocks = list(ch.blocks or [])
+        # Omit a leading level-1 Heading that just repeats the chapter title.
+        if blocks:
+            b0 = blocks[0]
+            if (getattr(b0, "kind", "") == "heading"
+                    and int(getattr(b0, "level", 1) or 1) == 1
+                    and _clean_terms(getattr(b0, "text", "")).strip()
+                    == model._safe_str(ch.title).strip()):
+                blocks = blocks[1:]
+
+        for block in blocks:
+            try:
+                seg = _serialize_block(block, meta, out_path, counter)
+            except Exception as e:  # noqa: BLE001
+                seg = _md_note(model.Note(text=model._safe_str(block)))
+                notes.append(
+                    f"bloque '{getattr(block, 'kind', '?')}' del capítulo "
+                    f"'{ch.id}' degradado: {e}")
+            if seg:
+                segments.append(seg)
+        chapters_meta.append({"id": ch.id, "version": ch.version})
+
+    content = "\n\n".join(segments) + "\n"
+    note = f"{len(content)} caracteres"
+    if notes:
+        note += " · " + "; ".join(notes)
+    return _write(out_path, content, chapters_meta, note)
+
+
+def _write(out_path: str, content: str, chapters_meta: list, note: str) -> dict:
+    """Write the Markdown to disk (creating parents). dict-no-throw."""
+    try:
+        parent = os.path.dirname(os.path.abspath(out_path))
+        os.makedirs(parent, exist_ok=True)
+        with open(out_path, "w", encoding="utf-8") as fh:
+            fh.write(content)
+    except Exception as e:  # noqa: BLE001 — never raise from the writer.
+        return {"path": None, "n_chars": 0, "chapters": [],
+                "note": f"no se pudo escribir el Markdown: {e}"}
+    return {"path": out_path, "n_chars": len(content),
+            "chapters": chapters_meta, "note": note}

python/functions/datascience/automatic_eda/render_pdf_impl.py

@@ -675,6 +675,61 @@ def _measure_figure_like(block) -> float:
     return target_h + 0.04 + cap_h + _GAP
 
 
+def _measure_kv_table(block) -> float:
+    """Faithful height of a KVTable — matches ``_place_kv_table``.
+
+    Counts the optional title heading and, per row, the wrapped VALUE column
+    (the label column never wraps in the placer). The previous estimate assumed
+    one line per row and ignored the title, so a column's keep-together Group
+    under-budgeted the figure and the chart spilled to the next page. Keep this in
+    sync with ``_place_kv_table``."""
+    h = 0.0
+    title = getattr(block, "title", None)
+    if title:
+        h += _measure_heading_text(title, 2)
+    rows = getattr(block, "rows", []) or []
+    key_w = 1.9
+    val_chars = tl.chars_per_line(_USABLE_W - key_w - 0.1, _FS_BODY)
+    lh = tl.line_height_in(_FS_BODY)
+    for row in rows:
+        try:
+            value = row[1]
+        except Exception:  # noqa: BLE001
+            value = ""
+        v_lines = tl.wrap(model._safe_str(value), val_chars)
+        h += lh * len(v_lines) + _ROW_VPAD
+    return h + _GAP
+
+
+def _measure_data_table(block) -> float:
+    """Faithful height of a DataTable — matches ``_place_data_table``.
+
+    Counts the optional title heading, the wrapped header row, every wrapped data
+    row (per-column wrap via the same ``_col_widths``/``_wrap_row`` the placer
+    uses) and the optional note. Keep this in sync with ``_place_data_table``."""
+    h = 0.0
+    title = getattr(block, "title", None)
+    if title:
+        h += _measure_heading_text(title, 2)
+    header = list(getattr(block, "header", []) or [])
+    rows = list(getattr(block, "rows", []) or [])
+    fs = _FS_CELL
+    widths = _col_widths(header, rows, fs)
+    lh = tl.line_height_in(fs)
+    if header:
+        header_lines = _wrap_row(header, widths, fs)
+        h += lh * max((len(c) for c in header_lines), default=1) + _ROW_VPAD * 2
+    for r in rows:
+        cells_lines = _wrap_row(r, widths, fs)
+        h += lh * max((len(c) for c in cells_lines), default=1) + _ROW_VPAD * 2
+    note = getattr(block, "note", None)
+    if note:
+        nlines = tl.wrap(model._safe_str(note),
+                         tl.chars_per_line(_USABLE_W, _FS_NOTE))
+        h += tl.line_height_in(_FS_NOTE) * len(nlines)
+    return h + _GAP
+
+
 def _measure_block(st: _PdfState, block) -> float:
     kind = getattr(block, "kind", "")
     try:
@@ -690,13 +745,9 @@ def _measure_block(st: _PdfState, block) -> float:
                             tl.chars_per_line(_USABLE_W, _FS_NOTE))
             return tl.line_height_in(_FS_NOTE) * len(lines) + _GAP
         if kind == "kv_table":
-            rows = getattr(block, "rows", []) or []
-            return (tl.line_height_in(_FS_BODY) + _ROW_VPAD) * (len(rows) + 1) \
-                + _GAP
+            return _measure_kv_table(block)
         if kind == "data_table":
-            rows = getattr(block, "rows", []) or []
-            return (tl.line_height_in(_FS_CELL) + _ROW_VPAD * 2) \
-                * (len(rows) + 1) + _GAP
+            return _measure_data_table(block)
         if kind == "group":
             return sum(_measure_block(st, b)
                        for b in (getattr(block, "blocks", []) or []))
@@ -735,6 +786,10 @@ def _place_group(st: _PdfState, block) -> None:
     blocks = getattr(block, "blocks", []) or []
     if not blocks:
         return
+    # Opt-in page break: start this group on a fresh page unless the current one
+    # is still empty (so a chapter can give each unit its own page).
+    if getattr(block, "page_break_before", False) and st.y > _CONTENT_TOP + 1e-6:
+        _new_page(st)
     avail_full = _CONTENT_BOTTOM - _CONTENT_TOP
     _shrink_group_figures(st, blocks, avail_full)
     total = sum(_measure_block(st, b) for b in blocks)

python/functions/datascience/automatic_eda/render_pptx_impl.py

@@ -625,6 +625,55 @@ def _measure_figure_like(block) -> float:
     return target_h + 0.05 + cap_h + _GAP
 
 
+def _measure_kv_table(block) -> float:
+    """Faithful KVTable height — matches ``_place_kv_table`` (rendered as a
+    Campo/Valor data table with wrapped cells). The previous estimate assumed one
+    line per row and ignored the title, so a keep-together Group under-budgeted
+    the figure and the chart spilled to the next slide. Keep in sync."""
+    h = 0.0
+    title = getattr(block, "title", None)
+    if title:
+        h += _measure_heading_text(title, 2)
+    rows = getattr(block, "rows", []) or []
+    data_rows = []
+    for row in rows:
+        try:
+            label, value = row[0], row[1]
+        except Exception:  # noqa: BLE001
+            label, value = str(row), ""
+        data_rows.append([model._safe_str(label), model._safe_str(value)])
+    header = ["Campo", "Valor"]
+    widths = _col_widths(header, data_rows)
+    fs = _FS_CELL
+    h += _row_height_in(header, widths, fs)
+    for r in data_rows:
+        h += _row_height_in(r, widths, fs)
+    return h + _GAP
+
+
+def _measure_data_table(block) -> float:
+    """Faithful DataTable height — matches ``_place_data_table`` (title heading +
+    wrapped header + every wrapped row + optional note). Keep in sync."""
+    h = 0.0
+    title = getattr(block, "title", None)
+    if title:
+        h += _measure_heading_text(title, 2)
+    header = list(getattr(block, "header", []) or [])
+    rows = list(getattr(block, "rows", []) or [])
+    fs = _FS_CELL
+    widths = _col_widths(header, rows)
+    if header:
+        h += _row_height_in(header, widths, fs)
+    for r in rows:
+        h += _row_height_in(r, widths, fs)
+    note = getattr(block, "note", None)
+    if note:
+        nlines = tl.wrap(model._safe_str(note),
+                         tl.chars_per_line(_USABLE_W, _FS_NOTE))
+        h += tl.line_height_in(_FS_NOTE) * len(nlines) + 0.05
+    return h + _GAP
+
+
 def _measure_block(st: _PptxState, block) -> float:
     kind = getattr(block, "kind", "")
     try:
@@ -639,9 +688,10 @@ def _measure_block(st: _PptxState, block) -> float:
             lines = tl.wrap(getattr(block, "text", ""),
                             tl.chars_per_line(_USABLE_W, _FS_NOTE))
             return tl.line_height_in(_FS_NOTE) * len(lines) + 0.05 + _GAP
-        if kind in ("kv_table", "data_table"):
-            rows = getattr(block, "rows", []) or []
-            return (tl.line_height_in(_FS_CELL) + 0.10) * (len(rows) + 1) + _GAP
+        if kind == "kv_table":
+            return _measure_kv_table(block)
+        if kind == "data_table":
+            return _measure_data_table(block)
         if kind == "group":
             return sum(_measure_block(st, b)
                        for b in (getattr(block, "blocks", []) or []))
@@ -664,10 +714,14 @@ def _shrink_group_figures(st: _PptxState, blocks: list, avail_full: float) -> No
                    if getattr(b, "kind", "") not in ("figure", "image"))
     fig_overhead = tl.line_height_in(_FS_NOTE) + 0.05 + 0.05 + _GAP
     budget = avail_full - nonfig_h - 0.10 * len(fig_blocks)
-    if budget <= 1.0:
+    # Low thresholds: a 16:9 slide is short, so a content-heavy column (cardinality
+    # table + top-k + chart) only fits if the chart is allowed to shrink small.
+    # Prefer a small-but-present chart on the SAME slide over splitting the column
+    # across slides (matches the PDF renderer's keep-together philosophy).
+    if budget <= 0.6:
         return  # not enough room to keep together; let it flow (degrade).
     per = budget / len(fig_blocks) - fig_overhead
-    if per <= 0.8:
+    if per <= 0.35:
         return
     for fb in fig_blocks:
         cur = getattr(fb, "height_in", None)
@@ -675,12 +729,90 @@ def _shrink_group_figures(st: _PptxState, blocks: list, avail_full: float) -> No
                         if isinstance(cur, (int, float)) and cur > 0 else per)
 
 
+# Minimum height (inches) reserved for a figure inside a keep-together group on
+# the short 16:9 slide. When a high-cardinality column's table(s) would otherwise
+# leave no room, the data table is trimmed (with an honest note) so the chart
+# stays on the SAME slide next to its table instead of spilling to the next one.
+_GROUP_MIN_FIG_H = 1.3
+
+
+def _trim_data_table_to_budget(block, budget: float):
+    """Return a copy of a DataTable whose rows fit within ``budget`` inches.
+
+    Keeps the title, header, as many leading rows as fit (at least one) and an
+    honest note reporting how many of the original rows are shown. NEVER mutates
+    the original block — the same Chapter blocks are rendered by the PDF renderer,
+    which keeps the full table (an A5 page fits it)."""
+    header = list(getattr(block, "header", []) or [])
+    rows = list(getattr(block, "rows", []) or [])
+    title = getattr(block, "title", None)
+    fs = _FS_CELL
+    widths = _col_widths(header, rows)
+    fixed = 0.0
+    if title:
+        fixed += _measure_heading_text(title, 2)
+    if header:
+        fixed += _row_height_in(header, widths, fs)
+    note_h = tl.line_height_in(_FS_NOTE) + 0.05
+    avail_rows = budget - fixed - note_h - _GAP
+    kept = []
+    used = 0.0
+    for r in rows:
+        rh = _row_height_in(r, widths, fs)
+        if used + rh > avail_rows and kept:
+            break
+        kept.append(r)
+        used += rh
+    if len(kept) >= len(rows):
+        return block  # already fits; keep the original (with its own note).
+    note = (f"top {len(kept)} de {len(rows)} categorías mostradas "
+            "(recortado para caber en el slide; el PDF muestra más)")
+    return model.DataTable(header=header, rows=kept, title=title, note=note)
+
+
+def _fit_group_blocks(st: _PptxState, blocks: list, avail_full: float) -> list:
+    """Return a slide-fitting copy of a keep-together group's blocks.
+
+    On the short 16:9 slide a high-cardinality column's top-k table plus its
+    chart can overflow. Reserve ``_GROUP_MIN_FIG_H`` for the (later shrunk) figure
+    and trim the data table(s) to what is left, so every column keeps its chart
+    next to its table on ONE slide. No-op when the group has no figure+table pair
+    (e.g. id-like columns already drop the top-k upstream, or it already fits)."""
+    has_fig = any(getattr(b, "kind", "") in ("figure", "image") for b in blocks)
+    tbls = [b for b in blocks if getattr(b, "kind", "") == "data_table"]
+    if not (has_fig and tbls):
+        return blocks
+    fixed_h = sum(_measure_block(st, b) for b in blocks
+                  if getattr(b, "kind", "") not in ("figure", "image",
+                                                    "data_table"))
+    tables_h = sum(_measure_block(st, b) for b in tbls)
+    budget_tables = avail_full - fixed_h - _GROUP_MIN_FIG_H
+    if tables_h <= budget_tables:
+        return blocks  # already fits next to a min-height figure; leave intact.
+    out = []
+    for b in blocks:
+        if getattr(b, "kind", "") != "data_table":
+            out.append(b)
+            continue
+        trimmed = _trim_data_table_to_budget(b, max(budget_tables, 0.8))
+        out.append(trimmed)
+        budget_tables -= _measure_data_table(trimmed)
+    return out
+
+
 def _place_group(st: _PptxState, block) -> None:
     """Render a keep-together Group: move it whole to the next slide if needed."""
     blocks = getattr(block, "blocks", []) or []
     if not blocks:
         return
+    # Opt-in slide break: start this group on a fresh slide unless the current one
+    # is still empty (so a chapter can give each unit its own slide).
+    if getattr(block, "page_break_before", False) and st.y > _CONTENT_TOP + 1e-6:
+        _new_slide(st, cont=True)
     avail_full = _CONTENT_BOTTOM - _CONTENT_TOP
+    # Trim oversized tables first (keeps the chart on the same slide), then shrink
+    # the figure to share the remaining room.
+    blocks = _fit_group_blocks(st, blocks, avail_full)
     _shrink_group_figures(st, blocks, avail_full)
     total = sum(_measure_block(st, b) for b in blocks)
     if total <= avail_full:

python/functions/datascience/confidence_interval_mean.md

@@ -0,0 +1,87 @@
+---
+name: confidence_interval_mean
+kind: function
+lang: py
+domain: datascience
+version: "1.0.0"
+purity: pure
+signature: "def confidence_interval_mean(data: list, other: list = None, confidence: float = 0.95) -> dict"
+description: "Intervalo de confianza (IC) de la media de una muestra con la t de Student, o de la DIFERENCIA de medias de dos muestras independientes con el metodo de Welch (sin asumir varianzas iguales). Una muestra: df=n-1, se=sd_muestral/sqrt(n) (sd con ddof=1), tcrit=t.ppf((1+confidence)/2, df), ci=mean+/-tcrit*se. Dos muestras: IC de mean(data)-mean(other) con se=sqrt(se1^2+se2^2) y grados de libertad de Welch-Satterthwaite. Pura y robusta: nunca lanza; ante casos degenerados (muestra vacia, n<2) devuelve nan + clave note, y con varianza cero el IC colapsa al punto (no es error). Usa scipy.stats y numpy."
+tags: [papers, statistics, confidence-interval, welch, t-test, python]
+params:
+  - name: data
+    desc: "muestra de observaciones numericas (lista de numeros). Si other es None, el IC es el de la media de data."
+  - name: other
+    desc: "segunda muestra independiente (lista de numeros) o None (default). Si se da, el IC es el de la diferencia de medias mean(data)-mean(other) calculada con Welch (no asume varianzas iguales)."
+  - name: confidence
+    desc: "nivel de confianza en (0, 1); 0.95 = IC del 95% (default). El cuantil critico es t.ppf((1+confidence)/2, df)."
+output: "dict {mean, ci_low, ci_high, se, df, confidence, n}. mean = media de data (una muestra) o la diferencia mean(data)-mean(other) (dos muestras). En el caso de dos muestras se anaden ademas n1 y n2 (y n = n1+n2). df son los grados de libertad de la t (Welch-Satterthwaite si dos muestras). Casos degenerados (muestra vacia, n<2) anaden la clave note y dejan ci_low/ci_high/se (y a veces df) en nan; con varianza cero y n>=2 el IC colapsa a [mean, mean] con se=0 (con note, sin nan). Nunca None ni excepcion."
+uses_functions: []
+uses_types: []
+returns: []
+returns_optional: false
+error_type: ""
+imports: [scipy, numpy]
+tested: true
+tests: ["test_one_sample_golden_contra_scipy", "test_one_sample_distinto_nivel_confianza", "test_welch_diferencia_golden_contra_scipy", "test_edge_un_solo_elemento_no_lanza_nan_note", "test_edge_lista_vacia_no_lanza_note", "test_edge_varianza_cero_colapsa_al_punto", "test_edge_welch_muestra_vacia_no_lanza_note", "test_edge_welch_n1_uno_no_lanza_note"]
+test_file_path: "python/functions/datascience/confidence_interval_mean_test.py"
+file_path: "python/functions/datascience/confidence_interval_mean.py"
+---
+
+## Ejemplo
+
+```python
+from datascience import confidence_interval_mean
+
+# IC del 95% de la media de una muestra (t de Student).
+data = [2, 4, 4, 4, 5, 5, 7, 9]
+ci = confidence_interval_mean(data, confidence=0.95)
+print(ci["mean"])     # -> 5.0
+print(ci["df"])       # -> 7.0  (n - 1)
+print(round(ci["ci_low"], 5), round(ci["ci_high"], 5))
+# -> 3.21251 6.78749   (se con sd muestral ddof=1 ~ 2.13809)
+
+# IC del 95% de la DIFERENCIA de medias (Welch, no asume varianzas iguales).
+control = [23.0, 21.0, 25.0, 22.0, 24.0, 26.0]
+tratado = [18.0, 20.0, 17.0, 19.0, 21.0]
+diff = confidence_interval_mean(control, tratado, confidence=0.95)
+print(diff["mean"])   # -> 4.5  (mean(control) - mean(tratado))
+print(round(diff["ci_low"], 4), round(diff["ci_high"], 4))
+# Si el intervalo no incluye 0, la diferencia es significativa al 5%.
+
+# Degenerados: nunca lanza.
+print(confidence_interval_mean([5])["note"])      # n < 2: ... indefinidos
+print(confidence_interval_mean([3, 3, 3])["se"])  # -> 0.0  (IC colapsa a [3, 3])
+```
+
+## Cuando usarla
+
+Cuando quieras cuantificar la **incertidumbre de una media estimada** a partir de
+una muestra: reporta `[ci_low, ci_high]` en vez de un punto suelto para mostrar
+el rango plausible del valor real al nivel de confianza pedido. Usala tambien
+para **comparar dos grupos** (A/B test, control vs tratamiento, antes vs
+despues con grupos independientes): pasa las dos muestras y, si el IC de la
+diferencia **no incluye el 0**, la diferencia es significativa al nivel
+`1 - confidence`. Es el complemento del p-valor: ademas de "hay efecto", te dice
+"de que tamano y con que margen". Para dos muestras usa Welch por defecto, asi
+que no necesitas comprobar antes si las varianzas son iguales.
+
+## Gotchas
+
+- Pura y determinista (no hace I/O, no muta las entradas), pero **no** es
+  stdlib-only: depende de `scipy.stats` y `numpy` (ambos en el venv del proyecto).
+- Con `other` usa **Welch** (df de Welch-Satterthwaite): NO asume varianzas
+  iguales ni tamanos de muestra iguales. Si necesitas el t-test clasico de
+  varianzas agrupadas (pooled), esta funcion no lo hace.
+- `sd` se calcula con **ddof=1** (sd muestral), que es lo correcto para el IC de
+  una media con la t. Atajos como `sd_poblacional/sqrt(n)` (ddof=0) dan un
+  intervalo demasiado estrecho.
+- En el caso de dos muestras, `mean` es la **diferencia** `mean(data) - mean(other)`
+  (no la media de data). El orden importa: el signo del IC depende de cual va
+  primero.
+- Nunca lanza. Casos degenerados devuelven `nan` en `ci_low`/`ci_high`/`se`
+  (y a veces `df`) mas una clave `note`: muestra vacia o `n < 2` en cualquiera de
+  las muestras. **Excepcion**: con varianza cero y `n >= 2` el IC colapsa al
+  punto `[mean, mean]` con `se = 0` (no es un error, no hay `nan`).
+- Comprueba `"note" in out` antes de usar `ci_low`/`ci_high` si la muestra puede
+  ser degenerada.

python/functions/datascience/confidence_interval_mean.py

@@ -0,0 +1,176 @@
+"""Intervalo de confianza de la media (una muestra) o de la diferencia de medias (Welch).
+
+Funcion pura del grupo papers. Calcula el intervalo de confianza (IC) de la media
+de una muestra usando la t de Student, o el IC de la diferencia de medias de dos
+muestras independientes con el metodo de Welch (sin asumir varianzas iguales).
+
+- Una muestra: ``df = n - 1``, ``se = sd / sqrt(n)`` (sd con ddof=1),
+  ``tcrit = t.ppf((1 + confidence) / 2, df)``, ``ci = mean +/- tcrit * se``.
+- Dos muestras (Welch): IC de ``mean(data) - mean(other)``, con
+  ``se = sqrt(se1^2 + se2^2)`` y grados de libertad de Welch-Satterthwaite.
+
+No lanza excepciones: ante casos degenerados (muestras vacias, ``n < 2``,
+varianza cero) devuelve un dict coherente con ``ci_low``/``ci_high``/``se`` en
+``nan`` (salvo el sub-caso de varianza cero, donde el IC colapsa al punto) y una
+clave ``note`` explicando el caso. Usa ``scipy.stats`` y ``numpy``.
+"""
+
+from __future__ import annotations
+
+import math
+
+import numpy as np
+from scipy import stats
+
+
+def confidence_interval_mean(
+    data: list, other: list = None, confidence: float = 0.95
+) -> dict:
+    """Intervalo de confianza de la media o de la diferencia de medias (Welch).
+
+    Si ``other`` es ``None``, calcula el IC de la media de ``data`` con la t de
+    Student. Si se proporciona ``other``, calcula el IC de la diferencia
+    ``mean(data) - mean(other)`` con el metodo de Welch (no asume varianzas
+    iguales) y grados de libertad de Welch-Satterthwaite.
+
+    Es una funcion pura y determinista: no hace I/O ni muta las entradas. No
+    lanza excepcion ante datos degenerados; en su lugar devuelve un dict con la
+    clave ``note`` y los campos numericos indefinidos a ``nan``.
+
+    Args:
+        data: muestra de observaciones numericas (lista de numeros).
+        other: segunda muestra independiente. Si se da, el IC es el de la
+            diferencia de medias ``mean(data) - mean(other)`` con Welch. Si es
+            ``None`` (default), el IC es el de la media de ``data``.
+        confidence: nivel de confianza en (0, 1), p.ej. 0.95 para el 95%.
+
+    Returns:
+        dict con las claves:
+            mean: media de ``data`` (una muestra) o la diferencia
+                ``mean(data) - mean(other)`` (dos muestras).
+            ci_low: extremo inferior del intervalo de confianza.
+            ci_high: extremo superior del intervalo de confianza.
+            se: error estandar de la media (o de la diferencia).
+            df: grados de libertad de la t (Welch-Satterthwaite si dos muestras).
+            confidence: nivel de confianza aplicado (float).
+            n: tamano de la muestra (una muestra) o tamano total ``n1 + n2``
+                (dos muestras; ademas se incluyen ``n1`` y ``n2``).
+
+        En el caso de dos muestras se incluyen ademas ``n1`` y ``n2``. Casos
+        degenerados (muestra vacia, ``n < 2``, etc.) anaden la clave ``note`` y
+        dejan ``ci_low``/``ci_high``/``se`` (y a veces ``df``) en ``nan``.
+    """
+    conf = float(confidence)
+
+    if other is None:
+        return _ci_one_sample(data, conf)
+    return _ci_welch(data, other, conf)
+
+
+def _ci_one_sample(data: list, conf: float) -> dict:
+    """IC de la media de una sola muestra con la t de Student."""
+    arr = np.asarray(list(data), dtype=float)
+    n = int(arr.size)
+
+    base = {
+        "mean": float("nan"),
+        "ci_low": float("nan"),
+        "ci_high": float("nan"),
+        "se": float("nan"),
+        "df": float("nan"),
+        "confidence": conf,
+        "n": n,
+    }
+
+    if n == 0:
+        base["note"] = "muestra vacia: media e intervalo indefinidos"
+        return base
+
+    mean = float(arr.mean())
+    base["mean"] = mean
+
+    if n < 2:
+        base["note"] = "n < 2: error estandar y grados de libertad indefinidos"
+        return base
+
+    df = n - 1
+    base["df"] = float(df)
+
+    sd = float(arr.std(ddof=1))
+    se = sd / math.sqrt(n)
+    base["se"] = se
+
+    # Varianza cero: el IC colapsa al punto (no es un error).
+    if se == 0.0:
+        base["ci_low"] = mean
+        base["ci_high"] = mean
+        base["note"] = "varianza cero: el intervalo colapsa a la media"
+        return base
+
+    tcrit = float(stats.t.ppf((1.0 + conf) / 2.0, df))
+    margin = tcrit * se
+    base["ci_low"] = mean - margin
+    base["ci_high"] = mean + margin
+    return base
+
+
+def _ci_welch(data: list, other: list, conf: float) -> dict:
+    """IC de la diferencia de medias de dos muestras con el metodo de Welch."""
+    a = np.asarray(list(data), dtype=float)
+    b = np.asarray(list(other), dtype=float)
+    n1 = int(a.size)
+    n2 = int(b.size)
+
+    base = {
+        "mean": float("nan"),
+        "ci_low": float("nan"),
+        "ci_high": float("nan"),
+        "se": float("nan"),
+        "df": float("nan"),
+        "confidence": conf,
+        "n": n1 + n2,
+        "n1": n1,
+        "n2": n2,
+    }
+
+    if n1 == 0 or n2 == 0:
+        base["note"] = "alguna muestra esta vacia: diferencia e intervalo indefinidos"
+        return base
+
+    mean1 = float(a.mean())
+    mean2 = float(b.mean())
+    diff = mean1 - mean2
+    base["mean"] = diff
+
+    if n1 < 2 or n2 < 2:
+        base["note"] = (
+            "n < 2 en alguna muestra: error estandar y grados de libertad indefinidos"
+        )
+        return base
+
+    sd1 = float(a.std(ddof=1))
+    sd2 = float(b.std(ddof=1))
+    se1 = sd1 / math.sqrt(n1)
+    se2 = sd2 / math.sqrt(n2)
+    se = math.sqrt(se1 * se1 + se2 * se2)
+    base["se"] = se
+
+    # Ambas varianzas cero: el IC de la diferencia colapsa al punto.
+    if se == 0.0:
+        base["ci_low"] = diff
+        base["ci_high"] = diff
+        base["df"] = float("nan")
+        base["note"] = "varianza cero en ambas muestras: el intervalo colapsa a la diferencia"
+        return base
+
+    # Grados de libertad de Welch-Satterthwaite.
+    df = (se1 * se1 + se2 * se2) ** 2 / (
+        (se1**4) / (n1 - 1) + (se2**4) / (n2 - 1)
+    )
+    base["df"] = float(df)
+
+    tcrit = float(stats.t.ppf((1.0 + conf) / 2.0, df))
+    margin = tcrit * se
+    base["ci_low"] = diff - margin
+    base["ci_high"] = diff + margin
+    return base

python/functions/datascience/confidence_interval_mean_test.py

@@ -0,0 +1,140 @@
+"""Tests para confidence_interval_mean (IC de la media / diferencia de medias Welch).
+
+Importa el modulo hoja directamente (`confidence_interval_mean`) para no depender
+de que el paquete reexporte la funcion en su __init__ (lo integra el orquestador
+al cerrar el grupo).
+
+Los golden se calculan con scipy dentro del propio test para que sean robustos:
+la funcion bajo prueba debe coincidir con la referencia de scipy a ~1e-9.
+"""
+
+import math
+
+import numpy as np
+from scipy import stats
+
+from confidence_interval_mean import confidence_interval_mean
+
+
+def test_one_sample_golden_contra_scipy():
+    # mean=5.0, n=8. Este dataset tiene sd POBLACIONAL (ddof=0) exactamente 2.0,
+    # pero la sd MUESTRAL (ddof=1, la que exige la spec y la que es correcta para
+    # el IC de una media con la t) es sqrt(32/7) ~ 2.13809. El golden robusto se
+    # calcula con scipy usando se con ddof=1, no con el atajo 2.0/sqrt(8).
+    data = [2, 4, 4, 4, 5, 5, 7, 9]
+    out = confidence_interval_mean(data, confidence=0.95)
+
+    n = len(data)
+    mean = float(np.mean(data))
+    sd = float(np.std(data, ddof=1))  # sample sd ~ 2.13809
+    se = sd / math.sqrt(n)
+    lo, hi = stats.t.interval(0.95, df=n - 1, loc=mean, scale=se)
+
+    assert abs(out["mean"] - 5.0) < 1e-9
+    assert abs(out["se"] - se) < 1e-12
+    assert out["df"] == 7.0
+    assert out["n"] == 8
+    assert out["confidence"] == 0.95
+    assert abs(out["ci_low"] - lo) < 1e-9
+    assert abs(out["ci_high"] - hi) < 1e-9
+    # Valores tabulados correctos para ddof=1 (no los 3.32793/6.67207 del
+    # enunciado, que asumian erroneamente sd=2.0 / ddof=0).
+    assert abs(out["ci_low"] - 3.21251) < 1e-3
+    assert abs(out["ci_high"] - 6.78749) < 1e-3
+    assert "note" not in out
+
+
+def test_one_sample_distinto_nivel_confianza():
+    data = [10.0, 12.0, 11.0, 13.0, 9.0, 14.0]
+    out = confidence_interval_mean(data, confidence=0.99)
+
+    n = len(data)
+    mean = float(np.mean(data))
+    se = float(np.std(data, ddof=1)) / math.sqrt(n)
+    lo, hi = stats.t.interval(0.99, df=n - 1, loc=mean, scale=se)
+
+    assert abs(out["mean"] - mean) < 1e-12
+    assert abs(out["ci_low"] - lo) < 1e-9
+    assert abs(out["ci_high"] - hi) < 1e-9
+    assert out["df"] == float(n - 1)
+
+
+def test_welch_diferencia_golden_contra_scipy():
+    data = [23.0, 21.0, 25.0, 22.0, 24.0, 26.0]
+    other = [18.0, 20.0, 17.0, 19.0, 21.0]
+    conf = 0.95
+    out = confidence_interval_mean(data, other, confidence=conf)
+
+    a = np.asarray(data, dtype=float)
+    b = np.asarray(other, dtype=float)
+    n1, n2 = a.size, b.size
+    mean1, mean2 = float(a.mean()), float(b.mean())
+    diff = mean1 - mean2
+    se1 = float(a.std(ddof=1)) / math.sqrt(n1)
+    se2 = float(b.std(ddof=1)) / math.sqrt(n2)
+    se = math.sqrt(se1**2 + se2**2)
+    df = (se1**2 + se2**2) ** 2 / (se1**4 / (n1 - 1) + se2**4 / (n2 - 1))
+    lo, hi = stats.t.interval(conf, df=df, loc=diff, scale=se)
+
+    assert abs(out["mean"] - diff) < 1e-9
+    assert abs(out["mean"] - (mean1 - mean2)) < 1e-9
+    assert abs(out["se"] - se) < 1e-12
+    assert abs(out["df"] - df) < 1e-9
+    assert abs(out["ci_low"] - lo) < 1e-9
+    assert abs(out["ci_high"] - hi) < 1e-9
+    assert out["n1"] == n1
+    assert out["n2"] == n2
+    assert out["n"] == n1 + n2
+    assert "note" not in out
+
+
+def test_edge_un_solo_elemento_no_lanza_nan_note():
+    out = confidence_interval_mean([5], confidence=0.95)
+    assert out["mean"] == 5.0  # la media si esta definida con n=1
+    assert math.isnan(out["se"])
+    assert math.isnan(out["ci_low"])
+    assert math.isnan(out["ci_high"])
+    assert math.isnan(out["df"])
+    assert out["n"] == 1
+    assert "note" in out
+
+
+def test_edge_lista_vacia_no_lanza_note():
+    out = confidence_interval_mean([], confidence=0.95)
+    assert math.isnan(out["mean"])
+    assert math.isnan(out["ci_low"])
+    assert math.isnan(out["ci_high"])
+    assert math.isnan(out["se"])
+    assert out["n"] == 0
+    assert "note" in out
+
+
+def test_edge_varianza_cero_colapsa_al_punto():
+    out = confidence_interval_mean([3, 3, 3], confidence=0.95)
+    assert out["mean"] == 3.0
+    assert out["se"] == 0.0
+    assert out["ci_low"] == 3.0
+    assert out["ci_high"] == 3.0
+    assert not math.isnan(out["ci_low"])
+    assert out["n"] == 3
+    assert "note" in out
+
+
+def test_edge_welch_muestra_vacia_no_lanza_note():
+    out = confidence_interval_mean([1.0, 2.0, 3.0], [], confidence=0.95)
+    assert math.isnan(out["mean"])
+    assert math.isnan(out["ci_low"])
+    assert math.isnan(out["se"])
+    assert out["n1"] == 3
+    assert out["n2"] == 0
+    assert "note" in out
+
+
+def test_edge_welch_n1_uno_no_lanza_note():
+    out = confidence_interval_mean([5.0], [1.0, 2.0, 3.0], confidence=0.95)
+    # La diferencia de medias si esta definida.
+    assert abs(out["mean"] - (5.0 - 2.0)) < 1e-9
+    assert math.isnan(out["se"])
+    assert math.isnan(out["ci_low"])
+    assert math.isnan(out["df"])
+    assert "note" in out

python/functions/datascience/effect_size_cohens_d.md

@@ -0,0 +1,80 @@
+---
+name: effect_size_cohens_d
+kind: function
+lang: py
+domain: datascience
+version: "1.0.0"
+purity: pure
+signature: "def effect_size_cohens_d(group_a: list, group_b: list) -> dict"
+description: "Tamano del efecto (effect size) entre dos grupos numericos: Cohen's d (diferencia de medias estandarizada por la desviacion tipica combinada, varianzas muestrales ddof=1), Hedges' g (d corregido por el sesgo al alza con muestras pequenas via el factor J) e interpretacion cualitativa de la magnitud segun los umbrales clasicos de Cohen (negligible/small/medium/large). El p-valor dice si hay diferencia; el effect size dice como de grande, de forma adimensional e independiente del N. Pura, sin dependencias externas; nunca lanza: los casos degenerados (varianza cero, N<2, listas vacias) devuelven NaN + una clave note."
+tags: [papers, statistics, effect-size, cohens-d, hedges-g, python]
+params:
+  - name: group_a
+    desc: "primera muestra (lista de numeros). Necesita >=2 observaciones para que exista la varianza muestral (ddof=1)."
+  - name: group_b
+    desc: "segunda muestra (lista de numeros). Necesita >=2 observaciones. El signo de cohens_d es positivo cuando mean_a > mean_b."
+output: "dict {cohens_d: float (diferencia de medias estandarizada, puede ser NaN), hedges_g: float (cohens_d * factor de correccion J, puede ser NaN), interpretation: str ('negligible'|'small'|'medium'|'large', o 'undefined' en casos degenerados), n_a: int, n_b: int, mean_a: float, mean_b: float, pooled_sd: float (desviacion tipica combinada)}. Casos degenerados (varianza cero en ambos grupos, N<2 en algun grupo, o listas vacias) anaden clave note. Nunca None ni excepcion."
+uses_functions: []
+uses_types: []
+returns: []
+returns_optional: false
+error_type: ""
+imports: [math]
+tested: true
+tests: ["test_golden_large_effect", "test_hedges_g_menor_en_magnitud_que_cohens_d", "test_interpretation_thresholds", "test_signo_positivo_cuando_a_mayor_que_b", "test_varianza_cero_no_lanza", "test_n_insuficiente_no_lanza", "test_listas_vacias_no_lanza", "test_un_grupo_vacio_no_lanza"]
+test_file_path: "python/functions/datascience/effect_size_cohens_d_test.py"
+file_path: "python/functions/datascience/effect_size_cohens_d.py"
+---
+
+## Ejemplo
+
+```python
+from datascience import effect_size_cohens_d
+
+# Dos grupos desplazados 2 unidades, misma dispersion.
+a = [1, 2, 3, 4, 5]   # media 3, varianza muestral 2.5
+b = [3, 4, 5, 6, 7]   # media 5, varianza muestral 2.5
+
+out = effect_size_cohens_d(a, b)
+print(out["cohens_d"])        # -> -1.264911...  (a esta 1.26 SD por debajo de b)
+print(out["hedges_g"])        # -> -1.142500...  (|g| < |d|: correccion N pequeno)
+print(out["interpretation"])  # -> "large"       (|d| >= 0.8)
+print(out["pooled_sd"])       # -> 1.581138...
+
+# Caso degenerado: varianza cero -> no lanza, NaN + note.
+deg = effect_size_cohens_d([5, 5, 5], [5, 5, 5])
+print(deg["interpretation"])  # -> "undefined"
+print(deg["note"])            # -> "varianza cero, effect size indefinido"
+```
+
+## Cuando usarla
+
+Cuando ya sepas que dos grupos difieren (o quieras cuantificar su diferencia)
+y necesites una medida **de magnitud, no de significancia**: comparar el antes
+y el despues de una intervencion, el grupo control frente al tratamiento, o dos
+cohortes. Reportala junto al p-valor para responder "¿como de grande es la
+diferencia?" — un p-valor minusculo con N enorme puede esconder un efecto
+trivial. Es adimensional (en unidades de desviaciones tipicas), asi que hace
+comparables resultados entre estudios y alimenta meta-analisis. Usa **Hedges' g**
+en lugar de Cohen's d cuando los grupos sean pequenos (decenas o menos): d
+sobreestima el efecto y g lo corrige.
+
+## Gotchas
+
+- Pura y sin dependencias externas (solo `math` de la stdlib).
+- Usa **varianza muestral** (ddof=1), no poblacional. Por eso cada grupo
+  necesita al menos 2 observaciones; con N=1 la varianza muestral no existe y la
+  funcion devuelve NaN + `note`.
+- **Nunca lanza excepcion**. Los casos degenerados devuelven `cohens_d` y
+  `hedges_g` a `float('nan')`, `interpretation="undefined"` y una clave `note`:
+  varianza cero en ambos grupos (`pooled_sd == 0`), N<2 en algun grupo, o listas
+  vacias. Comprueba con `math.isnan(out["cohens_d"])` o la presencia de `note`
+  antes de usar el resultado.
+- El **signo** de `cohens_d` depende del orden de los argumentos: positivo si
+  `mean_a > mean_b`, negativo en caso contrario. La `interpretation` usa `|d|`,
+  asi que no depende del orden.
+- `pooled_sd` asume varianzas comparables entre grupos (homogeneidad). Si las
+  dispersiones son muy distintas, Cohen's d clasico pierde precision; considera
+  variantes (Glass's delta) fuera del alcance de esta funcion.
+- Los umbrales de Cohen (0.2 / 0.5 / 0.8) son convencion, no ley: interpretalos
+  segun el dominio.

python/functions/datascience/effect_size_cohens_d.py

@@ -0,0 +1,156 @@
+"""Effect size de dos grupos: Cohen's d, Hedges' g e interpretacion cualitativa.
+
+Funcion pura del grupo papers. El p-valor responde a "¿hay diferencia?" pero no
+a "¿como de grande es?". El tamano del efecto (effect size) cuantifica la
+magnitud de la diferencia entre dos grupos de forma adimensional, independiente
+del N, y es lo que hace comparables resultados entre estudios (meta-analisis).
+
+- Cohen's d: diferencia de medias estandarizada por la desviacion tipica
+  combinada (pooled SD), con varianzas muestrales (ddof=1).
+- Hedges' g: Cohen's d corregido por el sesgo al alza que sufre d con muestras
+  pequenas, multiplicando por el factor de correccion J.
+- interpretation: etiqueta cualitativa de |d| segun los umbrales clasicos de
+  Cohen (negligible / small / medium / large).
+
+No usa dependencias externas: aritmetica de la libreria estandar (``math``).
+"""
+
+from __future__ import annotations
+
+import math
+
+
+def _mean(xs: list) -> float:
+    """Media aritmetica de una lista no vacia de numeros."""
+    return sum(float(x) for x in xs) / len(xs)
+
+
+def _sample_variance(xs: list, mean: float) -> float:
+    """Varianza muestral (ddof=1) de una lista con al menos 2 elementos."""
+    n = len(xs)
+    return sum((float(x) - mean) ** 2 for x in xs) / (n - 1)
+
+
+def _interpret(abs_d: float) -> str:
+    """Etiqueta cualitativa del tamano del efecto segun |d| (umbrales de Cohen)."""
+    if abs_d < 0.2:
+        return "negligible"
+    if abs_d < 0.5:
+        return "small"
+    if abs_d < 0.8:
+        return "medium"
+    return "large"
+
+
+def effect_size_cohens_d(group_a: list, group_b: list) -> dict:
+    """Calcula el tamano del efecto entre dos grupos numericos.
+
+    Devuelve Cohen's d (diferencia de medias estandarizada por la pooled SD),
+    Hedges' g (d corregido por sesgo de muestra pequena) y una etiqueta
+    cualitativa de la magnitud segun los umbrales de Cohen.
+
+    Es una funcion pura y determinista: no hace I/O, no muta la entrada. No lanza
+    excepcion ante datos degenerados; en su lugar devuelve un dict con
+    ``cohens_d`` / ``hedges_g`` a ``float('nan')``, ``interpretation`` a
+    ``"undefined"`` y una clave ``note`` explicando el caso.
+
+    Definiciones:
+        s_pooled = sqrt(((n1-1)*s1^2 + (n2-1)*s2^2) / (n1+n2-2)), con s1^2, s2^2
+            varianzas muestrales (ddof=1).
+        cohens_d = (mean_a - mean_b) / s_pooled.
+        J = 1 - 3 / (4*(n1+n2) - 9)  (factor de correccion de Hedges).
+        hedges_g = cohens_d * J.
+
+    Args:
+        group_a: primera muestra (lista de numeros). Necesita >=2 elementos para
+            que exista la varianza muestral.
+        group_b: segunda muestra (lista de numeros). Necesita >=2 elementos.
+
+    Returns:
+        dict con las claves:
+            cohens_d: float, diferencia de medias estandarizada (puede ser NaN).
+            hedges_g: float, Cohen's d corregido por sesgo (puede ser NaN).
+            interpretation: str, "negligible" | "small" | "medium" | "large", o
+                "undefined" en casos degenerados.
+            n_a: int, tamano de group_a.
+            n_b: int, tamano de group_b.
+            mean_a: float, media de group_a (NaN si vacio).
+            mean_b: float, media de group_b (NaN si vacio).
+            pooled_sd: float, desviacion tipica combinada (NaN si indefinida).
+
+        Casos degenerados (lista vacia, N<2 en algun grupo, o varianza cero en
+        ambos grupos -> pooled_sd == 0) anaden ademas una clave ``note``.
+    """
+    nan = float("nan")
+    n_a = len(group_a)
+    n_b = len(group_b)
+
+    # Listas vacias: ni media ni varianza definidas.
+    if n_a == 0 or n_b == 0:
+        return {
+            "cohens_d": nan,
+            "hedges_g": nan,
+            "interpretation": "undefined",
+            "n_a": n_a,
+            "n_b": n_b,
+            "mean_a": _mean(group_a) if n_a else nan,
+            "mean_b": _mean(group_b) if n_b else nan,
+            "pooled_sd": nan,
+            "note": "grupo vacio: media y varianza indefinidas, effect size indefinido",
+        }
+
+    mean_a = _mean(group_a)
+    mean_b = _mean(group_b)
+
+    # N insuficiente: la varianza muestral (ddof=1) no existe con un solo dato,
+    # y la correccion de Hedges no es fiable.
+    if n_a < 2 or n_b < 2:
+        return {
+            "cohens_d": nan,
+            "hedges_g": nan,
+            "interpretation": "undefined",
+            "n_a": n_a,
+            "n_b": n_b,
+            "mean_a": mean_a,
+            "mean_b": mean_b,
+            "pooled_sd": nan,
+            "note": (
+                "N insuficiente: cada grupo necesita >=2 observaciones para la "
+                "varianza muestral; effect size indefinido"
+            ),
+        }
+
+    var_a = _sample_variance(group_a, mean_a)
+    var_b = _sample_variance(group_b, mean_b)
+    pooled_sd = math.sqrt(
+        ((n_a - 1) * var_a + (n_b - 1) * var_b) / (n_a + n_b - 2)
+    )
+
+    # Varianza cero en ambos grupos: no se puede estandarizar (division por 0).
+    if pooled_sd == 0.0:
+        return {
+            "cohens_d": nan,
+            "hedges_g": nan,
+            "interpretation": "undefined",
+            "n_a": n_a,
+            "n_b": n_b,
+            "mean_a": mean_a,
+            "mean_b": mean_b,
+            "pooled_sd": 0.0,
+            "note": "varianza cero, effect size indefinido",
+        }
+
+    cohens_d = (mean_a - mean_b) / pooled_sd
+    j = 1.0 - 3.0 / (4.0 * (n_a + n_b) - 9.0)
+    hedges_g = cohens_d * j
+
+    return {
+        "cohens_d": cohens_d,
+        "hedges_g": hedges_g,
+        "interpretation": _interpret(abs(cohens_d)),
+        "n_a": n_a,
+        "n_b": n_b,
+        "mean_a": mean_a,
+        "mean_b": mean_b,
+        "pooled_sd": pooled_sd,
+    }

python/functions/datascience/effect_size_cohens_d_test.py

@@ -0,0 +1,96 @@
+"""Tests para effect_size_cohens_d (tamano del efecto de dos grupos).
+
+Importa el modulo hoja directamente (`effect_size_cohens_d`) para no depender de
+que el paquete reexporte la funcion en su __init__ (lo integra el orquestador al
+cerrar el grupo papers). El pytest del repo tiene pythonpath=["functions", ...],
+asi que el modulo hoja se resuelve por su nombre directo.
+"""
+
+import math
+
+from effect_size_cohens_d import effect_size_cohens_d
+
+
+def test_golden_large_effect():
+    # group_a: mean 3, var muestral 2.5; group_b: mean 5, var 2.5.
+    # pooled_sd = sqrt(2.5) ~= 1.5811388.
+    # cohens_d = (3-5)/1.5811388 ~= -1.264911.
+    # J = 1 - 3/(4*10-9) = 1 - 3/31 = 0.9032258.
+    # hedges_g = d * J = -1.2649111 * 0.9032258 ~= -1.142500.
+    out = effect_size_cohens_d([1, 2, 3, 4, 5], [3, 4, 5, 6, 7])
+    assert abs(out["cohens_d"] - (-1.26491)) < 1e-4
+    assert abs(out["hedges_g"] - (-1.14250)) < 1e-4
+    assert out["interpretation"] == "large"
+    assert out["n_a"] == 5
+    assert out["n_b"] == 5
+    assert abs(out["mean_a"] - 3.0) < 1e-12
+    assert abs(out["mean_b"] - 5.0) < 1e-12
+    assert abs(out["pooled_sd"] - math.sqrt(2.5)) < 1e-9
+    assert "note" not in out
+
+
+def test_hedges_g_menor_en_magnitud_que_cohens_d():
+    # La correccion J esta en (0, 1), asi que |g| < |d| siempre.
+    out = effect_size_cohens_d([1, 2, 3, 4, 5], [3, 4, 5, 6, 7])
+    assert abs(out["hedges_g"]) < abs(out["cohens_d"])
+
+
+def test_interpretation_thresholds():
+    # negligible: |d| < 0.2. Medias casi iguales con varianza grande.
+    neg = effect_size_cohens_d([0, 10, 20, 30], [1, 11, 21, 31])
+    assert neg["interpretation"] == "negligible"
+    assert abs(neg["cohens_d"]) < 0.2
+
+    # small: 0.2 <= |d| < 0.5.
+    small = effect_size_cohens_d([0, 10, 20, 30], [4, 14, 24, 34])
+    assert small["interpretation"] == "small"
+    assert 0.2 <= abs(small["cohens_d"]) < 0.5
+
+    # medium: 0.5 <= |d| < 0.8.
+    medium = effect_size_cohens_d([0, 10, 20, 30], [9, 19, 29, 39])
+    assert medium["interpretation"] == "medium"
+    assert 0.5 <= abs(medium["cohens_d"]) < 0.8
+
+
+def test_signo_positivo_cuando_a_mayor_que_b():
+    out = effect_size_cohens_d([10, 12, 14, 16], [1, 2, 3, 4])
+    assert out["cohens_d"] > 0
+    assert out["interpretation"] == "large"
+
+
+def test_varianza_cero_no_lanza():
+    out = effect_size_cohens_d([5, 5, 5], [5, 5, 5])
+    assert math.isnan(out["cohens_d"])
+    assert math.isnan(out["hedges_g"])
+    assert out["interpretation"] == "undefined"
+    assert out["pooled_sd"] == 0.0
+    assert "note" in out
+    assert "varianza cero" in out["note"]
+
+
+def test_n_insuficiente_no_lanza():
+    out = effect_size_cohens_d([3], [1, 2, 3])
+    assert math.isnan(out["cohens_d"])
+    assert math.isnan(out["hedges_g"])
+    assert out["interpretation"] == "undefined"
+    assert out["n_a"] == 1
+    assert out["n_b"] == 3
+    assert "note" in out
+
+
+def test_listas_vacias_no_lanza():
+    out = effect_size_cohens_d([], [])
+    assert math.isnan(out["cohens_d"])
+    assert math.isnan(out["hedges_g"])
+    assert out["interpretation"] == "undefined"
+    assert out["n_a"] == 0
+    assert out["n_b"] == 0
+    assert "note" in out
+
+
+def test_un_grupo_vacio_no_lanza():
+    out = effect_size_cohens_d([1, 2, 3], [])
+    assert math.isnan(out["cohens_d"])
+    assert out["interpretation"] == "undefined"
+    assert out["n_b"] == 0
+    assert "note" in out

python/functions/datascience/fdr_correction.md

@@ -3,19 +3,19 @@ name: fdr_correction
 kind: function
 lang: py
 domain: datascience
-version: "1.0.0"
+version: "1.1.0"
 purity: pure
 signature: "def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = \"bh\") -> dict"
-description: "Correccion de comparaciones multiples (multiple-testing) sobre una lista de p-valores: Benjamini-Hochberg (FDR, 'bh') o Bonferroni (FWER, 'bonferroni'). Antidoto al sesgo de mineria de datos (data-mining bias): al evaluar muchas hipotesis a la vez (todos los pares de una matriz), el azar produce falsos positivos; esta funcion ajusta los p-valores y marca cuales siguen siendo significativos tras corregir. Pura, sin dependencias externas, alineada 1:1 con la entrada (admite None en posiciones sin test)."
-tags: [eda, statistics, multiple-testing, fdr, benjamini-hochberg, bonferroni, p-value, data-mining-bias, python]
+description: "Correccion de comparaciones multiples (multiple-testing) sobre una lista de p-valores: Benjamini-Hochberg (FDR, 'bh'), Bonferroni (FWER, 'bonferroni') o Holm-Bonferroni (FWER step-down, 'holm', mas potente que Bonferroni simple). Antidoto al sesgo de mineria de datos (data-mining bias): al evaluar muchas hipotesis a la vez (todos los pares de una matriz), el azar produce falsos positivos; esta funcion ajusta los p-valores y marca cuales siguen siendo significativos tras corregir. Pura, sin dependencias externas, alineada 1:1 con la entrada (admite None en posiciones sin test)."
+tags: [eda, statistics, multiple-testing, fdr, benjamini-hochberg, bonferroni, holm, holm-bonferroni, fwer, p-value, data-mining-bias, python]
 params:
   - name: pvalues
     desc: "lista de p-valores (floats en [0, 1]). Se admiten None u otros valores no validos en posiciones sin test disponible; se propagan como None en la salida y no cuentan como prueba (m)."
   - name: alpha
     desc: "nivel de significancia objetivo tras la correccion (default 0.05). Para BH es el umbral del FDR; para Bonferroni, del FWER (tasa de error por familia)."
   - name: method
-    desc: "'bh' = Benjamini-Hochberg (controla FDR, menos conservador, mas potencia); 'bonferroni' = controla FWER (mas conservador). Cualquier otro valor devuelve un dict con note."
-output: "dict {p_values_adjusted: lista alineada con pvalues (float ajustado o None), reject: lista de bool (True = significativo tras corregir), n_tests: nº de p-valores validos (m), n_rejected: nº de hipotesis rechazadas, alpha: float aplicado, method: str}. Casos degenerados (vacio, sin p validos, metodo desconocido) anaden clave note. Nunca None ni excepcion."
+    desc: "'bh' = Benjamini-Hochberg (controla FDR, menos conservador, mas potencia); 'bonferroni' = controla FWER (mas conservador); 'holm' = Holm-Bonferroni (controla FWER, step-down, uniformemente mas potente que Bonferroni simple). Cualquier otro valor devuelve un dict con note."
+output: "dict {p_values_adjusted: lista alineada con pvalues (float ajustado o None), reject: lista de bool (True = significativo tras corregir), n_tests: nº de p-valores validos (m), n_rejected: nº de hipotesis rechazadas, alpha: float aplicado, method: str ('bh' | 'bonferroni' | 'holm')}. Casos degenerados (vacio, sin p validos, metodo desconocido) anaden clave note. Nunca None ni excepcion."
 uses_functions: []
 uses_types: []
 returns: []
@@ -23,7 +23,7 @@ returns_optional: false
 error_type: ""
 imports: [math]
 tested: true
-tests: ["test_bh_golden_rechaza_dos_de_tres", "test_bonferroni_mas_conservador_que_bh", "test_p_values_adjusted_alineados_y_en_rango", "test_none_se_propaga_alineado", "test_lista_vacia_devuelve_note", "test_solo_none_devuelve_note", "test_metodo_desconocido_devuelve_note", "test_todos_significativos"]
+tests: ["test_bh_golden_rechaza_dos_de_tres", "test_bonferroni_mas_conservador_que_bh", "test_p_values_adjusted_alineados_y_en_rango", "test_none_se_propaga_alineado", "test_lista_vacia_devuelve_note", "test_solo_none_devuelve_note", "test_metodo_desconocido_devuelve_note", "test_todos_significativos", "test_holm_golden_rechaza_dos_de_cuatro", "test_holm_entre_bonferroni_y_bh", "test_none_se_propaga_alineado_holm", "test_lista_vacia_holm_devuelve_note"]
 test_file_path: "python/functions/datascience/fdr_correction_test.py"
 file_path: "python/functions/datascience/fdr_correction.py"
 ---
@@ -45,6 +45,13 @@ bon = fdr_correction(pvalues, alpha=0.05, method="bonferroni")
 print(bon["reject"])       # -> [True, False, False]
 print(bon["p_values_adjusted"])  # -> [0.03, 0.06, 1.0]
 
+# Holm-Bonferroni (step-down): controla el FWER como Bonferroni pero es mas
+# potente; rechaza al menos tanto como Bonferroni simple, nunca menos.
+holm = fdr_correction([0.01, 0.04, 0.03, 0.005], alpha=0.05, method="holm")
+print(holm["reject"])      # -> [True, False, False, True]
+print(holm["p_values_adjusted"])  # -> [0.03, 0.06, 0.06, 0.02]
+print(holm["n_rejected"])  # -> 2
+
 # Posiciones sin test (None) se propagan alineadas: el llamador puede pasar la
 # lista completa de pares y recuperar el mapeo 1:1.
 mix = fdr_correction([0.001, None, 0.9])
@@ -61,8 +68,11 @@ combinaciones y se quede con las que "pasan". Sin corregir, con N pruebas y
 alpha=0.05 esperas ~5% de falsos positivos *por azar*: cuantas mas pruebas, mas
 correlaciones espurias. Llama a `fdr_correction` con todos los p-valores de la
 familia y usa `reject` (no el umbral crudo) para decidir que es real. Usa `"bh"`
-por defecto (mejor potencia); `"bonferroni"` cuando un falso positivo sea muy
-costoso y prefieras maxima cautela.
+por defecto (mejor potencia); `"holm"` (Holm-Bonferroni, FWER step-down) cuando
+quieras controlar el FWER pero sin la perdida de potencia de Bonferroni simple
+(rechaza al menos tanto como `"bonferroni"`, nunca menos); `"bonferroni"` cuando
+un falso positivo sea muy costoso y prefieras la maxima cautela del metodo mas
+simple.
 
 ## Gotchas
 
@@ -76,8 +86,16 @@ costoso y prefieras maxima cautela.
   eso puedes pasar la lista completa de pares aunque algunos no tengan test.
 - `n_tests` es el numero de p-valores **validos** (m), que puede ser menor que
   `len(pvalues)` si hay `None`.
-- BH y Bonferroni controlan cosas distintas: BH la tasa de falsos
-  descubrimientos (FDR), Bonferroni la probabilidad de *cualquier* falso
+- BH controla cosa distinta que Bonferroni/Holm: BH la tasa de falsos
+  descubrimientos (FDR); Bonferroni y Holm la probabilidad de *cualquier* falso
   positivo (FWER). No son intercambiables; elige segun el coste de equivocarte.
+- `"holm"` y `"bonferroni"` controlan ambos el FWER, pero Holm es step-down y
+  uniformemente mas potente: rechaza al menos tantas hipotesis como Bonferroni
+  simple sobre el mismo set, nunca menos. Si controlas FWER, `"holm"` domina a
+  `"bonferroni"` salvo que necesites el ajuste mas simple por interpretabilidad.
 - Metodo desconocido o lista vacia/sin p validos no lanzan: devuelven un dict
-  con `note`.
+  con `note`. Los metodos validos son `"bh"`, `"bonferroni"` y `"holm"`.
+
+## Capability growth log
+
+- v1.1.0 (2026-06-30) — añade method="holm" (Holm-Bonferroni step-down, FWER, más potente que Bonferroni simple).

python/functions/datascience/fdr_correction.py

@@ -5,12 +5,15 @@ todos los pares de una matriz de asociacion), la probabilidad de obtener al meno
 un falso positivo por azar crece con el numero de pruebas: es el sesgo de mineria
 de datos (data-mining bias) descrito por Aronson en *Evidence-Based Technical
 Analysis* (cap. 6). Esta funcion ajusta los p-valores para controlar ese sesgo
-mediante dos metodos clasicos:
+mediante tres metodos clasicos:
 
 - Benjamini-Hochberg (``"bh"``): controla la tasa de falsos descubrimientos
   (False Discovery Rate, FDR). Menos conservador, mas potencia estadistica.
 - Bonferroni (``"bonferroni"``): controla la tasa de error por familia
   (Family-Wise Error Rate, FWER). Mas conservador.
+- Holm-Bonferroni (``"holm"``): controla el FWER como Bonferroni pero es un
+  procedimiento step-down uniformemente mas potente; rechaza al menos tantas
+  hipotesis como Bonferroni simple, nunca menos.
 
 No usa dependencias externas: aritmetica de la libreria estandar.
 """
@@ -35,8 +38,9 @@ def _is_valid_p(v) -> bool:
 def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = "bh") -> dict:
     """Corrige una lista de p-valores por comparaciones multiples.
 
-    Aplica Benjamini-Hochberg (FDR) o Bonferroni (FWER) sobre ``pvalues`` y
-    devuelve, alineado posicion a posicion con la entrada, el p-valor ajustado y
+    Aplica Benjamini-Hochberg (FDR), Bonferroni (FWER) o Holm-Bonferroni
+    (FWER, step-down) sobre ``pvalues`` y devuelve, alineado posicion a
+    posicion con la entrada, el p-valor ajustado y
     si cada hipotesis se rechaza al nivel ``alpha`` tras la correccion. Las
     posiciones cuyo valor no sea un p-valor valido (``None``, ``NaN``, fuera de
     ``[0, 1]`` o no numerico) se conservan en la salida como ``None`` /
@@ -53,8 +57,10 @@ def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = "bh") -> di
             otros valores no validos en posiciones sin test disponible; se
             propagan como ``None`` en la salida y no cuentan como prueba.
         alpha: nivel de significancia objetivo tras la correccion (default 0.05).
-            Para BH es el umbral del FDR; para Bonferroni, del FWER.
-        method: ``"bh"`` (Benjamini-Hochberg, FDR) o ``"bonferroni"`` (FWER).
+            Para BH es el umbral del FDR; para Bonferroni y Holm, del FWER.
+        method: ``"bh"`` (Benjamini-Hochberg, FDR), ``"bonferroni"`` (FWER) o
+            ``"holm"`` (Holm-Bonferroni, FWER step-down, mas potente que
+            Bonferroni simple).
 
     Returns:
         dict con las claves:
@@ -68,7 +74,7 @@ def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = "bh") -> di
             n_tests: numero de p-valores validos usados en la correccion (m).
             n_rejected: numero de hipotesis rechazadas (significativas).
             alpha: nivel de significancia aplicado (float).
-            method: metodo aplicado (``"bh"`` o ``"bonferroni"``).
+            method: metodo aplicado (``"bh"``, ``"bonferroni"`` o ``"holm"``).
 
         Casos degenerados (lista vacia, sin p-valores validos o metodo
         desconocido) anaden ademas una clave ``note`` y devuelven listas
@@ -76,7 +82,7 @@ def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = "bh") -> di
         en las posiciones invalidas).
     """
     method_norm = (method or "").strip().lower()
-    if method_norm not in {"bh", "bonferroni"}:
+    if method_norm not in {"bh", "bonferroni", "holm"}:
         n = len(pvalues)
         return {
             "p_values_adjusted": [None] * n,
@@ -86,8 +92,8 @@ def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = "bh") -> di
             "alpha": float(alpha),
             "method": method,
             "note": (
-                f"metodo desconocido '{method}'; usa 'bh' (Benjamini-Hochberg) "
-                "o 'bonferroni'"
+                f"metodo desconocido '{method}'; usa 'bh' (Benjamini-Hochberg), "
+                "'bonferroni' o 'holm' (Holm-Bonferroni)"
             ),
         }
 
@@ -129,6 +135,20 @@ def fdr_correction(pvalues: list, alpha: float = 0.05, method: str = "bh") -> di
             padj = min(1.0, p * m)
             adjusted[orig_idx] = padj
             reject[orig_idx] = padj <= a
+    elif method_norm == "holm":
+        # Holm-Bonferroni (step-down). Ordena p ascendente; para el rank k
+        # (1-indexed) el p ajustado crudo es (m - k + 1) * p_(k). Impon
+        # monotonicidad acumulada (no decreciente) recorriendo de menor a mayor:
+        # padj_(k) = max(padj_(k-1), min(1, (m-k+1)*p_(k))), con padj_(0)=0.
+        order = sorted(valid, key=lambda t: t[1])  # [(orig_idx, p), ...] por p asc
+        prev = 0.0
+        for k in range(1, m + 1):
+            orig_idx, p = order[k - 1]
+            raw = min(1.0, (m - k + 1) * p)
+            padj = max(prev, raw)
+            prev = padj
+            adjusted[orig_idx] = padj
+            reject[orig_idx] = padj <= a
     else:
         # Benjamini-Hochberg (step-up). Ordena p ascendente y calcula q-valores
         # con la monotonicidad acumulada de derecha a izquierda.

python/functions/datascience/fdr_correction_test.py

@@ -82,7 +82,8 @@ def test_solo_none_devuelve_note():
 
 
 def test_metodo_desconocido_devuelve_note():
-    out = fdr_correction([0.01, 0.02], method="holm")
+    # 'holm' ya es un metodo valido (v1.1.0); usamos uno realmente desconocido.
+    out = fdr_correction([0.01, 0.02], method="sidak")
     assert "note" in out
     assert out["n_rejected"] == 0
     assert out["reject"] == [False, False]
@@ -97,3 +98,66 @@ def test_todos_significativos():
     assert bon["n_rejected"] == 3
     assert all(bh["reject"])
     assert all(bon["reject"])
+
+
+def test_holm_golden_rechaza_dos_de_cuatro():
+    # Holm-Bonferroni (step-down) sobre [0.01, 0.04, 0.03, 0.005], m=4, alpha=0.05.
+    # Ordenado ascendente: 0.005, 0.01, 0.03, 0.04.
+    #   padj_(1) = 4*0.005 = 0.02
+    #   padj_(2) = max(0.02, 3*0.01=0.03) = 0.03
+    #   padj_(3) = max(0.03, 2*0.03=0.06) = 0.06
+    #   padj_(4) = max(0.06, 1*0.04=0.04) = 0.06
+    # Mapeado al orden de entrada [0.01, 0.04, 0.03, 0.005]:
+    #   0.01 -> 0.03, 0.04 -> 0.06, 0.03 -> 0.06, 0.005 -> 0.02
+    out = fdr_correction([0.01, 0.04, 0.03, 0.005], alpha=0.05, method="holm")
+    assert out["method"] == "holm"
+    assert out["n_tests"] == 4
+    adj = out["p_values_adjusted"]
+    assert abs(adj[0] - 0.03) < 1e-9
+    assert abs(adj[1] - 0.06) < 1e-9
+    assert abs(adj[2] - 0.06) < 1e-9
+    assert abs(adj[3] - 0.02) < 1e-9
+    assert out["reject"] == [True, False, False, True]
+    assert out["n_rejected"] == 2
+
+
+def test_holm_entre_bonferroni_y_bh():
+    # Holm controla FWER como Bonferroni pero es step-down: rechaza AL MENOS
+    # tanto como Bonferroni simple, y a lo sumo tanto como BH (FDR, menos
+    # conservador). Cadena de potencia: bonferroni <= holm <= bh.
+    pvalues = [0.01, 0.02, 0.04, 0.005]
+    bon = fdr_correction(pvalues, alpha=0.05, method="bonferroni")
+    holm = fdr_correction(pvalues, alpha=0.05, method="holm")
+    bh = fdr_correction(pvalues, alpha=0.05, method="bh")
+    assert holm["n_rejected"] >= bon["n_rejected"]
+    assert holm["n_rejected"] <= bh["n_rejected"]
+    # En este set Holm gana potencia frente a Bonferroni simple (estricto).
+    assert holm["n_rejected"] > bon["n_rejected"]
+
+    # Un set donde Holm es estrictamente mas conservador que BH.
+    pvals2 = [0.01, 0.02, 0.03, 0.04]
+    bon2 = fdr_correction(pvals2, alpha=0.05, method="bonferroni")
+    holm2 = fdr_correction(pvals2, alpha=0.05, method="holm")
+    bh2 = fdr_correction(pvals2, alpha=0.05, method="bh")
+    assert holm2["n_rejected"] >= bon2["n_rejected"]
+    assert holm2["n_rejected"] < bh2["n_rejected"]
+
+
+def test_none_se_propaga_alineado_holm():
+    # None se propaga alineado tambien con holm: la posicion central no cuenta
+    # como prueba (m=2) y se devuelve como None / False.
+    out = fdr_correction([0.001, None, 0.9], method="holm")
+    assert out["n_tests"] == 2
+    assert out["p_values_adjusted"][1] is None
+    assert out["reject"][1] is False
+    assert out["reject"][0] is True
+    assert len(out["reject"]) == 3
+
+
+def test_lista_vacia_holm_devuelve_note():
+    out = fdr_correction([], method="holm")
+    assert out["p_values_adjusted"] == []
+    assert out["reject"] == []
+    assert out["n_tests"] == 0
+    assert out["n_rejected"] == 0
+    assert "note" in out

python/functions/datascience/preregister_hypothesis.md

@@ -0,0 +1,100 @@
+---
+name: preregister_hypothesis
+kind: function
+lang: py
+domain: datascience
+version: "1.0.0"
+purity: impure
+signature: "def preregister_hypothesis(paper_dir: str, hypotheses: dict, analysis_plan: dict) -> dict"
+description: "Pre-registra (congela) la hipotesis y el plan de analisis de un paper ANTES de mirar los datos: antidoto al HARKing (Hypothesizing After the Results are Known). Escribe/actualiza <paper_dir>/preregistration.md con un frontmatter (paper_slug, frozen_at, content_hash, status) y un cuerpo markdown DETERMINISTA derivado de (hypotheses, analysis_plan) (mismo input -> mismo cuerpo byte a byte, claves ordenadas alfabeticamente). El content_hash es sha256 del cuerpo NORMALIZADO (strip por linea + colapso de blancos), nunca del frontmatter. Una vez status=frozen es INMUTABLE: re-congelar con el mismo contenido es idempotente (no reescribe, devuelve unchanged) y re-congelar con contenido distinto se RECHAZA (no sobrescribe, devuelve error) para que no se pueda ajustar la hipotesis a los resultados. Estilo dict-no-throw: nunca lanza."
+tags: [papers, preregistration, reproducibility, anti-harking, python]
+params:
+  - name: paper_dir
+    desc: "ruta del directorio del paper, p.ej. 'papers/0001-mi-paper'. Debe existir (no se crea aqui). El paper_slug del frontmatter es el basename del dir. Si no existe o no es str -> {status:error, path, note} sin crash ni creacion."
+  - name: hypotheses
+    desc: "dict de hipotesis, p.ej. {'h0': 'no hay diferencia ...', 'h1': 'el grupo A > grupo B ...'}. Se renderiza en la seccion '## Hypotheses' con una linea por clave, ordenadas alfabeticamente para determinismo."
+  - name: analysis_plan
+    desc: "dict con el plan de analisis, p.ej. {'test': 'welch_t_test', 'effect_size_metric': 'cohens_d', 'decision_rule': 'rechazar H0 si p<0.05 tras Holm y |d|>=0.5', 'planned_n': 100, 'multiple_correction': 'holm'}. Se renderiza en '## Analysis plan' con una linea por clave (ordenadas alfabeticamente). Acepta valores no-str (int, etc.)."
+output: "dict dict-no-throw (NUNCA lanza). status='frozen' cuando escribe el archivo por primera vez o congela un draft previo ({status, path, content_hash, frozen_at}). status='unchanged' cuando ya estaba frozen con el mismo content_hash: no reescribe y preserva el archivo byte-identico incl. el frozen_at original ({status, path, content_hash, frozen_at}). status='error' cuando paper_dir no existe, ya esta frozen con un hash distinto (rechazo anti-HARKing, no sobrescribe), inputs invalidos o error de I/O ({status, path, note, [content_hash]})."
+uses_functions: []
+uses_types: []
+returns: []
+returns_optional: false
+error_type: "error_go_core"
+imports: [hashlib]
+tested: true
+tests: ["test_golden_congela_y_escribe_archivo", "test_idempotente_mismo_input_no_reescribe", "test_inmutabilidad_anti_harking_rechaza_contenido_distinto", "test_error_paper_dir_inexistente_no_crash_no_crea"]
+test_file_path: "python/functions/datascience/preregister_hypothesis_test.py"
+file_path: "python/functions/datascience/preregister_hypothesis.py"
+---
+
+## Ejemplo
+
+```python
+import os, tempfile
+from datascience import preregister_hypothesis
+
+# Un directorio de paper que ya existe.
+paper_dir = tempfile.mkdtemp(prefix="0001-")
+
+hypotheses = {
+    "h0": "no hay diferencia entre el grupo A y el grupo B",
+    "h1": "el grupo A tiene mayor conversion que el grupo B",
+}
+analysis_plan = {
+    "test": "welch_t_test",
+    "effect_size_metric": "cohens_d",
+    "decision_rule": "rechazar H0 si p<0.05 tras Holm y |d|>=0.5",
+    "planned_n": 100,
+    "multiple_correction": "holm",
+}
+
+# 1) Primera vez: congela y escribe <paper_dir>/preregistration.md
+r1 = preregister_hypothesis(paper_dir, hypotheses, analysis_plan)
+print(r1["status"])        # -> "frozen"
+print(r1["content_hash"])  # sha256 del cuerpo
+
+# 2) Mismo input: idempotente, no reescribe.
+r2 = preregister_hypothesis(paper_dir, hypotheses, analysis_plan)
+print(r2["status"])        # -> "unchanged"
+
+# 3) Cambiar la hipotesis tras congelar (HARKing): rechazado, archivo intacto.
+r3 = preregister_hypothesis(paper_dir, {"h0": "...", "h1": "otra cosa"}, analysis_plan)
+print(r3["status"])        # -> "error"
+```
+
+## Cuando usarla
+
+Llamala al ARRANCAR el analisis de un paper, antes de tocar los datos, para
+dejar por escrito (y firmado por hash) que vas a probar y como vas a decidir.
+Es el primer paso de un flujo reproducible: pre-registras la hipotesis y el plan
+(`test`, `effect_size_metric`, `decision_rule`, `planned_n`,
+`multiple_correction`), y solo despues corres el analisis y comparas con lo
+pre-registrado. Si mas tarde el analisis "descubre" otra hipotesis que encaja
+mejor con los datos, el pre-registro congelado deja en evidencia el cambio: no se
+puede reescribir. Combinala con `effect_size_cohens_d` y `fdr_correction` para
+cerrar el plan declarado (effect size + correccion de multiples comparaciones).
+
+## Gotchas
+
+- **Inmutabilidad (el corazon)**: una vez `status: frozen`, el pre-registro NO se
+  puede editar. Re-congelar con el MISMO contenido es idempotente (`unchanged`,
+  no reescribe, preserva incluso el `frozen_at` original). Re-congelar con
+  contenido DISTINTO devuelve `error` y deja el archivo intacto: asi se mata el
+  HARKing. Para cambiar de verdad la hipotesis hay que borrar el archivo a mano y
+  asumir explicitamente que ya no es un pre-registro valido.
+- **dict-no-throw**: la funcion NUNCA lanza. Cualquier error previsible
+  (directorio inexistente, inputs no-dict, fallo de I/O, excepcion inesperada) se
+  captura y se devuelve como `{"status": "error", "note": ...}`. Siempre incluye
+  `path` (la ruta esperada del `preregistration.md`).
+- **El hash es SOLO del cuerpo, nunca del frontmatter**: el frontmatter contiene
+  el propio `content_hash` y el `frozen_at` (timestamp), asi que incluirlos en el
+  hash seria circular y romperia la idempotencia. El cuerpo se normaliza antes de
+  hashear (strip por linea + colapso de lineas en blanco + strip final): cambios
+  irrelevantes de whitespace no alteran el hash, pero cambios de contenido SI.
+- **Determinismo**: el cuerpo se genera con las claves de `hypotheses` y
+  `analysis_plan` ordenadas alfabeticamente, de modo que el orden de insercion del
+  dict no afecta al hash. Mismo `(hypotheses, analysis_plan)` -> mismo cuerpo y
+  mismo hash, byte a byte.
+- **No crea el directorio del paper**: si `paper_dir` no existe, devuelve `error`
+  sin crear nada (ni el dir ni el archivo).

python/functions/datascience/preregister_hypothesis.py

@@ -0,0 +1,202 @@
+"""Congela (pre-registra) la hipotesis y el plan de analisis de un paper.
+
+Anti-HARKing (Hypothesizing After the Results are Known): el pre-registro fija
+la hipotesis y el plan de analisis ANTES de mirar los datos. Una vez congelado
+(``status: frozen``) es INMUTABLE: cualquier intento posterior de re-congelar con
+un contenido distinto se RECHAZA en vez de sobrescribir, de modo que no se puede
+"ajustar" la hipotesis a los resultados despues de verlos.
+
+Escribe/actualiza ``<paper_dir>/preregistration.md`` con un frontmatter
+(``paper_slug``, ``frozen_at``, ``content_hash``, ``status``) y un cuerpo
+markdown DETERMINISTA derivado de ``(hypotheses, analysis_plan)``.
+
+Estilo dict-no-throw: NUNCA lanza; cualquier error previsible se captura y se
+devuelve como ``{"status": "error", "note": ...}``.
+"""
+
+import hashlib
+import os
+from datetime import datetime, timezone
+
+
+def _build_body(hypotheses: dict, analysis_plan: dict) -> str:
+    """Construye el cuerpo markdown del pre-registro de forma DETERMINISTA.
+
+    Mismo ``(hypotheses, analysis_plan)`` -> mismo cuerpo byte a byte. Las claves
+    se ordenan alfabeticamente para no depender del orden de insercion del dict.
+    """
+    lines = ["## Hypotheses", ""]
+    for k in sorted(hypotheses.keys()):
+        lines.append(f"- **{k}**: {hypotheses[k]}")
+    lines.append("")
+    lines.append("## Analysis plan")
+    lines.append("")
+    for k in sorted(analysis_plan.keys()):
+        lines.append(f"- **{k}**: {analysis_plan[k]}")
+    return "\n".join(lines)
+
+
+def _normalize(body: str) -> str:
+    """Normaliza el cuerpo para el hash: strip por linea + colapsa blancos.
+
+    Cambios irrelevantes de whitespace (espacios al final, dobles lineas en
+    blanco) no alteran el hash; cambios de contenido SI. Esto hace el hash
+    robusto sin perder la capacidad de detectar ediciones reales.
+    """
+    out = []
+    prev_blank = False
+    for raw in body.splitlines():
+        line = raw.strip()
+        if line == "":
+            if prev_blank:
+                continue
+            prev_blank = True
+        else:
+            prev_blank = False
+        out.append(line)
+    return "\n".join(out).strip()
+
+
+def _content_hash(body: str) -> str:
+    """sha256 hex del cuerpo NORMALIZADO (nunca del frontmatter)."""
+    return hashlib.sha256(_normalize(body).encode("utf-8")).hexdigest()
+
+
+def _parse_frontmatter(text: str) -> dict:
+    """Parsea el frontmatter ``--- ... ---`` simple (key: value) de un .md."""
+    if not text.startswith("---"):
+        return {}
+    parts = text.split("---", 2)
+    if len(parts) < 3:
+        return {}
+    fm = {}
+    for line in parts[1].splitlines():
+        line = line.strip()
+        if not line or ":" not in line:
+            continue
+        key, _, value = line.partition(":")
+        fm[key.strip()] = value.strip()
+    return fm
+
+
+def _render_file(slug: str, frozen_at: str, content_hash: str, body: str) -> str:
+    """Compone el archivo completo: frontmatter frozen + cuerpo."""
+    return (
+        "---\n"
+        f"paper_slug: {slug}\n"
+        f"frozen_at: {frozen_at}\n"
+        f"content_hash: {content_hash}\n"
+        "status: frozen\n"
+        "---\n"
+        "\n"
+        f"{body}\n"
+    )
+
+
+def preregister_hypothesis(paper_dir: str, hypotheses: dict, analysis_plan: dict) -> dict:
+    """Congela la hipotesis y el plan de analisis de un paper (anti-HARKing).
+
+    Escribe ``<paper_dir>/preregistration.md`` con frontmatter ``status: frozen``
+    y un cuerpo markdown determinista. Una vez congelado es inmutable.
+
+    Args:
+        paper_dir: ruta del directorio del paper (p.ej. ``"papers/0001-mi-paper"``).
+            El ``paper_slug`` es el basename del directorio. Debe existir.
+        hypotheses: dict de hipotesis, p.ej.
+            ``{"h0": "no hay diferencia ...", "h1": "grupo A > grupo B ..."}``.
+        analysis_plan: dict con el plan, p.ej.
+            ``{"test": "welch_t_test", "effect_size_metric": "cohens_d",
+            "decision_rule": "...", "planned_n": 100, "multiple_correction": "holm"}``.
+
+    Returns:
+        dict dict-no-throw (NUNCA lanza). Claves segun el caso:
+          - frozen:    {"status": "frozen", "path", "content_hash", "frozen_at"}
+          - unchanged: {"status": "unchanged", "path", "content_hash", "frozen_at"}
+          - error:     {"status": "error", "path", "note", ...}
+    """
+    expected_path = os.path.join(paper_dir, "preregistration.md")
+    try:
+        # 1) El directorio del paper debe existir; no se crea aqui.
+        if not isinstance(paper_dir, str) or not os.path.isdir(paper_dir):
+            return {
+                "status": "error",
+                "path": expected_path,
+                "note": f"paper_dir no existe: {paper_dir}",
+            }
+
+        if not isinstance(hypotheses, dict) or not isinstance(analysis_plan, dict):
+            return {
+                "status": "error",
+                "path": expected_path,
+                "note": "hypotheses y analysis_plan deben ser dict",
+            }
+
+        slug = os.path.basename(os.path.normpath(paper_dir))
+
+        # 2) + 3) Cuerpo determinista y su hash (solo del cuerpo, no del frontmatter).
+        body = _build_body(hypotheses, analysis_plan)
+        new_hash = _content_hash(body)
+
+        # 5) Logica de escritura.
+        if os.path.exists(expected_path):
+            existing = ""
+            try:
+                with open(expected_path, "r", encoding="utf-8") as fh:
+                    existing = fh.read()
+            except OSError as exc:
+                return {
+                    "status": "error",
+                    "path": expected_path,
+                    "note": f"no se pudo leer el pre-registro existente: {exc}",
+                }
+            fm = _parse_frontmatter(existing)
+            old_status = fm.get("status", "")
+            old_hash = fm.get("content_hash", "")
+            old_frozen_at = fm.get("frozen_at", "")
+
+            if old_status == "frozen":
+                if old_hash == new_hash:
+                    # Idempotente: mismo contenido ya congelado. No se reescribe.
+                    return {
+                        "status": "unchanged",
+                        "path": expected_path,
+                        "content_hash": new_hash,
+                        "frozen_at": old_frozen_at,
+                    }
+                # Inmutabilidad: ya congelado con OTRO hash -> se rechaza (anti-HARKing).
+                return {
+                    "status": "error",
+                    "path": expected_path,
+                    "content_hash": new_hash,
+                    "note": (
+                        "pre-registro inmutable: ya esta congelado (frozen) con un "
+                        "hash distinto; un pre-registro no se puede editar tras "
+                        "congelarse"
+                    ),
+                }
+            # status != "frozen" (p.ej. draft) -> se congela ahora.
+
+        # Archivo nuevo o draft existente: congelar con timestamp actual.
+        frozen_at = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+        file_text = _render_file(slug, frozen_at, new_hash, body)
+        try:
+            with open(expected_path, "w", encoding="utf-8") as fh:
+                fh.write(file_text)
+        except OSError as exc:
+            return {
+                "status": "error",
+                "path": expected_path,
+                "note": f"no se pudo escribir el pre-registro: {exc}",
+            }
+        return {
+            "status": "frozen",
+            "path": expected_path,
+            "content_hash": new_hash,
+            "frozen_at": frozen_at,
+        }
+    except Exception as exc:  # noqa: BLE001 - dict-no-throw: nunca propagar.
+        return {
+            "status": "error",
+            "path": expected_path,
+            "note": f"error inesperado: {exc}",
+        }

python/functions/datascience/preregister_hypothesis_test.py

@@ -0,0 +1,99 @@
+"""Tests para preregister_hypothesis (pre-registro inmutable, anti-HARKing).
+
+Importa el modulo hoja directamente (`preregister_hypothesis`) para no depender
+de que el paquete reexporte la funcion en su __init__ (lo integra el orquestador
+al cerrar el grupo papers). El pytest del repo resuelve el modulo hoja por su
+nombre directo.
+
+Todos los tests son hermeticos y deterministas: usan el fixture `tmp_path` de
+pytest; NUNCA escriben en `papers/`.
+"""
+
+from preregister_hypothesis import preregister_hypothesis
+
+
+def _parse_frontmatter(text: str) -> dict:
+    parts = text.split("---", 2)
+    fm = {}
+    for line in parts[1].splitlines():
+        line = line.strip()
+        if not line or ":" not in line:
+            continue
+        key, _, value = line.partition(":")
+        fm[key.strip()] = value.strip()
+    return fm
+
+
+HYP = {"h0": "no hay diferencia entre A y B", "h1": "el grupo A > grupo B"}
+PLAN = {
+    "test": "welch_t_test",
+    "effect_size_metric": "cohens_d",
+    "decision_rule": "rechazar H0 si p<0.05 tras Holm y |d|>=0.5",
+    "planned_n": 100,
+    "multiple_correction": "holm",
+}
+
+
+def test_golden_congela_y_escribe_archivo(tmp_path):
+    paper = tmp_path / "0001-x"
+    paper.mkdir()
+
+    res = preregister_hypothesis(str(paper), HYP, PLAN)
+
+    assert res["status"] == "frozen"
+    pre = paper / "preregistration.md"
+    assert pre.exists()
+
+    text = pre.read_text(encoding="utf-8")
+    fm = _parse_frontmatter(text)
+    assert fm["status"] == "frozen"
+    assert fm["paper_slug"] == "0001-x"
+    assert fm["content_hash"]  # no vacio
+    assert fm["frozen_at"]     # no vacio
+    assert res["content_hash"] == fm["content_hash"]
+    assert res["frozen_at"] == fm["frozen_at"]
+
+
+def test_idempotente_mismo_input_no_reescribe(tmp_path):
+    paper = tmp_path / "0001-x"
+    paper.mkdir()
+    pre = paper / "preregistration.md"
+
+    first = preregister_hypothesis(str(paper), HYP, PLAN)
+    assert first["status"] == "frozen"
+    bytes_before = pre.read_bytes()
+
+    second = preregister_hypothesis(str(paper), HYP, PLAN)
+    assert second["status"] == "unchanged"
+    # Mismo hash y frozen_at original preservado.
+    assert second["content_hash"] == first["content_hash"]
+    assert second["frozen_at"] == first["frozen_at"]
+    # El archivo NO cambio byte a byte (incl. frozen_at).
+    assert pre.read_bytes() == bytes_before
+
+
+def test_inmutabilidad_anti_harking_rechaza_contenido_distinto(tmp_path):
+    paper = tmp_path / "0001-x"
+    paper.mkdir()
+    pre = paper / "preregistration.md"
+
+    preregister_hypothesis(str(paper), HYP, PLAN)
+    bytes_frozen = pre.read_bytes()
+
+    # Intento de re-congelar con una hipotesis DISTINTA (HARKing) -> rechazado.
+    hyp_tramposo = {"h0": "no hay diferencia", "h1": "el grupo B > grupo A (cambiado tras ver datos)"}
+    res = preregister_hypothesis(str(paper), hyp_tramposo, PLAN)
+
+    assert res["status"] == "error"
+    # Asercion mas importante: el archivo en disco SIGUE siendo el original.
+    assert pre.read_bytes() == bytes_frozen
+
+
+def test_error_paper_dir_inexistente_no_crash_no_crea(tmp_path):
+    missing = tmp_path / "no-existe"
+    res = preregister_hypothesis(str(missing), HYP, PLAN)
+
+    assert res["status"] == "error"
+    # No se creo el directorio ni el archivo.
+    assert not missing.exists()
+    assert not (missing / "preregistration.md").exists()

python/functions/datascience/render_automatic_eda_markdown.md

@@ -0,0 +1,89 @@
+---
+name: render_automatic_eda_markdown
+kind: function
+lang: py
+domain: datascience
+version: "1.0.0"
+purity: impure
+signature: "def render_automatic_eda_markdown(chapters_or_profile, out_path: str, meta: dict = None) -> dict"
+description: "Renderiza un documento AutomaticEDA por CAPÍTULOS (modelo de bloques independiente del formato) en un único MARKDOWN autocontenido pensado para PEGAR A UN LLM. Acepta una lista de capítulos del modelo o directamente un TableProfile del grupo eda (construye los capítulos canónicos con build_document). Prioriza TEXTO + DATOS sobre lo visual: las tablas se vuelcan como tablas markdown con TODAS las filas (sin paginar — no hay páginas que cortar), una figura matplotlib se reduce a su caption más la tabla de datos subyacente (Desde/Hasta/Frecuencia de las barras del histograma) porque un LLM no ve la imagen, y los marcadores de glosario se eliminan conservando el **negrita**. Lleva cabecera (# título), bloque de metadatos en blockquote e índice numerado con anclas GitHub. Espejo de render_automatic_eda_pdf/render_automatic_eda_pptx pero SIN manifest (KISS, el markdown es un único artefacto de texto). dict-no-throw: nunca lanza, devuelve {path, n_chars, chapters, note}; en error fatal path es None y note explica la causa. Flag opcional meta['embed_figures'] exporta PNGs junto al .md (off por defecto)."
+tags: [eda, markdown, render, report, llm, automatic-eda, chapters, versioned, no-cut, text, datascience, python]
+uses_functions: []
+uses_types: []
+returns: []
+returns_optional: false
+error_type: "error_go_core"
+imports: [os, re, matplotlib, "datascience.automatic_eda"]
+params:
+  - name: chapters_or_profile
+    desc: "una lista de capítulos del modelo AutomaticEDA (dataclasses Chapter o dicts {id,title,version,blocks}) O un TableProfile dict del grupo eda. Si es un TableProfile, los capítulos canónicos se construyen con build_document(profile, meta['ctx']). Bloques soportados: heading, markdown, kv_table, data_table, figure, image, caption, note, group, glossary_entry. Lectura defensiva: lo no reconocido se degrada a Note, nunca lanza."
+  - name: out_path
+    desc: "ruta del archivo .md de salida. Los directorios padre se crean si faltan. Directorio no escribible → {path:None, note:<causa>} sin lanzar."
+  - name: meta
+    desc: "dict opcional. Claves: title (título del documento), ctx (dict con dataset_name→Dataset, source_origin→Fuente, storage→Almacenamiento, n_rows/n_cols→Dimensiones; también lo consumen los builders de capítulo cuando se da un profile), generated_at (timestamp; si falta se genera ISO UTC), embed_figures (True para exportar PNGs <basename>_figN.png junto al .md; por defecto False y el markdown queda autocontenido)."
+output: "dict (nunca lanza): {path: str|None, n_chars: int, chapters: list[{id,version}], note: str}. En error fatal (p.ej. directorio no escribible) path es None y note explica la causa. Un documento sin capítulos aplicables produce un markdown mínimo válido con 'documento vacío' y chapters=[]."
+tested: true
+tests: ["test_golden_bloques_sinteticos_serializa_todo_a_markdown", "test_edge_documento_vacio_no_revienta", "test_profile_path_construye_capitulos_y_escribe"]
+test_file_path: "python/functions/datascience/render_automatic_eda_markdown_test.py"
+file_path: "python/functions/datascience/render_automatic_eda_markdown.py"
+---
+
+## Ejemplo
+
+```python
+from datascience import render_automatic_eda_markdown
+
+# Desde un TableProfile del grupo eda (mismo modelo que los renderers PDF/PPTX).
+profile = {
+    "table": "ventas", "source": "/data/ventas.csv",
+    "n_rows": 1000, "n_cols": 2, "quality_score": 92.5,
+    "columns": [
+        {"name": "precio", "inferred_type": "numeric", "null_pct": 0.01,
+         "numeric": {"mean": 42.5, "median": 40.0, "min": 1.0, "max": 100.0,
+                     "std": 12.3}},
+        {"name": "categoria", "inferred_type": "categorical", "null_pct": 0.0,
+         "categorical": {"top": [{"value": "neumaticos", "count": 500}]}},
+    ],
+}
+res = render_automatic_eda_markdown(
+    profile, "reports/ventas_aeda.md",
+    {"title": "EDA — ventas",
+     "ctx": {"dataset_name": "Ventas", "source_origin": "ERP export",
+             "n_rows": 1000, "n_cols": 2}})
+print(res["path"], res["n_chars"], res["chapters"])
+# -> reports/ventas_aeda.md 4123 [{'id':'portada','version':'1.0.0'}, ...]
+```
+
+## Cuando usarla
+
+Cuando quieras **pegar el EDA a un LLM** (ChatGPT, Claude, ...) o tenerlo en texto
+plano versionable: mismo documento por capítulos que el PDF/PPTX, pero serializado a
+Markdown sin binarios. Úsala como tercera salida junto a `render_automatic_eda_pdf`
+(móvil) y `render_automatic_eda_pptx` (compartir) desde el MISMO modelo de capítulos.
+A diferencia de esas dos, no hay páginas ni slides: todas las filas de cada tabla se
+vuelcan (nada se corta) y cada figura se reduce a su caption + la tabla de datos
+subyacente, que es lo que un LLM puede leer. Para añadir capítulos al documento, ver
+`docs/capabilities/automatic_eda.md`.
+
+## Gotchas
+
+- **Impura**: escribe el `.md` en `out_path` (crea los directorios padre). Con
+  `meta['embed_figures']=True` además exporta un PNG `<basename>_figN.png` por figura
+  junto al `.md`; por defecto NO exporta nada y el markdown queda autocontenido.
+- **Nunca lanza** (dict-no-throw): un bloque que falle se degrada a una nota y se anota
+  en `note`; el documento se escribe igual. Un profile/lista vacíos producen un markdown
+  mínimo válido con `*(documento vacío …)*` y `chapters=[]`.
+- **Figuras = datos, no imagen**: un bloque `figure` se serializa como `*Figura: caption*`
+  más, si la figura matplotlib trae barras (histograma / barras), una tabla
+  `| Desde | Hasta | Frecuencia |` extraída de los `Rectangle` patches (máx 100 filas;
+  el resto se trunca con `*… (N filas más)*`). Si no hay barras o algo falla, solo sale
+  el caption. La figura se cierra (`plt.close`) tras leerla.
+- **Glosario vs negrita**: se eliminan SOLO los marcadores de glosario
+  `[[term:key]]visible[[/term]]` (queda `visible`); el `**negrita**` markdown SE
+  CONSERVA (es válido). No se usa `strip_inline_md` aquí porque ese también quita el bold.
+- **Anclas del índice**: el `## Índice` enlaza cada capítulo con un ancla estilo GitHub
+  del encabezado `## N. Título` (minúsculas, espacios→`-`, sin signos). Si dos capítulos
+  comparten título exacto sus anclas colisionan (caso raro; los capítulos canónicos tienen
+  títulos únicos).
+- **Tablas**: las celdas escapan `|` (→ `\|`) y pliegan saltos de línea a `<br>` para no
+  romper la columna. No hay reparto por ancho — un LLM no lo necesita.

python/functions/datascience/render_automatic_eda_markdown.py

@@ -0,0 +1,55 @@
+"""render_automatic_eda_markdown — chapter-based EDA report as one Markdown file.
+
+Public ``eda``-group entry point that serializes an AutomaticEDA document (a list
+of chapters, or an ``eda`` TableProfile from which the canonical chapters are
+built) into a single self-contained Markdown file optimised to be **pasted into
+an LLM**: plain text, Markdown tables (every row dumped — there are no pages to
+cut), figures reduced to caption + underlying data, no binaries. It mirrors
+``render_automatic_eda_pdf`` / ``render_automatic_eda_pptx`` but for text output;
+unlike those it writes no manifest (KISS — Markdown is a single text artefact).
+
+dict-no-throw: never raises. Returns ``{path, n_chars, chapters, note}``; on a
+fatal error ``path`` is None and ``note`` explains why.
+"""
+
+from __future__ import annotations
+
+from datascience.automatic_eda import build_document, render_md
+from datascience.automatic_eda.model import as_chapter, as_chapters
+
+
+def _coerce_chapters(chapters_or_profile, meta: dict) -> list:
+    """Accept chapters OR an eda profile and return a list of Chapter."""
+    arg = chapters_or_profile
+    if isinstance(arg, (list, tuple)):
+        return as_chapters(list(arg))
+    if isinstance(arg, dict):
+        if "blocks" in arg and "columns" not in arg:
+            ch = as_chapter(arg)
+            return [ch] if ch is not None else []
+        return build_document(arg, (meta or {}).get("ctx"))
+    return []
+
+
+def render_automatic_eda_markdown(chapters_or_profile, out_path: str,
+                                  meta: dict = None) -> dict:
+    """Render an AutomaticEDA document into a single self-contained Markdown file.
+
+    Args:
+        chapters_or_profile: a list of chapters (``Chapter`` dataclasses or
+            dicts) or an ``eda`` TableProfile dict (chapters built via
+            ``build_document(profile, meta['ctx'])``).
+        out_path: filesystem path for the ``.md`` (parent dirs are created).
+        meta: optional dict. Recognised keys: ``title``, ``ctx`` (dict with
+            ``dataset_name``/``source_origin``/``storage``/``n_rows``/``n_cols``),
+            ``generated_at``, ``embed_figures`` (export PNGs beside the .md,
+            default False — off keeps the Markdown self-contained).
+
+    Returns:
+        dict (never raises): ``{path: str|None, n_chars: int,
+        chapters: list[{id, version}], note: str}``. On a fatal error ``path`` is
+        None and ``note`` explains the cause.
+    """
+    meta = dict(meta or {})
+    chapters = _coerce_chapters(chapters_or_profile, meta)
+    return render_md(chapters, out_path, meta)

python/functions/datascience/render_automatic_eda_markdown_test.py

@@ -0,0 +1,168 @@
+"""Tests for render_automatic_eda_markdown — DoD: golden + edge + profile path.
+
+Self-contained synthetic blocks (no DuckDB). Verifies every block kind serializes
+to Markdown (heading, markdown with glossary+bold, kv/data tables, a figure whose
+histogram bars become a data table, caption, note, group, glossary entry), that a
+leading level-1 heading equal to the chapter title is omitted, that an empty
+document degrades to a valid minimal Markdown without raising, and that passing a
+minimal TableProfile builds chapters and writes the file.
+"""
+
+import os
+import tempfile
+
+from datascience.render_automatic_eda_markdown import render_automatic_eda_markdown
+from datascience.automatic_eda.model import (
+    Caption, Chapter, DataTable, Figure, GlossaryEntry, Group, Heading, KVTable,
+    Markdown, Note,
+)
+
+
+def _hist_fig():
+    import matplotlib
+    matplotlib.use("Agg")
+    import matplotlib.pyplot as plt
+    fig, ax = plt.subplots()
+    ax.hist([1, 1, 2, 2, 2, 3, 4, 4, 5, 5, 5, 5], bins=5)
+    return fig
+
+
+def _chapters() -> list:
+    blocks = [
+        Heading("Demo", 1),                       # == chapter title -> omitted.
+        Heading("Seccion dos", 2),                # -> ####
+        Markdown("Texto con [[term:ent]]entropia[[/term]] y **bold** aqui."),
+        KVTable(rows=[("Filas", 1000), ("Columnas", 5)], title="Resumen"),
+        DataTable(header=["col", "valor"],
+                  rows=[["alpha", "111"], ["beta", "222"], ["gamma", "333"]],
+                  title="Datos", note="nota inferior"),
+        Figure(make=_hist_fig, caption="Histograma demo"),
+        Caption("pie de figura"),
+        Note("una nota aparte"),
+        Group(title="Grupo X", blocks=[Markdown("dentro del grupo")]),
+        GlossaryEntry(key="ent", label="Entropia",
+                      definition="Medida de incertidumbre."),
+    ]
+    return [Chapter(id="demo", title="Demo", version="1.0.0", blocks=blocks)]
+
+
+def _read(path: str) -> str:
+    with open(path, "r", encoding="utf-8") as fh:
+        return fh.read()
+
+
+def test_golden_bloques_sinteticos_serializa_todo_a_markdown():
+    with tempfile.TemporaryDirectory() as d:
+        out = os.path.join(d, "demo.md")
+        res = render_automatic_eda_markdown(
+            _chapters(), out,
+            {"title": "EDA Demo",
+             "ctx": {"dataset_name": "Demo", "n_rows": 12, "n_cols": 2}})
+        assert res["path"] == out
+        assert os.path.exists(out)
+        assert res["n_chars"] > 0
+        assert res["chapters"] == [{"id": "demo", "version": "1.0.0"}]
+
+        content = _read(out)
+        # Document structure.
+        assert content.startswith("# ")
+        assert "## Índice" in content
+        # A Markdown table is present (header + separator row).
+        assert "| " in content and "| --- " in content
+        # DataTable values are all dumped.
+        for v in ("alpha", "111", "beta", "222", "gamma", "333"):
+            assert v in content
+        # Glossary markers stripped, bold kept.
+        assert "[[term" not in content
+        assert "[[/term]]" not in content
+        assert "**bold**" in content
+        assert "entropia" in content  # visible glossary text preserved.
+        # Figure histogram bars became a data table.
+        assert "| Desde | Hasta | Frecuencia |" in content
+        # Glossary entry rendered as a level-3 heading.
+        assert "### Entropia" in content
+        # Level-2 heading -> ####.
+        assert "#### Seccion dos" in content
+        # Leading level-1 heading equal to the title was omitted.
+        assert "### Demo" not in content
+        # Group title rendered.
+        assert "### Grupo X" in content
+
+
+def _hist_fig_with_span():
+    """Histogram with a wide ``axvspan`` (±1σ band) over it.
+
+    Reproduces the num_distr figure shape: matplotlib keeps the span as a lone
+    Rectangle in ``ax.patches`` alongside the bin bars; it must NOT leak into the
+    extracted bins table as a fake bin (it is ~5x wider than a bin)."""
+    import matplotlib
+    matplotlib.use("Agg")
+    import matplotlib.pyplot as plt
+    fig, ax = plt.subplots()
+    data = [1, 1, 2, 2, 2, 3, 4, 4, 5, 5, 5, 5]
+    ax.hist(data, bins=5)
+    ax.axvspan(2.0, 4.0, alpha=0.2)   # mean±σ band — a wide stray rectangle.
+    return fig
+
+
+def test_figura_descarta_axvspan_de_la_tabla_de_bins():
+    """The ±1σ band rectangle must not appear as a row in the bins table."""
+    blocks = [Figure(make=_hist_fig_with_span, caption="Hist con banda")]
+    chapters = [Chapter(id="f", title="Fig", version="1.0.0", blocks=blocks)]
+    with tempfile.TemporaryDirectory() as d:
+        out = os.path.join(d, "fig.md")
+        render_automatic_eda_markdown(chapters, out, {"title": "T"})
+        content = _read(out)
+        assert "| Desde | Hasta | Frecuencia |" in content
+        # Extract the rows of the bins table: lines between the header/separator
+        # and the next blank line.
+        lines = content.splitlines()
+        hi = next(i for i, ln in enumerate(lines)
+                  if ln.startswith("| Desde | Hasta | Frecuencia |"))
+        rows = []
+        for ln in lines[hi + 2:]:           # skip header + separator
+            if not ln.startswith("|"):
+                break
+            rows.append(ln)
+        # 5 histogram bins, no extra wide span row.
+        assert len(rows) == 5, rows
+        # No row spans a width of ~2.0 (the axvspan from x=2 to x=4).
+        for ln in rows:
+            cells = [c.strip() for c in ln.strip("|").split("|")]
+            lo, hi_v = float(cells[0]), float(cells[1])
+            assert (hi_v - lo) < 1.5, f"wide span leaked: {ln}"
+
+
+def test_edge_documento_vacio_no_revienta():
+    with tempfile.TemporaryDirectory() as d:
+        out = os.path.join(d, "empty.md")
+        res = render_automatic_eda_markdown([], out, {})
+        assert res["path"] == out
+        assert os.path.exists(out)
+        assert res["chapters"] == []
+        content = _read(out)
+        assert "documento vacío" in content
+        assert content.startswith("# ")
+
+
+def test_profile_path_construye_capitulos_y_escribe():
+    profile = {
+        "table": "mini",
+        "source": "/data/mini.csv",
+        "n_rows": 10,
+        "n_cols": 1,
+        "quality_score": 88.0,
+        "columns": [
+            {"name": "x", "inferred_type": "numeric", "null_pct": 0.0,
+             "null_count": 0,
+             "numeric": {"mean": 1.0, "median": 1.0, "min": 0.0, "max": 2.0,
+                         "std": 0.5}},
+        ],
+    }
+    with tempfile.TemporaryDirectory() as d:
+        out = os.path.join(d, "mini.md")
+        res = render_automatic_eda_markdown(
+            profile, out, {"title": "Mini", "ctx": {"dataset_name": "Mini"}})
+        assert res["path"] == out  # not None — no exception, file written.
+        assert os.path.exists(out)
+        assert res["n_chars"] > 0

python/functions/pipelines/render_automatic_eda.py

@@ -1,9 +1,10 @@
-"""render_automatic_eda — EDA completo one-shot: perfil → ctx → PDF + PPTX.
+"""render_automatic_eda — EDA completo one-shot: perfil → ctx → PDF + PPTX + MD.
 
 Pipeline impuro del grupo de capacidad `eda`. Dada UNA tabla DuckDB (o
-PostgreSQL), produce el informe AutomaticEDA COMPLETO en sus dos formatos a la
-vez (PDF móvil A5 + PPTX 16:9) con los 11 capítulos POBLADOS, en una sola
-llamada. Compone, sin reimplementar su lógica, cuatro funciones del registry:
+PostgreSQL), produce el informe AutomaticEDA COMPLETO en sus tres formatos a la
+vez (PDF móvil A5 + PPTX 16:9 + Markdown autocontenido para pegar a un LLM) con
+los capítulos POBLADOS, en una sola llamada. Compone, sin reimplementar su
+lógica, varias funciones del registry:
 
   - profile_table          : perfila la tabla end-to-end (TableProfile agregado),
                              opcionalmente con modelos baratos y análisis de serie.
@@ -12,8 +13,11 @@ llamada. Compone, sin reimplementar su lógica, cuatro funciones del registry:
                              modelos/geo, timeseries_raw para series, geo_points
                              para el mapa, db_path/table para la agregación
                              push-down). Sin él, esos capítulos degradan.
-  - render_automatic_eda_pdf  : renderiza el documento por capítulos a PDF.
-  - render_automatic_eda_pptx : renderiza el mismo documento a PPTX.
+  - render_automatic_eda_pdf      : renderiza el documento por capítulos a PDF.
+  - render_automatic_eda_pptx     : renderiza el mismo documento a PPTX.
+  - render_automatic_eda_markdown : serializa el mismo documento a Markdown
+                                    autocontenido (texto + tablas markdown, sin
+                                    binarios) para incorporar a un LLM.
 
 El TableProfile agregado basta para portada/overview/distribuciones/calidad/
 correlación, pero los capítulos `modelos`, `timeseries`, `geospatial` y
@@ -32,6 +36,7 @@ from datetime import datetime, timezone
 
 from datascience import (
     build_eda_render_ctx,
+    render_automatic_eda_markdown,
     render_automatic_eda_pdf,
     render_automatic_eda_pptx,
     run_eda_models,
@@ -93,6 +98,7 @@ def render_automatic_eda(
     out_dir: str = "reports",
     basename: str = None,
     ctx_extra: dict = None,
+    emit_md: bool = True,
 ) -> dict:
     """Perfila una tabla y emite el informe AutomaticEDA completo (PDF + PPTX).
 
@@ -140,13 +146,19 @@ def render_automatic_eda(
         ctx_extra: dict opcional con claves de presentación/contexto extra que se
             mezclan en el ctx (p.ej. dataset_name, description, source_origin).
             No pisan las claves de datos calculadas por build_eda_render_ctx.
+        emit_md: además del PDF y el PPTX, emite un Markdown autocontenido del
+            MISMO documento por capítulos (texto plano + tablas markdown, sin
+            binarios), pensado para pegar a un LLM. Default True. La ruta sale en
+            la clave de retorno ``aeda_md_path``. No altera las demás salidas.
 
     Returns:
         dict (nunca lanza). En éxito::
 
             {"status": "ok", "pdf_path": str, "pptx_path": str,
-             "manifest_path": str|None, "n_pages": int, "n_slides": int,
-             "pdf_note": str, "pptx_note": str, "profile": <TableProfile>}
+             "aeda_md_path": str|None, "manifest_path": str|None,
+             "n_pages": int, "n_slides": int, "md_chars": int|None,
+             "pdf_note": str, "pptx_note": str, "md_note": str|None,
+             "profile": <TableProfile>}
 
         En error: {"status": "error", "error": str}.
     """
@@ -243,15 +255,26 @@ def render_automatic_eda(
         rpdf = render_automatic_eda_pdf(prof, pdf_path, meta) or {}
         rpptx = render_automatic_eda_pptx(prof, pptx_path, meta) or {}
 
+        # Salida Markdown autocontenida (mismo documento por capítulos) para
+        # pegar a un LLM. Aditiva: no afecta a PDF/PPTX/manifest. dict-no-throw.
+        rmd = {}
+        md_path = None
+        if emit_md:
+            md_path = os.path.join(out_dir, base + ".md")
+            rmd = render_automatic_eda_markdown(prof, md_path, meta) or {}
+
         return {
             "status": "ok",
             "pdf_path": rpdf.get("path"),
             "pptx_path": rpptx.get("path"),
+            "aeda_md_path": rmd.get("path"),
             "manifest_path": rpdf.get("manifest_path"),
             "n_pages": rpdf.get("n_pages"),
             "n_slides": rpptx.get("n_slides"),
+            "md_chars": rmd.get("n_chars"),
             "pdf_note": rpdf.get("note"),
             "pptx_note": rpptx.get("note"),
+            "md_note": rmd.get("note"),
             "profile": prof,
         }
     except Exception as e:  # noqa: BLE001 — dict-no-throw: degradar, nunca lanzar.