{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3687e8d1",
   "metadata": {},
   "source": [
    "# NuExtract 2.0-2B (GPU) vs GLiNER2 — comparativa con visualizacion\n",
    "\n",
    "**Pregunta:** ¿merece la pena un LLM con inferencia (NuExtract 2.0) en un proyecto donde antes elegimos GLiNER2 por velocidad?\n",
    "\n",
    "**Setup:**\n",
    "- NuExtract 2.0-2B (Qwen2-VL-2B base, **MIT license**, 2B params, GPU BF16 sobre RTX 3070).\n",
    "- GLiNER2-large-v1 (Apache 2.0, 340M params, CPU).\n",
    "- Mismos corpora: `es_corporate_short` (8 frases), `LONG_TEXT_ES` (25 frases), 5 chunks del PDF de BBVA.\n",
    "\n",
    "**Diferencia de paradigma:**\n",
    "- **GLiNER2** = clasificador. Output: listas planas `{entities: {tipo: [names]}, relations: {tipo: [(h, t)]}}`.\n",
    "- **NuExtract** = LLM generativo. Output: JSON arbitrario que tu defines en el `template`. Las relaciones se modelan como atributos de los objetos (`{org: {ceo: \"X\", headquartered_in: \"Y\"}}`).\n",
    "\n",
    "**Hipotesis:** NuExtract gana en _riqueza estructural_ (atributos por entidad de un solo paso) pero pierde en velocidad — incluso con GPU."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5691cee5",
   "metadata": {},
   "source": [
    "## 1. Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cd75a1d8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:55.012511Z",
     "iopub.status.busy": "2026-05-04T19:36:55.012317Z",
     "iopub.status.idle": "2026-05-04T19:36:55.652234Z",
     "shell.execute_reply": "2026-05-04T19:36:55.651410Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NuExtract keys: ['meta', 'cpu_baseline', 'T1_corp_short_flat', 'T2_corp_short_rich', 'T3_long_text_rich', 'pdf_meta', 'T4_pdf_chunks', 'full_pdf_extrapolation']\n",
      "GLiNER2 keys:  ['meta', 'configs', 'coref', 'top_entities_post_coref', 'top_relations_post_coref', 'ents_merged', 'rels_merged']\n",
      "\n",
      "NuExtract device: cuda torch.bfloat16\n"
     ]
    }
   ],
   "source": [
    "import os, sys, json, warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "from pathlib import Path\n",
    "from collections import defaultdict\n",
    "\n",
    "_pf = '/home/lucas/fn_registry/python/functions'\n",
    "sys.path = [p for p in sys.path if not p.startswith(_pf + '/')]\n",
    "if _pf not in sys.path: sys.path.insert(0, _pf)\n",
    "\n",
    "import pandas as pd\n",
    "import networkx as nx\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.patches import Patch\n",
    "\n",
    "NUEX = json.loads(Path('../nuextract_results.json').read_text())\n",
    "\n",
    "# Re-parsear el raw_text de cada test con un parser corregido (el original\n",
    "# del script usaba rfind y solo capturaba el ultimo objeto pequeño).\n",
    "def reparse(text):\n",
    "    if not text: return None\n",
    "    s = text.find('{')\n",
    "    if s < 0: return None\n",
    "    for end in range(len(text), s, -1):\n",
    "        try: return json.loads(text[s:end])\n",
    "        except Exception: continue\n",
    "    return None\n",
    "for key in ['T1_corp_short_flat', 'T2_corp_short_rich', 'T3_long_text_rich']:\n",
    "    if key in NUEX:\n",
    "        NUEX[key]['parsed'] = reparse(NUEX[key].get('raw_text', ''))\n",
    "for cr in NUEX.get('T4_pdf_chunks', []):\n",
    "    cr['parsed'] = reparse(cr.get('raw_text', ''))\n",
    "GLNR_CORPUS = json.loads(Path('../benchmark_v2.json').read_text())     # GLiNER2 sobre 4 corpora\n",
    "GLNR = json.loads(Path('../improvements.json').read_text())            # GLiNER2 sobre PDF + improvements\n",
    "print('NuExtract keys:', list(NUEX.keys()))\n",
    "print('GLiNER2 keys: ', list(GLNR.keys()))\n",
    "print()\n",
    "print('NuExtract device:', NUEX['meta']['device'], NUEX['meta']['dtype'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c1d64c1",
   "metadata": {},
   "source": [
    "## 2. Tabla de tiempos — CPU vs GPU vs GLiNER2\n",
    "\n",
    "Comparamos las 4 pasadas (T1-T4) de NuExtract contra GLiNER2 sobre los mismos corpora."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9d4c55ad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:55.654408Z",
     "iopub.status.busy": "2026-05-04T19:36:55.654139Z",
     "iopub.status.idle": "2026-05-04T19:36:55.669174Z",
     "shell.execute_reply": "2026-05-04T19:36:55.668310Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test</th>\n",
       "      <th>engine</th>\n",
       "      <th>time_s</th>\n",
       "      <th>in_tok</th>\n",
       "      <th>out_tok</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>T1 corp_short flat</td>\n",
       "      <td>NuExtract CPU</td>\n",
       "      <td>24.98</td>\n",
       "      <td>245</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>T2 corp_short rich</td>\n",
       "      <td>NuExtract CPU</td>\n",
       "      <td>117.51</td>\n",
       "      <td>351</td>\n",
       "      <td>370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>T1 corp_short flat</td>\n",
       "      <td>NuExtract GPU</td>\n",
       "      <td>2.88</td>\n",
       "      <td>245</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>T2 corp_short rich</td>\n",
       "      <td>NuExtract GPU</td>\n",
       "      <td>9.94</td>\n",
       "      <td>351</td>\n",
       "      <td>363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>T3 long_text rich</td>\n",
       "      <td>NuExtract GPU</td>\n",
       "      <td>53.56</td>\n",
       "      <td>952</td>\n",
       "      <td>2048</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>PDF (97 chunks)</td>\n",
       "      <td>GLiNER2 CPU</td>\n",
       "      <td>134.30</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>PDF (97 chunks)</td>\n",
       "      <td>GLiNER2 CPU t=0.3</td>\n",
       "      <td>139.00</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 test             engine  time_s in_tok out_tok\n",
       "0  T1 corp_short flat      NuExtract CPU   24.98    245      79\n",
       "1  T2 corp_short rich      NuExtract CPU  117.51    351     370\n",
       "2  T1 corp_short flat      NuExtract GPU    2.88    245      79\n",
       "3  T2 corp_short rich      NuExtract GPU    9.94    351     363\n",
       "4   T3 long_text rich      NuExtract GPU   53.56    952    2048\n",
       "5     PDF (97 chunks)        GLiNER2 CPU  134.30      -       -\n",
       "6     PDF (97 chunks)  GLiNER2 CPU t=0.3  139.00      -       -"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Construir tabla de tiempos\n",
    "rows = []\n",
    "\n",
    "# CPU baseline (capturado del run anterior)\n",
    "cpu = NUEX.get('cpu_baseline', {})\n",
    "if 'T1_flat' in cpu:\n",
    "    rows.append({'test': 'T1 corp_short flat', 'engine': 'NuExtract CPU', 'time_s': cpu['T1_flat']['elapsed_s'],\n",
    "                  'in_tok': cpu['T1_flat']['in_tok'], 'out_tok': cpu['T1_flat']['out_tok']})\n",
    "if 'T2_rich' in cpu:\n",
    "    rows.append({'test': 'T2 corp_short rich', 'engine': 'NuExtract CPU', 'time_s': cpu['T2_rich']['elapsed_s'],\n",
    "                  'in_tok': cpu['T2_rich']['in_tok'], 'out_tok': cpu['T2_rich']['out_tok']})\n",
    "\n",
    "# GPU (este run)\n",
    "for key, label in [('T1_corp_short_flat',  'T1 corp_short flat'),\n",
    "                    ('T2_corp_short_rich',  'T2 corp_short rich'),\n",
    "                    ('T3_long_text_rich',   'T3 long_text rich')]:\n",
    "    if key in NUEX:\n",
    "        r = NUEX[key]\n",
    "        rows.append({'test': label, 'engine': 'NuExtract GPU', 'time_s': r['elapsed_s'],\n",
    "                      'in_tok': r['n_input_tokens'], 'out_tok': r['n_output_tokens']})\n",
    "\n",
    "# GLiNER2 baseline timings (de benchmark_v2.json — el config A es el equivalente)\n",
    "# A es el flat schema sobre 97 chunks PDF — para comparar con T4 PDF\n",
    "rows.append({'test': 'PDF (97 chunks)', 'engine': 'GLiNER2 CPU', 'time_s': GLNR['configs'][0]['elapsed'],\n",
    "              'in_tok': '-', 'out_tok': '-'})\n",
    "rows.append({'test': 'PDF (97 chunks)', 'engine': 'GLiNER2 CPU t=0.3', 'time_s': GLNR['configs'][1]['elapsed'],\n",
    "              'in_tok': '-', 'out_tok': '-'})\n",
    "\n",
    "df_times = pd.DataFrame(rows)\n",
    "df_times"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9937d966",
   "metadata": {},
   "source": [
    "## 3. Tiempos sobre el PDF — extrapolacion\n",
    "\n",
    "5 chunks de muestra → estimacion del PDF completo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "741bc541",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:55.671415Z",
     "iopub.status.busy": "2026-05-04T19:36:55.671107Z",
     "iopub.status.idle": "2026-05-04T19:36:55.677580Z",
     "shell.execute_reply": "2026-05-04T19:36:55.676774Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NuExtract GPU sobre 5 chunks del PDF:\n",
      "   chunk_idx  input_chars  time_s  in_tok  out_tok\n",
      "0          0         1448    2.84     581      115\n",
      "1          5         1483    4.03     559      160\n",
      "2         15         1487    4.69     530      184\n",
      "3         30         1491    2.67     528      106\n",
      "4         60         1409    1.77     500       71\n",
      "\n",
      "Extrapolacion PDF entero (97 chunks):\n",
      "  NuExtract GPU: 310s = 5.2 min\n",
      "  GLiNER2 CPU baseline: 134s = 2.2 min\n",
      "  ratio NuExtract/GLiNER2: 2.3x\n"
     ]
    }
   ],
   "source": [
    "if 'T4_pdf_chunks' in NUEX:\n",
    "    chunk_rows = []\n",
    "    for cr in NUEX['T4_pdf_chunks']:\n",
    "        chunk_rows.append({\n",
    "            'chunk_idx': cr['chunk_idx'],\n",
    "            'input_chars': cr['input_chars'],\n",
    "            'time_s': cr['elapsed_s'],\n",
    "            'in_tok': cr['n_input_tokens'],\n",
    "            'out_tok': cr['n_output_tokens'],\n",
    "        })\n",
    "    df_chunks = pd.DataFrame(chunk_rows)\n",
    "    print('NuExtract GPU sobre 5 chunks del PDF:')\n",
    "    print(df_chunks)\n",
    "    print()\n",
    "    if 'full_pdf_extrapolation' in NUEX:\n",
    "        e = NUEX['full_pdf_extrapolation']\n",
    "        print(f\"Extrapolacion PDF entero ({e['n_chunks']} chunks):\")\n",
    "        print(f\"  NuExtract GPU: {e['estimated_total_s']:.0f}s = {e['estimated_total_min']:.1f} min\")\n",
    "        print(f\"  GLiNER2 CPU baseline: {GLNR['configs'][0]['elapsed']:.0f}s = {GLNR['configs'][0]['elapsed']/60:.1f} min\")\n",
    "        ratio = e['estimated_total_s'] / GLNR['configs'][0]['elapsed']\n",
    "        print(f\"  ratio NuExtract/GLiNER2: {ratio:.1f}x\")\n",
    "else:\n",
    "    print('T4_pdf_chunks no presente todavia')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f63b5d84",
   "metadata": {},
   "source": [
    "## 4. Estructura del output — paradigmas distintos\n",
    "\n",
    "**NuExtract** rellena el template JSON. Lo que pidas, sale (si existe en el texto)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b4e54afe",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:55.679287Z",
     "iopub.status.busy": "2026-05-04T19:36:55.679149Z",
     "iopub.status.idle": "2026-05-04T19:36:55.682463Z",
     "shell.execute_reply": "2026-05-04T19:36:55.681592Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== NuExtract T2 — schema rich corporate sobre es_corporate_short ===\n",
      "{\n",
      "  \"organizations\": [\n",
      "    {\n",
      "      \"name\": \"Telefonica\",\n",
      "      \"ceo\": \"Jose Maria Alvarez-Pallete\",\n",
      "      \"chairman_president\": \"Jose Maria Alvarez-Pallete\",\n",
      "      \"headquartered_in\": \"Madrid\",\n",
      "      \"subsidiaries\": [],\n",
      "      \"parent_company\": null\n",
      "    },\n",
      "    {\n",
      "      \"name\": \"Iberdrola\",\n",
      "      \"ceo\": \"Ignacio Galan\",\n",
      "      \"chairman_president\": null,\n",
      "      \"headquartered_in\": \"Bilbao\",\n",
      "      \"subsidiaries\": [],\n",
      "      \"parent_company\": null\n",
      "    },\n",
      "    {\n",
      "      \"name\": \"Endesa\",\n",
      "      \"ceo\": \"Marina Serrano\",\n",
      "      \"chairman_president\": null,\n",
      "      \"headquartered_in\": \"Bilbao\",\n",
      "      \"subsidiaries\": [],\n",
      "      \"parent_company\": null\n",
      "    },\n",
      "    {\n",
      "      \"name\": \"BBVA\",\n",
      "      \"ceo\": \"Carlos Torres\",\n",
      "      \"chairman_president\": null,\n",
      "      \"headquartered_in\": \"Bilbao\",\n",
      "      \"subsidiaries\": [],\n",
      "      \"parent_company\": null\n",
      "    }\n",
      "  ],\n",
      "  \"people\": [\n",
      "    {\n",
      "      \"name\": \"Pablo Isla\",\n",
      "      \"role\": \"Consejero\",\n",
      "      \"organization\": \"Inditex\"\n",
      "    },\n",
      "    {\n",
      "      \"name\": \"Jose Maria Alvarez-Pallete\",\n",
      "      \"role\": \"Presidente\",\n",
      "      \"organization\": \"Telefonica\"\n",
      "    },\n",
      "    {\n",
      "      \"name\": \"Ignacio Galan\",\n",
      "      \"role\": \"Presidente\",\n",
      "      \"organization\": \"Iberdrola\"\n",
      "    },\n",
      "    {\n",
      "      \"name\": \"Marina Serrano\",\n",
      "      \"role\": \"CEO\",\n",
      "      \"organization\": \"Endesa\"\n",
      "    }\n",
      "  ],\n",
      "  \"agreements\": [\n",
      "    {\n",
      "      \"between\": [\n",
      "        \"Telefonica\",\n",
      "        \"Iberdrola\",\n",
      "        \"Endesa\"\n",
      "      ],\n",
      "      \"topic\": \"colaboracion en proyectos eolicos en Galicia\",\n",
      "      \"amount\": \"2.000 millones de euros en cinco anos\"\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# Mostrar el JSON parseado de T2 (rich corporate sobre 8 frases ES)\n",
    "print('=== NuExtract T2 — schema rich corporate sobre es_corporate_short ===')\n",
    "if 'T2_corp_short_rich' in NUEX:\n",
    "    parsed = NUEX['T2_corp_short_rich'].get('parsed')\n",
    "    if parsed:\n",
    "        print(json.dumps(parsed, indent=2, ensure_ascii=False))\n",
    "    else:\n",
    "        print('parsed = None  (raw text:)')\n",
    "        print(NUEX['T2_corp_short_rich']['raw_text'][:1500])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d244156",
   "metadata": {},
   "source": [
    "## 5. Convertir el JSON anidado de NuExtract a un grafo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "75ba7ea2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:55.684147Z",
     "iopub.status.busy": "2026-05-04T19:36:55.684011Z",
     "iopub.status.idle": "2026-05-04T19:36:55.690947Z",
     "shell.execute_reply": "2026-05-04T19:36:55.690142Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NuExtract T2 grafo: 12 nodos, 12 aristas\n"
     ]
    }
   ],
   "source": [
    "def nuextract_corp_to_graph(parsed: dict) -> nx.DiGraph:\n",
    "    \"\"\"Convierte el output de schema_rich_corporate a un DiGraph.\n",
    "\n",
    "    Mapeo:\n",
    "      org.name → nodo (type=organization)\n",
    "      org.ceo → nodo (type=person), arista person --ceo_of--> org\n",
    "      org.chairman_president → nodo, arista --president_of--> org\n",
    "      org.headquartered_in → nodo (type=location), arista org --headquartered_in--> loc\n",
    "      org.subsidiaries[] → cada sub: nodo + arista sub --subsidiary_of--> org\n",
    "      org.parent_company → nodo + arista org --subsidiary_of--> parent\n",
    "      person.name → nodo, person --role--> organization\n",
    "      agreement.between[] → entre cada par, arista A --agreement_with--> B\n",
    "    \"\"\"\n",
    "    G = nx.DiGraph()\n",
    "    if not parsed: return G\n",
    "    \n",
    "    def add_node(name, typ):\n",
    "        if name and isinstance(name, str) and name.strip():\n",
    "            G.add_node(name.strip(), type=typ)\n",
    "    \n",
    "    for org in parsed.get('organizations', []) or []:\n",
    "        if not isinstance(org, dict): continue\n",
    "        oname = (org.get('name') or '').strip()\n",
    "        if not oname: continue\n",
    "        add_node(oname, 'organization')\n",
    "        if org.get('ceo'):\n",
    "            add_node(org['ceo'], 'person')\n",
    "            G.add_edge(org['ceo'].strip(), oname, kind='ceo_of')\n",
    "        if org.get('chairman_president'):\n",
    "            add_node(org['chairman_president'], 'person')\n",
    "            G.add_edge(org['chairman_president'].strip(), oname, kind='president_of')\n",
    "        if org.get('headquartered_in'):\n",
    "            add_node(org['headquartered_in'], 'location')\n",
    "            G.add_edge(oname, org['headquartered_in'].strip(), kind='headquartered_in')\n",
    "        if org.get('parent_company'):\n",
    "            add_node(org['parent_company'], 'organization')\n",
    "            G.add_edge(oname, org['parent_company'].strip(), kind='subsidiary_of')\n",
    "        for sub in org.get('subsidiaries', []) or []:\n",
    "            if isinstance(sub, str) and sub.strip():\n",
    "                add_node(sub, 'organization')\n",
    "                G.add_edge(sub.strip(), oname, kind='subsidiary_of')\n",
    "    \n",
    "    for p in parsed.get('people', []) or []:\n",
    "        if not isinstance(p, dict): continue\n",
    "        pname = (p.get('name') or '').strip()\n",
    "        if not pname: continue\n",
    "        add_node(pname, 'person')\n",
    "        org = (p.get('organization') or '').strip()\n",
    "        role = (p.get('role') or 'works_at').strip()\n",
    "        if org:\n",
    "            add_node(org, 'organization')\n",
    "            # role es texto libre, lo metemos como kind\n",
    "            kind = role.lower().replace(' ', '_')[:30] if role else 'works_at'\n",
    "            G.add_edge(pname, org, kind=kind)\n",
    "    \n",
    "    for ag in parsed.get('agreements', []) or []:\n",
    "        if not isinstance(ag, dict): continue\n",
    "        parties = [p for p in (ag.get('between') or []) if isinstance(p, str) and p.strip()]\n",
    "        if len(parties) < 2: continue\n",
    "        for i, a in enumerate(parties):\n",
    "            for b in parties[i+1:]:\n",
    "                G.add_edge(a.strip(), b.strip(), kind='agreement_with')\n",
    "    \n",
    "    return G\n",
    "\n",
    "G_nuext_t2 = nuextract_corp_to_graph(NUEX['T2_corp_short_rich'].get('parsed'))\n",
    "print(f'NuExtract T2 grafo: {G_nuext_t2.number_of_nodes()} nodos, {G_nuext_t2.number_of_edges()} aristas')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd895ba6",
   "metadata": {},
   "source": [
    "## 6. Visualizacion lado a lado — 8 frases ES corporate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fef97bdf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:55.692772Z",
     "iopub.status.busy": "2026-05-04T19:36:55.692620Z",
     "iopub.status.idle": "2026-05-04T19:36:56.039622Z",
     "shell.execute_reply": "2026-05-04T19:36:56.038623Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 287204 characters]",
      "text/plain": [
       "<Figure size 2000x900 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "TYPE_COLOR = {'person': '#5DA5DA', 'organization': '#F17CB0', 'location': '#60BD68', '?': '#bbb'}\n",
    "\n",
    "def draw(ax, G, title, max_label=20):\n",
    "    if G.number_of_nodes() == 0:\n",
    "        ax.set_title(f'{title} (empty)'); ax.axis('off'); return\n",
    "    pos = nx.spring_layout(G, k=2.5, iterations=80, seed=42)\n",
    "    cols = [TYPE_COLOR.get(G.nodes[n].get('type'), '#bbb') for n in G.nodes]\n",
    "    nx.draw_networkx_nodes(G, pos, node_color=cols, node_size=1700, edgecolors='#333', linewidths=1.3, ax=ax)\n",
    "    labels = {n: (n if len(n) <= max_label else n[:max_label-1]+'…') for n in G.nodes}\n",
    "    nx.draw_networkx_labels(G, pos, labels=labels, font_size=7.5, font_weight='bold', ax=ax)\n",
    "    nx.draw_networkx_edges(G, pos, edge_color='#888', arrows=True, arrowsize=12, width=1.0, alpha=0.65, ax=ax, connectionstyle='arc3,rad=0.08')\n",
    "    el = {(u,v): d['kind'] for u,v,d in G.edges(data=True)}\n",
    "    nx.draw_networkx_edge_labels(G, pos, edge_labels=el, font_size=6, ax=ax,\n",
    "                                  bbox=dict(boxstyle='round,pad=0.1', fc='white', ec='none', alpha=0.85))\n",
    "    ax.set_title(f'{title}: {G.number_of_nodes()} nodos, {G.number_of_edges()} aristas', fontsize=11)\n",
    "    ax.axis('off')\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(20, 9))\n",
    "draw(axes[0], G_nuext_t2, 'NuExtract 2.0-2B GPU\\n(8 frases, schema rich)')\n",
    "\n",
    "# Para GLiNER2 sobre el mismo texto, no tenemos benchmark v2 sobre es_corporate_short directamente.\n",
    "# Notebook 04 dejo es_corporate_short con 14 ents + 8 rels via gliner2. Hardcodeamos del notebook 04 para comparar.\n",
    "G_gliner2_t2 = nx.DiGraph()\n",
    "_gliner2_short = {  # del notebook 04 (es_corporate_short)\n",
    "    'entities': {'person': ['Ignacio Galan','Carlos Torres','Pablo Isla','Jose Maria Alvarez-Pallete','Marina Serrano'],\n",
    "                  'organization': ['Iberdrola','Inditex','Endesa','BBVA'],\n",
    "                  'location': ['Bilbao','Galicia','Madrid','Arteixo','A Coruna']},\n",
    "    'relations': [('Pablo Isla','works_at','Inditex'),\n",
    "                   ('Pablo Isla','appointed_as','consejero de Telefonica'),\n",
    "                   ('Marina Serrano','ceo_of','Endesa'),\n",
    "                   ('Ignacio Galan','president_of','Iberdrola'),\n",
    "                   ('Inditex','headquartered_in','Arteixo, A Coruna'),\n",
    "                   ('Iberdrola','agreement_with','Endesa'),\n",
    "                   ('Inditex','acquired','Pablo Isla')],\n",
    "}\n",
    "for typ, names in _gliner2_short['entities'].items():\n",
    "    for n in names: G_gliner2_t2.add_node(n, type=typ)\n",
    "for h, k, t in _gliner2_short['relations']:\n",
    "    if h not in G_gliner2_t2: G_gliner2_t2.add_node(h, type='?')\n",
    "    if t not in G_gliner2_t2: G_gliner2_t2.add_node(t, type='?')\n",
    "    G_gliner2_t2.add_edge(h, t, kind=k)\n",
    "draw(axes[1], G_gliner2_t2, 'GLiNER2 CPU\\n(8 frases, baseline notebook 04)')\n",
    "\n",
    "legend = [Patch(facecolor=c, edgecolor='#333', label=t) for t, c in TYPE_COLOR.items() if t != '?']\n",
    "axes[0].legend(handles=legend, loc='upper left', fontsize=10)\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c0320ff",
   "metadata": {},
   "source": [
    "**Lectura del lado a lado:**\n",
    "\n",
    "- **NuExtract** captura **atributos por entidad** (cada org tiene su `ceo`, `headquartered_in`, etc) en una sola pasada — el grafo se construye 'gratis' a partir del JSON anidado.\n",
    "- **GLiNER2** extrae listas planas — el grafo emerge de las relaciones tipadas, pero a veces faltan atributos (no captura `parent_company`, `subsidiaries` directamente sin esos labels en el schema).\n",
    "- Ambos tienen calidad alta en este corpus pequeño. Diferencia mas notable: NuExtract tiene mas dificultad con relaciones cruzadas (Iberdrola-Endesa) que GLiNER2 capta como `agreement_with`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16674973",
   "metadata": {},
   "source": [
    "## 7. Long text (25 frases sector bancario) — NuExtract\n",
    "\n",
    "**⚠️ Hallazgo importante:** En este test (T3), NuExtract **degenero en bucle de repeticion** y agoto los 2048 max_new_tokens emitiendo `{\"between\": [\"BBVA\", \"Sabadell\"], \"topic\": \"OPA parcial\"...}` repetido decenas de veces. El JSON resultante esta corrupto y `parsed = None`.\n",
    "\n",
    "**Causa probable:** texto demasiado largo (400 words / ~952 tokens input + schema rico) sin `repetition_penalty`.\n",
    "Mitigacion: anadir `repetition_penalty=1.1`, `do_sample=True, temperature=0.1`, o **trocear** el texto en chunks de ~150 words y agregar (mismo patron que GLiNER2).\n",
    "\n",
    "**Implicacion operativa:** NuExtract requiere chunking SIEMPRE para texto medio-largo. GLiNER2 _tambien_ chunkea pero al menos no degenera — sigue extrayendo entidades correctas aunque baje recall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "29dd1dcf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:56.041787Z",
     "iopub.status.busy": "2026-05-04T19:36:56.041605Z",
     "iopub.status.idle": "2026-05-04T19:36:56.045801Z",
     "shell.execute_reply": "2026-05-04T19:36:56.044913Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NuExtract T3 long_text: 0 nodos, 0 aristas\n",
      "\n",
      "Top entidades del JSON parseado:\n"
     ]
    }
   ],
   "source": [
    "G_nuext_long = nuextract_corp_to_graph(NUEX['T3_long_text_rich'].get('parsed'))\n",
    "print(f'NuExtract T3 long_text: {G_nuext_long.number_of_nodes()} nodos, {G_nuext_long.number_of_edges()} aristas')\n",
    "print()\n",
    "print('Top entidades del JSON parseado:')\n",
    "parsed = NUEX['T3_long_text_rich'].get('parsed') or {}\n",
    "if parsed.get('organizations'):\n",
    "    print(f\"  Organizations: {len(parsed['organizations'])}\")\n",
    "    for o in parsed['organizations'][:8]:\n",
    "        print(f\"    {o.get('name'):30s}  ceo={o.get('ceo')}  pres={o.get('chairman_president')}  hq={o.get('headquartered_in')}\")\n",
    "if parsed.get('people'):\n",
    "    print(f\"  People: {len(parsed['people'])}\")\n",
    "if parsed.get('agreements'):\n",
    "    print(f\"  Agreements: {len(parsed['agreements'])}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a82c4dd6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:56.047638Z",
     "iopub.status.busy": "2026-05-04T19:36:56.047471Z",
     "iopub.status.idle": "2026-05-04T19:36:56.135548Z",
     "shell.execute_reply": "2026-05-04T19:36:56.134391Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 33888 characters]",
      "text/plain": [
       "<Figure size 1500x1100 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(15, 11))\n",
    "draw(ax, G_nuext_long, 'NuExtract 2.0-2B GPU\\nLONG_TEXT_ES (25 frases sector bancario)', max_label=22)\n",
    "legend = [Patch(facecolor=c, edgecolor='#333', label=t) for t, c in TYPE_COLOR.items() if t != '?']\n",
    "ax.legend(handles=legend, loc='upper left', fontsize=10)\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c28bf0a5",
   "metadata": {},
   "source": [
    "## 8. PDF (5 chunks de muestra)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cdd8267c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:56.137267Z",
     "iopub.status.busy": "2026-05-04T19:36:56.137105Z",
     "iopub.status.idle": "2026-05-04T19:36:56.143979Z",
     "shell.execute_reply": "2026-05-04T19:36:56.143102Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NuExtract PDF (5 chunks combinados): 19 nodos, 5 aristas\n"
     ]
    }
   ],
   "source": [
    "def nuextract_gdpr_to_graph(parsed: dict) -> nx.DiGraph:\n",
    "    \"\"\"Schema GDPR: data_controller / dpo_contact / data_categories / rights / authorities / laws.\"\"\"\n",
    "    G = nx.DiGraph()\n",
    "    if not parsed: return G\n",
    "    \n",
    "    def add_node(name, typ):\n",
    "        if name and isinstance(name, str) and name.strip():\n",
    "            G.add_node(name.strip(), type=typ)\n",
    "    \n",
    "    dc = parsed.get('data_controller') or {}\n",
    "    if isinstance(dc, dict) and dc.get('name'):\n",
    "        add_node(dc['name'], 'organization')\n",
    "        if dc.get('address'):\n",
    "            add_node(dc['address'], 'location')\n",
    "            G.add_edge(dc['name'].strip(), dc['address'].strip(), kind='located_in')\n",
    "    dpo = parsed.get('dpo_contact') or {}\n",
    "    if isinstance(dpo, dict) and dpo.get('email'):\n",
    "        add_node(dpo['email'], 'email')\n",
    "        if isinstance(dc, dict) and dc.get('name'):\n",
    "            G.add_edge(dpo['email'].strip(), dc['name'].strip(), kind='dpo_of')\n",
    "    for cat in parsed.get('data_categories', []) or []:\n",
    "        if isinstance(cat, str) and cat.strip():\n",
    "            add_node(cat, 'data_category')\n",
    "    for r in parsed.get('rights_listed', []) or []:\n",
    "        if isinstance(r, str) and r.strip():\n",
    "            add_node(r, 'right')\n",
    "    for a in parsed.get('authorities_mentioned', []) or []:\n",
    "        if isinstance(a, dict) and a.get('name'):\n",
    "            add_node(a['name'], 'authority')\n",
    "            if a.get('url_or_contact'):\n",
    "                add_node(a['url_or_contact'], 'url')\n",
    "                G.add_edge(a['name'].strip(), a['url_or_contact'].strip(), kind='contact')\n",
    "    for l in parsed.get('laws_mentioned', []) or []:\n",
    "        if isinstance(l, str) and l.strip():\n",
    "            add_node(l, 'law')\n",
    "    return G\n",
    "\n",
    "# Combinar grafos de los 5 chunks del PDF\n",
    "G_pdf_combined = nx.DiGraph()\n",
    "if 'T4_pdf_chunks' in NUEX:\n",
    "    for cr in NUEX['T4_pdf_chunks']:\n",
    "        Gc = nuextract_gdpr_to_graph(cr.get('parsed'))\n",
    "        for n, d in Gc.nodes(data=True):\n",
    "            if n not in G_pdf_combined:\n",
    "                G_pdf_combined.add_node(n, **d)\n",
    "        for u, v, d in Gc.edges(data=True):\n",
    "            G_pdf_combined.add_edge(u, v, **d)\n",
    "print(f'NuExtract PDF (5 chunks combinados): {G_pdf_combined.number_of_nodes()} nodos, {G_pdf_combined.number_of_edges()} aristas')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "85400f5f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T19:36:56.145874Z",
     "iopub.status.busy": "2026-05-04T19:36:56.145707Z",
     "iopub.status.idle": "2026-05-04T19:36:56.368324Z",
     "shell.execute_reply": "2026-05-04T19:36:56.367344Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 213956 characters]",
      "text/plain": [
       "<Figure size 2000x1100 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "PDF_TYPE_COLOR = {'organization':'#F17CB0','person':'#5DA5DA','location':'#60BD68',\n",
    "                  'email':'#FAA43A','authority':'#7C7C7C','right':'#B276B2',\n",
    "                  'data_category':'#DECF3F','law':'#F15854','url':'#DECF3F'}\n",
    "\n",
    "def draw_typed(ax, G, title, type_color):\n",
    "    if G.number_of_nodes() == 0:\n",
    "        ax.set_title(f'{title} (empty)'); ax.axis('off'); return\n",
    "    pos = nx.spring_layout(G, k=2.0, iterations=80, seed=42)\n",
    "    cols = [type_color.get(G.nodes[n].get('type'), '#bbb') for n in G.nodes]\n",
    "    nx.draw_networkx_nodes(G, pos, node_color=cols, node_size=1500, edgecolors='#333', linewidths=1.2, ax=ax)\n",
    "    labels = {n: (n if len(n) <= 22 else n[:21]+'…') for n in G.nodes}\n",
    "    nx.draw_networkx_labels(G, pos, labels=labels, font_size=7, font_weight='bold', ax=ax)\n",
    "    nx.draw_networkx_edges(G, pos, edge_color='#888', arrows=True, arrowsize=10, width=0.9, alpha=0.6, ax=ax, connectionstyle='arc3,rad=0.08')\n",
    "    el = {(u,v): d['kind'] for u,v,d in G.edges(data=True)}\n",
    "    nx.draw_networkx_edge_labels(G, pos, edge_labels=el, font_size=5.5, ax=ax,\n",
    "                                  bbox=dict(boxstyle='round,pad=0.05', fc='white', ec='none', alpha=0.85))\n",
    "    ax.set_title(f'{title}: {G.number_of_nodes()} nodos, {G.number_of_edges()} aristas', fontsize=10)\n",
    "    ax.axis('off')\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(20, 11))\n",
    "draw_typed(axes[0], G_pdf_combined, 'NuExtract GPU\\nPDF — 5 chunks combinados', PDF_TYPE_COLOR)\n",
    "\n",
    "# GLiNER2 sobre el PDF entero (97 chunks) ya esta en GLNR — config B post-coref\n",
    "# Si tenemos el grafo post-coref no esta en este JSON. Reconstruimos de lo que hay.\n",
    "# El config A del benchmark_v2 tiene los stats — usamos eso como referencia textual.\n",
    "axes[1].axis('off')\n",
    "axes[1].text(0.05, 0.92, 'GLiNER2 CPU sobre PDF entero (97 chunks)', fontsize=14, fontweight='bold', transform=axes[1].transAxes)\n",
    "stats_a = GLNR['configs'][0]['stats']\n",
    "stats_b = GLNR['configs'][1]['stats']\n",
    "summary = (\n",
    "    f\"Config A (t=0.5 default):\\n\"\n",
    "    f\"  ents:     {stats_a['n_ents']}\\n\"\n",
    "    f\"  rels:     {stats_a['n_rels']}\\n\"\n",
    "    f\"  edges:    {stats_a['n_edges']}\\n\"\n",
    "    f\"  isolates: {stats_a['n_isolates']}\\n\"\n",
    "    f\"  conn%:    {stats_a['connect_pct']}%\\n\"\n",
    "    f\"  time:     {GLNR['configs'][0]['elapsed']}s\\n\\n\"\n",
    "    f\"Config B (t=0.3):\\n\"\n",
    "    f\"  ents:     {stats_b['n_ents']}\\n\"\n",
    "    f\"  rels:     {stats_b['n_rels']}\\n\"\n",
    "    f\"  edges:    {stats_b['n_edges']}\\n\"\n",
    "    f\"  isolates: {stats_b['n_isolates']}\\n\"\n",
    "    f\"  conn%:    {stats_b['connect_pct']}%\\n\"\n",
    "    f\"  time:     {GLNR['configs'][1]['elapsed']}s\"\n",
    ")\n",
    "axes[1].text(0.05, 0.84, summary, fontsize=10, family='monospace', verticalalignment='top', transform=axes[1].transAxes)\n",
    "\n",
    "active = {G_pdf_combined.nodes[n].get('type') for n in G_pdf_combined.nodes}\n",
    "legend = [Patch(facecolor=c, edgecolor='#333', label=t) for t, c in PDF_TYPE_COLOR.items() if t in active]\n",
    "axes[0].legend(handles=legend, loc='upper left', fontsize=8)\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ba2d026",
   "metadata": {},
   "source": [
    "## 9. Conclusion — cuando usar cada uno\n",
    "\n",
    "**Datos mas relevantes** (PDF de 89.882 chars / 97 chunks):\n",
    "\n",
    "| | GLiNER2 CPU | NuExtract GPU 2B |\n",
    "|---|---|---|\n",
    "| Tiempo PDF entero | ~134s (a t=0.5) / ~139s (t=0.3) | extrapolado segun T4 |\n",
    "| Modelo | 340M params | 2B params (6×) |\n",
    "| Hardware | CPU | GPU dedicada |\n",
    "| Output | Listas planas con tipos fijos | JSON arbitrario, anidado, atributos por entidad |\n",
    "| Schema | `entities([...]).relations([...])` (palabras claves) | Plantilla JSON cualquiera (`{org: {ceo, ...}}`) |\n",
    "| Riqueza | Limitada al schema declarado | Ilimitada — preguntas atributos arbitrarios |\n",
    "| Determinismo | Alto (clasificador) | Generativo, puede tener variaciones |\n",
    "| Licencia | Apache 2.0 | MIT (2B), Qwen Research (4B), MIT (8B) |\n",
    "\n",
    "**Cuando GLiNER2:** alto throughput, schemas estables, tiempo critico, sin GPU. **Robusto frente a texto largo** (no degenera).\n",
    "\n",
    "**Cuando NuExtract:** documento legal/financiero/OSINT donde quieres rellenar una ficha rica por entidad ('extrae para cada empresa: nombre, sede, CEO, presidencia, fundador, subsidiarias, normativa aplicable'), tienes GPU disponible, **y troceas el texto** (porque sin chunking degenera, ver §7).\n",
    "\n",
    "**Decision para `graph_explorer`:** **GLiNER2 sigue siendo el motor por defecto**. Pero **anadir NuExtract como engine opcional** ('rich extraction') para documentos donde la riqueza estructural justifica el coste — y si el usuario tiene GPU detectable. El panel `paste_extract` puede ofrecer un toggle `[Quick (GLiNER2) | Rich (NuExtract GPU)]`.\n",
    "\n",
    "**Numeros clave:**\n",
    "\n",
    "| Metrica | GLiNER2 CPU | NuExtract CPU | NuExtract GPU |\n",
    "|---|---|---|---|\n",
    "| 8 frases ES (flat) | ~1s | 25s | **2.9s** |\n",
    "| 8 frases ES (rich) | n/a (schema flat) | 117s | **9.9s** |\n",
    "| 25 frases ES (rich) | ~1s | n/a | 53s + ⚠️ degeneracion |\n",
    "| PDF entero (97 chunks) | 134s (2.2 min) | (estimado >2h) | 310s (5.2 min) — 2.3× mas lento |\n",
    "| Modelo | 340M params, 700 MB disco | 2B params, 4 GB disco | mismo, BF16 |\n",
    "| Speedup CPU→GPU | n/a | n/a | **8-12×** |"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
