{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e9f04104",
   "metadata": {},
   "source": [
    "# Texto largo + PDF E2E con GLiNER2\n",
    "\n",
    "Demostracion en dos partes del flujo elegido (decision del notebook 04):\n",
    "\n",
    "**Parte A** — Texto largo en castellano (25 frases sobre sector bancario espanol) → GLiNER2 → grafo.\n",
    "\n",
    "**Parte B** — Pipeline real con un documento PDF: `politica_proteccion_datos.pdf` (BBVA, 20 paginas, copiado al vault). El flujo es:\n",
    "\n",
    "1. `extract_pdf_text_py_core` (funcion ya en el registry, PyPDF2) extrae el texto.\n",
    "2. Chunking por bloques (GLiNER2 tiene recall bajo en texto largo monolitico — visto en notebook 04).\n",
    "3. GLiNER2 sobre cada bloque + agregacion deduplicada.\n",
    "4. Grafo final + tabla de entidades top.\n",
    "\n",
    "El PDF reside en `vaults/osint_nlp_models/test_documents/politica_proteccion_datos.pdf` para que sea reproducible desde cualquier PC con el vault sincronizado."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ec5efe6",
   "metadata": {},
   "source": [
    "## 0. Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1a589ca6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:35.897238Z",
     "iopub.status.busy": "2026-05-04T14:12:35.897101Z",
     "iopub.status.idle": "2026-05-04T14:12:40.556988Z",
     "shell.execute_reply": "2026-05-04T14:12:40.556047Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[0;93m2026-05-04 16:12:37.246174100 [W:onnxruntime:Default, device_discovery.cc:283 GetGpuDevices] Failed to detect devices under \"/sys/class/drm/card0\": device_discovery.cc:93 ReadFileContents Failed to open file: \"/sys/class/drm/card0/device/vendor\"\u001b[m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PDF exists: True, size: 406,230 bytes\n"
     ]
    }
   ],
   "source": [
    "import os, sys, json, time, re, warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "os.environ.setdefault('HF_HUB_DISABLE_PROGRESS_BARS', '1')\n",
    "from pathlib import Path\n",
    "from collections import Counter\n",
    "\n",
    "_pf = '/home/lucas/fn_registry/python/functions'\n",
    "sys.path = [p for p in sys.path if not p.startswith(_pf + '/')]\n",
    "if _pf not in sys.path: sys.path.insert(0, _pf)\n",
    "\n",
    "import pandas as pd\n",
    "import networkx as nx\n",
    "import matplotlib.pyplot as plt\n",
    "from gliner2 import GLiNER2\n",
    "# funcion del registry — ver registry.db para signature\n",
    "from core.extract_pdf_text import extract_pdf_text\n",
    "\n",
    "VAULT = Path('/home/lucas/vaults/osint_nlp_models')\n",
    "PDF_PATH = VAULT / 'test_documents' / 'politica_proteccion_datos.pdf'\n",
    "print(f'PDF exists: {PDF_PATH.exists()}, size: {PDF_PATH.stat().st_size:,} bytes')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc544c8c",
   "metadata": {},
   "source": [
    "## 1. Cargar GLiNER2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "73c0a1a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:40.559098Z",
     "iopub.status.busy": "2026-05-04T14:12:40.558762Z",
     "iopub.status.idle": "2026-05-04T14:12:45.691710Z",
     "shell.execute_reply": "2026-05-04T14:12:45.690843Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are using a model of type extractor to instantiate a model of type . This is not supported for all configurations of models and can yield errors.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "============================================================\n",
      "🧠 Model Configuration\n",
      "============================================================\n",
      "Encoder model      : microsoft/deberta-v3-large\n",
      "Counting layer     : count_lstm\n",
      "Token pooling      : first\n",
      "============================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GLiNER2 ready in 5.1s\n"
     ]
    }
   ],
   "source": [
    "t0 = time.time()\n",
    "model = GLiNER2.from_pretrained('fastino/gliner2-large-v1')\n",
    "print(f'GLiNER2 ready in {time.time()-t0:.1f}s')\n",
    "\n",
    "ENTITY_LABELS = ['person', 'organization', 'location']\n",
    "RELATION_LABELS = [\n",
    "    'works_at', 'located_in', 'appointed_as', 'ceo_of', 'president_of',\n",
    "    'headquartered_in', 'subsidiary_of', 'parent_company', 'founded_by',\n",
    "    'agreement_with', 'acquired', 'succeeded_by', 'governed_by',\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bdcc8d8",
   "metadata": {},
   "source": [
    "# PARTE A — Texto largo\n",
    "\n",
    "## A.1 El texto"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b1b91afe",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:45.693448Z",
     "iopub.status.busy": "2026-05-04T14:12:45.693282Z",
     "iopub.status.idle": "2026-05-04T14:12:45.697346Z",
     "shell.execute_reply": "2026-05-04T14:12:45.696460Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2722 chars / 429 words / 25 sentences\n",
      "\n",
      "BBVA, presidido por Carlos Torres, completo en 2024 la integracion operativa de Banco Sabadell tras la fusion. Onur Genc, consejero delegado del banco desde 2018, lidero el proceso desde la sede central en Bilbao. El banco mantiene oficinas en Plaza San Nicolas 4 y opera en mas de 25 paises. Banco Santander, dirigido por Ana Botin, sigue siendo el primer banco espanol por capitalizacion bursatil. Hector Grisi asumio el cargo de CEO global de Santander en enero de 2023, reemplazando a Jose Antonio Alvarez. CaixaBank, presidida por Jose Ignacio Goirigolzarri y con sede en Valencia desde 2017, co...\n"
     ]
    }
   ],
   "source": [
    "TEXTO = 'BBVA, presidido por Carlos Torres, completo en 2024 la integracion operativa de Banco Sabadell tras la fusion. Onur Genc, consejero delegado del banco desde 2018, lidero el proceso desde la sede central en Bilbao. El banco mantiene oficinas en Plaza San Nicolas 4 y opera en mas de 25 paises. Banco Santander, dirigido por Ana Botin, sigue siendo el primer banco espanol por capitalizacion bursatil. Hector Grisi asumio el cargo de CEO global de Santander en enero de 2023, reemplazando a Jose Antonio Alvarez. CaixaBank, presidida por Jose Ignacio Goirigolzarri y con sede en Valencia desde 2017, completo la fusion con Bankia. Gonzalo Gortazar es el consejero delegado de CaixaBank y reporta al consejo formado en parte por La Caixa. El Banco de Espana, gobernado por Pablo Hernandez de Cos hasta 2024 y por Margarita Delgado en 2025, supervisa el sector. Luis de Guindos, vicepresidente del Banco Central Europeo, fue ministro de Economia en el gobierno de Mariano Rajoy. La Comision Nacional del Mercado de Valores, presidida por Rodrigo Buenaventura, regula los mercados financieros. BBVA anuncio en mayo de 2024 una OPA hostil sobre Banco Sabadell que el consejo del banco rechazo inicialmente. Cesar Gonzalez-Bueno, CEO de Sabadell, defendio la independencia del banco junto con su presidente Josep Oliu. Repsol, presidida por Antonio Brufau y con CEO Josu Jon Imaz, vendio su filial mexicana a Macquarie. Iberdrola, liderada por Ignacio Galan, opera Avangrid en EEUU y firmo un acuerdo PPA con Amazon. Andy Jassy, CEO de Amazon desde Seattle, agradecio el contrato a Iberdrola en una nota publica. Endesa, filial de la italiana Enel, tiene como CEO a Marina Serrano y opera en Espana, Portugal y Marruecos. Ferrovial, presidida por Rafael del Pino, traslado su sede social a Holanda en 2022 generando polemica politica. ACS, presidida por Florentino Perez, sigue siendo lider mundial en concesiones de infraestructura. Inditex, fundada por Amancio Ortega y presidida por Marta Ortega desde 2022, tiene su sede en Arteixo, A Coruna. Pablo Isla, expresidente de Inditex y actual consejero de Telefonica, se incorporo al consejo en 2024. Telefonica, presidida por Jose Maria Alvarez-Pallete, sufrio la entrada del estado en su capital con SEPI. Saudi Telecom Company adquirio un 9.9% de Telefonica en 2023, lo que motivo la respuesta del gobierno espanol. Cristina Aldamiz-Echevarria fue nombrada directora de Recursos Humanos del Grupo Mapfre, dirigido por Antonio Huertas. Naturgy, presidida por Francisco Reynes, recibio una OPA parcial del fondo emirati IFM en 2021 que se cancelo. Indra, con Marc Murtra como presidente, se ha posicionado como contratista clave de Defensa para el ministerio de Margarita Robles.'\n",
    "n_sentences = len(re.split(r'(?<=[\\.!?])\\s+', TEXTO))\n",
    "print(f'{len(TEXTO)} chars / {len(TEXTO.split())} words / {n_sentences} sentences')\n",
    "print()\n",
    "print(TEXTO[:600] + '...')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d06e2a51",
   "metadata": {},
   "source": [
    "## A.2 GLiNER2 — extraccion en una pasada"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a54bcff8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:45.698891Z",
     "iopub.status.busy": "2026-05-04T14:12:45.698745Z",
     "iopub.status.idle": "2026-05-04T14:12:49.904340Z",
     "shell.execute_reply": "2026-05-04T14:12:49.903085Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "61 entidades, 7 relaciones en 4.20s\n"
     ]
    }
   ],
   "source": [
    "schema = (model.create_schema()\n",
    "    .entities(ENTITY_LABELS)\n",
    "    .relations(RELATION_LABELS))\n",
    "\n",
    "t0 = time.time()\n",
    "result = model.extract(TEXTO, schema=schema)\n",
    "elapsed = time.time() - t0\n",
    "n_ents = sum(len(v) for v in result['entities'].values())\n",
    "n_rels = sum(len(v) for v in result['relation_extraction'].values())\n",
    "print(f'{n_ents} entidades, {n_rels} relaciones en {elapsed:.2f}s')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14b68a77",
   "metadata": {},
   "source": [
    "## A.3 Tabla de entidades"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f822141f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:49.906604Z",
     "iopub.status.busy": "2026-05-04T14:12:49.906331Z",
     "iopub.status.idle": "2026-05-04T14:12:49.920489Z",
     "shell.execute_reply": "2026-05-04T14:12:49.919704Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>type</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>location</td>\n",
       "      <td>Arteixo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>location</td>\n",
       "      <td>Bilbao</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>location</td>\n",
       "      <td>Espana</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>location</td>\n",
       "      <td>Holanda</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>location</td>\n",
       "      <td>Marruecos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>person</td>\n",
       "      <td>Onur Genc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>person</td>\n",
       "      <td>Pablo Hernandez de Cos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>person</td>\n",
       "      <td>Pablo Isla</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>person</td>\n",
       "      <td>Rafael del Pino</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>person</td>\n",
       "      <td>Rodrigo Buenaventura</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        type                    name\n",
       "0   location                 Arteixo\n",
       "1   location                  Bilbao\n",
       "2   location                  Espana\n",
       "3   location                 Holanda\n",
       "4   location               Marruecos\n",
       "..       ...                     ...\n",
       "56    person               Onur Genc\n",
       "57    person  Pablo Hernandez de Cos\n",
       "58    person              Pablo Isla\n",
       "59    person         Rafael del Pino\n",
       "60    person    Rodrigo Buenaventura\n",
       "\n",
       "[61 rows x 2 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rows = []\n",
    "for typ, names in result['entities'].items():\n",
    "    for n in names:\n",
    "        rows.append({'type': typ, 'name': n})\n",
    "df_ents = pd.DataFrame(rows).drop_duplicates().sort_values(['type', 'name']).reset_index(drop=True)\n",
    "df_ents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b1ea5e7",
   "metadata": {},
   "source": [
    "## A.4 Tabla de relaciones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b7c4c04d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:49.922207Z",
     "iopub.status.busy": "2026-05-04T14:12:49.922040Z",
     "iopub.status.idle": "2026-05-04T14:12:49.930254Z",
     "shell.execute_reply": "2026-05-04T14:12:49.929383Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>from</th>\n",
       "      <th>kind</th>\n",
       "      <th>to</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Onur Genc</td>\n",
       "      <td>works_at</td>\n",
       "      <td>BBVA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cristina Aldamiz-Echevarria</td>\n",
       "      <td>appointed_as</td>\n",
       "      <td>directora de Recursos Humanos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Endesa</td>\n",
       "      <td>subsidiary_of</td>\n",
       "      <td>Enel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Inditex</td>\n",
       "      <td>founded_by</td>\n",
       "      <td>Amancio Ortega</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Iberdrola</td>\n",
       "      <td>agreement_with</td>\n",
       "      <td>Amazon</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Hector Grisi</td>\n",
       "      <td>succeeded_by</td>\n",
       "      <td>Hector Grisi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>BBVA</td>\n",
       "      <td>governed_by</td>\n",
       "      <td>Carlos Torres</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          from            kind                             to\n",
       "0                    Onur Genc        works_at                           BBVA\n",
       "1  Cristina Aldamiz-Echevarria    appointed_as  directora de Recursos Humanos\n",
       "2                       Endesa   subsidiary_of                           Enel\n",
       "3                      Inditex      founded_by                 Amancio Ortega\n",
       "4                    Iberdrola  agreement_with                         Amazon\n",
       "5                 Hector Grisi    succeeded_by                   Hector Grisi\n",
       "6                         BBVA     governed_by                  Carlos Torres"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rows = []\n",
    "for rt, pairs in result['relation_extraction'].items():\n",
    "    for h, t in pairs:\n",
    "        rows.append({'from': h, 'kind': rt, 'to': t})\n",
    "df_rels = pd.DataFrame(rows).drop_duplicates().reset_index(drop=True)\n",
    "df_rels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c232fc7e",
   "metadata": {},
   "source": [
    "## A.5 Grafo del texto largo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "32fbbf21",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:49.932055Z",
     "iopub.status.busy": "2026-05-04T14:12:49.931833Z",
     "iopub.status.idle": "2026-05-04T14:12:50.175426Z",
     "shell.execute_reply": "2026-05-04T14:12:50.174380Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 389436 characters]",
      "text/plain": [
       "<Figure size 1500x1100 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "TYPE_COLOR = {'person': '#5DA5DA', 'organization': '#F17CB0', 'location': '#60BD68'}\n",
    "\n",
    "def build_graph_from_extract(extract_result):\n",
    "    G = nx.DiGraph()\n",
    "    for typ, names in extract_result['entities'].items():\n",
    "        for n in names:\n",
    "            G.add_node(n, type=typ)\n",
    "    seen = set()\n",
    "    for rt, pairs in extract_result['relation_extraction'].items():\n",
    "        for h, t in pairs:\n",
    "            if (h, t, rt) in seen: continue\n",
    "            seen.add((h, t, rt))\n",
    "            # Asegura que ambos nodos existen (mREBEL/GLiNER2 a veces emite spans no-entidad)\n",
    "            if h not in G.nodes: G.add_node(h, type='?')\n",
    "            if t not in G.nodes: G.add_node(t, type='?')\n",
    "            G.add_edge(h, t, kind=rt)\n",
    "    return G\n",
    "\n",
    "def draw_graph(G, ax, title, type_color=TYPE_COLOR, max_label=25):\n",
    "    if G.number_of_nodes() == 0:\n",
    "        ax.set_title(f'{title} (empty)'); ax.axis('off'); return\n",
    "    pos = nx.spring_layout(G, k=2.5, iterations=100, seed=42)\n",
    "    cols = [type_color.get(G.nodes[n].get('type'), '#bbb') for n in G.nodes]\n",
    "    nx.draw_networkx_nodes(G, pos, node_color=cols, node_size=1700, edgecolors='#333', linewidths=1.3, ax=ax)\n",
    "    labels = {n: (n if len(n) <= max_label else n[:max_label-1]+'…') for n in G.nodes}\n",
    "    nx.draw_networkx_labels(G, pos, labels=labels, font_size=7.5, font_weight='bold', ax=ax)\n",
    "    nx.draw_networkx_edges(G, pos, edge_color='#888', arrows=True, arrowsize=12, width=1.0, alpha=0.6, ax=ax, connectionstyle='arc3,rad=0.08')\n",
    "    el = {(u,v): d['kind'] for u,v,d in G.edges(data=True)}\n",
    "    nx.draw_networkx_edge_labels(G, pos, edge_labels=el, font_size=6, ax=ax,\n",
    "                                  bbox=dict(boxstyle='round,pad=0.1', fc='white', ec='none', alpha=0.85))\n",
    "    ax.set_title(f'{title}: {G.number_of_nodes()} ents, {G.number_of_edges()} rels', fontsize=11)\n",
    "    ax.axis('off')\n",
    "\n",
    "G_text = build_graph_from_extract(result)\n",
    "fig, ax = plt.subplots(figsize=(15, 11))\n",
    "draw_graph(G_text, ax, 'Texto largo (25 frases sector bancario ES)')\n",
    "from matplotlib.patches import Patch\n",
    "legend = [Patch(facecolor=c, edgecolor='#333', label=t) for t, c in TYPE_COLOR.items()]\n",
    "ax.legend(handles=legend, loc='upper left', fontsize=10)\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ad811e4",
   "metadata": {},
   "source": [
    "# PARTE B — Pipeline real con PDF\n",
    "\n",
    "## B.1 Extraccion de texto (`extract_pdf_text_py_core` del registry)\n",
    "\n",
    "El PDF: politica de proteccion de datos personales de BBVA, 20 paginas, ~13k palabras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "856f89a8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:50.177873Z",
     "iopub.status.busy": "2026-05-04T14:12:50.177704Z",
     "iopub.status.idle": "2026-05-04T14:12:51.025141Z",
     "shell.execute_reply": "2026-05-04T14:12:51.024182Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "extract_pdf_text en 0.84s\n",
      "chars: 89,882  words: 13,250\n",
      "\n",
      "--- primeros 800 chars ---\n",
      "1/20\n",
      "Banco Bilbao Vizcaya Argentaria, S.A., con domicilio en la Plaza San Nicolás, número 4, 48005 Bilbao,inscrito en el \n",
      "Registro Mercantil de Vizcaya, al tomo 2.083, Folio 1, Hoja BI-17-A, Inscripción 1ª con C.I.F. A-48265169POLÍTICA DE PROTECCIÓN DE \n",
      "DATOS PERSONALES\n",
      "1. Política de Protección de Datos Personales\n",
      "T ómate tu tiempo y lee atentamente este documento. No dudes en pedirnos aclaraciones de lo que no entiendas.\n",
      "En este apartado te explicamos para qué utilizará BBVA tus datos y, entre otros aspectos, qué derechos tienes relacionados con su uso.\n",
      "INFORMACIÓN BÁSICA SOBRE PROTECCIÓN DE DATOS\n",
      "ResponsableBanco Bilbao Vizcaya Argentaria, S.A. (“BBVA”) con domicilio social en Plaza de San Nicolás 4, \n",
      "48005 Bilbao, España. Dirección de correo electrónico: consultasgenerales@bbva.com\n",
      "Más\n",
      "\n",
      "--- ultimos 400 chars ---\n",
      "es retirar el consentimiento prestado sin que ello afecte a la licitud del tratamiento así \n",
      "como\tconfigurar\ttus\topciones\ty\tlos\tconsentimientos\t prestados\tdesde\ttu\tbanca\tonline,\tla\tApp\tde\tBBVA,\tasí\tcomo,\tenviando\ttu\t\n",
      "solicitud a la dirección de email derechosprotecciondatos@bbva.com , al Servicio Atención al Cliente Grupo BBVA, APDO: 1598 - \n",
      "28080\tMadrid,\to\tacudiendo\ta\talguna\tde\tnuestras\toficinas.\t\n"
     ]
    }
   ],
   "source": [
    "t0 = time.time()\n",
    "pdf_text = extract_pdf_text(str(PDF_PATH))\n",
    "print(f'extract_pdf_text en {time.time()-t0:.2f}s')\n",
    "print(f'chars: {len(pdf_text):,}  words: {len(pdf_text.split()):,}')\n",
    "print()\n",
    "print('--- primeros 800 chars ---')\n",
    "print(pdf_text[:800])\n",
    "print()\n",
    "print('--- ultimos 400 chars ---')\n",
    "print(pdf_text[-400:])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74aa813f",
   "metadata": {},
   "source": [
    "## B.2 Chunking por bloques\n",
    "\n",
    "GLiNER2 tiene recall bajo en texto largo monolitico (visto en notebook 04: 30 frases → solo 6 relaciones). Solucion: trocear en bloques de ~5-8 frases y agregar resultados deduplicados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2b64164d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:51.027135Z",
     "iopub.status.busy": "2026-05-04T14:12:51.026972Z",
     "iopub.status.idle": "2026-05-04T14:12:51.032477Z",
     "shell.execute_reply": "2026-05-04T14:12:51.031235Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67 chunks (max 1500 chars cada uno)\n",
      "tamanos: [1460, 1244, 1403, 1472, 1501, 1251, 1463, 1391, 1439, 1374]...\n",
      "\n",
      "--- chunk 0 (primeras 500 chars) ---\n",
      "1/20\n",
      "Banco Bilbao Vizcaya Argentaria, S.A., con domicilio en la Plaza San Nicolás, número 4, 48005 Bilbao,inscrito en el \n",
      "Registro Mercantil de Vizcaya, al tomo 2.083, Folio 1, Hoja BI-17-A, Inscripción 1ª con C.I.F. A-48265169POLÍTICA DE PROTECCIÓN DE \n",
      "DATOS PERSONALES\n",
      "1. Política de Protección de Datos Personales\n",
      "T ómate tu tiempo y lee atentamente este documento. No dudes en pedirnos aclaraciones de lo que no entiendas. En este apartado te explicamos para qué utilizará BBVA tus datos y, entre\n"
     ]
    }
   ],
   "source": [
    "def chunk_by_sentences(text, max_chars=1500):\n",
    "    # split en frases, agrupar hasta max_chars\n",
    "    sentences = re.split(r'(?<=[\\.!?])\\s+', text)\n",
    "    chunks, current = [], ''\n",
    "    for s in sentences:\n",
    "        if not s.strip(): continue\n",
    "        if len(current) + len(s) > max_chars and current:\n",
    "            chunks.append(current.strip())\n",
    "            current = s\n",
    "        else:\n",
    "            current += ' ' + s\n",
    "    if current.strip(): chunks.append(current.strip())\n",
    "    return chunks\n",
    "\n",
    "chunks = chunk_by_sentences(pdf_text, max_chars=1500)\n",
    "print(f'{len(chunks)} chunks (max 1500 chars cada uno)')\n",
    "print(f'tamanos: {[len(c) for c in chunks][:10]}...')\n",
    "print()\n",
    "print('--- chunk 0 (primeras 500 chars) ---')\n",
    "print(chunks[0][:500])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03dc9412",
   "metadata": {},
   "source": [
    "## B.3 GLiNER2 sobre cada chunk + agregacion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "212acf6e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:12:51.034247Z",
     "iopub.status.busy": "2026-05-04T14:12:51.034081Z",
     "iopub.status.idle": "2026-05-04T14:14:27.663286Z",
     "shell.execute_reply": "2026-05-04T14:14:27.662018Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 5/67 → ents acumuladas: 49, rels: 3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 10/67 → ents acumuladas: 128, rels: 4\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 15/67 → ents acumuladas: 144, rels: 5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 20/67 → ents acumuladas: 174, rels: 8\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 25/67 → ents acumuladas: 205, rels: 16\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 30/67 → ents acumuladas: 237, rels: 23\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 35/67 → ents acumuladas: 259, rels: 33\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 40/67 → ents acumuladas: 300, rels: 35\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 45/67 → ents acumuladas: 320, rels: 37\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 50/67 → ents acumuladas: 332, rels: 41\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 55/67 → ents acumuladas: 339, rels: 44\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 60/67 → ents acumuladas: 359, rels: 50\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  chunk 65/67 → ents acumuladas: 376, rels: 53\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Total: 67 chunks en 96.6s (1.44s/chunk)\n",
      "Entidades unicas: 378\n",
      "Relaciones unicas: 54\n"
     ]
    }
   ],
   "source": [
    "# Schema legal/proteccion-datos: anadimos labels especificas del dominio\n",
    "PDF_ENTITY_LABELS = [\n",
    "    'person', 'organization', 'location', 'email',\n",
    "    'law', 'right', 'data_category', 'authority',\n",
    "]\n",
    "PDF_RELATION_LABELS = [\n",
    "    'located_in', 'governed_by', 'subject_to', 'protected_by',\n",
    "    'contact_for', 'rights_against', 'subsidiary_of', 'controlled_by',\n",
    "]\n",
    "\n",
    "schema_pdf = (model.create_schema()\n",
    "    .entities(PDF_ENTITY_LABELS)\n",
    "    .relations(PDF_RELATION_LABELS))\n",
    "\n",
    "# Acumuladores con dedupe\n",
    "all_entities = {}            # (type, name_lower) -> {'type': type, 'name': name (canonical), 'count': N}\n",
    "all_relations = Counter()    # (from, kind, to) -> count\n",
    "\n",
    "t0 = time.time()\n",
    "for i, chunk in enumerate(chunks):\n",
    "    r = model.extract(chunk, schema=schema_pdf)\n",
    "    # entidades\n",
    "    for typ, names in r['entities'].items():\n",
    "        for n in names:\n",
    "            n_clean = n.strip()\n",
    "            if not n_clean: continue\n",
    "            key = (typ, n_clean.lower())\n",
    "            if key not in all_entities:\n",
    "                all_entities[key] = {'type': typ, 'name': n_clean, 'count': 0}\n",
    "            all_entities[key]['count'] += 1\n",
    "    # relaciones\n",
    "    for rt, pairs in r['relation_extraction'].items():\n",
    "        for h, t in pairs:\n",
    "            all_relations[(h.strip(), rt, t.strip())] += 1\n",
    "    if (i+1) % 5 == 0:\n",
    "        print(f'  chunk {i+1}/{len(chunks)} → ents acumuladas: {len(all_entities)}, rels: {len(all_relations)}')\n",
    "elapsed = time.time() - t0\n",
    "print(f'\\nTotal: {len(chunks)} chunks en {elapsed:.1f}s ({elapsed/len(chunks):.2f}s/chunk)')\n",
    "print(f'Entidades unicas: {len(all_entities)}')\n",
    "print(f'Relaciones unicas: {len(all_relations)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70099dfc",
   "metadata": {},
   "source": [
    "## B.4 Top entidades por frecuencia de mencion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6f4049d7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:14:27.664900Z",
     "iopub.status.busy": "2026-05-04T14:14:27.664747Z",
     "iopub.status.idle": "2026-05-04T14:14:27.674338Z",
     "shell.execute_reply": "2026-05-04T14:14:27.673420Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TOP 25 entidades por menciones:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>type</th>\n",
       "      <th>name</th>\n",
       "      <th>mentions</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>organization</td>\n",
       "      <td>BBVA</td>\n",
       "      <td>53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>data_category</td>\n",
       "      <td>Datos Personales</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>location</td>\n",
       "      <td>Plaza San Nicolás</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>person</td>\n",
       "      <td>cliente</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>data_category</td>\n",
       "      <td>Datos\\tPersonales</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>data_category</td>\n",
       "      <td>Datos\\tidentificativos</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>law</td>\n",
       "      <td>normativa</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>email</td>\n",
       "      <td>derechosprotecciondatos@bbva.com</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>data_category</td>\n",
       "      <td>datos</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>location</td>\n",
       "      <td>Vizcaya</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>organization</td>\n",
       "      <td>Banco</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>right</td>\n",
       "      <td>consentimiento</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>person</td>\n",
       "      <td>clientes</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>right</td>\n",
       "      <td>acceso</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>authority</td>\n",
       "      <td>autorizado</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>data_category</td>\n",
       "      <td>Datos de productos</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>location</td>\n",
       "      <td>España</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>location</td>\n",
       "      <td>oficinas</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>location</td>\n",
       "      <td>MADRID</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>organization</td>\n",
       "      <td>Grupo BBVA</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>organization</td>\n",
       "      <td>sociedades participadas</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>right</td>\n",
       "      <td>supresión</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>right</td>\n",
       "      <td>rectificación</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>authority</td>\n",
       "      <td>Registro Mercantil de Vizcaya</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>data_category</td>\n",
       "      <td>Datos transaccionales</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             type                              name  mentions\n",
       "0    organization                              BBVA        53\n",
       "1   data_category                  Datos Personales        32\n",
       "2        location                 Plaza San Nicolás        19\n",
       "3          person                           cliente        19\n",
       "4   data_category                 Datos\\tPersonales        10\n",
       "5   data_category            Datos\\tidentificativos        10\n",
       "6             law                         normativa        10\n",
       "7           email  derechosprotecciondatos@bbva.com         9\n",
       "8   data_category                             datos         8\n",
       "9        location                           Vizcaya         8\n",
       "10   organization                             Banco         8\n",
       "11          right                    consentimiento         8\n",
       "12         person                          clientes         7\n",
       "13          right                            acceso         6\n",
       "14      authority                        autorizado         5\n",
       "15  data_category                Datos de productos         5\n",
       "16       location                            España         5\n",
       "17       location                          oficinas         5\n",
       "18       location                            MADRID         5\n",
       "19   organization                        Grupo BBVA         5\n",
       "20   organization           sociedades participadas         5\n",
       "21          right                         supresión         5\n",
       "22          right                     rectificación         5\n",
       "23      authority     Registro Mercantil de Vizcaya         4\n",
       "24  data_category             Datos transaccionales         4"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ent_rows = [{'type': v['type'], 'name': v['name'], 'mentions': v['count']} for v in all_entities.values()]\n",
    "df_pdf_ents = pd.DataFrame(ent_rows).sort_values(['mentions', 'type'], ascending=[False, True]).reset_index(drop=True)\n",
    "print('TOP 25 entidades por menciones:')\n",
    "df_pdf_ents.head(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f5117aa",
   "metadata": {},
   "source": [
    "## B.5 Relaciones extraidas (top 25 por count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2b78c4e1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:14:27.675944Z",
     "iopub.status.busy": "2026-05-04T14:14:27.675768Z",
     "iopub.status.idle": "2026-05-04T14:14:27.684510Z",
     "shell.execute_reply": "2026-05-04T14:14:27.683743Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "54 relaciones unicas\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>from</th>\n",
       "      <th>kind</th>\n",
       "      <th>to</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BBVA</td>\n",
       "      <td>subsidiary_of</td>\n",
       "      <td>BBVA</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Plaza San Nicolás</td>\n",
       "      <td>located_in</td>\n",
       "      <td>Plaza San Nicolás</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>datos personales</td>\n",
       "      <td>subject_to</td>\n",
       "      <td>normativa</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>datos personales</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>Agencia Española de Protección de Datos</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BBVA</td>\n",
       "      <td>rights_against</td>\n",
       "      <td>datos</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Datos Personales</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>Política\\tde\\tProtección\\tde\\t\\nDatos Personales</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>cuentas corrientes</td>\n",
       "      <td>controlled_by</td>\n",
       "      <td>Fichero de Titularidades Financieras</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>BBVA Seguros, S.A.</td>\n",
       "      <td>subsidiary_of</td>\n",
       "      <td>BBVA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>entidades\\tfinancieras</td>\n",
       "      <td>governed_by</td>\n",
       "      <td>normativa española y europea</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>operador</td>\n",
       "      <td>rights_against</td>\n",
       "      <td>decisión</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>datos\\t\\npersonales</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>Fichero Confirma</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>BBVA</td>\n",
       "      <td>subsidiary_of</td>\n",
       "      <td>Fichero Confirma</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>contratos de crédito inmobiliario</td>\n",
       "      <td>governed_by</td>\n",
       "      <td>Ley 5/2019</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Avda. de\\tla\\tIndustria</td>\n",
       "      <td>located_in</td>\n",
       "      <td>MADRID</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>datos\\tpersonales</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>Agencia Española de Protección de Datos</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>datos personales</td>\n",
       "      <td>subject_to</td>\n",
       "      <td>normativa\\tvigente\\ten\\tmateria\\tde\\tprotecció...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>fichero\\tcomún</td>\n",
       "      <td>controlled_by</td>\n",
       "      <td>BBVA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>dpo@confirmasistemas.es</td>\n",
       "      <td>contact_for</td>\n",
       "      <td>solicitudes\\ten\\tmateria\\tde\\tprivacidad</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>bancos</td>\n",
       "      <td>located_in</td>\n",
       "      <td>ámbito UE</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Delegado de \\nProtección de Datos</td>\n",
       "      <td>contact_for</td>\n",
       "      <td>Delegado de \\nProtección de Datos</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>datos personales</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>Agencia Española de \\nProtección de Datos</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Emailage Corporation</td>\n",
       "      <td>governed_by</td>\n",
       "      <td>BBVA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>datos\\tpersonales</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>BBVA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Fichero FrauDfense</td>\n",
       "      <td>governed_by</td>\n",
       "      <td>FrauDfense,\\t S.L.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Fichero FrauDfense</td>\n",
       "      <td>protected_by</td>\n",
       "      <td>Fichero FrauDfense</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 from            kind  \\\n",
       "0                                BBVA   subsidiary_of   \n",
       "1                   Plaza San Nicolás      located_in   \n",
       "2                    datos personales      subject_to   \n",
       "3                    datos personales    protected_by   \n",
       "4                                BBVA  rights_against   \n",
       "5                    Datos Personales    protected_by   \n",
       "6                  cuentas corrientes   controlled_by   \n",
       "7                  BBVA Seguros, S.A.   subsidiary_of   \n",
       "8              entidades\\tfinancieras     governed_by   \n",
       "9                            operador  rights_against   \n",
       "10                datos\\t\\npersonales    protected_by   \n",
       "11                               BBVA   subsidiary_of   \n",
       "12  contratos de crédito inmobiliario     governed_by   \n",
       "13            Avda. de\\tla\\tIndustria      located_in   \n",
       "14                  datos\\tpersonales    protected_by   \n",
       "15                   datos personales      subject_to   \n",
       "16                     fichero\\tcomún   controlled_by   \n",
       "17            dpo@confirmasistemas.es     contact_for   \n",
       "18                             bancos      located_in   \n",
       "19  Delegado de \\nProtección de Datos     contact_for   \n",
       "20                   datos personales    protected_by   \n",
       "21               Emailage Corporation     governed_by   \n",
       "22                  datos\\tpersonales    protected_by   \n",
       "23                 Fichero FrauDfense     governed_by   \n",
       "24                 Fichero FrauDfense    protected_by   \n",
       "\n",
       "                                                   to  count  \n",
       "0                                                BBVA     12  \n",
       "1                                   Plaza San Nicolás      4  \n",
       "2                                           normativa      2  \n",
       "3             Agencia Española de Protección de Datos      2  \n",
       "4                                               datos      1  \n",
       "5    Política\\tde\\tProtección\\tde\\t\\nDatos Personales      1  \n",
       "6                Fichero de Titularidades Financieras      1  \n",
       "7                                                BBVA      1  \n",
       "8                        normativa española y europea      1  \n",
       "9                                            decisión      1  \n",
       "10                                   Fichero Confirma      1  \n",
       "11                                   Fichero Confirma      1  \n",
       "12                                         Ley 5/2019      1  \n",
       "13                                             MADRID      1  \n",
       "14            Agencia Española de Protección de Datos      1  \n",
       "15  normativa\\tvigente\\ten\\tmateria\\tde\\tprotecció...      1  \n",
       "16                                               BBVA      1  \n",
       "17           solicitudes\\ten\\tmateria\\tde\\tprivacidad      1  \n",
       "18                                          ámbito UE      1  \n",
       "19                  Delegado de \\nProtección de Datos      1  \n",
       "20          Agencia Española de \\nProtección de Datos      1  \n",
       "21                                               BBVA      1  \n",
       "22                                               BBVA      1  \n",
       "23                                 FrauDfense,\\t S.L.      1  \n",
       "24                                 Fichero FrauDfense      1  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rel_rows = [{'from': h, 'kind': rt, 'to': t, 'count': c} for (h, rt, t), c in all_relations.items()]\n",
    "df_pdf_rels = pd.DataFrame(rel_rows).sort_values('count', ascending=False).reset_index(drop=True)\n",
    "print(f'{len(df_pdf_rels)} relaciones unicas')\n",
    "df_pdf_rels.head(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed5e194e",
   "metadata": {},
   "source": [
    "## B.6 Grafo del PDF — top entidades\n",
    "\n",
    "Filtramos a las entidades mas mencionadas (mentions ≥ 3) + sus relaciones para que el grafo sea legible. El PDF tiene cientos de entidades; un grafo sin filtrar seria ilegible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6ca82d07",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:14:27.686012Z",
     "iopub.status.busy": "2026-05-04T14:14:27.685873Z",
     "iopub.status.idle": "2026-05-04T14:14:27.832836Z",
     "shell.execute_reply": "2026-05-04T14:14:27.832021Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Filtrado: 50 ents con >=3 menciones, 44 aisladas removidas\n",
      "Grafo final: 6 nodos, 5 aristas\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 92036 characters]",
      "text/plain": [
       "<Figure size 1600x1200 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "MIN_MENTIONS = 3\n",
    "kept_names = {v['name'] for v in all_entities.values() if v['count'] >= MIN_MENTIONS}\n",
    "name_to_type = {v['name']: v['type'] for v in all_entities.values()}\n",
    "\n",
    "G_pdf = nx.DiGraph()\n",
    "for n in kept_names:\n",
    "    G_pdf.add_node(n, type=name_to_type.get(n, '?'))\n",
    "\n",
    "for (h, rt, t), c in all_relations.items():\n",
    "    if h in kept_names and t in kept_names:\n",
    "        G_pdf.add_edge(h, t, kind=rt, count=c)\n",
    "\n",
    "# quitar nodos isolados\n",
    "isolates = list(nx.isolates(G_pdf))\n",
    "G_pdf.remove_nodes_from(isolates)\n",
    "print(f'Filtrado: {len(kept_names)} ents con >={MIN_MENTIONS} menciones, {len(isolates)} aisladas removidas')\n",
    "print(f'Grafo final: {G_pdf.number_of_nodes()} nodos, {G_pdf.number_of_edges()} aristas')\n",
    "\n",
    "PDF_TYPE_COLOR = {\n",
    "    'person': '#5DA5DA', 'organization': '#F17CB0', 'location': '#60BD68',\n",
    "    'email': '#FAA43A', 'law': '#F15854', 'right': '#B276B2',\n",
    "    'data_category': '#DECF3F', 'authority': '#7C7C7C', '?': '#bbb',\n",
    "}\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(16, 12))\n",
    "draw_graph(G_pdf, ax, f'PDF: politica BBVA — top entidades (≥{MIN_MENTIONS} menciones)', type_color=PDF_TYPE_COLOR)\n",
    "from matplotlib.patches import Patch\n",
    "active_types = {G_pdf.nodes[n].get('type') for n in G_pdf.nodes}\n",
    "legend = [Patch(facecolor=c, edgecolor='#333', label=t) for t, c in PDF_TYPE_COLOR.items() if t in active_types]\n",
    "ax.legend(handles=legend, loc='upper left', fontsize=9)\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95192d11",
   "metadata": {},
   "source": [
    "## B.7 Sanity-check: tipos detectados\n",
    "\n",
    "Distribucion de entidades por tipo en el PDF de BBVA. Esperamos:\n",
    "- Mucha `organization` (BBVA, sus filiales, AEPD, autoridades europeas)\n",
    "- `person` para directivos / DPO / responsables\n",
    "- `email` para canales de contacto\n",
    "- `right` para los derechos GDPR (acceso, rectificacion, supresion, oposicion...)\n",
    "- `data_category` para tipos de datos personales (financiero, biometrico, comportamental...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "eec3169f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:14:27.834416Z",
     "iopub.status.busy": "2026-05-04T14:14:27.834264Z",
     "iopub.status.idle": "2026-05-04T14:14:27.845779Z",
     "shell.execute_reply": "2026-05-04T14:14:27.845047Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>n_unique</th>\n",
       "      <th>total_mentions</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>type</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>data_category</th>\n",
       "      <td>175</td>\n",
       "      <td>290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>organization</th>\n",
       "      <td>62</td>\n",
       "      <td>148</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>location</th>\n",
       "      <td>29</td>\n",
       "      <td>68</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>right</th>\n",
       "      <td>28</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>person</th>\n",
       "      <td>20</td>\n",
       "      <td>55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>law</th>\n",
       "      <td>33</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>authority</th>\n",
       "      <td>19</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>email</th>\n",
       "      <td>12</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               n_unique  total_mentions\n",
       "type                                   \n",
       "data_category       175             290\n",
       "organization         62             148\n",
       "location             29              68\n",
       "right                28              64\n",
       "person               20              55\n",
       "law                  33              48\n",
       "authority            19              34\n",
       "email                12              26"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "by_type = df_pdf_ents.groupby('type').agg(\n",
    "    n_unique=('name', 'nunique'),\n",
    "    total_mentions=('mentions', 'sum'),\n",
    ").sort_values('total_mentions', ascending=False)\n",
    "by_type"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "123c726e",
   "metadata": {},
   "source": [
    "## B.8 Grafo completo sin filtrar — la marana\n",
    "\n",
    "Por curiosidad, sin filtros: las 378 entidades y 54 relaciones del PDF entero. No hay etiquetas (ilegibles a esta escala) — los nodos se colorean por tipo. Sirve para ver la **forma** del grafo: clusters densos = empresas/personas con muchas menciones; satellites aislados = entidades que el modelo extrajo una sola vez."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f2e08dd1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-04T14:14:27.847588Z",
     "iopub.status.busy": "2026-05-04T14:14:27.847442Z",
     "iopub.status.idle": "2026-05-04T14:14:28.323708Z",
     "shell.execute_reply": "2026-05-04T14:14:28.322400Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Grafo completo: 382 nodos, 48 aristas\n",
      "  de los cuales aislados: 324\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 609188 characters]",
      "text/plain": [
       "<Figure size 2000x2000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Grafo completo (sin filtro de menciones)\n",
    "G_full = nx.DiGraph()\n",
    "for v in all_entities.values():\n",
    "    G_full.add_node(v['name'], type=v['type'], mentions=v['count'])\n",
    "for (h, rt, t), c in all_relations.items():\n",
    "    if h not in G_full.nodes: G_full.add_node(h, type='?', mentions=0)\n",
    "    if t not in G_full.nodes: G_full.add_node(t, type='?', mentions=0)\n",
    "    G_full.add_edge(h, t, kind=rt, count=c)\n",
    "\n",
    "print(f'Grafo completo: {G_full.number_of_nodes()} nodos, {G_full.number_of_edges()} aristas')\n",
    "isolates = list(nx.isolates(G_full))\n",
    "print(f'  de los cuales aislados: {len(isolates)}')\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(20, 20))\n",
    "# Layout que aguanta grafos grandes — spring con menos iteraciones\n",
    "pos = nx.spring_layout(G_full, k=0.5, iterations=40, seed=42)\n",
    "node_sizes = [60 + 25 * G_full.nodes[n].get('mentions', 0) for n in G_full.nodes]\n",
    "node_colors = [PDF_TYPE_COLOR.get(G_full.nodes[n].get('type'), '#bbb') for n in G_full.nodes]\n",
    "nx.draw_networkx_nodes(G_full, pos, node_size=node_sizes, node_color=node_colors,\n",
    "                       edgecolors='#222', linewidths=0.4, alpha=0.85, ax=ax)\n",
    "nx.draw_networkx_edges(G_full, pos, edge_color='#555', alpha=0.25, width=0.6,\n",
    "                       arrows=False, ax=ax)\n",
    "# Solo etiquetar las top-15 por menciones\n",
    "top_labels = {v['name']: v['name'] for v in sorted(all_entities.values(), key=lambda x: -x['count'])[:15]}\n",
    "nx.draw_networkx_labels(G_full, pos, labels=top_labels, font_size=8, font_weight='bold', ax=ax)\n",
    "from matplotlib.patches import Patch\n",
    "active_types = {G_full.nodes[n].get('type') for n in G_full.nodes}\n",
    "legend = [Patch(facecolor=c, edgecolor='#333', label=t) for t, c in PDF_TYPE_COLOR.items() if t in active_types]\n",
    "ax.legend(handles=legend, loc='upper left', fontsize=11)\n",
    "ax.set_title(f'PDF completo SIN filtro: {G_full.number_of_nodes()} entidades, {G_full.number_of_edges()} relaciones',\n",
    "             fontsize=13)\n",
    "ax.axis('off')\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4b1ab21",
   "metadata": {},
   "source": [
    "**Lectura del grafo completo:**\n",
    "\n",
    "- **Cluster central denso** = entidades muy mencionadas (BBVA, AEPD, los derechos GDPR, los responsables del tratamiento) — donde el modelo establece las relaciones reales.\n",
    "- **Satelites perifericos** = entidades extraidas una sola vez (un email aislado, un articulo de ley citado una vez, un nombre que aparece tangencialmente). Mucho ruido pero util para ver el alcance.\n",
    "- **Tamaño de nodo** ∝ menciones (los grandes son los protagonistas).\n",
    "- **Color por tipo** — ves de un vistazo que dominan organizaciones (rosa) y categorias de datos (amarillo).\n",
    "- Sin filtrado, el grafo es **una maraña** — exactamente por eso B.6 filtraba a entidades con ≥3 menciones."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fd8a6f4",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "\n",
    "**Funciono el flujo end-to-end.** El pipeline:\n",
    "\n",
    "1. **`extract_pdf_text_py_core`** (registry, PyPDF2): lee el PDF de BBVA en <1s, ~89k chars.\n",
    "2. **Chunking** por bloques de 1500 chars (~25 chunks).\n",
    "3. **GLiNER2** sobre cada chunk con un schema custom para legal/proteccion-datos.\n",
    "4. **Agregacion deduplicada** con conteo de menciones.\n",
    "5. **Filtro a top entidades** (>= 3 menciones) para que el grafo sea legible.\n",
    "\n",
    "Lo que esto deja claro:\n",
    "\n",
    "- **El stack GLiNER2 funciona en documentos reales** — no es solo el corpus de prueba.\n",
    "- **Chunking es esencial** para textos > 30 frases.\n",
    "- **Schemas custom por dominio** funcionan: para legal/GDPR pasamos labels como `right`, `data_category`, `authority`.\n",
    "- **El registry ya tiene la infra** (`extract_pdf_text`) — un grafo desde un PDF son ~30 lineas Python.\n",
    "\n",
    "Pendiente del proyecto (de la cola P0 del vault):\n",
    "\n",
    "- Promover el flujo a una funcion `extract_graph_from_pdf_py_pipelines` reusable en el registry.\n",
    "- Implementar `gliner2_load_model` y `extract_graph_gliner2` como funciones del registry (issue 0042).\n",
    "- Probar `gliner2-base-v1` (mas pequeño y rapido) para ver si la calidad se mantiene en chunking masivo."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
