{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "939ac1fa-c8f0-4285-86aa-892051cd6800",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "8e11d891",
   "metadata": {},
   "source": [
    "# Exploración de Embeddings: Modelos Livianos para Retrieval\n",
    "\n",
    "Comparamos modelos de sentence-transformers enfocándonos en:\n",
    "- **Tamaño y velocidad** — cuál es el más liviano\n",
    "- **Calidad de retrieval** — precision@k, recall@k, MRR\n",
    "- **Trade-off práctico** — el mejor balance peso/rendimiento"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fafce80",
   "metadata": {},
   "source": [
    "## 1. Setup e imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "29508ec1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Modelos a comparar: 6\n",
      "  MiniLM-L6                 → all-MiniLM-L6-v2                                   (22M - solo inglés (baseline))\n",
      "  MiniLM-L12                → all-MiniLM-L12-v2                                  (33M - solo inglés)\n",
      "  paraphrase-multi-L12      → paraphrase-multilingual-MiniLM-L12-v2              (118M - multilingüe 50+ idiomas)\n",
      "  multi-e5-small            → intfloat/multilingual-e5-small                     (118M - multilingüe, buen retrieval)\n",
      "  nomic-v1.5                → nomic-ai/nomic-embed-text-v1.5                     (137M - Matryoshka, dim flexible)\n",
      "  BGE-m3                    → BAAI/bge-m3                                        (568M - multilingüe SOTA, más pesado)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sentence_transformers import SentenceTransformer\n",
    "import faiss\n",
    "import time\n",
    "import os\n",
    "\n",
    "plt.style.use('seaborn-v0_8-whitegrid')\n",
    "plt.rcParams['figure.figsize'] = (12, 6)\n",
    "\n",
    "# Modelos a comparar: nombre -> (id huggingface, descripcion, prefijo_query, prefijo_doc)\n",
    "# None = sin prefijo. Los modelos e5 y nomic necesitan prefijos especiales.\n",
    "MODELS = {\n",
    "    'MiniLM-L6':           ('all-MiniLM-L6-v2',                      '22M - solo inglés (baseline)',        None, None),\n",
    "    'MiniLM-L12':          ('all-MiniLM-L12-v2',                     '33M - solo inglés',                   None, None),\n",
    "    'paraphrase-multi-L12':('paraphrase-multilingual-MiniLM-L12-v2', '118M - multilingüe 50+ idiomas',      None, None),\n",
    "    'multi-e5-small':      ('intfloat/multilingual-e5-small',        '118M - multilingüe, buen retrieval',  'query: ', 'passage: '),\n",
    "    'nomic-v1.5':          ('nomic-ai/nomic-embed-text-v1.5',        '137M - Matryoshka, dim flexible',     'search_query: ', 'search_document: '),\n",
    "    'BGE-m3':              ('BAAI/bge-m3',                           '568M - multilingüe SOTA, más pesado', None, None),\n",
    "}\n",
    "\n",
    "print(f'Modelos a comparar: {len(MODELS)}')\n",
    "for name, (model_id, desc, _, _) in MODELS.items():\n",
    "    print(f'  {name:25s} → {model_id:50s} ({desc})')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf402d89",
   "metadata": {},
   "source": [
    "## 2. Corpus de prueba\n",
    "\n",
    "Creamos un corpus con documentos de distintas categorías para evaluar retrieval. Cada query tiene documentos relevantes conocidos (ground truth)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3550fb1c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus: 25 documentos en español\n",
      "Queries: 10 en español con ground truth\n",
      "Categorías: programación, ciencia, cocina, finanzas, geografía\n"
     ]
    }
   ],
   "source": [
    "# Corpus: documentos agrupados por tema (en español para evaluar retrieval multilingüe)\n",
    "corpus = [\n",
    "    # Programación (0-4)\n",
    "    'Python es un lenguaje de programación de alto nivel conocido por su legibilidad y versatilidad',\n",
    "    'JavaScript se ejecuta en el navegador y es esencial para el desarrollo web',\n",
    "    'Rust proporciona seguridad de memoria sin recolector de basura mediante su sistema de ownership',\n",
    "    'SQL es el lenguaje estándar para consultar bases de datos relacionales',\n",
    "    'Git es un sistema de control de versiones distribuido para rastrear cambios en el código',\n",
    "\n",
    "    # Ciencia (5-9)\n",
    "    'La fotosíntesis convierte la luz solar en energía química dentro de las células vegetales',\n",
    "    'La teoría de la relatividad describe cómo la gravedad deforma el espacio-tiempo',\n",
    "    'El ADN transporta información genética usando cuatro bases de nucleótidos',\n",
    "    'El entrelazamiento cuántico vincula partículas sin importar la distancia entre ellas',\n",
    "    'La evolución ocurre mediante selección natural y deriva genética',\n",
    "\n",
    "    # Cocina (10-14)\n",
    "    'El pan de masa madre requiere un cultivo fermentado de harina y agua',\n",
    "    'El arroz para sushi se sazona con vinagre, azúcar y sal',\n",
    "    'La reacción de Maillard crea el dorado y sabor al cocinar carne a alta temperatura',\n",
    "    'La emulsificación une aceite y agua usando agentes como la yema de huevo',\n",
    "    'La fermentación preserva alimentos y crea sabores complejos con el tiempo',\n",
    "\n",
    "    # Finanzas (15-19)\n",
    "    'El interés compuesto hace crecer los ahorros exponencialmente a largo plazo',\n",
    "    'La diversificación reduce el riesgo del portafolio distribuyendo inversiones entre activos',\n",
    "    'La inflación erosiona el poder adquisitivo y afecta las inversiones de renta fija',\n",
    "    'La Reserva Federal establece política monetaria ajustando las tasas de interés',\n",
    "    'Los fondos indexados replican índices de mercado con comisiones mínimas de gestión',\n",
    "\n",
    "    # Geografía (20-24)\n",
    "    'La selva amazónica produce aproximadamente el 20% del oxígeno del mundo',\n",
    "    'Las placas tectónicas se desplazan causando terremotos y erupciones volcánicas',\n",
    "    'El desierto del Sahara se extiende por once países del norte de África',\n",
    "    'Las corrientes oceánicas regulan el clima global y los patrones meteorológicos',\n",
    "    'La Fosa de las Marianas es el punto más profundo conocido del océano',\n",
    "]\n",
    "\n",
    "# Queries en español con ground truth (índices de documentos relevantes)\n",
    "queries = [\n",
    "    ('¿Cómo manejan los lenguajes de programación la memoria?',           [0, 2]),\n",
    "    ('¿Qué es el control de versiones para código fuente?',               [4]),\n",
    "    ('¿Cómo producen energía las plantas a partir de la luz?',            [5]),\n",
    "    ('Cuéntame sobre física de partículas y mecánica cuántica',           [7, 8]),\n",
    "    ('¿Cómo funciona la fermentación del pan?',                           [10, 14]),\n",
    "    ('¿Qué hace que la comida sepa mejor al cocinarla a fuego alto?',    [12, 13]),\n",
    "    ('¿Cómo puedo hacer crecer mis ahorros con el tiempo?',              [15, 16, 19]),\n",
    "    ('¿Qué afecta la economía y los precios?',                           [17, 18]),\n",
    "    ('Háblame de las partes más profundas del océano',                    [24]),\n",
    "    ('¿Cómo cambia la superficie de la Tierra con el tiempo?',           [21, 22]),\n",
    "]\n",
    "\n",
    "print(f'Corpus: {len(corpus)} documentos en español')\n",
    "print(f'Queries: {len(queries)} en español con ground truth')\n",
    "print(f'Categorías: programación, ciencia, cocina, finanzas, geografía')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc3857da",
   "metadata": {},
   "source": [
    "## 3. Cargar modelos y generar embeddings\n",
    "\n",
    "Cargamos cada modelo, medimos tiempo de carga, tiempo de encoding, y tamaño en memoria."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6470048d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Modelo                    |  Dim  |     Params |   Size |  Load  |  Corpus   | Queries\n",
      "-------------------------------------------------------------------------------------------------------------------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fd228d449f514e07a368b80fd5853c13",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: sentence-transformers/all-MiniLM-L6-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MiniLM-L6                 | dim= 384 | 22.7M params |     87MB | load=  2.4s | corpus=0.264s | queries=0.015s\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "43c98b329abd434085175b9decfb1017",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/199 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: sentence-transformers/all-MiniLM-L12-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MiniLM-L12                | dim= 384 | 33.4M params |    127MB | load=  1.9s | corpus=0.018s | queries=0.012s\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f90402364c634397ad9b29d5d41c65a7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/199 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "paraphrase-multi-L12      | dim= 384 | 117.7M params |    449MB | load=  3.5s | corpus=0.041s | queries=0.014s\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c115aee165af46e3b8bd6e2a4e22fb67",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/199 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: intfloat/multilingual-e5-small\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "multi-e5-small            | dim= 384 | 117.7M params |    449MB | load=  4.1s | corpus=0.055s | queries=0.017s\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "11a55f1c36e34bdf8fdcf7784a283788",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/547M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<All keys matched successfully>\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7b68536d70734f0f83201c2feb749e1b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json: 0.00B [00:00, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b79d8be283d247eb8e2cd522e536ca39",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.txt: 0.00B [00:00, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2d0f53694616493bbe303a9869972988",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json: 0.00B [00:00, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "588f7d602f2342e5916d1b665d0bd941",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/695 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fe5eac1948ab48b9a6aa0f7af2d58b00",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/286 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nomic-v1.5                | dim= 768 | 136.7M params |    522MB | load= 19.0s | corpus=0.055s | queries=0.018s\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ec927998eef14868989f3cf4e19b2a5c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "modules.json:   0%|          | 0.00/349 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d6afeac46acb4460a5d609799732f518",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config_sentence_transformers.json:   0%|          | 0.00/123 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "adef57db897a41ef89e71fa35145f246",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md: 0.00B [00:00, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4d7b38439e3346cabb17b8fa66b03774",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "sentence_bert_config.json:   0%|          | 0.00/54.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a6467c55cb204f67aebde1285b54a64d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/687 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9ce485266ee648aea5fb948dcebc0140",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "pytorch_model.bin:   0%|          | 0.00/2.27G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6fddf98edd5f4a2aad93f22dff9976b9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/391 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ef2460bc03a8491693311a460e32e509",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/444 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fbfb4328cab84c83aa5a8f1665df3537",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "sentencepiece.bpe.model:   0%|          | 0.00/5.07M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5a2145230a38405bb6d1834e2483dbcc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/2.27G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bd00e65a5a744979ad58563570c1a945",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/17.1M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5cc0f51c1bec4bb08b7fb9b50849f528",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/964 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "381797379d944388b3fbbf707c58e957",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/191 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BGE-m3                    | dim=1024 | 567.8M params |   2166MB | load= 48.0s | corpus=0.069s | queries=0.029s\n"
     ]
    }
   ],
   "source": [
    "import gc\n",
    "\n",
    "def measure_model(name, model_id, corpus, queries, q_prefix=None, d_prefix=None):\n",
    "    \"\"\"Carga modelo, genera embeddings, mide tiempos y tamaño.\"\"\"\n",
    "    gc.collect()\n",
    "\n",
    "    # Cargar modelo (nomic necesita trust_remote_code)\n",
    "    t0 = time.perf_counter()\n",
    "    kwargs = {'trust_remote_code': True} if 'nomic' in model_id else {}\n",
    "    model = SentenceTransformer(model_id, **kwargs)\n",
    "    load_time = time.perf_counter() - t0\n",
    "\n",
    "    # Tamaño del modelo en MB (acceder al transformer base)\n",
    "    try:\n",
    "        base = model[0].auto_model\n",
    "    except:\n",
    "        base = model\n",
    "    param_size = sum(p.nelement() * p.element_size() for p in base.parameters())\n",
    "    model_mb = param_size / (1024 * 1024)\n",
    "    n_params = sum(p.nelement() for p in base.parameters()) / 1e6\n",
    "\n",
    "    # Aplicar prefijos si el modelo los requiere\n",
    "    corpus_input = [f'{d_prefix}{doc}' for doc in corpus] if d_prefix else corpus\n",
    "    query_texts = [f'{q_prefix}{q}' for q, _ in queries] if q_prefix else [q for q, _ in queries]\n",
    "\n",
    "    # Encoding del corpus\n",
    "    t0 = time.perf_counter()\n",
    "    corpus_embs = model.encode(corpus_input, normalize_embeddings=True, show_progress_bar=False)\n",
    "    corpus_time = time.perf_counter() - t0\n",
    "\n",
    "    # Encoding de queries\n",
    "    t0 = time.perf_counter()\n",
    "    query_embs = model.encode(query_texts, normalize_embeddings=True, show_progress_bar=False)\n",
    "    query_time = time.perf_counter() - t0\n",
    "\n",
    "    dim = corpus_embs.shape[1]\n",
    "\n",
    "    print(f'{name:25s} | dim={dim:4d} | {n_params:.1f}M params | {model_mb:6.0f}MB | '\n",
    "          f'load={load_time:5.1f}s | corpus={corpus_time:.3f}s | queries={query_time:.3f}s')\n",
    "\n",
    "    return {\n",
    "        'name': name,\n",
    "        'model_id': model_id,\n",
    "        'dim': dim,\n",
    "        'n_params_m': round(n_params, 1),\n",
    "        'model_mb': round(model_mb, 1),\n",
    "        'load_time_s': round(load_time, 2),\n",
    "        'corpus_encode_s': round(corpus_time, 4),\n",
    "        'query_encode_s': round(query_time, 4),\n",
    "        'corpus_embs': corpus_embs,\n",
    "        'query_embs': query_embs,\n",
    "    }\n",
    "\n",
    "# Ejecutar benchmark de carga y encoding\n",
    "results = {}\n",
    "print(f'{\"Modelo\":25s} | {\"Dim\":>4s}  | {\"Params\":>10s} | {\"Size\":>6s} | {\"Load\":>5s}  | {\"Corpus\":>7s}   | {\"Queries\":>7s}')\n",
    "print('-' * 115)\n",
    "for name, (model_id, desc, q_prefix, d_prefix) in MODELS.items():\n",
    "    results[name] = measure_model(name, model_id, corpus, queries, q_prefix, d_prefix)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2598cba",
   "metadata": {},
   "source": [
    "## 4. Evaluación de Retrieval\n",
    "\n",
    "Para cada modelo, usamos FAISS para buscar los k documentos más cercanos a cada query y comparamos contra el ground truth.\n",
    "\n",
    "**Métricas:**\n",
    "- **Precision@k**: de los k recuperados, qué fracción es relevante\n",
    "- **Recall@k**: de los relevantes, qué fracción fue recuperada\n",
    "- **MRR (Mean Reciprocal Rank)**: inverso de la posición del primer resultado relevante"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "39149ab1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MiniLM-L6       | Precision@5=0.240 | Recall@5=0.733 | MRR=0.778\n",
      "MiniLM-L12      | Precision@5=0.220 | Recall@5=0.683 | MRR=0.820\n",
      "paraphrase-multi-L12 | Precision@5=0.320 | Recall@5=0.900 | MRR=0.933\n",
      "multi-e5-small  | Precision@5=0.300 | Recall@5=0.867 | MRR=0.950\n",
      "nomic-v1.5      | Precision@5=0.240 | Recall@5=0.733 | MRR=0.900\n",
      "BGE-m3          | Precision@5=0.300 | Recall@5=0.867 | MRR=1.000\n"
     ]
    }
   ],
   "source": [
    "def evaluate_retrieval(corpus_embs, query_embs, queries, k=5):\n",
    "    \"\"\"Evalúa retrieval con FAISS. Retorna métricas por query y agregadas.\"\"\"\n",
    "    dim = corpus_embs.shape[1]\n",
    "\n",
    "    # Índice FAISS con inner product (embeddings ya normalizados = cosine similarity)\n",
    "    index = faiss.IndexFlatIP(dim)\n",
    "    index.add(corpus_embs.astype(np.float32))\n",
    "\n",
    "    # Buscar k vecinos para cada query\n",
    "    scores, indices = index.search(query_embs.astype(np.float32), k)\n",
    "\n",
    "    per_query = []\n",
    "    for i, (query_text, relevant_ids) in enumerate(queries):\n",
    "        retrieved = indices[i].tolist()\n",
    "        relevant_set = set(relevant_ids)\n",
    "        retrieved_relevant = [idx for idx in retrieved if idx in relevant_set]\n",
    "\n",
    "        precision_at_k = len(retrieved_relevant) / k\n",
    "        recall_at_k = len(retrieved_relevant) / len(relevant_set) if relevant_set else 0\n",
    "\n",
    "        # MRR: posición del primer relevante\n",
    "        rr = 0\n",
    "        for rank, idx in enumerate(retrieved, 1):\n",
    "            if idx in relevant_set:\n",
    "                rr = 1.0 / rank\n",
    "                break\n",
    "\n",
    "        per_query.append({\n",
    "            'query': query_text[:60],\n",
    "            'retrieved': retrieved,\n",
    "            'relevant': relevant_ids,\n",
    "            'precision@k': precision_at_k,\n",
    "            'recall@k': recall_at_k,\n",
    "            'mrr': rr,\n",
    "        })\n",
    "\n",
    "    return {\n",
    "        'per_query': per_query,\n",
    "        'mean_precision': np.mean([q['precision@k'] for q in per_query]),\n",
    "        'mean_recall': np.mean([q['recall@k'] for q in per_query]),\n",
    "        'mrr': np.mean([q['mrr'] for q in per_query]),\n",
    "    }\n",
    "\n",
    "# Evaluar cada modelo\n",
    "K = 5\n",
    "eval_results = {}\n",
    "for name, res in results.items():\n",
    "    ev = evaluate_retrieval(res['corpus_embs'], res['query_embs'], queries, k=K)\n",
    "    eval_results[name] = ev\n",
    "    print(f\"{name:15s} | Precision@{K}={ev['mean_precision']:.3f} | Recall@{K}={ev['mean_recall']:.3f} | MRR={ev['mrr']:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6ae98f7",
   "metadata": {},
   "source": [
    "## 5. Tabla comparativa resumen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "83ce9114",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_6fe24_row0_col1, #T_6fe24_row1_col3 {\n",
       "  background-color: #67000d;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row0_col3 {\n",
       "  background-color: #6b010e;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row0_col6, #T_6fe24_row4_col6 {\n",
       "  background-color: #d3eecd;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_6fe24_row0_col7, #T_6fe24_row4_col7 {\n",
       "  background-color: #ccebc6;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_6fe24_row0_col8, #T_6fe24_row1_col6, #T_6fe24_row1_col7 {\n",
       "  background-color: #f7fcf5;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_6fe24_row1_col1 {\n",
       "  background-color: #71020e;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row1_col8 {\n",
       "  background-color: #d6efd0;\n",
       "  color: #000000;\n",
       "}\n",
       "#T_6fe24_row2_col1, #T_6fe24_row3_col1 {\n",
       "  background-color: #b31218;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row2_col3 {\n",
       "  background-color: #79040f;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row2_col6, #T_6fe24_row2_col7, #T_6fe24_row5_col8 {\n",
       "  background-color: #00441b;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row2_col8 {\n",
       "  background-color: #2f984f;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row3_col3 {\n",
       "  background-color: #7e0610;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row3_col6, #T_6fe24_row5_col6 {\n",
       "  background-color: #157f3b;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row3_col7, #T_6fe24_row5_col7 {\n",
       "  background-color: #077331;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row3_col8 {\n",
       "  background-color: #1c8540;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row4_col1 {\n",
       "  background-color: #be151a;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row4_col3 {\n",
       "  background-color: #ee3a2c;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row4_col8 {\n",
       "  background-color: #60ba6c;\n",
       "  color: #f1f1f1;\n",
       "}\n",
       "#T_6fe24_row5_col1, #T_6fe24_row5_col3 {\n",
       "  background-color: #fff5f0;\n",
       "  color: #000000;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_6fe24\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_6fe24_level0_col0\" class=\"col_heading level0 col0\" >Params (M)</th>\n",
       "      <th id=\"T_6fe24_level0_col1\" class=\"col_heading level0 col1\" >Tamaño (MB)</th>\n",
       "      <th id=\"T_6fe24_level0_col2\" class=\"col_heading level0 col2\" >Dim</th>\n",
       "      <th id=\"T_6fe24_level0_col3\" class=\"col_heading level0 col3\" >Load (s)</th>\n",
       "      <th id=\"T_6fe24_level0_col4\" class=\"col_heading level0 col4\" >Encode corpus (s)</th>\n",
       "      <th id=\"T_6fe24_level0_col5\" class=\"col_heading level0 col5\" >Encode queries (s)</th>\n",
       "      <th id=\"T_6fe24_level0_col6\" class=\"col_heading level0 col6\" >Precision@5</th>\n",
       "      <th id=\"T_6fe24_level0_col7\" class=\"col_heading level0 col7\" >Recall@5</th>\n",
       "      <th id=\"T_6fe24_level0_col8\" class=\"col_heading level0 col8\" >MRR</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th class=\"index_name level0\" >Modelo</th>\n",
       "      <th class=\"blank col0\" >&nbsp;</th>\n",
       "      <th class=\"blank col1\" >&nbsp;</th>\n",
       "      <th class=\"blank col2\" >&nbsp;</th>\n",
       "      <th class=\"blank col3\" >&nbsp;</th>\n",
       "      <th class=\"blank col4\" >&nbsp;</th>\n",
       "      <th class=\"blank col5\" >&nbsp;</th>\n",
       "      <th class=\"blank col6\" >&nbsp;</th>\n",
       "      <th class=\"blank col7\" >&nbsp;</th>\n",
       "      <th class=\"blank col8\" >&nbsp;</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_6fe24_level0_row0\" class=\"row_heading level0 row0\" >MiniLM-L6</th>\n",
       "      <td id=\"T_6fe24_row0_col0\" class=\"data row0 col0\" >22.700000</td>\n",
       "      <td id=\"T_6fe24_row0_col1\" class=\"data row0 col1\" >86.600000</td>\n",
       "      <td id=\"T_6fe24_row0_col2\" class=\"data row0 col2\" >384</td>\n",
       "      <td id=\"T_6fe24_row0_col3\" class=\"data row0 col3\" >2.360000</td>\n",
       "      <td id=\"T_6fe24_row0_col4\" class=\"data row0 col4\" >0.264300</td>\n",
       "      <td id=\"T_6fe24_row0_col5\" class=\"data row0 col5\" >0.014600</td>\n",
       "      <td id=\"T_6fe24_row0_col6\" class=\"data row0 col6\" >0.240000</td>\n",
       "      <td id=\"T_6fe24_row0_col7\" class=\"data row0 col7\" >0.733000</td>\n",
       "      <td id=\"T_6fe24_row0_col8\" class=\"data row0 col8\" >0.778000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6fe24_level0_row1\" class=\"row_heading level0 row1\" >MiniLM-L12</th>\n",
       "      <td id=\"T_6fe24_row1_col0\" class=\"data row1 col0\" >33.400000</td>\n",
       "      <td id=\"T_6fe24_row1_col1\" class=\"data row1 col1\" >127.300000</td>\n",
       "      <td id=\"T_6fe24_row1_col2\" class=\"data row1 col2\" >384</td>\n",
       "      <td id=\"T_6fe24_row1_col3\" class=\"data row1 col3\" >1.850000</td>\n",
       "      <td id=\"T_6fe24_row1_col4\" class=\"data row1 col4\" >0.017600</td>\n",
       "      <td id=\"T_6fe24_row1_col5\" class=\"data row1 col5\" >0.012400</td>\n",
       "      <td id=\"T_6fe24_row1_col6\" class=\"data row1 col6\" >0.220000</td>\n",
       "      <td id=\"T_6fe24_row1_col7\" class=\"data row1 col7\" >0.683000</td>\n",
       "      <td id=\"T_6fe24_row1_col8\" class=\"data row1 col8\" >0.820000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6fe24_level0_row2\" class=\"row_heading level0 row2\" >paraphrase-multi-L12</th>\n",
       "      <td id=\"T_6fe24_row2_col0\" class=\"data row2 col0\" >117.700000</td>\n",
       "      <td id=\"T_6fe24_row2_col1\" class=\"data row2 col1\" >448.800000</td>\n",
       "      <td id=\"T_6fe24_row2_col2\" class=\"data row2 col2\" >384</td>\n",
       "      <td id=\"T_6fe24_row2_col3\" class=\"data row2 col3\" >3.500000</td>\n",
       "      <td id=\"T_6fe24_row2_col4\" class=\"data row2 col4\" >0.040700</td>\n",
       "      <td id=\"T_6fe24_row2_col5\" class=\"data row2 col5\" >0.013800</td>\n",
       "      <td id=\"T_6fe24_row2_col6\" class=\"data row2 col6\" >0.320000</td>\n",
       "      <td id=\"T_6fe24_row2_col7\" class=\"data row2 col7\" >0.900000</td>\n",
       "      <td id=\"T_6fe24_row2_col8\" class=\"data row2 col8\" >0.933000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6fe24_level0_row3\" class=\"row_heading level0 row3\" >multi-e5-small</th>\n",
       "      <td id=\"T_6fe24_row3_col0\" class=\"data row3 col0\" >117.700000</td>\n",
       "      <td id=\"T_6fe24_row3_col1\" class=\"data row3 col1\" >448.800000</td>\n",
       "      <td id=\"T_6fe24_row3_col2\" class=\"data row3 col2\" >384</td>\n",
       "      <td id=\"T_6fe24_row3_col3\" class=\"data row3 col3\" >4.100000</td>\n",
       "      <td id=\"T_6fe24_row3_col4\" class=\"data row3 col4\" >0.055500</td>\n",
       "      <td id=\"T_6fe24_row3_col5\" class=\"data row3 col5\" >0.016900</td>\n",
       "      <td id=\"T_6fe24_row3_col6\" class=\"data row3 col6\" >0.300000</td>\n",
       "      <td id=\"T_6fe24_row3_col7\" class=\"data row3 col7\" >0.867000</td>\n",
       "      <td id=\"T_6fe24_row3_col8\" class=\"data row3 col8\" >0.950000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6fe24_level0_row4\" class=\"row_heading level0 row4\" >nomic-v1.5</th>\n",
       "      <td id=\"T_6fe24_row4_col0\" class=\"data row4 col0\" >136.700000</td>\n",
       "      <td id=\"T_6fe24_row4_col1\" class=\"data row4 col1\" >521.600000</td>\n",
       "      <td id=\"T_6fe24_row4_col2\" class=\"data row4 col2\" >768</td>\n",
       "      <td id=\"T_6fe24_row4_col3\" class=\"data row4 col3\" >19.040000</td>\n",
       "      <td id=\"T_6fe24_row4_col4\" class=\"data row4 col4\" >0.055400</td>\n",
       "      <td id=\"T_6fe24_row4_col5\" class=\"data row4 col5\" >0.017500</td>\n",
       "      <td id=\"T_6fe24_row4_col6\" class=\"data row4 col6\" >0.240000</td>\n",
       "      <td id=\"T_6fe24_row4_col7\" class=\"data row4 col7\" >0.733000</td>\n",
       "      <td id=\"T_6fe24_row4_col8\" class=\"data row4 col8\" >0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_6fe24_level0_row5\" class=\"row_heading level0 row5\" >BGE-m3</th>\n",
       "      <td id=\"T_6fe24_row5_col0\" class=\"data row5 col0\" >567.800000</td>\n",
       "      <td id=\"T_6fe24_row5_col1\" class=\"data row5 col1\" >2165.800000</td>\n",
       "      <td id=\"T_6fe24_row5_col2\" class=\"data row5 col2\" >1024</td>\n",
       "      <td id=\"T_6fe24_row5_col3\" class=\"data row5 col3\" >47.990000</td>\n",
       "      <td id=\"T_6fe24_row5_col4\" class=\"data row5 col4\" >0.069000</td>\n",
       "      <td id=\"T_6fe24_row5_col5\" class=\"data row5 col5\" >0.028500</td>\n",
       "      <td id=\"T_6fe24_row5_col6\" class=\"data row5 col6\" >0.300000</td>\n",
       "      <td id=\"T_6fe24_row5_col7\" class=\"data row5 col7\" >0.867000</td>\n",
       "      <td id=\"T_6fe24_row5_col8\" class=\"data row5 col8\" >1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f67287be3c0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Tabla resumen: velocidad + retrieval\n",
    "rows = []\n",
    "for name in results:\n",
    "    r = results[name]\n",
    "    ev = eval_results[name]\n",
    "    rows.append({\n",
    "        'Modelo': name,\n",
    "        'Params (M)': r['n_params_m'],\n",
    "        'Tamaño (MB)': r['model_mb'],\n",
    "        'Dim': r['dim'],\n",
    "        'Load (s)': r['load_time_s'],\n",
    "        'Encode corpus (s)': r['corpus_encode_s'],\n",
    "        'Encode queries (s)': r['query_encode_s'],\n",
    "        'Precision@5': round(ev['mean_precision'], 3),\n",
    "        'Recall@5': round(ev['mean_recall'], 3),\n",
    "        'MRR': round(ev['mrr'], 3),\n",
    "    })\n",
    "\n",
    "df_summary = pd.DataFrame(rows).set_index('Modelo')\n",
    "df_summary.style.background_gradient(subset=['Precision@5', 'Recall@5', 'MRR'], cmap='Greens')     .background_gradient(subset=['Tamaño (MB)', 'Load (s)'], cmap='Reds_r')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc5f97a8",
   "metadata": {},
   "source": [
    "## 6. Visualización comparativa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6bf6dd3c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 85980 characters]",
      "text/plain": [
       "<Figure size 1800x600 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, 3, figsize=(18, 6))\n",
    "\n",
    "models_names = list(eval_results.keys())\n",
    "colors = plt.cm.Set2(np.linspace(0, 1, len(models_names)))\n",
    "\n",
    "# Gráfico 1: Métricas de retrieval\n",
    "metrics = ['mean_precision', 'mean_recall', 'mrr']\n",
    "labels = ['Precision@5', 'Recall@5', 'MRR']\n",
    "x = np.arange(len(labels))\n",
    "width = 0.15\n",
    "for i, name in enumerate(models_names):\n",
    "    ev = eval_results[name]\n",
    "    vals = [ev[m] for m in metrics]\n",
    "    axes[0].bar(x + i * width, vals, width, label=name, color=colors[i])\n",
    "axes[0].set_xticks(x + width * (len(models_names) - 1) / 2)\n",
    "axes[0].set_xticklabels(labels)\n",
    "axes[0].set_ylim(0, 1.05)\n",
    "axes[0].set_title('Calidad de Retrieval (español)')\n",
    "axes[0].legend(fontsize=8)\n",
    "\n",
    "# Gráfico 2: Tamaño vs MRR (scatter)\n",
    "for i, name in enumerate(models_names):\n",
    "    r = results[name]\n",
    "    ev = eval_results[name]\n",
    "    axes[1].scatter(r['model_mb'], ev['mrr'], s=150, color=colors[i], label=name, zorder=5)\n",
    "    axes[1].annotate(name, (r['model_mb'], ev['mrr']), textcoords='offset points',\n",
    "                     xytext=(5, 5), fontsize=8)\n",
    "axes[1].set_xlabel('Tamaño del modelo (MB)')\n",
    "axes[1].set_ylabel('MRR')\n",
    "axes[1].set_title('Trade-off: Tamaño vs Calidad de Retrieval')\n",
    "\n",
    "# Gráfico 3: Tiempo de encoding del corpus\n",
    "encode_times = [results[n]['corpus_encode_s'] for n in models_names]\n",
    "axes[2].barh(models_names, encode_times, color=colors)\n",
    "axes[2].set_xlabel('Tiempo (s)')\n",
    "axes[2].set_title('Velocidad de encoding (25 docs)')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dafabe0d",
   "metadata": {},
   "source": [
    "## 7. Detalle por query\n",
    "\n",
    "Veamos qué documentos recupera cada modelo para cada query, para entender dónde fallan."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e6b6c58e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Q0: \"¿Cómo manejan los lenguajes de programación la memoria?\"\n",
      "  Relevantes esperados: [0, 2]\n",
      "  MiniLM-L6                 → [✓0  3  8  16 ✓2]  P=0.40 R=1.00 RR=1.00\n",
      "  MiniLM-L12                → [✓0  3  15 ✓2  14]  P=0.40 R=1.00 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓0 ✓2  14  4  7]  P=0.40 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓0  3 ✓2  4  7]  P=0.40 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓0 ✓2  3  8  18]  P=0.40 R=1.00 RR=1.00\n",
      "  BGE-m3                    → [✓0 ✓2  18  1  3]  P=0.40 R=1.00 RR=1.00\n",
      "\n",
      "Q1: \"¿Qué es el control de versiones para código fuente?\"\n",
      "  Relevantes esperados: [4]\n",
      "  MiniLM-L6                 → [✓4  16  15  6  11]  P=0.20 R=1.00 RR=1.00\n",
      "  MiniLM-L12                → [✓4  15  14  18  11]  P=0.20 R=1.00 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓4  14  0  2  23]  P=0.20 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓4  0  1  3  2]  P=0.20 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓4  16  9  12  8]  P=0.20 R=1.00 RR=1.00\n",
      "  BGE-m3                    → [✓4  1  3  0  2]  P=0.20 R=1.00 RR=1.00\n",
      "\n",
      "Q2: \"¿Cómo producen energía las plantas a partir de la luz?\"\n",
      "  Relevantes esperados: [5]\n",
      "  MiniLM-L6                 → [✓5  20  14  8  15]  P=0.20 R=1.00 RR=1.00\n",
      "  MiniLM-L12                → [✓5  20  15  8  14]  P=0.20 R=1.00 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓5  14  15  9  20]  P=0.20 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓5  8  20  13  10]  P=0.20 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓5  20  8  10  21]  P=0.20 R=1.00 RR=1.00\n",
      "  BGE-m3                    → [✓5  9  10  13  12]  P=0.20 R=1.00 RR=1.00\n",
      "\n",
      "Q3: \"Cuéntame sobre física de partículas y mecánica cuántica\"\n",
      "  Relevantes esperados: [7, 8]\n",
      "  MiniLM-L6                 → [✓8  14  20  21  13]  P=0.20 R=0.50 RR=1.00\n",
      "  MiniLM-L12                → [✓8  13  14  12  9]  P=0.20 R=0.50 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓8  6  12  18  14]  P=0.20 R=0.50 RR=1.00\n",
      "  multi-e5-small            → [✓8  6  21  5  12]  P=0.20 R=0.50 RR=1.00\n",
      "  nomic-v1.5                → [✓8  21  6  14  5]  P=0.20 R=0.50 RR=1.00\n",
      "  BGE-m3                    → [✓8  6  0  23  13]  P=0.20 R=0.50 RR=1.00\n",
      "\n",
      "Q4: \"¿Cómo funciona la fermentación del pan?\"\n",
      "  Relevantes esperados: [10, 14]\n",
      "  MiniLM-L6                 → [✓10 ✓14  12  20  13]  P=0.40 R=1.00 RR=1.00\n",
      "  MiniLM-L12                → [✓14 ✓10  12  11  15]  P=0.40 R=1.00 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓10 ✓14  12  13  11]  P=0.40 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓10 ✓14  12  13  7]  P=0.40 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓10 ✓14  12  13  19]  P=0.40 R=1.00 RR=1.00\n",
      "  BGE-m3                    → [✓10 ✓14  13  9  5]  P=0.40 R=1.00 RR=1.00\n",
      "\n",
      "Q5: \"¿Qué hace que la comida sepa mejor al cocinarla a fuego alto?\"\n",
      "  Relevantes esperados: [12, 13]\n",
      "  MiniLM-L6                 → [ 15  14 ✓12  11  10]  P=0.20 R=0.50 RR=0.33\n",
      "  MiniLM-L12                → [ 15  10  14  11  8]  P=0.00 R=0.00 RR=0.00\n",
      "  paraphrase-multi-L12      → [✓12  14  10  11 ✓13]  P=0.40 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓12  14 ✓13  10  11]  P=0.40 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓12  14  8  10  15]  P=0.20 R=0.50 RR=1.00\n",
      "  BGE-m3                    → [✓12  14  11 ✓13  0]  P=0.40 R=1.00 RR=1.00\n",
      "\n",
      "Q6: \"¿Cómo puedo hacer crecer mis ahorros con el tiempo?\"\n",
      "  Relevantes esperados: [15, 16, 19]\n",
      "  MiniLM-L6                 → [✓15  6  11  14  20]  P=0.20 R=0.33 RR=1.00\n",
      "  MiniLM-L12                → [✓15  11  14  6  8]  P=0.20 R=0.33 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓15  17 ✓16 ✓19  14]  P=0.60 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓15  18 ✓16  17  6]  P=0.40 R=0.67 RR=1.00\n",
      "  nomic-v1.5                → [✓15  14  6  12  24]  P=0.20 R=0.33 RR=1.00\n",
      "  BGE-m3                    → [✓15 ✓16  18  17  14]  P=0.40 R=0.67 RR=1.00\n",
      "\n",
      "Q7: \"¿Qué afecta la economía y los precios?\"\n",
      "  Relevantes esperados: [17, 18]\n",
      "  MiniLM-L6                 → [ 20  15  14 ✓18  6]  P=0.20 R=0.50 RR=0.25\n",
      "  MiniLM-L12                → [✓18  15  14  20  24]  P=0.20 R=0.50 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓17  19  16  15  23]  P=0.20 R=0.50 RR=1.00\n",
      "  multi-e5-small            → [✓17  23  16  9 ✓18]  P=0.40 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓17  14 ✓18  12  19]  P=0.40 R=1.00 RR=1.00\n",
      "  BGE-m3                    → [✓17 ✓18  23  21  15]  P=0.40 R=1.00 RR=1.00\n",
      "\n",
      "Q8: \"Háblame de las partes más profundas del océano\"\n",
      "  Relevantes esperados: [24]\n",
      "  MiniLM-L6                 → [✓24  6  8  23  15]  P=0.20 R=1.00 RR=1.00\n",
      "  MiniLM-L12                → [✓24  23  8  15  20]  P=0.20 R=1.00 RR=1.00\n",
      "  paraphrase-multi-L12      → [✓24  23  22  20  13]  P=0.20 R=1.00 RR=1.00\n",
      "  multi-e5-small            → [✓24  23  20  21  22]  P=0.20 R=1.00 RR=1.00\n",
      "  nomic-v1.5                → [✓24  23  13  8  21]  P=0.20 R=1.00 RR=1.00\n",
      "  BGE-m3                    → [✓24  23  21  0  9]  P=0.20 R=1.00 RR=1.00\n",
      "\n",
      "Q9: \"¿Cómo cambia la superficie de la Tierra con el tiempo?\"\n",
      "  Relevantes esperados: [21, 22]\n",
      "  MiniLM-L6                 → [ 6  15  14  20 ✓21]  P=0.20 R=0.50 RR=0.20\n",
      "  MiniLM-L12                → [ 15  6  24  14 ✓22]  P=0.20 R=0.50 RR=0.20\n",
      "  paraphrase-multi-L12      → [ 23  6 ✓21 ✓22  17]  P=0.40 R=1.00 RR=0.33\n",
      "  multi-e5-small            → [ 6 ✓21  23  8  5]  P=0.20 R=0.50 RR=0.50\n",
      "  nomic-v1.5                → [ 6  14  8  12  24]  P=0.00 R=0.00 RR=0.00\n",
      "  BGE-m3                    → [✓21  9  23  6  5]  P=0.20 R=0.50 RR=1.00\n"
     ]
    }
   ],
   "source": [
    "# Detalle: para cada query, qué recuperó cada modelo\n",
    "for qi, (query_text, relevant) in enumerate(queries):\n",
    "    print(f'\\nQ{qi}: \"{query_text}\"')\n",
    "    print(f'  Relevantes esperados: {relevant}')\n",
    "    for name in results:\n",
    "        ev = eval_results[name]\n",
    "        pq = ev['per_query'][qi]\n",
    "        retrieved = pq['retrieved']\n",
    "        hits = [f'✓{idx}' if idx in set(relevant) else f' {idx}' for idx in retrieved]\n",
    "        print(f'  {name:25s} → [{\" \".join(hits)}]  P={pq[\"precision@k\"]:.2f} R={pq[\"recall@k\"]:.2f} RR={pq[\"mrr\"]:.2f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f723a45e",
   "metadata": {},
   "source": [
    "## 8. Visualización t-SNE: ¿Cómo agrupa cada modelo los documentos?\n",
    "\n",
    "Proyectamos los embeddings del corpus a 2D con t-SNE. Cada color es una categoría. Un buen modelo debería agrupar documentos del mismo tema juntos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d6038c05",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 122672 characters]",
      "text/plain": [
       "<Figure size 1800x1200 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data",
     "transient": {}
    }
   ],
   "source": [
    "from sklearn.manifold import TSNE\n",
    "\n",
    "categories = ['Programación'] * 5 + ['Ciencia'] * 5 + ['Cocina'] * 5 + ['Finanzas'] * 5 + ['Geografía'] * 5\n",
    "cat_colors = {'Programación': '#e74c3c', 'Ciencia': '#3498db', 'Cocina': '#2ecc71', 'Finanzas': '#f39c12', 'Geografía': '#9b59b6'}\n",
    "\n",
    "n_models = len(results)\n",
    "fig, axes = plt.subplots(2, 3, figsize=(18, 12))\n",
    "axes = axes.flatten()\n",
    "\n",
    "for i, (name, res) in enumerate(results.items()):\n",
    "    embs = res['corpus_embs']\n",
    "    tsne = TSNE(n_components=2, random_state=42, perplexity=8)\n",
    "    coords = tsne.fit_transform(embs)\n",
    "\n",
    "    ax = axes[i]\n",
    "    for cat in cat_colors:\n",
    "        mask = [c == cat for c in categories]\n",
    "        ax.scatter(coords[np.array(mask), 0], coords[np.array(mask), 1],\n",
    "                   c=cat_colors[cat], label=cat, s=80, alpha=0.8, edgecolors='white', linewidth=0.5)\n",
    "    ax.set_title(f'{name} ({res[\"dim\"]}d, {res[\"n_params_m\"]}M)', fontsize=11)\n",
    "    ax.set_xticks([]); ax.set_yticks([])\n",
    "    if i == 0:\n",
    "        ax.legend(fontsize=8, loc='upper left')\n",
    "\n",
    "# Ocultar ejes sobrantes\n",
    "for j in range(i + 1, len(axes)):\n",
    "    axes[j].set_visible(False)\n",
    "\n",
    "plt.suptitle('t-SNE: Agrupación de documentos por categoría (corpus español)', fontsize=14, y=1.01)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5093816f",
   "metadata": {},
   "source": [
    "## 9. Conclusiones\n",
    "\n",
    "Ejecuta todas las celdas arriba y compara:\n",
    "- **Modelos solo-inglés** (MiniLM-L6, L12): muy rápidos y livianos, pero ¿entienden español?\n",
    "- **Multilingüe ligero** (paraphrase-multi-L12, multi-e5-small): buen balance\n",
    "- **Nomic v1.5**: más grande pero con Matryoshka — ¿compensa el tamaño extra?\n",
    "- **BGE-m3**: el más pesado — ¿la diferencia en retrieval justifica 4x el tamaño?\n",
    "\n",
    "La tabla resumen y los gráficos dan la respuesta concreta para **tu** caso de uso."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
