# Extraction Pipeline (orquestador)

Diseño original para fuzzygraph.

## Pipeline a implementar

### extraction_pipeline

- **Dominio:** pipelines
- **Lang:** Python
- **Kind:** pipeline
- **Purity:** impure
- **Signature:**
  ```python
  def extraction_pipeline(
      file_path: str,
      entity_presets: list[dict],
      relation_types: list[str],
      llm_chat_json: Callable[[list[dict]], dict],
      chunk_size: int = 500,
      chunk_overlap: int = 50,
      confidence_threshold: float = 0.5,
      dedup_threshold: float = 0.85,
      on_progress: Callable[[str, float], None] | None = None,
  ) -> ExtractionResult:
  ```
- **Descripcion:** Pipeline completa de extraccion de entidades y relaciones desde un documento. Orquesta todas las funciones del stack.
- **uses_functions:**
  - `extract_text_from_file` (mf_09)
  - `preprocess_text` (mf_02)
  - `split_text_into_chunks` (mf_01)
  - `build_entity_schema_prompt` (fg_05)
  - `build_relation_schema_prompt` (fg_05)
  - `extract_entities_llm` (fg_01)
  - `extract_relations_llm` (fg_02)
  - `deduplicate_entities` (fg_03)
  - `deduplicate_relations` (fg_04)
- **Algoritmo:**
  1. **Extract:** `extract_text_from_file(file_path)` → texto crudo
  2. **Preprocess:** `preprocess_text(text)` → texto limpio
  3. **Split:** `split_text_into_chunks(text, chunk_size, chunk_overlap)` → chunks
  4. **Extract entities per chunk:** Para cada chunk (con retry):
     - `extract_entities_llm(chunk, entity_presets, llm_chat_json)`
     - Anotar `source_chunk_index` en cada candidato
     - progress callback: `(f"Extracting entities from chunk {i}/{n}", i/n * 0.4)`
  5. **Flatten + filter:** Juntar todas las entidades, filtrar por `confidence >= confidence_threshold`
  6. **Deduplicate entities:** `deduplicate_entities(all_entities, dedup_threshold)`
  7. **Extract relations per chunk:** Para cada chunk:
     - Obtener entidades relevantes de ese chunk
     - `extract_relations_llm(chunk, chunk_entities, relation_types, llm_chat_json)`
     - progress callback: `("Extracting relations...", 0.4 + i/n * 0.4)`
  8. **Deduplicate relations:** `deduplicate_relations(all_relations, entity_id_map)`
  9. **Return** ExtractionResult con entities, relations, stats
- **Progress:** 0-40% entity extraction, 40-80% relation extraction, 80-100% dedup
- **Error type:** FileNotFoundError, ValueError
- **Tests:** documento con entidades y relaciones, documento vacio, documento sin entidades detectables

## Integracion con fuzzygraph

La pipeline retorna `ExtractionResult` que contiene listas de entidades y relaciones listas para insertar en `operations.db` via `fn ops entity add` / `fn ops relation add`. El mapping a fuzzygraph seria:

```python
for entity in result.entities:
    # entity.name → ops entity name
    # entity.type_ref → ops entity type_ref
    # entity.attributes → ops entity metadata (JSON)
    db.add_entity(name=entity.name, type_ref=entity.type_ref, metadata=entity.attributes)

for relation in result.relations:
    db.add_relation(name=relation.relation_type, from_entity=relation.from_id, to_entity=relation.to_id)
```
