egutierrez
837563c3ba
Datascience: aggregate_by_group, deduplicate_entities/relations, detect_drift, diff_entities/relations, extract_entities/relations_llm, hotness_score, melt, merge_graphs, pivot, build_entity/relation_schema_prompt. Finance: avellaneda_stoikov_quotes, generate_gbm_prices, generate_taker_order, hawkes_intensity + módulo finance.py. Cybersecurity: envelope_encrypt/decrypt + módulo cybersecurity.py. Pipelines: extraction_pipeline, monte_carlo_market, run_market_sim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
69 lines
2.8 KiB
Markdown
69 lines
2.8 KiB
Markdown
---
|
|
name: merge_graphs
|
|
kind: function
|
|
lang: py
|
|
domain: datascience
|
|
version: "1.0.0"
|
|
purity: pure
|
|
signature: "def merge_graphs(graphs: list[dict], entity_key: str = 'name', similarity_threshold: float = 0.85) -> dict"
|
|
description: "Mergea multiples grafos de conocimiento en uno deduplicando entities por similitud de nombre (Levenshtein normalizado). Relaciones se re-apuntan a las entities canonicas. Atributos se combinan por union."
|
|
tags: [graph, merge, deduplication, knowledge-graph, levenshtein, similarity, datascience]
|
|
uses_functions: [levenshtein_distance_py_cybersecurity]
|
|
uses_types: []
|
|
returns: []
|
|
returns_optional: false
|
|
error_type: ""
|
|
imports: [sys, os]
|
|
tested: true
|
|
tests:
|
|
- "dos grafos con entity duplicada → merge"
|
|
- "entities similares pero bajo threshold → no merge"
|
|
- "relaciones re-apuntadas correctamente"
|
|
- "merge log registra cada merge"
|
|
- "tres grafos → merge transitivo"
|
|
- "grafos sin overlap → concatenacion simple"
|
|
test_file_path: "python/functions/datascience/merge_graphs_test.py"
|
|
file_path: "python/functions/datascience/merge_graphs.py"
|
|
---
|
|
|
|
## Ejemplo
|
|
|
|
```python
|
|
g1 = {
|
|
"entities": [
|
|
{"id": "1", "name": "Alice Corp", "type": "company"},
|
|
{"id": "2", "name": "Bob", "type": "person"},
|
|
],
|
|
"relations": [
|
|
{"source_id": "2", "target_id": "1", "relation_type": "works_at"},
|
|
],
|
|
}
|
|
g2 = {
|
|
"entities": [
|
|
{"id": "3", "name": "Alice Corp.", "type": "company", "country": "US"},
|
|
],
|
|
"relations": [],
|
|
}
|
|
|
|
result = merge_graphs([g1, g2], similarity_threshold=0.85)
|
|
# result["entities"] -> 2 entities (Alice Corp mergeada, Bob)
|
|
# result["merge_log"] -> [{"merged": ["3", "1"], "into": "1", "similarity": 0.909}]
|
|
# "Alice Corp." mergeada en "Alice Corp" porque similitud > 0.85
|
|
```
|
|
|
|
## Notas
|
|
|
|
Funcion pura. Reutiliza `levenshtein_distance_py_cybersecurity` para calcular similitud normalizada entre nombres.
|
|
|
|
**Algoritmo de merge transitivo**: si A~B y B~C, entonces A, B, C se mergean en uno solo. Se implementa via union-find (path compression simple).
|
|
|
|
**Eleccion de canonical**: la entity con mas campos no-null gana. En caso de empate, la primera encontrada en el par.
|
|
|
|
**Conflictos de atributos**: si ambas entities tienen un campo con valor, el canonical conserva el suyo (primero gana). Solo se copian campos que el canonical no tiene o tiene null.
|
|
|
|
**Deduplicacion de relaciones**: por (source_id, target_id, relation_type). Si dos relaciones son identicas tras re-apuntar los IDs, se conserva la primera encontrada.
|
|
|
|
**Complejidad**: O(n^2) en numero de entities por la comparacion de pares. Adecuado para grafos de knowledge tipicos (< 10K entities). Para grafos muy grandes, usar indexado por prefijo antes de comparar.
|
|
|
|
**Importacion**: intenta importar `levenshtein_distance` desde el paquete `cybersecurity` del registry. Si no esta disponible, usa una reimplementacion inline equivalente.
|