egutierrez
faac610745
Extrae al registry funciones del proyecto interno footprint_aurgi: - core (6): slugify_ascii, normalize_for_join, cp_provincia_es, infer_provincia_from_cp, safe_read_csv_fallback, csv_to_parquet_duckdb - geo puras (7): haversine_km, point_in_ring, point_in_polygon, point_in_polygons_bbox, polygon_bbox, extent_with_padding, distance_bucket - geo I/O (4): load_geojson_polygons, load_boundary_gdf, add_basemap_osm, add_basemap_with_timeout - valhalla client (4): valhalla_route, valhalla_isochrone, valhalla_isochrones_async, valhalla_matrix_1_to_n - datascience stats (7): trimmed_mean, geometric_mean, detect_distribution_type, best_central_tendency, summary_stats, kde_density_levels, alpha_shape_concave_hull - datascience fuzzy (3): fuzzy_merge_adaptive (rapidfuzz), words_to_dataset, remove_words_from_column - datascience viz (2): plot_kde_2d, plot_heatmap_log - infra (4): compress_pdf_ghostscript, render_table_page_pdfpages, add_header_logo, osm2pgsql_ingest - pipelines (4): setup_geo_stack_docker, compute_centers_reachability, generate_isochrones_by_zone, count_points_per_zone - types geo (4): LonLat, BBox, IsochroneRequest, Centro Incluye: - apps/footprint_geo_stack/ (PostGIS + Martin + Valhalla via docker-compose) - 131/132 tests pasan (1 skip esperado: osm2pgsql en PATH) - Issue tracker dev/issues/0052-footprint-aurgi-extraction.md - Atribucion uniforme: source_repo internal:footprint_aurgi, source_license internal-aurgi - Build con 9 agentes en paralelo (8 wave 1 + 1 wave 2 pipelines) Tambien commitea trabajo previo no commiteado: aggregate_extraction_results, chunk_with_overlap, clean_pdf_text, merge_entity_aliases, extract_graph_gliner2, extract_relations_mrebel, extract_triples_spacy_es, gliner2/mrebel/marianmt/rebel/spacy_es load_model, parse_rebel_output, translate_es_to_en, issue 0050/0051. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
61 lines
2.1 KiB
Python
61 lines
2.1 KiB
Python
"""Extraccion de entidades + relaciones en una pasada con GLiNER2."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import time
|
|
from typing import Any
|
|
|
|
|
|
def extract_graph_gliner2(
|
|
text: str,
|
|
entity_labels: list[str],
|
|
relation_labels: list | dict,
|
|
model: Any,
|
|
threshold: float = 0.3,
|
|
include_confidence: bool = False,
|
|
) -> dict:
|
|
"""Extract entities + relations using GLiNER2 with one schema pass.
|
|
|
|
Wrapper de alto nivel sobre la API de GLiNER2. Construye el schema,
|
|
ejecuta la extraccion y normaliza el resultado a un dict plano.
|
|
NO aplica post-filtrado ni coreference — eso lo hace el caller con
|
|
filter_relations_by_entity_types y merge_entity_aliases.
|
|
|
|
Args:
|
|
text: Texto a analizar. Recomendado: <= 1500 chars (pre-chunked).
|
|
entity_labels: Lista de strings con los tipos de entidad.
|
|
E.g. ["person", "organization", "location"]
|
|
relation_labels: Lista de strings o dict {label: description} con
|
|
los tipos de relacion.
|
|
E.g. ["works_at", "ceo_of"] o
|
|
{"works_at": "person works at organization"}
|
|
model: Instancia GLiNER2 cargada con gliner2_load_model.
|
|
threshold: Umbral de confianza (0-1). 0.3 es el valor validado
|
|
empiricamente en los notebooks del analisis.
|
|
include_confidence: Si True, el modelo devuelve scores por entidad
|
|
y relacion (formato interno de GLiNER2).
|
|
|
|
Returns:
|
|
{
|
|
"entities": {type: [name, ...]},
|
|
"relation_extraction": {rel_type: [(head, tail), ...]},
|
|
"elapsed_s": float
|
|
}
|
|
"""
|
|
schema = model.create_schema().entities(entity_labels).relations(relation_labels)
|
|
|
|
t0 = time.time()
|
|
r = model.extract(
|
|
text,
|
|
schema=schema,
|
|
threshold=threshold,
|
|
include_confidence=include_confidence,
|
|
)
|
|
elapsed = round(time.time() - t0, 3)
|
|
|
|
return {
|
|
"entities": r.get("entities", {}),
|
|
"relation_extraction": r.get("relation_extraction", {}),
|
|
"elapsed_s": elapsed,
|
|
}
|