feat: extraccion masiva footprint_aurgi (41 funcs + 4 types + stack Docker geo)

Extrae al registry funciones del proyecto interno footprint_aurgi: - core (6): slugify_ascii, normalize_for_join, cp_provincia_es, infer_provincia_from_cp, safe_read_csv_fallback, csv_to_parquet_duckdb - geo puras (7): haversine_km, point_in_ring, point_in_polygon, point_in_polygons_bbox, polygon_bbox, extent_with_padding, distance_bucket - geo I/O (4): load_geojson_polygons, load_boundary_gdf, add_basemap_osm, add_basemap_with_timeout - valhalla client (4): valhalla_route, valhalla_isochrone, valhalla_isochrones_async, valhalla_matrix_1_to_n - datascience stats (7): trimmed_mean, geometric_mean, detect_distribution_type, best_central_tendency, summary_stats, kde_density_levels, alpha_shape_concave_hull - datascience fuzzy (3): fuzzy_merge_adaptive (rapidfuzz), words_to_dataset, remove_words_from_column - datascience viz (2): plot_kde_2d, plot_heatmap_log - infra (4): compress_pdf_ghostscript, render_table_page_pdfpages, add_header_logo, osm2pgsql_ingest - pipelines (4): setup_geo_stack_docker, compute_centers_reachability, generate_isochrones_by_zone, count_points_per_zone - types geo (4): LonLat, BBox, IsochroneRequest, Centro Incluye: - apps/footprint_geo_stack/ (PostGIS + Martin + Valhalla via docker-compose) - 131/132 tests pasan (1 skip esperado: osm2pgsql en PATH) - Issue tracker dev/issues/0052-footprint-aurgi-extraction.md - Atribucion uniforme: source_repo internal:footprint_aurgi, source_license internal-aurgi - Build con 9 agentes en paralelo (8 wave 1 + 1 wave 2 pipelines) Tambien commitea trabajo previo no commiteado: aggregate_extraction_results, chunk_with_overlap, clean_pdf_text, merge_entity_aliases, extract_graph_gliner2, extract_relations_mrebel, extract_triples_spacy_es, gliner2/mrebel/marianmt/rebel/spacy_es load_model, parse_rebel_output, translate_es_to_en, issue 0050/0051. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 23:35:22 +02:00
parent f73ea072bd
commit faac610745
193 changed files with 13146 additions and 3 deletions
@@ -0,0 +1,97 @@
+"""Ingest an OSM PBF file into a PostGIS database using osm2pgsql."""
+from __future__ import annotations
+
+import os
+import shutil
+import subprocess
+from pathlib import Path
+
+
+def osm2pgsql_ingest(
+    osm_pbf_path: "str | Path",
+    host: str = "localhost",
+    port: int = 5432,
+    dbname: str = "gis",
+    user: str = "geoserver",
+    password: str = "geoserver",
+    style: "str | None" = None,
+    ensure_hstore: bool = True,
+) -> dict:
+    """Ingest an OSM PBF file into PostGIS using osm2pgsql.
+
+    Verifies osm2pgsql is in PATH (raises RuntimeError if not). If
+    ensure_hstore=True, runs psql to CREATE EXTENSION IF NOT EXISTS hstore.
+    Then runs osm2pgsql with --create --slim --hstore --multi-geometry.
+
+    Args:
+        osm_pbf_path: Path to the .osm.pbf file to ingest.
+        host: PostGIS host (default: localhost).
+        port: PostGIS port (default: 5432).
+        dbname: Database name (default: gis).
+        user: Database user (default: geoserver).
+        password: Database password (default: geoserver).
+        style: Optional path to a .style file for osm2pgsql. If None, uses
+               osm2pgsql's built-in default style.
+        ensure_hstore: If True, create the hstore extension before ingesting.
+
+    Returns:
+        dict with keys:
+          - ok (bool): True if ingestion succeeded.
+          - rows_loaded (int | None): Not directly available from osm2pgsql stdout;
+            always None (osm2pgsql does not report row counts).
+          - stderr (str): Combined stdout+stderr output from osm2pgsql.
+
+    Raises:
+        RuntimeError: If osm2pgsql is not found in PATH.
+        FileNotFoundError: If osm_pbf_path does not exist.
+    """
+    pbf = Path(osm_pbf_path)
+    if not pbf.exists():
+        raise FileNotFoundError(f"OSM PBF file not found: {pbf}")
+
+    if shutil.which("osm2pgsql") is None:
+        raise RuntimeError(
+            "osm2pgsql not found in PATH. Install it before calling this function."
+        )
+
+    env = os.environ.copy()
+    env["PGPASSWORD"] = password
+
+    if ensure_hstore and shutil.which("psql") is not None:
+        psql_cmd = [
+            "psql",
+            f"--host={host}",
+            f"--port={port}",
+            f"--dbname={dbname}",
+            f"--username={user}",
+            "--command",
+            "CREATE EXTENSION IF NOT EXISTS hstore;",
+        ]
+        subprocess.run(psql_cmd, env=env, capture_output=True, text=True)
+
+    cmd = [
+        "osm2pgsql",
+        f"--host={host}",
+        f"--port={port}",
+        f"--database={dbname}",
+        f"--user={user}",
+        "--create",
+        "--slim",
+        "--hstore",
+        "--multi-geometry",
+    ]
+    if style:
+        cmd += ["--style", str(style)]
+    cmd.append(str(pbf))
+
+    result = subprocess.run(
+        cmd,
+        env=env,
+        capture_output=True,
+        text=True,
+    )
+
+    combined = result.stdout + result.stderr
+    if result.returncode == 0:
+        return {"ok": True, "rows_loaded": None, "stderr": combined}
+    return {"ok": False, "rows_loaded": None, "stderr": combined}