egutierrez
a03675113a
- .claude/agents/fn-orquestador/SKILL.md - .claude/commands/fn_claude.md - .claude/rules/INDEX.md - .claude/rules/cpp_apps.md - .claude/rules/ids_naming.md - CHANGELOG.md - apps/dag_engine/README.md - apps/dag_engine/api.go - apps/dag_engine/dags_migrated/example.yaml - apps/dag_engine/dags_migrated/example_lineage_tracking.yaml - ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
70 lines
2.5 KiB
Markdown
70 lines
2.5 KiB
Markdown
---
|
|
name: aemet-madrid
|
|
id: 0002
|
|
status: pending
|
|
created: 2026-05-16
|
|
updated: 2026-05-16
|
|
priority: medium
|
|
risk: low
|
|
related_issues: [0097]
|
|
apps:
|
|
- dag_engine
|
|
- data_factory
|
|
- footprint_geo_stack
|
|
trigger: cron
|
|
schedule: "0 * * * *"
|
|
expected_runtime_s: 10
|
|
tags: [api, weather, geo, http-only]
|
|
---
|
|
|
|
## Goal
|
|
|
|
Probar path HTTP-only (sin Chrome/CDP). Extractor REST -> data_factory -> sink geo (PostGIS via footprint_geo_stack). Demuestra que el stack tambien sirve para APIs publicas + datos georeferenciados.
|
|
|
|
## Pre-requisitos
|
|
|
|
- API key AEMET (gratis, signup). Guardar en `pass insert aemet/api-key`.
|
|
- `footprint_geo_stack` corriendo (PostGIS :5432 + Martin tiles :3000).
|
|
- dag_engine activo.
|
|
|
|
## Flow
|
|
|
|
1. Crear funcion del registry `aemet_get_weather_py_infra` (o usar `http_get_json_py_infra` directamente si la API responde JSON simple).
|
|
2. Endpoint AEMET observacion convencional: `GET https://opendata.aemet.es/opendata/api/observacion/convencional/datos/estacion/<id>` con header `api_key`.
|
|
3. Schema esperado: `[{ts, temp, humidity, pressure, wind_speed, lat, lon}, ...]`.
|
|
4. Sink: INSERT en PostGIS tabla `weather_madrid (ts, temp, humidity, pressure, geom geometry(Point, 4326))`.
|
|
5. Crear node en data_factory: `{kind: 'database', label: 'postgis_weather'}` (sink declarado).
|
|
6. DAG `aemet_madrid_hourly.yaml`:
|
|
```yaml
|
|
name: aemet-madrid
|
|
schedule: "0 * * * *"
|
|
steps:
|
|
- name: extract
|
|
function: aemet_get_weather_py_infra
|
|
args: ["--station", "3195", "--out", "/tmp/aemet.json"]
|
|
- name: load_postgis
|
|
function: db_insert_row_go_infra
|
|
args: ["--db", "postgres://...", "--table", "weather_madrid", "--from-json", "/tmp/aemet.json"]
|
|
depends: [extract]
|
|
```
|
|
7. Verificar Martin tiles renderiza overlay (opcional).
|
|
|
|
## Acceptance
|
|
|
|
- [ ] Funcion AEMET extractor existe (creada o reusada `http_get_json_*`).
|
|
- [ ] DAG corre 2x consecutivas via scheduler.
|
|
- [ ] PostGIS tabla `weather_madrid` tiene >=2 filas.
|
|
- [ ] data_factory muestra node `aemet_madrid` kind=extractor + node `postgis_weather` kind=database con `last_seen_at` reciente.
|
|
- [ ] Martin tile server sirve overlay weather (opcional).
|
|
|
|
## Telemetria esperada
|
|
|
|
- `function_stats.http_get_json_py_infra` o `aemet_get_weather_py_infra`: calls_24h += 24 (1/hora).
|
|
- `data_factory.runs`: 24 entries/dia.
|
|
- `data_factory.databases.last_seen_at` actualizado por sink.
|
|
|
|
## Notas
|
|
|
|
- Sin LLM/CDP. Mas barato que flow 0001.
|
|
- Caso minimal para servicios geo. Si funciona, sirve de plantilla para mas extractores API.
|