97 lines
3.9 KiB
Markdown
97 lines
3.9 KiB
Markdown
---
|
|
name: aemet-madrid
|
|
id: 0002
|
|
status: pending
|
|
created: 2026-05-16
|
|
updated: 2026-05-16
|
|
priority: medium
|
|
risk: low
|
|
related_issues: [0097]
|
|
apps:
|
|
- dag_engine
|
|
- data_factory
|
|
- footprint_geo_stack
|
|
trigger: cron
|
|
schedule: "0 * * * *"
|
|
expected_runtime_s: 10
|
|
tags: [api, weather, geo, http-only]
|
|
---
|
|
|
|
## Goal
|
|
|
|
Probar path HTTP-only (sin Chrome/CDP). Extractor REST -> data_factory -> sink geo (PostGIS via footprint_geo_stack). Demuestra que el stack tambien sirve para APIs publicas + datos georeferenciados.
|
|
|
|
## Pre-requisitos
|
|
|
|
- API key AEMET (gratis, signup). Guardar en `pass insert aemet/api-key`.
|
|
- `footprint_geo_stack` corriendo (PostGIS :5432 + Martin tiles :3000).
|
|
- dag_engine activo.
|
|
|
|
## Flow
|
|
|
|
1. Crear funcion del registry `aemet_get_weather_py_infra` (o usar `http_get_json_py_infra` directamente si la API responde JSON simple).
|
|
2. Endpoint AEMET observacion convencional: `GET https://opendata.aemet.es/opendata/api/observacion/convencional/datos/estacion/<id>` con header `api_key`.
|
|
3. Schema esperado: `[{ts, temp, humidity, pressure, wind_speed, lat, lon}, ...]`.
|
|
4. Sink: INSERT en PostGIS tabla `weather_madrid (ts, temp, humidity, pressure, geom geometry(Point, 4326))`.
|
|
5. Crear node en data_factory: `{kind: 'database', label: 'postgis_weather'}` (sink declarado).
|
|
6. DAG `aemet_madrid_hourly.yaml`:
|
|
```yaml
|
|
name: aemet-madrid
|
|
schedule: "0 * * * *"
|
|
steps:
|
|
- name: extract
|
|
function: aemet_get_weather_py_infra
|
|
args: ["--station", "3195", "--out", "/tmp/aemet.json"]
|
|
- name: load_postgis
|
|
function: db_insert_row_go_infra
|
|
args: ["--db", "postgres://...", "--table", "weather_madrid", "--from-json", "/tmp/aemet.json"]
|
|
depends: [extract]
|
|
```
|
|
7. Verificar Martin tiles renderiza overlay (opcional).
|
|
|
|
## Acceptance
|
|
|
|
- [ ] Funcion AEMET extractor existe (creada o reusada `http_get_json_*`).
|
|
- [ ] DAG corre 2x consecutivas via scheduler.
|
|
- [ ] PostGIS tabla `weather_madrid` tiene >=2 filas.
|
|
- [ ] data_factory muestra node `aemet_madrid` kind=extractor + node `postgis_weather` kind=database con `last_seen_at` reciente.
|
|
- [ ] Martin tile server sirve overlay weather (opcional).
|
|
|
|
## Telemetria esperada
|
|
|
|
- `function_stats.http_get_json_py_infra` o `aemet_get_weather_py_infra`: calls_24h += 24 (1/hora).
|
|
- `data_factory.runs`: 24 entries/dia.
|
|
- `data_factory.databases.last_seen_at` actualizado por sink.
|
|
|
|
## Definition of Done
|
|
|
|
Ver `README.md` seccion DoD + user-facing.
|
|
|
|
### Generico
|
|
|
|
- [ ] **Repetibilidad**: cron `0 * * * *` corre 3h consecutivas sin error.
|
|
- [ ] **Observabilidad**: extractor en `call_monitor.calls`, runs en `data_factory.runs`, fila en `databases.last_seen_at`.
|
|
- [ ] **Error-path**: AEMET 5xx → 3 reintentos exp-backoff, despues marca run failed (no crash).
|
|
- [ ] **Idempotencia**: re-run mismo timestamp = upsert PostGIS, sin duplicar puntos.
|
|
- [ ] **Secrets**: API key AEMET en `pass aemet/api-key`, nunca en el DAG.
|
|
- [ ] **Docs**: `## Notas` con comandos + onboarding.
|
|
- [ ] **Registry-first**: extractor AEMET creado como funcion del registry (`aemet_get_madrid_observations_py_*` o reuso de `http_get_json_*`), nada inline.
|
|
- [ ] **INDEX + status**: `status: done` + INDEX + movido.
|
|
|
|
### User-facing
|
|
|
|
- [ ] **User-facing**: usuario abre `footprint_geo_stack` → preset `madrid-weather` → ve overlay tiles con puntos meteo + tooltip (temp/humidity).
|
|
- [ ] **User-facing repeat**: mismo preset manana muestra datos refrescados; tooltip ultima hora.
|
|
- [ ] **User-facing onboarding**: parrafo en `## Notas`: "Para ver weather Madrid: `footprint_geo_stack.exe` → File → Open preset `madrid-weather`. Tile server local en :3000."
|
|
- [ ] **User-facing latencia**: cron 1h → mapa refleja datos en <61 min.
|
|
|
|
### Custom
|
|
|
|
- [ ] PostGIS schema via `migrations/NNN_*.sql` (no `CREATE TABLE` inline).
|
|
- [ ] Tile overlay sirve en <3s desde click.
|
|
|
|
## Notas
|
|
|
|
- Sin LLM/CDP. Mas barato que flow 0001.
|
|
- Caso minimal para servicios geo. Si funciona, sirve de plantilla para mas extractores API.
|