egutierrez
837563c3ba
Datascience: aggregate_by_group, deduplicate_entities/relations, detect_drift, diff_entities/relations, extract_entities/relations_llm, hotness_score, melt, merge_graphs, pivot, build_entity/relation_schema_prompt. Finance: avellaneda_stoikov_quotes, generate_gbm_prices, generate_taker_order, hawkes_intensity + módulo finance.py. Cybersecurity: envelope_encrypt/decrypt + módulo cybersecurity.py. Pipelines: extraction_pipeline, monte_carlo_market, run_market_sim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2.2 KiB
2.2 KiB
name, kind, lang, domain, version, purity, signature, description, tags, uses_functions, uses_types, returns, returns_optional, error_type, imports, tested, tests, test_file_path, file_path
| name | kind | lang | domain | version | purity | signature | description | tags | uses_functions | uses_types | returns | returns_optional | error_type | imports | tested | tests | test_file_path | file_path | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| detect_drift | function | py | datascience | 1.0.0 | pure | def detect_drift(history: list[dict], current: dict, fields: list[str], threshold: float = 2.0) -> list[dict] | Detecta drift estadistico comparando metricas de la ejecucion actual contra el historial usando z-score. Si |z| > threshold, el campo ha drifteado. Util para monitorizar executions en operations.db. |
|
false |
|
true |
|
python/functions/datascience/detect_drift_test.py | python/functions/datascience/detect_drift.py |
Ejemplo
history = [
{"records_out": 100, "duration_ms": 500},
{"records_out": 105, "duration_ms": 480},
{"records_out": 98, "duration_ms": 510},
]
current = {"records_out": 50, "duration_ms": 2000}
results = detect_drift(history, current, ["records_out", "duration_ms"])
# [
# {"field": "records_out", "current": 50, "mean": 101.0, "std": 3.6, "z_score": -14.2, "drifted": True},
# {"field": "duration_ms", "current": 2000, "mean": 496.7, "std": 15.3, "z_score": 98.3, "drifted": True},
# ]
Notas
Funcion pura. Solo stdlib (math).
El z-score usa desviacion estandar poblacional (dividir por N, no N-1) para ser consistente con historial de cualquier tamanio.
Casos especiales:
- Historial vacio: z_score=0.0, drifted=False para todos los campos.
- Un solo punto en historial: std=0.0, z_score=0.0, drifted=False. No hay suficiente historia para calcular variabilidad.
- Std=0 con N>=2: todos los valores historicos identicos. z_score=0.0, drifted=False (cualquier desviacion seria tecnicamente infinita, pero se asume que el sistema es muy estable).
Pensado para el paso ANALIZAR del bucle reactivo: comparar metrics de la ejecucion actual con executions historicas de operations.db.