feat: notebook 04 — JetStream a fondo + simulador de rendimiento interactivo
JetStream: anatomia de streams (storage/retention/limits), consumers pull durables con ack y cursor, dedup por Nats-Msg-Id, retencion workqueue, deliver policies. Simulador: boton ipywidgets que lanza 1 publisher -> N subscribers con miles de mensajes y grafica en movimiento (acumulado + throughput instantaneo).
This commit is contained in:
Binary file not shown.
@@ -6,7 +6,17 @@
|
|||||||
},
|
},
|
||||||
"17291f8f-336e-4a5f-8407-a4d22149d581": {
|
"17291f8f-336e-4a5f-8407-a4d22149d581": {
|
||||||
"version": "2.4.0",
|
"version": "2.4.0",
|
||||||
"created_at": "2026-06-03T17:46:15.674895+00:00",
|
"created_at": "2026-06-03T19:04:11.779086+00:00",
|
||||||
|
"document_version": "2.0.0"
|
||||||
|
},
|
||||||
|
"6a0e97d4-34d4-44f3-8a47-243934257256": {
|
||||||
|
"version": "2.4.0",
|
||||||
|
"created_at": "2026-06-03T19:16:39.520669+00:00",
|
||||||
|
"document_version": "2.0.0"
|
||||||
|
},
|
||||||
|
"bc0d99c9-d3bf-49c2-a87a-b38af47d1bff": {
|
||||||
|
"version": "2.4.0",
|
||||||
|
"created_at": "2026-06-03T19:18:34.134263+00:00",
|
||||||
"document_version": "2.0.0"
|
"document_version": "2.0.0"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Binary file not shown.
@@ -1,12 +1,16 @@
|
|||||||
{
|
{
|
||||||
"mcpServers": {
|
"mcpServers": {
|
||||||
"jupyter": {
|
"jupyter": {
|
||||||
"command": "/home/enmanuel/fn_registry/analysis/nats/.venv/bin/jupyter-mcp-server",
|
"command": "bash",
|
||||||
"args": [
|
"args": [
|
||||||
"--transport", "stdio",
|
"/home/enmanuel/fn_registry/bash/functions/infra/jupyter_mcp_serve.sh"
|
||||||
"--jupyter-url", "http://localhost:8890",
|
],
|
||||||
"--jupyter-token", ""
|
"env": {
|
||||||
]
|
"JUPYTER_MCP_VENV": "/home/enmanuel/fn_registry/analysis/nats/.venv",
|
||||||
|
"JUPYTER_MCP_ROOT": "/home/enmanuel/fn_registry/analysis/nats",
|
||||||
|
"JUPYTER_MCP_PORT": "8890",
|
||||||
|
"JUPYTER_MCP_TOKEN": ""
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -21,9 +21,12 @@ Analisis didactico de **NATS** como sistema de mensajeria pub/sub entre procesos
|
|||||||
| `01_core_pubsub.ipynb` | Modelo base: conexion, publish/subscribe, fan-out a N subscribers, wildcards `*` y `>`. |
|
| `01_core_pubsub.ipynb` | Modelo base: conexion, publish/subscribe, fan-out a N subscribers, wildcards `*` y `>`. |
|
||||||
| `02_queue_request_jetstream.ipynb` | Queue groups (reparto de carga), request/reply (RPC con inbox temporal), JetStream (stream persistente + consumer durable + replay). |
|
| `02_queue_request_jetstream.ipynb` | Queue groups (reparto de carga), request/reply (RPC con inbox temporal), JetStream (stream persistente + consumer durable + replay). |
|
||||||
| `03_procesos_reales.ipynb` | Publisher y subscribers como **procesos del SO independientes** (`subprocess`), cada uno con su PID. Demuestra el desacople real: el publisher no conoce a sus subscribers. |
|
| `03_procesos_reales.ipynb` | Publisher y subscribers como **procesos del SO independientes** (`subprocess`), cada uno con su PID. Demuestra el desacople real: el publisher no conoce a sus subscribers. |
|
||||||
|
| `04_jetstream_benchmark.ipynb` | **JetStream a fondo** (storage/retention/limits, consumers durables + ack + cursor, dedup por `Nats-Msg-Id`, retención `workqueue`, deliver policies) + **simulador de rendimiento interactivo**: un botón `ipywidgets` que lanza 1 publisher → N subscribers con miles de mensajes y una gráfica en movimiento (acumulado pub vs subs + throughput instantáneo). |
|
||||||
|
|
||||||
Los scripts `notebooks/procs/publisher.py` y `notebooks/procs/subscriber.py` son los programas que el notebook 03 lanza como procesos reales.
|
Los scripts `notebooks/procs/publisher.py` y `notebooks/procs/subscriber.py` son los programas que el notebook 03 lanza como procesos reales.
|
||||||
|
|
||||||
|
El notebook 04 requiere `ipywidgets` (incluido en el `.venv` del análisis). El simulador es interactivo: al abrir el notebook en JupyterLab, ejecuta sus celdas hasta el widget y pulsa **▶ Ejecutar benchmark** (los sliders ajustan número de mensajes y de subscribers). La gráfica se anima mientras corre.
|
||||||
|
|
||||||
### Como usar
|
### Como usar
|
||||||
|
|
||||||
1. Requiere Docker disponible (con permisos para el usuario actual). La primera celda de cada notebook arranca el broker de forma idempotente con la funcion `ensure_nats`, asi que cada notebook funciona de forma aislada.
|
1. Requiere Docker disponible (con permisos para el usuario actual). La primera celda de cada notebook arranca el broker de forma idempotente con la funcion `ensure_nats`, asi que cada notebook funciona de forma aislada.
|
||||||
|
|||||||
@@ -0,0 +1,435 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Generador del notebook 04 del analisis NATS: JetStream a fondo + simulador de
|
||||||
|
rendimiento interactivo.
|
||||||
|
|
||||||
|
Construye el .ipynb con nbformat (sin ejecutar). La ejecucion se hace despues
|
||||||
|
contra el servidor Jupyter del analisis (puerto 8890, su propio venv) para que
|
||||||
|
los outputs queden persistidos y el widget interactivo se renderice en JupyterLab.
|
||||||
|
|
||||||
|
Reproducible: re-ejecutar este script regenera el notebook 04 desde cero.
|
||||||
|
"""
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import nbformat as nbf
|
||||||
|
|
||||||
|
NBDIR = Path("/home/enmanuel/fn_registry/analysis/nats/notebooks")
|
||||||
|
|
||||||
|
|
||||||
|
def build(filename: str, cells: list[tuple[str, str]]) -> None:
|
||||||
|
nb = nbf.v4.new_notebook()
|
||||||
|
nb.metadata["kernelspec"] = {
|
||||||
|
"name": "python3",
|
||||||
|
"display_name": "Python 3 (ipykernel)",
|
||||||
|
"language": "python",
|
||||||
|
}
|
||||||
|
nb.cells = [
|
||||||
|
nbf.v4.new_markdown_cell(src) if typ == "md" else nbf.v4.new_code_cell(src)
|
||||||
|
for typ, src in cells
|
||||||
|
]
|
||||||
|
nbf.write(nb, str(NBDIR / filename))
|
||||||
|
print(f"escrito {filename}: {len(cells)} celdas")
|
||||||
|
|
||||||
|
|
||||||
|
ENSURE_NATS = '''import subprocess, time, json
|
||||||
|
|
||||||
|
NATS_CONTAINER = "nats_demo"
|
||||||
|
NATS_PORT = 4222
|
||||||
|
NATS_URL = f"nats://127.0.0.1:{NATS_PORT}"
|
||||||
|
|
||||||
|
def _docker(*args, check=True):
|
||||||
|
return subprocess.run(["docker", *args], capture_output=True, text=True, check=check)
|
||||||
|
|
||||||
|
def ensure_nats(name=NATS_CONTAINER, port=NATS_PORT):
|
||||||
|
"""Arranca un broker NATS en Docker de forma idempotente. Devuelve el estado."""
|
||||||
|
out = _docker("ps", "-a", "--filter", f"name=^{name}$", "--format", "{{.State}}", check=False).stdout.strip()
|
||||||
|
if out == "running":
|
||||||
|
state = "already-running"
|
||||||
|
elif out in ("exited", "created", "paused"):
|
||||||
|
_docker("start", name)
|
||||||
|
state = "started"
|
||||||
|
else:
|
||||||
|
_docker("run", "-d", "--name", name, "-p", f"{port}:4222", "-p", "8222:8222",
|
||||||
|
"nats:latest", "-js", "-m", "8222")
|
||||||
|
state = "created"
|
||||||
|
time.sleep(1.0)
|
||||||
|
return state'''
|
||||||
|
|
||||||
|
|
||||||
|
nb4 = [
|
||||||
|
("md", """# NATS pub/sub — 04 · JetStream a fondo y simulador de rendimiento
|
||||||
|
|
||||||
|
Este notebook tiene dos partes:
|
||||||
|
|
||||||
|
1. **JetStream a fondo** — más allá del replay básico del notebook 02: anatomía de un stream (almacenamiento, políticas de retención, límites), tipos de consumer, *acks*, deduplicación y políticas de entrega.
|
||||||
|
2. **Simulador de rendimiento interactivo** — un botón que, al pulsarlo, lanza un publisher que envía **miles de mensajes** a varios subscribers, con una **gráfica en movimiento** que muestra el throughput en tiempo real.
|
||||||
|
|
||||||
|
> Requiere el broker `nats_demo` (arrancado por la primera celda) y `ipywidgets` (incluido en el venv del análisis)."""),
|
||||||
|
|
||||||
|
# ---- Parte A: JetStream a fondo ----
|
||||||
|
("md", """## Parte A · JetStream a fondo
|
||||||
|
|
||||||
|
### Setup
|
||||||
|
|
||||||
|
JetStream es la capa de persistencia de NATS. Mientras el core es *fire-and-forget*, JetStream **almacena** los mensajes en un *stream* y permite leerlos con *consumers* que controlan el ritmo, confirman (*ack*) cada mensaje y pueden reproducir el historial."""),
|
||||||
|
|
||||||
|
("code", ENSURE_NATS + '''
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import nats
|
||||||
|
|
||||||
|
print("Broker:", ensure_nats())
|
||||||
|
nc = await nats.connect(NATS_URL, name="notebook-04")
|
||||||
|
js = nc.jetstream()
|
||||||
|
print("JetStream context listo. account info:")
|
||||||
|
ai = await js.account_info()
|
||||||
|
print(f" streams={ai.streams} consumers={ai.consumers} memory={ai.memory} storage={ai.storage}")'''),
|
||||||
|
|
||||||
|
("md", """### 1 · Anatomía de un stream
|
||||||
|
|
||||||
|
Un **stream** se define por:
|
||||||
|
|
||||||
|
- **subjects** — qué subjects captura (`pedidos.>`).
|
||||||
|
- **storage** — `file` (persistente en disco) o `memory` (rápido, se pierde al reiniciar).
|
||||||
|
- **retention** — cuándo se descartan los mensajes:
|
||||||
|
- `limits` (por defecto): se guardan hasta tocar un límite (`max_msgs`, `max_bytes`, `max_age`).
|
||||||
|
- `interest`: se descartan cuando todos los consumers interesados los han recibido.
|
||||||
|
- `workqueue`: cada mensaje se borra en cuanto **un** consumer lo confirma (cola de trabajo).
|
||||||
|
- **límites** — `max_msgs`, `max_bytes`, `max_age` (segundos), `max_msg_size`.
|
||||||
|
- **duplicate_window** — ventana de deduplicación (ver §3).
|
||||||
|
|
||||||
|
Creamos un stream `limits` con almacenamiento en disco y un tope de mensajes."""),
|
||||||
|
|
||||||
|
("code", '''from nats.js.api import StreamConfig, RetentionPolicy, StorageType, DiscardPolicy
|
||||||
|
|
||||||
|
# Recrear limpio para que la demo sea determinista
|
||||||
|
for s in ("DEMO_LIMITS", "DEMO_DEDUP", "DEMO_WQ"):
|
||||||
|
try:
|
||||||
|
await js.delete_stream(s)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
cfg = StreamConfig(
|
||||||
|
name="DEMO_LIMITS",
|
||||||
|
subjects=["demo.limits.>"],
|
||||||
|
storage=StorageType.FILE,
|
||||||
|
retention=RetentionPolicy.LIMITS,
|
||||||
|
max_msgs=1000, # tope de mensajes
|
||||||
|
max_age=3600, # 1 hora (segundos)
|
||||||
|
discard=DiscardPolicy.OLD, # al llegar al tope, descarta los más viejos
|
||||||
|
duplicate_window=120, # ventana de dedup: 120 s
|
||||||
|
)
|
||||||
|
info = await js.add_stream(cfg)
|
||||||
|
c = info.config
|
||||||
|
print("Stream creado:")
|
||||||
|
print(f" name : {c.name}")
|
||||||
|
print(f" subjects : {c.subjects}")
|
||||||
|
print(f" storage : {c.storage}")
|
||||||
|
print(f" retention : {c.retention}")
|
||||||
|
print(f" max_msgs : {c.max_msgs}")
|
||||||
|
print(f" max_age (s) : {c.max_age}")
|
||||||
|
print(f" discard : {c.discard}")
|
||||||
|
print(f" dup_window (s): {c.duplicate_window}")'''),
|
||||||
|
|
||||||
|
("md", """### 2 · Consumers: pull, durable, ack
|
||||||
|
|
||||||
|
Un **consumer** es la vista de lectura sobre un stream. Dos ejes:
|
||||||
|
|
||||||
|
- **pull vs push**: en *pull* el cliente pide mensajes cuando quiere (`fetch`); en *push* el servidor los empuja según llegan.
|
||||||
|
- **durable vs ephemeral**: un consumer *durable* tiene nombre y **recuerda su posición** (cursor) entre reconexiones; uno *ephemeral* desaparece al cerrarse.
|
||||||
|
|
||||||
|
El **ack** es la confirmación de procesado. Hasta que un mensaje no se confirma, el consumer lo considera *pendiente* y, si pasa el `ack_wait`, lo **reentrega**. Esto da entrega *at-least-once*."""),
|
||||||
|
|
||||||
|
("code", '''# Publicar 6 mensajes en el stream de límites
|
||||||
|
for i in range(6):
|
||||||
|
await js.publish("demo.limits.eventos", f"evento-{i}".encode())
|
||||||
|
|
||||||
|
# Pull consumer DURABLE: recuerda su cursor entre fetches
|
||||||
|
psub = await js.pull_subscribe("demo.limits.>", durable="procesador-A")
|
||||||
|
|
||||||
|
# Traer 4 y confirmarlos (ack)
|
||||||
|
batch = await psub.fetch(4, timeout=2)
|
||||||
|
print("Primer fetch (4 msgs):")
|
||||||
|
for m in batch:
|
||||||
|
print(f" seq={m.metadata.sequence.stream} {m.data.decode()}")
|
||||||
|
await m.ack()
|
||||||
|
|
||||||
|
# Estado del consumer: cuántos quedan pendientes / entregados
|
||||||
|
ci = await psub.consumer_info()
|
||||||
|
print()
|
||||||
|
print(f"Consumer 'procesador-A':")
|
||||||
|
print(f" num_pending : {ci.num_pending} (mensajes sin entregar todavía)")
|
||||||
|
print(f" num_ack_pending: {ci.num_ack_pending} (entregados sin ack)")
|
||||||
|
print(f" delivered.stream_seq: {ci.delivered.stream_seq}")
|
||||||
|
|
||||||
|
# Segundo fetch: continúa donde se quedó (recuerda el cursor)
|
||||||
|
batch2 = await psub.fetch(10, timeout=1)
|
||||||
|
print()
|
||||||
|
print(f"Segundo fetch: {len(batch2)} msgs restantes ->", [m.data.decode() for m in batch2])
|
||||||
|
for m in batch2:
|
||||||
|
await m.ack()'''),
|
||||||
|
|
||||||
|
("md", """### 3 · Deduplicación por `Nats-Msg-Id`
|
||||||
|
|
||||||
|
Si un publisher reintenta por un timeout de red, podría enviar el mismo mensaje dos veces. JetStream lo evita: si dos publicaciones llevan el mismo header **`Nats-Msg-Id`** dentro de la `duplicate_window`, la segunda se reconoce como **duplicada** y **no** se almacena. El `PubAck` lo indica con `duplicate=True`."""),
|
||||||
|
|
||||||
|
("code", '''await js.add_stream(name="DEMO_DEDUP", subjects=["demo.dedup.>"],
|
||||||
|
storage=StorageType.FILE, duplicate_window=120)
|
||||||
|
|
||||||
|
# Publicar dos veces el MISMO Nats-Msg-Id
|
||||||
|
ack1 = await js.publish("demo.dedup.pago", b"cobro 50e", headers={"Nats-Msg-Id": "pago-0001"})
|
||||||
|
ack2 = await js.publish("demo.dedup.pago", b"cobro 50e", headers={"Nats-Msg-Id": "pago-0001"})
|
||||||
|
|
||||||
|
print(f"1a publicacion: seq={ack1.seq} duplicate={ack1.duplicate}")
|
||||||
|
print(f"2a publicacion: seq={ack2.seq} duplicate={ack2.duplicate} <- detectada como duplicado")
|
||||||
|
|
||||||
|
# Un Msg-Id distinto sí se almacena
|
||||||
|
ack3 = await js.publish("demo.dedup.pago", b"cobro 30e", headers={"Nats-Msg-Id": "pago-0002"})
|
||||||
|
print(f"3a publicacion (id nuevo): seq={ack3.seq} duplicate={ack3.duplicate}")
|
||||||
|
|
||||||
|
st = (await js.stream_info("DEMO_DEDUP")).state
|
||||||
|
print()
|
||||||
|
print(f"Mensajes realmente almacenados en el stream: {st.messages} (2 publicaciones unicas, 1 descartada)")'''),
|
||||||
|
|
||||||
|
("md", """### 4 · Retención `workqueue`: la cola de trabajo
|
||||||
|
|
||||||
|
Con `retention=workqueue`, cada mensaje se **borra del stream en cuanto un consumer lo confirma**. Es el patrón de cola de tareas distribuida: los mensajes se reparten entre workers y desaparecen al procesarse, así el stream no crece sin fin."""),
|
||||||
|
|
||||||
|
("code", '''from nats.js.api import RetentionPolicy, StorageType
|
||||||
|
|
||||||
|
await js.add_stream(name="DEMO_WQ", subjects=["demo.wq.>"],
|
||||||
|
storage=StorageType.FILE, retention=RetentionPolicy.WORK_QUEUE)
|
||||||
|
|
||||||
|
# Encolar 5 tareas
|
||||||
|
for i in range(5):
|
||||||
|
await js.publish("demo.wq.tareas", f"tarea-{i}".encode())
|
||||||
|
|
||||||
|
antes = (await js.stream_info("DEMO_WQ")).state.messages
|
||||||
|
print(f"Tareas encoladas en el stream: {antes}")
|
||||||
|
|
||||||
|
# Un worker consume y confirma 3
|
||||||
|
wsub = await js.pull_subscribe("demo.wq.>", durable="worker")
|
||||||
|
tres = await wsub.fetch(3, timeout=2)
|
||||||
|
for m in tres:
|
||||||
|
print(f" procesada: {m.data.decode()}")
|
||||||
|
await m.ack() # al confirmar, JetStream BORRA el mensaje del stream
|
||||||
|
|
||||||
|
await asyncio.sleep(0.3)
|
||||||
|
despues = (await js.stream_info("DEMO_WQ")).state.messages
|
||||||
|
print()
|
||||||
|
print(f"Mensajes restantes en el stream tras 3 acks: {despues} (workqueue borra lo confirmado: {antes} -> {despues})")'''),
|
||||||
|
|
||||||
|
("md", """### 5 · Políticas de entrega (replay)
|
||||||
|
|
||||||
|
Al crear un consumer se elige **desde dónde** empieza a leer (`DeliverPolicy`):
|
||||||
|
|
||||||
|
- `ALL` — todo el historial desde el principio (lo habitual para reprocesar).
|
||||||
|
- `LAST` — solo el último mensaje del stream.
|
||||||
|
- `NEW` — solo lo que llegue a partir de ahora.
|
||||||
|
- `BY_START_SEQUENCE` / `BY_START_TIME` — desde una secuencia o instante concretos.
|
||||||
|
|
||||||
|
Comparamos `ALL` vs `LAST` sobre el stream de límites (que tiene 6 mensajes)."""),
|
||||||
|
|
||||||
|
("code", '''from nats.js.api import ConsumerConfig, DeliverPolicy
|
||||||
|
|
||||||
|
# Consumer que reproduce TODO el historial
|
||||||
|
all_sub = await js.pull_subscribe(
|
||||||
|
"demo.limits.>", durable="replay-all",
|
||||||
|
config=ConsumerConfig(deliver_policy=DeliverPolicy.ALL),
|
||||||
|
)
|
||||||
|
todos = await all_sub.fetch(50, timeout=1)
|
||||||
|
print(f"DeliverPolicy.ALL -> {len(todos)} mensajes:", [m.data.decode() for m in todos])
|
||||||
|
for m in todos:
|
||||||
|
await m.ack()
|
||||||
|
|
||||||
|
# Consumer que solo entrega el ÚLTIMO
|
||||||
|
last_sub = await js.pull_subscribe(
|
||||||
|
"demo.limits.>", durable="replay-last",
|
||||||
|
config=ConsumerConfig(deliver_policy=DeliverPolicy.LAST),
|
||||||
|
)
|
||||||
|
ultimo = await last_sub.fetch(50, timeout=1)
|
||||||
|
print(f"DeliverPolicy.LAST -> {len(ultimo)} mensaje :", [m.data.decode() for m in ultimo])
|
||||||
|
for m in ultimo:
|
||||||
|
await m.ack()'''),
|
||||||
|
|
||||||
|
# ---- Parte B: simulador interactivo ----
|
||||||
|
("md", """## Parte B · Simulador de rendimiento (interactivo)
|
||||||
|
|
||||||
|
Pulsa **▶ Ejecutar benchmark** y verás cómo **un publisher** inunda el broker con miles de mensajes que **varios subscribers** reciben simultáneamente (fan-out). La gráfica se actualiza **en movimiento** mientras corre:
|
||||||
|
|
||||||
|
- **Izquierda** — mensajes acumulados: enviados (publisher) vs recibidos (suma de todos los subs).
|
||||||
|
- **Derecha** — throughput instantáneo (msgs/s recibidos) muestreado cada ~80 ms.
|
||||||
|
|
||||||
|
Ajusta los sliders para cambiar el número de mensajes y de subscribers. Con más mensajes (p. ej. 100.000) la animación dura más y se aprecia mejor la curva."""),
|
||||||
|
|
||||||
|
("code", '''import ipywidgets as widgets
|
||||||
|
from IPython.display import display, clear_output
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import asyncio, time
|
||||||
|
import nats
|
||||||
|
|
||||||
|
# --- widgets ---
|
||||||
|
n_msgs_w = widgets.IntSlider(value=20000, min=1000, max=100000, step=1000,
|
||||||
|
description="Mensajes:", style={"description_width": "initial"},
|
||||||
|
layout=widgets.Layout(width="380px"))
|
||||||
|
n_subs_w = widgets.IntSlider(value=3, min=1, max=8, step=1,
|
||||||
|
description="Subscribers:", style={"description_width": "initial"})
|
||||||
|
run_btn = widgets.Button(description="▶ Ejecutar benchmark", button_style="success",
|
||||||
|
layout=widgets.Layout(width="220px"))
|
||||||
|
plot_out = widgets.Output()
|
||||||
|
log_out = widgets.Output()
|
||||||
|
|
||||||
|
SUBJECT = "bench.load"
|
||||||
|
PAYLOAD = b"x" * 128 # 128 bytes por mensaje
|
||||||
|
|
||||||
|
def _throughput(ts, recv):
|
||||||
|
thr = [0.0]
|
||||||
|
for i in range(1, len(ts)):
|
||||||
|
dt = ts[i] - ts[i-1]
|
||||||
|
thr.append((recv[i] - recv[i-1]) / dt if dt > 0 else 0.0)
|
||||||
|
return thr
|
||||||
|
|
||||||
|
def render(history, n_subs, n_msgs, done=False):
|
||||||
|
ts = [h[0] for h in history]
|
||||||
|
sent = [h[1] for h in history]
|
||||||
|
recv = [h[2] for h in history]
|
||||||
|
thr = _throughput(ts, recv)
|
||||||
|
with plot_out:
|
||||||
|
clear_output(wait=True)
|
||||||
|
fig, (a1, a2) = plt.subplots(1, 2, figsize=(11, 3.6))
|
||||||
|
a1.plot(ts, sent, label="enviados (pub)", color="#2563eb", lw=2)
|
||||||
|
a1.plot(ts, recv, label=f"recibidos (Σ {n_subs} subs)", color="#16a34a", lw=2)
|
||||||
|
a1.set_xlabel("segundos"); a1.set_ylabel("mensajes acumulados")
|
||||||
|
a1.set_title("Publisher vs subscribers"); a1.legend(loc="upper left")
|
||||||
|
a2.plot(ts, thr, color="#db2777", lw=2)
|
||||||
|
a2.set_xlabel("segundos"); a2.set_ylabel("msgs/s recibidos")
|
||||||
|
a2.set_title("Throughput instantáneo")
|
||||||
|
estado = "✓ DONE" if done else "● corriendo…"
|
||||||
|
fig.suptitle(f"[{estado}] {n_msgs:,} msgs → {n_subs} subs "
|
||||||
|
f"enviados={sent[-1]:,} recibidos={recv[-1]:,}", fontsize=11)
|
||||||
|
plt.tight_layout(); plt.show()
|
||||||
|
|
||||||
|
async def run_benchmark(n_msgs, n_subs, live=True):
|
||||||
|
"""1 publisher -> n_subs subscribers. Devuelve (history, counters)."""
|
||||||
|
nc = await nats.connect(NATS_URL, name="benchmark")
|
||||||
|
counters = [0] * n_subs
|
||||||
|
|
||||||
|
def make_cb(i):
|
||||||
|
async def cb(msg):
|
||||||
|
counters[i] += 1
|
||||||
|
return cb
|
||||||
|
|
||||||
|
subs = [await nc.subscribe(SUBJECT, cb=make_cb(i)) for i in range(n_subs)]
|
||||||
|
history = [] # (t, enviados, recibidos_total)
|
||||||
|
sent = 0
|
||||||
|
t0 = time.monotonic()
|
||||||
|
|
||||||
|
async def publish_all():
|
||||||
|
nonlocal sent
|
||||||
|
for k in range(n_msgs):
|
||||||
|
await nc.publish(SUBJECT, PAYLOAD)
|
||||||
|
sent += 1
|
||||||
|
if k % 1000 == 0:
|
||||||
|
await nc.flush()
|
||||||
|
await asyncio.sleep(0) # ceder al event loop (deja correr callbacks)
|
||||||
|
await nc.flush()
|
||||||
|
|
||||||
|
task = asyncio.create_task(publish_all())
|
||||||
|
|
||||||
|
# Muestreo para la gráfica en movimiento
|
||||||
|
while not task.done() or sum(counters) < sent:
|
||||||
|
await asyncio.sleep(0.08)
|
||||||
|
history.append((time.monotonic() - t0, sent, sum(counters)))
|
||||||
|
if live:
|
||||||
|
render(history, n_subs, n_msgs)
|
||||||
|
if time.monotonic() - t0 > 30: # tope de seguridad
|
||||||
|
break
|
||||||
|
await task
|
||||||
|
|
||||||
|
# Drenaje final (que los callbacks alcancen al publisher)
|
||||||
|
for _ in range(40):
|
||||||
|
if sum(counters) >= sent:
|
||||||
|
break
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
history.append((time.monotonic() - t0, sent, sum(counters)))
|
||||||
|
if live:
|
||||||
|
render(history, n_subs, n_msgs, done=True)
|
||||||
|
|
||||||
|
for s in subs:
|
||||||
|
await s.unsubscribe()
|
||||||
|
await nc.drain()
|
||||||
|
return history, counters
|
||||||
|
|
||||||
|
def on_click(_):
|
||||||
|
run_btn.disabled = True
|
||||||
|
with log_out:
|
||||||
|
clear_output()
|
||||||
|
print(f"Lanzando: {n_msgs_w.value:,} mensajes → {n_subs_w.value} subscribers …")
|
||||||
|
async def go():
|
||||||
|
try:
|
||||||
|
history, counters = await run_benchmark(n_msgs_w.value, n_subs_w.value, live=True)
|
||||||
|
dur = history[-1][0]
|
||||||
|
recv = sum(counters)
|
||||||
|
with log_out:
|
||||||
|
print(f"OK en {dur:.2f}s")
|
||||||
|
print(f" enviados : {n_msgs_w.value:,}")
|
||||||
|
print(f" recibidos: {recv:,} (fan-out ×{n_subs_w.value} = {recv/max(n_msgs_w.value,1):.2f} por mensaje)")
|
||||||
|
print(f" throughput pub : {n_msgs_w.value/dur:,.0f} msgs/s")
|
||||||
|
print(f" throughput recv: {recv/dur:,.0f} msgs/s (entregas totales)")
|
||||||
|
print(f" por subscriber : {counters}")
|
||||||
|
finally:
|
||||||
|
run_btn.disabled = False
|
||||||
|
asyncio.ensure_future(go())
|
||||||
|
|
||||||
|
run_btn.on_click(on_click)
|
||||||
|
display(widgets.HBox([n_msgs_w, n_subs_w]), run_btn, plot_out, log_out)
|
||||||
|
print("Simulador listo. Pulsa el botón para lanzar el benchmark.")'''),
|
||||||
|
|
||||||
|
("md", """### Verificación (headless)
|
||||||
|
|
||||||
|
La celda anterior renderiza el widget para pulsarlo en JupyterLab. Aquí ejecutamos el mismo benchmark **una vez de forma programática** (sin botón) para dejar evidencia ejecutada: una corrida real de 15.000 mensajes a 3 subscribers con su gráfica final."""),
|
||||||
|
|
||||||
|
("code", '''hist, counters = await run_benchmark(15000, 3, live=False)
|
||||||
|
|
||||||
|
ts = [h[0] for h in hist]
|
||||||
|
sent = [h[1] for h in hist]
|
||||||
|
recv = [h[2] for h in hist]
|
||||||
|
dur = ts[-1]
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(9, 3.6))
|
||||||
|
ax.plot(ts, sent, label="enviados (pub)", color="#2563eb", lw=2)
|
||||||
|
ax.plot(ts, recv, label="recibidos (Σ 3 subs)", color="#16a34a", lw=2)
|
||||||
|
ax.set_xlabel("segundos"); ax.set_ylabel("mensajes acumulados")
|
||||||
|
ax.set_title(f"Benchmark headless: 15.000 msgs → 3 subs en {dur:.2f}s")
|
||||||
|
ax.legend(loc="upper left")
|
||||||
|
plt.tight_layout(); plt.show()
|
||||||
|
|
||||||
|
print(f"enviados : 15,000")
|
||||||
|
print(f"recibidos: {sum(counters):,} por sub -> {counters}")
|
||||||
|
print(f"throughput pub : {15000/dur:,.0f} msgs/s")
|
||||||
|
print(f"throughput recv: {sum(counters)/dur:,.0f} msgs/s (entregas totales, fan-out x3)")'''),
|
||||||
|
|
||||||
|
("md", """## Resumen
|
||||||
|
|
||||||
|
**JetStream a fondo:**
|
||||||
|
- Un **stream** persiste mensajes con políticas de **storage** (file/memory), **retention** (limits/interest/workqueue) y **límites** (max_msgs/max_age).
|
||||||
|
- Los **consumers** (pull/push, durable/ephemeral) leen a su ritmo y **confirman** (ack) cada mensaje → entrega *at-least-once*.
|
||||||
|
- **Dedup** por `Nats-Msg-Id` evita duplicados por reintentos.
|
||||||
|
- **workqueue** borra cada mensaje al confirmarse → cola de trabajo.
|
||||||
|
- **DeliverPolicy** controla el replay (all/last/new/by_sequence/by_time).
|
||||||
|
|
||||||
|
**Simulador:** demuestra el fan-out a escala — un publisher alimenta a N subscribers con miles de mensajes y la gráfica en vivo muestra que el throughput de recepción sigue al de envío (cada mensaje se entrega a los N subscribers).
|
||||||
|
|
||||||
|
### Limpieza
|
||||||
|
|
||||||
|
```python
|
||||||
|
for s in ("DEMO_LIMITS", "DEMO_DEDUP", "DEMO_WQ"):
|
||||||
|
try: await js.delete_stream(s)
|
||||||
|
except Exception: pass
|
||||||
|
await nc.drain()
|
||||||
|
```"""),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
build("04_jetstream_benchmark.ipynb", nb4)
|
||||||
|
print("OK: notebook 04 generado en", NBDIR)
|
||||||
+285
-77
@@ -5,21 +5,21 @@
|
|||||||
Abre: http://localhost:8890
|
Abre: http://localhost:8890
|
||||||
Ctrl+C para detener
|
Ctrl+C para detener
|
||||||
|
|
||||||
[W 2026-06-03 19:41:26.217 ServerApp] ServerApp.token config is deprecated in 2.0. Use IdentityProvider.token.
|
[W 2026-06-03 21:18:04.909 ServerApp] ServerApp.token config is deprecated in 2.0. Use IdentityProvider.token.
|
||||||
[I 2026-06-03 19:41:26.656 ServerApp] jupyter_lsp | extension was successfully linked.
|
[I 2026-06-03 21:18:05.346 ServerApp] jupyter_lsp | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.658 ServerApp] jupyter_mcp_server | extension was successfully linked.
|
[I 2026-06-03 21:18:05.348 ServerApp] jupyter_mcp_server | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.658 ServerApp] jupyter_mcp_tools | extension was successfully linked.
|
[I 2026-06-03 21:18:05.348 ServerApp] jupyter_mcp_tools | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.660 ServerApp] jupyter_server_fileid | extension was successfully linked.
|
[I 2026-06-03 21:18:05.349 ServerApp] jupyter_server_fileid | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.662 ServerApp] jupyter_server_nbmodel | extension was successfully linked.
|
[I 2026-06-03 21:18:05.351 ServerApp] jupyter_server_nbmodel | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.663 ServerApp] jupyter_server_terminals | extension was successfully linked.
|
[I 2026-06-03 21:18:05.352 ServerApp] jupyter_server_terminals | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.665 ServerApp] jupyter_server_ydoc | extension was successfully linked.
|
[I 2026-06-03 21:18:05.354 ServerApp] jupyter_server_ydoc | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.667 ServerApp] jupyterlab | extension was successfully linked.
|
[I 2026-06-03 21:18:05.356 ServerApp] jupyterlab | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.669 ServerApp] notebook | extension was successfully linked.
|
[I 2026-06-03 21:18:05.358 ServerApp] notebook | extension was successfully linked.
|
||||||
[I 2026-06-03 19:41:26.670 ServerApp] notebook_shim | extension was successfully linked.
|
[I 2026-06-03 21:18:05.360 ServerApp] notebook_shim | extension was successfully linked.
|
||||||
[W 2026-06-03 19:41:26.680 ServerApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
|
[W 2026-06-03 21:18:05.369 ServerApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
|
||||||
[I 2026-06-03 19:41:26.681 ServerApp] notebook_shim | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.369 ServerApp] notebook_shim | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.682 ServerApp] jupyter_lsp | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.370 ServerApp] jupyter_lsp | extension was successfully loaded.
|
||||||
[06/03/26 19:41:26] INFO Auto-enrolled document extension.py:195
|
[06/03/26 21:18:05] INFO Auto-enrolled document extension.py:195
|
||||||
'notebook.ipynb' as 'default'
|
'notebook.ipynb' as 'default'
|
||||||
INFO Jupyter MCP Server Extension extension.py:197
|
INFO Jupyter MCP Server Extension extension.py:197
|
||||||
settings initialized
|
settings initialized
|
||||||
@@ -28,62 +28,43 @@
|
|||||||
INFO - Health check: /mcp/healthz extension.py:235
|
INFO - Health check: /mcp/healthz extension.py:235
|
||||||
INFO - List tools: /mcp/tools/list extension.py:236
|
INFO - List tools: /mcp/tools/list extension.py:236
|
||||||
INFO - Call tool: /mcp/tools/call extension.py:237
|
INFO - Call tool: /mcp/tools/call extension.py:237
|
||||||
[I 2026-06-03 19:41:26.784 ServerApp] jupyter_mcp_server | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.464 ServerApp] jupyter_mcp_server | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.784 ServerApp] Registered jupyter_mcp_tools server extension
|
[I 2026-06-03 21:18:05.464 ServerApp] Registered jupyter_mcp_tools server extension
|
||||||
[I 2026-06-03 19:41:26.784 ServerApp] jupyter_mcp_tools | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.464 ServerApp] jupyter_mcp_tools | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.784 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
|
[I 2026-06-03 21:18:05.464 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
|
||||||
[I 2026-06-03 19:41:26.784 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/enmanuel/fn_registry/analysis/nats
|
[I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/enmanuel/fn_registry/analysis/nats
|
||||||
[I 2026-06-03 19:41:26.784 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/enmanuel/.local/share/jupyter/file_id_manager.db
|
[I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/enmanuel/.local/share/jupyter/file_id_manager.db
|
||||||
[I 2026-06-03 19:41:26.785 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
|
[I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
|
||||||
[I 2026-06-03 19:41:26.785 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
|
[I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
|
||||||
[I 2026-06-03 19:41:26.785 FileIdExtension] Attached event listeners.
|
[I 2026-06-03 21:18:05.464 FileIdExtension] Attached event listeners.
|
||||||
[I 2026-06-03 19:41:26.785 ServerApp] jupyter_server_fileid | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.464 ServerApp] jupyter_server_fileid | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.785 ServerApp] jupyter_server_nbmodel | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.465 ServerApp] jupyter_server_nbmodel | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.786 ServerApp] jupyter_server_terminals | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.465 ServerApp] jupyter_server_terminals | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.787 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.467 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.789 LabApp] JupyterLab extension loaded from /home/enmanuel/fn_registry/analysis/nats/.venv/lib/python3.12/site-packages/jupyterlab
|
[I 2026-06-03 21:18:05.468 LabApp] JupyterLab extension loaded from /home/enmanuel/fn_registry/analysis/nats/.venv/lib/python3.12/site-packages/jupyterlab
|
||||||
[I 2026-06-03 19:41:26.789 LabApp] JupyterLab application directory is /home/enmanuel/fn_registry/analysis/nats/.venv/share/jupyter/lab
|
[I 2026-06-03 21:18:05.468 LabApp] JupyterLab application directory is /home/enmanuel/fn_registry/analysis/nats/.venv/share/jupyter/lab
|
||||||
[I 2026-06-03 19:41:26.789 LabApp] Extension Manager is 'pypi'.
|
[I 2026-06-03 21:18:05.469 LabApp] Extension Manager is 'pypi'.
|
||||||
[I 2026-06-03 19:41:26.810 ServerApp] jupyterlab | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.489 ServerApp] jupyterlab | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.811 ServerApp] notebook | extension was successfully loaded.
|
[I 2026-06-03 21:18:05.491 ServerApp] notebook | extension was successfully loaded.
|
||||||
[I 2026-06-03 19:41:26.812 ServerApp] Serving notebooks from local directory: /home/enmanuel/fn_registry/analysis/nats
|
[I 2026-06-03 21:18:05.491 ServerApp] Serving notebooks from local directory: /home/enmanuel/fn_registry/analysis/nats
|
||||||
[I 2026-06-03 19:41:26.812 ServerApp] Jupyter Server 2.19.0 is running at:
|
[I 2026-06-03 21:18:05.491 ServerApp] Jupyter Server 2.19.0 is running at:
|
||||||
[I 2026-06-03 19:41:26.812 ServerApp] http://localhost:8890/lab
|
[I 2026-06-03 21:18:05.491 ServerApp] http://localhost:8890/lab
|
||||||
[I 2026-06-03 19:41:26.812 ServerApp] http://127.0.0.1:8890/lab
|
[I 2026-06-03 21:18:05.491 ServerApp] http://127.0.0.1:8890/lab
|
||||||
[I 2026-06-03 19:41:26.812 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
|
[I 2026-06-03 21:18:05.491 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
|
||||||
[I 2026-06-03 19:41:26.876 ServerApp] Skipped non-installed server(s): basedpyright, bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyrefly, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
|
[I 2026-06-03 21:18:05.555 ServerApp] Skipped non-installed server(s): basedpyright, bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyrefly, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
|
||||||
[I 2026-06-03 19:46:15.569 ServerApp] Request for Y document (previously indexed) 'notebooks/01_core_pubsub.ipynb' with room ID: f0c81420-7285-42fa-afcf-aa618e651df6
|
[I 2026-06-03 21:18:25.232 ServerApp] 302 GET / (@127.0.0.1) 0.24ms
|
||||||
[I 2026-06-03 19:46:15.654 LabApp] Build is up to date
|
[I 2026-06-03 21:18:33.966 LabApp] Build is up to date
|
||||||
[I 2026-06-03 19:46:15.656 ServerApp] MCP Tools WebSocket connection opened
|
[I 2026-06-03 21:18:33.967 ServerApp] Request for Y document (previously indexed) 'notebooks/01_core_pubsub.ipynb' with room ID: f0c81420-7285-42fa-afcf-aa618e651df6
|
||||||
[W 2026-06-03 19:46:15.669 ServerApp] The websocket_ping_timeout (90000) cannot be longer than the websocket_ping_interval (30000).
|
[I 2026-06-03 21:18:34.060 ServerApp] MCP Tools WebSocket connection opened
|
||||||
|
[I 2026-06-03 21:18:34.061 ServerApp] Request for Y document (previously indexed) 'notebooks/02_queue_request_jetstream.ipynb' with room ID: fadc19b0-8f98-429c-a6d5-056a05bf47cd
|
||||||
|
[I 2026-06-03 21:18:34.061 ServerApp] Request for Y document (previously indexed) 'notebooks/03_procesos_reales.ipynb' with room ID: 61f241c1-9f7e-46e7-8fce-e61e1c568d1c
|
||||||
|
[W 2026-06-03 21:18:34.072 ServerApp] The websocket_ping_timeout (90000) cannot be longer than the websocket_ping_interval (30000).
|
||||||
Setting websocket_ping_timeout=30000
|
Setting websocket_ping_timeout=30000
|
||||||
[I 2026-06-03 19:46:15.672 YDocExtension] Creating FileLoader for: notebooks/01_core_pubsub.ipynb
|
[I 2026-06-03 21:18:34.076 YDocExtension] Creating FileLoader for: notebooks/01_core_pubsub.ipynb
|
||||||
[I 2026-06-03 19:46:15.673 YDocExtension] Watching file: notebooks/01_core_pubsub.ipynb
|
[I 2026-06-03 21:18:34.078 YDocExtension] Watching file: notebooks/01_core_pubsub.ipynb
|
||||||
[I 2026-06-03 19:46:15.675 ServerApp] Initializing room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
[I 2026-06-03 21:18:34.078 ServerApp] Registered 414 tools
|
||||||
[I 2026-06-03 19:46:15.676 ServerApp] Registered 414 tools
|
[I 2026-06-03 21:18:34.081 ServerApp] Initializing room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
||||||
[I 2026-06-03 19:46:15.716 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from the ystore SQLiteYStore
|
[06/03/26 21:18:34] ERROR Notebook JSON is invalid: Additional __init__.py:98
|
||||||
[I 2026-06-03 19:46:15.719 ServerApp] Content in file notebooks/01_core_pubsub.ipynb is out-of-sync with the ystore SQLiteYStore
|
|
||||||
[I 2026-06-03 19:46:15.720 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from file notebooks/01_core_pubsub.ipynb
|
|
||||||
[I 2026-06-03 19:46:16.735 ServerApp] Saving the content from room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:46:16.738 YDocExtension] Saving file: notebooks/01_core_pubsub.ipynb
|
|
||||||
[I 2026-06-03 19:47:15.407 YDocExtension] Processed 31 Y patches in one minute
|
|
||||||
[I 2026-06-03 19:47:15.407 YDocExtension] Connected Y users: 2
|
|
||||||
[I 2026-06-03 19:48:15.407 YDocExtension] Processed 16 Y patches in one minute
|
|
||||||
[I 2026-06-03 19:48:15.408 YDocExtension] Connected Y users: 2
|
|
||||||
[I 2026-06-03 19:49:15.408 YDocExtension] Processed 16 Y patches in one minute
|
|
||||||
[I 2026-06-03 19:49:15.409 YDocExtension] Connected Y users: 2
|
|
||||||
[I 2026-06-03 19:49:52.746 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[W 2026-06-03 19:49:52.770 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
|
|
||||||
[I 2026-06-03 19:50:15.409 YDocExtension] Processed 16 Y patches in one minute
|
|
||||||
[I 2026-06-03 19:50:15.409 YDocExtension] Connected Y users: 2
|
|
||||||
[I 2026-06-03 19:50:16.900 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:50:17.929 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:50:22.970 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:50:26.008 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:50:30.051 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:50:34.088 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[I 2026-06-03 19:50:37.127 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
|
||||||
[06/03/26 19:50:37] ERROR Notebook JSON is invalid: Additional __init__.py:98
|
|
||||||
properties are not allowed
|
properties are not allowed
|
||||||
('transient' was unexpected)
|
('transient' was unexpected)
|
||||||
|
|
||||||
@@ -101,10 +82,237 @@
|
|||||||
'metadata': {},
|
'metadata': {},
|
||||||
'output_type': 'display_data',
|
'output_type': 'display_data',
|
||||||
'transient': {}}
|
'transient': {}}
|
||||||
[W 2026-06-03 19:50:37.149 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
|
[I 2026-06-03 21:18:34.118 YDocExtension] Creating FileLoader for: notebooks/02_queue_request_jetstream.ipynb
|
||||||
[I 2026-06-03 19:51:15.410 YDocExtension] Processed 16 Y patches in one minute
|
[W 2026-06-03 21:18:34.120 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
|
||||||
[I 2026-06-03 19:51:15.411 YDocExtension] Connected Y users: 2
|
[I 2026-06-03 21:18:34.121 YDocExtension] Watching file: notebooks/02_queue_request_jetstream.ipynb
|
||||||
[I 2026-06-03 19:52:15.414 YDocExtension] Processed 16 Y patches in one minute
|
[I 2026-06-03 21:18:34.122 ServerApp] Initializing room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd
|
||||||
[I 2026-06-03 19:52:15.414 YDocExtension] Connected Y users: 2
|
ERROR Notebook JSON is invalid: Additional __init__.py:98
|
||||||
[I 2026-06-03 19:53:15.417 YDocExtension] Processed 16 Y patches in one minute
|
properties are not allowed
|
||||||
[I 2026-06-03 19:53:15.417 YDocExtension] Connected Y users: 2
|
('transient' was unexpected)
|
||||||
|
|
||||||
|
Failed validating
|
||||||
|
'additionalProperties' in
|
||||||
|
display_data:
|
||||||
|
|
||||||
|
On
|
||||||
|
instance['cells'][5]['outputs'][0]:
|
||||||
|
{'data': {'image/png':
|
||||||
|
'iVBORw0KGgoAAAANSUhEUgAAArIAAAEiCAY
|
||||||
|
AAAAF9zFeAAAAOnRFWHRTb2Z0d2Fy...',
|
||||||
|
'text/plain': '<Figure
|
||||||
|
size 700x300 with 1 Axes>'},
|
||||||
|
'metadata': {},
|
||||||
|
'output_type': 'display_data',
|
||||||
|
'transient': {}}
|
||||||
|
[W 2026-06-03 21:18:34.130 ServerApp] Notebook notebooks/02_queue_request_jetstream.ipynb is not trusted
|
||||||
|
[I 2026-06-03 21:18:34.133 YDocExtension] Creating FileLoader for: notebooks/03_procesos_reales.ipynb
|
||||||
|
[I 2026-06-03 21:18:34.133 YDocExtension] Watching file: notebooks/03_procesos_reales.ipynb
|
||||||
|
[I 2026-06-03 21:18:34.134 ServerApp] Initializing room json:notebook:61f241c1-9f7e-46e7-8fce-e61e1c568d1c
|
||||||
|
ERROR Notebook JSON is invalid: Additional __init__.py:98
|
||||||
|
properties are not allowed
|
||||||
|
('transient' was unexpected)
|
||||||
|
|
||||||
|
Failed validating
|
||||||
|
'additionalProperties' in
|
||||||
|
display_data:
|
||||||
|
|
||||||
|
On
|
||||||
|
instance['cells'][8]['outputs'][0]:
|
||||||
|
{'data': {'image/png':
|
||||||
|
'iVBORw0KGgoAAAANSUhEUgAAArIAAAExCAY
|
||||||
|
AAACAtUFrAAAAOnRFWHRTb2Z0d2Fy...',
|
||||||
|
'text/plain': '<Figure
|
||||||
|
size 700x320 with 1 Axes>'},
|
||||||
|
'metadata': {},
|
||||||
|
'output_type': 'display_data',
|
||||||
|
'transient': {}}
|
||||||
|
[W 2026-06-03 21:18:34.148 ServerApp] Notebook notebooks/03_procesos_reales.ipynb is not trusted
|
||||||
|
[I 2026-06-03 21:18:34.153 ServerApp] Content in room json:notebook:61f241c1-9f7e-46e7-8fce-e61e1c568d1c loaded from the ystore SQLiteYStore
|
||||||
|
[I 2026-06-03 21:18:34.157 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from the ystore SQLiteYStore
|
||||||
|
[I 2026-06-03 21:18:34.158 ServerApp] Content in file notebooks/01_core_pubsub.ipynb is out-of-sync with the ystore SQLiteYStore
|
||||||
|
[I 2026-06-03 21:18:34.158 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from file notebooks/01_core_pubsub.ipynb
|
||||||
|
[I 2026-06-03 21:18:34.162 ServerApp] Content in room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd loaded from the ystore SQLiteYStore
|
||||||
|
[I 2026-06-03 21:18:34.164 ServerApp] Content in file notebooks/02_queue_request_jetstream.ipynb is out-of-sync with the ystore SQLiteYStore
|
||||||
|
[I 2026-06-03 21:18:34.164 ServerApp] Content in room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd loaded from file notebooks/02_queue_request_jetstream.ipynb
|
||||||
|
[I 2026-06-03 21:18:34.561 ServerApp] Kernel started: adb9c688-aaa0-442b-a665-77464065e37c
|
||||||
|
[I 2026-06-03 21:18:34.563 ServerApp] Kernel started: 97684b48-3a52-4e4b-9c7f-4d1dd08ab5a6
|
||||||
|
[I 2026-06-03 21:18:34.564 ServerApp] Kernel started: 3a4122ce-ea0f-4bd5-8953-7726be1f5de2
|
||||||
|
[I 2026-06-03 21:18:34.860 ServerApp] Adapting from protocol version 5.3 (kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:18:34.861 ServerApp] Connecting to kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2.
|
||||||
|
[I 2026-06-03 21:18:34.883 ServerApp] Adapting from protocol version 5.3 (kernel adb9c688-aaa0-442b-a665-77464065e37c) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:18:34.884 ServerApp] Connecting to kernel adb9c688-aaa0-442b-a665-77464065e37c.
|
||||||
|
[I 2026-06-03 21:18:34.905 ServerApp] Adapting from protocol version 5.3 (kernel 97684b48-3a52-4e4b-9c7f-4d1dd08ab5a6) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:18:34.906 ServerApp] Connecting to kernel 97684b48-3a52-4e4b-9c7f-4d1dd08ab5a6.
|
||||||
|
[I 2026-06-03 21:18:34.921 ServerApp] Adapting from protocol version 5.3 (kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:18:34.922 ServerApp] Connecting to kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2.
|
||||||
|
[I 2026-06-03 21:18:34.939 ServerApp] Adapting from protocol version 5.3 (kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:18:34.944 ServerApp] Connecting to kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2.
|
||||||
|
[I 2026-06-03 21:18:35.172 ServerApp] Saving the content from room json:notebook:61f241c1-9f7e-46e7-8fce-e61e1c568d1c
|
||||||
|
[I 2026-06-03 21:18:35.178 YDocExtension] Saving file: notebooks/03_procesos_reales.ipynb
|
||||||
|
[W 2026-06-03 21:18:35.179 ServerApp] Notebook notebooks/03_procesos_reales.ipynb is not trusted
|
||||||
|
[06/03/26 21:18:35] ERROR Notebook JSON is invalid: __init__.py:134
|
||||||
|
Additional properties are not
|
||||||
|
allowed ('transient' was
|
||||||
|
unexpected)
|
||||||
|
|
||||||
|
Failed validating
|
||||||
|
'additionalProperties' in
|
||||||
|
display_data:
|
||||||
|
|
||||||
|
On
|
||||||
|
instance['cells'][8]['outputs'][0]:
|
||||||
|
{'data': {'image/png':
|
||||||
|
'iVBORw0KGgoAAAANSUhEUgAAArIAAAExCA
|
||||||
|
YAAACAtUFrAAAAOnRFWHRTb2Z0d2Fy...',
|
||||||
|
'text/plain': '<Figure
|
||||||
|
size 700x320 with 1 Axes>'},
|
||||||
|
'metadata': {},
|
||||||
|
'output_type': 'display_data',
|
||||||
|
'transient': {}}
|
||||||
|
[I 2026-06-03 21:18:35.269 ServerApp] Saving the content from room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
|
||||||
|
[I 2026-06-03 21:18:35.275 YDocExtension] Saving file: notebooks/01_core_pubsub.ipynb
|
||||||
|
[W 2026-06-03 21:18:35.276 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
|
||||||
|
ERROR Notebook JSON is invalid: __init__.py:134
|
||||||
|
Additional properties are not
|
||||||
|
allowed ('transient' was
|
||||||
|
unexpected)
|
||||||
|
|
||||||
|
Failed validating
|
||||||
|
'additionalProperties' in
|
||||||
|
display_data:
|
||||||
|
|
||||||
|
On
|
||||||
|
instance['cells'][12]['outputs'][0]
|
||||||
|
:
|
||||||
|
{'data': {'image/png':
|
||||||
|
'iVBORw0KGgoAAAANSUhEUgAAA3oAAAD5CA
|
||||||
|
YAAABxn0eTAAAAOnRFWHRTb2Z0d2Fy...',
|
||||||
|
'text/plain': '<Figure
|
||||||
|
size 900x260 with 1 Axes>'},
|
||||||
|
'metadata': {},
|
||||||
|
'output_type': 'display_data',
|
||||||
|
'transient': {}}
|
||||||
|
[I 2026-06-03 21:18:35.436 ServerApp] Saving the content from room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd
|
||||||
|
[I 2026-06-03 21:18:35.442 YDocExtension] Saving file: notebooks/02_queue_request_jetstream.ipynb
|
||||||
|
[W 2026-06-03 21:18:35.444 ServerApp] Notebook notebooks/02_queue_request_jetstream.ipynb is not trusted
|
||||||
|
ERROR Notebook JSON is invalid: __init__.py:134
|
||||||
|
Additional properties are not
|
||||||
|
allowed ('transient' was
|
||||||
|
unexpected)
|
||||||
|
|
||||||
|
Failed validating
|
||||||
|
'additionalProperties' in
|
||||||
|
display_data:
|
||||||
|
|
||||||
|
On
|
||||||
|
instance['cells'][5]['outputs'][0]:
|
||||||
|
{'data': {'image/png':
|
||||||
|
'iVBORw0KGgoAAAANSUhEUgAAArIAAAEiCA
|
||||||
|
YAAAAF9zFeAAAAOnRFWHRTb2Z0d2Fy...',
|
||||||
|
'text/plain': '<Figure
|
||||||
|
size 700x300 with 1 Axes>'},
|
||||||
|
'metadata': {},
|
||||||
|
'output_type': 'display_data',
|
||||||
|
'transient': {}}
|
||||||
|
[I 2026-06-03 21:19:33.804 YDocExtension] Processed 59 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:19:33.805 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:20:33.805 YDocExtension] Processed 34 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:20:33.806 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:21:33.811 YDocExtension] Processed 32 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:21:33.811 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:22:33.815 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:22:33.815 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:23:33.818 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:23:33.818 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:24:33.824 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:24:33.824 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:25:33.828 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:25:33.828 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:26:33.830 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:26:33.830 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:27:33.831 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:27:33.831 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:28:33.836 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:28:33.836 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:29:33.838 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:29:33.838 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:30:33.838 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:30:33.839 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:31:33.842 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:31:33.842 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:32:33.845 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:32:33.845 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:33:33.851 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:33:33.851 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:34:33.854 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:34:33.855 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:35:33.857 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:35:33.857 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:36:33.863 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:36:33.863 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:37:33.866 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:37:33.867 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:38:33.868 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:38:33.868 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:39:33.875 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:39:33.875 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:40:33.878 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:40:33.879 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:41:33.881 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:41:33.881 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:42:33.882 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:42:33.883 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:43:33.886 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:43:33.886 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:44:33.888 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:44:33.888 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:45:33.889 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:45:33.890 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:46:33.894 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:46:33.894 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:47:33.896 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:47:33.896 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:48:33.898 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:48:33.898 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:49:33.903 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:49:33.903 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:50:33.906 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:50:33.906 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:51:18.556 ServerApp] Kernel started: 8391112c-c8cd-40ef-a0fd-2590d1861f16
|
||||||
|
[W 2026-06-03 21:51:18.559 ServerApp] Notebook notebooks/04_jetstream_benchmark.ipynb is not trusted
|
||||||
|
[I 2026-06-03 21:51:18.873 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:18.875 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:20.065 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:edf9cdd2-e2e57dbe9e1db13c2369bd49
|
||||||
|
[I 2026-06-03 21:51:20.070 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:20.129 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:20.131 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:20.190 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:ce162d97-bcd67ae41b07a72954c689b8
|
||||||
|
[I 2026-06-03 21:51:20.194 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:20.250 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:20.251 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:20.314 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:833d59f8-a7593d93633152b0c03ea130
|
||||||
|
[I 2026-06-03 21:51:20.318 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:20.379 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:20.380 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:20.433 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:cbe487ea-c1f5e2dffb3b06659f933964
|
||||||
|
[I 2026-06-03 21:51:20.439 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:20.490 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:20.492 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:20.849 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:e5ea749a-9c5b0957ba57e53d7545e31c
|
||||||
|
[I 2026-06-03 21:51:20.853 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:20.904 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:20.906 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:20.957 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:30826324-d09e241fc54796adcd6a376c
|
||||||
|
[I 2026-06-03 21:51:20.959 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:33.909 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:51:33.909 YDocExtension] Connected Y users: 4
|
||||||
|
[I 2026-06-03 21:51:42.074 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:42.075 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:42.431 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:c27a340b-675498de282e4e38aec57c86
|
||||||
|
[I 2026-06-03 21:51:42.436 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:51:42.492 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
|
||||||
|
[I 2026-06-03 21:51:42.493 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
|
||||||
|
[I 2026-06-03 21:51:42.762 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:a54123fe-7e8fcf758646f2ed9ec519e2
|
||||||
|
[I 2026-06-03 21:51:42.764 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
|
||||||
|
[I 2026-06-03 21:52:33.911 YDocExtension] Processed 20 Y patches in one minute
|
||||||
|
[I 2026-06-03 21:52:33.912 YDocExtension] Connected Y users: 4
|
||||||
|
|||||||
@@ -0,0 +1,539 @@
|
|||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "72277c66",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"# NATS pub/sub — 04 · JetStream a fondo y simulador de rendimiento\n",
|
||||||
|
"\n",
|
||||||
|
"Este notebook tiene dos partes:\n",
|
||||||
|
"\n",
|
||||||
|
"1. **JetStream a fondo** — más allá del replay básico del notebook 02: anatomía de un stream (almacenamiento, políticas de retención, límites), tipos de consumer, *acks*, deduplicación y políticas de entrega.\n",
|
||||||
|
"2. **Simulador de rendimiento interactivo** — un botón que, al pulsarlo, lanza un publisher que envía **miles de mensajes** a varios subscribers, con una **gráfica en movimiento** que muestra el throughput en tiempo real.\n",
|
||||||
|
"\n",
|
||||||
|
"> Requiere el broker `nats_demo` (arrancado por la primera celda) y `ipywidgets` (incluido en el venv del análisis)."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "530eac60",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Parte A · JetStream a fondo\n",
|
||||||
|
"\n",
|
||||||
|
"### Setup\n",
|
||||||
|
"\n",
|
||||||
|
"JetStream es la capa de persistencia de NATS. Mientras el core es *fire-and-forget*, JetStream **almacena** los mensajes en un *stream* y permite leerlos con *consumers* que controlan el ritmo, confirman (*ack*) cada mensaje y pueden reproducir el historial."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 1,
|
||||||
|
"id": "847ca130",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Broker: already-running\n",
|
||||||
|
"JetStream context listo. account info:\n",
|
||||||
|
" streams=1 consumers=1 memory=0 storage=260\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"import subprocess, time, json\n",
|
||||||
|
"\n",
|
||||||
|
"NATS_CONTAINER = \"nats_demo\"\n",
|
||||||
|
"NATS_PORT = 4222\n",
|
||||||
|
"NATS_URL = f\"nats://127.0.0.1:{NATS_PORT}\"\n",
|
||||||
|
"\n",
|
||||||
|
"def _docker(*args, check=True):\n",
|
||||||
|
" return subprocess.run([\"docker\", *args], capture_output=True, text=True, check=check)\n",
|
||||||
|
"\n",
|
||||||
|
"def ensure_nats(name=NATS_CONTAINER, port=NATS_PORT):\n",
|
||||||
|
" \"\"\"Arranca un broker NATS en Docker de forma idempotente. Devuelve el estado.\"\"\"\n",
|
||||||
|
" out = _docker(\"ps\", \"-a\", \"--filter\", f\"name=^{name}$\", \"--format\", \"{{.State}}\", check=False).stdout.strip()\n",
|
||||||
|
" if out == \"running\":\n",
|
||||||
|
" state = \"already-running\"\n",
|
||||||
|
" elif out in (\"exited\", \"created\", \"paused\"):\n",
|
||||||
|
" _docker(\"start\", name)\n",
|
||||||
|
" state = \"started\"\n",
|
||||||
|
" else:\n",
|
||||||
|
" _docker(\"run\", \"-d\", \"--name\", name, \"-p\", f\"{port}:4222\", \"-p\", \"8222:8222\",\n",
|
||||||
|
" \"nats:latest\", \"-js\", \"-m\", \"8222\")\n",
|
||||||
|
" state = \"created\"\n",
|
||||||
|
" time.sleep(1.0)\n",
|
||||||
|
" return state\n",
|
||||||
|
"\n",
|
||||||
|
"import asyncio\n",
|
||||||
|
"import nats\n",
|
||||||
|
"\n",
|
||||||
|
"print(\"Broker:\", ensure_nats())\n",
|
||||||
|
"nc = await nats.connect(NATS_URL, name=\"notebook-04\")\n",
|
||||||
|
"js = nc.jetstream()\n",
|
||||||
|
"print(\"JetStream context listo. account info:\")\n",
|
||||||
|
"ai = await js.account_info()\n",
|
||||||
|
"print(f\" streams={ai.streams} consumers={ai.consumers} memory={ai.memory} storage={ai.storage}\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "b55873ec",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"### 1 · Anatomía de un stream\n",
|
||||||
|
"\n",
|
||||||
|
"Un **stream** se define por:\n",
|
||||||
|
"\n",
|
||||||
|
"- **subjects** — qué subjects captura (`pedidos.>`).\n",
|
||||||
|
"- **storage** — `file` (persistente en disco) o `memory` (rápido, se pierde al reiniciar).\n",
|
||||||
|
"- **retention** — cuándo se descartan los mensajes:\n",
|
||||||
|
" - `limits` (por defecto): se guardan hasta tocar un límite (`max_msgs`, `max_bytes`, `max_age`).\n",
|
||||||
|
" - `interest`: se descartan cuando todos los consumers interesados los han recibido.\n",
|
||||||
|
" - `workqueue`: cada mensaje se borra en cuanto **un** consumer lo confirma (cola de trabajo).\n",
|
||||||
|
"- **límites** — `max_msgs`, `max_bytes`, `max_age` (segundos), `max_msg_size`.\n",
|
||||||
|
"- **duplicate_window** — ventana de deduplicación (ver §3).\n",
|
||||||
|
"\n",
|
||||||
|
"Creamos un stream `limits` con almacenamiento en disco y un tope de mensajes."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "67118a66",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"from nats.js.api import StreamConfig, RetentionPolicy, StorageType, DiscardPolicy\n",
|
||||||
|
"\n",
|
||||||
|
"# Recrear limpio para que la demo sea determinista\n",
|
||||||
|
"for s in (\"DEMO_LIMITS\", \"DEMO_DEDUP\", \"DEMO_WQ\"):\n",
|
||||||
|
" try:\n",
|
||||||
|
" await js.delete_stream(s)\n",
|
||||||
|
" except Exception:\n",
|
||||||
|
" pass\n",
|
||||||
|
"\n",
|
||||||
|
"cfg = StreamConfig(\n",
|
||||||
|
" name=\"DEMO_LIMITS\",\n",
|
||||||
|
" subjects=[\"demo.limits.>\"],\n",
|
||||||
|
" storage=StorageType.FILE,\n",
|
||||||
|
" retention=RetentionPolicy.LIMITS,\n",
|
||||||
|
" max_msgs=1000, # tope de mensajes\n",
|
||||||
|
" max_age=3600, # 1 hora (segundos)\n",
|
||||||
|
" discard=DiscardPolicy.OLD, # al llegar al tope, descarta los más viejos\n",
|
||||||
|
" duplicate_window=120, # ventana de dedup: 120 s\n",
|
||||||
|
")\n",
|
||||||
|
"info = await js.add_stream(cfg)\n",
|
||||||
|
"c = info.config\n",
|
||||||
|
"print(\"Stream creado:\")\n",
|
||||||
|
"print(f\" name : {c.name}\")\n",
|
||||||
|
"print(f\" subjects : {c.subjects}\")\n",
|
||||||
|
"print(f\" storage : {c.storage}\")\n",
|
||||||
|
"print(f\" retention : {c.retention}\")\n",
|
||||||
|
"print(f\" max_msgs : {c.max_msgs}\")\n",
|
||||||
|
"print(f\" max_age (s) : {c.max_age}\")\n",
|
||||||
|
"print(f\" discard : {c.discard}\")\n",
|
||||||
|
"print(f\" dup_window (s): {c.duplicate_window}\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "9bf78aac",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"### 2 · Consumers: pull, durable, ack\n",
|
||||||
|
"\n",
|
||||||
|
"Un **consumer** es la vista de lectura sobre un stream. Dos ejes:\n",
|
||||||
|
"\n",
|
||||||
|
"- **pull vs push**: en *pull* el cliente pide mensajes cuando quiere (`fetch`); en *push* el servidor los empuja según llegan.\n",
|
||||||
|
"- **durable vs ephemeral**: un consumer *durable* tiene nombre y **recuerda su posición** (cursor) entre reconexiones; uno *ephemeral* desaparece al cerrarse.\n",
|
||||||
|
"\n",
|
||||||
|
"El **ack** es la confirmación de procesado. Hasta que un mensaje no se confirma, el consumer lo considera *pendiente* y, si pasa el `ack_wait`, lo **reentrega**. Esto da entrega *at-least-once*."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "499e73f7",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"# Publicar 6 mensajes en el stream de límites\n",
|
||||||
|
"for i in range(6):\n",
|
||||||
|
" await js.publish(\"demo.limits.eventos\", f\"evento-{i}\".encode())\n",
|
||||||
|
"\n",
|
||||||
|
"# Pull consumer DURABLE: recuerda su cursor entre fetches\n",
|
||||||
|
"psub = await js.pull_subscribe(\"demo.limits.>\", durable=\"procesador-A\")\n",
|
||||||
|
"\n",
|
||||||
|
"# Traer 4 y confirmarlos (ack)\n",
|
||||||
|
"batch = await psub.fetch(4, timeout=2)\n",
|
||||||
|
"print(\"Primer fetch (4 msgs):\")\n",
|
||||||
|
"for m in batch:\n",
|
||||||
|
" print(f\" seq={m.metadata.sequence.stream} {m.data.decode()}\")\n",
|
||||||
|
" await m.ack()\n",
|
||||||
|
"\n",
|
||||||
|
"# Estado del consumer: cuántos quedan pendientes / entregados\n",
|
||||||
|
"ci = await psub.consumer_info()\n",
|
||||||
|
"print()\n",
|
||||||
|
"print(f\"Consumer 'procesador-A':\")\n",
|
||||||
|
"print(f\" num_pending : {ci.num_pending} (mensajes sin entregar todavía)\")\n",
|
||||||
|
"print(f\" num_ack_pending: {ci.num_ack_pending} (entregados sin ack)\")\n",
|
||||||
|
"print(f\" delivered.stream_seq: {ci.delivered.stream_seq}\")\n",
|
||||||
|
"\n",
|
||||||
|
"# Segundo fetch: continúa donde se quedó (recuerda el cursor)\n",
|
||||||
|
"batch2 = await psub.fetch(10, timeout=1)\n",
|
||||||
|
"print()\n",
|
||||||
|
"print(f\"Segundo fetch: {len(batch2)} msgs restantes ->\", [m.data.decode() for m in batch2])\n",
|
||||||
|
"for m in batch2:\n",
|
||||||
|
" await m.ack()"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "15b96e37",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"### 3 · Deduplicación por `Nats-Msg-Id`\n",
|
||||||
|
"\n",
|
||||||
|
"Si un publisher reintenta por un timeout de red, podría enviar el mismo mensaje dos veces. JetStream lo evita: si dos publicaciones llevan el mismo header **`Nats-Msg-Id`** dentro de la `duplicate_window`, la segunda se reconoce como **duplicada** y **no** se almacena. El `PubAck` lo indica con `duplicate=True`."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "fdc4c498",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"await js.add_stream(name=\"DEMO_DEDUP\", subjects=[\"demo.dedup.>\"],\n",
|
||||||
|
" storage=StorageType.FILE, duplicate_window=120)\n",
|
||||||
|
"\n",
|
||||||
|
"# Publicar dos veces el MISMO Nats-Msg-Id\n",
|
||||||
|
"ack1 = await js.publish(\"demo.dedup.pago\", b\"cobro 50e\", headers={\"Nats-Msg-Id\": \"pago-0001\"})\n",
|
||||||
|
"ack2 = await js.publish(\"demo.dedup.pago\", b\"cobro 50e\", headers={\"Nats-Msg-Id\": \"pago-0001\"})\n",
|
||||||
|
"\n",
|
||||||
|
"print(f\"1a publicacion: seq={ack1.seq} duplicate={ack1.duplicate}\")\n",
|
||||||
|
"print(f\"2a publicacion: seq={ack2.seq} duplicate={ack2.duplicate} <- detectada como duplicado\")\n",
|
||||||
|
"\n",
|
||||||
|
"# Un Msg-Id distinto sí se almacena\n",
|
||||||
|
"ack3 = await js.publish(\"demo.dedup.pago\", b\"cobro 30e\", headers={\"Nats-Msg-Id\": \"pago-0002\"})\n",
|
||||||
|
"print(f\"3a publicacion (id nuevo): seq={ack3.seq} duplicate={ack3.duplicate}\")\n",
|
||||||
|
"\n",
|
||||||
|
"st = (await js.stream_info(\"DEMO_DEDUP\")).state\n",
|
||||||
|
"print()\n",
|
||||||
|
"print(f\"Mensajes realmente almacenados en el stream: {st.messages} (2 publicaciones unicas, 1 descartada)\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "da997d2e",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"### 4 · Retención `workqueue`: la cola de trabajo\n",
|
||||||
|
"\n",
|
||||||
|
"Con `retention=workqueue`, cada mensaje se **borra del stream en cuanto un consumer lo confirma**. Es el patrón de cola de tareas distribuida: los mensajes se reparten entre workers y desaparecen al procesarse, así el stream no crece sin fin."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "c7c4a35f",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"from nats.js.api import RetentionPolicy, StorageType\n",
|
||||||
|
"\n",
|
||||||
|
"await js.add_stream(name=\"DEMO_WQ\", subjects=[\"demo.wq.>\"],\n",
|
||||||
|
" storage=StorageType.FILE, retention=RetentionPolicy.WORK_QUEUE)\n",
|
||||||
|
"\n",
|
||||||
|
"# Encolar 5 tareas\n",
|
||||||
|
"for i in range(5):\n",
|
||||||
|
" await js.publish(\"demo.wq.tareas\", f\"tarea-{i}\".encode())\n",
|
||||||
|
"\n",
|
||||||
|
"antes = (await js.stream_info(\"DEMO_WQ\")).state.messages\n",
|
||||||
|
"print(f\"Tareas encoladas en el stream: {antes}\")\n",
|
||||||
|
"\n",
|
||||||
|
"# Un worker consume y confirma 3\n",
|
||||||
|
"wsub = await js.pull_subscribe(\"demo.wq.>\", durable=\"worker\")\n",
|
||||||
|
"tres = await wsub.fetch(3, timeout=2)\n",
|
||||||
|
"for m in tres:\n",
|
||||||
|
" print(f\" procesada: {m.data.decode()}\")\n",
|
||||||
|
" await m.ack() # al confirmar, JetStream BORRA el mensaje del stream\n",
|
||||||
|
"\n",
|
||||||
|
"await asyncio.sleep(0.3)\n",
|
||||||
|
"despues = (await js.stream_info(\"DEMO_WQ\")).state.messages\n",
|
||||||
|
"print()\n",
|
||||||
|
"print(f\"Mensajes restantes en el stream tras 3 acks: {despues} (workqueue borra lo confirmado: {antes} -> {despues})\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "5a68f26f",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"### 5 · Políticas de entrega (replay)\n",
|
||||||
|
"\n",
|
||||||
|
"Al crear un consumer se elige **desde dónde** empieza a leer (`DeliverPolicy`):\n",
|
||||||
|
"\n",
|
||||||
|
"- `ALL` — todo el historial desde el principio (lo habitual para reprocesar).\n",
|
||||||
|
"- `LAST` — solo el último mensaje del stream.\n",
|
||||||
|
"- `NEW` — solo lo que llegue a partir de ahora.\n",
|
||||||
|
"- `BY_START_SEQUENCE` / `BY_START_TIME` — desde una secuencia o instante concretos.\n",
|
||||||
|
"\n",
|
||||||
|
"Comparamos `ALL` vs `LAST` sobre el stream de límites (que tiene 6 mensajes)."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "a341929e",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"from nats.js.api import ConsumerConfig, DeliverPolicy\n",
|
||||||
|
"\n",
|
||||||
|
"# Consumer que reproduce TODO el historial\n",
|
||||||
|
"all_sub = await js.pull_subscribe(\n",
|
||||||
|
" \"demo.limits.>\", durable=\"replay-all\",\n",
|
||||||
|
" config=ConsumerConfig(deliver_policy=DeliverPolicy.ALL),\n",
|
||||||
|
")\n",
|
||||||
|
"todos = await all_sub.fetch(50, timeout=1)\n",
|
||||||
|
"print(f\"DeliverPolicy.ALL -> {len(todos)} mensajes:\", [m.data.decode() for m in todos])\n",
|
||||||
|
"for m in todos:\n",
|
||||||
|
" await m.ack()\n",
|
||||||
|
"\n",
|
||||||
|
"# Consumer que solo entrega el ÚLTIMO\n",
|
||||||
|
"last_sub = await js.pull_subscribe(\n",
|
||||||
|
" \"demo.limits.>\", durable=\"replay-last\",\n",
|
||||||
|
" config=ConsumerConfig(deliver_policy=DeliverPolicy.LAST),\n",
|
||||||
|
")\n",
|
||||||
|
"ultimo = await last_sub.fetch(50, timeout=1)\n",
|
||||||
|
"print(f\"DeliverPolicy.LAST -> {len(ultimo)} mensaje :\", [m.data.decode() for m in ultimo])\n",
|
||||||
|
"for m in ultimo:\n",
|
||||||
|
" await m.ack()"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "0fba1dc2",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Parte B · Simulador de rendimiento (interactivo)\n",
|
||||||
|
"\n",
|
||||||
|
"Pulsa **▶ Ejecutar benchmark** y verás cómo **un publisher** inunda el broker con miles de mensajes que **varios subscribers** reciben simultáneamente (fan-out). La gráfica se actualiza **en movimiento** mientras corre:\n",
|
||||||
|
"\n",
|
||||||
|
"- **Izquierda** — mensajes acumulados: enviados (publisher) vs recibidos (suma de todos los subs).\n",
|
||||||
|
"- **Derecha** — throughput instantáneo (msgs/s recibidos) muestreado cada ~80 ms.\n",
|
||||||
|
"\n",
|
||||||
|
"Ajusta los sliders para cambiar el número de mensajes y de subscribers. Con más mensajes (p. ej. 100.000) la animación dura más y se aprecia mejor la curva."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "4240cd89",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"import ipywidgets as widgets\n",
|
||||||
|
"from IPython.display import display, clear_output\n",
|
||||||
|
"import matplotlib.pyplot as plt\n",
|
||||||
|
"import asyncio, time\n",
|
||||||
|
"import nats\n",
|
||||||
|
"\n",
|
||||||
|
"# --- widgets ---\n",
|
||||||
|
"n_msgs_w = widgets.IntSlider(value=20000, min=1000, max=100000, step=1000,\n",
|
||||||
|
" description=\"Mensajes:\", style={\"description_width\": \"initial\"},\n",
|
||||||
|
" layout=widgets.Layout(width=\"380px\"))\n",
|
||||||
|
"n_subs_w = widgets.IntSlider(value=3, min=1, max=8, step=1,\n",
|
||||||
|
" description=\"Subscribers:\", style={\"description_width\": \"initial\"})\n",
|
||||||
|
"run_btn = widgets.Button(description=\"▶ Ejecutar benchmark\", button_style=\"success\",\n",
|
||||||
|
" layout=widgets.Layout(width=\"220px\"))\n",
|
||||||
|
"plot_out = widgets.Output()\n",
|
||||||
|
"log_out = widgets.Output()\n",
|
||||||
|
"\n",
|
||||||
|
"SUBJECT = \"bench.load\"\n",
|
||||||
|
"PAYLOAD = b\"x\" * 128 # 128 bytes por mensaje\n",
|
||||||
|
"\n",
|
||||||
|
"def _throughput(ts, recv):\n",
|
||||||
|
" thr = [0.0]\n",
|
||||||
|
" for i in range(1, len(ts)):\n",
|
||||||
|
" dt = ts[i] - ts[i-1]\n",
|
||||||
|
" thr.append((recv[i] - recv[i-1]) / dt if dt > 0 else 0.0)\n",
|
||||||
|
" return thr\n",
|
||||||
|
"\n",
|
||||||
|
"def render(history, n_subs, n_msgs, done=False):\n",
|
||||||
|
" ts = [h[0] for h in history]\n",
|
||||||
|
" sent = [h[1] for h in history]\n",
|
||||||
|
" recv = [h[2] for h in history]\n",
|
||||||
|
" thr = _throughput(ts, recv)\n",
|
||||||
|
" with plot_out:\n",
|
||||||
|
" clear_output(wait=True)\n",
|
||||||
|
" fig, (a1, a2) = plt.subplots(1, 2, figsize=(11, 3.6))\n",
|
||||||
|
" a1.plot(ts, sent, label=\"enviados (pub)\", color=\"#2563eb\", lw=2)\n",
|
||||||
|
" a1.plot(ts, recv, label=f\"recibidos (Σ {n_subs} subs)\", color=\"#16a34a\", lw=2)\n",
|
||||||
|
" a1.set_xlabel(\"segundos\"); a1.set_ylabel(\"mensajes acumulados\")\n",
|
||||||
|
" a1.set_title(\"Publisher vs subscribers\"); a1.legend(loc=\"upper left\")\n",
|
||||||
|
" a2.plot(ts, thr, color=\"#db2777\", lw=2)\n",
|
||||||
|
" a2.set_xlabel(\"segundos\"); a2.set_ylabel(\"msgs/s recibidos\")\n",
|
||||||
|
" a2.set_title(\"Throughput instantáneo\")\n",
|
||||||
|
" estado = \"✓ DONE\" if done else \"● corriendo…\"\n",
|
||||||
|
" fig.suptitle(f\"[{estado}] {n_msgs:,} msgs → {n_subs} subs \"\n",
|
||||||
|
" f\"enviados={sent[-1]:,} recibidos={recv[-1]:,}\", fontsize=11)\n",
|
||||||
|
" plt.tight_layout(); plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"async def run_benchmark(n_msgs, n_subs, live=True):\n",
|
||||||
|
" \"\"\"1 publisher -> n_subs subscribers. Devuelve (history, counters).\"\"\"\n",
|
||||||
|
" nc = await nats.connect(NATS_URL, name=\"benchmark\")\n",
|
||||||
|
" counters = [0] * n_subs\n",
|
||||||
|
"\n",
|
||||||
|
" def make_cb(i):\n",
|
||||||
|
" async def cb(msg):\n",
|
||||||
|
" counters[i] += 1\n",
|
||||||
|
" return cb\n",
|
||||||
|
"\n",
|
||||||
|
" subs = [await nc.subscribe(SUBJECT, cb=make_cb(i)) for i in range(n_subs)]\n",
|
||||||
|
" history = [] # (t, enviados, recibidos_total)\n",
|
||||||
|
" sent = 0\n",
|
||||||
|
" t0 = time.monotonic()\n",
|
||||||
|
"\n",
|
||||||
|
" async def publish_all():\n",
|
||||||
|
" nonlocal sent\n",
|
||||||
|
" for k in range(n_msgs):\n",
|
||||||
|
" await nc.publish(SUBJECT, PAYLOAD)\n",
|
||||||
|
" sent += 1\n",
|
||||||
|
" if k % 1000 == 0:\n",
|
||||||
|
" await nc.flush()\n",
|
||||||
|
" await asyncio.sleep(0) # ceder al event loop (deja correr callbacks)\n",
|
||||||
|
" await nc.flush()\n",
|
||||||
|
"\n",
|
||||||
|
" task = asyncio.create_task(publish_all())\n",
|
||||||
|
"\n",
|
||||||
|
" # Muestreo para la gráfica en movimiento\n",
|
||||||
|
" while not task.done() or sum(counters) < sent:\n",
|
||||||
|
" await asyncio.sleep(0.08)\n",
|
||||||
|
" history.append((time.monotonic() - t0, sent, sum(counters)))\n",
|
||||||
|
" if live:\n",
|
||||||
|
" render(history, n_subs, n_msgs)\n",
|
||||||
|
" if time.monotonic() - t0 > 30: # tope de seguridad\n",
|
||||||
|
" break\n",
|
||||||
|
" await task\n",
|
||||||
|
"\n",
|
||||||
|
" # Drenaje final (que los callbacks alcancen al publisher)\n",
|
||||||
|
" for _ in range(40):\n",
|
||||||
|
" if sum(counters) >= sent:\n",
|
||||||
|
" break\n",
|
||||||
|
" await asyncio.sleep(0.05)\n",
|
||||||
|
" history.append((time.monotonic() - t0, sent, sum(counters)))\n",
|
||||||
|
" if live:\n",
|
||||||
|
" render(history, n_subs, n_msgs, done=True)\n",
|
||||||
|
"\n",
|
||||||
|
" for s in subs:\n",
|
||||||
|
" await s.unsubscribe()\n",
|
||||||
|
" await nc.drain()\n",
|
||||||
|
" return history, counters\n",
|
||||||
|
"\n",
|
||||||
|
"def on_click(_):\n",
|
||||||
|
" run_btn.disabled = True\n",
|
||||||
|
" with log_out:\n",
|
||||||
|
" clear_output()\n",
|
||||||
|
" print(f\"Lanzando: {n_msgs_w.value:,} mensajes → {n_subs_w.value} subscribers …\")\n",
|
||||||
|
" async def go():\n",
|
||||||
|
" try:\n",
|
||||||
|
" history, counters = await run_benchmark(n_msgs_w.value, n_subs_w.value, live=True)\n",
|
||||||
|
" dur = history[-1][0]\n",
|
||||||
|
" recv = sum(counters)\n",
|
||||||
|
" with log_out:\n",
|
||||||
|
" print(f\"OK en {dur:.2f}s\")\n",
|
||||||
|
" print(f\" enviados : {n_msgs_w.value:,}\")\n",
|
||||||
|
" print(f\" recibidos: {recv:,} (fan-out ×{n_subs_w.value} = {recv/max(n_msgs_w.value,1):.2f} por mensaje)\")\n",
|
||||||
|
" print(f\" throughput pub : {n_msgs_w.value/dur:,.0f} msgs/s\")\n",
|
||||||
|
" print(f\" throughput recv: {recv/dur:,.0f} msgs/s (entregas totales)\")\n",
|
||||||
|
" print(f\" por subscriber : {counters}\")\n",
|
||||||
|
" finally:\n",
|
||||||
|
" run_btn.disabled = False\n",
|
||||||
|
" asyncio.ensure_future(go())\n",
|
||||||
|
"\n",
|
||||||
|
"run_btn.on_click(on_click)\n",
|
||||||
|
"display(widgets.HBox([n_msgs_w, n_subs_w]), run_btn, plot_out, log_out)\n",
|
||||||
|
"print(\"Simulador listo. Pulsa el botón para lanzar el benchmark.\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "de3caaa5",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"### Verificación (headless)\n",
|
||||||
|
"\n",
|
||||||
|
"La celda anterior renderiza el widget para pulsarlo en JupyterLab. Aquí ejecutamos el mismo benchmark **una vez de forma programática** (sin botón) para dejar evidencia ejecutada: una corrida real de 15.000 mensajes a 3 subscribers con su gráfica final."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": null,
|
||||||
|
"id": "34f06087",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"hist, counters = await run_benchmark(15000, 3, live=False)\n",
|
||||||
|
"\n",
|
||||||
|
"ts = [h[0] for h in hist]\n",
|
||||||
|
"sent = [h[1] for h in hist]\n",
|
||||||
|
"recv = [h[2] for h in hist]\n",
|
||||||
|
"dur = ts[-1]\n",
|
||||||
|
"\n",
|
||||||
|
"fig, ax = plt.subplots(figsize=(9, 3.6))\n",
|
||||||
|
"ax.plot(ts, sent, label=\"enviados (pub)\", color=\"#2563eb\", lw=2)\n",
|
||||||
|
"ax.plot(ts, recv, label=\"recibidos (Σ 3 subs)\", color=\"#16a34a\", lw=2)\n",
|
||||||
|
"ax.set_xlabel(\"segundos\"); ax.set_ylabel(\"mensajes acumulados\")\n",
|
||||||
|
"ax.set_title(f\"Benchmark headless: 15.000 msgs → 3 subs en {dur:.2f}s\")\n",
|
||||||
|
"ax.legend(loc=\"upper left\")\n",
|
||||||
|
"plt.tight_layout(); plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"print(f\"enviados : 15,000\")\n",
|
||||||
|
"print(f\"recibidos: {sum(counters):,} por sub -> {counters}\")\n",
|
||||||
|
"print(f\"throughput pub : {15000/dur:,.0f} msgs/s\")\n",
|
||||||
|
"print(f\"throughput recv: {sum(counters)/dur:,.0f} msgs/s (entregas totales, fan-out x3)\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "501f5185",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Resumen\n",
|
||||||
|
"\n",
|
||||||
|
"**JetStream a fondo:**\n",
|
||||||
|
"- Un **stream** persiste mensajes con políticas de **storage** (file/memory), **retention** (limits/interest/workqueue) y **límites** (max_msgs/max_age).\n",
|
||||||
|
"- Los **consumers** (pull/push, durable/ephemeral) leen a su ritmo y **confirman** (ack) cada mensaje → entrega *at-least-once*.\n",
|
||||||
|
"- **Dedup** por `Nats-Msg-Id` evita duplicados por reintentos.\n",
|
||||||
|
"- **workqueue** borra cada mensaje al confirmarse → cola de trabajo.\n",
|
||||||
|
"- **DeliverPolicy** controla el replay (all/last/new/by_sequence/by_time).\n",
|
||||||
|
"\n",
|
||||||
|
"**Simulador:** demuestra el fan-out a escala — un publisher alimenta a N subscribers con miles de mensajes y la gráfica en vivo muestra que el throughput de recepción sigue al de envío (cada mensaje se entrega a los N subscribers).\n",
|
||||||
|
"\n",
|
||||||
|
"### Limpieza\n",
|
||||||
|
"\n",
|
||||||
|
"```python\n",
|
||||||
|
"for s in (\"DEMO_LIMITS\", \"DEMO_DEDUP\", \"DEMO_WQ\"):\n",
|
||||||
|
" try: await js.delete_stream(s)\n",
|
||||||
|
" except Exception: pass\n",
|
||||||
|
"await nc.drain()\n",
|
||||||
|
"```"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "Python 3 (ipykernel)",
|
||||||
|
"language": "python",
|
||||||
|
"name": "python3"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 5
|
||||||
|
}
|
||||||
@@ -427,12 +427,21 @@
|
|||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3 (ipykernel)",
|
"display_name": "NATS analysis",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "nats"
|
||||||
},
|
},
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"name": ""
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.12.3"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
|
|||||||
@@ -1,21 +1,5 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "a5e95055",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"# NATS pub/sub — 02 · Queue groups, Request/Reply y JetStream\n",
|
|
||||||
"\n",
|
|
||||||
"En el notebook 01 vimos el *fan-out* del core: una publicación llega a **todos** los subscribers. Aquí cubrimos tres patrones que construyen sobre esa base:\n",
|
|
||||||
"\n",
|
|
||||||
"1. **Queue groups** — repartir la carga: varios *workers* comparten el trabajo y cada mensaje lo procesa **uno solo**.\n",
|
|
||||||
"2. **Request/Reply** — RPC sobre mensajería: un cliente pregunta y espera la respuesta de un servicio.\n",
|
|
||||||
"3. **JetStream** — la capa de **persistencia**: streams que almacenan los mensajes y permiten *replay*, a diferencia del core *fire-and-forget*.\n",
|
|
||||||
"\n",
|
|
||||||
"> Requiere el broker `nats_demo` del notebook 01. La primera celda lo arranca de forma idempotente, así que este notebook también funciona de forma aislada."
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"id": "1ab4e0d1",
|
"id": "1ab4e0d1",
|
||||||
@@ -374,6 +358,22 @@
|
|||||||
"\n",
|
"\n",
|
||||||
"**Siguiente** → `03_procesos_reales.ipynb`: hasta ahora todo ha ocurrido dentro de un mismo kernel. Allí lanzamos publisher y subscribers como **procesos del sistema operativo independientes** para ver el desacople real."
|
"**Siguiente** → `03_procesos_reales.ipynb`: hasta ahora todo ha ocurrido dentro de un mismo kernel. Allí lanzamos publisher y subscribers como **procesos del sistema operativo independientes** para ver el desacople real."
|
||||||
]
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "a5e95055",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"# NATS pub/sub — 02 · Queue groups, Request/Reply y JetStream\n",
|
||||||
|
"\n",
|
||||||
|
"En el notebook 01 vimos el *fan-out* del core: una publicación llega a **todos** los subscribers. Aquí cubrimos tres patrones que construyen sobre esa base:\n",
|
||||||
|
"\n",
|
||||||
|
"1. **Queue groups** — repartir la carga: varios *workers* comparten el trabajo y cada mensaje lo procesa **uno solo**.\n",
|
||||||
|
"2. **Request/Reply** — RPC sobre mensajería: un cliente pregunta y espera la respuesta de un servicio.\n",
|
||||||
|
"3. **JetStream** — la capa de **persistencia**: streams que almacenan los mensajes y permiten *replay*, a diferencia del core *fire-and-forget*.\n",
|
||||||
|
"\n",
|
||||||
|
"> Requiere el broker `nats_demo` del notebook 01. La primera celda lo arranca de forma idempotente, así que este notebook también funciona de forma aislada."
|
||||||
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
@@ -383,7 +383,16 @@
|
|||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"name": ""
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.12.3"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
|
|||||||
@@ -510,7 +510,16 @@
|
|||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"name": ""
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.12.3"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user