feat: notebook 04 — JetStream a fondo + simulador de rendimiento interactivo

JetStream: anatomia de streams (storage/retention/limits), consumers pull durables
con ack y cursor, dedup por Nats-Msg-Id, retencion workqueue, deliver policies.
Simulador: boton ipywidgets que lanza 1 publisher -> N subscribers con miles de
mensajes y grafica en movimiento (acumulado + throughput instantaneo).
This commit is contained in:
Egutierrez
2026-06-03 21:52:38 +02:00
parent 595930f3c8
commit c9e28b8135
12 changed files with 2029 additions and 104 deletions
Binary file not shown.
+11 -1
View File
@@ -6,7 +6,17 @@
}, },
"17291f8f-336e-4a5f-8407-a4d22149d581": { "17291f8f-336e-4a5f-8407-a4d22149d581": {
"version": "2.4.0", "version": "2.4.0",
"created_at": "2026-06-03T17:46:15.674895+00:00", "created_at": "2026-06-03T19:04:11.779086+00:00",
"document_version": "2.0.0"
},
"6a0e97d4-34d4-44f3-8a47-243934257256": {
"version": "2.4.0",
"created_at": "2026-06-03T19:16:39.520669+00:00",
"document_version": "2.0.0"
},
"bc0d99c9-d3bf-49c2-a87a-b38af47d1bff": {
"version": "2.4.0",
"created_at": "2026-06-03T19:18:34.134263+00:00",
"document_version": "2.0.0" "document_version": "2.0.0"
} }
} }
Binary file not shown.
+9 -5
View File
@@ -1,12 +1,16 @@
{ {
"mcpServers": { "mcpServers": {
"jupyter": { "jupyter": {
"command": "/home/enmanuel/fn_registry/analysis/nats/.venv/bin/jupyter-mcp-server", "command": "bash",
"args": [ "args": [
"--transport", "stdio", "/home/enmanuel/fn_registry/bash/functions/infra/jupyter_mcp_serve.sh"
"--jupyter-url", "http://localhost:8890", ],
"--jupyter-token", "" "env": {
] "JUPYTER_MCP_VENV": "/home/enmanuel/fn_registry/analysis/nats/.venv",
"JUPYTER_MCP_ROOT": "/home/enmanuel/fn_registry/analysis/nats",
"JUPYTER_MCP_PORT": "8890",
"JUPYTER_MCP_TOKEN": ""
}
} }
} }
} }
+3
View File
@@ -21,9 +21,12 @@ Analisis didactico de **NATS** como sistema de mensajeria pub/sub entre procesos
| `01_core_pubsub.ipynb` | Modelo base: conexion, publish/subscribe, fan-out a N subscribers, wildcards `*` y `>`. | | `01_core_pubsub.ipynb` | Modelo base: conexion, publish/subscribe, fan-out a N subscribers, wildcards `*` y `>`. |
| `02_queue_request_jetstream.ipynb` | Queue groups (reparto de carga), request/reply (RPC con inbox temporal), JetStream (stream persistente + consumer durable + replay). | | `02_queue_request_jetstream.ipynb` | Queue groups (reparto de carga), request/reply (RPC con inbox temporal), JetStream (stream persistente + consumer durable + replay). |
| `03_procesos_reales.ipynb` | Publisher y subscribers como **procesos del SO independientes** (`subprocess`), cada uno con su PID. Demuestra el desacople real: el publisher no conoce a sus subscribers. | | `03_procesos_reales.ipynb` | Publisher y subscribers como **procesos del SO independientes** (`subprocess`), cada uno con su PID. Demuestra el desacople real: el publisher no conoce a sus subscribers. |
| `04_jetstream_benchmark.ipynb` | **JetStream a fondo** (storage/retention/limits, consumers durables + ack + cursor, dedup por `Nats-Msg-Id`, retención `workqueue`, deliver policies) + **simulador de rendimiento interactivo**: un botón `ipywidgets` que lanza 1 publisher → N subscribers con miles de mensajes y una gráfica en movimiento (acumulado pub vs subs + throughput instantáneo). |
Los scripts `notebooks/procs/publisher.py` y `notebooks/procs/subscriber.py` son los programas que el notebook 03 lanza como procesos reales. Los scripts `notebooks/procs/publisher.py` y `notebooks/procs/subscriber.py` son los programas que el notebook 03 lanza como procesos reales.
El notebook 04 requiere `ipywidgets` (incluido en el `.venv` del análisis). El simulador es interactivo: al abrir el notebook en JupyterLab, ejecuta sus celdas hasta el widget y pulsa **▶ Ejecutar benchmark** (los sliders ajustan número de mensajes y de subscribers). La gráfica se anima mientras corre.
### Como usar ### Como usar
1. Requiere Docker disponible (con permisos para el usuario actual). La primera celda de cada notebook arranca el broker de forma idempotente con la funcion `ensure_nats`, asi que cada notebook funciona de forma aislada. 1. Requiere Docker disponible (con permisos para el usuario actual). La primera celda de cada notebook arranca el broker de forma idempotente con la funcion `ensure_nats`, asi que cada notebook funciona de forma aislada.
+435
View File
@@ -0,0 +1,435 @@
#!/usr/bin/env python3
"""Generador del notebook 04 del analisis NATS: JetStream a fondo + simulador de
rendimiento interactivo.
Construye el .ipynb con nbformat (sin ejecutar). La ejecucion se hace despues
contra el servidor Jupyter del analisis (puerto 8890, su propio venv) para que
los outputs queden persistidos y el widget interactivo se renderice en JupyterLab.
Reproducible: re-ejecutar este script regenera el notebook 04 desde cero.
"""
from pathlib import Path
import nbformat as nbf
NBDIR = Path("/home/enmanuel/fn_registry/analysis/nats/notebooks")
def build(filename: str, cells: list[tuple[str, str]]) -> None:
nb = nbf.v4.new_notebook()
nb.metadata["kernelspec"] = {
"name": "python3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
}
nb.cells = [
nbf.v4.new_markdown_cell(src) if typ == "md" else nbf.v4.new_code_cell(src)
for typ, src in cells
]
nbf.write(nb, str(NBDIR / filename))
print(f"escrito {filename}: {len(cells)} celdas")
ENSURE_NATS = '''import subprocess, time, json
NATS_CONTAINER = "nats_demo"
NATS_PORT = 4222
NATS_URL = f"nats://127.0.0.1:{NATS_PORT}"
def _docker(*args, check=True):
return subprocess.run(["docker", *args], capture_output=True, text=True, check=check)
def ensure_nats(name=NATS_CONTAINER, port=NATS_PORT):
"""Arranca un broker NATS en Docker de forma idempotente. Devuelve el estado."""
out = _docker("ps", "-a", "--filter", f"name=^{name}$", "--format", "{{.State}}", check=False).stdout.strip()
if out == "running":
state = "already-running"
elif out in ("exited", "created", "paused"):
_docker("start", name)
state = "started"
else:
_docker("run", "-d", "--name", name, "-p", f"{port}:4222", "-p", "8222:8222",
"nats:latest", "-js", "-m", "8222")
state = "created"
time.sleep(1.0)
return state'''
nb4 = [
("md", """# NATS pub/sub — 04 · JetStream a fondo y simulador de rendimiento
Este notebook tiene dos partes:
1. **JetStream a fondo** — más allá del replay básico del notebook 02: anatomía de un stream (almacenamiento, políticas de retención, límites), tipos de consumer, *acks*, deduplicación y políticas de entrega.
2. **Simulador de rendimiento interactivo** — un botón que, al pulsarlo, lanza un publisher que envía **miles de mensajes** a varios subscribers, con una **gráfica en movimiento** que muestra el throughput en tiempo real.
> Requiere el broker `nats_demo` (arrancado por la primera celda) y `ipywidgets` (incluido en el venv del análisis)."""),
# ---- Parte A: JetStream a fondo ----
("md", """## Parte A · JetStream a fondo
### Setup
JetStream es la capa de persistencia de NATS. Mientras el core es *fire-and-forget*, JetStream **almacena** los mensajes en un *stream* y permite leerlos con *consumers* que controlan el ritmo, confirman (*ack*) cada mensaje y pueden reproducir el historial."""),
("code", ENSURE_NATS + '''
import asyncio
import nats
print("Broker:", ensure_nats())
nc = await nats.connect(NATS_URL, name="notebook-04")
js = nc.jetstream()
print("JetStream context listo. account info:")
ai = await js.account_info()
print(f" streams={ai.streams} consumers={ai.consumers} memory={ai.memory} storage={ai.storage}")'''),
("md", """### 1 · Anatomía de un stream
Un **stream** se define por:
- **subjects** — qué subjects captura (`pedidos.>`).
- **storage** — `file` (persistente en disco) o `memory` (rápido, se pierde al reiniciar).
- **retention** — cuándo se descartan los mensajes:
- `limits` (por defecto): se guardan hasta tocar un límite (`max_msgs`, `max_bytes`, `max_age`).
- `interest`: se descartan cuando todos los consumers interesados los han recibido.
- `workqueue`: cada mensaje se borra en cuanto **un** consumer lo confirma (cola de trabajo).
- **límites** — `max_msgs`, `max_bytes`, `max_age` (segundos), `max_msg_size`.
- **duplicate_window** — ventana de deduplicación (ver §3).
Creamos un stream `limits` con almacenamiento en disco y un tope de mensajes."""),
("code", '''from nats.js.api import StreamConfig, RetentionPolicy, StorageType, DiscardPolicy
# Recrear limpio para que la demo sea determinista
for s in ("DEMO_LIMITS", "DEMO_DEDUP", "DEMO_WQ"):
try:
await js.delete_stream(s)
except Exception:
pass
cfg = StreamConfig(
name="DEMO_LIMITS",
subjects=["demo.limits.>"],
storage=StorageType.FILE,
retention=RetentionPolicy.LIMITS,
max_msgs=1000, # tope de mensajes
max_age=3600, # 1 hora (segundos)
discard=DiscardPolicy.OLD, # al llegar al tope, descarta los más viejos
duplicate_window=120, # ventana de dedup: 120 s
)
info = await js.add_stream(cfg)
c = info.config
print("Stream creado:")
print(f" name : {c.name}")
print(f" subjects : {c.subjects}")
print(f" storage : {c.storage}")
print(f" retention : {c.retention}")
print(f" max_msgs : {c.max_msgs}")
print(f" max_age (s) : {c.max_age}")
print(f" discard : {c.discard}")
print(f" dup_window (s): {c.duplicate_window}")'''),
("md", """### 2 · Consumers: pull, durable, ack
Un **consumer** es la vista de lectura sobre un stream. Dos ejes:
- **pull vs push**: en *pull* el cliente pide mensajes cuando quiere (`fetch`); en *push* el servidor los empuja según llegan.
- **durable vs ephemeral**: un consumer *durable* tiene nombre y **recuerda su posición** (cursor) entre reconexiones; uno *ephemeral* desaparece al cerrarse.
El **ack** es la confirmación de procesado. Hasta que un mensaje no se confirma, el consumer lo considera *pendiente* y, si pasa el `ack_wait`, lo **reentrega**. Esto da entrega *at-least-once*."""),
("code", '''# Publicar 6 mensajes en el stream de límites
for i in range(6):
await js.publish("demo.limits.eventos", f"evento-{i}".encode())
# Pull consumer DURABLE: recuerda su cursor entre fetches
psub = await js.pull_subscribe("demo.limits.>", durable="procesador-A")
# Traer 4 y confirmarlos (ack)
batch = await psub.fetch(4, timeout=2)
print("Primer fetch (4 msgs):")
for m in batch:
print(f" seq={m.metadata.sequence.stream} {m.data.decode()}")
await m.ack()
# Estado del consumer: cuántos quedan pendientes / entregados
ci = await psub.consumer_info()
print()
print(f"Consumer 'procesador-A':")
print(f" num_pending : {ci.num_pending} (mensajes sin entregar todavía)")
print(f" num_ack_pending: {ci.num_ack_pending} (entregados sin ack)")
print(f" delivered.stream_seq: {ci.delivered.stream_seq}")
# Segundo fetch: continúa donde se quedó (recuerda el cursor)
batch2 = await psub.fetch(10, timeout=1)
print()
print(f"Segundo fetch: {len(batch2)} msgs restantes ->", [m.data.decode() for m in batch2])
for m in batch2:
await m.ack()'''),
("md", """### 3 · Deduplicación por `Nats-Msg-Id`
Si un publisher reintenta por un timeout de red, podría enviar el mismo mensaje dos veces. JetStream lo evita: si dos publicaciones llevan el mismo header **`Nats-Msg-Id`** dentro de la `duplicate_window`, la segunda se reconoce como **duplicada** y **no** se almacena. El `PubAck` lo indica con `duplicate=True`."""),
("code", '''await js.add_stream(name="DEMO_DEDUP", subjects=["demo.dedup.>"],
storage=StorageType.FILE, duplicate_window=120)
# Publicar dos veces el MISMO Nats-Msg-Id
ack1 = await js.publish("demo.dedup.pago", b"cobro 50e", headers={"Nats-Msg-Id": "pago-0001"})
ack2 = await js.publish("demo.dedup.pago", b"cobro 50e", headers={"Nats-Msg-Id": "pago-0001"})
print(f"1a publicacion: seq={ack1.seq} duplicate={ack1.duplicate}")
print(f"2a publicacion: seq={ack2.seq} duplicate={ack2.duplicate} <- detectada como duplicado")
# Un Msg-Id distinto sí se almacena
ack3 = await js.publish("demo.dedup.pago", b"cobro 30e", headers={"Nats-Msg-Id": "pago-0002"})
print(f"3a publicacion (id nuevo): seq={ack3.seq} duplicate={ack3.duplicate}")
st = (await js.stream_info("DEMO_DEDUP")).state
print()
print(f"Mensajes realmente almacenados en el stream: {st.messages} (2 publicaciones unicas, 1 descartada)")'''),
("md", """### 4 · Retención `workqueue`: la cola de trabajo
Con `retention=workqueue`, cada mensaje se **borra del stream en cuanto un consumer lo confirma**. Es el patrón de cola de tareas distribuida: los mensajes se reparten entre workers y desaparecen al procesarse, así el stream no crece sin fin."""),
("code", '''from nats.js.api import RetentionPolicy, StorageType
await js.add_stream(name="DEMO_WQ", subjects=["demo.wq.>"],
storage=StorageType.FILE, retention=RetentionPolicy.WORK_QUEUE)
# Encolar 5 tareas
for i in range(5):
await js.publish("demo.wq.tareas", f"tarea-{i}".encode())
antes = (await js.stream_info("DEMO_WQ")).state.messages
print(f"Tareas encoladas en el stream: {antes}")
# Un worker consume y confirma 3
wsub = await js.pull_subscribe("demo.wq.>", durable="worker")
tres = await wsub.fetch(3, timeout=2)
for m in tres:
print(f" procesada: {m.data.decode()}")
await m.ack() # al confirmar, JetStream BORRA el mensaje del stream
await asyncio.sleep(0.3)
despues = (await js.stream_info("DEMO_WQ")).state.messages
print()
print(f"Mensajes restantes en el stream tras 3 acks: {despues} (workqueue borra lo confirmado: {antes} -> {despues})")'''),
("md", """### 5 · Políticas de entrega (replay)
Al crear un consumer se elige **desde dónde** empieza a leer (`DeliverPolicy`):
- `ALL` — todo el historial desde el principio (lo habitual para reprocesar).
- `LAST` — solo el último mensaje del stream.
- `NEW` — solo lo que llegue a partir de ahora.
- `BY_START_SEQUENCE` / `BY_START_TIME` — desde una secuencia o instante concretos.
Comparamos `ALL` vs `LAST` sobre el stream de límites (que tiene 6 mensajes)."""),
("code", '''from nats.js.api import ConsumerConfig, DeliverPolicy
# Consumer que reproduce TODO el historial
all_sub = await js.pull_subscribe(
"demo.limits.>", durable="replay-all",
config=ConsumerConfig(deliver_policy=DeliverPolicy.ALL),
)
todos = await all_sub.fetch(50, timeout=1)
print(f"DeliverPolicy.ALL -> {len(todos)} mensajes:", [m.data.decode() for m in todos])
for m in todos:
await m.ack()
# Consumer que solo entrega el ÚLTIMO
last_sub = await js.pull_subscribe(
"demo.limits.>", durable="replay-last",
config=ConsumerConfig(deliver_policy=DeliverPolicy.LAST),
)
ultimo = await last_sub.fetch(50, timeout=1)
print(f"DeliverPolicy.LAST -> {len(ultimo)} mensaje :", [m.data.decode() for m in ultimo])
for m in ultimo:
await m.ack()'''),
# ---- Parte B: simulador interactivo ----
("md", """## Parte B · Simulador de rendimiento (interactivo)
Pulsa **▶ Ejecutar benchmark** y verás cómo **un publisher** inunda el broker con miles de mensajes que **varios subscribers** reciben simultáneamente (fan-out). La gráfica se actualiza **en movimiento** mientras corre:
- **Izquierda** — mensajes acumulados: enviados (publisher) vs recibidos (suma de todos los subs).
- **Derecha** — throughput instantáneo (msgs/s recibidos) muestreado cada ~80 ms.
Ajusta los sliders para cambiar el número de mensajes y de subscribers. Con más mensajes (p. ej. 100.000) la animación dura más y se aprecia mejor la curva."""),
("code", '''import ipywidgets as widgets
from IPython.display import display, clear_output
import matplotlib.pyplot as plt
import asyncio, time
import nats
# --- widgets ---
n_msgs_w = widgets.IntSlider(value=20000, min=1000, max=100000, step=1000,
description="Mensajes:", style={"description_width": "initial"},
layout=widgets.Layout(width="380px"))
n_subs_w = widgets.IntSlider(value=3, min=1, max=8, step=1,
description="Subscribers:", style={"description_width": "initial"})
run_btn = widgets.Button(description="▶ Ejecutar benchmark", button_style="success",
layout=widgets.Layout(width="220px"))
plot_out = widgets.Output()
log_out = widgets.Output()
SUBJECT = "bench.load"
PAYLOAD = b"x" * 128 # 128 bytes por mensaje
def _throughput(ts, recv):
thr = [0.0]
for i in range(1, len(ts)):
dt = ts[i] - ts[i-1]
thr.append((recv[i] - recv[i-1]) / dt if dt > 0 else 0.0)
return thr
def render(history, n_subs, n_msgs, done=False):
ts = [h[0] for h in history]
sent = [h[1] for h in history]
recv = [h[2] for h in history]
thr = _throughput(ts, recv)
with plot_out:
clear_output(wait=True)
fig, (a1, a2) = plt.subplots(1, 2, figsize=(11, 3.6))
a1.plot(ts, sent, label="enviados (pub)", color="#2563eb", lw=2)
a1.plot(ts, recv, label=f"recibidos (Σ {n_subs} subs)", color="#16a34a", lw=2)
a1.set_xlabel("segundos"); a1.set_ylabel("mensajes acumulados")
a1.set_title("Publisher vs subscribers"); a1.legend(loc="upper left")
a2.plot(ts, thr, color="#db2777", lw=2)
a2.set_xlabel("segundos"); a2.set_ylabel("msgs/s recibidos")
a2.set_title("Throughput instantáneo")
estado = "✓ DONE" if done else "● corriendo…"
fig.suptitle(f"[{estado}] {n_msgs:,} msgs → {n_subs} subs "
f"enviados={sent[-1]:,} recibidos={recv[-1]:,}", fontsize=11)
plt.tight_layout(); plt.show()
async def run_benchmark(n_msgs, n_subs, live=True):
"""1 publisher -> n_subs subscribers. Devuelve (history, counters)."""
nc = await nats.connect(NATS_URL, name="benchmark")
counters = [0] * n_subs
def make_cb(i):
async def cb(msg):
counters[i] += 1
return cb
subs = [await nc.subscribe(SUBJECT, cb=make_cb(i)) for i in range(n_subs)]
history = [] # (t, enviados, recibidos_total)
sent = 0
t0 = time.monotonic()
async def publish_all():
nonlocal sent
for k in range(n_msgs):
await nc.publish(SUBJECT, PAYLOAD)
sent += 1
if k % 1000 == 0:
await nc.flush()
await asyncio.sleep(0) # ceder al event loop (deja correr callbacks)
await nc.flush()
task = asyncio.create_task(publish_all())
# Muestreo para la gráfica en movimiento
while not task.done() or sum(counters) < sent:
await asyncio.sleep(0.08)
history.append((time.monotonic() - t0, sent, sum(counters)))
if live:
render(history, n_subs, n_msgs)
if time.monotonic() - t0 > 30: # tope de seguridad
break
await task
# Drenaje final (que los callbacks alcancen al publisher)
for _ in range(40):
if sum(counters) >= sent:
break
await asyncio.sleep(0.05)
history.append((time.monotonic() - t0, sent, sum(counters)))
if live:
render(history, n_subs, n_msgs, done=True)
for s in subs:
await s.unsubscribe()
await nc.drain()
return history, counters
def on_click(_):
run_btn.disabled = True
with log_out:
clear_output()
print(f"Lanzando: {n_msgs_w.value:,} mensajes → {n_subs_w.value} subscribers …")
async def go():
try:
history, counters = await run_benchmark(n_msgs_w.value, n_subs_w.value, live=True)
dur = history[-1][0]
recv = sum(counters)
with log_out:
print(f"OK en {dur:.2f}s")
print(f" enviados : {n_msgs_w.value:,}")
print(f" recibidos: {recv:,} (fan-out ×{n_subs_w.value} = {recv/max(n_msgs_w.value,1):.2f} por mensaje)")
print(f" throughput pub : {n_msgs_w.value/dur:,.0f} msgs/s")
print(f" throughput recv: {recv/dur:,.0f} msgs/s (entregas totales)")
print(f" por subscriber : {counters}")
finally:
run_btn.disabled = False
asyncio.ensure_future(go())
run_btn.on_click(on_click)
display(widgets.HBox([n_msgs_w, n_subs_w]), run_btn, plot_out, log_out)
print("Simulador listo. Pulsa el botón para lanzar el benchmark.")'''),
("md", """### Verificación (headless)
La celda anterior renderiza el widget para pulsarlo en JupyterLab. Aquí ejecutamos el mismo benchmark **una vez de forma programática** (sin botón) para dejar evidencia ejecutada: una corrida real de 15.000 mensajes a 3 subscribers con su gráfica final."""),
("code", '''hist, counters = await run_benchmark(15000, 3, live=False)
ts = [h[0] for h in hist]
sent = [h[1] for h in hist]
recv = [h[2] for h in hist]
dur = ts[-1]
fig, ax = plt.subplots(figsize=(9, 3.6))
ax.plot(ts, sent, label="enviados (pub)", color="#2563eb", lw=2)
ax.plot(ts, recv, label="recibidos (Σ 3 subs)", color="#16a34a", lw=2)
ax.set_xlabel("segundos"); ax.set_ylabel("mensajes acumulados")
ax.set_title(f"Benchmark headless: 15.000 msgs → 3 subs en {dur:.2f}s")
ax.legend(loc="upper left")
plt.tight_layout(); plt.show()
print(f"enviados : 15,000")
print(f"recibidos: {sum(counters):,} por sub -> {counters}")
print(f"throughput pub : {15000/dur:,.0f} msgs/s")
print(f"throughput recv: {sum(counters)/dur:,.0f} msgs/s (entregas totales, fan-out x3)")'''),
("md", """## Resumen
**JetStream a fondo:**
- Un **stream** persiste mensajes con políticas de **storage** (file/memory), **retention** (limits/interest/workqueue) y **límites** (max_msgs/max_age).
- Los **consumers** (pull/push, durable/ephemeral) leen a su ritmo y **confirman** (ack) cada mensaje → entrega *at-least-once*.
- **Dedup** por `Nats-Msg-Id` evita duplicados por reintentos.
- **workqueue** borra cada mensaje al confirmarse → cola de trabajo.
- **DeliverPolicy** controla el replay (all/last/new/by_sequence/by_time).
**Simulador:** demuestra el fan-out a escala — un publisher alimenta a N subscribers con miles de mensajes y la gráfica en vivo muestra que el throughput de recepción sigue al de envío (cada mensaje se entrega a los N subscribers).
### Limpieza
```python
for s in ("DEMO_LIMITS", "DEMO_DEDUP", "DEMO_WQ"):
try: await js.delete_stream(s)
except Exception: pass
await nc.drain()
```"""),
]
if __name__ == "__main__":
build("04_jetstream_benchmark.ipynb", nb4)
print("OK: notebook 04 generado en", NBDIR)
+285 -77
View File
@@ -5,21 +5,21 @@
Abre: http://localhost:8890 Abre: http://localhost:8890
Ctrl+C para detener Ctrl+C para detener
[W 2026-06-03 19:41:26.217 ServerApp] ServerApp.token config is deprecated in 2.0. Use IdentityProvider.token. [W 2026-06-03 21:18:04.909 ServerApp] ServerApp.token config is deprecated in 2.0. Use IdentityProvider.token.
[I 2026-06-03 19:41:26.656 ServerApp] jupyter_lsp | extension was successfully linked. [I 2026-06-03 21:18:05.346 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2026-06-03 19:41:26.658 ServerApp] jupyter_mcp_server | extension was successfully linked. [I 2026-06-03 21:18:05.348 ServerApp] jupyter_mcp_server | extension was successfully linked.
[I 2026-06-03 19:41:26.658 ServerApp] jupyter_mcp_tools | extension was successfully linked. [I 2026-06-03 21:18:05.348 ServerApp] jupyter_mcp_tools | extension was successfully linked.
[I 2026-06-03 19:41:26.660 ServerApp] jupyter_server_fileid | extension was successfully linked. [I 2026-06-03 21:18:05.349 ServerApp] jupyter_server_fileid | extension was successfully linked.
[I 2026-06-03 19:41:26.662 ServerApp] jupyter_server_nbmodel | extension was successfully linked. [I 2026-06-03 21:18:05.351 ServerApp] jupyter_server_nbmodel | extension was successfully linked.
[I 2026-06-03 19:41:26.663 ServerApp] jupyter_server_terminals | extension was successfully linked. [I 2026-06-03 21:18:05.352 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2026-06-03 19:41:26.665 ServerApp] jupyter_server_ydoc | extension was successfully linked. [I 2026-06-03 21:18:05.354 ServerApp] jupyter_server_ydoc | extension was successfully linked.
[I 2026-06-03 19:41:26.667 ServerApp] jupyterlab | extension was successfully linked. [I 2026-06-03 21:18:05.356 ServerApp] jupyterlab | extension was successfully linked.
[I 2026-06-03 19:41:26.669 ServerApp] notebook | extension was successfully linked. [I 2026-06-03 21:18:05.358 ServerApp] notebook | extension was successfully linked.
[I 2026-06-03 19:41:26.670 ServerApp] notebook_shim | extension was successfully linked. [I 2026-06-03 21:18:05.360 ServerApp] notebook_shim | extension was successfully linked.
[W 2026-06-03 19:41:26.680 ServerApp] All authentication is disabled. Anyone who can connect to this server will be able to run code. [W 2026-06-03 21:18:05.369 ServerApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
[I 2026-06-03 19:41:26.681 ServerApp] notebook_shim | extension was successfully loaded. [I 2026-06-03 21:18:05.369 ServerApp] notebook_shim | extension was successfully loaded.
[I 2026-06-03 19:41:26.682 ServerApp] jupyter_lsp | extension was successfully loaded. [I 2026-06-03 21:18:05.370 ServerApp] jupyter_lsp | extension was successfully loaded.
[06/03/26 19:41:26] INFO Auto-enrolled document extension.py:195 [06/03/26 21:18:05] INFO Auto-enrolled document extension.py:195
'notebook.ipynb' as 'default' 'notebook.ipynb' as 'default'
INFO Jupyter MCP Server Extension extension.py:197 INFO Jupyter MCP Server Extension extension.py:197
settings initialized settings initialized
@@ -28,62 +28,43 @@
INFO - Health check: /mcp/healthz extension.py:235 INFO - Health check: /mcp/healthz extension.py:235
INFO - List tools: /mcp/tools/list extension.py:236 INFO - List tools: /mcp/tools/list extension.py:236
INFO - Call tool: /mcp/tools/call extension.py:237 INFO - Call tool: /mcp/tools/call extension.py:237
[I 2026-06-03 19:41:26.784 ServerApp] jupyter_mcp_server | extension was successfully loaded. [I 2026-06-03 21:18:05.464 ServerApp] jupyter_mcp_server | extension was successfully loaded.
[I 2026-06-03 19:41:26.784 ServerApp] Registered jupyter_mcp_tools server extension [I 2026-06-03 21:18:05.464 ServerApp] Registered jupyter_mcp_tools server extension
[I 2026-06-03 19:41:26.784 ServerApp] jupyter_mcp_tools | extension was successfully loaded. [I 2026-06-03 21:18:05.464 ServerApp] jupyter_mcp_tools | extension was successfully loaded.
[I 2026-06-03 19:41:26.784 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager [I 2026-06-03 21:18:05.464 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
[I 2026-06-03 19:41:26.784 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/enmanuel/fn_registry/analysis/nats [I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/enmanuel/fn_registry/analysis/nats
[I 2026-06-03 19:41:26.784 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/enmanuel/.local/share/jupyter/file_id_manager.db [I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/enmanuel/.local/share/jupyter/file_id_manager.db
[I 2026-06-03 19:41:26.785 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file. [I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
[I 2026-06-03 19:41:26.785 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE [I 2026-06-03 21:18:05.464 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
[I 2026-06-03 19:41:26.785 FileIdExtension] Attached event listeners. [I 2026-06-03 21:18:05.464 FileIdExtension] Attached event listeners.
[I 2026-06-03 19:41:26.785 ServerApp] jupyter_server_fileid | extension was successfully loaded. [I 2026-06-03 21:18:05.464 ServerApp] jupyter_server_fileid | extension was successfully loaded.
[I 2026-06-03 19:41:26.785 ServerApp] jupyter_server_nbmodel | extension was successfully loaded. [I 2026-06-03 21:18:05.465 ServerApp] jupyter_server_nbmodel | extension was successfully loaded.
[I 2026-06-03 19:41:26.786 ServerApp] jupyter_server_terminals | extension was successfully loaded. [I 2026-06-03 21:18:05.465 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2026-06-03 19:41:26.787 ServerApp] jupyter_server_ydoc | extension was successfully loaded. [I 2026-06-03 21:18:05.467 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
[I 2026-06-03 19:41:26.789 LabApp] JupyterLab extension loaded from /home/enmanuel/fn_registry/analysis/nats/.venv/lib/python3.12/site-packages/jupyterlab [I 2026-06-03 21:18:05.468 LabApp] JupyterLab extension loaded from /home/enmanuel/fn_registry/analysis/nats/.venv/lib/python3.12/site-packages/jupyterlab
[I 2026-06-03 19:41:26.789 LabApp] JupyterLab application directory is /home/enmanuel/fn_registry/analysis/nats/.venv/share/jupyter/lab [I 2026-06-03 21:18:05.468 LabApp] JupyterLab application directory is /home/enmanuel/fn_registry/analysis/nats/.venv/share/jupyter/lab
[I 2026-06-03 19:41:26.789 LabApp] Extension Manager is 'pypi'. [I 2026-06-03 21:18:05.469 LabApp] Extension Manager is 'pypi'.
[I 2026-06-03 19:41:26.810 ServerApp] jupyterlab | extension was successfully loaded. [I 2026-06-03 21:18:05.489 ServerApp] jupyterlab | extension was successfully loaded.
[I 2026-06-03 19:41:26.811 ServerApp] notebook | extension was successfully loaded. [I 2026-06-03 21:18:05.491 ServerApp] notebook | extension was successfully loaded.
[I 2026-06-03 19:41:26.812 ServerApp] Serving notebooks from local directory: /home/enmanuel/fn_registry/analysis/nats [I 2026-06-03 21:18:05.491 ServerApp] Serving notebooks from local directory: /home/enmanuel/fn_registry/analysis/nats
[I 2026-06-03 19:41:26.812 ServerApp] Jupyter Server 2.19.0 is running at: [I 2026-06-03 21:18:05.491 ServerApp] Jupyter Server 2.19.0 is running at:
[I 2026-06-03 19:41:26.812 ServerApp] http://localhost:8890/lab [I 2026-06-03 21:18:05.491 ServerApp] http://localhost:8890/lab
[I 2026-06-03 19:41:26.812 ServerApp] http://127.0.0.1:8890/lab [I 2026-06-03 21:18:05.491 ServerApp] http://127.0.0.1:8890/lab
[I 2026-06-03 19:41:26.812 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [I 2026-06-03 21:18:05.491 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2026-06-03 19:41:26.876 ServerApp] Skipped non-installed server(s): basedpyright, bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyrefly, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server [I 2026-06-03 21:18:05.555 ServerApp] Skipped non-installed server(s): basedpyright, bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyrefly, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[I 2026-06-03 19:46:15.569 ServerApp] Request for Y document (previously indexed) 'notebooks/01_core_pubsub.ipynb' with room ID: f0c81420-7285-42fa-afcf-aa618e651df6 [I 2026-06-03 21:18:25.232 ServerApp] 302 GET / (@127.0.0.1) 0.24ms
[I 2026-06-03 19:46:15.654 LabApp] Build is up to date [I 2026-06-03 21:18:33.966 LabApp] Build is up to date
[I 2026-06-03 19:46:15.656 ServerApp] MCP Tools WebSocket connection opened [I 2026-06-03 21:18:33.967 ServerApp] Request for Y document (previously indexed) 'notebooks/01_core_pubsub.ipynb' with room ID: f0c81420-7285-42fa-afcf-aa618e651df6
[W 2026-06-03 19:46:15.669 ServerApp] The websocket_ping_timeout (90000) cannot be longer than the websocket_ping_interval (30000). [I 2026-06-03 21:18:34.060 ServerApp] MCP Tools WebSocket connection opened
[I 2026-06-03 21:18:34.061 ServerApp] Request for Y document (previously indexed) 'notebooks/02_queue_request_jetstream.ipynb' with room ID: fadc19b0-8f98-429c-a6d5-056a05bf47cd
[I 2026-06-03 21:18:34.061 ServerApp] Request for Y document (previously indexed) 'notebooks/03_procesos_reales.ipynb' with room ID: 61f241c1-9f7e-46e7-8fce-e61e1c568d1c
[W 2026-06-03 21:18:34.072 ServerApp] The websocket_ping_timeout (90000) cannot be longer than the websocket_ping_interval (30000).
Setting websocket_ping_timeout=30000 Setting websocket_ping_timeout=30000
[I 2026-06-03 19:46:15.672 YDocExtension] Creating FileLoader for: notebooks/01_core_pubsub.ipynb [I 2026-06-03 21:18:34.076 YDocExtension] Creating FileLoader for: notebooks/01_core_pubsub.ipynb
[I 2026-06-03 19:46:15.673 YDocExtension] Watching file: notebooks/01_core_pubsub.ipynb [I 2026-06-03 21:18:34.078 YDocExtension] Watching file: notebooks/01_core_pubsub.ipynb
[I 2026-06-03 19:46:15.675 ServerApp] Initializing room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 [I 2026-06-03 21:18:34.078 ServerApp] Registered 414 tools
[I 2026-06-03 19:46:15.676 ServerApp] Registered 414 tools [I 2026-06-03 21:18:34.081 ServerApp] Initializing room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:46:15.716 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from the ystore SQLiteYStore [06/03/26 21:18:34] ERROR Notebook JSON is invalid: Additional __init__.py:98
[I 2026-06-03 19:46:15.719 ServerApp] Content in file notebooks/01_core_pubsub.ipynb is out-of-sync with the ystore SQLiteYStore
[I 2026-06-03 19:46:15.720 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from file notebooks/01_core_pubsub.ipynb
[I 2026-06-03 19:46:16.735 ServerApp] Saving the content from room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:46:16.738 YDocExtension] Saving file: notebooks/01_core_pubsub.ipynb
[I 2026-06-03 19:47:15.407 YDocExtension] Processed 31 Y patches in one minute
[I 2026-06-03 19:47:15.407 YDocExtension] Connected Y users: 2
[I 2026-06-03 19:48:15.407 YDocExtension] Processed 16 Y patches in one minute
[I 2026-06-03 19:48:15.408 YDocExtension] Connected Y users: 2
[I 2026-06-03 19:49:15.408 YDocExtension] Processed 16 Y patches in one minute
[I 2026-06-03 19:49:15.409 YDocExtension] Connected Y users: 2
[I 2026-06-03 19:49:52.746 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[W 2026-06-03 19:49:52.770 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
[I 2026-06-03 19:50:15.409 YDocExtension] Processed 16 Y patches in one minute
[I 2026-06-03 19:50:15.409 YDocExtension] Connected Y users: 2
[I 2026-06-03 19:50:16.900 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:50:17.929 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:50:22.970 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:50:26.008 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:50:30.051 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:50:34.088 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 19:50:37.127 ServerApp] Out-of-band changes. Overwriting the content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[06/03/26 19:50:37] ERROR Notebook JSON is invalid: Additional __init__.py:98
properties are not allowed properties are not allowed
('transient' was unexpected) ('transient' was unexpected)
@@ -101,10 +82,237 @@
'metadata': {}, 'metadata': {},
'output_type': 'display_data', 'output_type': 'display_data',
'transient': {}} 'transient': {}}
[W 2026-06-03 19:50:37.149 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted [I 2026-06-03 21:18:34.118 YDocExtension] Creating FileLoader for: notebooks/02_queue_request_jetstream.ipynb
[I 2026-06-03 19:51:15.410 YDocExtension] Processed 16 Y patches in one minute [W 2026-06-03 21:18:34.120 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
[I 2026-06-03 19:51:15.411 YDocExtension] Connected Y users: 2 [I 2026-06-03 21:18:34.121 YDocExtension] Watching file: notebooks/02_queue_request_jetstream.ipynb
[I 2026-06-03 19:52:15.414 YDocExtension] Processed 16 Y patches in one minute [I 2026-06-03 21:18:34.122 ServerApp] Initializing room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd
[I 2026-06-03 19:52:15.414 YDocExtension] Connected Y users: 2 ERROR Notebook JSON is invalid: Additional __init__.py:98
[I 2026-06-03 19:53:15.417 YDocExtension] Processed 16 Y patches in one minute properties are not allowed
[I 2026-06-03 19:53:15.417 YDocExtension] Connected Y users: 2 ('transient' was unexpected)
Failed validating
'additionalProperties' in
display_data:
On
instance['cells'][5]['outputs'][0]:
{'data': {'image/png':
'iVBORw0KGgoAAAANSUhEUgAAArIAAAEiCAY
AAAAF9zFeAAAAOnRFWHRTb2Z0d2Fy...',
'text/plain': '<Figure
size 700x300 with 1 Axes>'},
'metadata': {},
'output_type': 'display_data',
'transient': {}}
[W 2026-06-03 21:18:34.130 ServerApp] Notebook notebooks/02_queue_request_jetstream.ipynb is not trusted
[I 2026-06-03 21:18:34.133 YDocExtension] Creating FileLoader for: notebooks/03_procesos_reales.ipynb
[I 2026-06-03 21:18:34.133 YDocExtension] Watching file: notebooks/03_procesos_reales.ipynb
[I 2026-06-03 21:18:34.134 ServerApp] Initializing room json:notebook:61f241c1-9f7e-46e7-8fce-e61e1c568d1c
ERROR Notebook JSON is invalid: Additional __init__.py:98
properties are not allowed
('transient' was unexpected)
Failed validating
'additionalProperties' in
display_data:
On
instance['cells'][8]['outputs'][0]:
{'data': {'image/png':
'iVBORw0KGgoAAAANSUhEUgAAArIAAAExCAY
AAACAtUFrAAAAOnRFWHRTb2Z0d2Fy...',
'text/plain': '<Figure
size 700x320 with 1 Axes>'},
'metadata': {},
'output_type': 'display_data',
'transient': {}}
[W 2026-06-03 21:18:34.148 ServerApp] Notebook notebooks/03_procesos_reales.ipynb is not trusted
[I 2026-06-03 21:18:34.153 ServerApp] Content in room json:notebook:61f241c1-9f7e-46e7-8fce-e61e1c568d1c loaded from the ystore SQLiteYStore
[I 2026-06-03 21:18:34.157 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from the ystore SQLiteYStore
[I 2026-06-03 21:18:34.158 ServerApp] Content in file notebooks/01_core_pubsub.ipynb is out-of-sync with the ystore SQLiteYStore
[I 2026-06-03 21:18:34.158 ServerApp] Content in room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6 loaded from file notebooks/01_core_pubsub.ipynb
[I 2026-06-03 21:18:34.162 ServerApp] Content in room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd loaded from the ystore SQLiteYStore
[I 2026-06-03 21:18:34.164 ServerApp] Content in file notebooks/02_queue_request_jetstream.ipynb is out-of-sync with the ystore SQLiteYStore
[I 2026-06-03 21:18:34.164 ServerApp] Content in room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd loaded from file notebooks/02_queue_request_jetstream.ipynb
[I 2026-06-03 21:18:34.561 ServerApp] Kernel started: adb9c688-aaa0-442b-a665-77464065e37c
[I 2026-06-03 21:18:34.563 ServerApp] Kernel started: 97684b48-3a52-4e4b-9c7f-4d1dd08ab5a6
[I 2026-06-03 21:18:34.564 ServerApp] Kernel started: 3a4122ce-ea0f-4bd5-8953-7726be1f5de2
[I 2026-06-03 21:18:34.860 ServerApp] Adapting from protocol version 5.3 (kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2) to 5.4 (client).
[I 2026-06-03 21:18:34.861 ServerApp] Connecting to kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2.
[I 2026-06-03 21:18:34.883 ServerApp] Adapting from protocol version 5.3 (kernel adb9c688-aaa0-442b-a665-77464065e37c) to 5.4 (client).
[I 2026-06-03 21:18:34.884 ServerApp] Connecting to kernel adb9c688-aaa0-442b-a665-77464065e37c.
[I 2026-06-03 21:18:34.905 ServerApp] Adapting from protocol version 5.3 (kernel 97684b48-3a52-4e4b-9c7f-4d1dd08ab5a6) to 5.4 (client).
[I 2026-06-03 21:18:34.906 ServerApp] Connecting to kernel 97684b48-3a52-4e4b-9c7f-4d1dd08ab5a6.
[I 2026-06-03 21:18:34.921 ServerApp] Adapting from protocol version 5.3 (kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2) to 5.4 (client).
[I 2026-06-03 21:18:34.922 ServerApp] Connecting to kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2.
[I 2026-06-03 21:18:34.939 ServerApp] Adapting from protocol version 5.3 (kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2) to 5.4 (client).
[I 2026-06-03 21:18:34.944 ServerApp] Connecting to kernel 3a4122ce-ea0f-4bd5-8953-7726be1f5de2.
[I 2026-06-03 21:18:35.172 ServerApp] Saving the content from room json:notebook:61f241c1-9f7e-46e7-8fce-e61e1c568d1c
[I 2026-06-03 21:18:35.178 YDocExtension] Saving file: notebooks/03_procesos_reales.ipynb
[W 2026-06-03 21:18:35.179 ServerApp] Notebook notebooks/03_procesos_reales.ipynb is not trusted
[06/03/26 21:18:35] ERROR Notebook JSON is invalid: __init__.py:134
Additional properties are not
allowed ('transient' was
unexpected)
Failed validating
'additionalProperties' in
display_data:
On
instance['cells'][8]['outputs'][0]:
{'data': {'image/png':
'iVBORw0KGgoAAAANSUhEUgAAArIAAAExCA
YAAACAtUFrAAAAOnRFWHRTb2Z0d2Fy...',
'text/plain': '<Figure
size 700x320 with 1 Axes>'},
'metadata': {},
'output_type': 'display_data',
'transient': {}}
[I 2026-06-03 21:18:35.269 ServerApp] Saving the content from room json:notebook:f0c81420-7285-42fa-afcf-aa618e651df6
[I 2026-06-03 21:18:35.275 YDocExtension] Saving file: notebooks/01_core_pubsub.ipynb
[W 2026-06-03 21:18:35.276 ServerApp] Notebook notebooks/01_core_pubsub.ipynb is not trusted
ERROR Notebook JSON is invalid: __init__.py:134
Additional properties are not
allowed ('transient' was
unexpected)
Failed validating
'additionalProperties' in
display_data:
On
instance['cells'][12]['outputs'][0]
:
{'data': {'image/png':
'iVBORw0KGgoAAAANSUhEUgAAA3oAAAD5CA
YAAABxn0eTAAAAOnRFWHRTb2Z0d2Fy...',
'text/plain': '<Figure
size 900x260 with 1 Axes>'},
'metadata': {},
'output_type': 'display_data',
'transient': {}}
[I 2026-06-03 21:18:35.436 ServerApp] Saving the content from room json:notebook:fadc19b0-8f98-429c-a6d5-056a05bf47cd
[I 2026-06-03 21:18:35.442 YDocExtension] Saving file: notebooks/02_queue_request_jetstream.ipynb
[W 2026-06-03 21:18:35.444 ServerApp] Notebook notebooks/02_queue_request_jetstream.ipynb is not trusted
ERROR Notebook JSON is invalid: __init__.py:134
Additional properties are not
allowed ('transient' was
unexpected)
Failed validating
'additionalProperties' in
display_data:
On
instance['cells'][5]['outputs'][0]:
{'data': {'image/png':
'iVBORw0KGgoAAAANSUhEUgAAArIAAAEiCA
YAAAAF9zFeAAAAOnRFWHRTb2Z0d2Fy...',
'text/plain': '<Figure
size 700x300 with 1 Axes>'},
'metadata': {},
'output_type': 'display_data',
'transient': {}}
[I 2026-06-03 21:19:33.804 YDocExtension] Processed 59 Y patches in one minute
[I 2026-06-03 21:19:33.805 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:20:33.805 YDocExtension] Processed 34 Y patches in one minute
[I 2026-06-03 21:20:33.806 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:21:33.811 YDocExtension] Processed 32 Y patches in one minute
[I 2026-06-03 21:21:33.811 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:22:33.815 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:22:33.815 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:23:33.818 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:23:33.818 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:24:33.824 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:24:33.824 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:25:33.828 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:25:33.828 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:26:33.830 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:26:33.830 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:27:33.831 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:27:33.831 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:28:33.836 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:28:33.836 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:29:33.838 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:29:33.838 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:30:33.838 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:30:33.839 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:31:33.842 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:31:33.842 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:32:33.845 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:32:33.845 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:33:33.851 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:33:33.851 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:34:33.854 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:34:33.855 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:35:33.857 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:35:33.857 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:36:33.863 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:36:33.863 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:37:33.866 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:37:33.867 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:38:33.868 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:38:33.868 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:39:33.875 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:39:33.875 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:40:33.878 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:40:33.879 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:41:33.881 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:41:33.881 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:42:33.882 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:42:33.883 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:43:33.886 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:43:33.886 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:44:33.888 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:44:33.888 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:45:33.889 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:45:33.890 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:46:33.894 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:46:33.894 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:47:33.896 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:47:33.896 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:48:33.898 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:48:33.898 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:49:33.903 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:49:33.903 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:50:33.906 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:50:33.906 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:51:18.556 ServerApp] Kernel started: 8391112c-c8cd-40ef-a0fd-2590d1861f16
[W 2026-06-03 21:51:18.559 ServerApp] Notebook notebooks/04_jetstream_benchmark.ipynb is not trusted
[I 2026-06-03 21:51:18.873 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:18.875 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:20.065 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:edf9cdd2-e2e57dbe9e1db13c2369bd49
[I 2026-06-03 21:51:20.070 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:20.129 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:20.131 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:20.190 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:ce162d97-bcd67ae41b07a72954c689b8
[I 2026-06-03 21:51:20.194 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:20.250 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:20.251 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:20.314 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:833d59f8-a7593d93633152b0c03ea130
[I 2026-06-03 21:51:20.318 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:20.379 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:20.380 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:20.433 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:cbe487ea-c1f5e2dffb3b06659f933964
[I 2026-06-03 21:51:20.439 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:20.490 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:20.492 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:20.849 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:e5ea749a-9c5b0957ba57e53d7545e31c
[I 2026-06-03 21:51:20.853 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:20.904 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:20.906 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:20.957 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:30826324-d09e241fc54796adcd6a376c
[I 2026-06-03 21:51:20.959 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:33.909 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:51:33.909 YDocExtension] Connected Y users: 4
[I 2026-06-03 21:51:42.074 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:42.075 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:42.431 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:c27a340b-675498de282e4e38aec57c86
[I 2026-06-03 21:51:42.436 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:51:42.492 ServerApp] Adapting from protocol version 5.3 (kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16) to 5.4 (client).
[I 2026-06-03 21:51:42.493 ServerApp] Connecting to kernel 8391112c-c8cd-40ef-a0fd-2590d1861f16.
[I 2026-06-03 21:51:42.762 ServerApp] Starting buffering for 8391112c-c8cd-40ef-a0fd-2590d1861f16:a54123fe-7e8fcf758646f2ed9ec519e2
[I 2026-06-03 21:51:42.764 ServerApp] Saving file at /notebooks/04_jetstream_benchmark.ipynb
[I 2026-06-03 21:52:33.911 YDocExtension] Processed 20 Y patches in one minute
[I 2026-06-03 21:52:33.912 YDocExtension] Connected Y users: 4
@@ -0,0 +1,539 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "72277c66",
"metadata": {},
"source": [
"# NATS pub/sub — 04 · JetStream a fondo y simulador de rendimiento\n",
"\n",
"Este notebook tiene dos partes:\n",
"\n",
"1. **JetStream a fondo** — más allá del replay básico del notebook 02: anatomía de un stream (almacenamiento, políticas de retención, límites), tipos de consumer, *acks*, deduplicación y políticas de entrega.\n",
"2. **Simulador de rendimiento interactivo** — un botón que, al pulsarlo, lanza un publisher que envía **miles de mensajes** a varios subscribers, con una **gráfica en movimiento** que muestra el throughput en tiempo real.\n",
"\n",
"> Requiere el broker `nats_demo` (arrancado por la primera celda) y `ipywidgets` (incluido en el venv del análisis)."
]
},
{
"cell_type": "markdown",
"id": "530eac60",
"metadata": {},
"source": [
"## Parte A · JetStream a fondo\n",
"\n",
"### Setup\n",
"\n",
"JetStream es la capa de persistencia de NATS. Mientras el core es *fire-and-forget*, JetStream **almacena** los mensajes en un *stream* y permite leerlos con *consumers* que controlan el ritmo, confirman (*ack*) cada mensaje y pueden reproducir el historial."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "847ca130",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Broker: already-running\n",
"JetStream context listo. account info:\n",
" streams=1 consumers=1 memory=0 storage=260\n"
]
}
],
"source": [
"import subprocess, time, json\n",
"\n",
"NATS_CONTAINER = \"nats_demo\"\n",
"NATS_PORT = 4222\n",
"NATS_URL = f\"nats://127.0.0.1:{NATS_PORT}\"\n",
"\n",
"def _docker(*args, check=True):\n",
" return subprocess.run([\"docker\", *args], capture_output=True, text=True, check=check)\n",
"\n",
"def ensure_nats(name=NATS_CONTAINER, port=NATS_PORT):\n",
" \"\"\"Arranca un broker NATS en Docker de forma idempotente. Devuelve el estado.\"\"\"\n",
" out = _docker(\"ps\", \"-a\", \"--filter\", f\"name=^{name}$\", \"--format\", \"{{.State}}\", check=False).stdout.strip()\n",
" if out == \"running\":\n",
" state = \"already-running\"\n",
" elif out in (\"exited\", \"created\", \"paused\"):\n",
" _docker(\"start\", name)\n",
" state = \"started\"\n",
" else:\n",
" _docker(\"run\", \"-d\", \"--name\", name, \"-p\", f\"{port}:4222\", \"-p\", \"8222:8222\",\n",
" \"nats:latest\", \"-js\", \"-m\", \"8222\")\n",
" state = \"created\"\n",
" time.sleep(1.0)\n",
" return state\n",
"\n",
"import asyncio\n",
"import nats\n",
"\n",
"print(\"Broker:\", ensure_nats())\n",
"nc = await nats.connect(NATS_URL, name=\"notebook-04\")\n",
"js = nc.jetstream()\n",
"print(\"JetStream context listo. account info:\")\n",
"ai = await js.account_info()\n",
"print(f\" streams={ai.streams} consumers={ai.consumers} memory={ai.memory} storage={ai.storage}\")"
]
},
{
"cell_type": "markdown",
"id": "b55873ec",
"metadata": {},
"source": [
"### 1 · Anatomía de un stream\n",
"\n",
"Un **stream** se define por:\n",
"\n",
"- **subjects** — qué subjects captura (`pedidos.>`).\n",
"- **storage** — `file` (persistente en disco) o `memory` (rápido, se pierde al reiniciar).\n",
"- **retention** — cuándo se descartan los mensajes:\n",
" - `limits` (por defecto): se guardan hasta tocar un límite (`max_msgs`, `max_bytes`, `max_age`).\n",
" - `interest`: se descartan cuando todos los consumers interesados los han recibido.\n",
" - `workqueue`: cada mensaje se borra en cuanto **un** consumer lo confirma (cola de trabajo).\n",
"- **límites** — `max_msgs`, `max_bytes`, `max_age` (segundos), `max_msg_size`.\n",
"- **duplicate_window** — ventana de deduplicación (ver §3).\n",
"\n",
"Creamos un stream `limits` con almacenamiento en disco y un tope de mensajes."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67118a66",
"metadata": {},
"outputs": [],
"source": [
"from nats.js.api import StreamConfig, RetentionPolicy, StorageType, DiscardPolicy\n",
"\n",
"# Recrear limpio para que la demo sea determinista\n",
"for s in (\"DEMO_LIMITS\", \"DEMO_DEDUP\", \"DEMO_WQ\"):\n",
" try:\n",
" await js.delete_stream(s)\n",
" except Exception:\n",
" pass\n",
"\n",
"cfg = StreamConfig(\n",
" name=\"DEMO_LIMITS\",\n",
" subjects=[\"demo.limits.>\"],\n",
" storage=StorageType.FILE,\n",
" retention=RetentionPolicy.LIMITS,\n",
" max_msgs=1000, # tope de mensajes\n",
" max_age=3600, # 1 hora (segundos)\n",
" discard=DiscardPolicy.OLD, # al llegar al tope, descarta los más viejos\n",
" duplicate_window=120, # ventana de dedup: 120 s\n",
")\n",
"info = await js.add_stream(cfg)\n",
"c = info.config\n",
"print(\"Stream creado:\")\n",
"print(f\" name : {c.name}\")\n",
"print(f\" subjects : {c.subjects}\")\n",
"print(f\" storage : {c.storage}\")\n",
"print(f\" retention : {c.retention}\")\n",
"print(f\" max_msgs : {c.max_msgs}\")\n",
"print(f\" max_age (s) : {c.max_age}\")\n",
"print(f\" discard : {c.discard}\")\n",
"print(f\" dup_window (s): {c.duplicate_window}\")"
]
},
{
"cell_type": "markdown",
"id": "9bf78aac",
"metadata": {},
"source": [
"### 2 · Consumers: pull, durable, ack\n",
"\n",
"Un **consumer** es la vista de lectura sobre un stream. Dos ejes:\n",
"\n",
"- **pull vs push**: en *pull* el cliente pide mensajes cuando quiere (`fetch`); en *push* el servidor los empuja según llegan.\n",
"- **durable vs ephemeral**: un consumer *durable* tiene nombre y **recuerda su posición** (cursor) entre reconexiones; uno *ephemeral* desaparece al cerrarse.\n",
"\n",
"El **ack** es la confirmación de procesado. Hasta que un mensaje no se confirma, el consumer lo considera *pendiente* y, si pasa el `ack_wait`, lo **reentrega**. Esto da entrega *at-least-once*."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "499e73f7",
"metadata": {},
"outputs": [],
"source": [
"# Publicar 6 mensajes en el stream de límites\n",
"for i in range(6):\n",
" await js.publish(\"demo.limits.eventos\", f\"evento-{i}\".encode())\n",
"\n",
"# Pull consumer DURABLE: recuerda su cursor entre fetches\n",
"psub = await js.pull_subscribe(\"demo.limits.>\", durable=\"procesador-A\")\n",
"\n",
"# Traer 4 y confirmarlos (ack)\n",
"batch = await psub.fetch(4, timeout=2)\n",
"print(\"Primer fetch (4 msgs):\")\n",
"for m in batch:\n",
" print(f\" seq={m.metadata.sequence.stream} {m.data.decode()}\")\n",
" await m.ack()\n",
"\n",
"# Estado del consumer: cuántos quedan pendientes / entregados\n",
"ci = await psub.consumer_info()\n",
"print()\n",
"print(f\"Consumer 'procesador-A':\")\n",
"print(f\" num_pending : {ci.num_pending} (mensajes sin entregar todavía)\")\n",
"print(f\" num_ack_pending: {ci.num_ack_pending} (entregados sin ack)\")\n",
"print(f\" delivered.stream_seq: {ci.delivered.stream_seq}\")\n",
"\n",
"# Segundo fetch: continúa donde se quedó (recuerda el cursor)\n",
"batch2 = await psub.fetch(10, timeout=1)\n",
"print()\n",
"print(f\"Segundo fetch: {len(batch2)} msgs restantes ->\", [m.data.decode() for m in batch2])\n",
"for m in batch2:\n",
" await m.ack()"
]
},
{
"cell_type": "markdown",
"id": "15b96e37",
"metadata": {},
"source": [
"### 3 · Deduplicación por `Nats-Msg-Id`\n",
"\n",
"Si un publisher reintenta por un timeout de red, podría enviar el mismo mensaje dos veces. JetStream lo evita: si dos publicaciones llevan el mismo header **`Nats-Msg-Id`** dentro de la `duplicate_window`, la segunda se reconoce como **duplicada** y **no** se almacena. El `PubAck` lo indica con `duplicate=True`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fdc4c498",
"metadata": {},
"outputs": [],
"source": [
"await js.add_stream(name=\"DEMO_DEDUP\", subjects=[\"demo.dedup.>\"],\n",
" storage=StorageType.FILE, duplicate_window=120)\n",
"\n",
"# Publicar dos veces el MISMO Nats-Msg-Id\n",
"ack1 = await js.publish(\"demo.dedup.pago\", b\"cobro 50e\", headers={\"Nats-Msg-Id\": \"pago-0001\"})\n",
"ack2 = await js.publish(\"demo.dedup.pago\", b\"cobro 50e\", headers={\"Nats-Msg-Id\": \"pago-0001\"})\n",
"\n",
"print(f\"1a publicacion: seq={ack1.seq} duplicate={ack1.duplicate}\")\n",
"print(f\"2a publicacion: seq={ack2.seq} duplicate={ack2.duplicate} <- detectada como duplicado\")\n",
"\n",
"# Un Msg-Id distinto sí se almacena\n",
"ack3 = await js.publish(\"demo.dedup.pago\", b\"cobro 30e\", headers={\"Nats-Msg-Id\": \"pago-0002\"})\n",
"print(f\"3a publicacion (id nuevo): seq={ack3.seq} duplicate={ack3.duplicate}\")\n",
"\n",
"st = (await js.stream_info(\"DEMO_DEDUP\")).state\n",
"print()\n",
"print(f\"Mensajes realmente almacenados en el stream: {st.messages} (2 publicaciones unicas, 1 descartada)\")"
]
},
{
"cell_type": "markdown",
"id": "da997d2e",
"metadata": {},
"source": [
"### 4 · Retención `workqueue`: la cola de trabajo\n",
"\n",
"Con `retention=workqueue`, cada mensaje se **borra del stream en cuanto un consumer lo confirma**. Es el patrón de cola de tareas distribuida: los mensajes se reparten entre workers y desaparecen al procesarse, así el stream no crece sin fin."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c7c4a35f",
"metadata": {},
"outputs": [],
"source": [
"from nats.js.api import RetentionPolicy, StorageType\n",
"\n",
"await js.add_stream(name=\"DEMO_WQ\", subjects=[\"demo.wq.>\"],\n",
" storage=StorageType.FILE, retention=RetentionPolicy.WORK_QUEUE)\n",
"\n",
"# Encolar 5 tareas\n",
"for i in range(5):\n",
" await js.publish(\"demo.wq.tareas\", f\"tarea-{i}\".encode())\n",
"\n",
"antes = (await js.stream_info(\"DEMO_WQ\")).state.messages\n",
"print(f\"Tareas encoladas en el stream: {antes}\")\n",
"\n",
"# Un worker consume y confirma 3\n",
"wsub = await js.pull_subscribe(\"demo.wq.>\", durable=\"worker\")\n",
"tres = await wsub.fetch(3, timeout=2)\n",
"for m in tres:\n",
" print(f\" procesada: {m.data.decode()}\")\n",
" await m.ack() # al confirmar, JetStream BORRA el mensaje del stream\n",
"\n",
"await asyncio.sleep(0.3)\n",
"despues = (await js.stream_info(\"DEMO_WQ\")).state.messages\n",
"print()\n",
"print(f\"Mensajes restantes en el stream tras 3 acks: {despues} (workqueue borra lo confirmado: {antes} -> {despues})\")"
]
},
{
"cell_type": "markdown",
"id": "5a68f26f",
"metadata": {},
"source": [
"### 5 · Políticas de entrega (replay)\n",
"\n",
"Al crear un consumer se elige **desde dónde** empieza a leer (`DeliverPolicy`):\n",
"\n",
"- `ALL` — todo el historial desde el principio (lo habitual para reprocesar).\n",
"- `LAST` — solo el último mensaje del stream.\n",
"- `NEW` — solo lo que llegue a partir de ahora.\n",
"- `BY_START_SEQUENCE` / `BY_START_TIME` — desde una secuencia o instante concretos.\n",
"\n",
"Comparamos `ALL` vs `LAST` sobre el stream de límites (que tiene 6 mensajes)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a341929e",
"metadata": {},
"outputs": [],
"source": [
"from nats.js.api import ConsumerConfig, DeliverPolicy\n",
"\n",
"# Consumer que reproduce TODO el historial\n",
"all_sub = await js.pull_subscribe(\n",
" \"demo.limits.>\", durable=\"replay-all\",\n",
" config=ConsumerConfig(deliver_policy=DeliverPolicy.ALL),\n",
")\n",
"todos = await all_sub.fetch(50, timeout=1)\n",
"print(f\"DeliverPolicy.ALL -> {len(todos)} mensajes:\", [m.data.decode() for m in todos])\n",
"for m in todos:\n",
" await m.ack()\n",
"\n",
"# Consumer que solo entrega el ÚLTIMO\n",
"last_sub = await js.pull_subscribe(\n",
" \"demo.limits.>\", durable=\"replay-last\",\n",
" config=ConsumerConfig(deliver_policy=DeliverPolicy.LAST),\n",
")\n",
"ultimo = await last_sub.fetch(50, timeout=1)\n",
"print(f\"DeliverPolicy.LAST -> {len(ultimo)} mensaje :\", [m.data.decode() for m in ultimo])\n",
"for m in ultimo:\n",
" await m.ack()"
]
},
{
"cell_type": "markdown",
"id": "0fba1dc2",
"metadata": {},
"source": [
"## Parte B · Simulador de rendimiento (interactivo)\n",
"\n",
"Pulsa **▶ Ejecutar benchmark** y verás cómo **un publisher** inunda el broker con miles de mensajes que **varios subscribers** reciben simultáneamente (fan-out). La gráfica se actualiza **en movimiento** mientras corre:\n",
"\n",
"- **Izquierda** — mensajes acumulados: enviados (publisher) vs recibidos (suma de todos los subs).\n",
"- **Derecha** — throughput instantáneo (msgs/s recibidos) muestreado cada ~80 ms.\n",
"\n",
"Ajusta los sliders para cambiar el número de mensajes y de subscribers. Con más mensajes (p. ej. 100.000) la animación dura más y se aprecia mejor la curva."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4240cd89",
"metadata": {},
"outputs": [],
"source": [
"import ipywidgets as widgets\n",
"from IPython.display import display, clear_output\n",
"import matplotlib.pyplot as plt\n",
"import asyncio, time\n",
"import nats\n",
"\n",
"# --- widgets ---\n",
"n_msgs_w = widgets.IntSlider(value=20000, min=1000, max=100000, step=1000,\n",
" description=\"Mensajes:\", style={\"description_width\": \"initial\"},\n",
" layout=widgets.Layout(width=\"380px\"))\n",
"n_subs_w = widgets.IntSlider(value=3, min=1, max=8, step=1,\n",
" description=\"Subscribers:\", style={\"description_width\": \"initial\"})\n",
"run_btn = widgets.Button(description=\"▶ Ejecutar benchmark\", button_style=\"success\",\n",
" layout=widgets.Layout(width=\"220px\"))\n",
"plot_out = widgets.Output()\n",
"log_out = widgets.Output()\n",
"\n",
"SUBJECT = \"bench.load\"\n",
"PAYLOAD = b\"x\" * 128 # 128 bytes por mensaje\n",
"\n",
"def _throughput(ts, recv):\n",
" thr = [0.0]\n",
" for i in range(1, len(ts)):\n",
" dt = ts[i] - ts[i-1]\n",
" thr.append((recv[i] - recv[i-1]) / dt if dt > 0 else 0.0)\n",
" return thr\n",
"\n",
"def render(history, n_subs, n_msgs, done=False):\n",
" ts = [h[0] for h in history]\n",
" sent = [h[1] for h in history]\n",
" recv = [h[2] for h in history]\n",
" thr = _throughput(ts, recv)\n",
" with plot_out:\n",
" clear_output(wait=True)\n",
" fig, (a1, a2) = plt.subplots(1, 2, figsize=(11, 3.6))\n",
" a1.plot(ts, sent, label=\"enviados (pub)\", color=\"#2563eb\", lw=2)\n",
" a1.plot(ts, recv, label=f\"recibidos (Σ {n_subs} subs)\", color=\"#16a34a\", lw=2)\n",
" a1.set_xlabel(\"segundos\"); a1.set_ylabel(\"mensajes acumulados\")\n",
" a1.set_title(\"Publisher vs subscribers\"); a1.legend(loc=\"upper left\")\n",
" a2.plot(ts, thr, color=\"#db2777\", lw=2)\n",
" a2.set_xlabel(\"segundos\"); a2.set_ylabel(\"msgs/s recibidos\")\n",
" a2.set_title(\"Throughput instantáneo\")\n",
" estado = \"✓ DONE\" if done else \"● corriendo…\"\n",
" fig.suptitle(f\"[{estado}] {n_msgs:,} msgs → {n_subs} subs \"\n",
" f\"enviados={sent[-1]:,} recibidos={recv[-1]:,}\", fontsize=11)\n",
" plt.tight_layout(); plt.show()\n",
"\n",
"async def run_benchmark(n_msgs, n_subs, live=True):\n",
" \"\"\"1 publisher -> n_subs subscribers. Devuelve (history, counters).\"\"\"\n",
" nc = await nats.connect(NATS_URL, name=\"benchmark\")\n",
" counters = [0] * n_subs\n",
"\n",
" def make_cb(i):\n",
" async def cb(msg):\n",
" counters[i] += 1\n",
" return cb\n",
"\n",
" subs = [await nc.subscribe(SUBJECT, cb=make_cb(i)) for i in range(n_subs)]\n",
" history = [] # (t, enviados, recibidos_total)\n",
" sent = 0\n",
" t0 = time.monotonic()\n",
"\n",
" async def publish_all():\n",
" nonlocal sent\n",
" for k in range(n_msgs):\n",
" await nc.publish(SUBJECT, PAYLOAD)\n",
" sent += 1\n",
" if k % 1000 == 0:\n",
" await nc.flush()\n",
" await asyncio.sleep(0) # ceder al event loop (deja correr callbacks)\n",
" await nc.flush()\n",
"\n",
" task = asyncio.create_task(publish_all())\n",
"\n",
" # Muestreo para la gráfica en movimiento\n",
" while not task.done() or sum(counters) < sent:\n",
" await asyncio.sleep(0.08)\n",
" history.append((time.monotonic() - t0, sent, sum(counters)))\n",
" if live:\n",
" render(history, n_subs, n_msgs)\n",
" if time.monotonic() - t0 > 30: # tope de seguridad\n",
" break\n",
" await task\n",
"\n",
" # Drenaje final (que los callbacks alcancen al publisher)\n",
" for _ in range(40):\n",
" if sum(counters) >= sent:\n",
" break\n",
" await asyncio.sleep(0.05)\n",
" history.append((time.monotonic() - t0, sent, sum(counters)))\n",
" if live:\n",
" render(history, n_subs, n_msgs, done=True)\n",
"\n",
" for s in subs:\n",
" await s.unsubscribe()\n",
" await nc.drain()\n",
" return history, counters\n",
"\n",
"def on_click(_):\n",
" run_btn.disabled = True\n",
" with log_out:\n",
" clear_output()\n",
" print(f\"Lanzando: {n_msgs_w.value:,} mensajes → {n_subs_w.value} subscribers …\")\n",
" async def go():\n",
" try:\n",
" history, counters = await run_benchmark(n_msgs_w.value, n_subs_w.value, live=True)\n",
" dur = history[-1][0]\n",
" recv = sum(counters)\n",
" with log_out:\n",
" print(f\"OK en {dur:.2f}s\")\n",
" print(f\" enviados : {n_msgs_w.value:,}\")\n",
" print(f\" recibidos: {recv:,} (fan-out ×{n_subs_w.value} = {recv/max(n_msgs_w.value,1):.2f} por mensaje)\")\n",
" print(f\" throughput pub : {n_msgs_w.value/dur:,.0f} msgs/s\")\n",
" print(f\" throughput recv: {recv/dur:,.0f} msgs/s (entregas totales)\")\n",
" print(f\" por subscriber : {counters}\")\n",
" finally:\n",
" run_btn.disabled = False\n",
" asyncio.ensure_future(go())\n",
"\n",
"run_btn.on_click(on_click)\n",
"display(widgets.HBox([n_msgs_w, n_subs_w]), run_btn, plot_out, log_out)\n",
"print(\"Simulador listo. Pulsa el botón para lanzar el benchmark.\")"
]
},
{
"cell_type": "markdown",
"id": "de3caaa5",
"metadata": {},
"source": [
"### Verificación (headless)\n",
"\n",
"La celda anterior renderiza el widget para pulsarlo en JupyterLab. Aquí ejecutamos el mismo benchmark **una vez de forma programática** (sin botón) para dejar evidencia ejecutada: una corrida real de 15.000 mensajes a 3 subscribers con su gráfica final."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "34f06087",
"metadata": {},
"outputs": [],
"source": [
"hist, counters = await run_benchmark(15000, 3, live=False)\n",
"\n",
"ts = [h[0] for h in hist]\n",
"sent = [h[1] for h in hist]\n",
"recv = [h[2] for h in hist]\n",
"dur = ts[-1]\n",
"\n",
"fig, ax = plt.subplots(figsize=(9, 3.6))\n",
"ax.plot(ts, sent, label=\"enviados (pub)\", color=\"#2563eb\", lw=2)\n",
"ax.plot(ts, recv, label=\"recibidos (Σ 3 subs)\", color=\"#16a34a\", lw=2)\n",
"ax.set_xlabel(\"segundos\"); ax.set_ylabel(\"mensajes acumulados\")\n",
"ax.set_title(f\"Benchmark headless: 15.000 msgs → 3 subs en {dur:.2f}s\")\n",
"ax.legend(loc=\"upper left\")\n",
"plt.tight_layout(); plt.show()\n",
"\n",
"print(f\"enviados : 15,000\")\n",
"print(f\"recibidos: {sum(counters):,} por sub -> {counters}\")\n",
"print(f\"throughput pub : {15000/dur:,.0f} msgs/s\")\n",
"print(f\"throughput recv: {sum(counters)/dur:,.0f} msgs/s (entregas totales, fan-out x3)\")"
]
},
{
"cell_type": "markdown",
"id": "501f5185",
"metadata": {},
"source": [
"## Resumen\n",
"\n",
"**JetStream a fondo:**\n",
"- Un **stream** persiste mensajes con políticas de **storage** (file/memory), **retention** (limits/interest/workqueue) y **límites** (max_msgs/max_age).\n",
"- Los **consumers** (pull/push, durable/ephemeral) leen a su ritmo y **confirman** (ack) cada mensaje → entrega *at-least-once*.\n",
"- **Dedup** por `Nats-Msg-Id` evita duplicados por reintentos.\n",
"- **workqueue** borra cada mensaje al confirmarse → cola de trabajo.\n",
"- **DeliverPolicy** controla el replay (all/last/new/by_sequence/by_time).\n",
"\n",
"**Simulador:** demuestra el fan-out a escala — un publisher alimenta a N subscribers con miles de mensajes y la gráfica en vivo muestra que el throughput de recepción sigue al de envío (cada mensaje se entrega a los N subscribers).\n",
"\n",
"### Limpieza\n",
"\n",
"```python\n",
"for s in (\"DEMO_LIMITS\", \"DEMO_DEDUP\", \"DEMO_WQ\"):\n",
" try: await js.delete_stream(s)\n",
" except Exception: pass\n",
"await nc.drain()\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
+12 -3
View File
@@ -427,12 +427,21 @@
], ],
"metadata": { "metadata": {
"kernelspec": { "kernelspec": {
"display_name": "Python 3 (ipykernel)", "display_name": "NATS analysis",
"language": "python", "language": "python",
"name": "python3" "name": "nats"
}, },
"language_info": { "language_info": {
"name": "" "codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
+26 -17
View File
@@ -1,21 +1,5 @@
{ {
"cells": [ "cells": [
{
"cell_type": "markdown",
"id": "a5e95055",
"metadata": {},
"source": [
"# NATS pub/sub — 02 · Queue groups, Request/Reply y JetStream\n",
"\n",
"En el notebook 01 vimos el *fan-out* del core: una publicación llega a **todos** los subscribers. Aquí cubrimos tres patrones que construyen sobre esa base:\n",
"\n",
"1. **Queue groups** — repartir la carga: varios *workers* comparten el trabajo y cada mensaje lo procesa **uno solo**.\n",
"2. **Request/Reply** — RPC sobre mensajería: un cliente pregunta y espera la respuesta de un servicio.\n",
"3. **JetStream** — la capa de **persistencia**: streams que almacenan los mensajes y permiten *replay*, a diferencia del core *fire-and-forget*.\n",
"\n",
"> Requiere el broker `nats_demo` del notebook 01. La primera celda lo arranca de forma idempotente, así que este notebook también funciona de forma aislada."
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "1ab4e0d1", "id": "1ab4e0d1",
@@ -374,6 +358,22 @@
"\n", "\n",
"**Siguiente** → `03_procesos_reales.ipynb`: hasta ahora todo ha ocurrido dentro de un mismo kernel. Allí lanzamos publisher y subscribers como **procesos del sistema operativo independientes** para ver el desacople real." "**Siguiente** → `03_procesos_reales.ipynb`: hasta ahora todo ha ocurrido dentro de un mismo kernel. Allí lanzamos publisher y subscribers como **procesos del sistema operativo independientes** para ver el desacople real."
] ]
},
{
"cell_type": "markdown",
"id": "a5e95055",
"metadata": {},
"source": [
"# NATS pub/sub — 02 · Queue groups, Request/Reply y JetStream\n",
"\n",
"En el notebook 01 vimos el *fan-out* del core: una publicación llega a **todos** los subscribers. Aquí cubrimos tres patrones que construyen sobre esa base:\n",
"\n",
"1. **Queue groups** — repartir la carga: varios *workers* comparten el trabajo y cada mensaje lo procesa **uno solo**.\n",
"2. **Request/Reply** — RPC sobre mensajería: un cliente pregunta y espera la respuesta de un servicio.\n",
"3. **JetStream** — la capa de **persistencia**: streams que almacenan los mensajes y permiten *replay*, a diferencia del core *fire-and-forget*.\n",
"\n",
"> Requiere el broker `nats_demo` del notebook 01. La primera celda lo arranca de forma idempotente, así que este notebook también funciona de forma aislada."
]
} }
], ],
"metadata": { "metadata": {
@@ -383,7 +383,16 @@
"name": "python3" "name": "python3"
}, },
"language_info": { "language_info": {
"name": "" "codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
+10 -1
View File
@@ -510,7 +510,16 @@
"name": "python3" "name": "python3"
}, },
"language_info": { "language_info": {
"name": "" "codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
File diff suppressed because one or more lines are too long