{ "cells": [ { "cell_type": "markdown", "id": "05860b9f", "metadata": {}, "source": [ "# NATS pub/sub — 03 · Procesos del sistema operativo reales\n", "\n", "En los notebooks 01 y 02 todo ocurrió dentro de un mismo kernel: varias conexiones `asyncio` simulaban procesos distintos. Eso es cómodo para explicar, pero NATS brilla precisamente cuando los participantes son **procesos del sistema operativo separados** —incluso en máquinas distintas— que solo comparten la dirección del broker y los nombres de subject.\n", "\n", "Aquí lanzamos **procesos reales** con `subprocess`:\n", "\n", "- un **publisher** (`procs/publisher.py`) que emite telemetría a `telemetria.cpu` y `telemetria.mem`;\n", "- dos **subscribers** independientes (`procs/subscriber.py`), cada uno con su propio PID:\n", " - `sub-todo` escucha `telemetria.>` (toda la telemetría),\n", " - `sub-cpu` escucha solo `telemetria.cpu`.\n", "\n", "Cada proceso abre su propia conexión al broker. El publisher **no sabe** cuántos subscribers hay ni qué escuchan: solo publica a un subject. Ese es el desacople real." ] }, { "cell_type": "markdown", "id": "c5127085", "metadata": {}, "source": [ "## 0 · Broker + scripts de los procesos\n", "\n", "Arrancamos el broker (idempotente) y mostramos el código de los dos scripts que vamos a lanzar como procesos. Cada uno es un programa autónomo que se conecta a `nats://127.0.0.1:4222` y emite eventos como líneas JSON en su stdout, que el notebook recogerá." ] }, { "cell_type": "code", "execution_count": 1, "id": "bb720c29", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Broker: already-running\n", "Scripts de proceso en: /home/enmanuel/fn_registry/analysis/nats/notebooks/procs\n", "\n", "=== procs/publisher.py ===\n", "#!/usr/bin/env python3\n", "\"\"\"Publisher NATS como proceso del sistema operativo independiente.\n", "\n", "Se conecta al broker y publica una rafaga de mensajes de telemetria,\n", "alternando entre los subjects `telemetria.cpu` y `telemetria.mem`.\n", "No sabe ni le importa cuantos subscribers hay escuchando: solo conoce el\n", "subject. Emite cada publicacion como linea JSON en stdout.\n", "\"\"\"\n", "import argparse\n", "import asyncio\n", "import json\n", "import os\n", "import random\n", "import time\n", "\n", "import nats\n", "\n", "NATS_URL = \"nats://127.0.0.1:4222\"\n", "\n", "\n", "def emit(event: dict) -> None:\n", " print(json.dumps(event), flush=True)\n", "\n", "\n", "async def main(count: int, interval: float) -> None:\n", " pid = os.getpid()\n", " nc = await nats.connect(NATS_URL, name=\"publisher\")\n", " emit({\"event\": \"ready\", \"pid\": pid, \"name\": \"publisher\"})\n", "\n", " for i in range(count):\n", " subject = \"telemetria.cpu\" if i % 2 == 0 else \"telemetria.mem\"\n", " payload = json.dumps({\"i\": i, \"valor\": round(random.uniform(0, 100), 1)})\n", " await nc.publish(subject, payload.encode())\n", " emit({\"event\": \"published\", \"pid\": pid, \"subject\": subject, \"i\": i})\n", " await asyncio.sleep(interval)\n", "\n", " await nc.flush()\n", " emit({\"event\": \"done\", \"pid\": pid, \"name\": \"publisher\", \"published\": count})\n", " await nc.drain()\n", "\n", "\n", "if __name__ == \"__main__\":\n", " parser = argparse.ArgumentParser(description=\"Publisher NATS de demostracion\")\n", " parser.add_argument(\"--count\", type=int, default=8, help=\"Numero de mensajes a publicar\")\n", " parser.add_argument(\"--interval\", type=float, default=0.15,\n", " help=\"Segundos entre publicaciones\")\n", " args = parser.parse_args()\n", " asyncio.run(main(args.count, args.interval))\n", "\n" ] } ], "source": [ "import subprocess, time, json\n", "\n", "NATS_CONTAINER = \"nats_demo\"\n", "NATS_PORT = 4222\n", "NATS_URL = f\"nats://127.0.0.1:{NATS_PORT}\"\n", "\n", "def _docker(*args, check=True):\n", " return subprocess.run([\"docker\", *args], capture_output=True, text=True, check=check)\n", "\n", "def ensure_nats(name=NATS_CONTAINER, port=NATS_PORT):\n", " \"\"\"Arranca un broker NATS en Docker de forma idempotente. Devuelve el estado.\"\"\"\n", " out = _docker(\"ps\", \"-a\", \"--filter\", f\"name=^{name}$\", \"--format\", \"{{.State}}\", check=False).stdout.strip()\n", " if out == \"running\":\n", " state = \"already-running\"\n", " elif out in (\"exited\", \"created\", \"paused\"):\n", " _docker(\"start\", name)\n", " state = \"started\"\n", " else:\n", " _docker(\"run\", \"-d\", \"--name\", name, \"-p\", f\"{port}:4222\", \"-p\", \"8222:8222\",\n", " \"nats:latest\", \"-js\", \"-m\", \"8222\")\n", " state = \"created\"\n", " time.sleep(1.0)\n", " return state\n", "\n", "from pathlib import Path\n", "\n", "PROCS = Path(r\"/home/enmanuel/fn_registry/analysis/nats/notebooks/procs\")\n", "print(\"Broker:\", ensure_nats())\n", "print(\"Scripts de proceso en:\", PROCS)\n", "print()\n", "print(\"=== procs/publisher.py ===\")\n", "print(Path(PROCS / \"publisher.py\").read_text())" ] }, { "cell_type": "code", "execution_count": 2, "id": "3412b705", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "=== procs/subscriber.py ===\n", "#!/usr/bin/env python3\n", "\"\"\"Subscriber NATS como proceso del sistema operativo independiente.\n", "\n", "Se conecta al broker, se suscribe a uno o varios subjects y emite cada evento\n", "como una linea JSON en stdout para que el proceso padre (el notebook) la lea.\n", "Termina solo tras `--seconds` segundos.\n", "\"\"\"\n", "import argparse\n", "import asyncio\n", "import json\n", "import os\n", "import time\n", "\n", "import nats\n", "\n", "NATS_URL = \"nats://127.0.0.1:4222\"\n", "\n", "\n", "def emit(event: dict) -> None:\n", " \"\"\"Escribe un evento como linea JSON en stdout, con flush inmediato.\"\"\"\n", " print(json.dumps(event), flush=True)\n", "\n", "\n", "async def main(name: str, subjects: list[str], seconds: float) -> None:\n", " pid = os.getpid()\n", " nc = await nats.connect(NATS_URL, name=name)\n", " received = 0\n", " t0 = time.monotonic()\n", "\n", " async def handler(msg):\n", " nonlocal received\n", " received += 1\n", " emit({\n", " \"event\": \"msg\",\n", " \"pid\": pid,\n", " \"name\": name,\n", " \"subject\": msg.subject,\n", " \"data\": msg.data.decode(),\n", " \"t\": round(time.monotonic() - t0, 4),\n", " })\n", "\n", " for subject in subjects:\n", " await nc.subscribe(subject, cb=handler)\n", "\n", " # Senal de que este proceso ya esta escuchando (el padre la espera).\n", " emit({\"event\": \"ready\", \"pid\": pid, \"name\": name, \"subjects\": subjects})\n", "\n", " await asyncio.sleep(seconds)\n", " emit({\"event\": \"done\", \"pid\": pid, \"name\": name, \"received\": received})\n", " await nc.drain()\n", "\n", "\n", "if __name__ == \"__main__\":\n", " parser = argparse.ArgumentParser(description=\"Subscriber NATS de demostracion\")\n", " parser.add_argument(\"--name\", required=True, help=\"Nombre logico del subscriber\")\n", " parser.add_argument(\"--subjects\", required=True,\n", " help=\"Subjects separados por coma (admite wildcards)\")\n", " parser.add_argument(\"--seconds\", type=float, default=4.0,\n", " help=\"Tiempo de escucha antes de terminar\")\n", " args = parser.parse_args()\n", " asyncio.run(main(args.name, args.subjects.split(\",\"), args.seconds))\n", "\n" ] } ], "source": [ "print(\"=== procs/subscriber.py ===\")\n", "print((PROCS / \"subscriber.py\").read_text())" ] }, { "cell_type": "markdown", "id": "e17dd705", "metadata": {}, "source": [ "## 1 · Lanzar los procesos y orquestarlos\n", "\n", "El notebook actúa de **orquestador**:\n", "\n", "1. Lanza los dos subscribers como procesos (`subprocess.Popen`), cada uno con su PID. Les damos 1.5 s para que conecten y se suscriban.\n", "2. Lanza el publisher, que emite 8 mensajes y termina.\n", "3. Espera a que los subscribers terminen solos (su `--seconds`) y recoge su stdout.\n", "\n", "Usamos `sys.executable` para que los procesos hijos usen el mismo intérprete (con `nats-py` instalado) que el kernel." ] }, { "cell_type": "code", "execution_count": 3, "id": "f647029e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Subscribers lanzados (PIDs del SO): {'sub-todo': 4191523, 'sub-cpu': 4191524}\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Publisher (PID 4191735) publicó 8 mensajes\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Total de entregas recibidas entre todos los procesos: 12\n" ] } ], "source": [ "import subprocess, sys, json, time\n", "\n", "def lanzar_subscriber(nombre, subjects, seconds=4.5):\n", " return subprocess.Popen(\n", " [sys.executable, str(PROCS / \"subscriber.py\"),\n", " \"--name\", nombre, \"--subjects\", subjects, \"--seconds\", str(seconds)],\n", " stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,\n", " )\n", "\n", "# 1. Subscribers: procesos OS independientes\n", "procs = {\n", " \"sub-todo\": lanzar_subscriber(\"sub-todo\", \"telemetria.>\"),\n", " \"sub-cpu\": lanzar_subscriber(\"sub-cpu\", \"telemetria.cpu\"),\n", "}\n", "print(\"Subscribers lanzados (PIDs del SO):\", {n: p.pid for n, p in procs.items()})\n", "time.sleep(1.5) # que conecten y se suscriban antes de publicar\n", "\n", "# 2. Publisher: otro proceso OS, publica 8 mensajes y termina\n", "pub = subprocess.run(\n", " [sys.executable, str(PROCS / \"publisher.py\"), \"--count\", \"8\", \"--interval\", \"0.15\"],\n", " stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,\n", ")\n", "pub_eventos = [json.loads(l) for l in pub.stdout.splitlines() if l.strip()]\n", "print(f\"Publisher (PID {pub_eventos[0]['pid']}) publicó {sum(1 for e in pub_eventos if e['event']=='published')} mensajes\")\n", "\n", "# 3. Recoger stdout de los subscribers (terminan solos por --seconds)\n", "eventos = []\n", "for nombre, p in procs.items():\n", " out, err = p.communicate(timeout=10)\n", " for l in out.splitlines():\n", " if l.strip():\n", " eventos.append(json.loads(l))\n", " if err.strip():\n", " print(f\"[{nombre} stderr] {err.strip()[:200]}\")\n", "\n", "msgs = [e for e in eventos if e[\"event\"] == \"msg\"]\n", "print(f\"\\nTotal de entregas recibidas entre todos los procesos: {len(msgs)}\")" ] }, { "cell_type": "markdown", "id": "33dcf1f4", "metadata": {}, "source": [ "## 2 · Qué recibió cada proceso\n", "\n", "Cada subscriber es un PID distinto. `sub-todo` (suscrito a `telemetria.>`) recibe los 8 mensajes; `sub-cpu` (suscrito solo a `telemetria.cpu`) recibe únicamente los 4 de CPU. El broker filtró por subject sin que el publisher supiera nada de ello." ] }, { "cell_type": "code", "execution_count": 4, "id": "01ae57ed", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PID de cada proceso subscriber: {'sub-todo': 4191523, 'sub-cpu': 4191524}\n", "\n", "Mensajes recibidos por proceso y subject:\n", "subject telemetria.cpu telemetria.mem\n", "name \n", "sub-cpu 4 0\n", "sub-todo 4 4\n" ] }, { "data": { "text/html": [ "
| subject | \n", "telemetria.cpu | \n", "telemetria.mem | \n", "
|---|---|---|
| name | \n", "\n", " | \n", " |
| sub-cpu | \n", "4 | \n", "0 | \n", "
| sub-todo | \n", "4 | \n", "4 | \n", "