dataforge/web_scraping
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: auto-commit (3 archivos)

Browse Source 
- .mcp.json
- CAPABILITIES_TODO.md
- demo_e2e/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Egutierrez
2026-06-06 12:49:54 +02:00
parent 2527fd306a
commit 23f9aa90e8
14 changed files with 1858 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
demo_e2e/RESUMEN.md
+128
View File
@@ -0,0 +1,128 @@
# Ejercicio e2e — validación de capacidades del browser_mcp
Fecha: 06/06/2026. Objetivo: comprobar, ejecutando de verdad contra sitios reales, que el servidor
`browser_mcp` (control de navegador vía CDP) puede hacer tareas distintas de recopilación de datos —
de simples a complejas. No "compila", sino "funciona".
## Montaje
- **Servidor**: `projects/web_scraping/apps/browser_mcp/browser_mcp` (36 tools, pool de conexiones).
- **Navegador**: Chrome/Chromium 148 aislado en el puerto CDP **9333** (no el 9222 del navegador diario),
`user-data-dir` dedicado. Lanzado headless para la batería y luego **sin headless** (ventana visible)
para inspección humana — ambos con 5/5.
- **Cliente**: `mcp_client.py` — cliente JSON-RPC stdio **secuencial** (espera la respuesta de cada tool
antes de mandar la siguiente, como hace un cliente MCP real). Un primer intento mandando todos los
mensajes de golpe falló por una race: el servidor procesa requests de forma concurrente.
- **Runner**: `run_demo.py` — ejecuta las 5 pruebas, guarda pasos/respuestas/veredicto en `results/`.
## Resultado: 8/8 PASS (headless y con ventana visible)
Las pruebas 1-5 son la batería inicial; 6-8 se añadieron para validar las tandas de fixes (A/D/E y B).
| # | Prueba | Sitio sandbox | Capacidades ejercitadas | Resultado |
|---|---|---|---|---|
| 1 | Extraer citas estructuradas | quotes.toscrape.com | navigate, wait_load, **eval → JSON real** | PASS — 10 citas `{text,author,tags}` |
| 2 | Percibir página como agente | the-internet.herokuapp.com | **page_perceive** (AX outline) | PASS — outline 4021 chars con `#ref` accionables |
| 3 | Submit de formulario con teclado | the-internet.herokuapp.com/login | dom_click, dom_type, **press_key Enter**, wait_element | PASS — "You logged into a secure area!" |
| 4 | Login + sesión persistente | the-internet.herokuapp.com | form, **storage_save/load**, cookie_clear | PASS — sesión restaurada sin re-login |
| 5 | Scraping paginado + dedup | books.toscrape.com | navegación multi-página, eval, composición | PASS — 60 libros únicos (3 páginas) |
| 6 | sessionStorage en storage_state | the-internet.herokuapp.com | set→save→clear→load→get | PASS — `clear=null`, `restore=demo_v` (fix D) |
| 7 | `find_by_text` honesto | quotes.toscrape.com | dom_find_by_text presente vs inexistente | PASS — texto inexistente devuelve error, no vacío (fix E) |
| 8 | Verificación post-acción | quotes.toscrape.com | dom_click sobre oculto / dom_type sin foco | PASS — ambos devuelven error en vez de actuar al vacío (fix B) |
Cobertura conjunta: lanzar/atar navegador, navegar, esperar carga, evaluar JS con JSON real, percibir
(AX outline), leer texto, teclado, formularios, cookies, estado de sesión, y composición multi-página.
## Valor del ejercicio: 3 bugs reales encontrados y arreglados
Ejecutar de verdad reveló defectos que "compila" jamás habría detectado:
1. **`page_perceive` roto** (bug de integración del MCP). Invocaba `fn run cdp_perceive_outline
--debug-port 9333 ...` con flags, pero `fn run` pasa los argumentos **posicionalmente** a la función
del pipeline → `int('--debug-port')` reventaba. Toda la percepción AX caía. Fix: argumentos
posicionales en el orden de la firma. (`tools_read.go`)
2. **`cdp_save_storage_state` guardaba cookies de todos los dominios.** Usaba `Network.getAllCookies`
(global), así que el `storage_state` arrastraba cookies de sitios visitados antes en la misma sesión
(en la prueba, cookies de Wikipedia contaminaban la sesión de the-internet). Fix: filtrar por el host
actual (`location.hostname`). (`cdp_save_storage_state.go`)
3. **`cdp_load_storage_state` no restauraba la sesión httpOnly.** `Network.setCookies` no aplicaba de
forma fiable la cookie de sesión (`rack.session`, httpOnly) porque a cada cookie le faltaba el campo
`url`. Fix: sintetizar `url` por cookie a partir de `domain`/`secure`/`path`. Con esto el login
persistente (la pieza estrella) funciona de verdad. (`cdp_load_storage_state.go`)
## Segunda tanda de fixes (A + D + E) — a partir del análisis de deuda
Tras la primera batería se atacaron tres deudas, cada una validada:
- **A — Aislamiento robusto del navegador del agente** (`chrome_launch.go`). El wrapper del sistema
`/etc/chromium.d/cdp` inyecta `--user-data-dir`/`--remote-debugging-port` globales a todo chromium;
el aislamiento dependía de que nuestros flags fueran al final (Chrome usa el último duplicado). Fix:
`findChrome` prefiere el **binario real** (`/usr/lib/chromium/chromium`), que al ejecutarse directo no
pasa por el wrapper y por tanto no hereda esos flags. Validado por construcción (el binario existe y va
primero; `browser_launch` devuelve PID correcto); la inspección observable del cmdline choca con el
exit-144 del entorno de pruebas al lanzar chromium desde el harness, no con el fix.
- **D — `sessionStorage` en `storage_state`** (`cdp_save_storage_state.go`, `cdp_load_storage_state.go`).
Antes solo cookies + localStorage. Ahora también el "cajón temporal". Validado por la prueba 6.
- **E — `cdp_find_by_text` honesto** (`cdp_find_by_text.go`). Antes devolvía `("", nil)` cuando no
encontraba (el caller creía que había encontrado algo). Ahora devuelve error explícito. Validado por la
prueba 7.
## Tercera tanda de fix (B) — verificación post-acción
- **B — fin del "fire-and-forget"** (`cdp_click.go`, `cdp_type_text.go`). `cdp_click` ahora verifica que
el elemento es **visible** antes de clicar (display:none / tamaño 0 / opacity 0 → error, en vez de clicar
en (0,0) sin efecto). `cdp_type_text` verifica que hay un **campo editable enfocado** (input/textarea/
select/contenteditable) antes de escribir (sin foco → error claro "usa CdpClick primero", en vez de
escribir a la nada). Validado por la prueba 8. Pendiente de esta familia: `cdp_scroll` con target
explícito (P1.5) y el puente percepción→acción por nodeId (P1.3).
## Hallazgos secundarios
- **`press_key Enter` no dispara widgets JS complejos.** Contra el buscador de Wikipedia (typeahead Vue
del skin Vector) el keyevent sintético se ejecutó sin error pero el widget no reaccionó. La prueba 3 se
reorientó a un formulario HTML normal (login de the-internet), donde Enter sí envía el form. Deuda: para
widgets JS-driven puede hacer falta disparar el evento del framework o submit explícito.
- **Las pruebas comparten un mismo Chrome**, por lo que el estado (cookies) se acumula entre ellas. Se
añadió `cookie_clear` al inicio de las pruebas con login para aislarlas. Un harness más estricto usaría
un contexto/perfil por prueba.
## Deuda pendiente (no bloqueó el ejercicio)
- `storage_state` aún no captura `sessionStorage` (sí cookies + localStorage). Suficiente para sesiones
basadas en cookie como the-internet; insuficiente para sitios que guardan el token en sessionStorage.
- Verificación post-acción (P1 del análisis LLM-readiness): `dom_click`/`dom_type` siguen siendo
fire-and-forget. En estas pruebas se compensó con `dom_wait_element` y checks por `eval`, pero las tools
no confirman su efecto por sí mismas.
## Cómo reproducir
```bash
# 1. Chrome aislado en 9333 (headless)
systemd-run --user -q --unit=browser_demo \
chromium --headless=new --remote-debugging-port=9333 \
--user-data-dir=/tmp/browser_mcp_userdata about:blank
# 1-bis. O con ventana visible (Linux con sesión gráfica):
systemd-run --user -q --unit=browser_demo \
--setenv=DISPLAY=:0 --setenv=XAUTHORITY=$HOME/.Xauthority \
chromium --remote-debugging-port=9333 \
--user-data-dir=/tmp/browser_mcp_visible --start-maximized about:blank
# 2. Compilar el MCP y ejecutar la batería
cd projects/web_scraping/apps/browser_mcp && go build -o browser_mcp .
cd ../../demo_e2e && python3 run_demo.py
# 3. Parar el navegador
systemctl --user stop browser_demo.service
```
## Archivos
- `mcp_client.py` — cliente MCP stdio secuencial (reutilizable).
- `run_demo.py` — las 5 pruebas.
- `results/prueba_N_*.json` — pasos, respuestas y datos extraídos por prueba.
- `results/run.log` — log de la corrida.
- `results/summary.json` — veredicto agregado.
- `results/mcp_stderr.log` — stderr del servidor MCP.
demo_e2e/mcp_client.py
+108
View File
@@ -0,0 +1,108 @@
"""Cliente JSON-RPC stdio mínimo para hablar con un servidor MCP (mark3labs/mcp-go).
Secuencial por diseño: cada call() manda un request y bloquea hasta recibir la
respuesta con el mismo id, ignorando notificaciones intermedias. Esto replica
cómo un cliente MCP real (Claude) usa el servidor — a diferencia de mandar todos
los mensajes de golpe, que provoca una race porque el servidor procesa los
requests en goroutines concurrentes.
Un hilo lector vuelca stdout a una cola; _read_until consume de la cola con
timeout para no colgarse si una tool no responde.
"""
import json
import os
import queue
import subprocess
import threading
import time
class MCPClient:
def __init__(self, exe, env=None, cwd=None, stderr_path=None):
full_env = dict(os.environ)
if env:
full_env.update(env)
self._stderr = open(stderr_path, "w") if stderr_path else subprocess.DEVNULL
self.p = subprocess.Popen(
[exe],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=self._stderr,
text=True,
bufsize=1,
env=full_env,
cwd=cwd,
)
self._id = 0
self._q = queue.Queue()
self._reader = threading.Thread(target=self._read_loop, daemon=True)
self._reader.start()
def _read_loop(self):
for line in self.p.stdout:
line = line.strip()
if line:
self._q.put(line)
def _send(self, obj):
self.p.stdin.write(json.dumps(obj) + "\n")
self.p.stdin.flush()
def _read_until(self, want_id, timeout):
deadline = time.time() + timeout
while time.time() < deadline:
try:
line = self._q.get(timeout=max(0.1, deadline - time.time()))
except queue.Empty:
break
try:
m = json.loads(line)
except json.JSONDecodeError:
continue
if m.get("id") == want_id:
return m
raise TimeoutError(f"sin respuesta para id {want_id} en {timeout}s")
def initialize(self):
self._id += 1
iid = self._id
self._send({
"jsonrpc": "2.0", "id": iid, "method": "initialize",
"params": {
"protocolVersion": "2024-11-05",
"capabilities": {},
"clientInfo": {"name": "demo_e2e", "version": "1"},
},
})
r = self._read_until(iid, 15)
self._send({"jsonrpc": "2.0", "method": "notifications/initialized"})
return r
def call(self, name, arguments, timeout=60):
"""Llama una tool. Devuelve (texto, is_error)."""
self._id += 1
cid = self._id
self._send({
"jsonrpc": "2.0", "id": cid, "method": "tools/call",
"params": {"name": name, "arguments": arguments},
})
r = self._read_until(cid, timeout)
if "error" in r:
return json.dumps(r["error"]), True
res = r.get("result", {})
content = res.get("content", [])
text = "".join(c.get("text", "") for c in content if isinstance(c, dict))
return text, bool(res.get("isError", False))
def close(self):
try:
self.p.stdin.close()
except Exception:
pass
try:
self.p.wait(timeout=5)
except Exception:
self.p.kill()
if self._stderr not in (subprocess.DEVNULL, None):
self._stderr.close()
demo_e2e/results/prueba_1_quotes.json
+68
View File
@@ -0,0 +1,68 @@
{
"name": "1 - Extraer citas estructuradas (quotes.toscrape.com)",
"verdict": "PASS",
"extracted_count": 10,
"sample": [
{
"author": "Albert Einstein",
"tags": [
"change",
"deep-thoughts",
"thinking",
"world"
],
"text": "“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”"
},
{
"author": "J.K. Rowling",
"tags": [
"abilities",
"choices"
],
"text": "“It is our choices, Harry, that show what we truly are, far more than our abilities.”"
},
{
"author": "Albert Einstein",
"tags": [
"inspirational",
"life",
"live",
"miracle",
"miracles"
],
"text": "“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”"
}
],
"steps": [
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://quotes.toscrape.com"
},
"ms": 735,
"is_error": false,
"response": "navigated to https://quotes.toscrape.com"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 437,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "[...document.querySelectorAll('.quote')].map(q=>({text:q.querySelector('.text').innerText,author:q.querySelector('.author').innerText,tags:[...q.querySelectorAll('.tag')].map(t=>t.innerText)}))"
},
"ms": 2,
"is_error": false,
"response": "[{\"author\":\"Albert Einstein\",\"tags\":[\"change\",\"deep-thoughts\",\"thinking\",\"world\"],\"text\":\"“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”\"},{\"author\":\"J.K. Rowling\",\"tags\":[\"abilities\",\"choices\"],\"text\":\"“It is our choices, Harry, that show what we truly are, far more than our abilities.”\"},{\"author\":\"Albert Einstein\",\"tags\":[\"inspirational\",\"life\",\"live\",\"miracle\",\"miracles\"],\"text\":\"“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”\"},{\"author\":\"Jane Austen\",\"tags\":[\"aliteracy\",\"books\",\"classic\",\"humor\"],\"text\":\"“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”\"},{\"author\":\"Marilyn Monroe\",\"tags\":[\"be-yourself\",\"inspirational\"],\"text\":\"“Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”\"},{\"author\":\"Albert Einstein\",\"tags\":[\"adulthood\",\"success\",\"value\"],\"text\":\"“Try not to become a man of success. Rather become a man of value.”\"},{\"author\":\"André Gide\",\"tags\":[\"life\",\"love\"],\"text\":\"“It is better to be hated for what you are than to be loved for what you are not.”\"},{\"author\":\"Thomas A. Edison\",\"tags\":[\"edison\",\"failure\",\"inspirational\",\"paraphrased\"],\"text\":\"“I have not failed. I've just found 10,000 ways that won't work.”\"},{\"author\":\"Eleanor Roosevelt\",\"tags\":[\"misattributed-eleanor-roosevelt\"],\"text\":\"“A woman is like a tea bag; you never know how strong it is until it's in hot water.”\"},{\"author\":\"Steve Martin\",\"tags\":[\"humor\",\"obvious\",\"simile\"],\"text\":\"“A day without sunshine is like, you know, night.”\"}]"
}
]
}
demo_e2e/results/prueba_2_perceive.json
+40
View File
@@ -0,0 +1,40 @@
{
"name": "2 - Percibir página (page_perceive AX outline)",
"verdict": "PASS",
"has_refs": true,
"has_link": true,
"outline_len": 4021,
"outline_preview": "RootWebArea \"The Internet\"\ngeneric\n generic\n separator\n generic\n StaticText \"Powered by \"\n InlineTextBox \"Powered by \"\n link \"Elemental Selenium\" #ref=169\n StaticText \"Elemental Selenium\"\n InlineTextBox \"Elemental \"\n InlineTextBox \"Selenium\"\nlink \"Fork me on GitHub\" #ref=72\n image \"Fork me on GitHub\"\ngeneric\n heading \"Welcome to the-internet\"\n StaticText \"Welcome to the-internet\"\n InlineTextBox \"Welcome to the-internet\"\n heading \"Available Examples\"\n StaticText \"Available Examples\"\n InlineTextBox \"Available Examples\"\n list\n listitem\n link \"A/B Testing\" #ref=79\n StaticText \"A/B Testing\"\n InlineTextBox \"A/B Testing\"\n listitem\n link \"Add/Remove Elements\" #ref=81\n StaticText \"Add/Remove Elements\"\n InlineTextBox \"Add/Remove Elements\"\n listitem\n link \"Basic Auth\" #ref=83\n StaticText \"Basic Auth\"\n InlineTextBox \"Basic Auth\"\n StaticText \" (user and pass: admin)\"\n InlineTextBox \" (user and pass: admin)\"\n listitem\n link \"Broken Images\" #ref=85\n StaticText \"Broken Images\"\n InlineTextBox \"Broken Images\"\n listitem\n link \"Challenging DOM\" #ref=87\n StaticText \"Challenging DOM\"\n ",
"steps": [
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com"
},
"ms": 388,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 833,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_perceive",
"args": {
"port": 9333,
"max_chars": 4000
},
"ms": 125,
"is_error": false,
"response": "RootWebArea \"The Internet\"\ngeneric\n generic\n separator\n generic\n StaticText \"Powered by \"\n InlineTextBox \"Powered by \"\n link \"Elemental Selenium\" #ref=169\n StaticText \"Elemental Selenium\"\n InlineTextBox \"Elemental \"\n InlineTextBox \"Selenium\"\nlink \"Fork me on GitHub\" #ref=72\n image \"Fork me on GitHub\"\ngeneric\n heading \"Welcome to the-internet\"\n StaticText \"Welcome to the-internet\"\n InlineTextBox \"Welcome to the-internet\"\n heading \"Available Examples\"\n StaticText \"Available Examples\"\n InlineTextBox \"Available Examples\"\n list\n listitem\n link \"A/B Testing\" #ref=79\n StaticText \"A/B Testing\"\n InlineTextBox \"A/B Testing\"\n listitem\n link \"Add/Remove Elements\" #ref=81\n StaticText \"Add/Remove Elements\"\n InlineTextBox \"Add/Remove Elements\"\n listitem\n link \"Basic Auth\" #ref=83\n StaticText \"Basic Auth\"\n InlineTextBox \"Basic Auth\"\n StaticText \" (user and pass: admin)\"\n InlineTextBox \" (user and pass: admin)\"\n listitem\n link \"Broken Images\" #ref=85\n StaticText \"Broken Images\"\n InlineTextBox \"Broken Images\"\n listitem\n link \"Challenging DOM\" #ref=87\n StaticText \"Challenging DOM\"\n InlineTextBox \"Challenging DOM\"\n listitem\n link \"Checkboxes\" #ref=89\n StaticText \"Checkboxes\"\n InlineTextBox \"Checkboxes\"\n listitem\n link \"Context Menu\" #ref=91\n StaticText \"Context Menu\"\n InlineTextBox \"Context Menu\"\n listitem\n link \"Digest Authentication\" #ref=93\n StaticText \"Digest Authentication\"\n InlineTextBox \"Digest Au"
}
]
}
demo_e2e/results/prueba_3_search.json
+118
View File
@@ -0,0 +1,118 @@
{
"name": "3 - Submit de formulario con teclado Enter (the-internet/login)",
"verdict": "PASS",
"flash": "You logged into a secure area!\n×",
"steps": [
{
"tool": "cookie_clear",
"args": {
"port": 9333
},
"ms": 1,
"is_error": false,
"response": "cookies cleared"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com/login"
},
"ms": 91,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com/login"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 249,
"is_error": false,
"response": "page loaded"
},
{
"tool": "dom_click",
"args": {
"port": 9333,
"selector": "#username"
},
"ms": 23,
"is_error": false,
"response": "clicked #username"
},
{
"tool": "dom_type",
"args": {
"port": 9333,
"text": "tomsmith"
},
"ms": 115,
"is_error": false,
"response": "typed text"
},
{
"tool": "dom_click",
"args": {
"port": 9333,
"selector": "#password"
},
"ms": 8,
"is_error": false,
"response": "clicked #password"
},
{
"tool": "dom_type",
"args": {
"port": 9333,
"text": "SuperSecretPassword!"
},
"ms": 279,
"is_error": false,
"response": "typed text"
},
{
"tool": "press_key",
"args": {
"port": 9333,
"key": "Enter"
},
"ms": 4,
"is_error": false,
"response": "pressed Enter"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 1,
"is_error": false,
"response": "page loaded"
},
{
"tool": "dom_wait_element",
"args": {
"port": 9333,
"selector": "#flash",
"timeout_ms": 8000
},
"ms": 435,
"is_error": false,
"response": "element appeared: #flash"
},
{
"tool": "page_get_text",
"args": {
"port": 9333,
"selector": "#flash",
"max_bytes": 200
},
"ms": 2,
"is_error": false,
"response": " You logged into a secure area!\n×"
}
]
}
demo_e2e/results/prueba_4_login_session.json
+233
View File
@@ -0,0 +1,233 @@
{
"name": "4 - Login + sesión persistente (storage_state)",
"verdict": "PASS",
"logged_in": true,
"kicked_after_clear": true,
"restored_after_load": true,
"flash_login": " You logged into a secure area!\n×",
"flash_after_clear": " You must login to view the secure area!\n×",
"flash_restored": "{\"path\":\"/secure\",\"secure\":true}",
"steps": [
{
"tool": "cookie_clear",
"args": {
"port": 9333
},
"ms": 2,
"is_error": false,
"response": "cookies cleared"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com/login"
},
"ms": 96,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com/login"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 247,
"is_error": false,
"response": "page loaded"
},
{
"tool": "dom_click",
"args": {
"port": 9333,
"selector": "#username"
},
"ms": 4,
"is_error": false,
"response": "clicked #username"
},
{
"tool": "dom_type",
"args": {
"port": 9333,
"text": "tomsmith"
},
"ms": 93,
"is_error": false,
"response": "typed text"
},
{
"tool": "dom_click",
"args": {
"port": 9333,
"selector": "#password"
},
"ms": 9,
"is_error": false,
"response": "clicked #password"
},
{
"tool": "dom_type",
"args": {
"port": 9333,
"text": "SuperSecretPassword!"
},
"ms": 290,
"is_error": false,
"response": "typed text"
},
{
"tool": "dom_click",
"args": {
"port": 9333,
"selector": "button[type=submit]"
},
"ms": 10,
"is_error": false,
"response": "clicked button[type=submit]"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 2,
"is_error": false,
"response": "page loaded"
},
{
"tool": "dom_wait_element",
"args": {
"port": 9333,
"selector": "#flash",
"timeout_ms": 8000
},
"ms": 431,
"is_error": false,
"response": "element appeared: #flash"
},
{
"tool": "page_get_text",
"args": {
"port": 9333,
"selector": "#flash",
"max_bytes": 300
},
"ms": 3,
"is_error": false,
"response": " You logged into a secure area!\n×"
},
{
"tool": "storage_save",
"args": {
"port": 9333,
"path": "/tmp/demo_session.json"
},
"ms": 9,
"is_error": false,
"response": "storage state saved to /tmp/demo_session.json"
},
{
"tool": "cookie_clear",
"args": {
"port": 9333
},
"ms": 4,
"is_error": false,
"response": "cookies cleared"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com/secure"
},
"ms": 195,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com/secure"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 229,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_get_text",
"args": {
"port": 9333,
"selector": "#flash",
"max_bytes": 300
},
"ms": 0,
"is_error": false,
"response": " You must login to view the secure area!\n×"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com"
},
"ms": 92,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 241,
"is_error": false,
"response": "page loaded"
},
{
"tool": "storage_load",
"args": {
"port": 9333,
"path": "/tmp/demo_session.json"
},
"ms": 5,
"is_error": false,
"response": "storage state loaded from /tmp/demo_session.json"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com/secure"
},
"ms": 102,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com/secure"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 220,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "JSON.stringify({path:location.pathname,secure:document.body.innerText.includes('Secure Area')})"
},
"ms": 2,
"is_error": false,
"response": "{\"path\":\"/secure\",\"secure\":true}"
}
]
}
demo_e2e/results/prueba_5_books.json
+115
View File
@@ -0,0 +1,115 @@
{
"name": "5 - Scraping paginado + dedup (books.toscrape.com, 3 páginas)",
"verdict": "PASS",
"total_scraped": 60,
"unique_count": 60,
"sample": [
{
"price": "£51.77",
"stock": "In stock",
"title": "A Light in the Attic"
},
{
"price": "£53.74",
"stock": "In stock",
"title": "Tipping the Velvet"
},
{
"price": "£50.10",
"stock": "In stock",
"title": "Soumission"
}
],
"steps": [
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://books.toscrape.com/catalogue/page-1.html"
},
"ms": 164,
"is_error": false,
"response": "navigated to https://books.toscrape.com/catalogue/page-1.html"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 659,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "[...document.querySelectorAll('.product_pod')].map(b=>({title:b.querySelector('h3 a').getAttribute('title'),price:b.querySelector('.price_color').innerText,stock:b.querySelector('.availability').innerText.trim()}))"
},
"ms": 3,
"is_error": false,
"response": "[{\"price\":\"£51.77\",\"stock\":\"In stock\",\"title\":\"A Light in the Attic\"},{\"price\":\"£53.74\",\"stock\":\"In stock\",\"title\":\"Tipping the Velvet\"},{\"price\":\"£50.10\",\"stock\":\"In stock\",\"title\":\"Soumission\"},{\"price\":\"£47.82\",\"stock\":\"In stock\",\"title\":\"Sharp Objects\"},{\"price\":\"£54.23\",\"stock\":\"In stock\",\"title\":\"Sapiens: A Brief History of Humankind\"},{\"price\":\"£22.65\",\"stock\":\"In stock\",\"title\":\"The Requiem Red\"},{\"price\":\"£33.34\",\"stock\":\"In stock\",\"title\":\"The Dirty Little Secrets of Getting Your Dream Job\"},{\"price\":\"£17.93\",\"stock\":\"In stock\",\"title\":\"The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull\"},{\"price\":\"£22.60\",\"stock\":\"In stock\",\"title\":\"The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics\"},{\"price\":\"£52.15\",\"stock\":\"In stock\",\"title\":\"The Black Maria\"},{\"price\":\"£13.99\",\"stock\":\"In stock\",\"title\":\"Starving Hearts (Triangular Trade Trilogy, #1)\"},{\"price\":\"£20.66\",\"stock\":\"In stock\",\"title\":\"Shakespeare's Sonnets\"},{\"price\":\"£17.46\",\"stock\":\"In stock\",\"title\":\"Set Me Free\"},{\"price\":\"£52.29\",\"stock\":\"In stock\",\"title\":\"Scott Pilgrim's Precious Little Life (Scott Pilgrim #1)\"},{\"price\":\"£35.02\",\"stock\":\"In stock\",\"title\":\"Rip it Up and Start Again\"},{\"price\":\"£57.25\",\"stock\":\"In stock\",\"title\":\"Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991\"},{\"price\":\"£23.88\",\"stock\":\"In stock\",\"title\":\"Olio\"},{\"price\":\"£37.59\",\"stock\":\"In stock\",\"title\":\"Mesaerion: The Best Science Fiction Stories 1800-1849\"},{\"price\":\"£51.33\",\"stock\":\"In stock\",\"title\":\"Libertarianism for Beginners\"},{\"price\":\"£45.17\",\"stock\":\"In stock\",\"title\":\"It's Only the Himalayas\"}]"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://books.toscrape.com/catalogue/page-2.html"
},
"ms": 107,
"is_error": false,
"response": "navigated to https://books.toscrape.com/catalogue/page-2.html"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 632,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "[...document.querySelectorAll('.product_pod')].map(b=>({title:b.querySelector('h3 a').getAttribute('title'),price:b.querySelector('.price_color').innerText,stock:b.querySelector('.availability').innerText.trim()}))"
},
"ms": 4,
"is_error": false,
"response": "[{\"price\":\"£12.84\",\"stock\":\"In stock\",\"title\":\"In Her Wake\"},{\"price\":\"£37.32\",\"stock\":\"In stock\",\"title\":\"How Music Works\"},{\"price\":\"£30.52\",\"stock\":\"In stock\",\"title\":\"Foolproof Preserving: A Guide to Small Batch Jams, Jellies, Pickles, Condiments, and More: A Foolproof Guide to Making Small Batch Jams, Jellies, Pickles, Condiments, and More\"},{\"price\":\"£25.27\",\"stock\":\"In stock\",\"title\":\"Chase Me (Paris Nights #2)\"},{\"price\":\"£34.53\",\"stock\":\"In stock\",\"title\":\"Black Dust\"},{\"price\":\"£54.64\",\"stock\":\"In stock\",\"title\":\"Birdsong: A Story in Pictures\"},{\"price\":\"£22.50\",\"stock\":\"In stock\",\"title\":\"America's Cradle of Quarterbacks: Western Pennsylvania's Football Factory from Johnny Unitas to Joe Montana\"},{\"price\":\"£53.13\",\"stock\":\"In stock\",\"title\":\"Aladdin and His Wonderful Lamp\"},{\"price\":\"£40.30\",\"stock\":\"In stock\",\"title\":\"Worlds Elsewhere: Journeys Around Shakespeare’s Globe\"},{\"price\":\"£44.18\",\"stock\":\"In stock\",\"title\":\"Wall and Piece\"},{\"price\":\"£17.66\",\"stock\":\"In stock\",\"title\":\"The Four Agreements: A Practical Guide to Personal Freedom\"},{\"price\":\"£31.05\",\"stock\":\"In stock\",\"title\":\"The Five Love Languages: How to Express Heartfelt Commitment to Your Mate\"},{\"price\":\"£23.82\",\"stock\":\"In stock\",\"title\":\"The Elephant Tree\"},{\"price\":\"£36.89\",\"stock\":\"In stock\",\"title\":\"The Bear and the Piano\"},{\"price\":\"£15.94\",\"stock\":\"In stock\",\"title\":\"Sophie's World\"},{\"price\":\"£33.29\",\"stock\":\"In stock\",\"title\":\"Penny Maybe\"},{\"price\":\"£18.02\",\"stock\":\"In stock\",\"title\":\"Maude (1883-1993):She Grew Up with the country\"},{\"price\":\"£19.63\",\"stock\":\"In stock\",\"title\":\"In a Dark, Dark Wood\"},{\"price\":\"£52.22\",\"stock\":\"In stock\",\"title\":\"Behind Closed Doors\"},{\"price\":\"£33.63\",\"stock\":\"In stock\",\"title\":\"You can't bury them all: Poems\"}]"
},
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://books.toscrape.com/catalogue/page-3.html"
},
"ms": 110,
"is_error": false,
"response": "navigated to https://books.toscrape.com/catalogue/page-3.html"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 429,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "[...document.querySelectorAll('.product_pod')].map(b=>({title:b.querySelector('h3 a').getAttribute('title'),price:b.querySelector('.price_color').innerText,stock:b.querySelector('.availability').innerText.trim()}))"
},
"ms": 4,
"is_error": false,
"response": "[{\"price\":\"£57.31\",\"stock\":\"In stock\",\"title\":\"Slow States of Collapse: Poems\"},{\"price\":\"£26.41\",\"stock\":\"In stock\",\"title\":\"Reasons to Stay Alive\"},{\"price\":\"£47.61\",\"stock\":\"In stock\",\"title\":\"Private Paris (Private #10)\"},{\"price\":\"£23.11\",\"stock\":\"In stock\",\"title\":\"#HigherSelfie: Wake Up Your Life. Free Your Soul. Find Your Tribe.\"},{\"price\":\"£45.07\",\"stock\":\"In stock\",\"title\":\"Without Borders (Wanderlove #1)\"},{\"price\":\"£31.77\",\"stock\":\"In stock\",\"title\":\"When We Collided\"},{\"price\":\"£50.27\",\"stock\":\"In stock\",\"title\":\"We Love You, Charlie Freeman\"},{\"price\":\"£14.27\",\"stock\":\"In stock\",\"title\":\"Untitled Collection: Sabbath Poems 2014\"},{\"price\":\"£44.18\",\"stock\":\"In stock\",\"title\":\"Unseen City: The Majesty of Pigeons, the Discreet Charm of Snails \\u0026 Other Wonders of the Urban Wilderness\"},{\"price\":\"£18.78\",\"stock\":\"In stock\",\"title\":\"Unicorn Tracks\"},{\"price\":\"£25.52\",\"stock\":\"In stock\",\"title\":\"Unbound: How Eight Technologies Made Us Human, Transformed Society, and Brought Our World to the Brink\"},{\"price\":\"£16.28\",\"stock\":\"In stock\",\"title\":\"Tsubasa: WoRLD CHRoNiCLE 2 (Tsubasa WoRLD CHRoNiCLE #2)\"},{\"price\":\"£31.12\",\"stock\":\"In stock\",\"title\":\"Throwing Rocks at the Google Bus: How Growth Became the Enemy of Prosperity\"},{\"price\":\"£19.49\",\"stock\":\"In stock\",\"title\":\"This One Summer\"},{\"price\":\"£17.27\",\"stock\":\"In stock\",\"title\":\"Thirst\"},{\"price\":\"£19.09\",\"stock\":\"In stock\",\"title\":\"The Torch Is Passed: A Harding Family Story\"},{\"price\":\"£56.13\",\"stock\":\"In stock\",\"title\":\"The Secret of Dreadwillow Carse\"},{\"price\":\"£56.41\",\"stock\":\"In stock\",\"title\":\"The Pioneer Woman Cooks: Dinnertime: Comfort Classics, Freezer Food, 16-Minute Meals, and Other Delicious Ways to Solve Supper!\"},{\"price\":\"£56.50\",\"stock\":\"In stock\",\"title\":\"The Past Never Ends\"},{\"price\":\"£45.22\",\"stock\":\"In stock\",\"title\":\"The Natural History of Us (The Fine Art of Pretending #2)\"}]"
}
]
}
demo_e2e/results/prueba_6_session_storage.json
+89
View File
@@ -0,0 +1,89 @@
{
"name": "6 - sessionStorage en storage_state (fix D)",
"verdict": "PASS",
"cleared_value": "null",
"restored_value": "demo_v",
"json_has_sessionstorage": true,
"steps": [
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://the-internet.herokuapp.com"
},
"ms": 112,
"is_error": false,
"response": "navigated to https://the-internet.herokuapp.com"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 224,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "window.sessionStorage.setItem('demo_k','demo_v'); 'set'"
},
"ms": 4,
"is_error": false,
"response": "set"
},
{
"tool": "storage_save",
"args": {
"port": 9333,
"path": "/tmp/demo_ss.json"
},
"ms": 10,
"is_error": false,
"response": "storage state saved to /tmp/demo_ss.json"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "window.sessionStorage.clear(); 'cleared'"
},
"ms": 2,
"is_error": false,
"response": "cleared"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "String(window.sessionStorage.getItem('demo_k'))"
},
"ms": 1,
"is_error": false,
"response": "null"
},
{
"tool": "storage_load",
"args": {
"port": 9333,
"path": "/tmp/demo_ss.json"
},
"ms": 6,
"is_error": false,
"response": "storage state loaded from /tmp/demo_ss.json"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "String(window.sessionStorage.getItem('demo_k'))"
},
"ms": 1,
"is_error": false,
"response": "demo_v"
}
]
}
demo_e2e/results/prueba_7_find_honesto.json
+49
View File
@@ -0,0 +1,49 @@
{
"name": "7 - find_by_text honesto: error en no-encontrado (fix E)",
"verdict": "PASS",
"found_present": "body > div > div:nth-of-type(1) > div:nth-of-type(2) > p > a",
"missing_is_error": true,
"missing_response": "cdp find by text: no se encontro elemento con texto \"ZZZ_texto_inexistente_42\"",
"steps": [
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://quotes.toscrape.com"
},
"ms": 122,
"is_error": false,
"response": "navigated to https://quotes.toscrape.com"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 232,
"is_error": false,
"response": "page loaded"
},
{
"tool": "dom_find_by_text",
"args": {
"port": 9333,
"text": "Login"
},
"ms": 7,
"is_error": false,
"response": "body > div > div:nth-of-type(1) > div:nth-of-type(2) > p > a"
},
{
"tool": "dom_find_by_text",
"args": {
"port": 9333,
"text": "ZZZ_texto_inexistente_42"
},
"ms": 5,
"is_error": true,
"response": "cdp find by text: no se encontro elemento con texto \"ZZZ_texto_inexistente_42\""
}
]
}
demo_e2e/results/prueba_8_verificacion.json
+68
View File
@@ -0,0 +1,68 @@
{
"name": "8 - Verificación post-acción: click oculto / type sin foco dan error (fix B)",
"verdict": "PASS",
"click_hidden_error": true,
"type_nofocus_error": true,
"steps": [
{
"tool": "tab_navigate",
"args": {
"port": 9333,
"url": "https://quotes.toscrape.com"
},
"ms": 112,
"is_error": false,
"response": "navigated to https://quotes.toscrape.com"
},
{
"tool": "page_wait_load",
"args": {
"port": 9333,
"timeout_ms": 12000
},
"ms": 212,
"is_error": false,
"response": "page loaded"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "var b=document.createElement('button');b.id='hidden_btn';b.textContent='x';b.style.display='none';document.body.appendChild(b);'injected'"
},
"ms": 2,
"is_error": false,
"response": "injected"
},
{
"tool": "dom_click",
"args": {
"port": 9333,
"selector": "#hidden_btn"
},
"ms": 2,
"is_error": true,
"response": "cdp click: elemento \"#hidden_btn\" existe pero no es visible/clickable (display:none, oculto, opacity 0 o tamaño 0)"
},
{
"tool": "page_eval_js",
"args": {
"port": 9333,
"expression": "if(document.activeElement){document.activeElement.blur();} document.body.focus(); 'blurred'"
},
"ms": 1,
"is_error": false,
"response": "blurred"
},
{
"tool": "dom_type",
"args": {
"port": 9333,
"text": "fantasma"
},
"ms": 1,
"is_error": true,
"response": "cdp type text: no hay campo de texto enfocado (activeElement: body); usa CdpClick sobre el input primero"
}
]
}
demo_e2e/results/summary.json
+42
View File
@@ -0,0 +1,42 @@
[
{
"prueba": "prueba_1_quotes",
"verdict": "PASS",
"detail": "10 citas"
},
{
"prueba": "prueba_2_perceive",
"verdict": "PASS",
"detail": "outline 4021 chars, refs=True"
},
{
"prueba": "prueba_3_search",
"verdict": "PASS",
"detail": "flash='You logged into a secure area!\n×'"
},
{
"prueba": "prueba_4_login_session",
"verdict": "PASS",
"detail": "login=True restore=True"
},
{
"prueba": "prueba_5_books",
"verdict": "PASS",
"detail": "60 libros únicos"
},
{
"prueba": "prueba_6_session_storage",
"verdict": "PASS",
"detail": "clear='null' restore='demo_v'"
},
{
"prueba": "prueba_7_find_honesto",
"verdict": "PASS",
"detail": "present_ok=True miss_error=True"
},
{
"prueba": "prueba_8_verificacion",
"verdict": "PASS",
"detail": "click_hidden_err=True type_nofocus_err=True"
}
]
demo_e2e/run_demo.py
+332
View File
@@ -0,0 +1,332 @@
"""Ejecuta 5 pruebas e2e graduadas contra el servidor browser_mcp para validar
las capacidades de control de navegador (CDP) sobre sitios sandbox estables.
Cada prueba se conecta al Chrome aislado del MCP en el puerto 9333 (que debe
estar ya corriendo) y ejerce un conjunto de tools. Los resultados (pasos,
respuestas, veredicto y datos extraídos) se guardan en results/.
Uso:
python3 run_demo.py
Requisitos:
- Chrome/Chromium headless en CDP 9333 (Chrome aislado del MCP).
- Binario browser_mcp compilado.
- FN_REGISTRY_ROOT para que la tool page_perceive pueda invocar fn run.
"""
import json
import os
import time
from mcp_client import MCPClient
ROOT = "/home/enmanuel/fn_registry"
EXE = os.path.join(ROOT, "projects/web_scraping/apps/browser_mcp/browser_mcp")
RESULTS = os.path.join(os.path.dirname(__file__), "results")
PORT = 9333
os.makedirs(RESULTS, exist_ok=True)
class Recorder:
def __init__(self, client, log):
self.c = client
self.log = log
self.steps = []
def step(self, tool, args, timeout=60):
t0 = time.time()
try:
text, is_err = self.c.call(tool, args, timeout=timeout)
except Exception as e:
text, is_err = f"EXCEPTION: {e}", True
dt = round((time.time() - t0) * 1000)
rec = {"tool": tool, "args": args, "ms": dt, "is_error": is_err,
"response": text[:2000]}
self.steps.append(rec)
self.log.write(f" [{tool}] {dt}ms err={is_err} -> {text[:160]}\n")
self.log.flush()
return text, is_err
def save(name, payload):
path = os.path.join(RESULTS, name)
with open(path, "w", encoding="utf-8") as f:
json.dump(payload, f, ensure_ascii=False, indent=2)
def prueba_1_quotes(c, log):
"""Extraer citas estructuradas (valida fix %v: array JS -> JSON real)."""
r = Recorder(c, log)
r.step("tab_navigate", {"port": PORT, "url": "https://quotes.toscrape.com"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
expr = ("[...document.querySelectorAll('.quote')].map(q=>({"
"text:q.querySelector('.text').innerText,"
"author:q.querySelector('.author').innerText,"
"tags:[...q.querySelectorAll('.tag')].map(t=>t.innerText)}))")
text, err = r.step("page_eval_js", {"port": PORT, "expression": expr})
quotes = []
try:
quotes = json.loads(text)
except Exception:
pass
ok = (not err) and isinstance(quotes, list) and len(quotes) >= 10 \
and all("author" in q for q in quotes)
verdict = "PASS" if ok else "FAIL"
save("prueba_1_quotes.json", {
"name": "1 - Extraer citas estructuradas (quotes.toscrape.com)",
"verdict": verdict,
"extracted_count": len(quotes),
"sample": quotes[:3],
"steps": r.steps,
})
return verdict, f"{len(quotes)} citas"
def prueba_2_perceive(c, log):
"""Percibir página como outline AX accionable (P0.1)."""
r = Recorder(c, log)
r.step("tab_navigate", {"port": PORT, "url": "https://the-internet.herokuapp.com"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
text, err = r.step("page_perceive", {"port": PORT, "max_chars": 4000}, timeout=90)
has_refs = "#ref=" in text
has_link = "link" in text.lower()
ok = (not err) and has_refs and has_link and len(text) > 100
verdict = "PASS" if ok else "FAIL"
save("prueba_2_perceive.json", {
"name": "2 - Percibir página (page_perceive AX outline)",
"verdict": verdict,
"has_refs": has_refs, "has_link": has_link, "outline_len": len(text),
"outline_preview": text[:1500],
"steps": r.steps,
})
return verdict, f"outline {len(text)} chars, refs={has_refs}"
def prueba_3_search(c, log):
"""Submit de formulario con teclado: type + press_key Enter (sin click submit)."""
r = Recorder(c, log)
# Form HTML normal (the-internet/login): tras escribir credenciales, Enter
# envía el form. Valida type + press_key Enter de forma fiable, sin depender
# de un widget JS (como el typeahead de Wikipedia, que ignora el keyevent).
base = "https://the-internet.herokuapp.com"
r.step("cookie_clear", {"port": PORT})
r.step("tab_navigate", {"port": PORT, "url": base + "/login"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
r.step("dom_click", {"port": PORT, "selector": "#username"})
r.step("dom_type", {"port": PORT, "text": "tomsmith"})
r.step("dom_click", {"port": PORT, "selector": "#password"})
r.step("dom_type", {"port": PORT, "text": "SuperSecretPassword!"})
r.step("press_key", {"port": PORT, "key": "Enter"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
r.step("dom_wait_element", {"port": PORT, "selector": "#flash", "timeout_ms": 8000})
flash, err = r.step("page_get_text", {"port": PORT, "selector": "#flash", "max_bytes": 200})
ok = (not err) and ("logged into" in flash.lower())
verdict = "PASS" if ok else "FAIL"
save("prueba_3_search.json", {
"name": "3 - Submit de formulario con teclado Enter (the-internet/login)",
"verdict": verdict,
"flash": flash.strip(),
"steps": r.steps,
})
return verdict, f"flash='{flash.strip()[:40]}'"
def prueba_4_login_session(c, log):
"""Login + persistir sesión: storage_save -> cookie_clear -> storage_load."""
r = Recorder(c, log)
base = "https://the-internet.herokuapp.com"
# Sesión limpia: las cookies de pruebas previas (otros dominios) no deben
# contaminar el storage_state que guardaremos.
r.step("cookie_clear", {"port": PORT})
r.step("tab_navigate", {"port": PORT, "url": base + "/login"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
r.step("dom_click", {"port": PORT, "selector": "#username"})
r.step("dom_type", {"port": PORT, "text": "tomsmith"})
r.step("dom_click", {"port": PORT, "selector": "#password"})
r.step("dom_type", {"port": PORT, "text": "SuperSecretPassword!"})
r.step("dom_click", {"port": PORT, "selector": "button[type=submit]"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
r.step("dom_wait_element", {"port": PORT, "selector": "#flash", "timeout_ms": 8000})
flash1, _ = r.step("page_get_text", {"port": PORT, "selector": "#flash", "max_bytes": 300})
# "logged into" sólo aparece en el flash de ÉXITO; evita colisión con el
# mensaje de error "view the secure area" que contiene "secure area".
logged_in = "logged into" in flash1.lower()
# Guardar sesión, limpiar cookies, restaurar.
r.step("storage_save", {"port": PORT, "path": "/tmp/demo_session.json"})
r.step("cookie_clear", {"port": PORT})
# Tras limpiar cookies, /secure debe expulsar a login.
r.step("tab_navigate", {"port": PORT, "url": base + "/secure"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
after_clear, _ = r.step("page_get_text", {"port": PORT, "selector": "#flash", "max_bytes": 300})
kicked = "must login" in after_clear.lower()
# Restaurar sesión: navegar al dominio, load, volver a /secure.
r.step("tab_navigate", {"port": PORT, "url": base})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
r.step("storage_load", {"port": PORT, "path": "/tmp/demo_session.json"})
r.step("tab_navigate", {"port": PORT, "url": base + "/secure"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
# Check robusto (no #flash, que sufre timing): si seguimos en /secure y el
# body menciona "Secure Area", la sesión se restauró; si nos echó, pathname
# vuelve a "/".
probe, _ = r.step("page_eval_js", {"port": PORT, "expression":
"JSON.stringify({path:location.pathname,secure:document.body.innerText.includes('Secure Area')})"})
flash2 = probe
try:
pj = json.loads(json.loads(probe) if probe.strip().startswith('"') else probe)
except Exception:
pj = {}
restored = (pj.get("path") == "/secure") and bool(pj.get("secure"))
ok = logged_in and kicked and restored
verdict = "PASS" if ok else "FAIL"
save("prueba_4_login_session.json", {
"name": "4 - Login + sesión persistente (storage_state)",
"verdict": verdict,
"logged_in": logged_in, "kicked_after_clear": kicked, "restored_after_load": restored,
"flash_login": flash1[:200], "flash_after_clear": after_clear[:200], "flash_restored": flash2[:200],
"steps": r.steps,
})
return verdict, f"login={logged_in} restore={restored}"
def prueba_5_books(c, log):
"""Scraping paginado multi-página + dedup (books.toscrape.com, 3 páginas)."""
r = Recorder(c, log)
all_books = []
for page in (1, 2, 3):
url = f"https://books.toscrape.com/catalogue/page-{page}.html"
r.step("tab_navigate", {"port": PORT, "url": url})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
expr = ("[...document.querySelectorAll('.product_pod')].map(b=>({"
"title:b.querySelector('h3 a').getAttribute('title'),"
"price:b.querySelector('.price_color').innerText,"
"stock:b.querySelector('.availability').innerText.trim()}))")
text, err = r.step("page_eval_js", {"port": PORT, "expression": expr})
try:
all_books.extend(json.loads(text))
except Exception:
pass
unique = {b["title"]: b for b in all_books if isinstance(b, dict) and b.get("title")}
ok = len(unique) >= 60
verdict = "PASS" if ok else "FAIL"
save("prueba_5_books.json", {
"name": "5 - Scraping paginado + dedup (books.toscrape.com, 3 páginas)",
"verdict": verdict,
"total_scraped": len(all_books), "unique_count": len(unique),
"sample": list(unique.values())[:3],
"steps": r.steps,
})
return verdict, f"{len(unique)} libros únicos"
def prueba_6_session_storage(c, log):
"""sessionStorage en storage_state: set -> save -> clear -> load -> get (fix D)."""
r = Recorder(c, log)
r.step("tab_navigate", {"port": PORT, "url": "https://the-internet.herokuapp.com"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
r.step("page_eval_js", {"port": PORT, "expression":
"window.sessionStorage.setItem('demo_k','demo_v'); 'set'"})
r.step("storage_save", {"port": PORT, "path": "/tmp/demo_ss.json"})
r.step("page_eval_js", {"port": PORT, "expression": "window.sessionStorage.clear(); 'cleared'"})
cleared, _ = r.step("page_eval_js", {"port": PORT, "expression":
"String(window.sessionStorage.getItem('demo_k'))"})
r.step("storage_load", {"port": PORT, "path": "/tmp/demo_ss.json"})
got, err = r.step("page_eval_js", {"port": PORT, "expression":
"String(window.sessionStorage.getItem('demo_k'))"})
# Verificar también que el JSON guardado incluye el campo sessionStorage.
saved_has_ss = False
try:
with open("/tmp/demo_ss.json", encoding="utf-8") as f:
saved_has_ss = json.load(f).get("sessionStorage", {}).get("demo_k") == "demo_v"
except Exception:
pass
ok = (not err) and (cleared.strip() == "null") and ("demo_v" in got) and saved_has_ss
verdict = "PASS" if ok else "FAIL"
save("prueba_6_session_storage.json", {
"name": "6 - sessionStorage en storage_state (fix D)",
"verdict": verdict,
"cleared_value": cleared.strip(), "restored_value": got.strip(), "json_has_sessionstorage": saved_has_ss,
"steps": r.steps,
})
return verdict, f"clear='{cleared.strip()}' restore='{got.strip()}'"
def prueba_7_find_honesto(c, log):
"""find_by_text con texto inexistente -> error explícito, no vacío (fix E)."""
r = Recorder(c, log)
r.step("tab_navigate", {"port": PORT, "url": "https://quotes.toscrape.com"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
# Texto presente: debe encontrar (no error).
found, ferr = r.step("dom_find_by_text", {"port": PORT, "text": "Login"})
# Texto inexistente: debe dar error explícito (antes: vacío sin error).
miss, merr = r.step("dom_find_by_text", {"port": PORT, "text": "ZZZ_texto_inexistente_42"})
ok = (not ferr) and bool(found.strip()) and merr and ("no se encontro" in miss.lower())
verdict = "PASS" if ok else "FAIL"
save("prueba_7_find_honesto.json", {
"name": "7 - find_by_text honesto: error en no-encontrado (fix E)",
"verdict": verdict,
"found_present": found.strip()[:80], "missing_is_error": merr, "missing_response": miss.strip()[:120],
"steps": r.steps,
})
return verdict, f"present_ok={bool(found.strip())} miss_error={merr}"
def prueba_8_verificacion(c, log):
"""Verificación post-acción: click oculto y type sin foco dan error (fix B / P1)."""
r = Recorder(c, log)
r.step("tab_navigate", {"port": PORT, "url": "https://quotes.toscrape.com"})
r.step("page_wait_load", {"port": PORT, "timeout_ms": 12000})
# Inyectar un botón oculto y comprobar que click da error (no clic al aire).
r.step("page_eval_js", {"port": PORT, "expression":
"var b=document.createElement('button');b.id='hidden_btn';b.textContent='x';"
"b.style.display='none';document.body.appendChild(b);'injected'"})
_, click_hidden_err = r.step("dom_click", {"port": PORT, "selector": "#hidden_btn"})
# Quitar el foco y comprobar que type da error (no escribe a la nada).
r.step("page_eval_js", {"port": PORT, "expression":
"if(document.activeElement){document.activeElement.blur();} document.body.focus(); 'blurred'"})
_, type_nofocus_err = r.step("dom_type", {"port": PORT, "text": "fantasma"})
ok = bool(click_hidden_err) and bool(type_nofocus_err)
verdict = "PASS" if ok else "FAIL"
save("prueba_8_verificacion.json", {
"name": "8 - Verificación post-acción: click oculto / type sin foco dan error (fix B)",
"verdict": verdict,
"click_hidden_error": bool(click_hidden_err),
"type_nofocus_error": bool(type_nofocus_err),
"steps": r.steps,
})
return verdict, f"click_hidden_err={bool(click_hidden_err)} type_nofocus_err={bool(type_nofocus_err)}"
def main():
log = open(os.path.join(RESULTS, "run.log"), "w", encoding="utf-8")
log.write(f"=== Demo e2e browser_mcp @ {time.strftime('%d/%m/%Y %H:%M')} ===\n")
client = MCPClient(EXE, env={"FN_REGISTRY_ROOT": ROOT}, cwd=ROOT,
stderr_path=os.path.join(RESULTS, "mcp_stderr.log"))
init = client.initialize()
log.write(f"initialize: {json.dumps(init.get('result', {}).get('serverInfo', {}))}\n")
pruebas = [prueba_1_quotes, prueba_2_perceive, prueba_3_search,
prueba_4_login_session, prueba_5_books,
prueba_6_session_storage, prueba_7_find_honesto,
prueba_8_verificacion]
summary = []
for fn in pruebas:
name = fn.__doc__.split("\n")[0]
log.write(f"\n--- {fn.__name__}: {name}\n")
try:
verdict, detail = fn(client, log)
except Exception as e:
verdict, detail = "ERROR", str(e)
summary.append({"prueba": fn.__name__, "verdict": verdict, "detail": detail})
log.write(f" => {verdict} ({detail})\n")
client.close()
save("summary.json", summary)
log.write("\n=== RESUMEN ===\n")
for s in summary:
log.write(f"{s['verdict']:6} {s['prueba']:24} {s['detail']}\n")
log.close()
print(json.dumps(summary, ensure_ascii=False, indent=2))
if __name__ == "__main__":
main()