arcade-mcp/libs/arcade-mcp-server/arcade_mcp_server/transports/stdio.py
Eric Gustin 3424ec8219
MCP Local (#563)
Versions:
* arcade-mcp\==1.0.0rc1
* arcade-mcp-server\==1.0.0rc1
* arcade-core\==2.5.0rc1
* arcade-tdk\==2.6.0rc1
* arcade-serve\==2.2.0rc1

### Summary
Adds first-class MCP support across Arcade, introduces a new MCP server
and CLI, unifies the project under the arcade-mcp name, overhauls
templates/scaffolding, and improves developer tooling, secrets
management, and examples.

### Highlights
- **MCP Server & Core**
- New MCP server with stdio and HTTP/SSE transports, session management,
resumability, and lifecycle handling.
- FastAPI-like `MCPApp` for building servers with lazy init; integrated
worker+MCP HTTP app option.
- Middleware system (logging and error handling), robust exception
hierarchy, and Pydantic-based settings.
- Async-safe managers for tools, resources, and prompts backed by
registries and locks.
- Developer-facing, transport-agnostic runtime context interfaces (logs,
tools, prompts, resources, sampling, UI, notifications).
- Conversion from Arcade ToolDefinition to MCP tool schema; OpenAI JSON
tool schema converter.
  - Parser supports `@app.tool`/`@app.tool(...)` decorators.

- **CLI**
  - New `mcp` command to run MCP servers with stdio or HTTP/SSE.
- New `secret` command to set/list/unset tool secrets (supports .env
input, preserves original casing for lookups).
- `new` command refactored; option to create a full toolkit package with
scaffolding.
  - `chat` command removed.
- `serve.py` imports updated to `arcade_serve.fastapi.telemetry`;
version retrieval now uses `arcade-mcp`.
  - `show.py` refactor to use new local catalog utilities.
- `display_tool_details` improved: adds “Default” column and handles
nested properties.

- **Configuration & Discovery**
- New `configure.py` to set up Claude Desktop, Cursor, and VS Code to
connect to local or Arcade Cloud MCP servers.
- Discovery utilities to find/install toolkits, build `ToolCatalog`s,
analyze files for tools, load kits from directories (pyproject parsing),
and build minimal toolkits.
- Better handling of provider API key resolution and evaluation suite
loading.

- **Templates & Scaffolding**
- Reorganized template structure (minimal vs full); moved
`.pre-commit-config.yaml`, `.ruff.toml`, license, Makefile, README,
tests, and tools layout to correct paths.
  - Minimal template adds `.env.example` for runtime secret injection.
- Template pyproject updated for MCP servers; includes sample server
with greeting and secret-reveal tools.
  - Authorization flow in templates simplified.

- **Repo-wide Renaming & Examples**
- Migrates references from `arcade-ai` to `arcade-mcp` across READMEs,
scripts, and package metadata.
- Examples updated (LangChain/LangGraph/AI SDK/TypeScript) and package
name changed to `arcade-mcp-sdk`.

- **Evals & Core Utilities**
- Evals now use OpenAI tooling format (`OpenAIToolList`, `to_openai`);
`tool_eval` takes `provider_api_key`.
- Core utilities: fixed `does_function_return_value` by dedenting before
parse; version bump to `2.5.0rc1` and dependency cleanup.

- **Tooling & CI**
- `setup-uv-env` action splits toolkit vs contrib dependency
installation.
- Pre-commit: excludes `libs/arcade-mcp-server/mkdocs.yml` and
`libs/tests/` from YAML and Ruff hooks; Ruff per-file ignores (e.g.,
C901 in `libs/**/*.py`, TRY400 in server docs paths).
- Makefile updates for uv env setup, quality checks, tests, builds, and
new `shell` target.
  - Added Makefile to MCP server library to streamline dev workflow.

- **Cleanup**
  - Removed `claude.json` config.
- Simplified stdio entrypoint; removed unused imports (`arcade_gmail`,
`arcade_search`).

### Breaking Changes
- **CLI**: `chat` command removed; use `mcp`, `secret`, and updated
`new`.
- **Naming**: All users should update references from `arcade-ai` to
`arcade-mcp`.
- **Templates**: File paths moved; downstream scripts referencing old
template locations may need updates.

### Getting Started
- Run an MCP server:
  - `arcade mcp --stdio --toolkits your_toolkit`
  - `arcade mcp --http --toolkits your_toolkit`
- Manage secrets:
  - `arcade secret set your_toolkit KEY=value`
  - `arcade secret list your_toolkit`
  - `arcade secret unset your_toolkit KEY`
- Configure clients:
- `arcade configure` to set up Claude Desktop, Cursor, and VS Code for
local/Arcade Cloud MCP.

---------

Co-authored-by: Sam Partee <sam@arcade-ai.com>
Co-authored-by: Shub <125150494+shubcodes@users.noreply.github.com>
2025-09-25 15:28:15 -07:00

209 lines
6.5 KiB
Python

"""
Stdio Transport
Provides stdio (stdin/stdout) transport for MCP communication.
"""
import asyncio
import contextlib
import logging
import queue
import signal
import sys
import threading
import uuid
from collections.abc import AsyncIterator
from typing import Any
from arcade_mcp_server.exceptions import TransportError
from arcade_mcp_server.session import ServerSession
logger = logging.getLogger("arcade.mcp.transports.stdio")
class StdioWriteStream:
"""Write stream implementation for stdio."""
def __init__(self, write_queue: queue.Queue[str | None]):
self.write_queue = write_queue
async def send(self, data: str) -> None:
"""Send data to stdout."""
if not data.endswith("\n"):
data += "\n"
await asyncio.to_thread(self.write_queue.put, data)
class StdioReadStream:
"""Read stream implementation for stdio."""
def __init__(self, read_queue: queue.Queue[str | None]):
self.read_queue = read_queue
self._running = True
def stop(self) -> None:
"""Stop the read stream."""
self._running = False
def __aiter__(self) -> AsyncIterator[str]:
return self
async def __anext__(self) -> str:
if not self._running:
raise StopAsyncIteration
try:
line = await asyncio.to_thread(self.read_queue.get)
except asyncio.CancelledError:
raise StopAsyncIteration
except Exception as e:
logger.exception("Error reading from stdin")
raise TransportError(f"Read error: {e}") from e
if line is None or not self._running:
raise StopAsyncIteration
return line
class StdioTransport:
"""
Stdio transport implementation for stdio communication.
This transport uses stdin/stdout for MCP communication,
suitable for command-line tools and scripts.
"""
def __init__(self, name: str = "stdio"):
"""Initialize stdio transport."""
self.name = name
self.read_queue: queue.Queue[str | None] = queue.Queue()
self.write_queue: queue.Queue[str | None] = queue.Queue()
self.reader_thread: threading.Thread | None = None
self.writer_thread: threading.Thread | None = None
self._shutdown_event = asyncio.Event()
self._running = False
self._sessions: dict[str, ServerSession] = {}
async def start(self) -> None:
"""Start the transport."""
# Component start is handled here directly
# Start I/O threads
self._running = True
self.reader_thread = threading.Thread(
target=self._reader_loop,
daemon=True,
name=f"{self.name}-reader",
)
self.writer_thread = threading.Thread(
target=self._writer_loop,
daemon=True,
name=f"{self.name}-writer",
)
self.reader_thread.start()
self.writer_thread.start()
# Set up signal handlers
loop = asyncio.get_running_loop()
for sig in (signal.SIGINT, signal.SIGTERM):
try:
loop.add_signal_handler(sig, lambda: asyncio.create_task(self.stop()))
except NotImplementedError:
# Windows doesn't support POSIX signals
if sys.platform == "win32":
logger.warning("Signal handling not fully supported on Windows")
else:
logger.warning(f"Failed to set up signal handler for {sig}")
async def stop(self) -> None:
"""Stop the transport."""
if not self._running:
return
logger.info("Stopping stdio transport")
self._running = False
# Signal threads to stop
self.read_queue.put(None)
self.write_queue.put(None)
# Wait for threads to finish
if self.reader_thread and self.reader_thread.is_alive():
self.reader_thread.join(timeout=1.0)
if self.writer_thread and self.writer_thread.is_alive():
self.writer_thread.join(timeout=1.0)
# Set shutdown event
self._shutdown_event.set()
def _reader_loop(self) -> None:
"""Reader thread loop."""
try:
for line in sys.stdin:
if not self._running:
break
self.read_queue.put(line.strip())
except Exception:
logger.exception("Error in reader thread")
finally:
self.read_queue.put(None) # Signal EOF
def _writer_loop(self) -> None:
"""Writer thread loop."""
try:
while self._running:
msg = self.write_queue.get()
if msg is None:
break
sys.stdout.write(msg)
sys.stdout.flush()
except Exception:
logger.exception("Error in writer thread")
@contextlib.asynccontextmanager
async def connect_session(self, **options: Any) -> AsyncIterator[ServerSession]:
"""
Create a stdio session.
Since stdio is inherently single-session, this will fail
if a session is already active.
"""
# Check if already have a session
sessions = await self.list_sessions()
if sessions:
raise TransportError("Stdio transport only supports one session")
# Create session
session_id = str(uuid.uuid4())
read_stream = StdioReadStream(self.read_queue)
write_stream = StdioWriteStream(self.write_queue)
session = ServerSession(
server=None, # set by the caller using run_connection; not used here
session_id=session_id,
read_stream=read_stream,
write_stream=write_stream,
init_options=options,
stateless=True,
)
# Register session
await self.register_session(session)
try:
yield session
finally:
# Cleanup
read_stream.stop()
await self.unregister_session(session_id)
async def wait_for_shutdown(self) -> None:
"""Wait for the transport to shut down."""
await self._shutdown_event.wait()
# Minimal session registry to support connect_session lifecycle
async def list_sessions(self) -> list[str]:
return list(self._sessions.keys())
async def register_session(self, session: ServerSession) -> None:
self._sessions[session.session_id] = session
async def unregister_session(self, session_id: str) -> None:
self._sessions.pop(session_id, None)