arcade-mcp/libs/arcade-serve
Eric Gustin 5602578b2f
Worker Stability (#688)
This PR does three things:
1. Executes synchronous tool calls in thread pool allowing for up to 4 +
# of CPUs executions in parallel.
2. Makes force quitting via double SIGINT/SIGTERM possible and via
single SIGINT/SIGTERM + graceful shutdown timeout expiry possible, even
if there are active connections.
3. Sets `timeout_graceful_shutdown` to
`ARCADE_UVICORN_TIMEOUT_GRACEFUL_SHUTDOWN` env var if set, else defaults
to 15.
4. Disable the worker health check span to reduce noise

Tradeoffs:
Since this PR introduces executing synchronous tools via `await
asyncio.to_thread(func, **func_args)`, this means that there is no way
for the thread to be killed until it finishes. The ramifications of this
is that the force quitting logic that is also implemented in this PR has
to be very harsh `os._exit(1)` just in case there is a sync tool
actively executing. This means that `MCPApp` teardown logic will not
execute when force quitting is required. Although this was already the
case because we weren't previously able to force quit! This tradeoff is
justified for now since "parallel" tool executions will relieve us of
many worker timeouts that we are seeing in prod.

Future work:
Minimize/eliminate the need for `os._exit(1)` such that `MCPApp`
teardown logic will always execute, even when force quitting. The
solution will likely be moving away from `await asyncio.to_thread(func,
**func_args)` (while maintaining "parallelism" and then utilize the
`TaskTrackerMiddleware` introduced in this PR to cancel all of the
active HTTP requests.

Resolves PLT-713
2025-11-20 11:13:41 -08:00
..
arcade_serve Worker Stability (#688) 2025-11-20 11:13:41 -08:00
pyproject.toml Worker Stability (#688) 2025-11-20 11:13:41 -08:00
README.md PyPI release arcade-serve and arcade-tdk (#432) 2025-06-13 13:06:11 -07:00

Arcade Serve

Serving infrastructure for Arcade tools and workers.

Overview

Arcade Serve provides the infrastructure for serving Arcade tools:

  • FastAPI Worker: High-performance FastAPI-based worker implementation
  • MCP Server: Model Context Protocol server for tool integration
  • Core Abstractions: Base worker classes and components
  • Authentication: Auth utilities and routing
  • Runtime Management: Tool execution and lifecycle management

Installation

pip install arcade-serve

Usage

To add a toolkit to a hosted worker such as FastAPI, you can register them in the worker itself. This allows you to explicitly define which tools should be included on a particular worker.

Here is an example of adding the math toolkit (pip install arcade-math) to a FastAPI Worker:

import arcade_math
from fastapi import FastAPI
from arcade_tdk import Toolkit
from arcade_serve.fastapi import FastAPIWorker

app = FastAPI()

worker_secret = os.environ.get("ARCADE_WORKER_SECRET")
worker = FastAPIWorker(app, secret=worker_secret)

worker.register_toolkit(Toolkit.from_module(arcade_math))

Here is an example of adding the math toolkit (pip install arcade-math) to a MCP Worker

import arcade_math
from arcade_core.catalog import ToolCatalog
from arcade_serve.mcp.stdio import StdioServer

# 1. Create and populate the tool catalog
catalog = ToolCatalog()
catalog.add_module(arcade_math)


# 2. Main entrypoint
async def main():
    # Create the worker with the tool catalog
    worker = StdioServer(catalog)

    # Run the worker
    await worker.run()


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

License

MIT License - see LICENSE file for details.