Commit graph

65 commits

Author SHA1 Message Date
Eric Gustin
d31a81ef3f
Add background update check & notification for arcade CLI (#800)
## Summary
- On every CLI command invocation (except `update`/`upgrade`/`mcp`), a
detached subprocess checks PyPI for newer versions (throttled to once
per 4 hours) and caches the result at `~/.arcade/update_cache.json`
- On the next invocation, if a newer version is known, a yellow
one-liner notification is printed suggesting `arcade update`
- Respects `ARCADE_DISABLE_AUTOUPDATE=1` environment variable to opt out
entirely


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Adds a background PyPI version check that spawns detached subprocesses
and may print update notifications on most CLI invocations; mistakes
could impact CLI reliability or corrupt MCP stdio output (mitigated by
explicit command exclusions).
> 
> **Overview**
> Adds `arcade update` (and hidden `arcade upgrade` alias) to
self-upgrade the `arcade-mcp` CLI by detecting the original install
method (`uv tool`, `pipx`, `uv pip`, or `pip`) and running the
appropriate upgrade command.
> 
> Introduces a **throttled background update check** on most CLI
invocations: a detached subprocess queries PyPI, writes
`~/.arcade/update_cache.json`, and on subsequent runs prints a one-line
notification when a newer version is cached; this is disabled via
`ARCADE_DISABLE_AUTOUPDATE=1` and explicitly skipped for
`update`/`upgrade`/`mcp` to avoid MCP stdio output corruption.
> 
> Bumps the package version to `1.13.0`, adds a `packaging` dependency,
and includes comprehensive tests covering PyPI/yanked/prerelease
handling, install-method detection, caching, and callback integration.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2d9646ecc2211e8cfecd6e4901d14b1f5b7bb306. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 11:30:55 -07:00
Eric Gustin
9eec003c72
Add full support for MCP Resources (#803)
Resolves
https://linear.app/arcadedev/issue/TOO-590/add-resources-support-to-server-framework


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Adds new resource registration/reading semantics (including URI
templates and duplicate/multiple-match policies) and changes JSON Schema
generation for tool I/O, which may affect MCP client compatibility and
runtime behavior across servers.
> 
> **Overview**
> **Adds first-class MCP Resources support across `arcade-mcp-server`.**
`MCPApp` can now register resources at build time via
`add_resource`/`@resource` plus convenience `add_text_resource` and
`add_file_resource`, and passes these through to `MCPServer` for startup
loading (including `ResourceTemplate` URIs with `{param}` and `{param*}`
matching).
> 
> **Extends `ResourceManager` behavior.** Resource reads now coerce
handler return types (including raw `bytes` to base64
`BlobResourceContents`), support template matching with
overlap/multiple-match detection, and introduce configurable duplicate
handling policies.
> 
> **Improves tool schema + MCP Apps linking.** Tool input/output JSON
Schema generation is refactored to recursively expand nested `json`
schemas and ensure `outputSchema` is always an object (wrapping
non-object returns in a `result` property); `MCPApp` also supports
attaching arbitrary tool `_meta` extensions (e.g., `ui.resourceUri`)
applied at server start.
> 
> Adds two new example servers (`resources`, `tools_with_output_schema`)
and broad test coverage for resource templates, static/file resources,
meta extensions, and schema wrapping/recursion.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
e785bee79d74110727519b00b81dcad6e9b74212. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 15:27:57 -07:00
Eric Gustin
bbba7aec90
Update arcade new --full (#805)
Monorepo has new linting and formatting preferences. Updated the
`--full` to reflect that.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Low risk: changes only affect the generated `--full` project template
and CLI help surface (flag is now hidden), plus a patch version bump.
> 
> **Overview**
> Updates `arcade new --full` to be *internal-only* by hiding the flag
and revises the full template to match monorepo conventions
(Ruff/pre-commit versions and hook IDs, `.ruff.toml` now extends the
repo config, `pyproject.toml` formatting/booleans, adds `pytest`
`asyncio_mode`, and removes `pre-commit install` from the template
`Makefile`).
> 
> Adds a regression test ensuring generated full-template files match
these conventions, and bumps the root package version to `1.12.2`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2c1a285752d67dc4dd1aa8e0b6f25ca2f0a33fa2. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-03-27 14:22:32 -07:00
Sankara R. Avula
78c8e6fb99
feat: Add TelemetryPassbackMiddleware for serverExecutionTelemetry capability (#797)
**Implements**: [SEP-2448: server execution telemetry]
(https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2448)



**Description:**

**The Observability Gap (The Problem)**

MCP clients propagate trace context to servers, but server-side
execution remains a black box. The client sees a single tools/call or
resources/read span; everything the server does (auth checks, policy
evaluation, API calls, sub-tool invocations) is invisible. In
cross-organization deployments, clients and servers use separate
observability backends with no shared collector access, making
traditional span export useless.

<img width="1015" height="450" alt="Screenshot 2026-03-23 at 3 43 21 PM"
src="https://github.com/user-attachments/assets/58c817b5-fee6-46a3-9877-d523a25368ad"
/>


**Server Execution Telemetry (The Solution)**

Servers advertise serverExecutionTelemetry and return a curated slice of
their execution spans directly in _meta.otel of the response. Clients
ingest these verbatim OTLP spans into their own collector, stitching
server-side execution into their distributed trace; no shared
infrastructure required. The black box becomes transparent.

<img width="945" height="574" alt="Screenshot 2026-03-23 at 3 43 44 PM"
src="https://github.com/user-attachments/assets/38d97c94-aa73-4e62-9b4e-3264600e5ed0"
/>

.
**Summary:**

Implement MCP serverExecutionTelemetry capability that enables
cross-organization distributed tracing by returning server-side
OpenTelemetry spans to clients inline via _meta.otel.traces.

  Server-side (middleware):
  - TelemetryPassbackMiddleware intercepts tools/call and resources/read
- ContextVarSpanCollector isolates span collection per-request via
ContextVar
- Propagates traceparent from client request for distributed trace
stitching
- Serializes collected spans to verbatim OTLP JSON (resourceSpans
format), directly POSTable to /v1/traces
- Top-level span filtering by default; full span tree via detailed
opt-in
- Middleware advertises capabilities via get_capabilities() on the
Middleware base class
  - Provisional API: FutureWarning emitted until SEP-2448 is ratified

  Client-side (reference agent):
- LangChain ReAct agent connects to MCP server via
streamable_http_client with OAuth 2.1
  - Detects serverExecutionTelemetry capability at initialization
- Dynamically wraps discovered MCP tools with traceparent propagation
and _meta.otel span request
- Ingests returned server spans into Jaeger (OTLP JSON) and Galileo
(OTLP protobuf)
- Two-act demo: --no-passback (black box) vs default (full server-side
visibility)

  Dependencies:
  - opentelemetry-api and opentelemetry-sdk added to arcade-mcp-server

  Bump arcade-mcp-server version to 1.18.0.
2026-03-25 15:57:50 -07:00
Eric Gustin
9bbdbe2b46
Fix outputSchema to conform to MCP spec's object type requirement (#799)
When a stdio server had a tool that didn't return a dict, then:
```
  {
    "code": "invalid_value",
    "values": [
      "object"
    ],
    "path": [
      "tools",
      2,
      "outputSchema",
      "type"
    ],
    "message": "Invalid input: expected \"object\""
  }
```

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes the generated `outputSchema` shape for all non-`json` return
types by wrapping them under a `result` property, which may affect
clients/tests expecting primitive/array schemas despite being
spec-correct.
> 
> **Overview**
> Adjusts MCP tool `outputSchema` generation to **always** emit an
object schema, per the MCP spec that `structuredContent` must be a JSON
object.
> 
> `json` outputs remain a direct object schema, while primitive/array
outputs are now wrapped as `{ "type": "object", "properties": {
"result": <inner> } }` (preserving `enum`/`items`), and tests are
expanded to cover these cases. Bumps `arcade-mcp-server` version to
`1.18.0`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
7dd13bd33d6fdf6ebb778e1a3d9167ca89806f55. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-03-20 15:50:54 -07:00
jottakka
9c47f73602
[TOO-518] Enforce semver for MCPApp Versioning (#793)
Here's the PR summary:

---

## Enforce semver validation for `MCPApp` versioning

### Problem

`MCPApp.__init__` accepted any string as `version` with no validation.
Invalid versions like `"1.0.0dev"` or `"latest"` silently propagated to
the Engine, where `compareToolVersions` fell back to lexicographic
`strings.Compare` instead of `semver.Compare` — causing incorrect
ordering (e.g. `1.10.0 < 1.9.0`).

### Solution

Validate and normalize `version` at `MCPApp` instantiation time using
the same acceptance rules as Go's `golang.org/x/mod/semver v0.31.0` (the
exact version used by the Engine).

### Changes

**`arcade_mcp_server/_validation.py`** (new file)
- Shared regex constants: `SEMVER_PATTERN` (semver.org spec),
`SHORT_VERSION_PATTERN`, `MAJOR_ONLY_PATTERN`

**`arcade_mcp_server/mcp_app.py`**
- Added `_validate_version()` mirroring the existing `_validate_name()`
pattern
- Added `version` property + setter (validates on mutation too)
- `__init__` now stores `self._version` via `_validate_version()`

**`arcade_mcp_server/settings.py`**
- Added `@field_validator("version")` on `ServerSettings` — covers the
`MCP_SERVER_VERSION` env var path
- Fixed default from `"0.1.0dev"` → `"0.1.0"` (the old default was
itself invalid)

**`pyproject.toml`** — bumped `arcade-mcp-server` `1.17.4` → `1.17.5`

### Normalization pipeline

All inputs are normalized to canonical `MAJOR.MINOR.PATCH` before
storage:

| Input | Stored as |
|-------|-----------|
| `v1.0.0` | `1.0.0` |
| `1.0` / `v1.0` | `1.0.0` |
| `1` / `v1` | `1.0.0` |

### Verification

Validated against `golang.org/x/mod/semver v0.31.0` (Engine's exact
pinned version) — 40/40 accept/reject cases match. The Engine's own
`store_test.go` uses `"1.0"` and `"1.1"` as `ToolkitVersion` values,
confirming short forms are intentionally supported.

### Breaking change

Any user currently passing a non-semver version string (e.g.
`"1.0.0dev"`, `"latest"`) will get a `ValueError` on upgrade. This is
intentional — those versions were silently causing incorrect tool
ordering in the Engine.

Closes TOO-518

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Introduces stricter version validation/normalization that will raise
errors for previously-accepted non-semver inputs (including via env
vars), which may break existing consumers depending on lax version
strings.
> 
> **Overview**
> **Enforces semver for server versioning** across both `MCPApp` and
`ServerSettings`, rejecting invalid strings and normalizing accepted
inputs (e.g., stripping leading `v`, expanding `1`/`1.2` to
`1.0.0`/`1.2.0`).
> 
> Adds shared `normalize_version` logic in
`arcade_mcp_server/_validation.py`, updates `MCPApp` to validate on init
and via a new `version` property/setter, and adds a Pydantic `version`
validator so `MCP_SERVER_VERSION` is checked. Defaults are updated from
`0.1.0dev` to `0.1.0`, tests are expanded to cover accept/reject cases,
and the package version is bumped to `1.17.5`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2ceabacb25372e67eef9720b901c1ee2b214868f. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2026-03-16 16:06:25 -07:00
jottakka
3ed66e663c
Removing Flanky tests (#790)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> **Medium Risk**
> Test-only change, but it removes coverage for concurrent tool
execution over HTTP, which could let concurrency regressions slip
through unnoticed.
> 
> **Overview**
> Removes three async end-to-end integration tests in
`test_end_to_end.py` that asserted **parallel/concurrent tool
execution** timing across the HTTP `POST /mcp` JSON-RPC route, the `POST
/worker/tools/invoke` route, and a mixed MCP+Worker scenario.
> 
> No production code changes; remaining HTTP and stdio E2E coverage
stays in place, but the suite no longer validates HTTP concurrency
behavior.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
3846363488c935771e79e0bb9b946b98137fdc55. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-03-06 16:03:49 -03:00
Renato Byrro
3e9ffb6bd9
Fix deploy timeout and improve error messages (#770)
- update_deployment() was using httpx default timeout (5s) instead of
the 360s used by deploy_server_to_engine(), causing "The write operation
timed out" errors on larger packages
- Catch httpx.TimeoutException in both deploy paths with an actionable
error message that points to package size as the likely cause
- Add proper error handling (ConnectError, HTTPStatusError) and
client.close() to update_deployment(), matching
deploy_server_to_engine()
- Add unit tests covering timeout handling and timeout constant usage
2026-03-06 10:03:48 -03:00
Eric Gustin
4d48bb765d
arcade new <name> --full generates an MCPApp (#787)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> **Medium Risk**
> Moderate risk because it changes the default `arcade new --full`
scaffolding (dependencies, entry points, Makefile workflow, licensing)
and removes interactive prompts, which could break existing expectations
for generated projects.
> 
> **Overview**
> `arcade new --full` scaffolding is simplified to be non-interactive
and always generate an `arcade_<name>` package, with derived name
variants (title/hyphenated) and updated next-step instructions (`make
install/dev/test/lint`).
> 
> The full template is updated to produce a runnable `arcade-mcp-server`
`MCPApp` (`__main__.py` + `project.scripts`), switch sample
code/tests/evals to a new async Reddit example tool (auth + metadata)
using `httpx`, and refresh dev tooling (new `Makefile`, ruff config
tweaks, added pytest `conftest.py`).
> 
> Template licensing/metadata is standardized to Arcade proprietary
(removes community/official conditionals and the templated README), and
the repo version is bumped to `1.12.0`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
fd39c9ed9beba068fe85cf96979f04a31a40daa4. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-03-05 17:50:10 -08:00
jottakka
bcee0f556f
Left over fixes for Windows Papercut PR (#781)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> **Low Risk**
> Mostly CI/test and CLI output tweaks, plus a small refactor to reuse
existing subprocess termination logic; low risk with minor potential for
CI environment/version compatibility issues.
> 
> **Overview**
> Expands CI coverage by adding Python `3.13` and `3.14` to the GitHub
Actions matrices (main tests, install test, and no-auth CLI
integration), and removes a redundant editable install step in the
no-auth workflow.
> 
> Cleans up Windows subprocess handling by dropping
`arcade_cli.deploy._graceful_terminate` and calling the shared
`arcade_core.subprocess_utils.graceful_terminate_process` directly, with
corresponding test updates.
> 
> Improves `arcade new` scaffolding guidance by printing numbered “Next
steps” with explicit stdio/HTTP run options, and adds/updates CLI tests
to assert this output. Also bumps package version to `1.11.2` and
tightens pre-commit `ruff` excludes (no longer excluding `_scratch`).
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
55c2ae106f13e5657acdbebf63e00d74c171181f. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-02-26 13:24:15 -03:00
Eric Gustin
4a737b9710
Improve .env discovery (#737)
Resolves TOO-201

Documentation PR for this is here:
https://github.com/ArcadeAI/docs/pull/626


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes how environment variables/secrets are discovered and loaded,
which can subtly alter runtime behavior depending on directory structure
and existing env vars; bounded traversal and added tests reduce but
don’t eliminate this risk.
> 
> **Overview**
> **Improves `.env` discovery across the MCP server and CLI.** Adds
`find_env_file()` (bounded by the nearest `pyproject.toml` by default)
and switches settings loading, `arcade deploy`, `arcade configure` stdio
env injection, and provider API-key resolution to use it.
> 
> Updates dev reload to also watch the discovered `.env` even when it
lives outside the current working directory, adjusts `deploy --secrets
all` to only run when a `.env` was found, and moves the minimal
scaffold’s `.env.example` to the project root with updated
tests/integration checks. Version bumps align examples and top-level
deps with `arcade-mcp-server` `1.17.4` and `arcade-mcp` `1.11.2`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
40cff1738c14674ce01f09fd325ece9c874cd072. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 23:20:28 -08:00
Eric Gustin
36584942f7
Fix runtime warning (#771)
When `python -m arcade_mcp_server` was executed, we would get the
following Runtime Warning:

```
<frozen runpy>:128: RuntimeWarning: 'arcade_mcp_server.__main__' found in sys.modules after import of package 'arcade_mcp_server', but prior to execution of 'arcade_mcp_server.__main__'; this may result in unpredictable behaviour
```

This PR resolves this. This PR is mainly just moving existing functions
to new locations; a refactor


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Primarily a module-organization refactor with minimal behavior change;
main risk is import-path regressions for internal callers and stdio/CLI
startup wiring.
> 
> **Overview**
> Fixes the `python -m arcade_mcp_server` runtime warning by refactoring
`arcade_mcp_server.__main__` to be a thin CLI entrypoint and moving its
reusable logic into import-safe modules.
> 
> Extracts stdio execution and tool discovery into a new
`arcade_mcp_server.stdio_runner` (`initialize_tool_catalog`,
`run_stdio_server`) and moves `setup_logging` into `logging_utils`,
updating `MCPApp`, the FastAPI `worker`, and tests to import from the
new locations. Bumps package version to `1.17.3`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
210475acea7c5df44fc66be2bde06f1f0c806c4e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-02-25 09:55:37 -08:00
Eric Gustin
25267ab6ee
JSON-safety validation for ToolMetadata.extras (#773)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> **Medium Risk**
> Tightens validation on tool metadata, which may break existing tools
that relied on non-JSON-serializable `extras` values or keys; changes
are localized and well-covered by tests.
> 
> **Overview**
> Adds **JSON-safety enforcement** for `ToolMetadata.extras`: top-level
keys must be strings at model construction, and `validate_for_tool()`
now recursively rejects non-JSON-native values (including non-finite
floats) with path-rich `ToolDefinitionError` messages.
> 
> Expands tests to cover valid/invalid nested `extras` cases and
error-message quality, and bumps `arcade-core` version to `4.5.0`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2bab0db3c17f0ddb97868764d10494da543b39e5. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-02-25 09:48:04 -08:00
jottakka
fe8ddfd500
[TOO-326] Windows papercuts (#768)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> **Medium Risk**
> Touches authentication/login flow, credentials-file permissions, and
subprocess lifecycle behavior across platforms; while mostly defensive,
regressions could impact login or process management on Windows/macOS
runners.
> 
> **Overview**
> Improves Windows/cross-platform reliability across the CLI and MCP
server: OAuth login now binds the callback server to `127.0.0.1`, avoids
slow loopback reverse-DNS, adds a configurable callback timeout
(`--timeout` + env default), and opens URLs via a Windows-friendly
`_open_browser` to avoid flashing console windows.
> 
> Centralizes CLI output via a shared `console` that forces UTF-8 on
Windows, standardizes UTF-8 file reads/writes throughout, tightens
credentials-file permissions on Windows using `icacls`, and adds shared
Windows subprocess helpers for **no-window** process creation and
graceful termination (used by `deploy`, MCP reload, and usage-tracking
worker).
> 
> Updates client configuration UX/robustness (Windows AppData resolution
via `platformdirs`, Cursor config path fallbacks + compatibility writes,
overwrite warnings, absolute `uv` path for GUI clients, safer path
display) and improves `deploy` child-process handling to avoid
pipe-buffer deadlocks while giving better debug-aware error messages.
> 
> Expands CI to run tests on Linux/Windows/macOS, adds a no-auth CLI
integration workflow, disables usage tracking in toolkits CI, and adds
extensive regression tests for Windows signals, subprocess cleanup,
UTF-8, and config-path edge cases; bumps `arcade-core` to `4.4.2` and
`arcade-mcp-server` to `1.17.2` (with updated dependency pin).
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
0fabd8ca1cd647039ba6ddbdf3f7809c330bab9e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-02-25 13:18:16 -03:00
Evan Tahler
7a28e7f988
Remove secret hints from CLI (#774)
## Summary
- Removes the `Hint` column from `arcade secret list` table output —
secret hints were removed from the API in
[monorepo#322](https://github.com/ArcadeAI/monorepo/pull/322), causing a
`KeyError: 'hint'` crash
- Adds a test for `print_secret_table` with populated secrets (verifying
no hint column)
- Bumps version from 1.10.0 → 1.11.0

Closes TOO-444

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 08:31:18 -08:00
Eric Gustin
a918eef037
Add Tool Metadata (#766) 2026-02-17 14:31:45 -08:00
jottakka
7472b18106
Fixing bug with multiple providers + stats for multiple runs (#752)
@EricGustin you can use this cli command:
```
uv run arcade evals mcp_building_evals_results/eval_toolkit_iteration_dict.py \
    -p openai:gpt-4o,gpt-4o-mini \
    -p anthropic:claude-sonnet-4-20250514 \
    -k openai:$OPENAI_API_KEY \
    -k anthropic:$ANTHROPIC_API_KEY \
    -d \
    --num-runs 3 \
    --seed random \
    --multi-run-pass-rule majority \
    --max-concurrent 6 \
    -o mcp_building_evals_results/results

```

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Touches core eval execution and all result formatters while adding new
CLI inputs and output schema (`run_stats`/`critic_stats` and capture
`runs`), so regressions could affect evaluation results and report
compatibility despite being additive and validated.
> 
> **Overview**
> Adds **multi-run evaluation support** to `arcade evals` via new flags
`--num-runs`, `--seed`, and `--multi-run-pass-rule`, with upfront
validation and plumbing through the CLI runner into eval/capture suite
execution.
> 
> Fixes provider selection UX/bug by making `--use-provider/-p`
**repeatable** (instead of a space-delimited string), updates
docs/examples accordingly, and extends capture mode to optionally record
**per-run tool calls** (`CapturedRun`) when `num_runs > 1`.
> 
> Enhances all output formatters (HTML/Markdown/Text/JSON) to
**propagate and display** per-case `run_stats` and `critic_stats`,
including new HTML UI for run tabs/cards and comparative tables showing
mean ± stddev when multi-run data is present.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2ee1654b7d1fbb9538373507355636164b16a066. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-02-09 14:25:28 -03:00
Eric Gustin
d7d765343e
Fix multiple worker log level bug (#758)
When running `arcade_mcp_server` with `workers > 1`, uvicorn spawns
worker subprocesses that directly call `create_arcade_mcp_factory()`
without going through `main()`. Since `setup_logging()` is only called
in `main()`, these subprocesses have no logging configuration, causing:

1. Standard Python logging not intercepted by Loguru
    
2. DEBUG-level logs from libraries like urllib3 appearing when OTEL is
enabled
    
3. Inconsistent log formats between main process and workers

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Touches process-wide logging initialization for uvicorn worker
subprocesses, which can affect log levels/handlers and output across the
server. Functional impact is limited to observability but could change
verbosity when OTEL or libraries emit logs.
> 
> **Overview**
> Fixes multi-worker/reload mode logging by configuring Loguru inside
`create_arcade_mcp_factory()` (using `ARCADE_MCP_DEBUG` to set `INFO` vs
`DEBUG`) so uvicorn-spawned worker subprocesses get the same
logging/interception as `main()`.
> 
> Adds regression tests that assert the factory filters DEBUG logs by
default and enables them when `ARCADE_MCP_DEBUG=true`, and bumps
`arcade-mcp-server` to `1.15.2`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
0c262eb9716ecbd589f1524842243a7aed80666e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-01-30 15:37:52 -08:00
Eric Gustin
a4160dd9fe
Four bug fixes (#754)
1. Resolves
[TOO-363](https://linear.app/arcadedev/issue/TOO-363/arcade-deploy-fails-when-additional-deps-are-added-to-the-server).
2. Resolves
[TOO-364](https://linear.app/arcadedev/issue/TOO-364/arcade-cores-tool-skip-logic-is-missing-case-for-direct-execution).
3. Resolves
[TOO-358](https://linear.app/arcadedev/issue/TOO-358/missing-evals-error-message-shows-wrong-command).
4. Resolves
[TOO-365](https://linear.app/arcadedev/issue/TOO-365/arcade-evals-unit-tests-are-hanging).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Medium risk because it changes how `arcade deploy` spawns the server
process and adjusts toolkit discovery skip logic, which can affect
deployments and tool discovery; however, the changes are small and
covered by new unit/integration tests.
> 
> **Overview**
> `arcade deploy` now starts the validation server using the project’s
`.venv` interpreter (via `find_python_interpreter`) instead of the CLI’s
own `sys.executable`, preventing missing dependency failures when the
CLI is installed in an isolated env.
> 
> `arcade-core`’s `Toolkit.tools_from_directory` skip logic is hardened
to also skip the currently executing entrypoint by module name
(`__main__.__spec__.name`) when file paths don’t match (e.g., bundled
execution). CLI error printing now escapes plain messages to avoid rich
markup issues, and `arcade-evals` lock acquisition accepts an optional
timeout default.
> 
> Adds unit tests for the new toolkit skip behavior and an integration
test that boots the MCP server via direct Python invocation to mirror
deployment behavior, and bumps `arcade-core`, `arcade-mcp-server`, and
root dependency versions accordingly.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
e7785634c231c059f2e0bd1bc73a56bd7470a494. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-01-29 15:12:06 -08:00
Eric Gustin
28c1863ee3
Support Ed25519 Algorithm (#742)
Ed25519 is needed for Arcade AS. This required migrating from
`python-jose` to `joserfc`, because `python-jose` didn't seem to support
Ed25519
2026-01-16 15:55:05 -08:00
jottakka
98fad93d21
Adding MCP Servers supports to Arcade Evals (#689)
# MCP Server Tool Evaluation Support

## Overview
Add support for evaluating tools from remote MCP servers without
requiring Python callables. Enables direct evaluation of any
MCP-compatible tool server.

## What's New

### Core Features
- **`MCPToolRegistry`**: Evaluate tools from a single MCP server
- **`CompositeMCPRegistry`**: Evaluate tools from multiple MCP servers
simultaneously
- **Automatic loaders**: `load_from_stdio()` and `load_from_http()` to
fetch tools from running servers
- **Automatic namespacing**: Tools prefixed with server name (e.g.,
`server_tool_name`)
- **Smart name resolution**: Use short names if unique, full names if
ambiguous
- **OpenAI strict mode**: Automatic schema conversion prevents parameter
hallucinations

### Usage

**Automatic Loading:**
```python
from arcade_evals import load_from_stdio, MCPToolRegistry

# Load tools automatically from MCP server
tools = load_from_stdio(["npx", "-y", "@modelcontextprotocol/server-github"])
registry = MCPToolRegistry(tools)
```

**Single MCP Server:**
```python
from arcade_evals import MCPToolRegistry, ExpectedToolCall

registry = MCPToolRegistry(mcp_tools)
suite = EvalSuite(catalog=registry)

suite.add_case(
    expected_tool_calls=[
        ExpectedToolCall(tool_name="tool_name", args={...})
    ]
)
```

**Multiple MCP Servers:**
```python
from arcade_evals import CompositeMCPRegistry, load_from_stdio

# Load from multiple servers
github_tools = load_from_stdio(["npx", "-y", "@modelcontextprotocol/server-github"])
slack_tools = load_from_stdio(["npx", "-y", "@modelcontextprotocol/server-slack"])

composite = CompositeMCPRegistry(
    tool_lists={
        "github": github_tools,
        "slack": slack_tools,
    }
)

suite = EvalSuite(catalog=composite)

suite.add_case(
    expected_tool_calls=[
        ExpectedToolCall(tool_name="github_list_issues", args={...})
    ]
)
```

## Implementation

### Files Changed
- **`libs/arcade-evals/arcade_evals/registry.py`** (NEW): Registry
abstractions and implementations
- **`libs/arcade-evals/arcade_evals/loaders.py`** (NEW): Automatic tool
loading from MCP servers
- **`libs/arcade-evals/arcade_evals/eval.py`** (MODIFIED): Enhanced
`ExpectedToolCall` and evaluation logic
- **`libs/arcade-evals/arcade_evals/__init__.py`** (MODIFIED): Exported
new registries and loaders

### Key Technical Details
- Added `BaseToolRegistry` interface for abstraction
- `MCPToolRegistry` handles single server tools
- `CompositeMCPRegistry` manages multiple servers with collision
detection
- `load_from_stdio()` and `load_from_http()` for automatic tool
discovery
- Fixed name normalization bug: MCP tools use underscores (not dots)
- Optimized tool copying: 2.5x faster via shallow copy

## Testing
-  41 tests passing (25 new tests added)
-  `test_eval_mcp_registry.py`: MCPToolRegistry functionality
-  `test_eval_composite_mcp.py`: CompositeMCPRegistry with multiple
servers
-  Verified backward compatibility with Python tools

## Backward Compatibility
 **100% backward compatible** - No breaking changes


## Breaking Changes
**None**


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds end-to-end eval UX: examples, a robust CLI runner, and rich
outputs.
> 
> - **New examples**: `eval_arcade_gateway.py`,
`eval_stdio_mcp_server.py`, `eval_http_mcp_server.py`,
`eval_comprehensive_comparison.py` with timeouts, error handling, and
track-based comparisons; detailed `README.md`
> - **CLI runner**: `arcade_cli/evals_runner.py` to execute
evals/capture in parallel with progress, error isolation, failed-only
filtering, context inclusion, and multi-provider/model support
> - **Output formatters**: `arcade_cli/formatters/` (txt, md, html,
json) for evals and capture; comparative and multi-model HTML with tabs
and context rendering
> - **Display refactor**: `display.py` now supports writing multiple
formats, failed-only disclaimers, include-context, and improved console
summaries
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
ff8acf9c34a6b61462a019a1ee9df081006517d0. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
Co-authored-by: Mateo Torres <torresmateo@gmail.com>
2026-01-07 20:26:23 -03:00
jottakka
7a06bdfa7e
PagerDuty typed OAuth object (#718)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Adds a typed `PagerDuty` OAuth2 provider and wires it through TDK/MCP
exports, with tests and coordinated version/dependency bumps.
> 
> - **Auth (core)**:
> - Add typed OAuth2 provider `PagerDuty` (`provider_id="pagerduty"`) in
`arcade_core/auth.py`.
> - **TDK & MCP Server**:
> - Re-export `PagerDuty` in `arcade_tdk/auth/__init__.py` and
`arcade_mcp_server/auth/__init__.py`.
> - **Tests**:
> - Extend `test_tool_decorator.py` and `test_create_tool_definition.py`
to cover `PagerDuty` success/failure and tool requirement generation.
> - **Versioning/Deps**:
> - Bump versions: `arcade-core`→`4.1.0`, `arcade-tdk`→`3.4.0`,
`arcade-mcp-server`→`1.14.0`, root `arcade-mcp`→`1.7.1`.
>   - Update dependency ranges to require the bumped versions.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2b60261b1962586ea58831ccb6ea66e57053ac86. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
2025-12-15 17:42:11 -03:00
Nate Barbettini
aae9b3a49c
feat: Support multiple orgs & projects in Arcade CLI (#717)
Fixes [PLT-720: Refactor CLI to support multiple orgs +
projects](https://linear.app/arcadedev/issue/PLT-720/refactor-cli-to-support-multiple-orgs-projects)

This PR removes the legacy login flow (login to get an API key) from
Arcade CLI. Believe it or not, this flow predates the ability to get an
API key from the Dashboard, or even the Dashboard itself!

Notable changes:

**Legacy handling** - When a user with an existing `credentials.yaml`
updates the CLI, they will get instructions on fixing their old
credentials:
<img width="978" height="146" alt="Screenshot 2025-12-08 at 10 10 37"
src="https://github.com/user-attachments/assets/5aeaef2c-bef7-4642-a2f7-f917b257c94b"
/>

Any commands that require login (non-public commands) will be blocked
with the above message until `arcade logout / arcade login` is performed
again.

**New login flow**

```sh
arcade login
Opening a browser to log you in...

 Logged in as nate@arcade.dev.

Active project: Nate Barbettini's organization / Default project
Run 'arcade org list' or 'arcade project list' to see available options.
```

**List and set the active organization**
```sh
arcade org list
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ Name                           ┃ ID                                   ┃ Default ┃ Active ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ Nate Barbettini's organization │ 1c64968e-fdc5-4c55-8612-2ce46cd7881b │ ✓       │ ✓      │
│ Sergio 743                     │ 1f1f6184-58dc-4bac-bdde-b9184e43fdf3 │         │        │
└────────────────────────────────┴──────────────────────────────────────┴─────────┴────────┘

Use 'arcade org set <org_id>' to switch organizations.
```
```sh
arcade org set 1c64968e-fdc5-4c55-8612-2ce46cd7881b 

✓ Switched to organization: Nate Barbettini's organization
  Active project: Default project
```

**List and set the active project**
```sh
arcade project list

Active organization: Nate Barbettini's organization
Use 'arcade org list' and 'arcade org set <org_id>' to switch organizations.

┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ Name            ┃ ID                                   ┃ Default ┃ Active ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ Default project │ 35166bf3-6e68-481e-bf16-f747fadc6c22 │ ✓       │ ✓      │
│ Second project  │ 62963205-31ea-4fda-9fc4-af10db89c06f │         │        │
└─────────────────┴──────────────────────────────────────┴─────────┴────────┘

Use 'arcade project set <project_id>' to switch projects.
```
```sh
arcade project set 35166bf3-6e68-481e-bf16-f747fadc6c22
✓ Switched to project: Default project
```

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Migrates CLI to OAuth2 (PKCE) with saved org/project context, adds
org/project commands, rewrites Engine calls to org-scoped endpoints, and
bumps core packages.
> 
> - **Auth & Config**
> - Implement OAuth2 Authorization Code + PKCE (`arcade_cli/authn.py`)
with local callback server and Jinja templates.
> - Persist tokens and active `context` (org/project) in
`credentials.yaml` via updated config models
(`arcade_core/config_model.py`).
> - Add token refresh and CLI config fetch utilities
(`arcade_core/auth_tokens.py`).
> - Detect legacy API-key credentials and block protected commands until
re-login; add `whoami` command.
> - **Org/Project Management**
> - New subcommands: `arcade org list|set`, `arcade project list|set`
(fetch via Coordinator).
> - **Engine API usage (org-scoped)**
> - Introduce org/project URL rewriting transports
(`arcade_core/network/org_transport.py`) and helpers
(`get_org_scoped_url`, `get_arcade_client`, `get_auth_headers`).
> - Update `deploy`, `server`, and `secret` commands to use Bearer
tokens and org-scoped paths; adjust log streaming/status, secrets CRUD,
and deployment workflows.
> - **CLI UX**
> - Replace legacy login URLs/constants; add success/failure HTML
templates for browser callback.
>   - Tweak `dashboard` to health-check without credentials.
>   - Usage tracking now includes `org_id`/`project_id` properties.
> - **Tests**
> - Update tests for dashboard, secrets, utils, and usage identity
(OAuth `/whoami`).
> - **Dependencies & Versions**
> - Bump packages: `arcade-core@4.0.0`, `arcade-mcp-server@1.12.0`,
`arcade-serve@3.2.0`, `arcade-tdk@3.3.0`; add `authlib`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
49702c2f74b9db15bb286d3ec71179b4e74a9134. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-11 12:58:55 -08:00
Eric Gustin
98fd13c4ed
Front-Door Auth (#696)
# Valuable references for the reviewer:
- Docs PR: https://github.com/ArcadeAI/docs/pull/583
- Implements Phase 1 of the following planning doc:
https://linear.app/arcadedev/project/arcade-mcp-supports-mcp-auth-front-door-auth-7cbaa20cb054/overview


https://github.com/user-attachments/assets/79ad43fd-f5e8-4793-a1dd-18b35acefdc3

# PR Description
Adds OAuth 2.1 Resource Server authentication to arcade-mcp-server,
enabling HTTP MCP servers to validate Bearer tokens on every request.
This unlocks tool-level authorization and secrets support for HTTP
servers.

- Multiple authorization server support
- Granular token validation options (verify_exp, verify_iat, verify_iss)
- Environment variable configuration
- OAuth discovery metadata endpoint
(/.well-known/oauth-protected-resource)
- Extracts sub claim from token as context.user_id
- Lifts transport restrictions for tools requiring auth/secrets on HTTP
when protected

```python
from arcade_mcp_server import MCPApp
from arcade_mcp_server.resource_server import ResourceServerAuth, AuthorizationServerEntry

resource_server_auth = ResourceServerAuth(
    canonical_url="http://127.0.0.1:8000/mcp",
    authorization_servers=[
        AuthorizationServerEntry(
            authorization_server_url="https://auth.example.com",
            issuer="https://auth.example.com",
            jwks_uri="https://auth.example.com/jwks",
        )
    ],
)

app = MCPApp(name="my_server", version="1.0.0", auth=resource_server_auth)
```

# Testing
Beyond the comprehensive unit tests, I also manually tested end-to-end
with WorkOS Authkit (DCR) and KeyCloak (non-DCR).

# Future Work
- CIMD support
- An `ArcadeResourceServer` to make adding front-door auth super easy
when using Arcade's Auth Server



<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds OAuth 2.1 front-door auth (JWKS validation + OAuth discovery) and
propagates user identity to tools, enabling auth/secret-requiring tools
over HTTP.
> 
> - **Authentication (Front-Door OAuth 2.1)**
> - New `resource_server` module with `ResourceServerAuth`
(multi-authorization-server, metadata) and `JWKSTokenValidator`
(JWKS-based JWT validation) plus granular validation options.
> - ASGI `ResourceServerMiddleware` validates Bearer tokens on every
HTTP request and injects `resource_owner`.
> - OAuth discovery endpoint via FastAPI router at
`/.well-known/oauth-protected-resource[/<path>]`.
> - **Integration**
> - `MCPApp`/`worker` accept `auth`/`resource_server_validator`, mount
middleware, expose discovery; logs accepted auth servers.
> - HTTP transport (`http_streamable`) carries `SessionMessage` with
`resource_owner` from request → session.
> - `Context`/`Session`/`Server` plumb `resource_owner`; `Server`
selects `user_id` preferring token `sub`.
> - **Behavior Changes**
> - HTTP transport restriction lifted for tools requiring
`authorization`/`secrets` when request is authenticated; otherwise
blocked with actionable error.
> - **Configuration**
> - Env-var based auth config via `MCP_RESOURCE_SERVER_*` in
`MCPSettings.ResourceServerSettings`; `.env` auto-load.
> - **Telemetry**
>   - Usage tracking records `resource_server_type` on server start.
> - **Examples**
> - New `examples/mcp_servers/authorization` sample server (HTTP auth,
secrets, Reddit tool) with Docker setup.
> - **Tests**
> - Extensive unit tests for validators, middleware, env config,
multi-AS, transport rules, and app integration.
> - **Version**
> - Bump `arcade-mcp-server` to `1.12.0`; minor docstring tweak in
`__init__.py`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
d1116cdcafb0c7cb8f91e66682eb1fbae380da31. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->





Resolves TOO-152
2025-12-11 12:51:20 -08:00
Evan Tahler
0fc9e21308
Improve error messages with fix instructions (#713)
Improve user-facing error messages to provide actionable fix
instructions, enhancing developer experience and reducing support
queries.

---
Linear Issue:
[TOO-199](https://linear.app/arcadedev/issue/TOO-199/audit-error-messages-for-actionable-fix-instructions)

<a
href="https://cursor.com/background-agent?bcId=bc-e764f9a0-3581-4ced-b34a-2c48f3df1021"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-e764f9a0-3581-4ced-b34a-2c48f3df1021"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Enhances MCP server/session error responses with clear, actionable
guidance across JSON-RPC, tools, and resources; updates tests to assert
new messages.
> 
> - **Server (`arcade_mcp_server/server.py`)**
> - **Actionable JSON-RPC errors**: Rich messages for `Invalid request`,
`Not initialized`, `Method not found`, and internal errors with
troubleshooting steps.
>   - **Tools**:
> - `tools/list`/`tools/call`: Improved internal error messages;
user-facing guidance on failures.
>     - Unknown tool: returns detailed fix instructions.
> - Transport restrictions: explicit "Unsupported transport" guidance
for HTTP vs `stdio` with docs link.
> - Auth flow: messages for missing API key, pending authorization (with
`authorization_url`), and authorization errors; includes next steps.
> - Secrets: clear "Missing secret(s)" with `.env`/env-var setup
instructions.
>   - **Resources/Prompts**:
> - `resources/list`, `resources/templates/list`, `resources/read`,
`prompts/list`, `prompts/get`: Detailed failure and not-found messages
with guidance.
> - **Session (`arcade_mcp_server/session.py`)**
> - Enhanced internal error response formatting with troubleshooting
steps.
> - **Tests (`libs/tests/arcade_mcp_server/test_server.py`)**
> - Updated assertions to match new, descriptive messages (e.g.,
"Authorization required", "Missing Arcade API key", "Unsupported
transport").
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
97a6db4ec80a1ea9597f3364b6325d47948c94e0. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2025-12-10 10:16:38 -08:00
Evan Tahler
65acf41b11
Add startup warnings for missing secrets (#712)
Add startup warnings for missing tool secrets to provide faster feedback
on configuration issues.

---
Linear Issue:
[TOO-198](https://linear.app/arcadedev/issue/TOO-198/add-startup-warnings-for-missing-tool-secrets)

<a
href="https://cursor.com/background-agent?bcId=bc-203d1b6a-80a7-4933-b3ff-b3a9220b5809"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-203d1b6a-80a7-4933-b3ff-b3a9220b5809"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Eric Gustin <eric@arcade.dev>
2025-12-05 13:39:04 -08:00
jottakka
178c194ef2
[TOO-192] Update tdk to add figma typed oauth (#711)
Closes TOO-192

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds a Figma OAuth2 auth provider and wires it through TDK and MCP
server, with tests updated and package versions bumped.
> 
> - **Auth**:
> - Add `Figma` OAuth2 provider in
`libs/arcade-core/arcade_core/auth.py`.
> - **Exports**:
> - Expose `Figma` in
`libs/arcade-mcp-server/arcade_mcp_server/auth/__init__.py` and
`libs/arcade-tdk/arcade_tdk/auth/__init__.py` (`__all__`).
> - **Tests**:
> - Add Figma auth requirement test case in
`libs/tests/tool/test_create_tool_definition.py` and import `Figma`.
> - **Versioning**:
>   - Bump `arcade-mcp-server` to `1.10.2` and `arcade-tdk` to `3.2.0`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2bacfdc5695b3e7fc5e4532dbd360c3b2263130e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
2025-12-05 16:38:12 -03:00
Evan Tahler
e75eeb7e77
Handle client disconnect for large payloads (#701)
Catch `ClientDisconnect` in FastAPI worker to return HTTP 499 for large
payloads and reduce noisy error logs.

---
Linear Issue:
[TOO-189](https://linear.app/arcadedev/issue/TOO-189/catch-clientdisconnect-and-return-499-for-large-payloads)

<a
href="https://cursor.com/background-agent?bcId=bc-f777f89b-d2bc-4c0c-bcb1-b76fcf601e05"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-f777f89b-d2bc-4c0c-bcb1-b76fcf601e05"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Eric Gustin <eric@arcade.dev>
2025-12-02 13:44:34 -08:00
jottakka
2389d89471
[TOO-145]GQL Error Adaptor (#692)
# 🎫
[TOO-145](https://linear.app/arcadedev/issue/TOO-145/adding-gql-error-interceptor-in-tdk)
- add GraphQL-specific error adapter, expose it, and slot it ahead of
HTTP fallback
- map gql query/transport errors (incl. rate-limit handoff) without
logging sensitive data
- cover adapter + tool chain behavior with new unit tests (pathload,
dedupe, edge cases)



closes TOO-145

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
2025-11-25 23:00:26 -03:00
Sterling Dreyer
6d8d71b9ac
Exclude send/receive spans (#698) 2025-11-25 14:38:16 -08:00
Sterling Dreyer
0af13aeab5
Exclude health span from telemetry (#697) 2025-11-25 13:03:13 -08:00
Eric Gustin
4f2c6abe36
Update OTEL resource attributes (#695)
Co-authored-by: Wils Dawson <wils@arcade-ai.com>
2025-11-24 18:13:28 -08:00
Eric Gustin
7fb097f20f
Use monkeypatch for tests that use ARCADE_WORKER_SECRET (#694)
Reverts the updates to unit tests in
https://github.com/ArcadeAI/arcade-mcp/pull/691 and replaces with
monkeypatch. They inadvertently changed global process state during the
test run causing failure of post-merge and failure of PyPI publish. See
https://github.com/ArcadeAI/arcade-mcp/actions/runs/19651637906/job/56283833231
to see what failed
2025-11-24 17:22:17 -08:00
Eric Gustin
44660d18ce
Only serve worker endpoints if secret is set (#691)
Default to `ARCADE_WORKER_SECRET` being unset. This env var must be
explicitly set now. Once it is set, the `worker/` endpoints will be
served.
2025-11-24 14:39:14 -08:00
Eric Gustin
5602578b2f
Worker Stability (#688)
This PR does three things:
1. Executes synchronous tool calls in thread pool allowing for up to 4 +
# of CPUs executions in parallel.
2. Makes force quitting via double SIGINT/SIGTERM possible and via
single SIGINT/SIGTERM + graceful shutdown timeout expiry possible, even
if there are active connections.
3. Sets `timeout_graceful_shutdown` to
`ARCADE_UVICORN_TIMEOUT_GRACEFUL_SHUTDOWN` env var if set, else defaults
to 15.
4. Disable the worker health check span to reduce noise

Tradeoffs:
Since this PR introduces executing synchronous tools via `await
asyncio.to_thread(func, **func_args)`, this means that there is no way
for the thread to be killed until it finishes. The ramifications of this
is that the force quitting logic that is also implemented in this PR has
to be very harsh `os._exit(1)` just in case there is a sync tool
actively executing. This means that `MCPApp` teardown logic will not
execute when force quitting is required. Although this was already the
case because we weren't previously able to force quit! This tradeoff is
justified for now since "parallel" tool executions will relieve us of
many worker timeouts that we are seeing in prod.

Future work:
Minimize/eliminate the need for `os._exit(1)` such that `MCPApp`
teardown logic will always execute, even when force quitting. The
solution will likely be moving away from `await asyncio.to_thread(func,
**func_args)` (while maintaining "parallelism" and then utilize the
`TaskTrackerMiddleware` introduced in this PR to cancel all of the
active HTTP requests.

Resolves PLT-713
2025-11-20 11:13:41 -08:00
jottakka
b4720c2988
Fixing 403 error always being RateLimiting (#685)
Reponse 403 was returning RateLimiting all the time, but it was due only
checking if rate limiting header exists, but it should be checked if it
is 0 also.

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
2025-11-13 22:20:32 -03:00
Eric Gustin
c15c07e12f
Better Handling of MCP-specific Context usage for managed servers (#679)
Since servers managed by Arcade use the `/worker` routes under the hood,
tools that use MCP-specific properties of `Context` will fail.

This PR helps reduce the 'blast radius' of the above fact. For
properties that were deemed 'non-critical' to the execution of a
deployed tool, we simply no-op. For properties that were deemed
'critical' to the execution of a deployed tool, we raise an error that
informs the caller that the feature is not supported for Arcade managed
servers.

- Non-critical property: A context property that returns None
- Critical property: A context property that may return something that
could be necessary for a tool execution to succeed.
2025-11-07 10:26:56 -08:00
Eric Gustin
d89b3a53d4
Fix: Skip currently executing file during tool discovery (#668)
### The Bug:
When an entrypoint file imports its parent package and
calls add_tools_from_module() on that package, and the same entrypoint
file also defines tools using @app.tool or @tool decorators, then the
server fails to start with an `AttributeError`. This is because the
tools would be discovered via AST parsing, but those tools weren't added
to the module's namespace yet because the file is still executing.

For example, this would fail on startup:
```py
#!/usr/bin/env python3
"""local_filesystem MCP server"""

import sys
from typing import Annotated

from arcade_mcp_server import MCPApp

import local_filesystem

app = MCPApp(name="eric_server", version="1.0.0", log_level="DEBUG")


app.add_tools_from_module(local_filesystem)


@app.tool
def eric(name: Annotated[str, "The name of the person to greet"]) -> str:
    """Greet a person by name."""
    return "return"


if __name__ == "__main__":
    transport = sys.argv[1] if len(sys.argv) > 1 else "stdio"

    app.run(transport="http", host="127.0.0.1", port=8074)
```

### The fix:
Skip the entrypoint file. This means that any tool defined inside of the
entrypoint file must be added via MCPApp.add_tool(...) or instead use
the recommended @app.tool.
2025-11-03 11:26:51 -08:00
Eric Gustin
4ca824cf8f
Improve arcade deploy CLI Command (#634)
Also fleshed out `arcade server` commands and MCPApp.name validation.

Example of output of `arcade deploy`:
<img width="2112" height="1320" alt="image"
src="https://github.com/user-attachments/assets/51fd3dd9-0ff1-442c-a9bb-1dbcd7337e7a"
/>
2025-11-03 11:19:04 -08:00
Eric Gustin
e727af3a21
Fix MCP capabilities, examples, tests, and more (#657)
# PR Description
Consider this PR the result of a full pass through of this repository.
## Add helper for adding tools to an `MCPApp`
You can now add all of the tools in a module to an `MCPApp` via
`app.add_tools_from_module(...)`
## Edit what `arcade new` generates
First, I updated the backend to use hatchling.

Second, the structure generated before this PR was simple, but did not
create a proper Python module.
This hindered developers in the following ways:
1. Difficult to add the tools in your server to an evaluation suite
2. Difficult to add more than one tool to an MCPApp at a time
3. All other niceties that come with being able to import modules
```
# Before
server/
├── .env.example
├── server.py
└── pyproject.toml
```
This PR updates the structure generated such that a valid Python module
is generated:
```
# After 
server/
├── pyproject.toml
└── src/
    └── server/
        ├── __init__.py
        ├── .env.example
        └── server.py
```
## Fix Tool Chaining
`self._ctx.server.executor.run(...)` was being called, but `MCPServer`
does not have an instance of `ToolExecutor` (and it's not intended to be
an instance anyways). I updated `Tool.call_raw` to pass the programmatic
tool call through the `MCPServer._handle_call_tool`. This means that the
programmatic tool calls now go through the same steps that a typical
tool call (initiated by the MCP client) would.

This means that **toolA**, which specifies **requirementsA**, is
permitted to call **toolB**, which specifies **requirementsB**, without
needing to explicitly declare or satisfy **requirementsB**. I believe
this is acceptable because the secrets and/or auth token associated with
**toolB's** `Context` are not exposed to **toolA**, and the secrets
and/or auth token associated with **toolA's** `Context` are not exposed
to **toolB**.

## Fix User Elicitation
1. The read & write streams were created with a maximum queue size of 0.
I increased this to 100.
2. I updated `ServerSession`'s run loop to both read messages from the
stream & process them concurrently. This enables server initiated
requests (like user elicitation and progress reporting) to be handled
while tools are being executed. Otherwise, the server initiated requests
would wait for the tool to finish executing and the tool execution would
wait for the server initiated request to finish.
3. 
## Fix Progress Reporting
Progress tokens sent by the client were not being stored. Therefore
there was no way to notify a client with progress updates. I am now
storing the `progressToken`, along with other `_meta` sent from the
client, in the `ServerSession`'s `_request_meta`. I am setting
`_request_meta` whenever the `MCPServer` is handling an incoming message
from a client.

## Fix handling of server names with spaces
Before: 
Server name: "The simple server name"
Tool name: whisper_secret
Name seen by client: "The_simple_server_name_WhisperSecret"

After
Server name: "The simple server name"
Tool name: whisper_secret
Name seen by client: "TheSimpleServerName_WhisperSecret"

## Add Integration Tests
The stdio integration test is much more comprehensive than the http
integration test. These tests will let me sleep a bit more at night

## Add Example MCP Servers
Example servers for sampling, user-elicitation, progress reporting,
logging, tool chaining, combining prebuilt tools with custom tools, tool
secrets, tool auth, evaluations, and more!

## Add Docker template
Added a Docker template for running an MCP server in Docker (and removed
the old docker stuff)
2025-10-30 11:59:00 -07:00
Eric Gustin
49e53d2b33
Server start events (#635)
1. Refactored the core usage logic from `arcade_cli` to `arcade_core`
2. Add "MCP server started" event

As always, opt out by setting `ARCADE_USAGE_TRACKING` to 0.
2025-10-22 16:14:52 -07:00
Eric Gustin
66a126bba5
Disallow executing auth/secret tools for unauthenticated servers using HTTP transport (#641)
## PR Description
This PR tackles 3 things:
1. At tool execution runtime, blocks local HTTP servers from executing
tools that have `requires_auth` or `requires_secrets`
2. Make `stdio` the default transport in various locations
3. Improve the `arcade configure` CLI command


<img width="1408" height="1194" alt="image"
src="https://github.com/user-attachments/assets/badf1b55-ec7d-4741-89f5-4b5fee294890"
/>
<img width="3034" height="906" alt="image"
src="https://github.com/user-attachments/assets/aea528c5-4ea6-4eed-b5d7-f946626e58a7"
/>

---------

Co-authored-by: Evan Tahler <evantahler@gmail.com>
2025-10-22 13:14:46 -07:00
Renato Byrro
4dfd0522a6
Remove arcade docs CLI command - moved to the 'ArcadeAI/docs' repo (#642) 2025-10-22 14:04:09 -03:00
Eric Gustin
19bbaddf75
Reload for MCPApp (#622)
Previously, MCPApp did not truly have reload capabilities. Instead, if
`reload=True`, then under the hood we would just change over to the
module execution code path (e.g., `arcade mcp`, or `python -m
arcade_mcp_server`). This was bad because custom `MCPApp` startup code
was not being executed and tools that were not added to `MCPApp`'s
catalog were being discovered and added to the server.

`MCPApp` now contains its own custom reload logic. It doesn't use
uvicorn's reload because uvicorn's discovery & factory pattern wasn't
the best fit for `MCPApp`'s self-contained pattern.

Now when `MCPApp.run(reload=True)` is called, `MCPApp` becomes the
parent process that manages reload itself.
2025-10-17 17:38:11 -07:00
Eric Gustin
a8fc6691e7
arcade deploy for MCP Servers (#618)
Blocked by https://github.com/ArcadeAI/arcade-mcp/pull/614 (and the
reason for failing tests)

# PR Description
`arcade deploy` will deploy your local MCP server to Arcade. `arcade
deploy` should be executed at the root of your MCP Server package.
Before deploying, the command runs your server locally to ensure your
project is setup correctly and the server runs properly. `arcade deploy`
assumes your entrypoint file will execute `MCPApp.run` when the file is
invoked directly. This means you must either have an `if __name__ ==
"__main__" block that contains `MCPApp.run`, or `MCPApp.run` should be
top-level code (unindented living directly in the body of the file).

<img width="3318" height="594" alt="image"
src="https://github.com/user-attachments/assets/8249843e-6f9d-4d01-854d-356b0aae5055"
/>

<img width="1662" height="1056" alt="image"
src="https://github.com/user-attachments/assets/f44951f2-2718-4799-aecc-0e22c1b951b8"
/>
2025-10-16 09:00:10 -07:00
Eric Gustin
668d674995
Rename _meta requirements field to arcade_requirements (#616)
Also we are now excluding None in the model dump
2025-10-14 19:01:05 -07:00
Eric Gustin
3d2665d36c
Rename some 'toolkit' references to 'server' (#624)
There are many more instances of toolkit within this repo, but the goal
of this PR is to get rid of user facing references as much as possible.

---------

Co-authored-by: Nate Barbettini <nate@arcade.dev>
2025-10-14 18:42:27 -07:00
Eric Gustin
83c0eeab2b
Fix server info bug (#614)
Name, title, version, etc. for an `MCPApp` were being overwritten by its
internal `MCPServer`.

⚠️ this is blocking `arcade deploy` from working
2025-10-13 13:04:18 -07:00
Eric Gustin
805ad2d888
Add envvar overrides for mcp app (#606)
Needed for deployed MCP servers so that the Engine can override port,
host, and transport.
2025-10-07 12:43:59 -07:00
Eric Gustin
a84ad07b96
Don't send events when running CLI commands in unit tests (#600)
The `arcade dashboard` command was quite popular for the last couple
hours
2025-10-03 16:29:53 -07:00