Commit graph

50 commits

Author SHA1 Message Date
Eric Gustin
7c448aaf2e
Fix PostHog dependency issue (#740)
The 'MCP server started' events would fail to send to posthog if the CLI
was not installed. This PR fixes this by moving PostHog from being a
dependency of the CLI to a dependency of arcade-core.

<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Aligns versions and dependency ranges across the CLI and server
packages; removes an unnecessary dependency.
> 
> - Bump `arcade-mcp-server` to `1.14.2` and `arcade-mcp` to `1.8.1`
> - Update `arcade-core` constraint to `>=4.2.1,<5.0.0`; CLI now
requires `arcade-mcp-server>=1.14.2`
> - Remove `posthog` from CLI dependencies
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
69f8bb397737d4c01f57630863762109819dbc4f. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-01-12 10:48:47 -08:00
jottakka
98fad93d21
Adding MCP Servers supports to Arcade Evals (#689)
# MCP Server Tool Evaluation Support

## Overview
Add support for evaluating tools from remote MCP servers without
requiring Python callables. Enables direct evaluation of any
MCP-compatible tool server.

## What's New

### Core Features
- **`MCPToolRegistry`**: Evaluate tools from a single MCP server
- **`CompositeMCPRegistry`**: Evaluate tools from multiple MCP servers
simultaneously
- **Automatic loaders**: `load_from_stdio()` and `load_from_http()` to
fetch tools from running servers
- **Automatic namespacing**: Tools prefixed with server name (e.g.,
`server_tool_name`)
- **Smart name resolution**: Use short names if unique, full names if
ambiguous
- **OpenAI strict mode**: Automatic schema conversion prevents parameter
hallucinations

### Usage

**Automatic Loading:**
```python
from arcade_evals import load_from_stdio, MCPToolRegistry

# Load tools automatically from MCP server
tools = load_from_stdio(["npx", "-y", "@modelcontextprotocol/server-github"])
registry = MCPToolRegistry(tools)
```

**Single MCP Server:**
```python
from arcade_evals import MCPToolRegistry, ExpectedToolCall

registry = MCPToolRegistry(mcp_tools)
suite = EvalSuite(catalog=registry)

suite.add_case(
    expected_tool_calls=[
        ExpectedToolCall(tool_name="tool_name", args={...})
    ]
)
```

**Multiple MCP Servers:**
```python
from arcade_evals import CompositeMCPRegistry, load_from_stdio

# Load from multiple servers
github_tools = load_from_stdio(["npx", "-y", "@modelcontextprotocol/server-github"])
slack_tools = load_from_stdio(["npx", "-y", "@modelcontextprotocol/server-slack"])

composite = CompositeMCPRegistry(
    tool_lists={
        "github": github_tools,
        "slack": slack_tools,
    }
)

suite = EvalSuite(catalog=composite)

suite.add_case(
    expected_tool_calls=[
        ExpectedToolCall(tool_name="github_list_issues", args={...})
    ]
)
```

## Implementation

### Files Changed
- **`libs/arcade-evals/arcade_evals/registry.py`** (NEW): Registry
abstractions and implementations
- **`libs/arcade-evals/arcade_evals/loaders.py`** (NEW): Automatic tool
loading from MCP servers
- **`libs/arcade-evals/arcade_evals/eval.py`** (MODIFIED): Enhanced
`ExpectedToolCall` and evaluation logic
- **`libs/arcade-evals/arcade_evals/__init__.py`** (MODIFIED): Exported
new registries and loaders

### Key Technical Details
- Added `BaseToolRegistry` interface for abstraction
- `MCPToolRegistry` handles single server tools
- `CompositeMCPRegistry` manages multiple servers with collision
detection
- `load_from_stdio()` and `load_from_http()` for automatic tool
discovery
- Fixed name normalization bug: MCP tools use underscores (not dots)
- Optimized tool copying: 2.5x faster via shallow copy

## Testing
-  41 tests passing (25 new tests added)
-  `test_eval_mcp_registry.py`: MCPToolRegistry functionality
-  `test_eval_composite_mcp.py`: CompositeMCPRegistry with multiple
servers
-  Verified backward compatibility with Python tools

## Backward Compatibility
 **100% backward compatible** - No breaking changes


## Breaking Changes
**None**


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds end-to-end eval UX: examples, a robust CLI runner, and rich
outputs.
> 
> - **New examples**: `eval_arcade_gateway.py`,
`eval_stdio_mcp_server.py`, `eval_http_mcp_server.py`,
`eval_comprehensive_comparison.py` with timeouts, error handling, and
track-based comparisons; detailed `README.md`
> - **CLI runner**: `arcade_cli/evals_runner.py` to execute
evals/capture in parallel with progress, error isolation, failed-only
filtering, context inclusion, and multi-provider/model support
> - **Output formatters**: `arcade_cli/formatters/` (txt, md, html,
json) for evals and capture; comparative and multi-model HTML with tabs
and context rendering
> - **Display refactor**: `display.py` now supports writing multiple
formats, failed-only disclaimers, include-context, and improved console
summaries
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
ff8acf9c34a6b61462a019a1ee9df081006517d0. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
Co-authored-by: Mateo Torres <torresmateo@gmail.com>
2026-01-07 20:26:23 -03:00
Eric Gustin
25309c4e15
Fix broken links (#738)
https://github.com/ArcadeAI/docs/pull/622 moved a lot of files to new
URLs

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Updates references to Arcade docs after site restructure and bumps
package versions.
> 
> - Update docs URLs in `README.md`, `SECURITY.md`, contrib READMEs
(CrewAI, LangChain), and CLI template README to new `/en/...` paths
> - Update `documentation_url` in `arcade_mcp_server/server.py` error
message to the new "compare server types" doc
> - Bump versions: `arcade-mcp-server` to `1.14.1` and root `arcade-mcp`
to `1.7.2`
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
673b1ee7c2e5be6885ffd64914e7600b4685aaac. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-01-05 13:27:16 -08:00
jottakka
7a06bdfa7e
PagerDuty typed OAuth object (#718)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Adds a typed `PagerDuty` OAuth2 provider and wires it through TDK/MCP
exports, with tests and coordinated version/dependency bumps.
> 
> - **Auth (core)**:
> - Add typed OAuth2 provider `PagerDuty` (`provider_id="pagerduty"`) in
`arcade_core/auth.py`.
> - **TDK & MCP Server**:
> - Re-export `PagerDuty` in `arcade_tdk/auth/__init__.py` and
`arcade_mcp_server/auth/__init__.py`.
> - **Tests**:
> - Extend `test_tool_decorator.py` and `test_create_tool_definition.py`
to cover `PagerDuty` success/failure and tool requirement generation.
> - **Versioning/Deps**:
> - Bump versions: `arcade-core`→`4.1.0`, `arcade-tdk`→`3.4.0`,
`arcade-mcp-server`→`1.14.0`, root `arcade-mcp`→`1.7.1`.
>   - Update dependency ranges to require the bumped versions.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2b60261b1962586ea58831ccb6ea66e57053ac86. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Francisco Liberal <francisco@arcade.dev>
2025-12-15 17:42:11 -03:00
Evan Tahler
0fb3b75b8b
Dependency conflict rich library (#729)
Update rich dependency to resolve conflict with `strands-agents-tools`
and bump `arcade-mcp` version.

The previous exact pin `rich==13.9.4` conflicted with
`strands-agents-tools` which requires `rich>=14.0.0,<15.0.0`. This
change loosens the `rich` dependency to allow `rich` 14.x, enabling
`arcade-ai` and `strands-agents-tools` to be used in the same
environment. `arcade-mcp`'s version was bumped following semver rules
for dependency updates. Compatibility with `rich` 14.x was verified.

---
Linear Issue:
[TOO-258](https://linear.app/arcadedev/issue/TOO-258/rich-version-pin-==1394-conflicts-with-strands-agents-tools)

<a
href="https://cursor.com/background-agent?bcId=bc-06ee2617-5b7c-4c06-a501-c76076fc56dd"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-06ee2617-5b7c-4c06-a501-c76076fc56dd"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-14 10:46:03 -08:00
dependabot[bot]
324a148b5b
Bump authlib from 1.3.0 to 1.6.5 (#724)
Bumps [authlib](https://github.com/authlib/authlib) from 1.3.0 to 1.6.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/authlib/authlib/releases">authlib's
releases</a>.</em></p>
<blockquote>
<h2>v1.6.5</h2>
<h2>What's Changed</h2>
<ul>
<li>Add a <code>request</code> param to RFC7591
<code>generate_client_info</code> and
<code>generate_client_secret</code> methods by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/825">authlib/authlib#825</a></li>
<li>feat: support list params in prepare_grant_uri by <a
href="https://github.com/lisongmin"><code>@​lisongmin</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/827">authlib/authlib#827</a></li>
<li>chore(deps): bump SonarSource/sonarqube-scan-action from 5 to 6 in
/.github/workflows by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/authlib/authlib/pull/828">authlib/authlib#828</a></li>
<li>fix(jose): add max size for JWE zip=DEF decompression by <a
href="https://github.com/lepture"><code>@​lepture</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/830">authlib/authlib#830</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/lisongmin"><code>@​lisongmin</code></a>
made their first contribution in <a
href="https://redirect.github.com/authlib/authlib/pull/827">authlib/authlib#827</a></li>
<li><a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
made their first contribution in <a
href="https://redirect.github.com/authlib/authlib/pull/828">authlib/authlib#828</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/authlib/authlib/compare/v1.6.4...v1.6.5">https://github.com/authlib/authlib/compare/v1.6.4...v1.6.5</a></p>
<h2>v1.6.4</h2>
<h2>What's Changed</h2>
<ul>
<li>fix(jose): prevent public/unprotected header overwriting protected
header by <a
href="https://github.com/lepture"><code>@​lepture</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/809">authlib/authlib#809</a></li>
<li>Fix <code>InsecureTransportError</code> raising by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/810">authlib/authlib#810</a></li>
<li>Add conventional-commits pre-commit hook by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/811">authlib/authlib#811</a></li>
<li>Fix response_mode=form_post with Starlette client by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/812">authlib/authlib#812</a></li>
<li>Specify README.md as project long description by <a
href="https://github.com/EpicWink"><code>@​EpicWink</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/817">authlib/authlib#817</a></li>
<li>Migrate tests to pytest paradigm by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/813">authlib/authlib#813</a></li>
<li>jose/jws: Reject unprotected ‘crit’ and enforce type; add tests by
<a href="https://github.com/AL-Cybision"><code>@​AL-Cybision</code></a>
in <a
href="https://redirect.github.com/authlib/authlib/pull/823">authlib/authlib#823</a></li>
<li>Use explicit *.test urls in unit tests by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/824">authlib/authlib#824</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/EpicWink"><code>@​EpicWink</code></a>
made their first contribution in <a
href="https://redirect.github.com/authlib/authlib/pull/817">authlib/authlib#817</a></li>
<li><a
href="https://github.com/AL-Cybision"><code>@​AL-Cybision</code></a>
made their first contribution in <a
href="https://redirect.github.com/authlib/authlib/pull/823">authlib/authlib#823</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/authlib/authlib/compare/v1.6.3...v1.6.4">https://github.com/authlib/authlib/compare/v1.6.3...v1.6.4</a></p>
<h2>Version 1.6.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Add diff-cover check in GHA by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/803">authlib/authlib#803</a></li>
<li>Run GHA unit tests with uv by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/805">authlib/authlib#805</a></li>
<li>Move from pre-commit to prek by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/804">authlib/authlib#804</a></li>
<li>Sign OIDC id_token according to
<code>id_token_signed_response_alg</code> client metadata by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/802">authlib/authlib#802</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/authlib/authlib/compare/v1.6.2...v1.6.3">https://github.com/authlib/authlib/compare/v1.6.2...v1.6.3</a></p>
<h2>Version 1.6.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Allow insecure transport for 127.0.0.1 for debugging by <a
href="https://github.com/geigerzaehler"><code>@​geigerzaehler</code></a>
in <a
href="https://redirect.github.com/authlib/authlib/pull/788">authlib/authlib#788</a></li>
<li>Raise a MissingCodeError when code parameter is missing by <a
href="https://github.com/lepture"><code>@​lepture</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/786">authlib/authlib#786</a></li>
<li>Temporarily restore OAuth2Request body parameter by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/791">authlib/authlib#791</a></li>
<li>Raise MissingCodeException when code parameter is missing by <a
href="https://github.com/lepture"><code>@​lepture</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/794">authlib/authlib#794</a></li>
<li>Fix id_token generation with EdDSA alg by <a
href="https://github.com/azmeuk"><code>@​azmeuk</code></a> in <a
href="https://redirect.github.com/authlib/authlib/pull/800">authlib/authlib#800</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/authlib/authlib/compare/v1.6.1...v1.6.2">https://github.com/authlib/authlib/compare/v1.6.1...v1.6.2</a></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/authlib/authlib/blob/main/docs/changelog.rst">authlib's
changelog</a>.</em></p>
<blockquote>
<h2>Version 1.6.5</h2>
<p><strong>Released on Oct 2, 2025</strong></p>
<ul>
<li>RFC7591 <code>generate_client_info</code> and
<code>generate_client_secret</code> take a <code>request</code>
parameter.</li>
<li>Add size limitation when decode JWS/JWE to prevent DoS.</li>
<li>Add size limitation for <code>DEF</code> JWE zip algorithm.</li>
</ul>
<h2>Version 1.6.4</h2>
<p><strong>Released on Sep 17, 2025</strong></p>
<ul>
<li>Fix <code>InsecureTransportError</code> error raising.
:issue:<code>795</code></li>
<li>Fix <code>response_mode=form_post</code> with Starlette client.
:issue:<code>793</code></li>
<li>Validate <code>crit</code> header value, reject unprotected header
in <code>crit</code> header.</li>
</ul>
<h2>Version 1.6.3</h2>
<p><strong>Released on Aug 26, 2025</strong></p>
<ul>
<li>OIDC <code>id_token</code> are signed according to
<code>id_token_signed_response_alg</code>
client metadata. :issue:<code>755</code></li>
</ul>
<h2>Version 1.6.2</h2>
<p><strong>Released on Aug 23, 2025</strong></p>
<ul>
<li>Temporarily restore <code>OAuth2Request</code> <code>body</code>
parameter. :issue:<code>781</code> :pr:<code>791</code></li>
<li>Allow <code>127.0.0.1</code> in insecure transport mode.
:pr:<code>788</code></li>
<li>Raise <code>MissingCodeException</code> when the <code>code</code>
parameter is missing. :issue:<code>793</code> :pr:<code>794</code></li>
<li>Fix <code>id_token</code> generation with <code>EdDSA</code> algs.
:issue:<code>799</code> :pr:<code>800</code></li>
</ul>
<h2>Version 1.6.1</h2>
<p><strong>Released on Jul 20, 2025</strong></p>
<ul>
<li>Filter key set with additional &quot;alg&quot; and &quot;use&quot;
parameters.</li>
<li>Restore and deprecate <code>OAuth2Request</code> <code>body</code>
parameter. :issue:<code>781</code></li>
</ul>
<h2>Version 1.6.0</h2>
<p><strong>Released on May 22, 2025</strong></p>
<ul>
<li>Fix issue when :rfc:<code>RFC9207 &lt;9207&gt;</code> is enabled and
the authorization endpoint response is not a redirection.
:pr:<code>733</code></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9ec42561cd"><code>9ec4256</code></a>
chore: release 1.6.5</li>
<li><a
href="b62b5b2757"><code>b62b5b2</code></a>
Merge branch 'fix-GHSA-pq5p-34cr-23v9'</li>
<li><a
href="e0863d5129"><code>e0863d5</code></a>
Merge pull request <a
href="https://redirect.github.com/authlib/authlib/issues/830">#830</a>
from authlib/fix-GHSA-g7f3-828f-7h7m</li>
<li><a
href="867e3f87b0"><code>867e3f8</code></a>
fix(jose): add size limitation to prevent DoS</li>
<li><a
href="75ad6d4d62"><code>75ad6d4</code></a>
Merge pull request <a
href="https://redirect.github.com/authlib/authlib/issues/828">#828</a>
from authlib/dependabot/github_actions/dot-github/wor...</li>
<li><a
href="68b982352d"><code>68b9823</code></a>
chore(deps): bump SonarSource/sonarqube-scan-action</li>
<li><a
href="5bdfc4bfff"><code>5bdfc4b</code></a>
Merge pull request <a
href="https://redirect.github.com/authlib/authlib/issues/827">#827</a>
from lisongmin/support-list-params-in-prepare-grant-uri</li>
<li><a
href="30ea3c5f85"><code>30ea3c5</code></a>
feat: support list params in prepare_grant_uri</li>
<li><a
href="4b5b570339"><code>4b5b570</code></a>
fix(jose): add max size for JWE zip=DEF decompression</li>
<li><a
href="6e35a02ecf"><code>6e35a02</code></a>
Merge pull request <a
href="https://redirect.github.com/authlib/authlib/issues/825">#825</a>
from azmeuk/request-params</li>
<li>Additional commits viewable in <a
href="https://github.com/authlib/authlib/compare/v1.3.0...v1.6.5">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=authlib&package-manager=pip&previous-version=1.3.0&new-version=1.6.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ArcadeAI/arcade-mcp/network/alerts).

</details>

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Skip GHA secret-backed toolkit tests for dependabot, add Dockerized
Postgres test setup and default to postgres user, and bump authlib to
1.6.5.
> 
> - **CI**:
> - Update `Test stand-alone toolkits (with secrets)` condition to also
exclude `github.actor == 'dependabot[bot]'`.
>   - Execute optional `tests/test_setup.sh` before pytest when present.
> - **Postgres toolkit tests**:
> - Default `POSTGRES_DATABASE_CONNECTION_STRING` user changed to
`postgres` in `toolkits/postgres/tests/test_postgres.py`.
> - Add `toolkits/postgres/tests/test_setup.sh` to spin up a Docker
`postgres` and wait until ready.
> - **Dependencies**:
>   - Upgrade `authlib` to `1.6.5` in `pyproject.toml`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
f3600e7536a409ecd8e645f473d747b9ba363765. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: evan <evan@arcade.dev>
2025-12-12 17:16:14 -08:00
Eric Gustin
4d54b28926
Bump some verisons (#723)
`arcade-mcp-server` version was not bumped in
https://github.com/ArcadeAI/arcade-mcp/pull/717, so this PR bumps
`arcade-mcp-server`, and then update's `arcade-mcp`'s dependency on
`arcade-mcp-server` by increasing the minimum version

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Bumps arcade-mcp-server to 1.13.0, updates arcade-mcp to 1.6.2, and
raises related dependency minimums (including example auth server).
> 
> - **Versions**:
> - Bump `libs/arcade-mcp-server` project version from `1.12.0` to
`1.13.0`.
>   - Bump `arcade-mcp` package version from `1.6.1` to `1.6.2`.
> - **Dependencies**:
> - Raise `arcade-mcp` dependency on `arcade-mcp-server` to `>=1.13.0`
in `pyproject.toml` (including `all` extra).
> - Increase example server
`examples/mcp_servers/authorization/pyproject.toml` minimum
`arcade-mcp-server` to `>=1.12.0`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
8a4f606bd8d0b48dd50e3e8e836d31bb679c6eba. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-11 14:09:23 -08:00
Nate Barbettini
592c3f73c0
fix: Don't double-include template files (#722)
Fixes broken publishing action:
https://github.com/ArcadeAI/arcade-mcp/actions/runs/20147239181

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Removes template force-include to avoid duplicate files and adds CI
wheel-duplicate validation; bumps version to 1.6.1.
> 
> - **Packaging**:
> - Bump `arcade-mcp` version from `1.6.0` to `1.6.1` in
`pyproject.toml`.
> - Remove `[tool.hatch.build.targets.wheel.force-include]` for
`arcade_cli/templates` to prevent double-including template files.
> - **CI/CD**:
> - In `.github/workflows/release-on-version-change.yml`, add a
post-build Python step to validate built wheels for duplicate filenames
before publishing.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
3a15e08772b2b4851b185b04c763f3f5898bdbd5. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-11 13:55:31 -08:00
Nate Barbettini
aae9b3a49c
feat: Support multiple orgs & projects in Arcade CLI (#717)
Fixes [PLT-720: Refactor CLI to support multiple orgs +
projects](https://linear.app/arcadedev/issue/PLT-720/refactor-cli-to-support-multiple-orgs-projects)

This PR removes the legacy login flow (login to get an API key) from
Arcade CLI. Believe it or not, this flow predates the ability to get an
API key from the Dashboard, or even the Dashboard itself!

Notable changes:

**Legacy handling** - When a user with an existing `credentials.yaml`
updates the CLI, they will get instructions on fixing their old
credentials:
<img width="978" height="146" alt="Screenshot 2025-12-08 at 10 10 37"
src="https://github.com/user-attachments/assets/5aeaef2c-bef7-4642-a2f7-f917b257c94b"
/>

Any commands that require login (non-public commands) will be blocked
with the above message until `arcade logout / arcade login` is performed
again.

**New login flow**

```sh
arcade login
Opening a browser to log you in...

 Logged in as nate@arcade.dev.

Active project: Nate Barbettini's organization / Default project
Run 'arcade org list' or 'arcade project list' to see available options.
```

**List and set the active organization**
```sh
arcade org list
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ Name                           ┃ ID                                   ┃ Default ┃ Active ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ Nate Barbettini's organization │ 1c64968e-fdc5-4c55-8612-2ce46cd7881b │ ✓       │ ✓      │
│ Sergio 743                     │ 1f1f6184-58dc-4bac-bdde-b9184e43fdf3 │         │        │
└────────────────────────────────┴──────────────────────────────────────┴─────────┴────────┘

Use 'arcade org set <org_id>' to switch organizations.
```
```sh
arcade org set 1c64968e-fdc5-4c55-8612-2ce46cd7881b 

✓ Switched to organization: Nate Barbettini's organization
  Active project: Default project
```

**List and set the active project**
```sh
arcade project list

Active organization: Nate Barbettini's organization
Use 'arcade org list' and 'arcade org set <org_id>' to switch organizations.

┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ Name            ┃ ID                                   ┃ Default ┃ Active ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ Default project │ 35166bf3-6e68-481e-bf16-f747fadc6c22 │ ✓       │ ✓      │
│ Second project  │ 62963205-31ea-4fda-9fc4-af10db89c06f │         │        │
└─────────────────┴──────────────────────────────────────┴─────────┴────────┘

Use 'arcade project set <project_id>' to switch projects.
```
```sh
arcade project set 35166bf3-6e68-481e-bf16-f747fadc6c22
✓ Switched to project: Default project
```

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Migrates CLI to OAuth2 (PKCE) with saved org/project context, adds
org/project commands, rewrites Engine calls to org-scoped endpoints, and
bumps core packages.
> 
> - **Auth & Config**
> - Implement OAuth2 Authorization Code + PKCE (`arcade_cli/authn.py`)
with local callback server and Jinja templates.
> - Persist tokens and active `context` (org/project) in
`credentials.yaml` via updated config models
(`arcade_core/config_model.py`).
> - Add token refresh and CLI config fetch utilities
(`arcade_core/auth_tokens.py`).
> - Detect legacy API-key credentials and block protected commands until
re-login; add `whoami` command.
> - **Org/Project Management**
> - New subcommands: `arcade org list|set`, `arcade project list|set`
(fetch via Coordinator).
> - **Engine API usage (org-scoped)**
> - Introduce org/project URL rewriting transports
(`arcade_core/network/org_transport.py`) and helpers
(`get_org_scoped_url`, `get_arcade_client`, `get_auth_headers`).
> - Update `deploy`, `server`, and `secret` commands to use Bearer
tokens and org-scoped paths; adjust log streaming/status, secrets CRUD,
and deployment workflows.
> - **CLI UX**
> - Replace legacy login URLs/constants; add success/failure HTML
templates for browser callback.
>   - Tweak `dashboard` to health-check without credentials.
>   - Usage tracking now includes `org_id`/`project_id` properties.
> - **Tests**
> - Update tests for dashboard, secrets, utils, and usage identity
(OAuth `/whoami`).
> - **Dependencies & Versions**
> - Bump packages: `arcade-core@4.0.0`, `arcade-mcp-server@1.12.0`,
`arcade-serve@3.2.0`, `arcade-tdk@3.3.0`; add `authlib`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
49702c2f74b9db15bb286d3ec71179b4e74a9134. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-11 12:58:55 -08:00
Eric Gustin
489e01e149
Improve message when evals isn't installed (#687)
New and improved error message

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Enhances dependency checks to display uv/pip install commands and
updates evals command accordingly; bumps package version to 1.5.9.
> 
> - **CLI utils**:
> - Update `require_dependency` to accept `uv_install_command` and
`pip_install_command` and format error message with both install
options.
> - **Evals command (`arcade_cli/main.py`)**:
> - Update `require_dependency` calls for `arcade_evals` and
`arcade_tdk` to provide uv/pip install commands.
> - **Version**:
> - Bump `project.version` in `pyproject.toml` from `1.5.8` to `1.5.9`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
217a6a87686e27747ef59d66bc0db05a270b294a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-09 15:41:15 -08:00
Eric Gustin
bdb7163313
Update ALL libs and their deps to most recent version of other libs (#716)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Bumps versions across core libs and MCP packages, updates dependency
constraints accordingly, and refreshes CLI template defaults to the new
minimums.
> 
> - **Versions**:
>   - `arcade-core` → `3.4.0`
>   - `arcade-serve` → `3.1.5`
>   - `arcade-tdk` → `3.2.2`
>   - `arcade-mcp-server` → `1.11.1`
>   - Root `arcade-mcp` → `1.5.8`
> - **Dependency constraints**:
> - Raise minimums to `arcade-core>=3.4.0`, `arcade-serve>=3.1.5`,
`arcade-tdk>=3.2.2`, `arcade-mcp-server>=1.11.1` across affected
`pyproject.toml` files.
> - **CLI** (`libs/arcade-cli/arcade_cli/new.py`):
> - Update template minimums for `arcade-mcp` (`1.5.8`), `arcade-tdk`
(`3.2.2`), `arcade-serve` (`3.1.5`), and `arcade-mcp-server` (`1.11.1`).
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
b1b43e4d04810b0b49d4de6c943c23e69c99aad4. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-05 14:57:10 -08:00
Sterling Dreyer
f887877a8f
Fix arcade tdk and core version incompatibility (#714)
Update `arcade-core` and `arcade-tdk` dependency versions to resolve
`ImportError` caused by incompatible `Figma` auth provider imports.

The `Figma` OAuth2 provider was introduced in `arcade-core` 3.3.5.
`arcade-tdk` 3.2.0 and `arcade-mcp-server` 1.10.2 started importing
`Figma`, but their `pyproject.toml` dependency constraints were not
updated to require `arcade-core>=3.3.5`. This led to `ImportError` when
`arcade-tdk` or `arcade-mcp-server` were installed with an older
`arcade-core` version. This PR updates the minimum required versions in
`pyproject.toml` files across `arcade-tdk`, `arcade-mcp-server`, and the
root project to ensure compatibility.

---
Linear Issue:
[TOO-231](https://linear.app/arcadedev/issue/TOO-231/worker-fails-to-start-due-to-arcade-core-auth-import)

<a
href="https://cursor.com/background-agent?bcId=bc-4383bd24-eb8c-4d2e-bafe-c116a9d83e8b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-4383bd24-eb8c-4d2e-bafe-c116a9d83e8b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Align dependency minimums to `arcade-core>=3.3.5` and
`arcade-tdk>=3.2.0` and bump package versions across projects.
> 
> - **Dependencies**:
> - Raise `arcade-core` minimum to `>=3.3.5,<4.0.0` in
`libs/arcade-mcp-server/pyproject.toml`,
`libs/arcade-tdk/pyproject.toml`, and root `pyproject.toml`.
> - Raise `arcade-tdk` minimum to `>=3.2.0,<4.0.0` in
`libs/arcade-mcp-server/pyproject.toml` and root `pyproject.toml`.
> - **Version bumps**:
>   - `libs/arcade-mcp-server` version `1.10.2` → `1.10.3`.
>   - `libs/arcade-tdk` version `3.2.0` → `3.2.1`.
>   - Root package `arcade-mcp` version `1.5.6` → `1.5.7`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
abec5dff0d18c9e9c1c5a0ceafa73c67b6af661a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-05 13:15:53 -08:00
Eric Gustin
7fb097f20f
Use monkeypatch for tests that use ARCADE_WORKER_SECRET (#694)
Reverts the updates to unit tests in
https://github.com/ArcadeAI/arcade-mcp/pull/691 and replaces with
monkeypatch. They inadvertently changed global process state during the
test run causing failure of post-merge and failure of PyPI publish. See
https://github.com/ArcadeAI/arcade-mcp/actions/runs/19651637906/job/56283833231
to see what failed
2025-11-24 17:22:17 -08:00
Eric Gustin
44660d18ce
Only serve worker endpoints if secret is set (#691)
Default to `ARCADE_WORKER_SECRET` being unset. This env var must be
explicitly set now. Once it is set, the `worker/` endpoints will be
served.
2025-11-24 14:39:14 -08:00
Eric Gustin
5602578b2f
Worker Stability (#688)
This PR does three things:
1. Executes synchronous tool calls in thread pool allowing for up to 4 +
# of CPUs executions in parallel.
2. Makes force quitting via double SIGINT/SIGTERM possible and via
single SIGINT/SIGTERM + graceful shutdown timeout expiry possible, even
if there are active connections.
3. Sets `timeout_graceful_shutdown` to
`ARCADE_UVICORN_TIMEOUT_GRACEFUL_SHUTDOWN` env var if set, else defaults
to 15.
4. Disable the worker health check span to reduce noise

Tradeoffs:
Since this PR introduces executing synchronous tools via `await
asyncio.to_thread(func, **func_args)`, this means that there is no way
for the thread to be killed until it finishes. The ramifications of this
is that the force quitting logic that is also implemented in this PR has
to be very harsh `os._exit(1)` just in case there is a sync tool
actively executing. This means that `MCPApp` teardown logic will not
execute when force quitting is required. Although this was already the
case because we weren't previously able to force quit! This tradeoff is
justified for now since "parallel" tool executions will relieve us of
many worker timeouts that we are seeing in prod.

Future work:
Minimize/eliminate the need for `os._exit(1)` such that `MCPApp`
teardown logic will always execute, even when force quitting. The
solution will likely be moving away from `await asyncio.to_thread(func,
**func_args)` (while maintaining "parallelism" and then utilize the
`TaskTrackerMiddleware` introduced in this PR to cancel all of the
active HTTP requests.

Resolves PLT-713
2025-11-20 11:13:41 -08:00
Eric Gustin
bddb383034
Do not require entrypoint for arcade configure for HTTP server (#684) 2025-11-12 10:21:44 -08:00
Eric Gustin
0d30fdadf9
Better explain why arcade deploy may have failed (#677)
Another small one.

When you `arcade deploy`, you need to
1. Run the command from the root of your project, and
2. Specify the relative path to your entrypoint file if it is not
located at the root of your repository or if it is named something other
than `server.py`.
2025-11-04 17:44:16 -08:00
Eric Gustin
a770edca4a
Consistent Server Description and Version (#674)
#672 was a quick fix. This PR makes it a long term fix.

Whether a tool is added via `MCPApp.add_tools_from_module`,
`MCPApp.add_tool`, or `@app.tool`, the server's version and description
will be the same.
2025-11-03 15:20:12 -08:00
Eric Gustin
4ca824cf8f
Improve arcade deploy CLI Command (#634)
Also fleshed out `arcade server` commands and MCPApp.name validation.

Example of output of `arcade deploy`:
<img width="2112" height="1320" alt="image"
src="https://github.com/user-attachments/assets/51fd3dd9-0ff1-442c-a9bb-1dbcd7337e7a"
/>
2025-11-03 11:19:04 -08:00
Eric Gustin
e727af3a21
Fix MCP capabilities, examples, tests, and more (#657)
# PR Description
Consider this PR the result of a full pass through of this repository.
## Add helper for adding tools to an `MCPApp`
You can now add all of the tools in a module to an `MCPApp` via
`app.add_tools_from_module(...)`
## Edit what `arcade new` generates
First, I updated the backend to use hatchling.

Second, the structure generated before this PR was simple, but did not
create a proper Python module.
This hindered developers in the following ways:
1. Difficult to add the tools in your server to an evaluation suite
2. Difficult to add more than one tool to an MCPApp at a time
3. All other niceties that come with being able to import modules
```
# Before
server/
├── .env.example
├── server.py
└── pyproject.toml
```
This PR updates the structure generated such that a valid Python module
is generated:
```
# After 
server/
├── pyproject.toml
└── src/
    └── server/
        ├── __init__.py
        ├── .env.example
        └── server.py
```
## Fix Tool Chaining
`self._ctx.server.executor.run(...)` was being called, but `MCPServer`
does not have an instance of `ToolExecutor` (and it's not intended to be
an instance anyways). I updated `Tool.call_raw` to pass the programmatic
tool call through the `MCPServer._handle_call_tool`. This means that the
programmatic tool calls now go through the same steps that a typical
tool call (initiated by the MCP client) would.

This means that **toolA**, which specifies **requirementsA**, is
permitted to call **toolB**, which specifies **requirementsB**, without
needing to explicitly declare or satisfy **requirementsB**. I believe
this is acceptable because the secrets and/or auth token associated with
**toolB's** `Context` are not exposed to **toolA**, and the secrets
and/or auth token associated with **toolA's** `Context` are not exposed
to **toolB**.

## Fix User Elicitation
1. The read & write streams were created with a maximum queue size of 0.
I increased this to 100.
2. I updated `ServerSession`'s run loop to both read messages from the
stream & process them concurrently. This enables server initiated
requests (like user elicitation and progress reporting) to be handled
while tools are being executed. Otherwise, the server initiated requests
would wait for the tool to finish executing and the tool execution would
wait for the server initiated request to finish.
3. 
## Fix Progress Reporting
Progress tokens sent by the client were not being stored. Therefore
there was no way to notify a client with progress updates. I am now
storing the `progressToken`, along with other `_meta` sent from the
client, in the `ServerSession`'s `_request_meta`. I am setting
`_request_meta` whenever the `MCPServer` is handling an incoming message
from a client.

## Fix handling of server names with spaces
Before: 
Server name: "The simple server name"
Tool name: whisper_secret
Name seen by client: "The_simple_server_name_WhisperSecret"

After
Server name: "The simple server name"
Tool name: whisper_secret
Name seen by client: "TheSimpleServerName_WhisperSecret"

## Add Integration Tests
The stdio integration test is much more comprehensive than the http
integration test. These tests will let me sleep a bit more at night

## Add Example MCP Servers
Example servers for sampling, user-elicitation, progress reporting,
logging, tool chaining, combining prebuilt tools with custom tools, tool
secrets, tool auth, evaluations, and more!

## Add Docker template
Added a Docker template for running an MCP server in Docker (and removed
the old docker stuff)
2025-10-30 11:59:00 -07:00
Eric Gustin
66a126bba5
Disallow executing auth/secret tools for unauthenticated servers using HTTP transport (#641)
## PR Description
This PR tackles 3 things:
1. At tool execution runtime, blocks local HTTP servers from executing
tools that have `requires_auth` or `requires_secrets`
2. Make `stdio` the default transport in various locations
3. Improve the `arcade configure` CLI command


<img width="1408" height="1194" alt="image"
src="https://github.com/user-attachments/assets/badf1b55-ec7d-4741-89f5-4b5fee294890"
/>
<img width="3034" height="906" alt="image"
src="https://github.com/user-attachments/assets/aea528c5-4ea6-4eed-b5d7-f946626e58a7"
/>

---------

Co-authored-by: Evan Tahler <evantahler@gmail.com>
2025-10-22 13:14:46 -07:00
Eric Gustin
7d284622ae
Fix stdio settings bug (#636) 2025-10-19 13:25:21 -07:00
Eric Gustin
baa262ec00
Re-import arcade_core errors into arcade_mcp_server (#620) 2025-10-13 17:48:54 -07:00
Eric Gustin
b780e5b807
Fix stdio bugs (#608)
1. Updates `arcade configure claude --from-local` to create a valid json
config for claude desktop. NOTE: The `arcade configure` command needs
some re-work. It's fragile.
2. Fixes bug where stdio servers were sending logs to the wrong sink.
3. Disabled colorized logs for stdio.
4. Added missing dependency `httpx` for servers created with `arcade
new`

## Claude Desktop json configuration for stdio
Personally I like option 1 because the configuration looks the simplest
### Option 1:
Equivalent to `python server.py stdio`
```
{
  "globalShortcut": "Alt+Ctrl+Space",
  "mcpServers": {
    "my_server": {
      "command": "/path/to/my/mcp/server/directory/.venv/bin/python",
      "args": [
        "/path/to/my/mcp/server/directory/server.py",
        "stdio"
      ]
    }
  }
}
```
### Option 2:
Equivalent to `uv run server.py stdio`
```
{
  "mcpServers": {
    "my_server": {
      "command": "uv",
      "args": [
        "run",
        "--directory",
        "/path/to/my/mcp/server/directory",
        "python",
        "server.py",
        "stdio"
      ]
    }
  }
}
```
### Option 3:
Equivalent to `python -m arcade_mcp_server stdio --cwd ./`
```
{
  "mcpServers": {
    "my_server": {
      "command": "/path/to/my/mcp/server/directory/.venv/bin/python",
      "args": [
        "-m",
        "arcade_mcp_server",
        "stdio",
        "--cwd",
        "/path/to/my/mcp/server/directory"
      ]
    }
  }
}
```
2025-10-07 18:53:53 -07:00
Eric Gustin
0cf1a8bd22
Bump versions (#607)
forgot to push this comment before merging #606
2025-10-07 12:55:04 -07:00
Eric Gustin
dcd0a02389
Fix bug and update readme (#599)
The README didn't make any sense for a server developer. Especially when
viewed from PyPI

Fix bug so now stdio works
2025-10-03 12:47:32 -07:00
Eric Gustin
c205bda092
Release libs for arcade-mcp (#598)
Seeing that arcade-ai==2.2.3 doesn't allow for core, serve, or tdk
versions 3.x.x and that it doesn't know about arcade-mcp-server or
arcade-mcp, I feel confident that we can get this past the release
candidate stage. The current state of our documentation
(docs.arcade.dev) still references the 'old way' of doing things, so we
can gradually introduce these new packages to users without the hassle
of specifying pre release flags when installing

### New packages:
arcade-mcp==1.0.0
arcade-mcp-server==1.0.0

### Breaking change with major bump:
arcade-core==3.0.0 from 2.4.0
arcade-serve==3.0.0 from 2.1.0
arcade-tdk==3.0.0 from 2.5.0

### Deprecated:
arcade-ai==2.2.3
2025-10-03 12:11:25 -07:00
Eric Gustin
113d0d3086
CLI Usage (#593)
TLDR; 

The philosophy of CLI usage is "fire and forget" and "best effort". You
can opt out by setting `ARCADE_USAGE_TRACKING=0`.

We are capturing two events: `CLI execution succeeded` and `CLI
execution failed`. Reporting to PostHog is a short lived (maximum 10
seconds) subprocess that does not block the main CLI execution process.

`~/.arcade/usage.json` persists two values `anon_id` and
`linked_principal_id`. The logged in status of the CLI user determines
which ID is used. Upon `arcade login`, the `anon_id` is aliased with
`linked_principal_id`. Upon `arcade logout` the `linked_principal_id` is
removed and the `anon_id` is rotated.

## CLI Usage Tracking - How It Works

The usage tracking system implements an identity management and event
tracking pipeline. Here's how the pieces work together:

### **Identity State Management (`usage.json`)**

The system maintains a persistent identity file at
`~/.arcade/usage.json` with this structure:
```json
{
  "anon_id": "uuid",
  "linked_principal_id": "uuid" | null
}
```

**Key mechanics:**
- **`anon_id`**: Generated once on first CLI use and persists across
sessions. This UUID tracks all anonymous activity.
- **`linked_principal_id`**: Initially `null`. Once the user logs in and
we successfully alias their identity, this field stores their
`principal_id` to indicate this `anon_id` has been linked.
- **Atomic writes**: All updates use a temp file + atomic rename pattern
to prevent corruption from concurrent CLI processes
- **File locking**: Uses `fcntl` (Unix) to coordinate reads/writes
across multiple simultaneous CLI invocations
- **In-memory cache**: The `UsageIdentity` class caches the loaded data
to avoid repeated file I/O within a single CLI invocation

### **Identity Resolution Flow**

When tracking an event, the system determines the `distinct_id` (who to
attribute the event to) via this waterfall:

1. **Check `linked_principal_id`** in `usage.json`
   - If present → use it (user was previously aliased)
   - This is the fastest path and avoids API calls

2. **Fetch `principal_id` from Arcade Cloud API**
- Makes HTTP request to `/api/v1/auth/validate` with the user's API key
from `~/.arcade/credentials.yaml`
   - If authenticated → returns `principal_id`
   - Has 2s timeout for responsiveness

3. **Fall back to `anon_id`**
   - If not authenticated or API call fails → use anonymous ID
   - Marks event with `is_anon=True` flag

### **The Aliasing Lifecycle**

PostHog aliasing links anonymous activity to authenticated users. Here's
the state machine:

#### **Stage 1: Anonymous User**
```
usage.json: { "anon_id": "abc-123", "linked_principal_id": null }
All events → sent with distinct_id="abc-123" and is_anon=True
```

#### **Stage 2: Login Event**
1. User runs `arcade login`
2. Command completes successfully (auth token saved)
3. `CommandTracker` detects successful login
4. Fetches `principal_id` from API
5. Checks `should_alias()` → returns `True` because
`linked_principal_id` is `null`
6. **Calls `alias()` synchronously** (blocking):
   ```python
   posthog.alias(previous_id="abc-123", distinct_id="zyx-321")
   ```
7. Updates `usage.json`:
   ```json
   { "anon_id": "abc-123", "linked_principal_id": "zyx-321" }
   ```
8. PostHog backend merges all events with `distinct_id="abc-123"` into
the user profile for `"zyx-321"`

#### **Stage 3: Authenticated User**
```
usage.json: { "anon_id": "abc-123", "linked_principal_id": "zyx-321" }
All events → sent with distinct_id="zyx-321" and is_anon=False
```
- Events are directly attributed to the authenticated user
- No more API calls needed (uses cached `linked_principal_id`)

#### **Stage 4: Logout Event**
1. User runs `arcade logout`
2. Logout event is sent with the authenticated `distinct_id`
3. `CommandTracker` detects successful logout
4. **Rotates identity** by calling `reset_to_anonymous()`:
   ```json
   { "anon_id": "xyz-789", "linked_principal_id": null }
   ```
5. New `anon_id` prevents cross-contamination if another user logs in

### **Critical Constraint: Alias Timing**

PostHog requires that `alias()` is called **BEFORE** any events are sent
with the new `distinct_id`. This is why:
- **`alias()` is synchronous (blocking)**: Guarantees it completes
before the login success event is sent
- **Subsequent events use `linked_principal_id`**: Once aliased, all
future events use the authenticated ID
- **Lazy aliasing**: If a user authenticates via another mechanism (not
through `arcade login`), the system detects this on the next command and
performs aliasing before sending that command's event

### **Event Capture Pipeline**

When `CommandTracker.track_command_execution()` is called:

1. **Resolve identity** → determines `distinct_id` and `is_anon` flag
2. **Build event properties**:
   ```python
   {
     "command_name": "toolkit.run",
     "cli_version": "1.2.3",
     "python_version": "3.11.0",
     "os_type": "Darwin",
     "os_release": "23.4.0",
     "duration": 1250.42,  # milliseconds
     "error_message": "..."  # if failed
   }
   ```
3. **Call `UsageService.capture()`**:
   - Serializes event data to JSON
   - Spawns detached subprocess: `python -m arcade_cli.usage`
   - Passes data via `ARCADE_USAGE_EVENT_DATA` env var
   - **Returns immediately** (non-blocking)

4. **Detached subprocess (`__main__.py`)**:
   - Runs independently, survives parent CLI exit
   - Deserializes event data
- If `is_anon=True`, sets `$process_person_profile=False` (tells PostHog
not to create a full profile)
   - Sends event to PostHog with 5s timeout
   - Exits (hard exit after 10s max via timeout thread)

### **Concurrency Handling**

Multiple CLI processes can run simultaneously. The system handles this
via:
- **File locking** on `usage.json` (shared lock for reads, exclusive for
writes)
- **Atomic writes** via temp files ensure incomplete writes never
corrupt the file
- **Idempotent aliasing**: `should_alias()` prevents redundant alias
calls

### **Edge Cases Handled**

1. **Side-channel authentication**: User authenticates outside of
`arcade login` (e.g., manually editing credentials)
   - Detected via "lazy aliasing" check on every command
- Performs alias if `linked_principal_id` doesn't match current
`principal_id`

2. **API failures during identity fetch**: Falls back to anonymous
tracking
   - 2s timeout prevents hanging
   - Silent failure doesn't disrupt CLI

3. **PostHog merge restrictions**: Can't alias returning users who
already have a profile
- System stores `linked_principal_id` to avoid retrying impossible
aliases
   - New users (never logged in before) get full history stitched

4. **Multiple accounts on same machine**: Logout rotates `anon_id`
   - User A's anonymous activity won't leak into User B's profile

### **Privacy & Performance**

- **Opt-out**: `ARCADE_USAGE_TRACKING=0` disables all tracking
- **Non-blocking**: Events never slow down CLI (detached subprocess)
- **Anonymous profiles**: `$process_person_profile=False` for `anon_id`
events minimizes data collection
- **Silent failures**: Network issues or PostHog errors never surface to
users
2025-10-03 10:15:08 -07:00
Eric Gustin
a11f79b32d
Update arcade-mcp-server docs (#597)
1. Updates docs to prefer `uv run server.py` instead of `arcade mcp` or
`python -m arcade_mcp_server`
2. Found a bug with running stdio servers while updating the docs, so i
snuck that in this PR
2025-10-02 17:16:38 -07:00
Renato Byrro
7dd62fcc89
Adjust arcade docs command to the new docs repo structure (#592)
- Updates the `arcade docs` templates, dir/file paths, and URL paths to
reflect the new docs repo structure
- References "MCP Server" instead of "toolkit"
- Auto-detects when it's a Starter MCP server and adds the corresponding
warning in the main doc page
- Fixes a bug that generated the wrong file path to the Python & JS
examples when the package name had an underscore character)
- Introduces some minor improvements, such as pulling the MCP Server
description for `ToolInfo` from the package `pyproject.toml`, instead of
a standard description varying only the MCP Server name

---------

Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2025-10-02 01:03:24 -03:00
Eric Gustin
9e4d36b8e3
Local MCP Fixes and Address General Feedback (#586)
# Release Candidate 2
## This PR:
- [x] No more confusing 307 redirect logs when using `/mcp` instead of
`/mcp/` (requested by @shubcodes)
- [x] Fix bug in `arcade configure` for Python < 3.12 (reported by
@evantahler
- [x] Fix bug where tools with unsatisfied secret requirements could
still be executed (reported by @evantahler, @shubcodes)
- [x] Auth providers can now be imported via `from
arcade_mcp_server.auth import Reddit` (requested by @shubcodes)
- [x] Add complete E2E oauth flow for tool calls with informational
errors about how to log into arcade and where to go to authorize
(requested by @evantahler, @shubcodes)
- [x] Add OAuth tool in `arcade new`'s generated server (requested by
@shubcodes)
- [x] Standardize on defaulting to running servers on port 8000
- [x] Improve credentials.yaml reading logic
- [x] CLI user friendliness (requested by @Spartee)
- [x] Remove `arcade serve` CLI command
- [x] Fix race condition in `arcade logout`
- [x] Update docs for desired developer onboarding flow

## Next PRs:
- Get `arcade deploy` working for MCP servers. (Command is hidden for
now)
- Rename all occurrences of `toolkit` to `server`/`tools` and rename all
occurrences of `worker` to `server`
2025-09-29 16:00:47 -07:00
Eric Gustin
3424ec8219
MCP Local (#563)
Versions:
* arcade-mcp\==1.0.0rc1
* arcade-mcp-server\==1.0.0rc1
* arcade-core\==2.5.0rc1
* arcade-tdk\==2.6.0rc1
* arcade-serve\==2.2.0rc1

### Summary
Adds first-class MCP support across Arcade, introduces a new MCP server
and CLI, unifies the project under the arcade-mcp name, overhauls
templates/scaffolding, and improves developer tooling, secrets
management, and examples.

### Highlights
- **MCP Server & Core**
- New MCP server with stdio and HTTP/SSE transports, session management,
resumability, and lifecycle handling.
- FastAPI-like `MCPApp` for building servers with lazy init; integrated
worker+MCP HTTP app option.
- Middleware system (logging and error handling), robust exception
hierarchy, and Pydantic-based settings.
- Async-safe managers for tools, resources, and prompts backed by
registries and locks.
- Developer-facing, transport-agnostic runtime context interfaces (logs,
tools, prompts, resources, sampling, UI, notifications).
- Conversion from Arcade ToolDefinition to MCP tool schema; OpenAI JSON
tool schema converter.
  - Parser supports `@app.tool`/`@app.tool(...)` decorators.

- **CLI**
  - New `mcp` command to run MCP servers with stdio or HTTP/SSE.
- New `secret` command to set/list/unset tool secrets (supports .env
input, preserves original casing for lookups).
- `new` command refactored; option to create a full toolkit package with
scaffolding.
  - `chat` command removed.
- `serve.py` imports updated to `arcade_serve.fastapi.telemetry`;
version retrieval now uses `arcade-mcp`.
  - `show.py` refactor to use new local catalog utilities.
- `display_tool_details` improved: adds “Default” column and handles
nested properties.

- **Configuration & Discovery**
- New `configure.py` to set up Claude Desktop, Cursor, and VS Code to
connect to local or Arcade Cloud MCP servers.
- Discovery utilities to find/install toolkits, build `ToolCatalog`s,
analyze files for tools, load kits from directories (pyproject parsing),
and build minimal toolkits.
- Better handling of provider API key resolution and evaluation suite
loading.

- **Templates & Scaffolding**
- Reorganized template structure (minimal vs full); moved
`.pre-commit-config.yaml`, `.ruff.toml`, license, Makefile, README,
tests, and tools layout to correct paths.
  - Minimal template adds `.env.example` for runtime secret injection.
- Template pyproject updated for MCP servers; includes sample server
with greeting and secret-reveal tools.
  - Authorization flow in templates simplified.

- **Repo-wide Renaming & Examples**
- Migrates references from `arcade-ai` to `arcade-mcp` across READMEs,
scripts, and package metadata.
- Examples updated (LangChain/LangGraph/AI SDK/TypeScript) and package
name changed to `arcade-mcp-sdk`.

- **Evals & Core Utilities**
- Evals now use OpenAI tooling format (`OpenAIToolList`, `to_openai`);
`tool_eval` takes `provider_api_key`.
- Core utilities: fixed `does_function_return_value` by dedenting before
parse; version bump to `2.5.0rc1` and dependency cleanup.

- **Tooling & CI**
- `setup-uv-env` action splits toolkit vs contrib dependency
installation.
- Pre-commit: excludes `libs/arcade-mcp-server/mkdocs.yml` and
`libs/tests/` from YAML and Ruff hooks; Ruff per-file ignores (e.g.,
C901 in `libs/**/*.py`, TRY400 in server docs paths).
- Makefile updates for uv env setup, quality checks, tests, builds, and
new `shell` target.
  - Added Makefile to MCP server library to streamline dev workflow.

- **Cleanup**
  - Removed `claude.json` config.
- Simplified stdio entrypoint; removed unused imports (`arcade_gmail`,
`arcade_search`).

### Breaking Changes
- **CLI**: `chat` command removed; use `mcp`, `secret`, and updated
`new`.
- **Naming**: All users should update references from `arcade-ai` to
`arcade-mcp`.
- **Templates**: File paths moved; downstream scripts referencing old
template locations may need updates.

### Getting Started
- Run an MCP server:
  - `arcade mcp --stdio --toolkits your_toolkit`
  - `arcade mcp --http --toolkits your_toolkit`
- Manage secrets:
  - `arcade secret set your_toolkit KEY=value`
  - `arcade secret list your_toolkit`
  - `arcade secret unset your_toolkit KEY`
- Configure clients:
- `arcade configure` to set up Claude Desktop, Cursor, and VS Code for
local/Arcade Cloud MCP.

---------

Co-authored-by: Sam Partee <sam@arcade-ai.com>
Co-authored-by: Shub <125150494+shubcodes@users.noreply.github.com>
2025-09-25 15:28:15 -07:00
Renato Byrro
a270472a09
Fix reference in arcade docs Python example template to USER_ID instead of TOOL_NAME (#579) 2025-09-25 11:08:19 -03:00
Eric Gustin
667495c389
Bump CLI's arcadepy dependency (#565) 2025-09-17 16:02:55 -07:00
Eric Gustin
f50e05aa9b
Extra safe error adapters (#556)
Catching and logging all unexpected errors that occur in the error
adapters _**just in case**_
2025-09-10 14:51:18 -07:00
Eric Gustin
f4558ef3a8
Tool Error Handling (#539)
# Improvements to Arcade TDK Error Handling
I tried my very best to not make any breaking changes in this PR. So,
you will notice various "Deprecation" notices throughout.

### Instructions for PR reviewers
1. Pull down this PR's branch
2. Pull down the Engine's tool error handling PR's branch
3. Update your installed arcadepy to have the following:
- In `arcadepy/resources/tools/tools.py`, if you want to test out
including stacktraces, then you need to update `ToolsResource.execute`
to accept a `include_error_stacktrace` argument and also include the
"include_error_stacktrace" argument to the POST to the Engine inside of
the function's execute method's body.
- In `arcadepy/types/execute_tool_response.py` add the following enum
      ```py
      class ErrorKind(str, Enum):
          """Error kind that is comprised of
          - the who (toolkit, tool, upstream)
          - the when (load time, definition parsing time, runtime)
- the what (bad_definition, bad_input, bad_output, retry,
context_required, fatal, etc.)"""
      
          TOOLKIT_LOAD_FAILED = "TOOLKIT_LOAD_FAILED"
TOOL_DEFINITION_BAD_DEFINITION = "TOOL_DEFINITION_BAD_DEFINITION"
TOOL_DEFINITION_BAD_INPUT_SCHEMA = "TOOL_DEFINITION_BAD_INPUT_SCHEMA"
TOOL_DEFINITION_BAD_OUTPUT_SCHEMA = "TOOL_DEFINITION_BAD_OUTPUT_SCHEMA"
          TOOL_RUNTIME_BAD_INPUT_VALUE = "TOOL_RUNTIME_BAD_INPUT_VALUE"
TOOL_RUNTIME_BAD_OUTPUT_VALUE = "TOOL_RUNTIME_BAD_OUTPUT_VALUE"
          TOOL_RUNTIME_RETRY = "TOOL_RUNTIME_RETRY"
TOOL_RUNTIME_CONTEXT_REQUIRED = "TOOL_RUNTIME_CONTEXT_REQUIRED"
          TOOL_RUNTIME_FATAL = "TOOL_RUNTIME_FATAL"
          UPSTREAM_RUNTIME_BAD_REQUEST = "UPSTREAM_RUNTIME_BAD_REQUEST"
          UPSTREAM_RUNTIME_AUTH_ERROR = "UPSTREAM_RUNTIME_AUTH_ERROR"
          UPSTREAM_RUNTIME_NOT_FOUND = "UPSTREAM_RUNTIME_NOT_FOUND"
UPSTREAM_RUNTIME_VALIDATION_ERROR = "UPSTREAM_RUNTIME_VALIDATION_ERROR"
          UPSTREAM_RUNTIME_RATE_LIMIT = "UPSTREAM_RUNTIME_RATE_LIMIT"
UPSTREAM_RUNTIME_SERVER_ERROR = "UPSTREAM_RUNTIME_SERVER_ERROR"
          UPSTREAM_RUNTIME_UNMAPPED = "UPSTREAM_RUNTIME_UNMAPPED"
          UNKNOWN = "UNKNOWN"
      ```
- In `arcadepy/types/execute_tool_response.py` add the following fields
to OutputError:
      ```py
      kind: ErrorKind
      status_code: Optional[int] = None
      stacktrace: Optional[str] = None
      extra: Optional[dict[str, Any]] = None
      ```
### Example Client Usage
```py
# Example of handling an upstream rate limit
error = response.output.error
if  error and error.kind == ErrorKind.UPSTREAM_RUNTIME_RATE_LIMIT:
    sleep_time = error.retry_after_ms / 1000
    time.sleep(sleep_time)
    # and then execute again
```
```py
# Examples of determining what type of runtime error it is
error = response.output.error
if error:
    is_retryable_error = error.kind == ErrorKind.TOOL_RUNTIME_RETRY
    is_a_bug_in_the_tool = error.kind == ErrorKind.TOOL_RUNTIME_FATAL
    is_additional_context_required = error.kind == ErrorKind.TOOL_RUNTIME_CONTEXT_REQUIRED
```

### Example Tool Usage
```py
# EXAMPLE 1 letting Arcade handle upstream error handling for you
reddit_client.post(params) # Arcade's httpx adapter will handle error handling for you!

# ------------------------------------

# EXAMPLE 2 handling upstream bad request yourself, but letting Arcade handle the rest
try:
    reddit_client.post(params)
except httpx.HTTPStatusError as e:
    if e.status_code == 400:
        raise UpstreamError("My extra custom message) from e
    raise
```
```py
# EXAMPLE 1 letting Arcade handle it for you
risky_element = my_risky_list[42] # Arcade will raise a FatalToolError for you

# ------------------------------------

# EXAMPLE 2 handling it yourself for extra flexibility
try:
    risky_element = my_risky_list[42]
except IndexError as e:
    raise FatalToolError("My extra custom message") from e
```
### Non-runtime Error Message Examples
Example ToolkitLoadError Messages:
```
- [TOOLKIT_LOAD_FAILED] ToolkitLoadError when loading toolkit 'sample_tool': Could not import module mock_module. Reason: Mock import error
- [TOOLKIT_LOAD_FAILED] ToolkitLoadError when loading toolkit 'test_toolkit': Tool 'ValidTool' in toolkit 'test_toolkit' already exists in the catalog.
```
Example ToolDefinitionError Messages
```
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_missing_description': Tool 'tool_missing_description' is missing a description
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_invalid_secret_type': Secret keys must be strings (error in tool ToolWithInvalidSecretType).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_empty_secret': Secrets must have a non-empty key (error in tool ToolWithEmptySecret).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_invalid_metadata_type': Metadata must be strings (error in tool ToolWithInvalidMetadataType).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_metadata_requiring_auth_without_auth': Tool ToolWithMetadataRequiringAuthWithoutAuth declares metadata key 'client_id', which requires that the tool has an auth requirement, but no auth requirement was provided. Please specify an auth requirement.
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_empty_metadata': Metadata must have a non-empty key (error in tool ToolWithEmptyMetadata).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_unsupported_param_type': Unsupported parameter type: <class 'test_catalog.MyFancyTestClass'>
```
Example ToolInputSchemaError Messages
```
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_missing_input_parameter_annotation': Parameter 'input_text' is missing a description
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_no_type_annotation': Parameter param has no type annotation.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_invalid_param_name': Invalid parameter name: '123invalid' is not a valid identifier. Identifiers must start with a letter or underscore, and can only contain letters, digits, or underscores.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_too_many_annotations': Parameter param: Annotated[str, 'name', 'desc', 'extra'] has too many string annotations. Expected 0, 1, or 2, got 3.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_required_union_param': Parameter param is a union type. Only optional types are supported.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_non_callable_default_factory': Default factory for parameter param: Annotated[str, 'Parameter'] = FieldInfo(annotation=NoneType, required=False, default_factory=str) is not callable.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_multiple_tool_contexts': Only one ToolContext parameter is supported, but tool tool_with_multiple_tool_contexts has multiple.
```
Example ToolOutputSchemaError Messages
```
- [TOOL_DEFINITION_BAD_OUTPUT_SCHEMA] ToolOutputSchemaError in definition of tool 'tool_missing_return_type_hint': Tool 'ToolMissingReturnTypeHint' must have a return type
- [TOOL_DEFINITION_BAD_OUTPUT_SCHEMA] ToolOutputSchemaError in definition of tool 'tool_with_unsupported_output_type': Unsupported output type '<class 'test_catalog.MyFancyTestClass'>'. Only built-in Python types, TypedDicts, Pydantic models, and standard collections are supported as tool output types.
```
### Runtime Error Message Examples
Example Tool Runtime Error Messages
```
- [TOOL_RUNTIME_FATAL] FatalToolError during execution of tool 'get_posts_in_subreddit': list index out of range
- [TOOL_RUNTIME_CONTEXT_REQUIRED] ContextRequiredToolError during execution of tool 'get_posts_in_subreddit': Ambiguous username. Please provide a more specific username
- [TOOL_RUNTIME_RETRY] RetryableToolError during execution of tool 'get_posts_in_subreddit': Retry with subreddit=learnpython or subreddit=learnprogramming
```

Example Upstream Runtime Error Messages
```
- [UPSTREAM_RUNTIME_RATE_LIMIT] UpstreamRateLimitError during execution of tool 'get_posts_in_subreddit': 429 Client Error: Too Many Requests
- [UPSTREAM_RUNTIME_BAD_REQUEST] UpstreamError during execution of tool 'get_posts_in_subreddit': 400 Client Error: Bad request. Missing 'id' parameter.
- [UPSTREAM_RUNTIME_BAD_REQUEST] UpstreamError during execution of tool 'search_files': Upstream Google API error: Invalid value '-23'. Values must be within the range: [value: 1\n, value: 1000\n]
```
2025-09-10 10:45:18 -07:00
Sterling Dreyer
7888dc505e
Fix venv files not being found (#525) 2025-08-01 12:12:35 -07:00
Eric Gustin
288ff61959
Update arcade new (#520)
This PR updates the `arcade new` CLI command.
* adds support for official arcade tools. 
* removes local libs sources for toolkits that are not a community or
official
* Stopped creating README for community toolkits
2025-07-29 10:35:36 -07:00
Sterling Dreyer
c97f2f7500
Bump CLI version (#521) 2025-07-28 16:22:05 -07:00
Sterling Dreyer
3f5c7aa6ba
Error on invalid toolkit file (#510)
Changes toolkit loading and deployments to error if there are syntax
errors in the file
2025-07-28 16:17:06 -07:00
Sterling Dreyer
f4480c3945
Fix arcade worker list endpoints (#504)
We weren't checking that the engine version of the worker was the same
as the cloud version that we were comparing against and incorrectly
saying the URL was wrong

Before
<img width="1447" height="340" alt="Screenshot 2025-07-21 at 1 55 13 PM"
src="https://github.com/user-attachments/assets/cf39ce9f-0c86-45fd-a68e-c92369876292"
/>

After
<img width="1454" height="308" alt="Screenshot 2025-07-21 at 1 55 07 PM"
src="https://github.com/user-attachments/assets/efcfe6c8-b892-45f7-bf4c-71edc66c8325"
/>
2025-07-21 14:43:58 -07:00
Eric Gustin
856606f38c
Remove arcade_ prefix requirement and add entry point toolkit discovery (#485)
## Summary
This PR removes the requirement that all toolkits must have the arcade_
prefix and introduces a more flexible toolkit discovery system using
Python entry points.

### 🏷️ Flexible Toolkit Naming
* Community toolkits: Only add arcade_ prefix when the user is in
arcade-ai/toolkits/ directory and explicitly chooses to create a
community contribution.
* External toolkits: No prefix requirement - developers can name their
toolkits however they want
* Toolkit names are now determined by user choice rather than enforced
automatically
### 🔍 Entry Point Discovery
* Added find_arcade_toolkits_from_entrypoints() method to discover
toolkits via entry points
* Entry point group: arcade_toolkits with name: toolkit_name
* Updated pyproject.toml template to include entry point configuration
* Entry point discovery takes precedence over prefix-based discovery for
deduplication
### 📦 Backward Compatibility
* Existing arcade_* prefixed toolkits continue to work via
find_arcade_toolkits_from_prefix()
find_all_arcade_toolkits() now combines both discovery methods
* Deduplication logic prefers entry point toolkits over prefix-based
ones when package names match
### 🛠️ `arcade new` Template Updates
* pyproject.toml template for `arcade new` now includes entry point
configuration: [project.entry-points.arcade_toolkits]
### 🔧 Minor Improvements
* Refactored _strip_arcade_prefix() into a separate method for
reusability
* Updated variable naming for clarity (community_toolkit →
is_community_toolkit)
### Benefits
* Developer Freedom: Toolkit developers are no longer forced to use the
arcade_ prefix. They are also no longer forced to use the package name
as the toolkit name.
* Cleaner Naming: External toolkits can use more natural names (e.g.,
my_company_toolkit instead of arcade_my_company_toolkit)
* Better Discovery: Entry points provide a more standard Python
mechanism for plugin discovery
* Flexible Distribution: Toolkits can be distributed with any package
name while still being discoverable
### Testing
* Added comprehensive tests for the new entry point functionality
* Tests cover edge cases like deduplication, error handling, and
backward compatibility
### Version Bumps
arcade-core: 2.0.0 → 2.1.0
arcade-ai: 2.0.5 → 2.1.0

This change makes the Arcade toolkit ecosystem more flexible and
developer-friendly while maintaining full backward compatibility with
existing toolkits.

---------

Co-authored-by: Mateo Torres <mateo@arcade.dev>
2025-07-16 09:51:21 -07:00
Sterling Dreyer
32292d4b39
Fix login callback (#489) 2025-07-15 11:21:33 -07:00
Mateo Torres
a30fc9379a
added ruff toml and pre-comit files to template if community (#477)
This reintroduces a question into `arcade new`, which adds the ruff and
pre-commit files into new toolkits that are aimed to be contributed back
to Arcade AI.

I use it in the toolkit building tutorial

---------

Co-authored-by: Evan Tahler <evantahler@gmail.com>
2025-07-09 14:04:30 -03:00
Eric Gustin
87b7d0fdaf
Bump CLI version (#467) 2025-07-01 11:35:13 -07:00
Eric Gustin
ff8675e4b6
Filter out unneeded files/directories before deploying workers (#464)
`arcade deploy` is failing for local packages that have large unneeded
files such as `uv.lock`. It is failing because it is taking too long for
the CLI to compress and PUT to the cloud.
2025-07-01 10:07:15 -07:00
Eric Gustin
0e9207a895
Bump arcade-ai PyPI Package (#462)
v2.0.1 was missed last week. Going straight to 2.0.2 to match the
container version.
2025-07-01 08:07:32 -07:00
Eric Gustin
3149058aa1
Release arcade-ai 2.0.0 (#446) 2025-06-17 12:57:44 -07:00
Eric Gustin
3687150c95
Release arcade-ai==2.0.0rc1 to PyPI (#435) 2025-06-13 15:02:05 -07:00
Sam Partee
b6b4cd0a4c
🏗️ Restructure: Multi-Package Architecture + uv Migration (#412)
### Overview
Major restructuring from monolithic `arcade-ai` package to modular
library architecture with standardized uv-based dependency management.

![arcade-ai Monorepo
(2)](https://github.com/user-attachments/assets/25f102b0-bb87-4a04-9701-d227d05664b1)

### New Package Structure
- **`arcade-tdk`** - Lightweight toolkit development kit (core
decorators, auth)
- **`arcade-core`** - Core execution engine and catalog functionality  
- **`arcade-serve`** - FastAPI/MCP server components
- **`arcade-ai`** - Meta package that includes CLI functionality.
Optionally include evals via the `evals` extra. Optionally include all
packages via the `all` extra.

### Key Benefits
- **Lighter Dependencies**: Toolkits now depend only on `arcade-tdk` (~2
deps) vs full `arcade-ai` (~30+ deps)
- **Faster Builds**: uv provides 10-100x faster dependency resolution
and installation
- **Better Modularity**: Clear separation of concerns, consumers import
only what they need
- **Standard Tooling**: Eliminates custom poetry scripts, uses standard
Python packaging

### Migration Impact
- All 20 toolkits converted from poetry → uv with `arcade-tdk`
dependencies plus `arcade-ai[evals]` and `arcade-serve` dev
dependencies. When developing locally, devs should install toolkits via
`make install-local`.
- Modern Python 3.10+ type hints throughout
- Standardized build system with hatchling backend
- Enhanced Makefile with robust toolkit management commands
- Removed `arcade dev` CLI command
- Reduce the number of files created by `arcade new` and add an option
to not generate a tests and evals folder.

This foundation enables faster development cycles and cleaner dependency
chains for the growing toolkit ecosystem.

### Todo After this PR is merged
- [ ] Post-merge workflow(s) (release & publish containers, etc)
- [ ] Release order plan. @EricGustin suggests releasing in the
following order:
    1. `arcade-core` version 0.1.0
    2. `arcade-serve` version 0.1.0 and `arcade-tdk` version 0.1.0
    3. `arcade-ai` version 2.0.0
4. Patch release for all toolkits (all changes in toolkits are internal
refactors)
- [ ] [Update docs](https://github.com/ArcadeAI/docs/pull/318)

---------

Co-authored-by: Eric Gustin <eric@arcade.dev>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2025-06-11 16:48:17 -07:00