Commit graph

75 commits

Author SHA1 Message Date
Eric Gustin
f15aff42c6
Fix changed files gha (#662) 2025-10-30 13:45:41 -07:00
Evan Tahler
bee349287f
remove mkdocs for arcade-mcp (#617)
Compainion to https://github.com/ArcadeAI/docs/pull/488 - This PR turns
off MKDocs. We have the docs in docs.arcade.dev now.
2025-10-13 10:47:41 -07:00
Eric Gustin
ab93356e7c
Fix slack status message on PyPI publish failure (#604) 2025-10-06 13:10:25 -07:00
Evan Tahler
9f339b7964
docs build commands for arcade-mcp (#587) 2025-09-26 15:30:52 -07:00
Eric Gustin
3424ec8219
MCP Local (#563)
Versions:
* arcade-mcp\==1.0.0rc1
* arcade-mcp-server\==1.0.0rc1
* arcade-core\==2.5.0rc1
* arcade-tdk\==2.6.0rc1
* arcade-serve\==2.2.0rc1

### Summary
Adds first-class MCP support across Arcade, introduces a new MCP server
and CLI, unifies the project under the arcade-mcp name, overhauls
templates/scaffolding, and improves developer tooling, secrets
management, and examples.

### Highlights
- **MCP Server & Core**
- New MCP server with stdio and HTTP/SSE transports, session management,
resumability, and lifecycle handling.
- FastAPI-like `MCPApp` for building servers with lazy init; integrated
worker+MCP HTTP app option.
- Middleware system (logging and error handling), robust exception
hierarchy, and Pydantic-based settings.
- Async-safe managers for tools, resources, and prompts backed by
registries and locks.
- Developer-facing, transport-agnostic runtime context interfaces (logs,
tools, prompts, resources, sampling, UI, notifications).
- Conversion from Arcade ToolDefinition to MCP tool schema; OpenAI JSON
tool schema converter.
  - Parser supports `@app.tool`/`@app.tool(...)` decorators.

- **CLI**
  - New `mcp` command to run MCP servers with stdio or HTTP/SSE.
- New `secret` command to set/list/unset tool secrets (supports .env
input, preserves original casing for lookups).
- `new` command refactored; option to create a full toolkit package with
scaffolding.
  - `chat` command removed.
- `serve.py` imports updated to `arcade_serve.fastapi.telemetry`;
version retrieval now uses `arcade-mcp`.
  - `show.py` refactor to use new local catalog utilities.
- `display_tool_details` improved: adds “Default” column and handles
nested properties.

- **Configuration & Discovery**
- New `configure.py` to set up Claude Desktop, Cursor, and VS Code to
connect to local or Arcade Cloud MCP servers.
- Discovery utilities to find/install toolkits, build `ToolCatalog`s,
analyze files for tools, load kits from directories (pyproject parsing),
and build minimal toolkits.
- Better handling of provider API key resolution and evaluation suite
loading.

- **Templates & Scaffolding**
- Reorganized template structure (minimal vs full); moved
`.pre-commit-config.yaml`, `.ruff.toml`, license, Makefile, README,
tests, and tools layout to correct paths.
  - Minimal template adds `.env.example` for runtime secret injection.
- Template pyproject updated for MCP servers; includes sample server
with greeting and secret-reveal tools.
  - Authorization flow in templates simplified.

- **Repo-wide Renaming & Examples**
- Migrates references from `arcade-ai` to `arcade-mcp` across READMEs,
scripts, and package metadata.
- Examples updated (LangChain/LangGraph/AI SDK/TypeScript) and package
name changed to `arcade-mcp-sdk`.

- **Evals & Core Utilities**
- Evals now use OpenAI tooling format (`OpenAIToolList`, `to_openai`);
`tool_eval` takes `provider_api_key`.
- Core utilities: fixed `does_function_return_value` by dedenting before
parse; version bump to `2.5.0rc1` and dependency cleanup.

- **Tooling & CI**
- `setup-uv-env` action splits toolkit vs contrib dependency
installation.
- Pre-commit: excludes `libs/arcade-mcp-server/mkdocs.yml` and
`libs/tests/` from YAML and Ruff hooks; Ruff per-file ignores (e.g.,
C901 in `libs/**/*.py`, TRY400 in server docs paths).
- Makefile updates for uv env setup, quality checks, tests, builds, and
new `shell` target.
  - Added Makefile to MCP server library to streamline dev workflow.

- **Cleanup**
  - Removed `claude.json` config.
- Simplified stdio entrypoint; removed unused imports (`arcade_gmail`,
`arcade_search`).

### Breaking Changes
- **CLI**: `chat` command removed; use `mcp`, `secret`, and updated
`new`.
- **Naming**: All users should update references from `arcade-ai` to
`arcade-mcp`.
- **Templates**: File paths moved; downstream scripts referencing old
template locations may need updates.

### Getting Started
- Run an MCP server:
  - `arcade mcp --stdio --toolkits your_toolkit`
  - `arcade mcp --http --toolkits your_toolkit`
- Manage secrets:
  - `arcade secret set your_toolkit KEY=value`
  - `arcade secret list your_toolkit`
  - `arcade secret unset your_toolkit KEY`
- Configure clients:
- `arcade configure` to set up Claude Desktop, Cursor, and VS Code for
local/Arcade Cloud MCP.

---------

Co-authored-by: Sam Partee <sam@arcade-ai.com>
Co-authored-by: Shub <125150494+shubcodes@users.noreply.github.com>
2025-09-25 15:28:15 -07:00
Evan Tahler
92ae94fd38
skip PRs for pylon sync (#576) 2025-09-22 10:28:55 -07:00
Evan Tahler
2493d892d2
remove line on comment edit with pylon comment (#573) 2025-09-18 20:48:38 -07:00
Evan Tahler
56121820d3
Pylon comment thread (#571) 2025-09-18 18:41:18 -07:00
Evan Tahler
4e6b66cfe0
create pylon issue if not found (#568) 2025-09-18 18:28:04 -07:00
Evan Tahler
768d70ed79
Pylon Issue and Discussion Sync (#564)
We are going to use Pylon for support. This PR syncs all github issues
and discussions to pylon so we can manage them

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-18 17:44:15 -07:00
Evan Tahler
766a262a25
MongoDB analytics toolkit (#548)
Design System: https://github.com/ArcadeAI/Design-System/pull/31
Docs: https://github.com/ArcadeAI/docs/pull/439

<img width="832" height="797" alt="Screenshot 2025-09-11 at 2 35 36 PM"
src="https://github.com/user-attachments/assets/198f194f-d41c-41a0-bedf-b77bccd70dfa"
/>
2025-09-12 18:41:23 -07:00
Mateo Torres
a97450b3af
Mateo/lchainversion (#551) 2025-09-08 18:29:32 -03:00
Mateo Torres
639f726b30
Mateo/lchain (#549)
The latest change to langchain-arcade was blocked becuase it still
relied on poetry, I moved that to uv
2025-09-08 13:55:06 -03:00
Evan Tahler
a85fa76997
Clickhouse Toolkit (#527)
Generated from the postgres toolkit
2025-08-05 14:05:09 -07:00
Sterling Dreyer
54da08f873
Disable Deployments (#513) 2025-07-24 16:51:53 -07:00
Eric Gustin
b50726d044
Remove Toolkits (#512)
As of today, Arcade's toolkits are moving to closed-source.  

Toolkit source code remains available upon request for our paying
customers. The history of this repository contains the toolkits as of
their most recent publication under their previous license terms.
2025-07-24 16:45:24 -07:00
Evan Tahler
e1cf58fe1b
skip tests with GHA secrets on forks (#496) 2025-07-17 10:27:24 -07:00
Evan Tahler
b81bb86dfb
Skip tests that require secrets in forks (#495)
We know that forks won't have our secrets, so we separate those toolkits
out to new a new test group and skip them if you aren't in this main
repo. We know if a test suite has a secret via a magic comment

---------

Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2025-07-16 16:53:26 -07:00
Eric Gustin
b1e3c334f0
Update Slack webhook step (#484)
This _should_ fix it
2025-07-11 18:14:05 -07:00
Evan Tahler
40cdf2018d
Postgres Database Toolkit (#459)
Adds an example of a good "general case" SQL tool:
* enforces read-only mode
* hints to the LLM to discover the tables and schemas for the tables it
needs before any query
* uses RetryableToolErrors to hint to the LLM about what to do next

Docs: https://github.com/ArcadeAI/docs/pull/345

For testing, `TEST_POSTGRES_DATABASE_CONNECTION_STRING` has been set in
the repo (from Neon). details in 1 password.

<img width="1178" height="1091"
alt="464977013-49aff5e5-e301-4ca0-83b5-3ea742db2283"
src="https://github.com/user-attachments/assets/9344c27b-015d-4b91-907e-84f2e4193e16"
/>
2025-07-11 17:30:40 -07:00
Eric Gustin
836c890840
Release candidates for new toolkits (#479)
New toolkit versions are release candidates now.

Also fixed a test for e2b toolkit.

Also, I am explicitly not running test for toolkits for the post merge
workflow because they are run on every PR commit (i.e., they must pass
before we even merge)
2025-07-10 10:34:19 -07:00
Eric Gustin
07c52100f3
Split and rename multiple toolkits (#438)
# PR Description
## Split toolkits

This PR splits the `Microsoft`, `Google`, and `Search` toolkits into
multiple toolkits each.
 * `Microsoft` --> `OutlookCalendar`, `OutlookMail`.
* `Google` -----> `GoogleCalendar`, `GoogleContacts`, `GoogleDocs`,
`GoogleDrive`, `Gmail`, `GoogleSheets`
* `Search` -----> `GoogleFinance`, `GoogleFlights`, `GoogleHotels`,
`GoogleJobs`, `GoogleMaps`, `GoogleNews`, `GoogleSearch`,
`GoogleShopping`, `Walmart`, `Youtube`

> The original monolithic toolkits (`Microsoft`, `Google`, `Search`) are
not removed in this PR. The plan is to keep those toolkits around while
we
> 1. Stop documenting the toolkits, 
> 2. Stop displaying the toolkits in the dashboard, and 
> 3. Help customers migrate over to the new split toolkits.

## Rename toolkits
This PR renames the following toolkits 
* `Web` ------------> `Firecrawl`
* `CodeSandbox` ---> `E2B`

> The `Web` and `CodeSandbox` toolkits are not removed in this PR. The
plan is to keep them around while we
> 1. Stop documenting the toolkits, 
> 2. Stop displaying the toolkits in the dashboard, and 
> 3. Help customers migrate over to the new renamed toolkits.

## Rename tools
Since toolkit names were changed, this called for some tools to be
renamed as well.
* `GoogleSearch.SearchGoogle` ----------------> `GoogleSearch.Search`
* `GoogleShopping.SearchShoppingProducts` --->
`GoogleShopping.SearchProducts`
* `Walmart.SearchWalmartProducts` ------------> `Walmart.SearchProducts`
* `Walmart.GetWalmartProductDetails` --------->
`Walmart.GetProductDetails`
* `Youtube.SearchYoutubeVideos` -------------->
`Youtube.SearchForVideos`

## Google File Picker
Improvements to the Google File Picker experience were also added in
this PR.

The following tools will ALWAYS provide llm_instructions in their
response to "let the end-user know that they have the option to select
more files via the file picker url if they want to":
* `GoogleDocs.SearchDocuments`
* `GoogleDocs.SearchAndRetrieveDocuments`
* `GoogleDrive.GetFileTreeStructure`

The following tools will only provide the file picker URL if a 404 or
403 from the Google API:
* `GoogleDocs.InsertTextAtEndOfDocument`
* `GoogleDocs.GetDocumentById`
* `GoogleSheets.GetSpreadsheet`
* `GoogleSheets.WriteToCell`

Also, a standalone `GoogleDrive.GenerateGoogleFilePickerUrl` tool
exists.

## Other
* The `SearchDocuments` and `SearchAndRetrieveDocuments` tools used to
be organized within the Drive portion of the Google toolkit, but I moved
these into the new GoogleDocs toolkit because they are specific to Docs.

# Progress

- [x] `OutlookCalendar`
- [x] `OutlookMail`
- [x] `GoogleFinance`
- [x] `GoogleFlights`
- [x] `GoogleHotels`
- [x] `GoogleJobs`
- [x] `GoogleMaps`
- [x] `GoogleNews`
- [x] `GoogleSearch`
- [x] `GoogleShopping`
- [x] `Walmart`
- [x] `Youtube`
- [x] `GoogleCalendar`
- [x] `GoogleContacts`
- [x] `GoogleDocs`
- [x] `GoogleDrive`
- [x] `Gmail`
- [x] `GoogleSheets`
- [x] `Firecrawl`
- [x] `E2B`
- [x] File picker

# Discussion
* Repeated code is a consequence of splitting toolkits that use the same
provider. I am open to any ideas that would allow multiple toolkits to
reference common code. Comment your ideas in this PR.
2025-07-09 16:00:09 -07:00
Eric Gustin
a336f43a83
Fix release workflow for toolkits (#469)
The workflow didn't work correctly for toolkits. see run logs for more
info on how it didn't work correctly:
https://github.com/ArcadeAI/arcade-ai/actions/runs/16008156518/job/45159613768

Bumped the code-sandbox toolkit to prove it works. Proof it worked:
https://github.com/ArcadeAI/arcade-ai/actions/runs/16009427831/job/45163629931?pr=469
2025-07-01 13:29:59 -07:00
Eric Gustin
30e36e1325
Update slack notification for PyPI publishes (#463)
Update the Slack notification to use the "package" notification instead
of the "toolkit" notification since this workflow runs for not only
toolkits but also our libs.
2025-07-01 10:00:35 -07:00
Sterling Dreyer
d86cc1c301
Add slack alerts to toolkit publish (#457) 2025-06-24 11:01:21 -07:00
Eric Gustin
bbe8cb3478
Update check unauthorized toolkit updates workflow (#449)
Forgot to add the change for the target versions too
2025-06-17 13:46:58 -07:00
Eric Gustin
99aa53d025
Fix check unauthorized toolkit updates workflow (#448)
Changing toolkits to uv broke this.
2025-06-17 13:32:08 -07:00
Eric Gustin
3149058aa1
Release arcade-ai 2.0.0 (#446) 2025-06-17 12:57:44 -07:00
Eric Gustin
093ac56040
Update release containers GH Action (#439)
1. Patch, Minor, or Major is selected during workflow dispatch (defaults
to patch)
2. Get most recent release version
3. Calculate new version based on findings from step 2
4. Build wheels
5. Build worker and base-worker
6. Push to GHCR
7. Create new release with release notes

Also, I removed everything related to ECR
2025-06-17 10:21:08 -07:00
Eric Gustin
d19ef6fbb7
Deploy worker to bosslevel (#433) 2025-06-13 14:09:15 -07:00
Eric Gustin
86cde2d9bd
Add PyPI release workflow (#429)
This is the first of a few PRs. Deploy to staging will fail until we
have `arcade-core`, `arcade-serve`, and `arcade-ai` released to PyPI.
This PR will release `arcade-core` to PyPI.


### PR Description
* Adds workflow that checks for changes in any pyproject.toml, and if
its version has changed, then tests, builds wheel, then publishes to
PyPI
* Updates the Dockerfile for our new structure
* Updates porter yamls
* Updates `make full-dist`
* Removes a couple unused workflows

Check out https://github.com/ArcadeAI/arcade-ai/actions/runs/15622059209
to see how the new workflow works (note that it failed publishing to
PyPI on purpose)
2025-06-13 11:22:31 -07:00
Sam Partee
b6b4cd0a4c
🏗️ Restructure: Multi-Package Architecture + uv Migration (#412)
### Overview
Major restructuring from monolithic `arcade-ai` package to modular
library architecture with standardized uv-based dependency management.

![arcade-ai Monorepo
(2)](https://github.com/user-attachments/assets/25f102b0-bb87-4a04-9701-d227d05664b1)

### New Package Structure
- **`arcade-tdk`** - Lightweight toolkit development kit (core
decorators, auth)
- **`arcade-core`** - Core execution engine and catalog functionality  
- **`arcade-serve`** - FastAPI/MCP server components
- **`arcade-ai`** - Meta package that includes CLI functionality.
Optionally include evals via the `evals` extra. Optionally include all
packages via the `all` extra.

### Key Benefits
- **Lighter Dependencies**: Toolkits now depend only on `arcade-tdk` (~2
deps) vs full `arcade-ai` (~30+ deps)
- **Faster Builds**: uv provides 10-100x faster dependency resolution
and installation
- **Better Modularity**: Clear separation of concerns, consumers import
only what they need
- **Standard Tooling**: Eliminates custom poetry scripts, uses standard
Python packaging

### Migration Impact
- All 20 toolkits converted from poetry → uv with `arcade-tdk`
dependencies plus `arcade-ai[evals]` and `arcade-serve` dev
dependencies. When developing locally, devs should install toolkits via
`make install-local`.
- Modern Python 3.10+ type hints throughout
- Standardized build system with hatchling backend
- Enhanced Makefile with robust toolkit management commands
- Removed `arcade dev` CLI command
- Reduce the number of files created by `arcade new` and add an option
to not generate a tests and evals folder.

This foundation enables faster development cycles and cleaner dependency
chains for the growing toolkit ecosystem.

### Todo After this PR is merged
- [ ] Post-merge workflow(s) (release & publish containers, etc)
- [ ] Release order plan. @EricGustin suggests releasing in the
following order:
    1. `arcade-core` version 0.1.0
    2. `arcade-serve` version 0.1.0 and `arcade-tdk` version 0.1.0
    3. `arcade-ai` version 2.0.0
4. Patch release for all toolkits (all changes in toolkits are internal
refactors)
- [ ] [Update docs](https://github.com/ArcadeAI/docs/pull/318)

---------

Co-authored-by: Eric Gustin <eric@arcade.dev>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2025-06-11 16:48:17 -07:00
Sterling Dreyer
ab1a2a8d31
Move prod build (#424) 2025-06-11 08:30:18 -07:00
Sterling Dreyer
3b50c0bb6e
Add promotion workflow (#422) 2025-06-10 13:12:04 -07:00
porter-deployment-app[bot]
f23d80fdff
Enable Porter Application worker (#423)
Hello 👋 from Porter! Please merge this PR to finish setting up your
application.

---------

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
Co-authored-by: sdreyer <sterling@arcade-ai.com>
2025-06-10 13:02:59 -07:00
Sterling Dreyer
268b268044
Build staging from source (#421) 2025-06-10 12:05:00 -07:00
porter-deployment-app[bot]
ce95b3abf2
Enable Porter Application worker (#418)
Hello 👋 from Porter! Please merge this PR to finish setting up your
application.

Co-authored-by: porter-deployment-app[bot] <87230664+porter-deployment-app[bot]@users.noreply.github.com>
2025-06-06 17:05:19 -07:00
Eric Gustin
ac92f9fd65
Fix GH Action 'publish langchain' (#348)
I've set up a new slack workflow for contrib/ packages
2025-04-06 19:42:51 -07:00
Eric Gustin
53a12802f6
More updates to Publish Toolkit Workflow (#305) 2025-03-17 16:33:51 -07:00
Eric Gustin
5afd31ba27
Update Publish Toolkit Workflow (#303) 2025-03-17 16:10:08 -07:00
Eric Gustin
e048f277fd
Fix publish toolkit workflow (#302)
There was a bug where if the poetry publish failed, then the slack
message would say that it succeeded.

I am setting the notion toolkit version to 0.0.1. This is expected to
fail. I'm doing this to ensure the E2E issue is fixed.

The grep'd string comes from
https://github.com/python-poetry/poetry/blob/main/src/poetry/publishing/uploader.py#L246-L249
2025-03-17 15:40:32 -07:00
Eric Gustin
b7d5736995
Replace changed-files GH Action (#295) 2025-03-15 15:15:01 -07:00
Sam Partee
140f4eca17
Langchain arcade 1.2 (#282)
- **New Class Structure**: Introduced `ToolManager` and
`AsyncToolManager` classes (`ArcadeToolManager` is deprecated)
- **Async Support**: Full async implementation for modern LangChain
applications
- **Better Tool Management**: New methods for adding individual tools
and toolkits
- **CI/CD**: for langchain_arcade


## Upgrade Changes

```python
# Old pattern
manager = ArcadeToolManager(api_key="...")
tools = manager.get_tools(toolkits=["Google"])

# New pattern
manager = ToolManager(api_key="...")
manager.init_tools(toolkits=["Google"])
tools = manager.to_langchain()
```

Now supports underscores vs dots in tool names for better model
compatibility.
2025-03-10 18:52:06 -07:00
Nate Barbettini
668e92934e
fix: Rename actor->worker in gh workflow dispatch (#263)
Fixes a typo (will be relevant later with future CI changes).
2025-02-20 14:15:30 -08:00
Sterling Dreyer
f31cabdf2b
Give release permissions (#245) 2025-02-10 16:07:25 -08:00
Sterling Dreyer
8226cb3316
Fix CICD test wait (#235) 2025-01-27 14:48:33 -08:00
Eric Gustin
3657fc79b6
Whitelist Toolkit Release Managers (#234)
# PR Description
The `github.event.pull_request.author_association` in the "Prevent
Unauthorized Version Updates" workflow was returning inconsistent
results by saying that MEMBERS were CONTRIBUTORS. This PR moves away
from `author_association` in favor of a whitelist text file containing
the GitHub usernames of authorized toolkit release managers.

A toolkit release manager has the following special permissions:
* Can change the version of an existing toolkit
* Can delete an existing toolkit 
* Can rename an existing toolkit
2025-01-27 14:35:45 -08:00
Sam Partee
09a0784cd5
Improve Docker Build and Deployment for Arcade Worker (#205)
This PR enhances the Docker build and deployment process for the Arcade
Worker by:

- **Modularizing Docker Builds:**
- Introduces a new `INSTALL_TOOLKITS` build argument in the `Dockerfile`
to conditionally include toolkits. this enables the creation of a
`arcadeai/worker-base` which can be used to build custom containers in a
multi-stage build. an example of this is included in the example dir.
- Adds `docker-base` Makefile target to build a lightweight base image
without toolkits.

- **Publishing to GitHub Container Registry (GHCR):**
- Adds Makefile targets `publish-ghcr` and `gh-login` for pushing images
to GHCR.
  - Supports publishing both base and full images with toolkits to GHCR.

- **Docker Compose:**
  - Add Docker compose file and setup
  - Renames the `actor` service to `worker`
- Adds an `nginx` service in `docker-compose.yml` to proxy requests to
the Arcade Engine.
  - Introduces an `nginx.conf` file for the Nginx service.

- **Streamlining Toolkit Installation:**
- Moves toolkit installation from the `start.sh` script to the Docker
build process.
- Creates a `toolkits.txt` file to manage toolkit dependencies which can
be edited easily when we want to add a new toolkit. Developers can also
use this approach as shown in the example.

- **Improving Configuration Files:**
- Updates `docker.engine.yaml` and `env.example` to align with the new
setup.


TODO:
- CI/CD needs to be adjusted so that images are pushed to ghcr on
release.
- AWS resources need to be renamed actor -> worker @EricGustin 

This can go in after the above two items are resolved.

---------

Co-authored-by: Wils Dawson <wils@arcade-ai.com>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
Co-authored-by: sdreyer <sterling@arcade-ai.com>
Co-authored-by: Sterling Dreyer <sdreyer21@gmail.com>
Co-authored-by: Nate Barbettini <nathanaelb@gmail.com>
2025-01-23 12:57:24 -08:00
Sterling Dreyer
adaa0da649
Cleanup release action (#217) 2025-01-21 16:18:11 -08:00
Sterling Dreyer
b1ebd2afab
Fix pending test check (#216) 2025-01-21 15:57:24 -08:00