admin-valentin/arcade-mcp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

🏗️ Restructure: Multi-Package Architecture + uv Migration (#412)

Browse source 
### Overview
Major restructuring from monolithic `arcade-ai` package to modular
library architecture with standardized uv-based dependency management.

![arcade-ai Monorepo
(2)](https://github.com/user-attachments/assets/25f102b0-bb87-4a04-9701-d227d05664b1)

### New Package Structure
- **`arcade-tdk`** - Lightweight toolkit development kit (core
decorators, auth)
- **`arcade-core`** - Core execution engine and catalog functionality  
- **`arcade-serve`** - FastAPI/MCP server components
- **`arcade-ai`** - Meta package that includes CLI functionality.
Optionally include evals via the `evals` extra. Optionally include all
packages via the `all` extra.

### Key Benefits
- **Lighter Dependencies**: Toolkits now depend only on `arcade-tdk` (~2
deps) vs full `arcade-ai` (~30+ deps)
- **Faster Builds**: uv provides 10-100x faster dependency resolution
and installation
- **Better Modularity**: Clear separation of concerns, consumers import
only what they need
- **Standard Tooling**: Eliminates custom poetry scripts, uses standard
Python packaging

### Migration Impact
- All 20 toolkits converted from poetry → uv with `arcade-tdk`
dependencies plus `arcade-ai[evals]` and `arcade-serve` dev
dependencies. When developing locally, devs should install toolkits via
`make install-local`.
- Modern Python 3.10+ type hints throughout
- Standardized build system with hatchling backend
- Enhanced Makefile with robust toolkit management commands
- Removed `arcade dev` CLI command
- Reduce the number of files created by `arcade new` and add an option
to not generate a tests and evals folder.

This foundation enables faster development cycles and cleaner dependency
chains for the growing toolkit ecosystem.

### Todo After this PR is merged
- [ ] Post-merge workflow(s) (release & publish containers, etc)
- [ ] Release order plan. @EricGustin suggests releasing in the
following order:
    1. `arcade-core` version 0.1.0
    2. `arcade-serve` version 0.1.0 and `arcade-tdk` version 0.1.0
    3. `arcade-ai` version 2.0.0
4. Patch release for all toolkits (all changes in toolkits are internal
refactors)
- [ ] [Update docs](https://github.com/ArcadeAI/docs/pull/318)

---------

Co-authored-by: Eric Gustin <eric@arcade.dev>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
This commit is contained in:
Sam Partee 2025-06-11 16:48:17 -07:00 committed by GitHub
parent 01a9efcf63
commit b6b4cd0a4c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
429 changed files with 3321 additions and 3184 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
.github/actions/setup-uv-env/action.yml vendored Normal file
View file

@ -0,0 +1,20 @@
name: "setup-uv-env"
description: "Composite action to setup the Python and uv environment."
inputs:
python-version:
required: false
description: "The python version to use"
default: "3.11"
runs:
using: "composite"
steps:
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ inputs.python-version }}
- name: Install dependencies
run: uv sync --dev --extra all
shell: bash

29
.github/workflows/main.yml vendored
View file

@ -20,12 +20,12 @@ jobs:
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Set up the environment
uses: ./.github/actions/setup-poetry-env
uses: ./.github/actions/setup-uv-env
- name: Run checks
run: make check
tox:
test:
runs-on: ubuntu-latest
strategy:
matrix:
@ -35,30 +35,13 @@ jobs:
- name: Check out
uses: actions/checkout@v4
- name: Set up python
uses: actions/setup-python@v5
- name: Set up the environment
uses: ./.github/actions/setup-uv-env
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.5
- name: Load cached venv
uses: actions/cache@v4
with:
path: .tox
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('arcade/poetry.lock') }}
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox tox-gh-actions
- name: Test with tox
run: cd arcade && tox
- name: Test libs
run: make test
- name: Upload coverage reports to Codecov with GitHub Action on Python 3.10
uses: codecov/codecov-action@v4.0.1
if: ${{ matrix.python-version == '3.10' }}

57
.github/workflows/test-toolkits.yml vendored
View file

@ -11,46 +11,43 @@ jobs:
setup:
runs-on: ubuntu-latest
outputs:
tool_matrix: ${{ steps.dataStep.outputs.tools }}
tool_matrix: ${{ steps.get-toolkits.outputs.toolkits }}
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Check out
uses: actions/checkout@v4
- name: Get Toolkits
id: dataStep
- name: Get toolkits
id: get-toolkits
run: |
TARGETS=$(./.github/scripts/get_toolkits.sh)
echo "tools=$(jq -cn --argjson environments "$TARGETS" '{target: $environments}')" >> $GITHUB_OUTPUT
# Find all directories in toolkits/ that have a pyproject.toml
TOOLKITS=$(find toolkits -maxdepth 1 -type d -not -name "toolkits" -exec test -f {}/pyproject.toml \; -exec basename {} \; | jq -R -s -c 'split("\n")[:-1]')
echo "Found toolkits: $TOOLKITS"
echo "toolkits=$TOOLKITS" >> $GITHUB_OUTPUT
test-toolkits:
needs: setup
runs-on: ubuntu-latest
strategy:
matrix: ${{ fromJson(needs.setup.outputs.tool_matrix) }}
matrix:
toolkit: ${{ fromJson(needs.setup.outputs.tool_matrix) }}
fail-fast: true
steps:
- run: echo ${{ matrix.target }}
- name: Check out
uses: actions/checkout@v4
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up the environment
uses: ./.github/actions/setup-uv-env
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.5
- name: Install toolkit dependencies
working-directory: toolkits/${{ matrix.toolkit }}
run: uv pip install -e ".[dev]"
- uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Check toolkit
working-directory: toolkits/${{ matrix.toolkit }}
run: |
uv run --active pre-commit run -a
uv run --active mypy --config-file=pyproject.toml
- name: Test Toolkit
id: Test_Toolkit
working-directory: toolkits/${{ matrix.target }}
run: |
make install
make check
make test
- name: Test toolkit
working-directory: toolkits/${{ matrix.toolkit }}
run: uv run --active pytest -W ignore -v --cov=arcade_${{ matrix.toolkit }} --cov-report=xml

1
.pre-commit-config.yaml
View file

@ -5,6 +5,7 @@ repos:
- id: check-case-conflict
- id: check-merge-conflict
- id: check-toml
exclude: ".*/templates/.*"
- id: check-yaml
exclude: ".*/templates/.*"
- id: end-of-file-fixer

4
.ruff.toml
View file

@ -2,6 +2,10 @@ target-version = "py39"
line-length = 100
fix = true
exclude = [
"libs/arcade-cli/arcade_cli/templates/",
]
[lint]
select = [
# flake8-2020

28
.vscode/launch.json vendored
View file

@ -5,7 +5,7 @@
"name": "Debug `arcade workerup --no-auth`",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/arcade/run_cli.py",
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
"args": ["workerup", "--no-auth"],
"console": "integratedTerminal",
"jinja": true,
@ -16,34 +16,34 @@
"name": "Debug `arcade chat -d -h localhost`",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/arcade/run_cli.py",
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
"args": ["chat", "-d", "-h", "localhost"],
"console": "integratedTerminal",
"jinja": true,
"justMyCode": true,
"cwd": "${workspaceFolder}"
},
{
"name": "Debug `arcade dev`",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/arcade/run_cli.py",
"args": ["dev"],
"console": "integratedTerminal",
"jinja": true,
"justMyCode": true,
"cwd": "${workspaceFolder}"
},
{
"name": "Debug `arcade evals -d` on current file",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/arcade/run_cli.py",
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
"args": ["evals", "-d", "${fileDirname}", "-h", "localhost"],
"console": "integratedTerminal",
"jinja": true,
"justMyCode": true,
"cwd": ""
},
{
"name": "Debug `arcade serve`",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
"args": ["serve"],
"console": "integratedTerminal",
"jinja": true,
"justMyCode": true,
"cwd": ""
}
]
}

37
CONTRIBUTING.md
View file

@ -45,7 +45,7 @@ If you are proposing a new feature:
# Get Started!
Ready to contribute? Here's how to set up `arcade-ai` for local development.
Please note this documentation assumes you already have `poetry` and `Git` installed and ready to go.
Please note this documentation assumes you already have `uv` and `Git` installed and ready to go.
1. Fork the `arcade-ai` repo on GitHub.
@ -53,7 +53,7 @@ Please note this documentation assumes you already have `poetry` and `Git` insta
```bash
cd <directory_in_which_repo_should_be_created>
git clone git@github.com:YOUR_NAME/arcade-ai.git
git clone git@github.com:YOUR_GITHUB_USERNAME/arcade-ai.git
```
3. Now we need to install the environment. Navigate into the directory
@ -62,25 +62,30 @@ git clone git@github.com:YOUR_NAME/arcade-ai.git
cd arcade-ai
```
If you are using `pyenv`, select a version to use locally. (See installed versions with `pyenv versions`)
Create your virtual environment
```bash
pyenv local <x.y.z>
uv venv --python 3.11.6
```
Then, install and activate the environment with:
4. Install the development environment and dependencies:
```bash
poetry install
poetry shell
# Install all packages and development dependencies via uv workspace
uv sync --extra all --dev
# Install pre-commit hooks for code quality
uv run pre-commit install
```
4. Install pre-commit to run linters/formatters at commit time:
Or use the convenient Makefile command that does both:
```bash
poetry run pre-commit install
make install
```
The uv workspace will automatically handle installing all lib packages in the correct dependency order.
5. Create a branch for local development:
```bash
@ -89,7 +94,7 @@ git checkout -b name-of-your-bugfix-or-feature
Now you can make your changes locally.
6. Don't forget to add test cases for your added functionality to the `tests` directory.
6. Don't forget to add test cases for your added functionality to the `libs/tests` directory.
7. When you're done making changes, check that your changes pass the formatting tests.
@ -103,15 +108,14 @@ Now, validate that all unit tests are passing:
make test
```
9. Before raising a pull request you should also run tox.
This will run the tests across different versions of Python:
8. You can also run tests for specific components:
```bash
tox
# Test all lib packages
make test
```
This requires you to have multiple versions of python installed.
This step is also triggered in the CI/CD pipeline, so you could also choose to skip this step locally.
9. The CI/CD pipeline will run additional checks across different Python versions, so local testing with a single version is usually sufficient.
10. Commit your changes and push your branch to GitHub:
@ -129,8 +133,7 @@ Before you submit a pull request, check that it meets these guidelines:
1. The pull request should include tests.
2. If the pull request adds functionality, the docs should be updated.
Put your new functionality into a function with a docstring, and add the feature to the list in `README.md`.
2. If the pull request adds functionality, the [docs](https://github.com/ArcadeAI/docs) should be updated.
3. If making contributions to multiple toolkits (i.e. Google and Slack, etc.), submit a separate pull request for each.
This helps us segregate the changes during the review process making it more efficient.

224
Makefile
View file

@ -1,30 +1,64 @@
VERSION ?= "0.1.0.dev0"
CLI_VERSION ?= "2.0.0"
TDK_VERSION ?= "2.0.0"
SERVE_VERSION ?= "2.0.0"
CORE_VERSION ?= "2.0.0"
.PHONY: install
install: ## Install the poetry environment and install the pre-commit hooks
@echo "🚀 Creating virtual environment using pyenv and poetry"
@cd arcade && poetry install --all-extras
@cd arcade && poetry run pre-commit install
install: ## Install the uv environment and all packages with dependencies
@echo "🚀 Creating virtual environment and installing all packages using uv workspace"
@uv sync --active --dev --extra all
@uv run pre-commit install
@echo "✅ All packages and dependencies installed via uv workspace"
.PHONY: install-toolkits
install-toolkits: ## Install dependencies for all toolkits
@echo "🚀 Installing dependencies for all toolkits"
@for dir in toolkits/*/ ; do \
echo "📦 Installing dependencies for $$dir"; \
(cd $$dir && poetry lock && poetry install); \
done
@failed=0; \
successful=0; \
for dir in toolkits/*/ ; do \
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
echo "📦 Installing dependencies for $$dir"; \
if (cd $$dir && uv pip install -e ".[dev]"); then \
successful=$$((successful + 1)); \
else \
echo "❌ Failed to install dependencies for $$dir"; \
failed=$$((failed + 1)); \
fi; \
else \
echo "⚠️ Skipping $$dir (no pyproject.toml found)"; \
fi; \
done; \
echo ""; \
echo "📊 Installation Summary:"; \
echo " ✅ Successful: $$successful toolkits"; \
echo " ❌ Failed: $$failed toolkits"; \
if [ $$failed -gt 0 ]; then \
echo ""; \
echo "⚠️ Some toolkit installations failed. Check the output above for details."; \
exit 1; \
else \
echo ""; \
echo "🎉 All toolkit dependencies installed successfully!"; \
fi
.PHONY: check
check: ## Run code quality tools.
@echo "🚀 Checking Poetry lock file consistency with 'pyproject.toml': Running poetry check --lock"
@cd arcade && poetry check --lock
@echo "🚀 Linting code: Running pre-commit"
@cd arcade && poetry run pre-commit run -a
@echo "🚀 Static type checking: Running mypy"
@cd arcade && poetry run mypy $(git ls-files '*.py')
@uv run pre-commit run -a
@echo "🚀 Static type checking: Running mypy on libs"
@for lib in libs/arcade*/ ; do \
echo "🔍 Type checking $$lib"; \
(cd $$lib && uv run mypy . || true); \
done
.PHONY: check-libs
check-libs: ## Run code quality tools for each lib package
@echo "🚀 Running checks on each lib package"
@for lib in libs/arcade*/ ; do \
echo "🛠️ Checking lib $$lib"; \
(cd $$lib && uv run pre-commit run -a || true); \
(cd $$lib && uv run mypy . || true); \
done
.PHONY: check-toolkits
check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
@ -32,7 +66,7 @@ check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
@for dir in toolkits/*/ ; do \
if [ -f "$$dir/Makefile" ]; then \
echo "🛠️ Checking toolkit $$dir"; \
(cd "$$dir" && make check); \
(cd "$$dir" && uv run --active pre-commit run -a && uv run --active mypy --config-file=pyproject.toml); \
else \
echo "🛠️ Skipping toolkit $$dir (no Makefile found)"; \
fi; \
@ -40,69 +74,129 @@ check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
.PHONY: test
test: ## Test the code with pytest
@echo "🚀 Testing code: Running pytest"
@cd arcade && poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
@echo "🚀 Testing libs: Running pytest"
@uv run pytest -W ignore -v --cov=libs/tests --cov-config=pyproject.toml --cov-report=xml
.PHONY: test-libs
test-libs: ## Test each lib package individually
@echo "🚀 Testing each lib package"
@for lib in libs/arcade*/ ; do \
echo "🧪 Testing $$lib"; \
(cd $$lib && uv run pytest -W ignore -v || true); \
done
.PHONY: test-toolkits
test-toolkits: ## Iterate over all toolkits and run pytest on each one
@echo "🚀 Testing code in toolkits: Running pytest"
@for dir in toolkits/*/ ; do \
(cd $$dir && poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml || exit 1); \
toolkit_name=$$(basename "$$dir"); \
echo "🧪 Testing $$toolkit_name toolkit"; \
(cd $$dir && uv run --active pytest -W ignore -v --cov=arcade_$$toolkit_name --cov-report=xml || exit 1); \
done
.PHONY: coverage
coverage: ## Generate coverage report
@echo "coverage report"
@cd arcade && coverage report
@uv run coverage report
@echo "Generating coverage report"
@cd arcade && coverage html
@uv run coverage html
.PHONY: set-version
set-version: ## Set the version in the pyproject.toml file
@echo "🚀 Setting version in pyproject.toml"
@cd arcade && poetry version $(VERSION)
set-version: ## Set the version in all lib pyproject.toml files
@echo "🚀 Setting versions in all lib packages"
@echo "Setting arcade-ai version to $(CLI_VERSION)"
@sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(CLI_VERSION)/' pyproject.toml && rm pyproject.toml.bak
@echo "Setting libs/arcade-tdk version to $(TDK_VERSION)"
@cd libs/arcade-tdk && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(TDK_VERSION)/' pyproject.toml && rm pyproject.toml.bak
@echo "Setting libs/arcade-serve version to $(SERVE_VERSION)"
@cd libs/arcade-serve && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(SERVE_VERSION)/' pyproject.toml && rm pyproject.toml.bak
@echo "Setting libs/arcade-core version to $(CORE_VERSION)"
@cd libs/arcade-core && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(CORE_VERSION)/' pyproject.toml && rm pyproject.toml.bak
.PHONY: unset-version
unset-version: ## Set the version in the pyproject.toml file
@echo "🚀 Setting version in pyproject.toml"
@cd arcade && poetry version 0.1.0
unset-version: ## Reset version to 0.1.0 in all lib pyproject.toml files
@echo "🚀 Resetting version to 0.1.0 in all lib packages"
@for lib in libs/arcade*/ ; do \
if [ -f "$$lib/pyproject.toml" ]; then \
echo "Resetting version in $$lib"; \
(cd $$lib && sed -i.bak 's/version = "[^"]*"/version = "0.1.0"/' pyproject.toml && rm pyproject.toml.bak); \
fi; \
done
.PHONY: build
build: clean-build ## Build wheel file using poetry
@echo "🚀 Creating wheel file"
@cd arcade && poetry build
build: clean-build ## Build wheel files using uv
@echo "🚀 Creating wheel files for all lib packages"
@for lib in libs/arcade*/ ; do \
if [ -f "$$lib/pyproject.toml" ]; then \
echo "🛠️ Building $$lib"; \
(cd $$lib && uv build); \
fi; \
done
.PHONY: build-toolkits
build-toolkits: ## Build wheel files for all toolkits
@echo "🚀 Creating wheel files for all toolkits"
@failed=0; \
successful=0; \
for dir in toolkits/*/ ; do \
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
toolkit_name=$$(basename "$$dir"); \
echo "🛠️ Building toolkit $$toolkit_name"; \
if (cd $$dir && uv build); then \
successful=$$((successful + 1)); \
else \
echo "❌ Failed to build toolkit $$toolkit_name"; \
failed=$$((failed + 1)); \
fi; \
else \
echo "⚠️ Skipping $$dir (no pyproject.toml found)"; \
fi; \
done; \
echo ""; \
echo "📊 Build Summary:"; \
echo " ✅ Successful: $$successful toolkits"; \
echo " ❌ Failed: $$failed toolkits"; \
if [ $$failed -gt 0 ]; then \
echo ""; \
echo "⚠️ Some toolkit builds failed. Check the output above for details."; \
exit 1; \
else \
echo ""; \
echo "🎉 All toolkit wheels built successfully!"; \
fi
.PHONY: clean-build
clean-build: ## clean build artifacts
@cd arcade && rm -rf dist
@echo "🗑️ Cleaning build artifacts"
@for lib in libs/arcade*/ ; do \
(cd $$lib && rm -rf dist); \
done
.PHONY: publish
publish: ## publish a release to pypi.
@echo "🚀 Publishing: Dry run."
@cd arcade && poetry config pypi-token.pypi $(PYPI_TOKEN)
@cd arcade && poetry publish --dry-run
@echo "🚀 Publishing."
@cd arcade && poetry publish
@echo "🚀 Publishing all lib packages to PyPI"
@for lib in libs/arcade*/ ; do \
if [ -f "$$lib/pyproject.toml" ]; then \
echo "📦 Publishing $$lib"; \
(cd $$lib && uv publish --token $(PYPI_TOKEN) || true); \
fi; \
done
.PHONY: build-and-publish
build-and-publish: build publish ## Build and publish.
.PHONY: docker
docker: ## Build and run the Docker container
@echo "🚀 Building arcade and toolkit wheels..."
@echo "🚀 Building lib packages and toolkit wheels..."
@make full-dist
@echo "Writing requirements.txt"
@cd arcade && poetry export --output ../dist/requirements.txt
@echo "🚀 Building Docker image"
@cd docker && make docker-build
@cd docker && make docker-run
.PHONY: docker-base
docker-base: ## Build and run the Docker container
@echo "🚀 Building arcade and toolkit wheels..."
@echo "🚀 Building lib packages and toolkit wheels..."
@make full-dist
@echo "Writing requirements.txt"
@cd arcade && poetry export --output ../dist/requirements.txt
@echo "🚀 Building Docker image"
@cd docker && INSTALL_TOOLKITS=false make docker-build
@cd docker && INSTALL_TOOLKITS=false make docker-run
@ -123,31 +217,42 @@ publish-ghcr: ## Publish to the GHCR
.PHONY: full-dist
full-dist: clean-dist ## Build all projects and copy wheels to ./dist
@echo " Building a full distribution with toolkits"
@echo "🛠️ Building a full distribution with lib packages and toolkits"
@echo "Setting version to $(VERSION)"
@echo "Setting version to $(CLI_VERSION)"
@make set-version
# @echo "🛠️ Building all projects and copying wheels to ./dist"
@echo "🛠️ Building all lib packages and copying wheels to ./dist"
@mkdir -p dist
# Build the main arcade project
@echo "🛠️ Building arcade project wheel..."
@cd arcade && poetry build
# Build all lib packages in dependency order
@for lib in arcade-core arcade-tdk arcade-serve ; do \
echo "🛠️ Building libs/$$lib wheel..."; \
(cd libs/$$lib && uv build); \
cp libs/$$lib/dist/*.whl dist/; \
done
# Copy the main arcade project wheel to the dist directory
@cp arcade/dist/*.whl dist/
@echo "🛠️ Building all toolkit packages and copying wheels to ./dist"
@for dir in toolkits/*/ ; do \
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
toolkit_name=$$(basename "$$dir"); \
echo "🛠️ Building toolkit $$toolkit_name wheel..."; \
(cd $$dir && uv build); \
cp $$dir/dist/*.whl dist/; \
fi; \
done
@echo "Reset version to default (0.1.0)"
@make unset-version
.PHONY: clean-dist
clean-dist: ## Clean all built distributions
@echo "🗑️ Cleaning dist directory"
@rm -rf dist
@echo "🗑️ Cleaning arcade/dist directory"
@rm -rf arcade/dist
@echo "🗑️ Cleaning libs/*/dist directories"
@for lib in libs/arcade*/ ; do \
rm -rf "$$lib"/dist; \
done
@echo "🗑️ Cleaning toolkits/*/dist directory"
@for toolkit_dir in toolkits/*; do \
if [ -d "$$toolkit_dir" ]; then \
@ -155,11 +260,18 @@ clean-dist: ## Clean all built distributions
fi; \
done
.PHONY: setup
setup: install ## Complete development setup (same as install)
.PHONY: lint
lint: check ## Alias for check command
.PHONY: clean
clean: clean-build clean-dist ## Clean all build and distribution artifacts
.PHONY: help
help:
@echo "🛠️ Arcade Dev Commands:\n"
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.DEFAULT_GOAL := help

52
README.md
View file

@ -38,11 +38,17 @@
<a href="https://docs.arcade.dev/home/quickstart" target="_blank">Quickstart</a> •
<a href="https://docs.arcade.dev/home/contact-us" target="_blank">Contact Us</a>
# Arcade Tool SDK
# Arcade AI Platform
Arcade is a developer platform that lets you build, deploy, and manage tools for AI agents.
The Tool SDK makes it easy to create powerful, secure tools that your agents can use to interact with the world.
This repository contains the core Arcade libraries, organized as separate packages for maximum flexibility and modularity:
- [**`arcade-core`**](libs/arcade-core) - Core platform functionality and schemas
- [**`arcade-tdk`**](libs/arcade-tdk) - Tool Development Kit with the `@tool` decorator
- [**`arcade-serve`**](libs/arcade-serve) - Serving infrastructure for workers and MCP servers
- [**`arcade-evals`**](libs/arcade-evals) - Evaluation framework for testing tool performance
- [**`arcade-cli`**](libs/arcade-cli) - Command-line interface for the Arcade platform
![diagram](https://github.com/user-attachments/assets/1a567e5f-d6b4-4b1e-9918-c401ad232ebb)
@ -54,6 +60,48 @@ _Pst. hey, you, give us a star if you like it!_
<img src="https://img.shields.io/github/stars/ArcadeAI/arcade-ai.svg" alt="GitHub stars">
</a>
## Quick Start
### Installation
For development, install all packages with dependencies using uv workspace:
```bash
# Install all packages and dev dependencies
uv sync --extra all --dev
# Or use the Makefile (includes pre-commit hooks)
make install
```
For production use, install individual packages as needed:
```bash
pip install arcade-ai # CLI
pip install 'arcade-ai[evals]' # CLI + Evaluation framework
pip install 'arcade-ai[all]' # CLI + Serving infra + eval framework + TDK
pip install arcade_serve # Serving infrastructure
pip install arcade-tdk # Tool Development Kit
```
### Development
Use the Makefile for standard tasks:
```bash
# Run tests
make test
# Run linting and type checking
make check
# Build all packages
make build
# See all available commands
make help
```
## Client Libraries
- **[ArcadeAI/arcade-py](https://github.com/ArcadeAI/arcade-py):**

13
arcade/README.md
View file

@ -1,13 +0,0 @@
# Arcade Python SDK and CLI
[Arcade](https://arcade.dev?ref=pypi) provides developer-focused tooling and APIs designed to improve the capabilities of LLM applications and agents.
By removing the complexity of connecting agentic applications with your users' data and services, Arcade enables developers to focus on building their agentic applications.
To learn more, check out our
- [Website](https://arcade.dev?ref=pypi)
- [GitHub](https://github.com/ArcadeAI/arcade-ai)
- [Documentation](https://docs.arcade.dev)
- [Discord](https://discord.com/invite/GUZEMpEZ9p)
- [X](https://x.com/TryArcade)
- [LinkedIn](https://www.linkedin.com/company/arcade-ai)

3
arcade/arcade/__init__.py
View file

@ -1,3 +0,0 @@
from importlib.metadata import version as get_version
__version__ = get_version("arcade-ai")

505
arcade/arcade/cli/launcher.py
View file

@ -1,505 +0,0 @@
import http.client
import io
import ipaddress
import logging
import os
import shutil
import signal
import socket
import subprocess
import sys
import threading
import time
from pathlib import Path
from typing import Callable
from rich.console import Console
console = Console(highlight=False)
logger = logging.getLogger(__name__)
known_engine_config_locations = [
"/etc/arcade-ai",
"/etc/arcade-engine",
"/opt/homebrew/etc/arcade-engine",
]
if os.environ.get("HOMEBREW_REPOSITORY") is not None:
homebrew_home = os.path.join(os.environ["HOMEBREW_REPOSITORY"], "etc", "arcade-engine")
if homebrew_home not in known_engine_config_locations:
known_engine_config_locations.append(homebrew_home)
def start_servers(
worker_host: str,
worker_port: int,
engine_config: str | None,
engine_env: str | None = None,
debug: bool = False,
) -> None:
"""
Start the worker and engine servers.
Args:
host: Host for the worker server.
port: Port for the worker server.
engine_config: Path to the engine configuration file.
engine_env: Path to the engine environment file.
debug: Whether to run in debug mode.
"""
# Validate host and port
worker_host = _validate_host(worker_host)
worker_port = _validate_port(worker_port)
# Ensure engine_config is provided and validated
engine_config = _get_config_file(engine_config, default_filename="engine.yaml")
# Ensure engine_env is provided or found and either way, validated
env_file = _get_config_file(engine_env, default_filename="engine.env", optional=True)
# Prepare command-line arguments for the worker server and engine
worker_cmd = _build_worker_command(worker_host, worker_port, debug)
# even if the user didn't pass an env file we may have found it in the default locations
engine_cmd = _build_engine_command(engine_config, engine_env=env_file if env_file else None)
# Start and manage the processes
_manage_processes(worker_cmd, worker_host, worker_port, engine_cmd, debug=debug)
def _validate_host(host: str) -> str:
"""
Validates the host input.
Args:
host: Host for the worker server.
Returns:
The validated host as a string.
Raises:
ValueError: If the host is invalid.
"""
try:
# Validate IP address
ipaddress.ip_address(host)
except ValueError:
# Optionally, validate hostname
if not host.isalnum() and "-" not in host and "." not in host:
console.print(f"❌ Invalid host: {host}", style="bold red")
raise ValueError("Invalid host.")
return host
def _validate_port(port: int) -> int:
"""
Validates the port input.
Args:
port: Port for the worker server.
Returns:
The validated port as an integer.
Raises:
ValueError: If the port is out of the valid range.
"""
if not (1 <= port <= 65535):
console.print(f"❌ Invalid port: {port}", style="bold red")
raise ValueError("Invalid port.")
return port
def _get_config_file(
file_path: str | None, default_filename: str = "engine.yaml", optional: bool = False
) -> str | None:
"""
Resolves and validates the config file path from a set of candidate locations.
If a file_path is provided, it is checked directly.
Otherwise, the following candidate locations are checked in order:
1. Current working directory.
2. User's home directory under .arcade.
3. Known engine config locations.
Args:
file_path: Optional path provided by the user.
default_filename: The default filename to look for.
optional: Whether the config file is optional.
Returns:
The resolved config file path. None if the file is optional and not found.
Raises:
RuntimeError: If the config file is not found and is not optional.
"""
if file_path:
candidate = Path(os.path.expanduser(file_path)).resolve()
if not candidate.is_file():
console.print(f"❌ Config file not found at {candidate}", style="bold red")
raise RuntimeError(f"Config file not found at {candidate}")
return str(candidate)
# List of all config file path locations to check.
candidates = [
Path(os.getcwd()) / default_filename,
Path.home() / ".arcade" / default_filename,
]
candidates.extend(Path(path) / default_filename for path in known_engine_config_locations)
# Find the first candidate that exists.
for candidate in candidates:
if candidate.is_file():
console.print(f"Using config file at {candidate}", style="bold green")
return str(candidate)
# No config file was found. Handle according to the optional flag.
if optional:
console.print(
f"⚠️ Optional config file '{default_filename}' not found in any of the following locations:",
style="bold yellow",
)
for i, candidate in enumerate(candidates, start=1):
console.print(f" {i}) {candidate}", style="bold yellow")
return None
console.print(
f"❌ Error: Required config file '{default_filename}' not found in any of the following locations:",
style="bold red",
)
for i, candidate in enumerate(candidates, start=1):
console.print(f" {i}) {candidate}", style="bold red")
console.print(
"\nTIP: Please install the Arcade Engine by following the instructions at:\n"
" https://docs.arcade.dev/home/install/local#install-the-engine\n",
style="bold green",
)
raise RuntimeError(f"Config file '{default_filename}' not found.")
def _build_worker_command(host: str, port: int, debug: bool) -> list[str]:
"""
Builds the command to start the worker server.
Args:
host: Host for the worker server.
port: Port for the worker server.
debug: Whether to run in debug mode.
Returns:
The command as a list.
"""
# Expand full path to "arcade" executable
arcade_bin = shutil.which("arcade")
if not arcade_bin:
console.print(
"❌ Arcade binary not found, please install with `pip install arcade-ai`",
style="bold red",
)
sys.exit(1)
cmd = [
arcade_bin,
"workerup",
"--host",
host,
"--port",
str(port),
]
if debug:
cmd.append("--debug")
return cmd
def _build_engine_command(engine_config: str | None, engine_env: str | None = None) -> list[str]:
"""
Builds the command to start the engine.
Args:
engine_config: Path to the engine configuration file.
engine_env: Path to the engine environment file.
Returns:
The command as a list.
"""
# This should never happen, but we'll check regardless
if not engine_config:
console.print("❌ Engine configuration file not found", style="bold red")
sys.exit(1)
engine_bin = shutil.which("arcade-engine")
if not engine_bin:
console.print(
"❌ Engine binary not found, refer to the installation guide at "
"https://docs.arcade.dev/guides/installation for how to install the engine",
style="bold red",
)
sys.exit(1)
cmd = [
engine_bin,
"-c",
engine_config,
]
if engine_env:
cmd.append("-e")
cmd.append(engine_env)
return cmd
def _manage_processes(
worker_cmd: list[str],
worker_host: str,
worker_port: int,
engine_cmd: list[str],
engine_env: dict[str, str] | None = None,
debug: bool = False,
) -> None:
"""
Manages the lifecycle of the worker and engine processes.
Args:
worker_cmd: The command to start the worker server.
engine_cmd: The command to start the engine.
engine_env: Environment variables to set for the engine.
debug: Whether to run in debug mode.
"""
worker_process: subprocess.Popen | None = None
engine_process: subprocess.Popen | None = None
def terminate_processes(exit_program: bool = False) -> None:
console.print("Terminating child processes...", style="bold yellow")
_terminate_process(worker_process)
_terminate_process(engine_process)
if exit_program:
sys.exit(0)
_setup_signal_handlers(terminate_processes)
retry_count = 0
max_retries = 1 # Define the maximum number of retries
while retry_count <= max_retries:
try:
# Start the worker server
console.print("Starting worker server...", style="bold green")
worker_process = _start_process("Worker", worker_cmd, debug=debug)
_wait_for_healthy_worker(worker_process, worker_host, worker_port)
# Start the engine
console.print("Starting engine...", style="bold green")
engine_process = _start_process("Engine", engine_cmd, env=engine_env, debug=debug)
# Monitor processes
_monitor_processes(worker_process, engine_process)
# If we reach here, one of the processes has exited
retry_count += 1
console.print(
f"Processes exited. Retry {retry_count} of {max_retries}.", style="bold yellow"
)
if retry_count >= max_retries:
console.print(f"❌ Exiting after {max_retries} retries", style="bold red")
terminate_processes(exit_program=True)
break # Exit the loop
except Exception as e:
console.print(f"❌ Exception occurred: {e}", style="bold red")
terminate_processes()
retry_count += 1
if retry_count > max_retries:
console.print(
f"❌ Exiting after {retry_count - 1} retries due to exceptions",
style="bold red",
)
sys.exit(1)
break # Not strictly necessary, but good practice
console.print("Exiting...", style="bold red")
sys.exit(1)
def _start_process(
name: str, cmd: list[str], env: dict[str, str] | None = None, debug: bool = False
) -> subprocess.Popen:
"""
Starts a subprocess and begins streaming its output.
Args:
name: Name of the process.
cmd: Command to execute.
env: Environment variables to set for the process.
debug: Whether to run in debug mode.
Returns:
The subprocess.Popen object.
Raises:
RuntimeError: If the process fails to start.
"""
_env = os.environ.copy()
if env:
_env.update(env)
if debug:
_env["GIN_MODE"] = "debug"
else:
_env["GIN_MODE"] = "release"
if name == "Worker":
_env["PYTHONUNBUFFERED"] = "1"
try:
process = subprocess.Popen( # noqa: S603, RUF100
cmd,
env=_env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True,
bufsize=1,
shell=False,
)
_stream_output(process, name)
return process # noqa: TRY300
except Exception as e:
console.print(f"❌ Failed to start {name}: {e}", style="bold red")
raise RuntimeError(f"Failed to start {name}")
def _wait_for_healthy_worker(
worker_process: subprocess.Popen, worker_host: str, worker_port: int
) -> None:
"""Wait until an HTTP request to `host:port/worker/health` returns 200"""
while worker_process.poll() is None: # Continue waiting UNLESS the worker process has exited
time.sleep(1)
try:
conn = http.client.HTTPConnection(worker_host, worker_port, timeout=1)
conn.request("GET", "/worker/health")
res = conn.getresponse()
if res.status == 200:
break
conn.close()
except (socket.gaierror, http.client.HTTPException, ConnectionRefusedError, TimeoutError):
pass # Handle expected exceptions gracefully
console.print("Waiting for worker to start...", style="bold yellow")
time.sleep(1) # Wait just a little longer for everything to settle (discovered experimentally)
console.print("Worker is healthy", style="bold green")
def _stream_output(process: subprocess.Popen, name: str) -> None:
"""
Streams the output from a subprocess to the console.
Args:
process: The subprocess.Popen object.
name: Name of the process.
"""
stdout_style = "green" if name == "Worker" else "#87CEFA"
def stream(pipe: io.TextIOWrapper | None, style: str) -> None:
if pipe is None:
return
with pipe:
for line in iter(pipe.readline, ""):
line = line.rstrip()
if "DEBUG" in line:
line = line.replace("DEBUG", "[#87CEFA]DEBUG[/#87CEFA]", 1)
if "INFO" in line:
line = line.replace("INFO", "[#109a10]INFO[/#109a10]", 1)
if "WARNING" in line:
line = line.replace("WARNING", "[#FFA500]WARNING[/#FFA500]", 1)
if "ERROR" in line:
line = line.replace("ERROR", "[#FF0000]ERROR[/#FF0000]", 1)
console.print(f"[{style}]{name}>[/{style}] {line}")
threading.Thread(target=stream, args=(process.stdout, stdout_style), daemon=True).start()
threading.Thread(target=stream, args=(process.stderr, "red"), daemon=True).start()
def _monitor_processes(worker_process: subprocess.Popen, engine_process: subprocess.Popen) -> None:
"""
Monitors the worker and engine processes, restarts them if they exit.
Args:
worker_process: The worker subprocess.
engine_process: The engine subprocess.
"""
while True:
worker_status = worker_process.poll()
engine_status = engine_process.poll()
if worker_status is not None or engine_status is not None:
if worker_status is not None:
console.print(
f"Worker process exited with code {worker_status}. Restarting both processes...",
style="bold red",
)
if engine_status is not None:
console.print(
f"Engine process exited with code {engine_status}. Restarting both processes...",
style="bold red",
)
_terminate_process(worker_process)
_terminate_process(engine_process)
time.sleep(1)
break # Exit to restart both processes
else:
time.sleep(1)
def _terminate_process(process: subprocess.Popen | None) -> None:
"""
Terminates a subprocess if it's running.
Args:
process: The subprocess.Popen object.
"""
if process and process.poll() is None:
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
def _setup_signal_handlers(terminate_processes: Callable[[bool], None]) -> None:
"""
Setup signal handlers to handle process termination signals.
Args:
terminate_processes: Function to call to terminate child processes.
"""
signals_to_handle = ["SIGINT", "SIGTERM", "SIGQUIT", "SIGHUP"]
for sig_name in signals_to_handle:
sig = getattr(signal, sig_name, None)
if sig is None:
continue # Signal not available on this platform
try:
# Use a lambda to pass the terminate_processes function
signal.signal(
sig,
lambda signum, frame: _handle_signal(signum, terminate_processes),
)
except (ValueError, RuntimeError):
# Signal handling not allowed in this thread or invalid signal
console.print(f"Warning: Cannot set handler for {sig_name}", style="bold yellow")
continue
def _handle_signal(signum: int, terminate_processes: Callable[[bool], None]) -> None:
"""
Handle received signal and terminate child processes.
Args:
signum: The signal number received.
terminate_processes: Function to call to terminate child processes.
"""
signal_name = signal.Signals(signum).name
console.print(f"Received {signal_name}. Shutting down...", style="bold yellow")
terminate_processes(exit_program=True) # type: ignore[call-arg]

143
arcade/arcade/cli/new.py
View file

@ -1,143 +0,0 @@
import re
import shutil
from datetime import datetime
from importlib.metadata import version as get_version
from pathlib import Path
from typing import Optional
import typer
from jinja2 import Environment, FileSystemLoader, select_autoescape
from rich.console import Console
from arcade.cli.deployment import (
create_demo_deployment,
)
console = Console()
# Retrieve the installed version of arcade-ai
try:
ARCADE_VERSION = get_version("arcade-ai")
except Exception as e:
console.print(f"[red]Failed to get arcade-ai version: {e}[/red]")
ARCADE_VERSION = "0.0.0" # Default version if unable to fetch
TEMPLATE_IGNORE_PATTERN = re.compile(
r"(__pycache__|\.DS_Store|Thumbs\.db|\.git|\.svn|\.hg|\.vscode|\.idea|build|dist|.*\.egg-info|.*\.pyc|.*\.pyo)$"
)
def ask_question(question: str, default: Optional[str] = None) -> str:
"""
Ask a question via input() and return the answer.
"""
answer = typer.prompt(question, default=default)
if not answer and default:
return default
return str(answer)
def render_template(env: Environment, template_string: str, context: dict) -> str:
"""Render a template string with the given variables."""
template = env.from_string(template_string)
return template.render(context)
def write_template(path: Path, content: str) -> None:
"""Write content to a file."""
path.write_text(content, encoding="utf-8")
def create_package(env: Environment, template_path: Path, output_path: Path, context: dict) -> None:
"""Recursively create a new toolkit directory structure from jinja2 templates."""
if TEMPLATE_IGNORE_PATTERN.match(template_path.name):
return
try:
if template_path.is_dir():
folder_name = render_template(env, template_path.name, context)
new_dir_path = output_path / folder_name
new_dir_path.mkdir(parents=True, exist_ok=True)
for item in template_path.iterdir():
create_package(env, item, new_dir_path, context)
else:
# Render the file name
file_name = render_template(env, template_path.name, context)
with open(template_path, encoding="utf-8") as f:
content = f.read()
# Render the file content
content = render_template(env, content, context)
write_template(output_path / file_name, content)
except Exception as e:
console.print(f"[red]Failed to create package: {e}[/red]")
raise
def remove_toolkit(toolkit_directory: Path, toolkit_name: str) -> None:
"""Teardown logic for when creating a new toolkit fails."""
toolkit_path = toolkit_directory / toolkit_name
if toolkit_path.exists():
shutil.rmtree(toolkit_path)
def create_new_toolkit(output_directory: str) -> None:
"""Create a new toolkit from a template with user input."""
toolkit_directory = Path(output_directory)
while True:
name = ask_question("Name of the new toolkit?")
package_name = name if name.startswith("arcade_") else f"arcade_{name}"
# Check for illegal characters in the toolkit name
if re.match(r"^[\w_]+$", package_name):
toolkit_name = package_name.replace("arcade_", "", 1)
if (toolkit_directory / toolkit_name).exists():
console.print(f"[red]Toolkit {toolkit_name} already exists.[/red]")
continue
break
else:
console.print(
"[red]Toolkit name contains illegal characters. "
"Only alphanumeric characters and underscores are allowed. "
"Please try again.[/red]"
)
toolkit_description = ask_question("Description of the toolkit?")
toolkit_author_name = ask_question("Github owner username?")
toolkit_author_email = ask_question("Author's email?")
context = {
"package_name": package_name,
"toolkit_name": toolkit_name,
"toolkit_description": toolkit_description,
"toolkit_author_name": toolkit_author_name,
"toolkit_author_email": toolkit_author_email,
"arcade_version": f"^{ARCADE_VERSION}",
"creation_year": datetime.now().year,
}
template_directory = Path(__file__).parent.parent / "templates" / "{{ toolkit_name }}"
env = Environment(
loader=FileSystemLoader(str(template_directory)),
autoescape=select_autoescape(["html", "xml"]),
)
try:
create_package(env, template_directory, toolkit_directory, context)
create_deployment(toolkit_directory, toolkit_name)
except Exception:
remove_toolkit(toolkit_directory, toolkit_name)
raise
def create_deployment(toolkit_directory: Path, toolkit_name: str) -> None:
worker_toml = toolkit_directory / "worker.toml"
if not worker_toml.exists():
create_demo_deployment(worker_toml, toolkit_name)
else:
pass
# Disabled pending bug fix
# update_deployment_with_local_packages(worker_toml, toolkit_name)

14
arcade/arcade/sdk/__init__.py
View file

@ -1,14 +0,0 @@
from arcade.core.catalog import ToolCatalog
from arcade.core.schema import ToolAuthorizationContext, ToolContext, ToolMetadataKey
from arcade.core.toolkit import Toolkit
from .tool import tool
__all__ = [
"ToolAuthorizationContext",
"ToolCatalog",
"ToolContext",
"ToolMetadataKey",
"Toolkit",
"tool",
]

3
arcade/arcade/sdk/annotations/__init__.py
View file

@ -1,3 +0,0 @@
from arcade.core.annotations import Inferrable
__all__ = ["Inferrable"]

15
arcade/arcade/templates/{{ toolkit_name }}/.editorconfig
View file

@ -1,15 +0,0 @@
# Stop the editor from looking for .editorconfig files in the parent directories
root = true
[*]
charset = utf-8
insert_final_newline = true
end_of_line = lf
indent_style = space
indent_size = 4
max_line_length = 100 # This is also set in .ruff.toml for ruff
[*.{json,jsonc,yml,yaml}]
indent_style = space
indent_size = 2 # This is also set in .prettierrc.toml

39
arcade/arcade/templates/{{ toolkit_name }}/.github/actions/setup-poetry-env/action.yml vendored
View file

@ -1,39 +0,0 @@
name: "setup-poetry-env"{% raw %}
description: "Composite action to setup the Python and poetry environment."
inputs:
python-version:
required: false
description: "The python version to use"
default: "3.11"
runs:
using: "composite"
steps:
- name: Set up python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.5
virtualenvs-in-project: true
- name: Generate poetry.lock
run: poetry lock --no-update
shell: bash
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-${{ runner.os }}-${{ inputs.python-version }}-${{ hashFiles('poetry.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --all-extras
shell: bash
{% endraw %}

61
arcade/arcade/templates/{{ toolkit_name }}/.github/workflows/main.yml vendored
View file

@ -1,61 +0,0 @@
name: Main{% raw %}
on:
push:
branches:
- main
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
jobs:
quality:
runs-on: ubuntu-latest
steps:
- name: Check out
uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Set up the environment
uses: ./.github/actions/setup-poetry-env
- name: Run checks
run: make check
tox:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
fail-fast: false
steps:
- name: Check out
uses: actions/checkout@v4
- name: Set up python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.5
- name: Load cached venv
uses: actions/cache@v4
with:
path: .tox
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('poetry.lock') }}
- name: Install tox
run: |
python -m pip install --upgrade pip
python -m pip install tox tox-gh-actions
- name: Test with tox
run: tox
{% endraw %}

38
arcade/arcade/templates/{{ toolkit_name }}/.github/workflows/publish-to-pypi.yml vendored
View file

@ -1,38 +0,0 @@
name: Publish to PyPI
on:
release:
types: [published]
jobs:
pypi-publish:
name: Publish to PyPi
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/project/{{ package_name }}
permissions:
id-token: write
steps:
- name: Check out
uses: actions/checkout@v4
- name: Set up python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build package
run: python -m build
- name: Publish to PyPI
env:
TWINE_USERNAME: "__token__"{% raw %}
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
run: twine upload dist/*
{% endraw %}

167
arcade/arcade/templates/{{ toolkit_name }}/.gitignore vendored
View file

@ -1,167 +0,0 @@
.DS_Store
*.lock
# From https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

19
arcade/arcade/templates/{{ toolkit_name }}/.pre-commit-config.yaml
View file

@ -1,19 +0,0 @@
files: ^./
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: "v4.4.0"
hooks:
- id: check-case-conflict
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.7
hooks:
- id: ruff
args: [--fix]
- id: ruff-format

3
arcade/arcade/templates/{{ toolkit_name }}/.prettierignore
View file

@ -1,3 +0,0 @@
# Ignore Python files for Prettier
*.py

16
arcade/arcade/templates/{{ toolkit_name }}/.prettierrc.toml
View file

@ -1,16 +0,0 @@
# See https://prettier.io/docs/en/configuration
# Note: This prettier config is only for the non-python files in this repo.
# Python files are formatted with ruff and ignored in .prettierignore
trailingComma = "es5"
tabWidth = 4
semi = false
singleQuote = false
[[overrides]]
files = [ "*.json", "*.jsonc", "*.yml", "*.yaml" ]
[overrides.options]
tabWidth = 2

45
arcade/arcade/templates/{{ toolkit_name }}/.ruff.toml
View file

@ -1,45 +0,0 @@
target-version = "py310"
line-length = 100
fix = true
[lint]
select = [
# flake8-2020
"YTT",
# flake8-bandit
"S",
# flake8-bugbear
"B",
# flake8-builtins
"A",
# flake8-comprehensions
"C4",
# flake8-debugger
"T10",
# flake8-simplify
"SIM",
# isort
"I",
# mccabe
"C90",
# pycodestyle
"E", "W",
# pyflakes
"F",
# pygrep-hooks
"PGH",
# pyupgrade
"UP",
# ruff
"RUF",
# tryceratops
"TRY",
]
[lint.per-file-ignores]
"**/tests/*" = ["S101"]
[format]
preview = true
skip-magic-trailing-comma = false

22
arcade/arcade/templates/{{ toolkit_name }}/LICENSE
View file

@ -1,22 +0,0 @@
MIT License
Copyright (c) {{ creation_year }}, {{ toolkit_author_name }}
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

59
arcade/arcade/templates/{{ toolkit_name }}/Makefile
View file

@ -1,59 +0,0 @@
.PHONY: help
help:
@echo "🛠️ {{ toolkit_name }} Commands:\n"
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.PHONY: install
install: ## Install the poetry environment and install the pre-commit hooks
@echo "📦 Checking if Poetry is installed"
@if ! command -v poetry >/dev/null 2>&1; then \
echo "📦 Poetry not found. Checking if pip is available"; \
if ! command -v pip >/dev/null 2>&1; then \
echo "❌ pip is not installed. Please install pip first."; \
exit 1; \
fi; \
echo "📦 Installing Poetry with pip"; \
pip install poetry==1.8.5; \
else \
echo "📦 Poetry is already installed"; \
fi
@echo "🚀 Installing package in development mode with all extras"
poetry install --all-extras
.PHONY: build
build: clean-build ## Build wheel file using poetry
@echo "🚀 Creating wheel file"
poetry build
.PHONY: clean-build
clean-build: ## clean build artifacts
@echo "🗑️ Cleaning dist directory"
rm -rf dist
.PHONY: test
test: ## Test the code with pytest
@echo "🚀 Testing code: Running pytest"
@poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
.PHONY: coverage
coverage: ## Generate coverage report
@echo "coverage report"
coverage report
@echo "Generating coverage report"
coverage html
.PHONY: bump-version
bump-version: ## Bump the version in the pyproject.toml file
@echo "🚀 Bumping version in pyproject.toml"
poetry version patch
.PHONY: check
check: ## Run code quality tools.
@echo "🚀 Checking Poetry lock file consistency with 'pyproject.toml': Running poetry check"
@poetry check
@echo "🚀 Linting code: Running pre-commit"
@poetry run pre-commit run -a
@echo "🚀 Static type checking: Running mypy"
@poetry run mypy --config-file=pyproject.toml

10
arcade/arcade/templates/{{ toolkit_name }}/codecov.yaml
View file

@ -1,10 +0,0 @@
coverage:
range: 70..100
round: down
precision: 1
status:
project:
default:
target: 90%
threshold: 0.5%

40
arcade/arcade/templates/{{ toolkit_name }}/pyproject.toml
View file

@ -1,40 +0,0 @@
[tool.poetry]
name = "{{ package_name }}"
version = "0.0.1"
description = "{{ toolkit_description }}"
authors = ["{{ toolkit_author_name }} <{{ toolkit_author_email }}>"]
[tool.poetry.dependencies]
python = "^3.10"
arcade-ai = "^1.0.5"
[tool.poetry.dev-dependencies]
pytest = "^8.3.0"
pytest-cov = "^4.0.0"
mypy = "^1.5.1"
pre-commit = "^3.4.0"
tox = "^4.11.1"
ruff = "^0.7.4"
[build-system]
requires = ["poetry-core>=1.0.0,<2.0.0"]
build-backend = "poetry.core.masonry.api"
[tool.mypy]
files = ["{{ package_name }}/**/*.py"]
python_version = "3.10"
disallow_untyped_defs = "True"
disallow_any_unimported = "True"
no_implicit_optional = "True"
check_untyped_defs = "True"
warn_return_any = "True"
warn_unused_ignores = "True"
show_error_codes = "True"
ignore_missing_imports = "True"
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.coverage.report]
skip_empty = true

17
arcade/arcade/templates/{{ toolkit_name }}/tox.ini
View file

@ -1,17 +0,0 @@
[tox]
skipsdist = true
envlist = py310, py311, py312
[gh-actions]
python =
3.10: py310
3.11: py311
3.12: py312
[testenv]
passenv = PYTHON_VERSION
allowlist_externals = poetry
commands =
poetry install -v --all-extras
pytest --doctest-modules tests --cov --cov-config=pyproject.toml --cov-report=xml

11
arcade/codecov.yaml
View file

@ -1,11 +0,0 @@
coverage:
range: 70..100
round: down
precision: 1
status:
project:
default:
target: 90%
threshold: 0.5%
exclude:
- arcade/cli/**

88
arcade/pyproject.toml
View file

@ -1,88 +0,0 @@
[tool.poetry]
name = "arcade-ai"
version = "1.1.0"
description = "Arcade Python SDK and CLI"
readme = "README.md"
packages = [
{include="arcade", from="."}
]
authors = ["Arcade <dev@arcade.dev>"]
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
[tool.poetry.dependencies]
python = ">=3.10,<4.0"
pydantic = "^2.7.0"
typer = ">=0.10.0"
rich = "^13.7.1"
Jinja2 = ">=3.1.5,<4.0.0"
pyyaml = "^6.0"
openai = "^1.36.0" # TODO: relax to an earlier version that still has what we need
arcadepy = "^1.3.1"
pyjwt = "^2.8.0"
loguru = "^0.7.0"
tqdm = "^4.1.0"
toml = "^0.10.2"
packaging = "^24.1"
types-python-dateutil = "2.9.0.20241003"
types-pytz = "2024.2.0.20241003"
types-toml = "0.10.8.20240310"
opentelemetry-instrumentation-fastapi = "0.48b0"
opentelemetry-exporter-otlp-proto-http = "1.27.0"
opentelemetry-exporter-otlp-proto-common = "1.27.0"
fastapi = "^0.115.3"
uvicorn = "^0.30.0"
scipy = {version = "^1.14.0", optional = true}
numpy = {version = "^2.0.0", optional = true}
scikit-learn = {version = "^1.5.0", optional = true}
pytz = {version = "^2024.1", optional = true}
python-dateutil = {version = "^2.8.2", optional = true}
watchfiles = "^1.0.5"
pyreadline3 = {version = "^3.5.4", platform = "win32"}
[tool.poetry.extras]
evals = ["scipy", "numpy", "scikit-learn", "pytz", "python-dateutil"]
[tool.poetry.group.dev.dependencies]
pytest = "^8.1.2"
pytest-cov = "^4.0.0"
mypy = "^1.5.1"
pre-commit = "^3.4.0"
tox = "^4.11.1"
pytest-asyncio = "^0.23.7"
types-pytz = "^2024.1"
types-python-dateutil = "^2.8.2"
types-PyYAML = "^6.0.0"
poetry-plugin-export = "^1.7.0"
[tool.poetry.scripts]
arcade = "arcade.cli.main:cli"
[tool.mypy]
files = ["arcade"]
exclude = "arcade/templates"
python_version = "3.10"
disallow_untyped_defs = "True"
disallow_any_unimported = "True"
no_implicit_optional = "True"
check_untyped_defs = "True"
warn_return_any = "True"
warn_unused_ignores = "True"
show_error_codes = "True"
ignore_missing_imports = "True"
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.coverage.run]
branch = true
source = ["arcade"]
omit = ["arcade/cli/*"]
[tool.coverage.report]
skip_empty = true

17
arcade/tox.ini
View file

@ -1,17 +0,0 @@
[tox]
skipsdist = true
envlist = py310, py311, py312
[gh-actions]
python =
3.10: py310
3.11: py311
3.12: py312
[testenv]
passenv = PYTHON_VERSION
allowlist_externals = poetry
commands =
poetry install -v --all-extras
pytest --doctest-modules tests --cov --cov-config=pyproject.toml --cov-report=xml
# mypy

5
examples/mcp/run_stdio.py
View file

@ -1,8 +1,7 @@
import arcade_google # pip install arcade_google
import arcade_search # pip install arcade_search
from arcade.core.catalog import ToolCatalog
from arcade.worker.mcp.stdio import StdioServer
from arcade_core.catalog import ToolCatalog
from arcade_serve.mcp.stdio import StdioServer
# 2. Create and populate the tool catalog
catalog = ToolCatalog()

9
examples/serving-tools/modal/run-arcade-worker.py
View file

@ -7,17 +7,18 @@ app = App("arcade-worker")
toolkits = ["arcade_google", "arcade_slack"]
image = Image.debian_slim().pip_install("arcade-ai").pip_install(toolkits)
image = (
Image.debian_slim().pip_install("arcade_tdk").pip_install("arcade_serve").pip_install(toolkits)
)
@app.function(image=image)
@asgi_app()
def fastapi_app():
from arcade_serve.fastapi.worker import FastAPIWorker
from arcade_tdk import Toolkit
from fastapi import FastAPI
from arcade.sdk import Toolkit
from arcade.worker.fastapi.worker import FastAPIWorker
web_app = FastAPI()
# Initialize app and Arcade FastAPIWorker

28
libs/arcade-cli/README.md Normal file
View file

@ -0,0 +1,28 @@
# Arcade CLI
Command-line interface for the Arcade platform.
## Overview
Arcade CLI provides a comprehensive command-line interface for the Arcade platform:
- **User Authentication**: Login, logout
- **Tool Development**: Create, test, and manage Arcade tools
- **Worker Deployment**: Deploy and manage Arcade workers
- **Interactive Chat**: Test tools in an interactive environment
- **Project Templates**: Generate new toolkit projects
## Installation
```bash
pip install arcade-ai
```
## Usage
Learn how to use the Arcade CLI and what commands are available to you.
```bash
arcade --help
```
## License
MIT License - see LICENSE file for details.

0
arcade/arcade/cli/__init__.py → libs/arcade-cli/arcade_cli/__init__.py
View file

2
arcade/arcade/cli/authn.py → libs/arcade-cli/arcade_cli/authn.py
View file

@ -7,7 +7,7 @@ from urllib.parse import parse_qs
import yaml
from rich.console import Console
from arcade.cli.constants import (
from arcade_cli.constants import (
ARCADE_CONFIG_PATH,
CREDENTIALS_FILE_PATH,
LOGIN_FAILED_HTML,

104
libs/arcade-cli/arcade_cli/config.py Normal file
View file

@ -0,0 +1,104 @@
"""
Configuration utilities for the Arcade CLI.
"""
from dataclasses import dataclass
from typing import Any
from rich.console import Console
from rich.table import Table
console = Console()
@dataclass
class UserConfig:
"""User configuration."""
email: str | None = None
name: str | None = None
@dataclass
class ApiConfig:
"""API configuration."""
key: str | None = None
url: str | None = None
@dataclass
class Config:
"""Arcade CLI configuration."""
user: UserConfig | None = None
api: ApiConfig | None = None
@classmethod
def from_dict(cls, data: dict[str, Any]) -> "Config":
"""Create a Config instance from a dictionary.
Args:
data: Dictionary with configuration
Returns:
Config instance
"""
user_data = data.get("user", {})
api_data = data.get("api", {})
return cls(
user=UserConfig(
email=user_data.get("email"),
name=user_data.get("name"),
),
api=ApiConfig(
key=api_data.get("key"),
url=api_data.get("url"),
),
)
def to_dict(self) -> dict[str, Any]:
"""Convert configuration to a dictionary.
Returns:
Configuration as a dictionary
"""
result = {}
if self.user:
result["user"] = {
"email": self.user.email,
"name": self.user.name,
}
if self.api:
result["api"] = {
"key": self.api.key,
"url": self.api.url,
}
return result
def print_config(config: dict[str, Any], name: str | None = None) -> None:
"""
Print the configuration in a formatted table.
Args:
config: Configuration dictionary
name: Optional name for the configuration
"""
table = Table(title=f"Configuration: {name}" if name else "Configuration")
table.add_column("Key", style="cyan")
table.add_column("Value", style="green")
for key, value in sorted(config.items()):
if isinstance(value, dict):
# For nested configurations
nested_value = "\n".join(f"{k}: {v}" for k, v in value.items())
table.add_row(key, nested_value)
else:
table.add_row(key, str(value))
console.print(table)

0
arcade/arcade/cli/constants.py → libs/arcade-cli/arcade_cli/constants.py
View file

0
arcade/arcade/cli/deployment.py → libs/arcade-cli/arcade_cli/deployment.py
View file

5
arcade/arcade/cli/display.py → libs/arcade-cli/arcade_cli/display.py
View file

@ -1,14 +1,13 @@
from typing import TYPE_CHECKING, Any
from arcade_core.schema import ToolDefinition
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from rich.text import Text
from arcade.core.schema import ToolDefinition
if TYPE_CHECKING:
from arcade.sdk.eval.eval import EvaluationResult
from arcade_evals.eval import EvaluationResult
console = Console()

84
arcade/arcade/cli/main.py → libs/arcade-cli/arcade_cli/main.py
View file

@ -16,23 +16,22 @@ from rich.markup import escape
from rich.text import Text
from tqdm import tqdm
import arcade.cli.worker as worker
from arcade.cli.authn import LocalAuthCallbackServer, check_existing_login
from arcade.cli.constants import (
import arcade_cli.worker as worker
from arcade_cli.authn import LocalAuthCallbackServer, check_existing_login
from arcade_cli.constants import (
CREDENTIALS_FILE_PATH,
LOCALHOST,
PROD_CLOUD_HOST,
PROD_ENGINE_HOST,
)
from arcade.cli.deployment import Deployment
from arcade.cli.display import (
from arcade_cli.deployment import Deployment
from arcade_cli.display import (
display_arcade_chat_header,
display_eval_results,
display_tool_messages,
)
from arcade.cli.launcher import start_servers
from arcade.cli.show import show_logic
from arcade.cli.utils import (
from arcade_cli.show import show_logic
from arcade_cli.utils import (
OrderCommands,
compute_base_url,
compute_login_url,
@ -45,6 +44,7 @@ from arcade.cli.utils import (
is_authorization_pending,
load_eval_suites,
log_engine_health,
require_dependency,
validate_and_get_config,
version_callback,
)
@ -133,17 +133,24 @@ def logout() -> None:
console.print("You're not logged in.", style="bold red")
@cli.command(help="Create a new toolkit package directory", rich_help_panel="Tool Development")
@cli.command(
help="Create a new toolkit package directory. Example usage: arcade new my_toolkit",
rich_help_panel="Tool Development",
)
def new(
toolkit_name: str = typer.Argument(
help="The name of the toolkit to create",
metavar="TOOLKIT_NAME",
),
directory: str = typer.Option(os.getcwd(), "--dir", help="tools directory path"),
) -> None:
"""
Creates a new toolkit with the given name, description, and result type.
"""
from arcade.cli.new import create_new_toolkit
from arcade_cli.new import create_new_toolkit
try:
create_new_toolkit(directory)
create_new_toolkit(directory, toolkit_name)
except Exception as e:
error_message = f"❌ Failed to create new Toolkit: {escape(str(e))}"
console.print(error_message, style="bold red")
@ -374,6 +381,20 @@ def evals(
Find all files starting with 'eval_' in the given directory,
execute any functions decorated with @tool_eval, and display the results.
"""
require_dependency(
package_name="arcade_evals",
command_name="evals",
install_command=r"pip install 'arcade-ai\[evals]'",
)
# Although Evals does not depend on the TDK, some evaluations import the
# ToolCatalog class from the TDK instead of from arcade_core, so we require
# the TDK to run the evals CLI command to avoid possible import errors.
require_dependency(
package_name="arcade_tdk",
command_name="evals",
install_command=r"pip install arcade-tdk",
)
config = validate_and_get_config()
host = PROD_ENGINE_HOST if cloud else host
@ -481,7 +502,13 @@ def serve(
"""
Start a local Arcade Worker server.
"""
from arcade.cli.serve import serve_default_worker
require_dependency(
package_name="arcade_serve",
command_name="serve",
install_command=r"pip install 'arcade-ai\[serve]'",
)
from arcade_cli.serve import serve_default_worker
try:
serve_default_worker(
@ -501,31 +528,6 @@ def serve(
typer.Exit(code=1)
@cli.command(help="Launch Arcade - requires 'arcade-engine'", rich_help_panel="Launch")
def dev(
host: str = typer.Option("127.0.0.1", help="Host for the toolkit server.", show_default=True),
port: int = typer.Option(
8002, "-p", "--port", help="Port for the toolkit server.", show_default=True
),
engine_config: str = typer.Option(
None, "-c", "--config", help="Path to the engine configuration file."
),
env_file: str = typer.Option(
None, "-e", "--env-file", help="Path to the environment variables file."
),
debug: bool = typer.Option(False, "-d", "--debug", help="Show debug information"),
) -> None:
"""
Start both the toolkit server and engine servers.
"""
try:
start_servers(host, port, engine_config, engine_env=env_file, debug=debug)
except Exception as e:
error_message = f"❌ Failed to start servers: {escape(str(e))}"
console.print(error_message, style="bold red")
typer.Exit(code=1)
@cli.command(
help="Start a server with locally installed Arcade tools", rich_help_panel="Launch", hidden=True
)
@ -553,7 +555,13 @@ def workerup(
Starts the worker with host, port, and reload options. Uses
Uvicorn as ASGI worker. Parameters allow runtime configuration.
"""
from arcade.cli.serve import serve_default_worker
require_dependency(
package_name="arcade_serve",
command_name="worker",
install_command=r"pip install 'arcade-ai\[worker]'",
)
from arcade_cli.serve import serve_default_worker
try:
serve_default_worker(

219
libs/arcade-cli/arcade_cli/new.py Normal file
View file

@ -0,0 +1,219 @@
import re
import shutil
from datetime import datetime
from importlib.metadata import version as get_version
from pathlib import Path
from typing import Optional
import typer
from jinja2 import Environment, FileSystemLoader, select_autoescape
from rich.console import Console
from arcade_cli.deployment import (
create_demo_deployment,
)
console = Console()
# Retrieve the installed version of arcade-ai
try:
ARCADE_AI_MIN_VERSION = get_version("arcade-ai")
ARCADE_AI_MAX_VERSION = str(int(ARCADE_AI_MIN_VERSION.split(".")[0]) + 1) + ".0.0"
except Exception as e:
console.print(f"[red]Failed to get arcade-ai version: {e}[/red]")
ARCADE_AI_MIN_VERSION = "2.0.0" # Default version if unable to fetch
ARCADE_AI_MAX_VERSION = "3.0.0"
ARCADE_TDK_MIN_VERSION = "0.1.0"
ARCADE_TDK_MAX_VERSION = "1.0.0"
ARCADE_SERVE_MIN_VERSION = "0.1.0"
ARCADE_SERVE_MAX_VERSION = "1.0.0"
def ask_question(question: str, default: Optional[str] = None) -> str:
"""
Ask a question via input() and return the answer.
"""
answer = typer.prompt(question, default=default, show_default=False)
if not answer and default:
return default
return str(answer)
def ask_yes_no_question(question: str, default: bool = True) -> bool:
"""
Ask a yes/no question via input() and return the bool answer.
"""
default_str = "Y/n" if default else "y/N"
answer = typer.prompt(
f"{question} ({default_str})", default="y" if default else "n", show_default=False
)
return answer.lower() in [
"y",
"y/",
"yes",
"true",
"1",
"ye",
"yes",
"yeah",
"yep",
"sure",
"ok",
"yup",
]
def render_template(env: Environment, template_string: str, context: dict) -> str:
"""Render a template string with the given variables."""
template = env.from_string(template_string)
return template.render(context)
def write_template(path: Path, content: str) -> None:
"""Write content to a file."""
path.write_text(content, encoding="utf-8")
def create_ignore_pattern(include_evals: bool) -> re.Pattern[str]:
"""Create an ignore pattern based on user preferences."""
patterns = [
"__pycache__",
r"\.DS_Store",
r"Thumbs\.db",
r"\.git",
r"\.svn",
r"\.hg",
r"\.vscode",
r"\.idea",
"build",
"dist",
r".*\.egg-info",
r".*\.pyc",
r".*\.pyo",
]
if not include_evals:
patterns.append("evals")
return re.compile(f"({'|'.join(patterns)})$")
def create_package(
env: Environment,
template_path: Path,
output_path: Path,
context: dict,
ignore_pattern: re.Pattern[str],
) -> None:
"""Recursively create a new toolkit directory structure from jinja2 templates."""
if ignore_pattern.match(template_path.name):
return
try:
if template_path.is_dir():
folder_name = render_template(env, template_path.name, context)
new_dir_path = output_path / folder_name
new_dir_path.mkdir(parents=True, exist_ok=True)
for item in template_path.iterdir():
create_package(env, item, new_dir_path, context, ignore_pattern)
else:
# Render the file name
file_name = render_template(env, template_path.name, context)
with open(template_path, encoding="utf-8") as f:
content = f.read()
# Render the file content
content = render_template(env, content, context)
write_template(output_path / file_name, content)
except Exception as e:
console.print(f"[red]Failed to create package: {e}[/red]")
raise
def remove_toolkit(toolkit_directory: Path, toolkit_name: str) -> None:
"""Teardown logic for when creating a new toolkit fails."""
toolkit_path = toolkit_directory / toolkit_name
if toolkit_path.exists():
shutil.rmtree(toolkit_path)
def create_new_toolkit(output_directory: str, toolkit_name: str) -> None:
"""Create a new toolkit from a template with user input."""
toolkit_directory = Path(output_directory)
package_name = toolkit_name if toolkit_name.startswith("arcade_") else f"arcade_{toolkit_name}"
# Check for illegal characters in the toolkit name
if re.match(r"^[a-z0-9_]+$", package_name):
toolkit_name = package_name.replace("arcade_", "", 1)
if (toolkit_directory / toolkit_name).exists():
console.print(f"[red]Toolkit '{toolkit_name}' already exists.[/red]")
exit(1)
else:
console.print(
"[red]Toolkit name contains illegal characters. "
"Only lowercase alphanumeric characters and underscores are allowed. "
"Please try again.[/red]"
)
exit(1)
toolkit_description = ask_question("Describe what your toolkit will do (optional)", default="")
toolkit_author_name = ask_question("Your GitHub username (optional)", default="")
while True:
toolkit_author_email = ask_question("Your email (optional)", default="")
if toolkit_author_email == "" or re.match(r"[^@ ]+@[^@ ]+\.[^@ ]+", toolkit_author_email):
break
console.print(
"[red]Invalid email format. Please enter a valid email address or leave it empty.[/red]"
)
include_evals = ask_yes_no_question(
"Do you want an evals directory created for you?", default=True
)
context = {
"package_name": package_name,
"toolkit_name": toolkit_name,
"toolkit_description": toolkit_description,
"toolkit_author_name": toolkit_author_name,
"toolkit_author_email": toolkit_author_email,
"arcade_tdk_min_version": ARCADE_TDK_MIN_VERSION,
"arcade_tdk_max_version": ARCADE_TDK_MAX_VERSION,
"arcade_serve_min_version": ARCADE_SERVE_MIN_VERSION,
"arcade_serve_max_version": ARCADE_SERVE_MAX_VERSION,
"arcade_ai_min_version": ARCADE_AI_MIN_VERSION,
"arcade_ai_max_version": ARCADE_AI_MAX_VERSION,
"creation_year": datetime.now().year,
}
template_directory = Path(__file__).parent / "templates" / "{{ toolkit_name }}"
env = Environment(
loader=FileSystemLoader(str(template_directory)),
autoescape=select_autoescape(["html", "xml"]),
)
# Create dynamic ignore pattern based on user preferences
ignore_pattern = create_ignore_pattern(include_evals)
try:
create_package(env, template_directory, toolkit_directory, context, ignore_pattern)
console.print(
f"[green]Toolkit '{toolkit_name}' created successfully at '{toolkit_directory}'.[/green]"
)
create_deployment(toolkit_directory, toolkit_name)
except Exception:
remove_toolkit(toolkit_directory, toolkit_name)
raise
def create_deployment(toolkit_directory: Path, toolkit_name: str) -> None:
worker_toml = toolkit_directory / "worker.toml"
if not worker_toml.exists():
create_demo_deployment(worker_toml, toolkit_name)
else:
pass
# Disabled pending bug fix
# update_deployment_with_local_packages(worker_toml, toolkit_name)

15
arcade/arcade/cli/serve.py → libs/arcade-cli/arcade_cli/serve.py
View file

@ -15,18 +15,18 @@ import uvicorn
# Watchfiles is used under the hood by Uvicorn's reload feature.
# Importing watchfiles here is an explicit acknowledgement that it needs to be installed
import watchfiles # noqa: F401
from arcade_core.telemetry import OTELHandler
from arcade_core.toolkit import Toolkit, get_package_directory
from arcade_serve.fastapi.worker import FastAPIWorker
from loguru import logger
from rich.console import Console
from arcade.cli.constants import ARCADE_CONFIG_PATH
from arcade.cli.utils import (
from arcade_cli.constants import ARCADE_CONFIG_PATH
from arcade_cli.utils import (
build_tool_catalog,
discover_toolkits,
load_dotenv,
)
from arcade.core.telemetry import OTELHandler
from arcade.core.toolkit import Toolkit, get_package_directory
from arcade.worker.fastapi.worker import FastAPIWorker
console = Console(width=70, color_system="auto")
@ -45,7 +45,6 @@ def create_arcade_app() -> fastapi.FastAPI:
setup_logging(log_level=logging.DEBUG if debug_mode else logging.INFO, mcp_mode=False)
logger.info(f"Debug: {debug_mode}, OTEL: {otel_enabled}, Auth Disabled: {auth_for_reload}")
version = get_pkg_version("arcade-ai")
toolkits = discover_toolkits()
@ -94,7 +93,7 @@ def _run_mcp_stdio(
) -> None:
"""Launch an MCP stdio server; blocks until it exits."""
from arcade.worker.mcp.stdio import StdioServer
from arcade_serve.mcp.stdio import StdioServer
# Load env vars before launching server (explicit path, config path, cwd)
if env_file:
@ -139,7 +138,7 @@ def _run_fastapi_server(
toolkits_for_reload_dirs: list[Toolkit] | None,
debug_flag: bool,
) -> None:
app_import_string = "arcade.cli.serve:create_arcade_app"
app_import_string = "arcade_cli.serve:create_arcade_app"
reload_dirs_str_list: list[str] | None = None
if reload:

4
arcade/arcade/cli/show.py → libs/arcade-cli/arcade_cli/show.py
View file

@ -2,8 +2,8 @@ import typer
from rich.console import Console
from rich.markup import escape
from arcade.cli.display import display_tool_details, display_tools_table
from arcade.cli.utils import create_cli_catalog, get_tools_from_engine
from arcade_cli.display import display_tool_details, display_tools_table
from arcade_cli.utils import create_cli_catalog, get_tools_from_engine
def show_logic(

46
libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/Makefile Normal file
View file

@ -0,0 +1,46 @@
.PHONY: help
help:
@echo "🛠️ github Commands:\n"
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.PHONY: install
install: ## Install the uv environment and install all packages with dependencies
@echo "🚀 Creating virtual environment and installing all packages using uv"
@uv sync --active --all-extras --no-sources
@uv run pre-commit install
@echo "✅ All packages and dependencies installed via uv"
.PHONY: install-local
install-local: ## Install the uv environment and install all packages with dependencies with local Arcade sources
@echo "🚀 Creating virtual environment and installing all packages using uv"
@uv sync --active --all-extras
@uv run pre-commit install
@echo "✅ All packages and dependencies installed via uv"
.PHONY: build
build: clean-build ## Build wheel file using poetry
@echo "🚀 Creating wheel file"
uv build
.PHONY: clean-build
clean-build: ## clean build artifacts
@echo "🗑️ Cleaning dist directory"
rm -rf dist
.PHONY: test
test: ## Test the code with pytest
@echo "🚀 Testing code: Running pytest"
@uv run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
.PHONY: bump-version
bump-version: ## Bump the version in the pyproject.toml file by a patch version
@echo "🚀 Bumping version in pyproject.toml"
uv version --bump patch
.PHONY: check
check: ## Run code quality tools.
@echo "🚀 Linting code: Running pre-commit"
@uv run pre-commit run -a
@echo "🚀 Static type checking: Running mypy"
@uv run mypy --config-file=pyproject.toml

16
arcade/arcade/templates/{{ toolkit_name }}/README.md → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/README.md
View file

@ -6,11 +6,14 @@
</div>
<div style="display: flex; justify-content: center; align-items: center; margin-bottom: 8px;">
{% if toolkit_author_name -%}
<img src="https://img.shields.io/github/v/release/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub release" style="margin: 0 2px;">
{% endif -%}
<img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python version" style="margin: 0 2px;">
<img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License" style="margin: 0 2px;">
<img src="https://img.shields.io/pypi/v/{{ package_name }}" alt="PyPI version" style="margin: 0 2px;">
</div>
{% if toolkit_author_name -%}
<div style="display: flex; justify-content: center; align-items: center;">
<a href="https://github.com/{{ toolkit_author_name }}/{{ toolkit_name }}" target="_blank">
<img src="https://img.shields.io/github/stars/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub stars" style="margin: 0 2px;">
@ -19,22 +22,19 @@
<img src="https://img.shields.io/github/forks/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub forks" style="margin: 0 2px;">
</a>
</div>
{% endif %}
<br>
<br>
# Arcade {{ toolkit_name }} Toolkit
{% if toolkit_description -%}
{{ toolkit_description }}
{% endif -%}
## Features
- The {{ toolkit_name }} toolkit does not have any features yet.
## Install
## Development
Install this toolkit using pip:
```bash
pip install {{ package_name }}
```
Read the docs on how to create a toolkit [here](https://docs.arcade.dev/home/build-tools/create-a-toolkit)

6
arcade/arcade/templates/{{ toolkit_name }}/evals/eval_{{ toolkit_name }}.py → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/evals/eval_{{ toolkit_name }}.py
View file

@ -1,11 +1,11 @@
from arcade.sdk import ToolCatalog
from arcade.sdk.eval import (
from arcade_tdk import ToolCatalog
from arcade_evals import (
EvalRubric,
EvalSuite,
ExpectedToolCall,
SimilarityCritic,
tool_eval,
)
from arcade_evals.critic import SimilarityCritic
import {{ package_name }}
from {{ package_name }}.tools.hello import say_hello

66
libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/pyproject.toml Normal file
View file

@ -0,0 +1,66 @@
[build-system]
requires = [ "hatchling",]
build-backend = "hatchling.build"
[project]
name = "{{ package_name }}"
version = "0.1.0"
{% if toolkit_description -%}
description = "{{ toolkit_description }}"
{% endif -%}
requires-python = ">=3.10"
dependencies = [
"arcade-tdk>={{ arcade_tdk_min_version }},<{{ arcade_tdk_max_version}}",
]
{% if toolkit_author_name or toolkit_author_email -%}
[[project.authors]]
{% if toolkit_author_name -%}
name = "{{ toolkit_author_name }}"
{% endif -%}
{% if toolkit_author_email -%}
email = "{{ toolkit_author_email }}"
{% endif -%}
{% endif %}
[project.optional-dependencies]
dev = [
"arcade-ai[evals]>={{ arcade_ai_min_version }},<{{ arcade_ai_max_version }}",
"arcade-serve>={{ arcade_serve_min_version }},<{{ arcade_serve_max_version }}",
"pytest>=8.3.0,<8.4.0",
"pytest-cov>=4.0.0,<4.1.0",
"pytest-mock>=3.11.1,<3.12.0",
"pytest-asyncio>=0.24.0,<0.25.0",
"mypy>=1.5.1,<1.6.0",
"pre-commit>=3.4.0,<3.5.0",
"tox>=4.11.1,<4.12.0",
"ruff>=0.7.4,<0.8.0",
]
# Use local path sources for arcade libs when working locally
[tool.uv.sources]
arcade-ai = { path = "../../", editable = true }
arcade-serve = { path = "../../libs/arcade-serve/", editable = true }
arcade-tdk = { path = "../../libs/arcade-tdk/", editable = true }
[tool.mypy]
files = [ "{{ package_name }}/**/*.py",]
python_version = "3.10"
disallow_untyped_defs = "True"
disallow_any_unimported = "True"
no_implicit_optional = "True"
check_untyped_defs = "True"
warn_return_any = "True"
warn_unused_ignores = "True"
show_error_codes = "True"
ignore_missing_imports = "True"
[tool.pytest.ini_options]
testpaths = [ "tests",]
[tool.coverage.report]
skip_empty = true
[tool.hatch.build.targets.wheel]
packages = [ "{{ package_name }}",]

0
arcade/arcade/templates/{{ toolkit_name }}/tests/__init__.py → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/tests/__init__.py
View file

2
arcade/arcade/templates/{{ toolkit_name }}/tests/test_{{ toolkit_name }}.py → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/tests/test_{{ toolkit_name }}.py
View file

@ -1,5 +1,5 @@
import pytest
from arcade.sdk.errors import ToolExecutionError
from arcade_tdk.errors import ToolExecutionError
from {{ package_name }}.tools.hello import say_hello

0
arcade/arcade/templates/{{ toolkit_name }}/{{ package_name }}/__init__.py → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/{{ package_name }}/__init__.py
View file

0
arcade/arcade/templates/{{ toolkit_name }}/{{ package_name }}/tools/__init__.py → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/{{ package_name }}/tools/__init__.py
View file

3
arcade/arcade/templates/{{ toolkit_name }}/{{ package_name }}/tools/hello.py → libs/arcade-cli/arcade_cli/templates/{{ toolkit_name }}/{{ package_name }}/tools/hello.py
View file

@ -1,6 +1,6 @@
from typing import Annotated
from arcade.sdk import tool
from arcade_tdk import tool
@tool
@ -8,4 +8,3 @@ def say_hello(name: Annotated[str, "The name of the person to greet"]) -> str:
"""Say a greeting!"""
return "Hello, " + name + "!"

44
arcade/arcade/cli/utils.py → libs/arcade-cli/arcade_cli/utils.py
View file

@ -6,6 +6,7 @@ import webbrowser
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
from importlib import metadata
from pathlib import Path
from textwrap import dedent
from typing import Any, Callable, Union, cast
@ -13,6 +14,10 @@ from urllib.parse import urlencode, urlparse
import idna
import typer
from arcade_core import ToolCatalog, Toolkit
from arcade_core.config_model import Config
from arcade_core.errors import ToolkitLoadError
from arcade_core.schema import ToolDefinition
from arcadepy import NOT_GIVEN, APIConnectionError, APIStatusError, APITimeoutError, Arcade
from arcadepy.types import AuthorizationResponse
from openai import OpenAI, Stream
@ -27,11 +32,7 @@ from rich.text import Text
from typer.core import TyperGroup
from typer.models import Context
from arcade.cli.constants import LOCALHOST
from arcade.core.config_model import Config
from arcade.core.errors import ToolkitLoadError
from arcade.core.schema import ToolDefinition
from arcade.sdk import ToolCatalog, Toolkit
from arcade_cli.constants import LOCALHOST
console = Console()
@ -300,7 +301,7 @@ def validate_and_get_config(
"""
Validates the configuration, user, and returns the Config object
"""
from arcade.core.config import config
from arcade_core.config import config
if validate_api and (not config.api or not config.api.key):
console.print(
@ -664,8 +665,8 @@ def version_callback(value: bool) -> None:
Prints the version of Arcade and exit.
"""
if value:
version = importlib.import_module("arcade").__version__
console.print(f"[bold]Arcade[/bold] (version {version})")
version = metadata.version("arcade-ai")
console.print(f"[bold]Arcade CLI[/bold] (version {version})")
exit()
@ -751,3 +752,30 @@ def load_dotenv(path: str | Path, *, override: bool = False) -> dict[str, str]:
loaded[k] = v
return loaded
def require_dependency(
package_name: str,
command_name: str,
install_command: str,
) -> None:
"""
Display a helpful error message if the required dependency is missing.
Args:
package_name: The name of the package to import (e.g., 'arcade_serve')
command_name: The command that requires the package (e.g., 'serve')
install_command: The command to install the package (e.g., "pip install 'arcade-ai[evals]'")
"""
try:
importlib.import_module(package_name.replace("-", "_"))
except ImportError:
console.print(
f"❌ The '{package_name}' package is required to run the '{command_name}' command but is not installed.",
style="bold red",
)
console.print(
f"To install it, run the following command:\n* [green]{install_command}[/green]",
style="bold",
)
raise typer.Exit(code=1)

4
arcade/arcade/cli/worker.py → libs/arcade-cli/arcade_cli/worker.py
View file

@ -4,11 +4,11 @@ from arcadepy import Arcade, NotFoundError
from rich.console import Console
from rich.table import Table
from arcade.cli.constants import (
from arcade_cli.constants import (
PROD_CLOUD_HOST,
PROD_ENGINE_HOST,
)
from arcade.cli.utils import (
from arcade_cli.utils import (
OrderCommands,
compute_base_url,
validate_and_get_config,

2
arcade/run_cli.py → libs/arcade-cli/run_cli.py
View file

@ -1,6 +1,6 @@
import sys
from arcade.cli.main import cli
from arcade_cli.main import cli
if __name__ == "__main__":
# Supports attaching debugger to cli. Run from ../.vscode/launch.json.

39
libs/arcade-core/README.md Normal file
View file

@ -0,0 +1,39 @@
# Arcade Core
Core library for the Arcade platform providing foundational components and utilities.
## Overview
Arcade Core provides the essential building blocks for the Arcade platform:
- **Tool Catalog & Toolkit Management**: Core classes for managing and organizing tools
- **Configuration & Schema Handling**: Configuration management and validation
- **Authentication & Authorization**: Auth providers and security utilities
- **Error Handling**: Comprehensive error types and handling
- **Telemetry & Observability**: Monitoring and tracing capabilities
- **Utilities**: Common helper functions and validators
## Installation
```bash
pip install arcade-core
```
## Usage
```python
from arcade_core import ToolCatalog, Toolkit, ArcadeConfig
# Create a tool catalog
catalog = ToolCatalog()
# Load a toolkit
toolkit = Toolkit.from_directory("path/to/toolkit")
# Configure Arcade
config = ArcadeConfig.from_file("config.yaml")
```
## License
MIT License - see LICENSE file for details.

2
libs/arcade-core/arcade_core/__init__.py Normal file
View file

@ -0,0 +1,2 @@
from .catalog import ToolCatalog as ToolCatalog
from .toolkit import Toolkit as Toolkit

0
arcade/arcade/core/annotations.py → libs/arcade-core/arcade_core/annotations.py
View file

0
arcade/arcade/core/auth.py → libs/arcade-core/arcade_core/auth.py
View file

12
arcade/arcade/core/catalog.py → libs/arcade-core/arcade_core/catalog.py
View file

@ -26,10 +26,10 @@ from pydantic import BaseModel, Field, create_model
from pydantic.fields import FieldInfo
from pydantic_core import PydanticUndefined
from arcade.core.annotations import Inferrable
from arcade.core.auth import OAuth2, ToolAuthorization
from arcade.core.errors import ToolDefinitionError
from arcade.core.schema import (
from arcade_core.annotations import Inferrable
from arcade_core.auth import OAuth2, ToolAuthorization
from arcade_core.errors import ToolDefinitionError
from arcade_core.schema import (
TOOL_NAME_SEPARATOR,
FullyQualifiedName,
InputParameter,
@ -46,8 +46,8 @@ from arcade.core.schema import (
ToolSecretRequirement,
ValueSchema,
)
from arcade.core.toolkit import Toolkit
from arcade.core.utils import (
from arcade_core.toolkit import Toolkit
from arcade_core.utils import (
does_function_return_value,
first_or_none,
is_strict_optional,

2
arcade/arcade/core/config.py → libs/arcade-core/arcade_core/config.py
View file

@ -1,6 +1,6 @@
from functools import lru_cache
from arcade.core.config_model import Config
from arcade_core.config_model import Config
@lru_cache(maxsize=1)

0
arcade/arcade/core/config_model.py → libs/arcade-core/arcade_core/config_model.py
View file

0
arcade/arcade/core/errors.py → libs/arcade-core/arcade_core/errors.py
View file

6
arcade/arcade/core/executor.py → libs/arcade-core/arcade_core/executor.py
View file

@ -4,15 +4,15 @@ from typing import Any, Callable
from pydantic import BaseModel, ValidationError
from arcade.core.errors import (
from arcade_core.errors import (
RetryableToolError,
ToolInputError,
ToolOutputError,
ToolRuntimeError,
ToolSerializationError,
)
from arcade.core.output import output_factory
from arcade.core.schema import ToolCallLog, ToolCallOutput, ToolContext, ToolDefinition
from arcade_core.output import output_factory
from arcade_core.schema import ToolCallLog, ToolCallOutput, ToolContext, ToolDefinition
class ToolExecutor:

4
arcade/arcade/core/output.py → libs/arcade-core/arcade_core/output.py
View file

@ -1,7 +1,7 @@
from typing import TypeVar
from arcade.core.schema import ToolCallError, ToolCallLog, ToolCallOutput
from arcade.core.utils import coerce_empty_list_to_none
from arcade_core.schema import ToolCallError, ToolCallLog, ToolCallOutput
from arcade_core.utils import coerce_empty_list_to_none
T = TypeVar("T")

0
arcade/arcade/core/parse.py → libs/arcade-core/arcade_core/parse.py
View file

0
arcade/arcade/py.typed → libs/arcade-core/arcade_core/py.typed
View file

0
arcade/arcade/core/schema.py → libs/arcade-core/arcade_core/schema.py
View file

0
arcade/arcade/core/telemetry.py → libs/arcade-core/arcade_core/telemetry.py
View file

4
arcade/arcade/core/toolkit.py → libs/arcade-core/arcade_core/toolkit.py
View file

@ -8,8 +8,8 @@ from pathlib import Path
from pydantic import BaseModel, ConfigDict, field_validator
from arcade.core.errors import ToolkitLoadError
from arcade.core.parse import get_tools_from_file
from arcade_core.errors import ToolkitLoadError
from arcade_core.parse import get_tools_from_file
logger = logging.getLogger(__name__)

0
arcade/arcade/core/utils.py → libs/arcade-core/arcade_core/utils.py
View file

0
arcade/arcade/core/version.py → libs/arcade-core/arcade_core/version.py
View file

65
libs/arcade-core/pyproject.toml Normal file
View file

@ -0,0 +1,65 @@
[project]
name = "arcade-core"
version = "2.0.0"
description = "Arcade Core - Core library for Arcade platform"
readme = "README.md"
license = {text = "MIT"}
authors = [
{name = "Arcade", email = "dev@arcade.dev"},
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
requires-python = ">=3.10"
dependencies = [
"pydantic>=2.7.0",
"pyyaml>=6.0",
"loguru>=0.7.0",
"pyjwt>=2.8.0",
"toml>=0.10.2",
"packaging>=24.1",
"types-python-dateutil==2.9.0.20241003",
"types-pytz==2024.2.0.20241003",
"types-toml==0.10.8.20240310",
"opentelemetry-instrumentation-fastapi==0.49b2",
"opentelemetry-exporter-otlp-proto-http==1.28.2",
"opentelemetry-exporter-otlp-proto-common==1.28.2",
]
[project.optional-dependencies]
dev = [
"pytest>=8.1.2",
"pytest-cov>=4.0.0",
"mypy>=1.5.1",
"pre-commit>=3.4.0",
"pytest-asyncio>=0.23.7",
"types-pytz>=2024.1",
"types-python-dateutil>=2.8.2",
"types-PyYAML>=6.0.0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["arcade_core"]
[tool.mypy]
files = ["arcade_core"]
python_version = "3.10"
disallow_untyped_defs = true
disallow_any_unimported = true
no_implicit_optional = true
check_untyped_defs = true
warn_return_any = true
warn_unused_ignores = true
show_error_codes = true
ignore_missing_imports = true

86
libs/arcade-evals/README.md Normal file
View file

@ -0,0 +1,86 @@
# Arcade Evals
Evaluation toolkit for testing Arcade tools.
## Overview
Arcade Evals provides comprehensive evaluation capabilities for Arcade tools:
- **Evaluation Framework**: Cases, suites, and rubrics for systematic testing
- **Critics**: Different types of comparisons (binary, numeric, similarity, datetime)
- **Tool Evaluation**: Decorators and utilities for evaluating tool performance
- **Result Analysis**: Comprehensive evaluation results and reporting
## Installation
```bash
pip install 'arcade-ai[evals]'
```
## Usage
### Basic Evaluation
```python
from arcade_evals import EvalCase, EvalSuite, tool_eval
# Create evaluation cases
case1 = EvalCase(
input={"query": "What is 2+2?"},
expected_output="4"
)
case2 = EvalCase(
input={"query": "What is the capital of France?"},
expected_output="Paris"
)
# Create evaluation suite
suite = EvalSuite(cases=[case1, case2])
# Evaluate a tool
@tool_eval(suite)
def my_calculator(query: str) -> str:
# Tool implementation
return "4" if "2+2" in query else "Unknown"
```
### Using Critics
```python
from arcade_evals import NumericCritic, SimilarityCritic
# Numeric comparison
numeric_critic = NumericCritic(tolerance=0.1)
result = numeric_critic.evaluate(expected=10.0, actual=10.05)
# Similarity comparison
similarity_critic = SimilarityCritic(threshold=0.8)
result = similarity_critic.evaluate(
expected="The capital of France is Paris",
actual="Paris is the capital of France"
)
```
### Advanced Evaluation
```python
from arcade_evals import EvalRubric, ExpectedToolCall
# Create rubric with tool calls
rubric = EvalRubric(
expected_tool_calls=[
ExpectedToolCall(
tool_name="calculator",
parameters={"operation": "add", "a": 2, "b": 2}
)
]
)
# Evaluate with rubric
suite = EvalSuite(cases=[case1], rubric=rubric)
```
## License
MIT License - see LICENSE file for details.

0
arcade/arcade/sdk/eval/__init__.py → libs/arcade-evals/arcade_evals/__init__.py
View file

5
arcade/arcade/sdk/eval/critic.py → libs/arcade-evals/arcade_evals/critic.py
View file

@ -6,7 +6,7 @@ from typing import Any, ClassVar
import pytz
from dateutil import parser
from arcade.sdk.errors import WeightError
from arcade_evals.errors import WeightError
@dataclass
@ -214,7 +214,7 @@ class SimilarityCritic(Critic):
from sklearn.metrics.pairwise import cosine_similarity
except ImportError:
raise ImportError(
"Use `pip install 'arcade-ai[evals]'` to install the required dependencies for similarity metrics."
"Use `pip install 'arcade-evals` to install the required dependencies for similarity metrics."
)
vectorizer = TfidfVectorizer()
tfidf_matrix = vectorizer.fit_transform([expected, actual])
@ -227,7 +227,6 @@ class SimilarityCritic(Critic):
}
@dataclass
@dataclass
class DatetimeCritic(Critic):
"""

12
libs/arcade-evals/arcade_evals/errors.py Normal file
View file

@ -0,0 +1,12 @@
__all__ = [
"EvalError",
"WeightError",
]
class EvalError(Exception):
"""Base class for all evaluation errors."""
class WeightError(EvalError):
"""Raised when the critic weights do not abide by evaluation weight constraints."""

24
arcade/arcade/sdk/eval/eval.py → libs/arcade-evals/arcade_evals/eval.py
View file

@ -5,25 +5,19 @@ import json
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Callable
from arcade.core.config_model import Config
from arcade.core.schema import TOOL_NAME_SEPARATOR
try:
import numpy as np
from scipy.optimize import linear_sum_assignment
except ImportError:
raise ImportError(
"Use `pip install 'arcade-ai[evals]'` to install the required dependencies for evaluation."
)
import numpy as np
from arcade_core.config_model import Config
from arcade_core.schema import TOOL_NAME_SEPARATOR
from openai import AsyncOpenAI
from scipy.optimize import linear_sum_assignment
from arcade.sdk.errors import WeightError
from arcade.sdk.eval.critic import NoneCritic
from arcade_evals.critic import NoneCritic
from arcade_evals.errors import WeightError
if TYPE_CHECKING:
from arcade.sdk import ToolCatalog
from arcade.sdk.eval.critic import Critic
from arcade_core import ToolCatalog
from arcade_evals.critic import Critic
@dataclass

0
arcade/arcade/worker/__init__.py → libs/arcade-evals/arcade_evals/py.typed
View file

70
libs/arcade-serve/README.md Normal file
View file

@ -0,0 +1,70 @@
# Arcade Serve
Serving infrastructure for Arcade tools and workers.
## Overview
Arcade Serve provides the infrastructure for serving Arcade tools:
- **FastAPI Worker**: High-performance FastAPI-based worker implementation
- **MCP Server**: Model Context Protocol server for tool integration
- **Core Abstractions**: Base worker classes and components
- **Authentication**: Auth utilities and routing
- **Runtime Management**: Tool execution and lifecycle management
## Installation
```bash
pip install arcade-serve
```
## Usage
### FastAPI Worker
```python
from arcade_serve import FastAPIWorker
# Create a FastAPI worker
worker = FastAPIWorker()
# Add tools to the worker
worker.add_toolkit("path/to/toolkit")
# Start the server
worker.start(host="0.0.0.0", port=8000)
```
### MCP Server
```python
from arcade_serve import StdioServer
# Create an MCP server
server = StdioServer()
# Add tools
server.add_toolkit("path/to/toolkit")
# Start the server
server.run()
```
### Custom Worker
```python
from arcade_serve import BaseWorker, WorkerComponent
class MyWorker(BaseWorker):
def __init__(self):
super().__init__()
self.add_component(MyCustomComponent())
async def handle_request(self, request):
# Custom request handling
return await super().handle_request(request)
```
## License
MIT License - see LICENSE file for details.

0
arcade/arcade/worker/core/__init__.py → libs/arcade-serve/arcade_serve/__init__.py
View file

0
arcade/tests/__init__.py → libs/arcade-serve/arcade_serve/core/__init__.py
View file

0
arcade/arcade/worker/core/auth.py → libs/arcade-serve/arcade_serve/core/auth.py
View file

16
arcade/arcade/worker/core/base.py → libs/arcade-serve/arcade_serve/core/base.py
View file

@ -4,18 +4,18 @@ import time
from datetime import datetime
from typing import Any, Callable, ClassVar
from opentelemetry import trace
from opentelemetry.metrics import Meter
from arcade.core.catalog import ToolCatalog, Toolkit
from arcade.core.executor import ToolExecutor
from arcade.core.schema import (
from arcade_core.catalog import ToolCatalog, Toolkit
from arcade_core.executor import ToolExecutor
from arcade_core.schema import (
ToolCallRequest,
ToolCallResponse,
ToolDefinition,
)
from arcade.worker.core.common import Router, Worker
from arcade.worker.core.components import (
from opentelemetry import trace
from opentelemetry.metrics import Meter
from arcade_serve.core.common import Router, Worker
from arcade_serve.core.components import (
CallToolComponent,
CatalogComponent,
HealthCheckComponent,

3
arcade/arcade/worker/core/common.py → libs/arcade-serve/arcade_serve/core/common.py
View file

@ -1,10 +1,9 @@
from abc import ABC, abstractmethod
from typing import Any, Callable
from arcade_core.schema import ToolCallRequest, ToolCallResponse, ToolDefinition
from pydantic import BaseModel
from arcade.core.schema import ToolCallRequest, ToolCallResponse, ToolDefinition
CatalogResponse = list[ToolDefinition]
HealthCheckResponse = dict[str, str]
JSONResponse = dict[str, Any]

2
arcade/arcade/worker/core/components.py → libs/arcade-serve/arcade_serve/core/components.py
View file

@ -1,6 +1,6 @@
from opentelemetry import trace
from arcade.worker.core.common import (
from arcade_serve.core.common import (
CatalogResponse,
HealthCheckResponse,
RequestData,

0
arcade/arcade/worker/fastapi/__init__.py → libs/arcade-serve/arcade_serve/fastapi/__init__.py
View file

2
arcade/arcade/worker/fastapi/auth.py → libs/arcade-serve/arcade_serve/fastapi/auth.py
View file

@ -1,7 +1,7 @@
from fastapi import HTTPException
from fastapi.security import HTTPAuthorizationCredentials
from arcade.worker.core.auth import validate_engine_token
from arcade_serve.core.auth import validate_engine_token
# Dependency function to validate JWT

8
arcade/arcade/worker/fastapi/worker.py → libs/arcade-serve/arcade_serve/fastapi/worker.py
View file

@ -5,13 +5,13 @@ from fastapi import Depends, FastAPI, Request
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from opentelemetry.metrics import Meter
from arcade.worker.core.base import (
from arcade_serve.core.base import (
BaseWorker,
Router,
)
from arcade.worker.core.common import RequestData, ResponseData, WorkerComponent
from arcade.worker.fastapi.auth import validate_engine_request
from arcade.worker.utils import is_async_callable
from arcade_serve.core.common import RequestData, ResponseData, WorkerComponent
from arcade_serve.fastapi.auth import validate_engine_request
from arcade_serve.utils import is_async_callable
class FastAPIWorker(BaseWorker):

2
arcade/arcade/worker/mcp/__init__.py → libs/arcade-serve/arcade_serve/mcp/__init__.py
View file

@ -2,6 +2,6 @@
MCP (Model Context Protocol) support for Arcade workers.
"""
from arcade.worker.mcp.stdio import StdioServer
from arcade_serve.mcp.stdio import StdioServer
__all__ = ["StdioServer"]

2
arcade/arcade/worker/mcp/convert.py → libs/arcade-serve/arcade_serve/mcp/convert.py
View file

@ -3,7 +3,7 @@ import logging
from enum import Enum
from typing import Any
from arcade.core.catalog import MaterializedTool
from arcade_core.catalog import MaterializedTool
# Type aliases for MCP types
MCPTool = dict[str, Any]

2
arcade/arcade/worker/mcp/logging.py → libs/arcade-serve/arcade_serve/mcp/logging.py
View file

@ -4,7 +4,7 @@ import sys
import time
from typing import Any
from arcade.worker.mcp.types import (
from arcade_serve.mcp.types import (
JSONRPCError,
JSONRPCRequest,
JSONRPCResponse,

2
arcade/arcade/worker/mcp/message_processor.py → libs/arcade-serve/arcade_serve/mcp/message_processor.py
View file

@ -3,7 +3,7 @@ import json
import logging
from typing import Any, Callable, TypeVar
from arcade.worker.mcp.types import InitializeRequest, JSONRPCRequest, MCPMessage
from arcade_serve.mcp.types import InitializeRequest, JSONRPCRequest, MCPMessage
logger = logging.getLogger("arcade.mcp")

16
arcade/arcade/worker/mcp/server.py → libs/arcade-serve/arcade_serve/mcp/server.py
View file

@ -5,17 +5,17 @@ import uuid
from enum import Enum
from typing import Any, Callable, Union
from arcade_core.catalog import MaterializedTool, ToolCatalog
from arcade_core.executor import ToolExecutor
from arcade_core.schema import ToolAuthorizationContext, ToolContext
from arcadepy import ArcadeError, AsyncArcade
from arcadepy.types.auth_authorize_params import AuthRequirement, AuthRequirementOauth2
from arcadepy.types.shared import AuthorizationResponse
from arcade.core.catalog import MaterializedTool, ToolCatalog
from arcade.core.executor import ToolExecutor
from arcade.core.schema import ToolAuthorizationContext, ToolContext
from arcade.worker.mcp.convert import convert_to_mcp_content, create_mcp_tool
from arcade.worker.mcp.logging import create_mcp_logging_middleware
from arcade.worker.mcp.message_processor import MCPMessageProcessor, create_message_processor
from arcade.worker.mcp.types import (
from arcade_serve.mcp.convert import convert_to_mcp_content, create_mcp_tool
from arcade_serve.mcp.logging import create_mcp_logging_middleware
from arcade_serve.mcp.message_processor import MCPMessageProcessor, create_message_processor
from arcade_serve.mcp.types import (
CallToolRequest,
CallToolResponse,
CallToolResult,
@ -158,7 +158,7 @@ class MCPServer:
A user ID string
"""
try:
from arcade.core.config import config
from arcade_core.config import config
# Prefer config.user.email if available
if config.user and config.user.email:

2
arcade/arcade/worker/mcp/stdio.py → libs/arcade-serve/arcade_serve/mcp/stdio.py
View file

@ -10,7 +10,7 @@ from typing import TYPE_CHECKING, Any, TypeVar
if TYPE_CHECKING:
pass
from arcade.worker.mcp.server import MCPServer
from arcade_serve.mcp.server import MCPServer
logger = logging.getLogger("arcade.mcp")

0
arcade/arcade/worker/mcp/types.py → libs/arcade-serve/arcade_serve/mcp/types.py
View file

0
arcade/poetry.toml → libs/arcade-serve/arcade_serve/py.typed
View file

0
arcade/arcade/worker/utils.py → libs/arcade-serve/arcade_serve/utils.py
View file

Some files were not shown because too many files have changed in this diff Show more