🏗️ Restructure: Multi-Package Architecture + uv Migration (#412)
### Overview Major restructuring from monolithic `arcade-ai` package to modular library architecture with standardized uv-based dependency management.  ### New Package Structure - **`arcade-tdk`** - Lightweight toolkit development kit (core decorators, auth) - **`arcade-core`** - Core execution engine and catalog functionality - **`arcade-serve`** - FastAPI/MCP server components - **`arcade-ai`** - Meta package that includes CLI functionality. Optionally include evals via the `evals` extra. Optionally include all packages via the `all` extra. ### Key Benefits - **Lighter Dependencies**: Toolkits now depend only on `arcade-tdk` (~2 deps) vs full `arcade-ai` (~30+ deps) - **Faster Builds**: uv provides 10-100x faster dependency resolution and installation - **Better Modularity**: Clear separation of concerns, consumers import only what they need - **Standard Tooling**: Eliminates custom poetry scripts, uses standard Python packaging ### Migration Impact - All 20 toolkits converted from poetry → uv with `arcade-tdk` dependencies plus `arcade-ai[evals]` and `arcade-serve` dev dependencies. When developing locally, devs should install toolkits via `make install-local`. - Modern Python 3.10+ type hints throughout - Standardized build system with hatchling backend - Enhanced Makefile with robust toolkit management commands - Removed `arcade dev` CLI command - Reduce the number of files created by `arcade new` and add an option to not generate a tests and evals folder. This foundation enables faster development cycles and cleaner dependency chains for the growing toolkit ecosystem. ### Todo After this PR is merged - [ ] Post-merge workflow(s) (release & publish containers, etc) - [ ] Release order plan. @EricGustin suggests releasing in the following order: 1. `arcade-core` version 0.1.0 2. `arcade-serve` version 0.1.0 and `arcade-tdk` version 0.1.0 3. `arcade-ai` version 2.0.0 4. Patch release for all toolkits (all changes in toolkits are internal refactors) - [ ] [Update docs](https://github.com/ArcadeAI/docs/pull/318) --------- Co-authored-by: Eric Gustin <eric@arcade.dev> Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
This commit is contained in:
Sam Partee
GitHub
parent
01a9efcf63
commit
b6b4cd0a4c
429 changed files with 3321 additions and 3184 deletions
20
.github/actions/setup-uv-env/action.yml
vendored
Normal file
20
.github/actions/setup-uv-env/action.yml
vendored
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
name: "setup-uv-env"
|
||||||
|
description: "Composite action to setup the Python and uv environment."
|
||||||
|
|
||||||
|
inputs:
|
||||||
|
python-version:
|
||||||
|
required: false
|
||||||
|
description: "The python version to use"
|
||||||
|
default: "3.11"
|
||||||
|
|
||||||
|
runs:
|
||||||
|
using: "composite"
|
||||||
|
steps:
|
||||||
|
- name: Install uv
|
||||||
|
uses: astral-sh/setup-uv@v5
|
||||||
|
with:
|
||||||
|
python-version: ${{ inputs.python-version }}
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: uv sync --dev --extra all
|
||||||
|
shell: bash
|
||||||
29
.github/workflows/main.yml
vendored
29
.github/workflows/main.yml
vendored
|
|
@ -20,12 +20,12 @@ jobs:
|
||||||
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
|
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
|
||||||
|
|
||||||
- name: Set up the environment
|
- name: Set up the environment
|
||||||
uses: ./.github/actions/setup-poetry-env
|
uses: ./.github/actions/setup-uv-env
|
||||||
|
|
||||||
- name: Run checks
|
- name: Run checks
|
||||||
run: make check
|
run: make check
|
||||||
|
|
||||||
tox:
|
test:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
strategy:
|
||||||
matrix:
|
matrix:
|
||||||
|
|
@ -35,30 +35,13 @@ jobs:
|
||||||
- name: Check out
|
- name: Check out
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
- name: Set up python
|
- name: Set up the environment
|
||||||
uses: actions/setup-python@v5
|
uses: ./.github/actions/setup-uv-env
|
||||||
with:
|
with:
|
||||||
python-version: ${{ matrix.python-version }}
|
python-version: ${{ matrix.python-version }}
|
||||||
|
|
||||||
- name: Install Poetry
|
- name: Test libs
|
||||||
uses: snok/install-poetry@v1
|
run: make test
|
||||||
with:
|
|
||||||
version: 1.8.5
|
|
||||||
|
|
||||||
- name: Load cached venv
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: .tox
|
|
||||||
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('arcade/poetry.lock') }}
|
|
||||||
|
|
||||||
- name: Install tox
|
|
||||||
run: |
|
|
||||||
python -m pip install --upgrade pip
|
|
||||||
python -m pip install tox tox-gh-actions
|
|
||||||
|
|
||||||
- name: Test with tox
|
|
||||||
run: cd arcade && tox
|
|
||||||
|
|
||||||
- name: Upload coverage reports to Codecov with GitHub Action on Python 3.10
|
- name: Upload coverage reports to Codecov with GitHub Action on Python 3.10
|
||||||
uses: codecov/codecov-action@v4.0.1
|
uses: codecov/codecov-action@v4.0.1
|
||||||
if: ${{ matrix.python-version == '3.10' }}
|
if: ${{ matrix.python-version == '3.10' }}
|
||||||
|
|
|
||||||
57
.github/workflows/test-toolkits.yml
vendored
57
.github/workflows/test-toolkits.yml
vendored
|
|
@ -11,46 +11,43 @@ jobs:
|
||||||
setup:
|
setup:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
outputs:
|
outputs:
|
||||||
tool_matrix: ${{ steps.dataStep.outputs.tools }}
|
tool_matrix: ${{ steps.get-toolkits.outputs.toolkits }}
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout code
|
- name: Check out
|
||||||
uses: actions/checkout@v3
|
uses: actions/checkout@v4
|
||||||
with:
|
|
||||||
fetch-depth: 0
|
|
||||||
|
|
||||||
- name: Get Toolkits
|
- name: Get toolkits
|
||||||
id: dataStep
|
id: get-toolkits
|
||||||
run: |
|
run: |
|
||||||
TARGETS=$(./.github/scripts/get_toolkits.sh)
|
# Find all directories in toolkits/ that have a pyproject.toml
|
||||||
echo "tools=$(jq -cn --argjson environments "$TARGETS" '{target: $environments}')" >> $GITHUB_OUTPUT
|
TOOLKITS=$(find toolkits -maxdepth 1 -type d -not -name "toolkits" -exec test -f {}/pyproject.toml \; -exec basename {} \; | jq -R -s -c 'split("\n")[:-1]')
|
||||||
|
echo "Found toolkits: $TOOLKITS"
|
||||||
|
echo "toolkits=$TOOLKITS" >> $GITHUB_OUTPUT
|
||||||
|
|
||||||
test-toolkits:
|
test-toolkits:
|
||||||
needs: setup
|
needs: setup
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
strategy:
|
||||||
matrix: ${{ fromJson(needs.setup.outputs.tool_matrix) }}
|
matrix:
|
||||||
|
toolkit: ${{ fromJson(needs.setup.outputs.tool_matrix) }}
|
||||||
|
fail-fast: true
|
||||||
steps:
|
steps:
|
||||||
- run: echo ${{ matrix.target }}
|
- name: Check out
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
- name: Checkout code
|
- name: Set up the environment
|
||||||
uses: actions/checkout@v3
|
uses: ./.github/actions/setup-uv-env
|
||||||
with:
|
|
||||||
fetch-depth: 0
|
|
||||||
|
|
||||||
- name: Install Poetry
|
- name: Install toolkit dependencies
|
||||||
uses: snok/install-poetry@v1
|
working-directory: toolkits/${{ matrix.toolkit }}
|
||||||
with:
|
run: uv pip install -e ".[dev]"
|
||||||
version: 1.8.5
|
|
||||||
|
|
||||||
- uses: actions/setup-python@v5
|
- name: Check toolkit
|
||||||
with:
|
working-directory: toolkits/${{ matrix.toolkit }}
|
||||||
python-version: '3.12'
|
run: |
|
||||||
cache: 'pip'
|
uv run --active pre-commit run -a
|
||||||
|
uv run --active mypy --config-file=pyproject.toml
|
||||||
|
|
||||||
- name: Test Toolkit
|
- name: Test toolkit
|
||||||
id: Test_Toolkit
|
working-directory: toolkits/${{ matrix.toolkit }}
|
||||||
working-directory: toolkits/${{ matrix.target }}
|
run: uv run --active pytest -W ignore -v --cov=arcade_${{ matrix.toolkit }} --cov-report=xml
|
||||||
run: |
|
|
||||||
make install
|
|
||||||
make check
|
|
||||||
make test
|
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,7 @@ repos:
|
||||||
- id: check-case-conflict
|
- id: check-case-conflict
|
||||||
- id: check-merge-conflict
|
- id: check-merge-conflict
|
||||||
- id: check-toml
|
- id: check-toml
|
||||||
|
exclude: ".*/templates/.*"
|
||||||
- id: check-yaml
|
- id: check-yaml
|
||||||
exclude: ".*/templates/.*"
|
exclude: ".*/templates/.*"
|
||||||
- id: end-of-file-fixer
|
- id: end-of-file-fixer
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,10 @@ target-version = "py39"
|
||||||
line-length = 100
|
line-length = 100
|
||||||
fix = true
|
fix = true
|
||||||
|
|
||||||
|
exclude = [
|
||||||
|
"libs/arcade-cli/arcade_cli/templates/",
|
||||||
|
]
|
||||||
|
|
||||||
[lint]
|
[lint]
|
||||||
select = [
|
select = [
|
||||||
# flake8-2020
|
# flake8-2020
|
||||||
|
|
|
||||||
28
.vscode/launch.json
vendored
28
.vscode/launch.json
vendored
|
|
@ -5,7 +5,7 @@
|
||||||
"name": "Debug `arcade workerup --no-auth`",
|
"name": "Debug `arcade workerup --no-auth`",
|
||||||
"type": "debugpy",
|
"type": "debugpy",
|
||||||
"request": "launch",
|
"request": "launch",
|
||||||
"program": "${workspaceFolder}/arcade/run_cli.py",
|
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
|
||||||
"args": ["workerup", "--no-auth"],
|
"args": ["workerup", "--no-auth"],
|
||||||
"console": "integratedTerminal",
|
"console": "integratedTerminal",
|
||||||
"jinja": true,
|
"jinja": true,
|
||||||
|
|
@ -16,34 +16,34 @@
|
||||||
"name": "Debug `arcade chat -d -h localhost`",
|
"name": "Debug `arcade chat -d -h localhost`",
|
||||||
"type": "debugpy",
|
"type": "debugpy",
|
||||||
"request": "launch",
|
"request": "launch",
|
||||||
"program": "${workspaceFolder}/arcade/run_cli.py",
|
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
|
||||||
"args": ["chat", "-d", "-h", "localhost"],
|
"args": ["chat", "-d", "-h", "localhost"],
|
||||||
"console": "integratedTerminal",
|
"console": "integratedTerminal",
|
||||||
"jinja": true,
|
"jinja": true,
|
||||||
"justMyCode": true,
|
"justMyCode": true,
|
||||||
"cwd": "${workspaceFolder}"
|
"cwd": "${workspaceFolder}"
|
||||||
},
|
},
|
||||||
{
|
|
||||||
"name": "Debug `arcade dev`",
|
|
||||||
"type": "debugpy",
|
|
||||||
"request": "launch",
|
|
||||||
"program": "${workspaceFolder}/arcade/run_cli.py",
|
|
||||||
"args": ["dev"],
|
|
||||||
"console": "integratedTerminal",
|
|
||||||
"jinja": true,
|
|
||||||
"justMyCode": true,
|
|
||||||
"cwd": "${workspaceFolder}"
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"name": "Debug `arcade evals -d` on current file",
|
"name": "Debug `arcade evals -d` on current file",
|
||||||
"type": "debugpy",
|
"type": "debugpy",
|
||||||
"request": "launch",
|
"request": "launch",
|
||||||
"program": "${workspaceFolder}/arcade/run_cli.py",
|
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
|
||||||
"args": ["evals", "-d", "${fileDirname}", "-h", "localhost"],
|
"args": ["evals", "-d", "${fileDirname}", "-h", "localhost"],
|
||||||
"console": "integratedTerminal",
|
"console": "integratedTerminal",
|
||||||
"jinja": true,
|
"jinja": true,
|
||||||
"justMyCode": true,
|
"justMyCode": true,
|
||||||
"cwd": ""
|
"cwd": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Debug `arcade serve`",
|
||||||
|
"type": "debugpy",
|
||||||
|
"request": "launch",
|
||||||
|
"program": "${workspaceFolder}/libs/arcade-cli/run_cli.py",
|
||||||
|
"args": ["serve"],
|
||||||
|
"console": "integratedTerminal",
|
||||||
|
"jinja": true,
|
||||||
|
"justMyCode": true,
|
||||||
|
"cwd": ""
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -45,7 +45,7 @@ If you are proposing a new feature:
|
||||||
# Get Started!
|
# Get Started!
|
||||||
|
|
||||||
Ready to contribute? Here's how to set up `arcade-ai` for local development.
|
Ready to contribute? Here's how to set up `arcade-ai` for local development.
|
||||||
Please note this documentation assumes you already have `poetry` and `Git` installed and ready to go.
|
Please note this documentation assumes you already have `uv` and `Git` installed and ready to go.
|
||||||
|
|
||||||
1. Fork the `arcade-ai` repo on GitHub.
|
1. Fork the `arcade-ai` repo on GitHub.
|
||||||
|
|
||||||
|
|
@ -53,7 +53,7 @@ Please note this documentation assumes you already have `poetry` and `Git` insta
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd <directory_in_which_repo_should_be_created>
|
cd <directory_in_which_repo_should_be_created>
|
||||||
git clone git@github.com:YOUR_NAME/arcade-ai.git
|
git clone git@github.com:YOUR_GITHUB_USERNAME/arcade-ai.git
|
||||||
```
|
```
|
||||||
|
|
||||||
3. Now we need to install the environment. Navigate into the directory
|
3. Now we need to install the environment. Navigate into the directory
|
||||||
|
|
@ -62,25 +62,30 @@ git clone git@github.com:YOUR_NAME/arcade-ai.git
|
||||||
cd arcade-ai
|
cd arcade-ai
|
||||||
```
|
```
|
||||||
|
|
||||||
If you are using `pyenv`, select a version to use locally. (See installed versions with `pyenv versions`)
|
Create your virtual environment
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pyenv local <x.y.z>
|
uv venv --python 3.11.6
|
||||||
```
|
```
|
||||||
|
|
||||||
Then, install and activate the environment with:
|
4. Install the development environment and dependencies:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
poetry install
|
# Install all packages and development dependencies via uv workspace
|
||||||
poetry shell
|
uv sync --extra all --dev
|
||||||
|
|
||||||
|
# Install pre-commit hooks for code quality
|
||||||
|
uv run pre-commit install
|
||||||
```
|
```
|
||||||
|
|
||||||
4. Install pre-commit to run linters/formatters at commit time:
|
Or use the convenient Makefile command that does both:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
poetry run pre-commit install
|
make install
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The uv workspace will automatically handle installing all lib packages in the correct dependency order.
|
||||||
|
|
||||||
5. Create a branch for local development:
|
5. Create a branch for local development:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -89,7 +94,7 @@ git checkout -b name-of-your-bugfix-or-feature
|
||||||
|
|
||||||
Now you can make your changes locally.
|
Now you can make your changes locally.
|
||||||
|
|
||||||
6. Don't forget to add test cases for your added functionality to the `tests` directory.
|
6. Don't forget to add test cases for your added functionality to the `libs/tests` directory.
|
||||||
|
|
||||||
7. When you're done making changes, check that your changes pass the formatting tests.
|
7. When you're done making changes, check that your changes pass the formatting tests.
|
||||||
|
|
||||||
|
|
@ -103,15 +108,14 @@ Now, validate that all unit tests are passing:
|
||||||
make test
|
make test
|
||||||
```
|
```
|
||||||
|
|
||||||
9. Before raising a pull request you should also run tox.
|
8. You can also run tests for specific components:
|
||||||
This will run the tests across different versions of Python:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tox
|
# Test all lib packages
|
||||||
|
make test
|
||||||
```
|
```
|
||||||
|
|
||||||
This requires you to have multiple versions of python installed.
|
9. The CI/CD pipeline will run additional checks across different Python versions, so local testing with a single version is usually sufficient.
|
||||||
This step is also triggered in the CI/CD pipeline, so you could also choose to skip this step locally.
|
|
||||||
|
|
||||||
10. Commit your changes and push your branch to GitHub:
|
10. Commit your changes and push your branch to GitHub:
|
||||||
|
|
||||||
|
|
@ -129,8 +133,7 @@ Before you submit a pull request, check that it meets these guidelines:
|
||||||
|
|
||||||
1. The pull request should include tests.
|
1. The pull request should include tests.
|
||||||
|
|
||||||
2. If the pull request adds functionality, the docs should be updated.
|
2. If the pull request adds functionality, the [docs](https://github.com/ArcadeAI/docs) should be updated.
|
||||||
Put your new functionality into a function with a docstring, and add the feature to the list in `README.md`.
|
|
||||||
|
|
||||||
3. If making contributions to multiple toolkits (i.e. Google and Slack, etc.), submit a separate pull request for each.
|
3. If making contributions to multiple toolkits (i.e. Google and Slack, etc.), submit a separate pull request for each.
|
||||||
This helps us segregate the changes during the review process making it more efficient.
|
This helps us segregate the changes during the review process making it more efficient.
|
||||||
|
|
|
||||||
224
Makefile
224
Makefile
|
|
@ -1,30 +1,64 @@
|
||||||
VERSION ?= "0.1.0.dev0"
|
CLI_VERSION ?= "2.0.0"
|
||||||
|
TDK_VERSION ?= "2.0.0"
|
||||||
|
SERVE_VERSION ?= "2.0.0"
|
||||||
|
CORE_VERSION ?= "2.0.0"
|
||||||
|
|
||||||
.PHONY: install
|
.PHONY: install
|
||||||
install: ## Install the poetry environment and install the pre-commit hooks
|
install: ## Install the uv environment and all packages with dependencies
|
||||||
@echo "🚀 Creating virtual environment using pyenv and poetry"
|
@echo "🚀 Creating virtual environment and installing all packages using uv workspace"
|
||||||
@cd arcade && poetry install --all-extras
|
@uv sync --active --dev --extra all
|
||||||
@cd arcade && poetry run pre-commit install
|
@uv run pre-commit install
|
||||||
|
@echo "✅ All packages and dependencies installed via uv workspace"
|
||||||
|
|
||||||
.PHONY: install-toolkits
|
.PHONY: install-toolkits
|
||||||
install-toolkits: ## Install dependencies for all toolkits
|
install-toolkits: ## Install dependencies for all toolkits
|
||||||
@echo "🚀 Installing dependencies for all toolkits"
|
@echo "🚀 Installing dependencies for all toolkits"
|
||||||
@for dir in toolkits/*/ ; do \
|
@failed=0; \
|
||||||
echo "📦 Installing dependencies for $$dir"; \
|
successful=0; \
|
||||||
(cd $$dir && poetry lock && poetry install); \
|
for dir in toolkits/*/ ; do \
|
||||||
done
|
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
|
||||||
|
echo "📦 Installing dependencies for $$dir"; \
|
||||||
|
if (cd $$dir && uv pip install -e ".[dev]"); then \
|
||||||
|
successful=$$((successful + 1)); \
|
||||||
|
else \
|
||||||
|
echo "❌ Failed to install dependencies for $$dir"; \
|
||||||
|
failed=$$((failed + 1)); \
|
||||||
|
fi; \
|
||||||
|
else \
|
||||||
|
echo "⚠️ Skipping $$dir (no pyproject.toml found)"; \
|
||||||
|
fi; \
|
||||||
|
done; \
|
||||||
|
echo ""; \
|
||||||
|
echo "📊 Installation Summary:"; \
|
||||||
|
echo " ✅ Successful: $$successful toolkits"; \
|
||||||
|
echo " ❌ Failed: $$failed toolkits"; \
|
||||||
|
if [ $$failed -gt 0 ]; then \
|
||||||
|
echo ""; \
|
||||||
|
echo "⚠️ Some toolkit installations failed. Check the output above for details."; \
|
||||||
|
exit 1; \
|
||||||
|
else \
|
||||||
|
echo ""; \
|
||||||
|
echo "🎉 All toolkit dependencies installed successfully!"; \
|
||||||
|
fi
|
||||||
|
|
||||||
.PHONY: check
|
.PHONY: check
|
||||||
check: ## Run code quality tools.
|
check: ## Run code quality tools.
|
||||||
@echo "🚀 Checking Poetry lock file consistency with 'pyproject.toml': Running poetry check --lock"
|
|
||||||
@cd arcade && poetry check --lock
|
|
||||||
@echo "🚀 Linting code: Running pre-commit"
|
@echo "🚀 Linting code: Running pre-commit"
|
||||||
@cd arcade && poetry run pre-commit run -a
|
@uv run pre-commit run -a
|
||||||
@echo "🚀 Static type checking: Running mypy"
|
@echo "🚀 Static type checking: Running mypy on libs"
|
||||||
@cd arcade && poetry run mypy $(git ls-files '*.py')
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
echo "🔍 Type checking $$lib"; \
|
||||||
|
(cd $$lib && uv run mypy . || true); \
|
||||||
|
done
|
||||||
|
|
||||||
|
.PHONY: check-libs
|
||||||
|
check-libs: ## Run code quality tools for each lib package
|
||||||
|
@echo "🚀 Running checks on each lib package"
|
||||||
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
echo "🛠️ Checking lib $$lib"; \
|
||||||
|
(cd $$lib && uv run pre-commit run -a || true); \
|
||||||
|
(cd $$lib && uv run mypy . || true); \
|
||||||
|
done
|
||||||
|
|
||||||
.PHONY: check-toolkits
|
.PHONY: check-toolkits
|
||||||
check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
|
check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
|
||||||
|
|
@ -32,7 +66,7 @@ check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
|
||||||
@for dir in toolkits/*/ ; do \
|
@for dir in toolkits/*/ ; do \
|
||||||
if [ -f "$$dir/Makefile" ]; then \
|
if [ -f "$$dir/Makefile" ]; then \
|
||||||
echo "🛠️ Checking toolkit $$dir"; \
|
echo "🛠️ Checking toolkit $$dir"; \
|
||||||
(cd "$$dir" && make check); \
|
(cd "$$dir" && uv run --active pre-commit run -a && uv run --active mypy --config-file=pyproject.toml); \
|
||||||
else \
|
else \
|
||||||
echo "🛠️ Skipping toolkit $$dir (no Makefile found)"; \
|
echo "🛠️ Skipping toolkit $$dir (no Makefile found)"; \
|
||||||
fi; \
|
fi; \
|
||||||
|
|
@ -40,69 +74,129 @@ check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
|
||||||
|
|
||||||
.PHONY: test
|
.PHONY: test
|
||||||
test: ## Test the code with pytest
|
test: ## Test the code with pytest
|
||||||
@echo "🚀 Testing code: Running pytest"
|
@echo "🚀 Testing libs: Running pytest"
|
||||||
@cd arcade && poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
|
@uv run pytest -W ignore -v --cov=libs/tests --cov-config=pyproject.toml --cov-report=xml
|
||||||
|
|
||||||
|
.PHONY: test-libs
|
||||||
|
test-libs: ## Test each lib package individually
|
||||||
|
@echo "🚀 Testing each lib package"
|
||||||
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
echo "🧪 Testing $$lib"; \
|
||||||
|
(cd $$lib && uv run pytest -W ignore -v || true); \
|
||||||
|
done
|
||||||
|
|
||||||
.PHONY: test-toolkits
|
.PHONY: test-toolkits
|
||||||
test-toolkits: ## Iterate over all toolkits and run pytest on each one
|
test-toolkits: ## Iterate over all toolkits and run pytest on each one
|
||||||
@echo "🚀 Testing code in toolkits: Running pytest"
|
@echo "🚀 Testing code in toolkits: Running pytest"
|
||||||
@for dir in toolkits/*/ ; do \
|
@for dir in toolkits/*/ ; do \
|
||||||
(cd $$dir && poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml || exit 1); \
|
toolkit_name=$$(basename "$$dir"); \
|
||||||
|
echo "🧪 Testing $$toolkit_name toolkit"; \
|
||||||
|
(cd $$dir && uv run --active pytest -W ignore -v --cov=arcade_$$toolkit_name --cov-report=xml || exit 1); \
|
||||||
done
|
done
|
||||||
|
|
||||||
.PHONY: coverage
|
.PHONY: coverage
|
||||||
coverage: ## Generate coverage report
|
coverage: ## Generate coverage report
|
||||||
@echo "coverage report"
|
@echo "coverage report"
|
||||||
@cd arcade && coverage report
|
@uv run coverage report
|
||||||
@echo "Generating coverage report"
|
@echo "Generating coverage report"
|
||||||
@cd arcade && coverage html
|
@uv run coverage html
|
||||||
|
|
||||||
.PHONY: set-version
|
.PHONY: set-version
|
||||||
set-version: ## Set the version in the pyproject.toml file
|
set-version: ## Set the version in all lib pyproject.toml files
|
||||||
@echo "🚀 Setting version in pyproject.toml"
|
@echo "🚀 Setting versions in all lib packages"
|
||||||
@cd arcade && poetry version $(VERSION)
|
@echo "Setting arcade-ai version to $(CLI_VERSION)"
|
||||||
|
@sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(CLI_VERSION)/' pyproject.toml && rm pyproject.toml.bak
|
||||||
|
@echo "Setting libs/arcade-tdk version to $(TDK_VERSION)"
|
||||||
|
@cd libs/arcade-tdk && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(TDK_VERSION)/' pyproject.toml && rm pyproject.toml.bak
|
||||||
|
@echo "Setting libs/arcade-serve version to $(SERVE_VERSION)"
|
||||||
|
@cd libs/arcade-serve && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(SERVE_VERSION)/' pyproject.toml && rm pyproject.toml.bak
|
||||||
|
@echo "Setting libs/arcade-core version to $(CORE_VERSION)"
|
||||||
|
@cd libs/arcade-core && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(CORE_VERSION)/' pyproject.toml && rm pyproject.toml.bak
|
||||||
|
|
||||||
.PHONY: unset-version
|
.PHONY: unset-version
|
||||||
unset-version: ## Set the version in the pyproject.toml file
|
unset-version: ## Reset version to 0.1.0 in all lib pyproject.toml files
|
||||||
@echo "🚀 Setting version in pyproject.toml"
|
@echo "🚀 Resetting version to 0.1.0 in all lib packages"
|
||||||
@cd arcade && poetry version 0.1.0
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
if [ -f "$$lib/pyproject.toml" ]; then \
|
||||||
|
echo "Resetting version in $$lib"; \
|
||||||
|
(cd $$lib && sed -i.bak 's/version = "[^"]*"/version = "0.1.0"/' pyproject.toml && rm pyproject.toml.bak); \
|
||||||
|
fi; \
|
||||||
|
done
|
||||||
|
|
||||||
.PHONY: build
|
.PHONY: build
|
||||||
build: clean-build ## Build wheel file using poetry
|
build: clean-build ## Build wheel files using uv
|
||||||
@echo "🚀 Creating wheel file"
|
@echo "🚀 Creating wheel files for all lib packages"
|
||||||
@cd arcade && poetry build
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
if [ -f "$$lib/pyproject.toml" ]; then \
|
||||||
|
echo "🛠️ Building $$lib"; \
|
||||||
|
(cd $$lib && uv build); \
|
||||||
|
fi; \
|
||||||
|
done
|
||||||
|
|
||||||
|
.PHONY: build-toolkits
|
||||||
|
build-toolkits: ## Build wheel files for all toolkits
|
||||||
|
@echo "🚀 Creating wheel files for all toolkits"
|
||||||
|
@failed=0; \
|
||||||
|
successful=0; \
|
||||||
|
for dir in toolkits/*/ ; do \
|
||||||
|
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
|
||||||
|
toolkit_name=$$(basename "$$dir"); \
|
||||||
|
echo "🛠️ Building toolkit $$toolkit_name"; \
|
||||||
|
if (cd $$dir && uv build); then \
|
||||||
|
successful=$$((successful + 1)); \
|
||||||
|
else \
|
||||||
|
echo "❌ Failed to build toolkit $$toolkit_name"; \
|
||||||
|
failed=$$((failed + 1)); \
|
||||||
|
fi; \
|
||||||
|
else \
|
||||||
|
echo "⚠️ Skipping $$dir (no pyproject.toml found)"; \
|
||||||
|
fi; \
|
||||||
|
done; \
|
||||||
|
echo ""; \
|
||||||
|
echo "📊 Build Summary:"; \
|
||||||
|
echo " ✅ Successful: $$successful toolkits"; \
|
||||||
|
echo " ❌ Failed: $$failed toolkits"; \
|
||||||
|
if [ $$failed -gt 0 ]; then \
|
||||||
|
echo ""; \
|
||||||
|
echo "⚠️ Some toolkit builds failed. Check the output above for details."; \
|
||||||
|
exit 1; \
|
||||||
|
else \
|
||||||
|
echo ""; \
|
||||||
|
echo "🎉 All toolkit wheels built successfully!"; \
|
||||||
|
fi
|
||||||
|
|
||||||
.PHONY: clean-build
|
.PHONY: clean-build
|
||||||
clean-build: ## clean build artifacts
|
clean-build: ## clean build artifacts
|
||||||
@cd arcade && rm -rf dist
|
@echo "🗑️ Cleaning build artifacts"
|
||||||
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
(cd $$lib && rm -rf dist); \
|
||||||
|
done
|
||||||
|
|
||||||
.PHONY: publish
|
.PHONY: publish
|
||||||
publish: ## publish a release to pypi.
|
publish: ## publish a release to pypi.
|
||||||
@echo "🚀 Publishing: Dry run."
|
@echo "🚀 Publishing all lib packages to PyPI"
|
||||||
@cd arcade && poetry config pypi-token.pypi $(PYPI_TOKEN)
|
@for lib in libs/arcade*/ ; do \
|
||||||
@cd arcade && poetry publish --dry-run
|
if [ -f "$$lib/pyproject.toml" ]; then \
|
||||||
@echo "🚀 Publishing."
|
echo "📦 Publishing $$lib"; \
|
||||||
@cd arcade && poetry publish
|
(cd $$lib && uv publish --token $(PYPI_TOKEN) || true); \
|
||||||
|
fi; \
|
||||||
|
done
|
||||||
|
|
||||||
.PHONY: build-and-publish
|
.PHONY: build-and-publish
|
||||||
build-and-publish: build publish ## Build and publish.
|
build-and-publish: build publish ## Build and publish.
|
||||||
|
|
||||||
.PHONY: docker
|
.PHONY: docker
|
||||||
docker: ## Build and run the Docker container
|
docker: ## Build and run the Docker container
|
||||||
@echo "🚀 Building arcade and toolkit wheels..."
|
@echo "🚀 Building lib packages and toolkit wheels..."
|
||||||
@make full-dist
|
@make full-dist
|
||||||
@echo "Writing requirements.txt"
|
|
||||||
@cd arcade && poetry export --output ../dist/requirements.txt
|
|
||||||
@echo "🚀 Building Docker image"
|
@echo "🚀 Building Docker image"
|
||||||
@cd docker && make docker-build
|
@cd docker && make docker-build
|
||||||
@cd docker && make docker-run
|
@cd docker && make docker-run
|
||||||
|
|
||||||
.PHONY: docker-base
|
.PHONY: docker-base
|
||||||
docker-base: ## Build and run the Docker container
|
docker-base: ## Build and run the Docker container
|
||||||
@echo "🚀 Building arcade and toolkit wheels..."
|
@echo "🚀 Building lib packages and toolkit wheels..."
|
||||||
@make full-dist
|
@make full-dist
|
||||||
@echo "Writing requirements.txt"
|
|
||||||
@cd arcade && poetry export --output ../dist/requirements.txt
|
|
||||||
@echo "🚀 Building Docker image"
|
@echo "🚀 Building Docker image"
|
||||||
@cd docker && INSTALL_TOOLKITS=false make docker-build
|
@cd docker && INSTALL_TOOLKITS=false make docker-build
|
||||||
@cd docker && INSTALL_TOOLKITS=false make docker-run
|
@cd docker && INSTALL_TOOLKITS=false make docker-run
|
||||||
|
|
@ -123,31 +217,42 @@ publish-ghcr: ## Publish to the GHCR
|
||||||
|
|
||||||
.PHONY: full-dist
|
.PHONY: full-dist
|
||||||
full-dist: clean-dist ## Build all projects and copy wheels to ./dist
|
full-dist: clean-dist ## Build all projects and copy wheels to ./dist
|
||||||
@echo " Building a full distribution with toolkits"
|
@echo "🛠️ Building a full distribution with lib packages and toolkits"
|
||||||
|
|
||||||
@echo "Setting version to $(VERSION)"
|
@echo "Setting version to $(CLI_VERSION)"
|
||||||
@make set-version
|
@make set-version
|
||||||
|
|
||||||
# @echo "🛠️ Building all projects and copying wheels to ./dist"
|
@echo "🛠️ Building all lib packages and copying wheels to ./dist"
|
||||||
@mkdir -p dist
|
@mkdir -p dist
|
||||||
|
|
||||||
# Build the main arcade project
|
# Build all lib packages in dependency order
|
||||||
@echo "🛠️ Building arcade project wheel..."
|
@for lib in arcade-core arcade-tdk arcade-serve ; do \
|
||||||
@cd arcade && poetry build
|
echo "🛠️ Building libs/$$lib wheel..."; \
|
||||||
|
(cd libs/$$lib && uv build); \
|
||||||
|
cp libs/$$lib/dist/*.whl dist/; \
|
||||||
|
done
|
||||||
|
|
||||||
# Copy the main arcade project wheel to the dist directory
|
@echo "🛠️ Building all toolkit packages and copying wheels to ./dist"
|
||||||
@cp arcade/dist/*.whl dist/
|
@for dir in toolkits/*/ ; do \
|
||||||
|
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
|
||||||
|
toolkit_name=$$(basename "$$dir"); \
|
||||||
|
echo "🛠️ Building toolkit $$toolkit_name wheel..."; \
|
||||||
|
(cd $$dir && uv build); \
|
||||||
|
cp $$dir/dist/*.whl dist/; \
|
||||||
|
fi; \
|
||||||
|
done
|
||||||
|
|
||||||
@echo "Reset version to default (0.1.0)"
|
@echo "Reset version to default (0.1.0)"
|
||||||
@make unset-version
|
@make unset-version
|
||||||
|
|
||||||
|
|
||||||
.PHONY: clean-dist
|
.PHONY: clean-dist
|
||||||
clean-dist: ## Clean all built distributions
|
clean-dist: ## Clean all built distributions
|
||||||
@echo "🗑️ Cleaning dist directory"
|
@echo "🗑️ Cleaning dist directory"
|
||||||
@rm -rf dist
|
@rm -rf dist
|
||||||
@echo "🗑️ Cleaning arcade/dist directory"
|
@echo "🗑️ Cleaning libs/*/dist directories"
|
||||||
@rm -rf arcade/dist
|
@for lib in libs/arcade*/ ; do \
|
||||||
|
rm -rf "$$lib"/dist; \
|
||||||
|
done
|
||||||
@echo "🗑️ Cleaning toolkits/*/dist directory"
|
@echo "🗑️ Cleaning toolkits/*/dist directory"
|
||||||
@for toolkit_dir in toolkits/*; do \
|
@for toolkit_dir in toolkits/*; do \
|
||||||
if [ -d "$$toolkit_dir" ]; then \
|
if [ -d "$$toolkit_dir" ]; then \
|
||||||
|
|
@ -155,11 +260,18 @@ clean-dist: ## Clean all built distributions
|
||||||
fi; \
|
fi; \
|
||||||
done
|
done
|
||||||
|
|
||||||
|
.PHONY: setup
|
||||||
|
setup: install ## Complete development setup (same as install)
|
||||||
|
|
||||||
|
.PHONY: lint
|
||||||
|
lint: check ## Alias for check command
|
||||||
|
|
||||||
|
.PHONY: clean
|
||||||
|
clean: clean-build clean-dist ## Clean all build and distribution artifacts
|
||||||
|
|
||||||
.PHONY: help
|
.PHONY: help
|
||||||
help:
|
help:
|
||||||
@echo "🛠️ Arcade Dev Commands:\n"
|
@echo "🛠️ Arcade Dev Commands:\n"
|
||||||
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
|
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.DEFAULT_GOAL := help
|
.DEFAULT_GOAL := help
|
||||||
|
|
|
||||||
52
README.md
52
README.md
|
|
@ -38,11 +38,17 @@
|
||||||
<a href="https://docs.arcade.dev/home/quickstart" target="_blank">Quickstart</a> •
|
<a href="https://docs.arcade.dev/home/quickstart" target="_blank">Quickstart</a> •
|
||||||
<a href="https://docs.arcade.dev/home/contact-us" target="_blank">Contact Us</a>
|
<a href="https://docs.arcade.dev/home/contact-us" target="_blank">Contact Us</a>
|
||||||
|
|
||||||
# Arcade Tool SDK
|
# Arcade AI Platform
|
||||||
|
|
||||||
Arcade is a developer platform that lets you build, deploy, and manage tools for AI agents.
|
Arcade is a developer platform that lets you build, deploy, and manage tools for AI agents.
|
||||||
|
|
||||||
The Tool SDK makes it easy to create powerful, secure tools that your agents can use to interact with the world.
|
This repository contains the core Arcade libraries, organized as separate packages for maximum flexibility and modularity:
|
||||||
|
|
||||||
|
- [**`arcade-core`**](libs/arcade-core) - Core platform functionality and schemas
|
||||||
|
- [**`arcade-tdk`**](libs/arcade-tdk) - Tool Development Kit with the `@tool` decorator
|
||||||
|
- [**`arcade-serve`**](libs/arcade-serve) - Serving infrastructure for workers and MCP servers
|
||||||
|
- [**`arcade-evals`**](libs/arcade-evals) - Evaluation framework for testing tool performance
|
||||||
|
- [**`arcade-cli`**](libs/arcade-cli) - Command-line interface for the Arcade platform
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -54,6 +60,48 @@ _Pst. hey, you, give us a star if you like it!_
|
||||||
<img src="https://img.shields.io/github/stars/ArcadeAI/arcade-ai.svg" alt="GitHub stars">
|
<img src="https://img.shields.io/github/stars/ArcadeAI/arcade-ai.svg" alt="GitHub stars">
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
For development, install all packages with dependencies using uv workspace:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install all packages and dev dependencies
|
||||||
|
uv sync --extra all --dev
|
||||||
|
|
||||||
|
# Or use the Makefile (includes pre-commit hooks)
|
||||||
|
make install
|
||||||
|
```
|
||||||
|
|
||||||
|
For production use, install individual packages as needed:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install arcade-ai # CLI
|
||||||
|
pip install 'arcade-ai[evals]' # CLI + Evaluation framework
|
||||||
|
pip install 'arcade-ai[all]' # CLI + Serving infra + eval framework + TDK
|
||||||
|
pip install arcade_serve # Serving infrastructure
|
||||||
|
pip install arcade-tdk # Tool Development Kit
|
||||||
|
```
|
||||||
|
|
||||||
|
### Development
|
||||||
|
|
||||||
|
Use the Makefile for standard tasks:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run tests
|
||||||
|
make test
|
||||||
|
|
||||||
|
# Run linting and type checking
|
||||||
|
make check
|
||||||
|
|
||||||
|
# Build all packages
|
||||||
|
make build
|
||||||
|
|
||||||
|
# See all available commands
|
||||||
|
make help
|
||||||
|
```
|
||||||
|
|
||||||
## Client Libraries
|
## Client Libraries
|
||||||
|
|
||||||
- **[ArcadeAI/arcade-py](https://github.com/ArcadeAI/arcade-py):**
|
- **[ArcadeAI/arcade-py](https://github.com/ArcadeAI/arcade-py):**
|
||||||
|
|
|
||||||
|
|
@ -1,13 +0,0 @@
|
||||||
# Arcade Python SDK and CLI
|
|
||||||
|
|
||||||
[Arcade](https://arcade.dev?ref=pypi) provides developer-focused tooling and APIs designed to improve the capabilities of LLM applications and agents.
|
|
||||||
|
|
||||||
By removing the complexity of connecting agentic applications with your users' data and services, Arcade enables developers to focus on building their agentic applications.
|
|
||||||
|
|
||||||
To learn more, check out our
|
|
||||||
- [Website](https://arcade.dev?ref=pypi)
|
|
||||||
- [GitHub](https://github.com/ArcadeAI/arcade-ai)
|
|
||||||
- [Documentation](https://docs.arcade.dev)
|
|
||||||
- [Discord](https://discord.com/invite/GUZEMpEZ9p)
|
|
||||||
- [X](https://x.com/TryArcade)
|
|
||||||
- [LinkedIn](https://www.linkedin.com/company/arcade-ai)
|
|
||||||
|
|
@ -1,3 +0,0 @@
|
||||||
from importlib.metadata import version as get_version
|
|
||||||
|
|
||||||
__version__ = get_version("arcade-ai")
|
|
||||||
|
|
@ -1,505 +0,0 @@
|
||||||
import http.client
|
|
||||||
import io
|
|
||||||
import ipaddress
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import signal
|
|
||||||
import socket
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
from rich.console import Console
|
|
||||||
|
|
||||||
console = Console(highlight=False)
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
known_engine_config_locations = [
|
|
||||||
"/etc/arcade-ai",
|
|
||||||
"/etc/arcade-engine",
|
|
||||||
"/opt/homebrew/etc/arcade-engine",
|
|
||||||
]
|
|
||||||
|
|
||||||
if os.environ.get("HOMEBREW_REPOSITORY") is not None:
|
|
||||||
homebrew_home = os.path.join(os.environ["HOMEBREW_REPOSITORY"], "etc", "arcade-engine")
|
|
||||||
if homebrew_home not in known_engine_config_locations:
|
|
||||||
known_engine_config_locations.append(homebrew_home)
|
|
||||||
|
|
||||||
|
|
||||||
def start_servers(
|
|
||||||
worker_host: str,
|
|
||||||
worker_port: int,
|
|
||||||
engine_config: str | None,
|
|
||||||
engine_env: str | None = None,
|
|
||||||
debug: bool = False,
|
|
||||||
) -> None:
|
|
||||||
"""
|
|
||||||
Start the worker and engine servers.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
host: Host for the worker server.
|
|
||||||
port: Port for the worker server.
|
|
||||||
engine_config: Path to the engine configuration file.
|
|
||||||
engine_env: Path to the engine environment file.
|
|
||||||
debug: Whether to run in debug mode.
|
|
||||||
"""
|
|
||||||
# Validate host and port
|
|
||||||
worker_host = _validate_host(worker_host)
|
|
||||||
worker_port = _validate_port(worker_port)
|
|
||||||
|
|
||||||
# Ensure engine_config is provided and validated
|
|
||||||
engine_config = _get_config_file(engine_config, default_filename="engine.yaml")
|
|
||||||
|
|
||||||
# Ensure engine_env is provided or found and either way, validated
|
|
||||||
env_file = _get_config_file(engine_env, default_filename="engine.env", optional=True)
|
|
||||||
|
|
||||||
# Prepare command-line arguments for the worker server and engine
|
|
||||||
worker_cmd = _build_worker_command(worker_host, worker_port, debug)
|
|
||||||
|
|
||||||
# even if the user didn't pass an env file we may have found it in the default locations
|
|
||||||
engine_cmd = _build_engine_command(engine_config, engine_env=env_file if env_file else None)
|
|
||||||
|
|
||||||
# Start and manage the processes
|
|
||||||
_manage_processes(worker_cmd, worker_host, worker_port, engine_cmd, debug=debug)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_host(host: str) -> str:
|
|
||||||
"""
|
|
||||||
Validates the host input.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
host: Host for the worker server.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The validated host as a string.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
ValueError: If the host is invalid.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
# Validate IP address
|
|
||||||
ipaddress.ip_address(host)
|
|
||||||
except ValueError:
|
|
||||||
# Optionally, validate hostname
|
|
||||||
if not host.isalnum() and "-" not in host and "." not in host:
|
|
||||||
console.print(f"❌ Invalid host: {host}", style="bold red")
|
|
||||||
raise ValueError("Invalid host.")
|
|
||||||
return host
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_port(port: int) -> int:
|
|
||||||
"""
|
|
||||||
Validates the port input.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
port: Port for the worker server.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The validated port as an integer.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
ValueError: If the port is out of the valid range.
|
|
||||||
"""
|
|
||||||
if not (1 <= port <= 65535):
|
|
||||||
console.print(f"❌ Invalid port: {port}", style="bold red")
|
|
||||||
raise ValueError("Invalid port.")
|
|
||||||
return port
|
|
||||||
|
|
||||||
|
|
||||||
def _get_config_file(
|
|
||||||
file_path: str | None, default_filename: str = "engine.yaml", optional: bool = False
|
|
||||||
) -> str | None:
|
|
||||||
"""
|
|
||||||
Resolves and validates the config file path from a set of candidate locations.
|
|
||||||
|
|
||||||
If a file_path is provided, it is checked directly.
|
|
||||||
Otherwise, the following candidate locations are checked in order:
|
|
||||||
1. Current working directory.
|
|
||||||
2. User's home directory under .arcade.
|
|
||||||
3. Known engine config locations.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
file_path: Optional path provided by the user.
|
|
||||||
default_filename: The default filename to look for.
|
|
||||||
optional: Whether the config file is optional.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The resolved config file path. None if the file is optional and not found.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
RuntimeError: If the config file is not found and is not optional.
|
|
||||||
"""
|
|
||||||
if file_path:
|
|
||||||
candidate = Path(os.path.expanduser(file_path)).resolve()
|
|
||||||
if not candidate.is_file():
|
|
||||||
console.print(f"❌ Config file not found at {candidate}", style="bold red")
|
|
||||||
raise RuntimeError(f"Config file not found at {candidate}")
|
|
||||||
return str(candidate)
|
|
||||||
|
|
||||||
# List of all config file path locations to check.
|
|
||||||
candidates = [
|
|
||||||
Path(os.getcwd()) / default_filename,
|
|
||||||
Path.home() / ".arcade" / default_filename,
|
|
||||||
]
|
|
||||||
candidates.extend(Path(path) / default_filename for path in known_engine_config_locations)
|
|
||||||
|
|
||||||
# Find the first candidate that exists.
|
|
||||||
for candidate in candidates:
|
|
||||||
if candidate.is_file():
|
|
||||||
console.print(f"Using config file at {candidate}", style="bold green")
|
|
||||||
return str(candidate)
|
|
||||||
|
|
||||||
# No config file was found. Handle according to the optional flag.
|
|
||||||
if optional:
|
|
||||||
console.print(
|
|
||||||
f"⚠️ Optional config file '{default_filename}' not found in any of the following locations:",
|
|
||||||
style="bold yellow",
|
|
||||||
)
|
|
||||||
for i, candidate in enumerate(candidates, start=1):
|
|
||||||
console.print(f" {i}) {candidate}", style="bold yellow")
|
|
||||||
return None
|
|
||||||
|
|
||||||
console.print(
|
|
||||||
f"❌ Error: Required config file '{default_filename}' not found in any of the following locations:",
|
|
||||||
style="bold red",
|
|
||||||
)
|
|
||||||
for i, candidate in enumerate(candidates, start=1):
|
|
||||||
console.print(f" {i}) {candidate}", style="bold red")
|
|
||||||
|
|
||||||
console.print(
|
|
||||||
"\nTIP: Please install the Arcade Engine by following the instructions at:\n"
|
|
||||||
" https://docs.arcade.dev/home/install/local#install-the-engine\n",
|
|
||||||
style="bold green",
|
|
||||||
)
|
|
||||||
raise RuntimeError(f"Config file '{default_filename}' not found.")
|
|
||||||
|
|
||||||
|
|
||||||
def _build_worker_command(host: str, port: int, debug: bool) -> list[str]:
|
|
||||||
"""
|
|
||||||
Builds the command to start the worker server.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
host: Host for the worker server.
|
|
||||||
port: Port for the worker server.
|
|
||||||
debug: Whether to run in debug mode.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The command as a list.
|
|
||||||
"""
|
|
||||||
# Expand full path to "arcade" executable
|
|
||||||
arcade_bin = shutil.which("arcade")
|
|
||||||
if not arcade_bin:
|
|
||||||
console.print(
|
|
||||||
"❌ Arcade binary not found, please install with `pip install arcade-ai`",
|
|
||||||
style="bold red",
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
cmd = [
|
|
||||||
arcade_bin,
|
|
||||||
"workerup",
|
|
||||||
"--host",
|
|
||||||
host,
|
|
||||||
"--port",
|
|
||||||
str(port),
|
|
||||||
]
|
|
||||||
if debug:
|
|
||||||
cmd.append("--debug")
|
|
||||||
return cmd
|
|
||||||
|
|
||||||
|
|
||||||
def _build_engine_command(engine_config: str | None, engine_env: str | None = None) -> list[str]:
|
|
||||||
"""
|
|
||||||
Builds the command to start the engine.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
engine_config: Path to the engine configuration file.
|
|
||||||
engine_env: Path to the engine environment file.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
The command as a list.
|
|
||||||
"""
|
|
||||||
# This should never happen, but we'll check regardless
|
|
||||||
if not engine_config:
|
|
||||||
console.print("❌ Engine configuration file not found", style="bold red")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
engine_bin = shutil.which("arcade-engine")
|
|
||||||
if not engine_bin:
|
|
||||||
console.print(
|
|
||||||
"❌ Engine binary not found, refer to the installation guide at "
|
|
||||||
"https://docs.arcade.dev/guides/installation for how to install the engine",
|
|
||||||
style="bold red",
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
cmd = [
|
|
||||||
engine_bin,
|
|
||||||
"-c",
|
|
||||||
engine_config,
|
|
||||||
]
|
|
||||||
if engine_env:
|
|
||||||
cmd.append("-e")
|
|
||||||
cmd.append(engine_env)
|
|
||||||
|
|
||||||
return cmd
|
|
||||||
|
|
||||||
|
|
||||||
def _manage_processes(
|
|
||||||
worker_cmd: list[str],
|
|
||||||
worker_host: str,
|
|
||||||
worker_port: int,
|
|
||||||
engine_cmd: list[str],
|
|
||||||
engine_env: dict[str, str] | None = None,
|
|
||||||
debug: bool = False,
|
|
||||||
) -> None:
|
|
||||||
"""
|
|
||||||
Manages the lifecycle of the worker and engine processes.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
worker_cmd: The command to start the worker server.
|
|
||||||
engine_cmd: The command to start the engine.
|
|
||||||
engine_env: Environment variables to set for the engine.
|
|
||||||
debug: Whether to run in debug mode.
|
|
||||||
"""
|
|
||||||
worker_process: subprocess.Popen | None = None
|
|
||||||
engine_process: subprocess.Popen | None = None
|
|
||||||
|
|
||||||
def terminate_processes(exit_program: bool = False) -> None:
|
|
||||||
console.print("Terminating child processes...", style="bold yellow")
|
|
||||||
_terminate_process(worker_process)
|
|
||||||
_terminate_process(engine_process)
|
|
||||||
if exit_program:
|
|
||||||
sys.exit(0)
|
|
||||||
|
|
||||||
_setup_signal_handlers(terminate_processes)
|
|
||||||
|
|
||||||
retry_count = 0
|
|
||||||
max_retries = 1 # Define the maximum number of retries
|
|
||||||
|
|
||||||
while retry_count <= max_retries:
|
|
||||||
try:
|
|
||||||
# Start the worker server
|
|
||||||
console.print("Starting worker server...", style="bold green")
|
|
||||||
worker_process = _start_process("Worker", worker_cmd, debug=debug)
|
|
||||||
|
|
||||||
_wait_for_healthy_worker(worker_process, worker_host, worker_port)
|
|
||||||
|
|
||||||
# Start the engine
|
|
||||||
console.print("Starting engine...", style="bold green")
|
|
||||||
engine_process = _start_process("Engine", engine_cmd, env=engine_env, debug=debug)
|
|
||||||
|
|
||||||
# Monitor processes
|
|
||||||
_monitor_processes(worker_process, engine_process)
|
|
||||||
|
|
||||||
# If we reach here, one of the processes has exited
|
|
||||||
retry_count += 1
|
|
||||||
console.print(
|
|
||||||
f"Processes exited. Retry {retry_count} of {max_retries}.", style="bold yellow"
|
|
||||||
)
|
|
||||||
|
|
||||||
if retry_count >= max_retries:
|
|
||||||
console.print(f"❌ Exiting after {max_retries} retries", style="bold red")
|
|
||||||
terminate_processes(exit_program=True)
|
|
||||||
break # Exit the loop
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
console.print(f"❌ Exception occurred: {e}", style="bold red")
|
|
||||||
terminate_processes()
|
|
||||||
retry_count += 1
|
|
||||||
if retry_count > max_retries:
|
|
||||||
console.print(
|
|
||||||
f"❌ Exiting after {retry_count - 1} retries due to exceptions",
|
|
||||||
style="bold red",
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
break # Not strictly necessary, but good practice
|
|
||||||
|
|
||||||
console.print("Exiting...", style="bold red")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
def _start_process(
|
|
||||||
name: str, cmd: list[str], env: dict[str, str] | None = None, debug: bool = False
|
|
||||||
) -> subprocess.Popen:
|
|
||||||
"""
|
|
||||||
Starts a subprocess and begins streaming its output.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
name: Name of the process.
|
|
||||||
cmd: Command to execute.
|
|
||||||
env: Environment variables to set for the process.
|
|
||||||
debug: Whether to run in debug mode.
|
|
||||||
Returns:
|
|
||||||
The subprocess.Popen object.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
RuntimeError: If the process fails to start.
|
|
||||||
"""
|
|
||||||
_env = os.environ.copy()
|
|
||||||
if env:
|
|
||||||
_env.update(env)
|
|
||||||
|
|
||||||
if debug:
|
|
||||||
_env["GIN_MODE"] = "debug"
|
|
||||||
else:
|
|
||||||
_env["GIN_MODE"] = "release"
|
|
||||||
|
|
||||||
if name == "Worker":
|
|
||||||
_env["PYTHONUNBUFFERED"] = "1"
|
|
||||||
|
|
||||||
try:
|
|
||||||
process = subprocess.Popen( # noqa: S603, RUF100
|
|
||||||
cmd,
|
|
||||||
env=_env,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.PIPE,
|
|
||||||
universal_newlines=True,
|
|
||||||
bufsize=1,
|
|
||||||
shell=False,
|
|
||||||
)
|
|
||||||
_stream_output(process, name)
|
|
||||||
return process # noqa: TRY300
|
|
||||||
except Exception as e:
|
|
||||||
console.print(f"❌ Failed to start {name}: {e}", style="bold red")
|
|
||||||
raise RuntimeError(f"Failed to start {name}")
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_for_healthy_worker(
|
|
||||||
worker_process: subprocess.Popen, worker_host: str, worker_port: int
|
|
||||||
) -> None:
|
|
||||||
"""Wait until an HTTP request to `host:port/worker/health` returns 200"""
|
|
||||||
|
|
||||||
while worker_process.poll() is None: # Continue waiting UNLESS the worker process has exited
|
|
||||||
time.sleep(1)
|
|
||||||
try:
|
|
||||||
conn = http.client.HTTPConnection(worker_host, worker_port, timeout=1)
|
|
||||||
conn.request("GET", "/worker/health")
|
|
||||||
res = conn.getresponse()
|
|
||||||
if res.status == 200:
|
|
||||||
break
|
|
||||||
conn.close()
|
|
||||||
except (socket.gaierror, http.client.HTTPException, ConnectionRefusedError, TimeoutError):
|
|
||||||
pass # Handle expected exceptions gracefully
|
|
||||||
console.print("Waiting for worker to start...", style="bold yellow")
|
|
||||||
|
|
||||||
time.sleep(1) # Wait just a little longer for everything to settle (discovered experimentally)
|
|
||||||
console.print("Worker is healthy", style="bold green")
|
|
||||||
|
|
||||||
|
|
||||||
def _stream_output(process: subprocess.Popen, name: str) -> None:
|
|
||||||
"""
|
|
||||||
Streams the output from a subprocess to the console.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
process: The subprocess.Popen object.
|
|
||||||
name: Name of the process.
|
|
||||||
"""
|
|
||||||
stdout_style = "green" if name == "Worker" else "#87CEFA"
|
|
||||||
|
|
||||||
def stream(pipe: io.TextIOWrapper | None, style: str) -> None:
|
|
||||||
if pipe is None:
|
|
||||||
return
|
|
||||||
with pipe:
|
|
||||||
for line in iter(pipe.readline, ""):
|
|
||||||
line = line.rstrip()
|
|
||||||
|
|
||||||
if "DEBUG" in line:
|
|
||||||
line = line.replace("DEBUG", "[#87CEFA]DEBUG[/#87CEFA]", 1)
|
|
||||||
if "INFO" in line:
|
|
||||||
line = line.replace("INFO", "[#109a10]INFO[/#109a10]", 1)
|
|
||||||
if "WARNING" in line:
|
|
||||||
line = line.replace("WARNING", "[#FFA500]WARNING[/#FFA500]", 1)
|
|
||||||
if "ERROR" in line:
|
|
||||||
line = line.replace("ERROR", "[#FF0000]ERROR[/#FF0000]", 1)
|
|
||||||
console.print(f"[{style}]{name}>[/{style}] {line}")
|
|
||||||
|
|
||||||
threading.Thread(target=stream, args=(process.stdout, stdout_style), daemon=True).start()
|
|
||||||
threading.Thread(target=stream, args=(process.stderr, "red"), daemon=True).start()
|
|
||||||
|
|
||||||
|
|
||||||
def _monitor_processes(worker_process: subprocess.Popen, engine_process: subprocess.Popen) -> None:
|
|
||||||
"""
|
|
||||||
Monitors the worker and engine processes, restarts them if they exit.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
worker_process: The worker subprocess.
|
|
||||||
engine_process: The engine subprocess.
|
|
||||||
"""
|
|
||||||
|
|
||||||
while True:
|
|
||||||
worker_status = worker_process.poll()
|
|
||||||
engine_status = engine_process.poll()
|
|
||||||
|
|
||||||
if worker_status is not None or engine_status is not None:
|
|
||||||
if worker_status is not None:
|
|
||||||
console.print(
|
|
||||||
f"Worker process exited with code {worker_status}. Restarting both processes...",
|
|
||||||
style="bold red",
|
|
||||||
)
|
|
||||||
if engine_status is not None:
|
|
||||||
console.print(
|
|
||||||
f"Engine process exited with code {engine_status}. Restarting both processes...",
|
|
||||||
style="bold red",
|
|
||||||
)
|
|
||||||
_terminate_process(worker_process)
|
|
||||||
_terminate_process(engine_process)
|
|
||||||
time.sleep(1)
|
|
||||||
break # Exit to restart both processes
|
|
||||||
else:
|
|
||||||
time.sleep(1)
|
|
||||||
|
|
||||||
|
|
||||||
def _terminate_process(process: subprocess.Popen | None) -> None:
|
|
||||||
"""
|
|
||||||
Terminates a subprocess if it's running.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
process: The subprocess.Popen object.
|
|
||||||
"""
|
|
||||||
if process and process.poll() is None:
|
|
||||||
process.terminate()
|
|
||||||
try:
|
|
||||||
process.wait(timeout=5)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
process.kill()
|
|
||||||
|
|
||||||
|
|
||||||
def _setup_signal_handlers(terminate_processes: Callable[[bool], None]) -> None:
|
|
||||||
"""
|
|
||||||
Setup signal handlers to handle process termination signals.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
terminate_processes: Function to call to terminate child processes.
|
|
||||||
"""
|
|
||||||
signals_to_handle = ["SIGINT", "SIGTERM", "SIGQUIT", "SIGHUP"]
|
|
||||||
|
|
||||||
for sig_name in signals_to_handle:
|
|
||||||
sig = getattr(signal, sig_name, None)
|
|
||||||
if sig is None:
|
|
||||||
continue # Signal not available on this platform
|
|
||||||
try:
|
|
||||||
# Use a lambda to pass the terminate_processes function
|
|
||||||
signal.signal(
|
|
||||||
sig,
|
|
||||||
lambda signum, frame: _handle_signal(signum, terminate_processes),
|
|
||||||
)
|
|
||||||
except (ValueError, RuntimeError):
|
|
||||||
# Signal handling not allowed in this thread or invalid signal
|
|
||||||
console.print(f"Warning: Cannot set handler for {sig_name}", style="bold yellow")
|
|
||||||
continue
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_signal(signum: int, terminate_processes: Callable[[bool], None]) -> None:
|
|
||||||
"""
|
|
||||||
Handle received signal and terminate child processes.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
signum: The signal number received.
|
|
||||||
terminate_processes: Function to call to terminate child processes.
|
|
||||||
"""
|
|
||||||
signal_name = signal.Signals(signum).name
|
|
||||||
console.print(f"Received {signal_name}. Shutting down...", style="bold yellow")
|
|
||||||
terminate_processes(exit_program=True) # type: ignore[call-arg]
|
|
||||||
|
|
@ -1,143 +0,0 @@
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
from datetime import datetime
|
|
||||||
from importlib.metadata import version as get_version
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
import typer
|
|
||||||
from jinja2 import Environment, FileSystemLoader, select_autoescape
|
|
||||||
from rich.console import Console
|
|
||||||
|
|
||||||
from arcade.cli.deployment import (
|
|
||||||
create_demo_deployment,
|
|
||||||
)
|
|
||||||
|
|
||||||
console = Console()
|
|
||||||
|
|
||||||
# Retrieve the installed version of arcade-ai
|
|
||||||
try:
|
|
||||||
ARCADE_VERSION = get_version("arcade-ai")
|
|
||||||
except Exception as e:
|
|
||||||
console.print(f"[red]Failed to get arcade-ai version: {e}[/red]")
|
|
||||||
ARCADE_VERSION = "0.0.0" # Default version if unable to fetch
|
|
||||||
|
|
||||||
TEMPLATE_IGNORE_PATTERN = re.compile(
|
|
||||||
r"(__pycache__|\.DS_Store|Thumbs\.db|\.git|\.svn|\.hg|\.vscode|\.idea|build|dist|.*\.egg-info|.*\.pyc|.*\.pyo)$"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def ask_question(question: str, default: Optional[str] = None) -> str:
|
|
||||||
"""
|
|
||||||
Ask a question via input() and return the answer.
|
|
||||||
"""
|
|
||||||
answer = typer.prompt(question, default=default)
|
|
||||||
if not answer and default:
|
|
||||||
return default
|
|
||||||
return str(answer)
|
|
||||||
|
|
||||||
|
|
||||||
def render_template(env: Environment, template_string: str, context: dict) -> str:
|
|
||||||
"""Render a template string with the given variables."""
|
|
||||||
template = env.from_string(template_string)
|
|
||||||
return template.render(context)
|
|
||||||
|
|
||||||
|
|
||||||
def write_template(path: Path, content: str) -> None:
|
|
||||||
"""Write content to a file."""
|
|
||||||
path.write_text(content, encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def create_package(env: Environment, template_path: Path, output_path: Path, context: dict) -> None:
|
|
||||||
"""Recursively create a new toolkit directory structure from jinja2 templates."""
|
|
||||||
if TEMPLATE_IGNORE_PATTERN.match(template_path.name):
|
|
||||||
return
|
|
||||||
|
|
||||||
try:
|
|
||||||
if template_path.is_dir():
|
|
||||||
folder_name = render_template(env, template_path.name, context)
|
|
||||||
new_dir_path = output_path / folder_name
|
|
||||||
new_dir_path.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
for item in template_path.iterdir():
|
|
||||||
create_package(env, item, new_dir_path, context)
|
|
||||||
|
|
||||||
else:
|
|
||||||
# Render the file name
|
|
||||||
file_name = render_template(env, template_path.name, context)
|
|
||||||
with open(template_path, encoding="utf-8") as f:
|
|
||||||
content = f.read()
|
|
||||||
# Render the file content
|
|
||||||
content = render_template(env, content, context)
|
|
||||||
|
|
||||||
write_template(output_path / file_name, content)
|
|
||||||
except Exception as e:
|
|
||||||
console.print(f"[red]Failed to create package: {e}[/red]")
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def remove_toolkit(toolkit_directory: Path, toolkit_name: str) -> None:
|
|
||||||
"""Teardown logic for when creating a new toolkit fails."""
|
|
||||||
toolkit_path = toolkit_directory / toolkit_name
|
|
||||||
if toolkit_path.exists():
|
|
||||||
shutil.rmtree(toolkit_path)
|
|
||||||
|
|
||||||
|
|
||||||
def create_new_toolkit(output_directory: str) -> None:
|
|
||||||
"""Create a new toolkit from a template with user input."""
|
|
||||||
toolkit_directory = Path(output_directory)
|
|
||||||
while True:
|
|
||||||
name = ask_question("Name of the new toolkit?")
|
|
||||||
package_name = name if name.startswith("arcade_") else f"arcade_{name}"
|
|
||||||
|
|
||||||
# Check for illegal characters in the toolkit name
|
|
||||||
if re.match(r"^[\w_]+$", package_name):
|
|
||||||
toolkit_name = package_name.replace("arcade_", "", 1)
|
|
||||||
|
|
||||||
if (toolkit_directory / toolkit_name).exists():
|
|
||||||
console.print(f"[red]Toolkit {toolkit_name} already exists.[/red]")
|
|
||||||
continue
|
|
||||||
break
|
|
||||||
else:
|
|
||||||
console.print(
|
|
||||||
"[red]Toolkit name contains illegal characters. "
|
|
||||||
"Only alphanumeric characters and underscores are allowed. "
|
|
||||||
"Please try again.[/red]"
|
|
||||||
)
|
|
||||||
|
|
||||||
toolkit_description = ask_question("Description of the toolkit?")
|
|
||||||
toolkit_author_name = ask_question("Github owner username?")
|
|
||||||
toolkit_author_email = ask_question("Author's email?")
|
|
||||||
|
|
||||||
context = {
|
|
||||||
"package_name": package_name,
|
|
||||||
"toolkit_name": toolkit_name,
|
|
||||||
"toolkit_description": toolkit_description,
|
|
||||||
"toolkit_author_name": toolkit_author_name,
|
|
||||||
"toolkit_author_email": toolkit_author_email,
|
|
||||||
"arcade_version": f"^{ARCADE_VERSION}",
|
|
||||||
"creation_year": datetime.now().year,
|
|
||||||
}
|
|
||||||
template_directory = Path(__file__).parent.parent / "templates" / "{{ toolkit_name }}"
|
|
||||||
|
|
||||||
env = Environment(
|
|
||||||
loader=FileSystemLoader(str(template_directory)),
|
|
||||||
autoescape=select_autoescape(["html", "xml"]),
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
create_package(env, template_directory, toolkit_directory, context)
|
|
||||||
create_deployment(toolkit_directory, toolkit_name)
|
|
||||||
except Exception:
|
|
||||||
remove_toolkit(toolkit_directory, toolkit_name)
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def create_deployment(toolkit_directory: Path, toolkit_name: str) -> None:
|
|
||||||
worker_toml = toolkit_directory / "worker.toml"
|
|
||||||
if not worker_toml.exists():
|
|
||||||
create_demo_deployment(worker_toml, toolkit_name)
|
|
||||||
else:
|
|
||||||
pass
|
|
||||||
# Disabled pending bug fix
|
|
||||||
# update_deployment_with_local_packages(worker_toml, toolkit_name)
|
|
||||||
|
|
@ -1,14 +0,0 @@
|
||||||
from arcade.core.catalog import ToolCatalog
|
|
||||||
from arcade.core.schema import ToolAuthorizationContext, ToolContext, ToolMetadataKey
|
|
||||||
from arcade.core.toolkit import Toolkit
|
|
||||||
|
|
||||||
from .tool import tool
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"ToolAuthorizationContext",
|
|
||||||
"ToolCatalog",
|
|
||||||
"ToolContext",
|
|
||||||
"ToolMetadataKey",
|
|
||||||
"Toolkit",
|
|
||||||
"tool",
|
|
||||||
]
|
|
||||||
|
|
@ -1,3 +0,0 @@
|
||||||
from arcade.core.annotations import Inferrable
|
|
||||||
|
|
||||||
__all__ = ["Inferrable"]
|
|
||||||
|
|
@ -1,15 +0,0 @@
|
||||||
# Stop the editor from looking for .editorconfig files in the parent directories
|
|
||||||
root = true
|
|
||||||
|
|
||||||
[*]
|
|
||||||
charset = utf-8
|
|
||||||
insert_final_newline = true
|
|
||||||
end_of_line = lf
|
|
||||||
indent_style = space
|
|
||||||
indent_size = 4
|
|
||||||
max_line_length = 100 # This is also set in .ruff.toml for ruff
|
|
||||||
|
|
||||||
[*.{json,jsonc,yml,yaml}]
|
|
||||||
indent_style = space
|
|
||||||
indent_size = 2 # This is also set in .prettierrc.toml
|
|
||||||
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
name: "setup-poetry-env"{% raw %}
|
|
||||||
description: "Composite action to setup the Python and poetry environment."
|
|
||||||
|
|
||||||
inputs:
|
|
||||||
python-version:
|
|
||||||
required: false
|
|
||||||
description: "The python version to use"
|
|
||||||
default: "3.11"
|
|
||||||
|
|
||||||
runs:
|
|
||||||
using: "composite"
|
|
||||||
steps:
|
|
||||||
- name: Set up python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ inputs.python-version }}
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
uses: snok/install-poetry@v1
|
|
||||||
with:
|
|
||||||
version: 1.8.5
|
|
||||||
virtualenvs-in-project: true
|
|
||||||
|
|
||||||
- name: Generate poetry.lock
|
|
||||||
run: poetry lock --no-update
|
|
||||||
shell: bash
|
|
||||||
|
|
||||||
- name: Load cached venv
|
|
||||||
id: cached-poetry-dependencies
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: .venv
|
|
||||||
key: venv-${{ runner.os }}-${{ inputs.python-version }}-${{ hashFiles('poetry.lock') }}
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
|
|
||||||
run: poetry install --no-interaction --all-extras
|
|
||||||
shell: bash
|
|
||||||
{% endraw %}
|
|
||||||
|
|
@ -1,61 +0,0 @@
|
||||||
name: Main{% raw %}
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches:
|
|
||||||
- main
|
|
||||||
pull_request:
|
|
||||||
types: [opened, synchronize, reopened, ready_for_review]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
quality:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- name: Check out
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: ~/.cache/pre-commit
|
|
||||||
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
|
|
||||||
|
|
||||||
- name: Set up the environment
|
|
||||||
uses: ./.github/actions/setup-poetry-env
|
|
||||||
|
|
||||||
- name: Run checks
|
|
||||||
run: make check
|
|
||||||
|
|
||||||
tox:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
strategy:
|
|
||||||
matrix:
|
|
||||||
python-version: ["3.10", "3.11", "3.12"]
|
|
||||||
fail-fast: false
|
|
||||||
steps:
|
|
||||||
- name: Check out
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ matrix.python-version }}
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
uses: snok/install-poetry@v1
|
|
||||||
with:
|
|
||||||
version: 1.8.5
|
|
||||||
|
|
||||||
- name: Load cached venv
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: .tox
|
|
||||||
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('poetry.lock') }}
|
|
||||||
|
|
||||||
- name: Install tox
|
|
||||||
run: |
|
|
||||||
python -m pip install --upgrade pip
|
|
||||||
python -m pip install tox tox-gh-actions
|
|
||||||
|
|
||||||
- name: Test with tox
|
|
||||||
run: tox
|
|
||||||
{% endraw %}
|
|
||||||
|
|
@ -1,38 +0,0 @@
|
||||||
name: Publish to PyPI
|
|
||||||
|
|
||||||
on:
|
|
||||||
release:
|
|
||||||
types: [published]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
pypi-publish:
|
|
||||||
name: Publish to PyPi
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
environment:
|
|
||||||
name: pypi
|
|
||||||
url: https://pypi.org/project/{{ package_name }}
|
|
||||||
permissions:
|
|
||||||
id-token: write
|
|
||||||
steps:
|
|
||||||
- name: Check out
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: '3.12'
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
python -m pip install --upgrade pip
|
|
||||||
pip install build twine
|
|
||||||
|
|
||||||
- name: Build package
|
|
||||||
run: python -m build
|
|
||||||
|
|
||||||
- name: Publish to PyPI
|
|
||||||
env:
|
|
||||||
TWINE_USERNAME: "__token__"{% raw %}
|
|
||||||
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
|
|
||||||
run: twine upload dist/*
|
|
||||||
{% endraw %}
|
|
||||||
|
|
@ -1,167 +0,0 @@
|
||||||
.DS_Store
|
|
||||||
|
|
||||||
*.lock
|
|
||||||
|
|
||||||
# From https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore
|
|
||||||
|
|
||||||
# Byte-compiled / optimized / DLL files
|
|
||||||
__pycache__/
|
|
||||||
*.py[cod]
|
|
||||||
*$py.class
|
|
||||||
|
|
||||||
# C extensions
|
|
||||||
*.so
|
|
||||||
|
|
||||||
# Distribution / packaging
|
|
||||||
.Python
|
|
||||||
build/
|
|
||||||
develop-eggs/
|
|
||||||
dist/
|
|
||||||
downloads/
|
|
||||||
eggs/
|
|
||||||
.eggs/
|
|
||||||
lib/
|
|
||||||
lib64/
|
|
||||||
parts/
|
|
||||||
sdist/
|
|
||||||
var/
|
|
||||||
wheels/
|
|
||||||
share/python-wheels/
|
|
||||||
*.egg-info/
|
|
||||||
.installed.cfg
|
|
||||||
*.egg
|
|
||||||
MANIFEST
|
|
||||||
|
|
||||||
# PyInstaller
|
|
||||||
# Usually these files are written by a python script from a template
|
|
||||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
|
||||||
*.manifest
|
|
||||||
*.spec
|
|
||||||
|
|
||||||
# Installer logs
|
|
||||||
pip-log.txt
|
|
||||||
pip-delete-this-directory.txt
|
|
||||||
|
|
||||||
# Unit test / coverage reports
|
|
||||||
htmlcov/
|
|
||||||
.tox/
|
|
||||||
.nox/
|
|
||||||
.coverage
|
|
||||||
.coverage.*
|
|
||||||
.cache
|
|
||||||
nosetests.xml
|
|
||||||
coverage.xml
|
|
||||||
*.cover
|
|
||||||
*.py,cover
|
|
||||||
.hypothesis/
|
|
||||||
.pytest_cache/
|
|
||||||
cover/
|
|
||||||
|
|
||||||
# Translations
|
|
||||||
*.mo
|
|
||||||
*.pot
|
|
||||||
|
|
||||||
# Django stuff:
|
|
||||||
*.log
|
|
||||||
local_settings.py
|
|
||||||
db.sqlite3
|
|
||||||
db.sqlite3-journal
|
|
||||||
|
|
||||||
# Flask stuff:
|
|
||||||
instance/
|
|
||||||
.webassets-cache
|
|
||||||
|
|
||||||
# Scrapy stuff:
|
|
||||||
.scrapy
|
|
||||||
|
|
||||||
# Sphinx documentation
|
|
||||||
docs/_build/
|
|
||||||
|
|
||||||
# PyBuilder
|
|
||||||
.pybuilder/
|
|
||||||
target/
|
|
||||||
|
|
||||||
# Jupyter Notebook
|
|
||||||
.ipynb_checkpoints
|
|
||||||
|
|
||||||
# IPython
|
|
||||||
profile_default/
|
|
||||||
ipython_config.py
|
|
||||||
|
|
||||||
# pyenv
|
|
||||||
# For a library or package, you might want to ignore these files since the code is
|
|
||||||
# intended to run in multiple environments; otherwise, check them in:
|
|
||||||
# .python-version
|
|
||||||
|
|
||||||
# pipenv
|
|
||||||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
|
||||||
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
|
||||||
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
|
||||||
# install all needed dependencies.
|
|
||||||
#Pipfile.lock
|
|
||||||
|
|
||||||
# poetry
|
|
||||||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
|
||||||
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
|
||||||
# commonly ignored for libraries.
|
|
||||||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
|
||||||
poetry.lock
|
|
||||||
|
|
||||||
# pdm
|
|
||||||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
|
||||||
#pdm.lock
|
|
||||||
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
|
|
||||||
# in version control.
|
|
||||||
# https://pdm.fming.dev/#use-with-ide
|
|
||||||
.pdm.toml
|
|
||||||
|
|
||||||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
|
||||||
__pypackages__/
|
|
||||||
|
|
||||||
# Celery stuff
|
|
||||||
celerybeat-schedule
|
|
||||||
celerybeat.pid
|
|
||||||
|
|
||||||
# SageMath parsed files
|
|
||||||
*.sage.py
|
|
||||||
|
|
||||||
# Environments
|
|
||||||
.env
|
|
||||||
.venv
|
|
||||||
env/
|
|
||||||
venv/
|
|
||||||
ENV/
|
|
||||||
env.bak/
|
|
||||||
venv.bak/
|
|
||||||
|
|
||||||
# Spyder project settings
|
|
||||||
.spyderproject
|
|
||||||
.spyproject
|
|
||||||
|
|
||||||
# Rope project settings
|
|
||||||
.ropeproject
|
|
||||||
|
|
||||||
# mkdocs documentation
|
|
||||||
/site
|
|
||||||
|
|
||||||
# mypy
|
|
||||||
.mypy_cache/
|
|
||||||
.dmypy.json
|
|
||||||
dmypy.json
|
|
||||||
|
|
||||||
# Pyre type checker
|
|
||||||
.pyre/
|
|
||||||
|
|
||||||
# pytype static type analyzer
|
|
||||||
.pytype/
|
|
||||||
|
|
||||||
# Cython debug symbols
|
|
||||||
cython_debug/
|
|
||||||
|
|
||||||
# PyCharm
|
|
||||||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
|
||||||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
|
||||||
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
|
||||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
|
||||||
#.idea/
|
|
||||||
|
|
||||||
|
|
@ -1,19 +0,0 @@
|
||||||
files: ^./
|
|
||||||
repos:
|
|
||||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
|
||||||
rev: "v4.4.0"
|
|
||||||
hooks:
|
|
||||||
- id: check-case-conflict
|
|
||||||
- id: check-merge-conflict
|
|
||||||
- id: check-toml
|
|
||||||
- id: check-yaml
|
|
||||||
- id: end-of-file-fixer
|
|
||||||
- id: trailing-whitespace
|
|
||||||
|
|
||||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
|
||||||
rev: v0.6.7
|
|
||||||
hooks:
|
|
||||||
- id: ruff
|
|
||||||
args: [--fix]
|
|
||||||
- id: ruff-format
|
|
||||||
|
|
||||||
|
|
@ -1,3 +0,0 @@
|
||||||
# Ignore Python files for Prettier
|
|
||||||
*.py
|
|
||||||
|
|
||||||
|
|
@ -1,16 +0,0 @@
|
||||||
# See https://prettier.io/docs/en/configuration
|
|
||||||
|
|
||||||
# Note: This prettier config is only for the non-python files in this repo.
|
|
||||||
# Python files are formatted with ruff and ignored in .prettierignore
|
|
||||||
|
|
||||||
trailingComma = "es5"
|
|
||||||
tabWidth = 4
|
|
||||||
semi = false
|
|
||||||
singleQuote = false
|
|
||||||
|
|
||||||
[[overrides]]
|
|
||||||
files = [ "*.json", "*.jsonc", "*.yml", "*.yaml" ]
|
|
||||||
|
|
||||||
[overrides.options]
|
|
||||||
tabWidth = 2
|
|
||||||
|
|
||||||
|
|
@ -1,45 +0,0 @@
|
||||||
target-version = "py310"
|
|
||||||
line-length = 100
|
|
||||||
fix = true
|
|
||||||
|
|
||||||
[lint]
|
|
||||||
select = [
|
|
||||||
# flake8-2020
|
|
||||||
"YTT",
|
|
||||||
# flake8-bandit
|
|
||||||
"S",
|
|
||||||
# flake8-bugbear
|
|
||||||
"B",
|
|
||||||
# flake8-builtins
|
|
||||||
"A",
|
|
||||||
# flake8-comprehensions
|
|
||||||
"C4",
|
|
||||||
# flake8-debugger
|
|
||||||
"T10",
|
|
||||||
# flake8-simplify
|
|
||||||
"SIM",
|
|
||||||
# isort
|
|
||||||
"I",
|
|
||||||
# mccabe
|
|
||||||
"C90",
|
|
||||||
# pycodestyle
|
|
||||||
"E", "W",
|
|
||||||
# pyflakes
|
|
||||||
"F",
|
|
||||||
# pygrep-hooks
|
|
||||||
"PGH",
|
|
||||||
# pyupgrade
|
|
||||||
"UP",
|
|
||||||
# ruff
|
|
||||||
"RUF",
|
|
||||||
# tryceratops
|
|
||||||
"TRY",
|
|
||||||
]
|
|
||||||
|
|
||||||
[lint.per-file-ignores]
|
|
||||||
"**/tests/*" = ["S101"]
|
|
||||||
|
|
||||||
[format]
|
|
||||||
preview = true
|
|
||||||
skip-magic-trailing-comma = false
|
|
||||||
|
|
||||||
|
|
@ -1,22 +0,0 @@
|
||||||
MIT License
|
|
||||||
|
|
||||||
Copyright (c) {{ creation_year }}, {{ toolkit_author_name }}
|
|
||||||
|
|
||||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
||||||
of this software and associated documentation files (the "Software"), to deal
|
|
||||||
in the Software without restriction, including without limitation the rights
|
|
||||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
||||||
copies of the Software, and to permit persons to whom the Software is
|
|
||||||
furnished to do so, subject to the following conditions:
|
|
||||||
|
|
||||||
The above copyright notice and this permission notice shall be included in all
|
|
||||||
copies or substantial portions of the Software.
|
|
||||||
|
|
||||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
||||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
||||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
||||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
||||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
||||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
||||||
SOFTWARE.
|
|
||||||
|
|
||||||
|
|
@ -1,59 +0,0 @@
|
||||||
.PHONY: help
|
|
||||||
|
|
||||||
help:
|
|
||||||
@echo "🛠️ {{ toolkit_name }} Commands:\n"
|
|
||||||
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
|
|
||||||
|
|
||||||
.PHONY: install
|
|
||||||
install: ## Install the poetry environment and install the pre-commit hooks
|
|
||||||
@echo "📦 Checking if Poetry is installed"
|
|
||||||
@if ! command -v poetry >/dev/null 2>&1; then \
|
|
||||||
echo "📦 Poetry not found. Checking if pip is available"; \
|
|
||||||
if ! command -v pip >/dev/null 2>&1; then \
|
|
||||||
echo "❌ pip is not installed. Please install pip first."; \
|
|
||||||
exit 1; \
|
|
||||||
fi; \
|
|
||||||
echo "📦 Installing Poetry with pip"; \
|
|
||||||
pip install poetry==1.8.5; \
|
|
||||||
else \
|
|
||||||
echo "📦 Poetry is already installed"; \
|
|
||||||
fi
|
|
||||||
@echo "🚀 Installing package in development mode with all extras"
|
|
||||||
poetry install --all-extras
|
|
||||||
|
|
||||||
.PHONY: build
|
|
||||||
build: clean-build ## Build wheel file using poetry
|
|
||||||
@echo "🚀 Creating wheel file"
|
|
||||||
poetry build
|
|
||||||
|
|
||||||
.PHONY: clean-build
|
|
||||||
clean-build: ## clean build artifacts
|
|
||||||
@echo "🗑️ Cleaning dist directory"
|
|
||||||
rm -rf dist
|
|
||||||
|
|
||||||
.PHONY: test
|
|
||||||
test: ## Test the code with pytest
|
|
||||||
@echo "🚀 Testing code: Running pytest"
|
|
||||||
@poetry run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
|
|
||||||
|
|
||||||
.PHONY: coverage
|
|
||||||
coverage: ## Generate coverage report
|
|
||||||
@echo "coverage report"
|
|
||||||
coverage report
|
|
||||||
@echo "Generating coverage report"
|
|
||||||
coverage html
|
|
||||||
|
|
||||||
.PHONY: bump-version
|
|
||||||
bump-version: ## Bump the version in the pyproject.toml file
|
|
||||||
@echo "🚀 Bumping version in pyproject.toml"
|
|
||||||
poetry version patch
|
|
||||||
|
|
||||||
.PHONY: check
|
|
||||||
check: ## Run code quality tools.
|
|
||||||
@echo "🚀 Checking Poetry lock file consistency with 'pyproject.toml': Running poetry check"
|
|
||||||
@poetry check
|
|
||||||
@echo "🚀 Linting code: Running pre-commit"
|
|
||||||
@poetry run pre-commit run -a
|
|
||||||
@echo "🚀 Static type checking: Running mypy"
|
|
||||||
@poetry run mypy --config-file=pyproject.toml
|
|
||||||
|
|
||||||
|
|
@ -1,10 +0,0 @@
|
||||||
coverage:
|
|
||||||
range: 70..100
|
|
||||||
round: down
|
|
||||||
precision: 1
|
|
||||||
status:
|
|
||||||
project:
|
|
||||||
default:
|
|
||||||
target: 90%
|
|
||||||
threshold: 0.5%
|
|
||||||
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
[tool.poetry]
|
|
||||||
name = "{{ package_name }}"
|
|
||||||
version = "0.0.1"
|
|
||||||
description = "{{ toolkit_description }}"
|
|
||||||
authors = ["{{ toolkit_author_name }} <{{ toolkit_author_email }}>"]
|
|
||||||
|
|
||||||
[tool.poetry.dependencies]
|
|
||||||
python = "^3.10"
|
|
||||||
arcade-ai = "^1.0.5"
|
|
||||||
|
|
||||||
[tool.poetry.dev-dependencies]
|
|
||||||
pytest = "^8.3.0"
|
|
||||||
pytest-cov = "^4.0.0"
|
|
||||||
mypy = "^1.5.1"
|
|
||||||
pre-commit = "^3.4.0"
|
|
||||||
tox = "^4.11.1"
|
|
||||||
ruff = "^0.7.4"
|
|
||||||
|
|
||||||
[build-system]
|
|
||||||
requires = ["poetry-core>=1.0.0,<2.0.0"]
|
|
||||||
build-backend = "poetry.core.masonry.api"
|
|
||||||
|
|
||||||
[tool.mypy]
|
|
||||||
files = ["{{ package_name }}/**/*.py"]
|
|
||||||
python_version = "3.10"
|
|
||||||
disallow_untyped_defs = "True"
|
|
||||||
disallow_any_unimported = "True"
|
|
||||||
no_implicit_optional = "True"
|
|
||||||
check_untyped_defs = "True"
|
|
||||||
warn_return_any = "True"
|
|
||||||
warn_unused_ignores = "True"
|
|
||||||
show_error_codes = "True"
|
|
||||||
ignore_missing_imports = "True"
|
|
||||||
|
|
||||||
[tool.pytest.ini_options]
|
|
||||||
testpaths = ["tests"]
|
|
||||||
|
|
||||||
[tool.coverage.report]
|
|
||||||
skip_empty = true
|
|
||||||
|
|
||||||
|
|
@ -1,17 +0,0 @@
|
||||||
[tox]
|
|
||||||
skipsdist = true
|
|
||||||
envlist = py310, py311, py312
|
|
||||||
|
|
||||||
[gh-actions]
|
|
||||||
python =
|
|
||||||
3.10: py310
|
|
||||||
3.11: py311
|
|
||||||
3.12: py312
|
|
||||||
|
|
||||||
[testenv]
|
|
||||||
passenv = PYTHON_VERSION
|
|
||||||
allowlist_externals = poetry
|
|
||||||
commands =
|
|
||||||
poetry install -v --all-extras
|
|
||||||
pytest --doctest-modules tests --cov --cov-config=pyproject.toml --cov-report=xml
|
|
||||||
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
coverage:
|
|
||||||
range: 70..100
|
|
||||||
round: down
|
|
||||||
precision: 1
|
|
||||||
status:
|
|
||||||
project:
|
|
||||||
default:
|
|
||||||
target: 90%
|
|
||||||
threshold: 0.5%
|
|
||||||
exclude:
|
|
||||||
- arcade/cli/**
|
|
||||||
|
|
@ -1,88 +0,0 @@
|
||||||
[tool.poetry]
|
|
||||||
name = "arcade-ai"
|
|
||||||
version = "1.1.0"
|
|
||||||
description = "Arcade Python SDK and CLI"
|
|
||||||
readme = "README.md"
|
|
||||||
packages = [
|
|
||||||
{include="arcade", from="."}
|
|
||||||
]
|
|
||||||
authors = ["Arcade <dev@arcade.dev>"]
|
|
||||||
|
|
||||||
[build-system]
|
|
||||||
requires = ["poetry-core>=1.0.0"]
|
|
||||||
build-backend = "poetry.core.masonry.api"
|
|
||||||
|
|
||||||
|
|
||||||
[tool.poetry.dependencies]
|
|
||||||
python = ">=3.10,<4.0"
|
|
||||||
pydantic = "^2.7.0"
|
|
||||||
typer = ">=0.10.0"
|
|
||||||
rich = "^13.7.1"
|
|
||||||
Jinja2 = ">=3.1.5,<4.0.0"
|
|
||||||
pyyaml = "^6.0"
|
|
||||||
openai = "^1.36.0" # TODO: relax to an earlier version that still has what we need
|
|
||||||
arcadepy = "^1.3.1"
|
|
||||||
pyjwt = "^2.8.0"
|
|
||||||
loguru = "^0.7.0"
|
|
||||||
tqdm = "^4.1.0"
|
|
||||||
toml = "^0.10.2"
|
|
||||||
packaging = "^24.1"
|
|
||||||
types-python-dateutil = "2.9.0.20241003"
|
|
||||||
types-pytz = "2024.2.0.20241003"
|
|
||||||
types-toml = "0.10.8.20240310"
|
|
||||||
opentelemetry-instrumentation-fastapi = "0.48b0"
|
|
||||||
opentelemetry-exporter-otlp-proto-http = "1.27.0"
|
|
||||||
opentelemetry-exporter-otlp-proto-common = "1.27.0"
|
|
||||||
fastapi = "^0.115.3"
|
|
||||||
uvicorn = "^0.30.0"
|
|
||||||
scipy = {version = "^1.14.0", optional = true}
|
|
||||||
numpy = {version = "^2.0.0", optional = true}
|
|
||||||
scikit-learn = {version = "^1.5.0", optional = true}
|
|
||||||
pytz = {version = "^2024.1", optional = true}
|
|
||||||
python-dateutil = {version = "^2.8.2", optional = true}
|
|
||||||
watchfiles = "^1.0.5"
|
|
||||||
|
|
||||||
pyreadline3 = {version = "^3.5.4", platform = "win32"}
|
|
||||||
[tool.poetry.extras]
|
|
||||||
evals = ["scipy", "numpy", "scikit-learn", "pytz", "python-dateutil"]
|
|
||||||
|
|
||||||
|
|
||||||
[tool.poetry.group.dev.dependencies]
|
|
||||||
pytest = "^8.1.2"
|
|
||||||
pytest-cov = "^4.0.0"
|
|
||||||
mypy = "^1.5.1"
|
|
||||||
pre-commit = "^3.4.0"
|
|
||||||
tox = "^4.11.1"
|
|
||||||
pytest-asyncio = "^0.23.7"
|
|
||||||
types-pytz = "^2024.1"
|
|
||||||
types-python-dateutil = "^2.8.2"
|
|
||||||
types-PyYAML = "^6.0.0"
|
|
||||||
poetry-plugin-export = "^1.7.0"
|
|
||||||
|
|
||||||
[tool.poetry.scripts]
|
|
||||||
arcade = "arcade.cli.main:cli"
|
|
||||||
|
|
||||||
[tool.mypy]
|
|
||||||
files = ["arcade"]
|
|
||||||
exclude = "arcade/templates"
|
|
||||||
python_version = "3.10"
|
|
||||||
disallow_untyped_defs = "True"
|
|
||||||
disallow_any_unimported = "True"
|
|
||||||
no_implicit_optional = "True"
|
|
||||||
check_untyped_defs = "True"
|
|
||||||
warn_return_any = "True"
|
|
||||||
warn_unused_ignores = "True"
|
|
||||||
show_error_codes = "True"
|
|
||||||
ignore_missing_imports = "True"
|
|
||||||
|
|
||||||
[tool.pytest.ini_options]
|
|
||||||
testpaths = ["tests"]
|
|
||||||
|
|
||||||
|
|
||||||
[tool.coverage.run]
|
|
||||||
branch = true
|
|
||||||
source = ["arcade"]
|
|
||||||
omit = ["arcade/cli/*"]
|
|
||||||
|
|
||||||
[tool.coverage.report]
|
|
||||||
skip_empty = true
|
|
||||||
|
|
@ -1,17 +0,0 @@
|
||||||
[tox]
|
|
||||||
skipsdist = true
|
|
||||||
envlist = py310, py311, py312
|
|
||||||
|
|
||||||
[gh-actions]
|
|
||||||
python =
|
|
||||||
3.10: py310
|
|
||||||
3.11: py311
|
|
||||||
3.12: py312
|
|
||||||
|
|
||||||
[testenv]
|
|
||||||
passenv = PYTHON_VERSION
|
|
||||||
allowlist_externals = poetry
|
|
||||||
commands =
|
|
||||||
poetry install -v --all-extras
|
|
||||||
pytest --doctest-modules tests --cov --cov-config=pyproject.toml --cov-report=xml
|
|
||||||
# mypy
|
|
||||||
|
|
@ -1,8 +1,7 @@
|
||||||
import arcade_google # pip install arcade_google
|
import arcade_google # pip install arcade_google
|
||||||
import arcade_search # pip install arcade_search
|
import arcade_search # pip install arcade_search
|
||||||
|
from arcade_core.catalog import ToolCatalog
|
||||||
from arcade.core.catalog import ToolCatalog
|
from arcade_serve.mcp.stdio import StdioServer
|
||||||
from arcade.worker.mcp.stdio import StdioServer
|
|
||||||
|
|
||||||
# 2. Create and populate the tool catalog
|
# 2. Create and populate the tool catalog
|
||||||
catalog = ToolCatalog()
|
catalog = ToolCatalog()
|
||||||
|
|
|
||||||
|
|
@ -7,17 +7,18 @@ app = App("arcade-worker")
|
||||||
|
|
||||||
toolkits = ["arcade_google", "arcade_slack"]
|
toolkits = ["arcade_google", "arcade_slack"]
|
||||||
|
|
||||||
image = Image.debian_slim().pip_install("arcade-ai").pip_install(toolkits)
|
image = (
|
||||||
|
Image.debian_slim().pip_install("arcade_tdk").pip_install("arcade_serve").pip_install(toolkits)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@app.function(image=image)
|
@app.function(image=image)
|
||||||
@asgi_app()
|
@asgi_app()
|
||||||
def fastapi_app():
|
def fastapi_app():
|
||||||
|
from arcade_serve.fastapi.worker import FastAPIWorker
|
||||||
|
from arcade_tdk import Toolkit
|
||||||
from fastapi import FastAPI
|
from fastapi import FastAPI
|
||||||
|
|
||||||
from arcade.sdk import Toolkit
|
|
||||||
from arcade.worker.fastapi.worker import FastAPIWorker
|
|
||||||
|
|
||||||
web_app = FastAPI()
|
web_app = FastAPI()
|
||||||
|
|
||||||
# Initialize app and Arcade FastAPIWorker
|
# Initialize app and Arcade FastAPIWorker
|
||||||
|
|
|
||||||
28
libs/arcade-cli/README.md
Normal file
28
libs/arcade-cli/README.md
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
# Arcade CLI
|
||||||
|
|
||||||
|
Command-line interface for the Arcade platform.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Arcade CLI provides a comprehensive command-line interface for the Arcade platform:
|
||||||
|
|
||||||
|
- **User Authentication**: Login, logout
|
||||||
|
- **Tool Development**: Create, test, and manage Arcade tools
|
||||||
|
- **Worker Deployment**: Deploy and manage Arcade workers
|
||||||
|
- **Interactive Chat**: Test tools in an interactive environment
|
||||||
|
- **Project Templates**: Generate new toolkit projects
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
```bash
|
||||||
|
pip install arcade-ai
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
Learn how to use the Arcade CLI and what commands are available to you.
|
||||||
|
```bash
|
||||||
|
arcade --help
|
||||||
|
```
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT License - see LICENSE file for details.
|
||||||
|
|
@ -7,7 +7,7 @@ from urllib.parse import parse_qs
|
||||||
import yaml
|
import yaml
|
||||||
from rich.console import Console
|
from rich.console import Console
|
||||||
|
|
||||||
from arcade.cli.constants import (
|
from arcade_cli.constants import (
|
||||||
ARCADE_CONFIG_PATH,
|
ARCADE_CONFIG_PATH,
|
||||||
CREDENTIALS_FILE_PATH,
|
CREDENTIALS_FILE_PATH,
|
||||||
LOGIN_FAILED_HTML,
|
LOGIN_FAILED_HTML,
|
||||||
104
libs/arcade-cli/arcade_cli/config.py
Normal file
104
libs/arcade-cli/arcade_cli/config.py
Normal file
|
|
@ -0,0 +1,104 @@
|
||||||
|
"""
|
||||||
|
Configuration utilities for the Arcade CLI.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from rich.console import Console
|
||||||
|
from rich.table import Table
|
||||||
|
|
||||||
|
console = Console()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class UserConfig:
|
||||||
|
"""User configuration."""
|
||||||
|
|
||||||
|
email: str | None = None
|
||||||
|
name: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ApiConfig:
|
||||||
|
"""API configuration."""
|
||||||
|
|
||||||
|
key: str | None = None
|
||||||
|
url: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Config:
|
||||||
|
"""Arcade CLI configuration."""
|
||||||
|
|
||||||
|
user: UserConfig | None = None
|
||||||
|
api: ApiConfig | None = None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, data: dict[str, Any]) -> "Config":
|
||||||
|
"""Create a Config instance from a dictionary.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: Dictionary with configuration
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Config instance
|
||||||
|
"""
|
||||||
|
user_data = data.get("user", {})
|
||||||
|
api_data = data.get("api", {})
|
||||||
|
|
||||||
|
return cls(
|
||||||
|
user=UserConfig(
|
||||||
|
email=user_data.get("email"),
|
||||||
|
name=user_data.get("name"),
|
||||||
|
),
|
||||||
|
api=ApiConfig(
|
||||||
|
key=api_data.get("key"),
|
||||||
|
url=api_data.get("url"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
"""Convert configuration to a dictionary.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Configuration as a dictionary
|
||||||
|
"""
|
||||||
|
result = {}
|
||||||
|
|
||||||
|
if self.user:
|
||||||
|
result["user"] = {
|
||||||
|
"email": self.user.email,
|
||||||
|
"name": self.user.name,
|
||||||
|
}
|
||||||
|
|
||||||
|
if self.api:
|
||||||
|
result["api"] = {
|
||||||
|
"key": self.api.key,
|
||||||
|
"url": self.api.url,
|
||||||
|
}
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def print_config(config: dict[str, Any], name: str | None = None) -> None:
|
||||||
|
"""
|
||||||
|
Print the configuration in a formatted table.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
config: Configuration dictionary
|
||||||
|
name: Optional name for the configuration
|
||||||
|
"""
|
||||||
|
table = Table(title=f"Configuration: {name}" if name else "Configuration")
|
||||||
|
table.add_column("Key", style="cyan")
|
||||||
|
table.add_column("Value", style="green")
|
||||||
|
|
||||||
|
for key, value in sorted(config.items()):
|
||||||
|
if isinstance(value, dict):
|
||||||
|
# For nested configurations
|
||||||
|
nested_value = "\n".join(f"{k}: {v}" for k, v in value.items())
|
||||||
|
table.add_row(key, nested_value)
|
||||||
|
else:
|
||||||
|
table.add_row(key, str(value))
|
||||||
|
|
||||||
|
console.print(table)
|
||||||
|
|
@ -1,14 +1,13 @@
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from arcade_core.schema import ToolDefinition
|
||||||
from rich.console import Console
|
from rich.console import Console
|
||||||
from rich.panel import Panel
|
from rich.panel import Panel
|
||||||
from rich.table import Table
|
from rich.table import Table
|
||||||
from rich.text import Text
|
from rich.text import Text
|
||||||
|
|
||||||
from arcade.core.schema import ToolDefinition
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from arcade.sdk.eval.eval import EvaluationResult
|
from arcade_evals.eval import EvaluationResult
|
||||||
console = Console()
|
console = Console()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -16,23 +16,22 @@ from rich.markup import escape
|
||||||
from rich.text import Text
|
from rich.text import Text
|
||||||
from tqdm import tqdm
|
from tqdm import tqdm
|
||||||
|
|
||||||
import arcade.cli.worker as worker
|
import arcade_cli.worker as worker
|
||||||
from arcade.cli.authn import LocalAuthCallbackServer, check_existing_login
|
from arcade_cli.authn import LocalAuthCallbackServer, check_existing_login
|
||||||
from arcade.cli.constants import (
|
from arcade_cli.constants import (
|
||||||
CREDENTIALS_FILE_PATH,
|
CREDENTIALS_FILE_PATH,
|
||||||
LOCALHOST,
|
LOCALHOST,
|
||||||
PROD_CLOUD_HOST,
|
PROD_CLOUD_HOST,
|
||||||
PROD_ENGINE_HOST,
|
PROD_ENGINE_HOST,
|
||||||
)
|
)
|
||||||
from arcade.cli.deployment import Deployment
|
from arcade_cli.deployment import Deployment
|
||||||
from arcade.cli.display import (
|
from arcade_cli.display import (
|
||||||
display_arcade_chat_header,
|
display_arcade_chat_header,
|
||||||
display_eval_results,
|
display_eval_results,
|
||||||
display_tool_messages,
|
display_tool_messages,
|
||||||
)
|
)
|
||||||
from arcade.cli.launcher import start_servers
|
from arcade_cli.show import show_logic
|
||||||
from arcade.cli.show import show_logic
|
from arcade_cli.utils import (
|
||||||
from arcade.cli.utils import (
|
|
||||||
OrderCommands,
|
OrderCommands,
|
||||||
compute_base_url,
|
compute_base_url,
|
||||||
compute_login_url,
|
compute_login_url,
|
||||||
|
|
@ -45,6 +44,7 @@ from arcade.cli.utils import (
|
||||||
is_authorization_pending,
|
is_authorization_pending,
|
||||||
load_eval_suites,
|
load_eval_suites,
|
||||||
log_engine_health,
|
log_engine_health,
|
||||||
|
require_dependency,
|
||||||
validate_and_get_config,
|
validate_and_get_config,
|
||||||
version_callback,
|
version_callback,
|
||||||
)
|
)
|
||||||
|
|
@ -133,17 +133,24 @@ def logout() -> None:
|
||||||
console.print("You're not logged in.", style="bold red")
|
console.print("You're not logged in.", style="bold red")
|
||||||
|
|
||||||
|
|
||||||
@cli.command(help="Create a new toolkit package directory", rich_help_panel="Tool Development")
|
@cli.command(
|
||||||
|
help="Create a new toolkit package directory. Example usage: arcade new my_toolkit",
|
||||||
|
rich_help_panel="Tool Development",
|
||||||
|
)
|
||||||
def new(
|
def new(
|
||||||
|
toolkit_name: str = typer.Argument(
|
||||||
|
help="The name of the toolkit to create",
|
||||||
|
metavar="TOOLKIT_NAME",
|
||||||
|
),
|
||||||
directory: str = typer.Option(os.getcwd(), "--dir", help="tools directory path"),
|
directory: str = typer.Option(os.getcwd(), "--dir", help="tools directory path"),
|
||||||
) -> None:
|
) -> None:
|
||||||
"""
|
"""
|
||||||
Creates a new toolkit with the given name, description, and result type.
|
Creates a new toolkit with the given name, description, and result type.
|
||||||
"""
|
"""
|
||||||
from arcade.cli.new import create_new_toolkit
|
from arcade_cli.new import create_new_toolkit
|
||||||
|
|
||||||
try:
|
try:
|
||||||
create_new_toolkit(directory)
|
create_new_toolkit(directory, toolkit_name)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
error_message = f"❌ Failed to create new Toolkit: {escape(str(e))}"
|
error_message = f"❌ Failed to create new Toolkit: {escape(str(e))}"
|
||||||
console.print(error_message, style="bold red")
|
console.print(error_message, style="bold red")
|
||||||
|
|
@ -374,6 +381,20 @@ def evals(
|
||||||
Find all files starting with 'eval_' in the given directory,
|
Find all files starting with 'eval_' in the given directory,
|
||||||
execute any functions decorated with @tool_eval, and display the results.
|
execute any functions decorated with @tool_eval, and display the results.
|
||||||
"""
|
"""
|
||||||
|
require_dependency(
|
||||||
|
package_name="arcade_evals",
|
||||||
|
command_name="evals",
|
||||||
|
install_command=r"pip install 'arcade-ai\[evals]'",
|
||||||
|
)
|
||||||
|
# Although Evals does not depend on the TDK, some evaluations import the
|
||||||
|
# ToolCatalog class from the TDK instead of from arcade_core, so we require
|
||||||
|
# the TDK to run the evals CLI command to avoid possible import errors.
|
||||||
|
require_dependency(
|
||||||
|
package_name="arcade_tdk",
|
||||||
|
command_name="evals",
|
||||||
|
install_command=r"pip install arcade-tdk",
|
||||||
|
)
|
||||||
|
|
||||||
config = validate_and_get_config()
|
config = validate_and_get_config()
|
||||||
|
|
||||||
host = PROD_ENGINE_HOST if cloud else host
|
host = PROD_ENGINE_HOST if cloud else host
|
||||||
|
|
@ -481,7 +502,13 @@ def serve(
|
||||||
"""
|
"""
|
||||||
Start a local Arcade Worker server.
|
Start a local Arcade Worker server.
|
||||||
"""
|
"""
|
||||||
from arcade.cli.serve import serve_default_worker
|
require_dependency(
|
||||||
|
package_name="arcade_serve",
|
||||||
|
command_name="serve",
|
||||||
|
install_command=r"pip install 'arcade-ai\[serve]'",
|
||||||
|
)
|
||||||
|
|
||||||
|
from arcade_cli.serve import serve_default_worker
|
||||||
|
|
||||||
try:
|
try:
|
||||||
serve_default_worker(
|
serve_default_worker(
|
||||||
|
|
@ -501,31 +528,6 @@ def serve(
|
||||||
typer.Exit(code=1)
|
typer.Exit(code=1)
|
||||||
|
|
||||||
|
|
||||||
@cli.command(help="Launch Arcade - requires 'arcade-engine'", rich_help_panel="Launch")
|
|
||||||
def dev(
|
|
||||||
host: str = typer.Option("127.0.0.1", help="Host for the toolkit server.", show_default=True),
|
|
||||||
port: int = typer.Option(
|
|
||||||
8002, "-p", "--port", help="Port for the toolkit server.", show_default=True
|
|
||||||
),
|
|
||||||
engine_config: str = typer.Option(
|
|
||||||
None, "-c", "--config", help="Path to the engine configuration file."
|
|
||||||
),
|
|
||||||
env_file: str = typer.Option(
|
|
||||||
None, "-e", "--env-file", help="Path to the environment variables file."
|
|
||||||
),
|
|
||||||
debug: bool = typer.Option(False, "-d", "--debug", help="Show debug information"),
|
|
||||||
) -> None:
|
|
||||||
"""
|
|
||||||
Start both the toolkit server and engine servers.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
start_servers(host, port, engine_config, engine_env=env_file, debug=debug)
|
|
||||||
except Exception as e:
|
|
||||||
error_message = f"❌ Failed to start servers: {escape(str(e))}"
|
|
||||||
console.print(error_message, style="bold red")
|
|
||||||
typer.Exit(code=1)
|
|
||||||
|
|
||||||
|
|
||||||
@cli.command(
|
@cli.command(
|
||||||
help="Start a server with locally installed Arcade tools", rich_help_panel="Launch", hidden=True
|
help="Start a server with locally installed Arcade tools", rich_help_panel="Launch", hidden=True
|
||||||
)
|
)
|
||||||
|
|
@ -553,7 +555,13 @@ def workerup(
|
||||||
Starts the worker with host, port, and reload options. Uses
|
Starts the worker with host, port, and reload options. Uses
|
||||||
Uvicorn as ASGI worker. Parameters allow runtime configuration.
|
Uvicorn as ASGI worker. Parameters allow runtime configuration.
|
||||||
"""
|
"""
|
||||||
from arcade.cli.serve import serve_default_worker
|
require_dependency(
|
||||||
|
package_name="arcade_serve",
|
||||||
|
command_name="worker",
|
||||||
|
install_command=r"pip install 'arcade-ai\[worker]'",
|
||||||
|
)
|
||||||
|
|
||||||
|
from arcade_cli.serve import serve_default_worker
|
||||||
|
|
||||||
try:
|
try:
|
||||||
serve_default_worker(
|
serve_default_worker(
|
||||||
219
libs/arcade-cli/arcade_cli/new.py
Normal file
219
libs/arcade-cli/arcade_cli/new.py
Normal file
|
|
@ -0,0 +1,219 @@
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
from datetime import datetime
|
||||||
|
from importlib.metadata import version as get_version
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import typer
|
||||||
|
from jinja2 import Environment, FileSystemLoader, select_autoescape
|
||||||
|
from rich.console import Console
|
||||||
|
|
||||||
|
from arcade_cli.deployment import (
|
||||||
|
create_demo_deployment,
|
||||||
|
)
|
||||||
|
|
||||||
|
console = Console()
|
||||||
|
|
||||||
|
# Retrieve the installed version of arcade-ai
|
||||||
|
try:
|
||||||
|
ARCADE_AI_MIN_VERSION = get_version("arcade-ai")
|
||||||
|
ARCADE_AI_MAX_VERSION = str(int(ARCADE_AI_MIN_VERSION.split(".")[0]) + 1) + ".0.0"
|
||||||
|
except Exception as e:
|
||||||
|
console.print(f"[red]Failed to get arcade-ai version: {e}[/red]")
|
||||||
|
ARCADE_AI_MIN_VERSION = "2.0.0" # Default version if unable to fetch
|
||||||
|
ARCADE_AI_MAX_VERSION = "3.0.0"
|
||||||
|
|
||||||
|
ARCADE_TDK_MIN_VERSION = "0.1.0"
|
||||||
|
ARCADE_TDK_MAX_VERSION = "1.0.0"
|
||||||
|
ARCADE_SERVE_MIN_VERSION = "0.1.0"
|
||||||
|
ARCADE_SERVE_MAX_VERSION = "1.0.0"
|
||||||
|
|
||||||
|
|
||||||
|
def ask_question(question: str, default: Optional[str] = None) -> str:
|
||||||
|
"""
|
||||||
|
Ask a question via input() and return the answer.
|
||||||
|
"""
|
||||||
|
answer = typer.prompt(question, default=default, show_default=False)
|
||||||
|
if not answer and default:
|
||||||
|
return default
|
||||||
|
return str(answer)
|
||||||
|
|
||||||
|
|
||||||
|
def ask_yes_no_question(question: str, default: bool = True) -> bool:
|
||||||
|
"""
|
||||||
|
Ask a yes/no question via input() and return the bool answer.
|
||||||
|
"""
|
||||||
|
default_str = "Y/n" if default else "y/N"
|
||||||
|
answer = typer.prompt(
|
||||||
|
f"{question} ({default_str})", default="y" if default else "n", show_default=False
|
||||||
|
)
|
||||||
|
return answer.lower() in [
|
||||||
|
"y",
|
||||||
|
"y/",
|
||||||
|
"yes",
|
||||||
|
"true",
|
||||||
|
"1",
|
||||||
|
"ye",
|
||||||
|
"yes",
|
||||||
|
"yeah",
|
||||||
|
"yep",
|
||||||
|
"sure",
|
||||||
|
"ok",
|
||||||
|
"yup",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def render_template(env: Environment, template_string: str, context: dict) -> str:
|
||||||
|
"""Render a template string with the given variables."""
|
||||||
|
template = env.from_string(template_string)
|
||||||
|
return template.render(context)
|
||||||
|
|
||||||
|
|
||||||
|
def write_template(path: Path, content: str) -> None:
|
||||||
|
"""Write content to a file."""
|
||||||
|
path.write_text(content, encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def create_ignore_pattern(include_evals: bool) -> re.Pattern[str]:
|
||||||
|
"""Create an ignore pattern based on user preferences."""
|
||||||
|
patterns = [
|
||||||
|
"__pycache__",
|
||||||
|
r"\.DS_Store",
|
||||||
|
r"Thumbs\.db",
|
||||||
|
r"\.git",
|
||||||
|
r"\.svn",
|
||||||
|
r"\.hg",
|
||||||
|
r"\.vscode",
|
||||||
|
r"\.idea",
|
||||||
|
"build",
|
||||||
|
"dist",
|
||||||
|
r".*\.egg-info",
|
||||||
|
r".*\.pyc",
|
||||||
|
r".*\.pyo",
|
||||||
|
]
|
||||||
|
|
||||||
|
if not include_evals:
|
||||||
|
patterns.append("evals")
|
||||||
|
|
||||||
|
return re.compile(f"({'|'.join(patterns)})$")
|
||||||
|
|
||||||
|
|
||||||
|
def create_package(
|
||||||
|
env: Environment,
|
||||||
|
template_path: Path,
|
||||||
|
output_path: Path,
|
||||||
|
context: dict,
|
||||||
|
ignore_pattern: re.Pattern[str],
|
||||||
|
) -> None:
|
||||||
|
"""Recursively create a new toolkit directory structure from jinja2 templates."""
|
||||||
|
if ignore_pattern.match(template_path.name):
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
if template_path.is_dir():
|
||||||
|
folder_name = render_template(env, template_path.name, context)
|
||||||
|
new_dir_path = output_path / folder_name
|
||||||
|
new_dir_path.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
for item in template_path.iterdir():
|
||||||
|
create_package(env, item, new_dir_path, context, ignore_pattern)
|
||||||
|
|
||||||
|
else:
|
||||||
|
# Render the file name
|
||||||
|
file_name = render_template(env, template_path.name, context)
|
||||||
|
with open(template_path, encoding="utf-8") as f:
|
||||||
|
content = f.read()
|
||||||
|
# Render the file content
|
||||||
|
content = render_template(env, content, context)
|
||||||
|
|
||||||
|
write_template(output_path / file_name, content)
|
||||||
|
except Exception as e:
|
||||||
|
console.print(f"[red]Failed to create package: {e}[/red]")
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def remove_toolkit(toolkit_directory: Path, toolkit_name: str) -> None:
|
||||||
|
"""Teardown logic for when creating a new toolkit fails."""
|
||||||
|
toolkit_path = toolkit_directory / toolkit_name
|
||||||
|
if toolkit_path.exists():
|
||||||
|
shutil.rmtree(toolkit_path)
|
||||||
|
|
||||||
|
|
||||||
|
def create_new_toolkit(output_directory: str, toolkit_name: str) -> None:
|
||||||
|
"""Create a new toolkit from a template with user input."""
|
||||||
|
toolkit_directory = Path(output_directory)
|
||||||
|
|
||||||
|
package_name = toolkit_name if toolkit_name.startswith("arcade_") else f"arcade_{toolkit_name}"
|
||||||
|
|
||||||
|
# Check for illegal characters in the toolkit name
|
||||||
|
if re.match(r"^[a-z0-9_]+$", package_name):
|
||||||
|
toolkit_name = package_name.replace("arcade_", "", 1)
|
||||||
|
|
||||||
|
if (toolkit_directory / toolkit_name).exists():
|
||||||
|
console.print(f"[red]Toolkit '{toolkit_name}' already exists.[/red]")
|
||||||
|
exit(1)
|
||||||
|
else:
|
||||||
|
console.print(
|
||||||
|
"[red]Toolkit name contains illegal characters. "
|
||||||
|
"Only lowercase alphanumeric characters and underscores are allowed. "
|
||||||
|
"Please try again.[/red]"
|
||||||
|
)
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
toolkit_description = ask_question("Describe what your toolkit will do (optional)", default="")
|
||||||
|
toolkit_author_name = ask_question("Your GitHub username (optional)", default="")
|
||||||
|
while True:
|
||||||
|
toolkit_author_email = ask_question("Your email (optional)", default="")
|
||||||
|
if toolkit_author_email == "" or re.match(r"[^@ ]+@[^@ ]+\.[^@ ]+", toolkit_author_email):
|
||||||
|
break
|
||||||
|
console.print(
|
||||||
|
"[red]Invalid email format. Please enter a valid email address or leave it empty.[/red]"
|
||||||
|
)
|
||||||
|
include_evals = ask_yes_no_question(
|
||||||
|
"Do you want an evals directory created for you?", default=True
|
||||||
|
)
|
||||||
|
|
||||||
|
context = {
|
||||||
|
"package_name": package_name,
|
||||||
|
"toolkit_name": toolkit_name,
|
||||||
|
"toolkit_description": toolkit_description,
|
||||||
|
"toolkit_author_name": toolkit_author_name,
|
||||||
|
"toolkit_author_email": toolkit_author_email,
|
||||||
|
"arcade_tdk_min_version": ARCADE_TDK_MIN_VERSION,
|
||||||
|
"arcade_tdk_max_version": ARCADE_TDK_MAX_VERSION,
|
||||||
|
"arcade_serve_min_version": ARCADE_SERVE_MIN_VERSION,
|
||||||
|
"arcade_serve_max_version": ARCADE_SERVE_MAX_VERSION,
|
||||||
|
"arcade_ai_min_version": ARCADE_AI_MIN_VERSION,
|
||||||
|
"arcade_ai_max_version": ARCADE_AI_MAX_VERSION,
|
||||||
|
"creation_year": datetime.now().year,
|
||||||
|
}
|
||||||
|
template_directory = Path(__file__).parent / "templates" / "{{ toolkit_name }}"
|
||||||
|
|
||||||
|
env = Environment(
|
||||||
|
loader=FileSystemLoader(str(template_directory)),
|
||||||
|
autoescape=select_autoescape(["html", "xml"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create dynamic ignore pattern based on user preferences
|
||||||
|
ignore_pattern = create_ignore_pattern(include_evals)
|
||||||
|
|
||||||
|
try:
|
||||||
|
create_package(env, template_directory, toolkit_directory, context, ignore_pattern)
|
||||||
|
console.print(
|
||||||
|
f"[green]Toolkit '{toolkit_name}' created successfully at '{toolkit_directory}'.[/green]"
|
||||||
|
)
|
||||||
|
create_deployment(toolkit_directory, toolkit_name)
|
||||||
|
except Exception:
|
||||||
|
remove_toolkit(toolkit_directory, toolkit_name)
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def create_deployment(toolkit_directory: Path, toolkit_name: str) -> None:
|
||||||
|
worker_toml = toolkit_directory / "worker.toml"
|
||||||
|
if not worker_toml.exists():
|
||||||
|
create_demo_deployment(worker_toml, toolkit_name)
|
||||||
|
else:
|
||||||
|
pass
|
||||||
|
# Disabled pending bug fix
|
||||||
|
# update_deployment_with_local_packages(worker_toml, toolkit_name)
|
||||||
|
|
@ -15,18 +15,18 @@ import uvicorn
|
||||||
# Watchfiles is used under the hood by Uvicorn's reload feature.
|
# Watchfiles is used under the hood by Uvicorn's reload feature.
|
||||||
# Importing watchfiles here is an explicit acknowledgement that it needs to be installed
|
# Importing watchfiles here is an explicit acknowledgement that it needs to be installed
|
||||||
import watchfiles # noqa: F401
|
import watchfiles # noqa: F401
|
||||||
|
from arcade_core.telemetry import OTELHandler
|
||||||
|
from arcade_core.toolkit import Toolkit, get_package_directory
|
||||||
|
from arcade_serve.fastapi.worker import FastAPIWorker
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from rich.console import Console
|
from rich.console import Console
|
||||||
|
|
||||||
from arcade.cli.constants import ARCADE_CONFIG_PATH
|
from arcade_cli.constants import ARCADE_CONFIG_PATH
|
||||||
from arcade.cli.utils import (
|
from arcade_cli.utils import (
|
||||||
build_tool_catalog,
|
build_tool_catalog,
|
||||||
discover_toolkits,
|
discover_toolkits,
|
||||||
load_dotenv,
|
load_dotenv,
|
||||||
)
|
)
|
||||||
from arcade.core.telemetry import OTELHandler
|
|
||||||
from arcade.core.toolkit import Toolkit, get_package_directory
|
|
||||||
from arcade.worker.fastapi.worker import FastAPIWorker
|
|
||||||
|
|
||||||
console = Console(width=70, color_system="auto")
|
console = Console(width=70, color_system="auto")
|
||||||
|
|
||||||
|
|
@ -45,7 +45,6 @@ def create_arcade_app() -> fastapi.FastAPI:
|
||||||
setup_logging(log_level=logging.DEBUG if debug_mode else logging.INFO, mcp_mode=False)
|
setup_logging(log_level=logging.DEBUG if debug_mode else logging.INFO, mcp_mode=False)
|
||||||
|
|
||||||
logger.info(f"Debug: {debug_mode}, OTEL: {otel_enabled}, Auth Disabled: {auth_for_reload}")
|
logger.info(f"Debug: {debug_mode}, OTEL: {otel_enabled}, Auth Disabled: {auth_for_reload}")
|
||||||
|
|
||||||
version = get_pkg_version("arcade-ai")
|
version = get_pkg_version("arcade-ai")
|
||||||
toolkits = discover_toolkits()
|
toolkits = discover_toolkits()
|
||||||
|
|
||||||
|
|
@ -94,7 +93,7 @@ def _run_mcp_stdio(
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Launch an MCP stdio server; blocks until it exits."""
|
"""Launch an MCP stdio server; blocks until it exits."""
|
||||||
|
|
||||||
from arcade.worker.mcp.stdio import StdioServer
|
from arcade_serve.mcp.stdio import StdioServer
|
||||||
|
|
||||||
# Load env vars before launching server (explicit path, config path, cwd)
|
# Load env vars before launching server (explicit path, config path, cwd)
|
||||||
if env_file:
|
if env_file:
|
||||||
|
|
@ -139,7 +138,7 @@ def _run_fastapi_server(
|
||||||
toolkits_for_reload_dirs: list[Toolkit] | None,
|
toolkits_for_reload_dirs: list[Toolkit] | None,
|
||||||
debug_flag: bool,
|
debug_flag: bool,
|
||||||
) -> None:
|
) -> None:
|
||||||
app_import_string = "arcade.cli.serve:create_arcade_app"
|
app_import_string = "arcade_cli.serve:create_arcade_app"
|
||||||
reload_dirs_str_list: list[str] | None = None
|
reload_dirs_str_list: list[str] | None = None
|
||||||
|
|
||||||
if reload:
|
if reload:
|
||||||
|
|
@ -2,8 +2,8 @@ import typer
|
||||||
from rich.console import Console
|
from rich.console import Console
|
||||||
from rich.markup import escape
|
from rich.markup import escape
|
||||||
|
|
||||||
from arcade.cli.display import display_tool_details, display_tools_table
|
from arcade_cli.display import display_tool_details, display_tools_table
|
||||||
from arcade.cli.utils import create_cli_catalog, get_tools_from_engine
|
from arcade_cli.utils import create_cli_catalog, get_tools_from_engine
|
||||||
|
|
||||||
|
|
||||||
def show_logic(
|
def show_logic(
|
||||||
|
|
@ -0,0 +1,46 @@
|
||||||
|
.PHONY: help
|
||||||
|
|
||||||
|
help:
|
||||||
|
@echo "🛠️ github Commands:\n"
|
||||||
|
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
|
||||||
|
|
||||||
|
.PHONY: install
|
||||||
|
install: ## Install the uv environment and install all packages with dependencies
|
||||||
|
@echo "🚀 Creating virtual environment and installing all packages using uv"
|
||||||
|
@uv sync --active --all-extras --no-sources
|
||||||
|
@uv run pre-commit install
|
||||||
|
@echo "✅ All packages and dependencies installed via uv"
|
||||||
|
|
||||||
|
.PHONY: install-local
|
||||||
|
install-local: ## Install the uv environment and install all packages with dependencies with local Arcade sources
|
||||||
|
@echo "🚀 Creating virtual environment and installing all packages using uv"
|
||||||
|
@uv sync --active --all-extras
|
||||||
|
@uv run pre-commit install
|
||||||
|
@echo "✅ All packages and dependencies installed via uv"
|
||||||
|
|
||||||
|
.PHONY: build
|
||||||
|
build: clean-build ## Build wheel file using poetry
|
||||||
|
@echo "🚀 Creating wheel file"
|
||||||
|
uv build
|
||||||
|
|
||||||
|
.PHONY: clean-build
|
||||||
|
clean-build: ## clean build artifacts
|
||||||
|
@echo "🗑️ Cleaning dist directory"
|
||||||
|
rm -rf dist
|
||||||
|
|
||||||
|
.PHONY: test
|
||||||
|
test: ## Test the code with pytest
|
||||||
|
@echo "🚀 Testing code: Running pytest"
|
||||||
|
@uv run pytest -W ignore -v --cov --cov-config=pyproject.toml --cov-report=xml
|
||||||
|
|
||||||
|
.PHONY: bump-version
|
||||||
|
bump-version: ## Bump the version in the pyproject.toml file by a patch version
|
||||||
|
@echo "🚀 Bumping version in pyproject.toml"
|
||||||
|
uv version --bump patch
|
||||||
|
|
||||||
|
.PHONY: check
|
||||||
|
check: ## Run code quality tools.
|
||||||
|
@echo "🚀 Linting code: Running pre-commit"
|
||||||
|
@uv run pre-commit run -a
|
||||||
|
@echo "🚀 Static type checking: Running mypy"
|
||||||
|
@uv run mypy --config-file=pyproject.toml
|
||||||
|
|
@ -6,11 +6,14 @@
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div style="display: flex; justify-content: center; align-items: center; margin-bottom: 8px;">
|
<div style="display: flex; justify-content: center; align-items: center; margin-bottom: 8px;">
|
||||||
|
{% if toolkit_author_name -%}
|
||||||
<img src="https://img.shields.io/github/v/release/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub release" style="margin: 0 2px;">
|
<img src="https://img.shields.io/github/v/release/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub release" style="margin: 0 2px;">
|
||||||
|
{% endif -%}
|
||||||
<img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python version" style="margin: 0 2px;">
|
<img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python version" style="margin: 0 2px;">
|
||||||
<img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License" style="margin: 0 2px;">
|
<img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License" style="margin: 0 2px;">
|
||||||
<img src="https://img.shields.io/pypi/v/{{ package_name }}" alt="PyPI version" style="margin: 0 2px;">
|
<img src="https://img.shields.io/pypi/v/{{ package_name }}" alt="PyPI version" style="margin: 0 2px;">
|
||||||
</div>
|
</div>
|
||||||
|
{% if toolkit_author_name -%}
|
||||||
<div style="display: flex; justify-content: center; align-items: center;">
|
<div style="display: flex; justify-content: center; align-items: center;">
|
||||||
<a href="https://github.com/{{ toolkit_author_name }}/{{ toolkit_name }}" target="_blank">
|
<a href="https://github.com/{{ toolkit_author_name }}/{{ toolkit_name }}" target="_blank">
|
||||||
<img src="https://img.shields.io/github/stars/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub stars" style="margin: 0 2px;">
|
<img src="https://img.shields.io/github/stars/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub stars" style="margin: 0 2px;">
|
||||||
|
|
@ -19,22 +22,19 @@
|
||||||
<img src="https://img.shields.io/github/forks/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub forks" style="margin: 0 2px;">
|
<img src="https://img.shields.io/github/forks/{{ toolkit_author_name }}/{{ toolkit_name }}" alt="GitHub forks" style="margin: 0 2px;">
|
||||||
</a>
|
</a>
|
||||||
</div>
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
<br>
|
<br>
|
||||||
<br>
|
<br>
|
||||||
|
|
||||||
# Arcade {{ toolkit_name }} Toolkit
|
# Arcade {{ toolkit_name }} Toolkit
|
||||||
|
{% if toolkit_description -%}
|
||||||
{{ toolkit_description }}
|
{{ toolkit_description }}
|
||||||
|
{% endif -%}
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- The {{ toolkit_name }} toolkit does not have any features yet.
|
- The {{ toolkit_name }} toolkit does not have any features yet.
|
||||||
|
|
||||||
## Install
|
## Development
|
||||||
|
|
||||||
Install this toolkit using pip:
|
Read the docs on how to create a toolkit [here](https://docs.arcade.dev/home/build-tools/create-a-toolkit)
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install {{ package_name }}
|
|
||||||
```
|
|
||||||
|
|
@ -1,11 +1,11 @@
|
||||||
from arcade.sdk import ToolCatalog
|
from arcade_tdk import ToolCatalog
|
||||||
from arcade.sdk.eval import (
|
from arcade_evals import (
|
||||||
EvalRubric,
|
EvalRubric,
|
||||||
EvalSuite,
|
EvalSuite,
|
||||||
ExpectedToolCall,
|
ExpectedToolCall,
|
||||||
SimilarityCritic,
|
|
||||||
tool_eval,
|
tool_eval,
|
||||||
)
|
)
|
||||||
|
from arcade_evals.critic import SimilarityCritic
|
||||||
|
|
||||||
import {{ package_name }}
|
import {{ package_name }}
|
||||||
from {{ package_name }}.tools.hello import say_hello
|
from {{ package_name }}.tools.hello import say_hello
|
||||||
|
|
@ -0,0 +1,66 @@
|
||||||
|
[build-system]
|
||||||
|
requires = [ "hatchling",]
|
||||||
|
build-backend = "hatchling.build"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "{{ package_name }}"
|
||||||
|
version = "0.1.0"
|
||||||
|
{% if toolkit_description -%}
|
||||||
|
description = "{{ toolkit_description }}"
|
||||||
|
{% endif -%}
|
||||||
|
requires-python = ">=3.10"
|
||||||
|
dependencies = [
|
||||||
|
"arcade-tdk>={{ arcade_tdk_min_version }},<{{ arcade_tdk_max_version}}",
|
||||||
|
]
|
||||||
|
{% if toolkit_author_name or toolkit_author_email -%}
|
||||||
|
[[project.authors]]
|
||||||
|
{% if toolkit_author_name -%}
|
||||||
|
name = "{{ toolkit_author_name }}"
|
||||||
|
{% endif -%}
|
||||||
|
{% if toolkit_author_email -%}
|
||||||
|
email = "{{ toolkit_author_email }}"
|
||||||
|
{% endif -%}
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
dev = [
|
||||||
|
"arcade-ai[evals]>={{ arcade_ai_min_version }},<{{ arcade_ai_max_version }}",
|
||||||
|
"arcade-serve>={{ arcade_serve_min_version }},<{{ arcade_serve_max_version }}",
|
||||||
|
"pytest>=8.3.0,<8.4.0",
|
||||||
|
"pytest-cov>=4.0.0,<4.1.0",
|
||||||
|
"pytest-mock>=3.11.1,<3.12.0",
|
||||||
|
"pytest-asyncio>=0.24.0,<0.25.0",
|
||||||
|
"mypy>=1.5.1,<1.6.0",
|
||||||
|
"pre-commit>=3.4.0,<3.5.0",
|
||||||
|
"tox>=4.11.1,<4.12.0",
|
||||||
|
"ruff>=0.7.4,<0.8.0",
|
||||||
|
]
|
||||||
|
|
||||||
|
# Use local path sources for arcade libs when working locally
|
||||||
|
[tool.uv.sources]
|
||||||
|
arcade-ai = { path = "../../", editable = true }
|
||||||
|
arcade-serve = { path = "../../libs/arcade-serve/", editable = true }
|
||||||
|
arcade-tdk = { path = "../../libs/arcade-tdk/", editable = true }
|
||||||
|
|
||||||
|
|
||||||
|
[tool.mypy]
|
||||||
|
files = [ "{{ package_name }}/**/*.py",]
|
||||||
|
python_version = "3.10"
|
||||||
|
disallow_untyped_defs = "True"
|
||||||
|
disallow_any_unimported = "True"
|
||||||
|
no_implicit_optional = "True"
|
||||||
|
check_untyped_defs = "True"
|
||||||
|
warn_return_any = "True"
|
||||||
|
warn_unused_ignores = "True"
|
||||||
|
show_error_codes = "True"
|
||||||
|
ignore_missing_imports = "True"
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
testpaths = [ "tests",]
|
||||||
|
|
||||||
|
[tool.coverage.report]
|
||||||
|
skip_empty = true
|
||||||
|
|
||||||
|
[tool.hatch.build.targets.wheel]
|
||||||
|
packages = [ "{{ package_name }}",]
|
||||||
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
import pytest
|
import pytest
|
||||||
from arcade.sdk.errors import ToolExecutionError
|
from arcade_tdk.errors import ToolExecutionError
|
||||||
|
|
||||||
from {{ package_name }}.tools.hello import say_hello
|
from {{ package_name }}.tools.hello import say_hello
|
||||||
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
from typing import Annotated
|
from typing import Annotated
|
||||||
|
|
||||||
from arcade.sdk import tool
|
from arcade_tdk import tool
|
||||||
|
|
||||||
|
|
||||||
@tool
|
@tool
|
||||||
|
|
@ -8,4 +8,3 @@ def say_hello(name: Annotated[str, "The name of the person to greet"]) -> str:
|
||||||
"""Say a greeting!"""
|
"""Say a greeting!"""
|
||||||
|
|
||||||
return "Hello, " + name + "!"
|
return "Hello, " + name + "!"
|
||||||
|
|
||||||
|
|
@ -6,6 +6,7 @@ import webbrowser
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from enum import Enum
|
from enum import Enum
|
||||||
|
from importlib import metadata
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from textwrap import dedent
|
from textwrap import dedent
|
||||||
from typing import Any, Callable, Union, cast
|
from typing import Any, Callable, Union, cast
|
||||||
|
|
@ -13,6 +14,10 @@ from urllib.parse import urlencode, urlparse
|
||||||
|
|
||||||
import idna
|
import idna
|
||||||
import typer
|
import typer
|
||||||
|
from arcade_core import ToolCatalog, Toolkit
|
||||||
|
from arcade_core.config_model import Config
|
||||||
|
from arcade_core.errors import ToolkitLoadError
|
||||||
|
from arcade_core.schema import ToolDefinition
|
||||||
from arcadepy import NOT_GIVEN, APIConnectionError, APIStatusError, APITimeoutError, Arcade
|
from arcadepy import NOT_GIVEN, APIConnectionError, APIStatusError, APITimeoutError, Arcade
|
||||||
from arcadepy.types import AuthorizationResponse
|
from arcadepy.types import AuthorizationResponse
|
||||||
from openai import OpenAI, Stream
|
from openai import OpenAI, Stream
|
||||||
|
|
@ -27,11 +32,7 @@ from rich.text import Text
|
||||||
from typer.core import TyperGroup
|
from typer.core import TyperGroup
|
||||||
from typer.models import Context
|
from typer.models import Context
|
||||||
|
|
||||||
from arcade.cli.constants import LOCALHOST
|
from arcade_cli.constants import LOCALHOST
|
||||||
from arcade.core.config_model import Config
|
|
||||||
from arcade.core.errors import ToolkitLoadError
|
|
||||||
from arcade.core.schema import ToolDefinition
|
|
||||||
from arcade.sdk import ToolCatalog, Toolkit
|
|
||||||
|
|
||||||
console = Console()
|
console = Console()
|
||||||
|
|
||||||
|
|
@ -300,7 +301,7 @@ def validate_and_get_config(
|
||||||
"""
|
"""
|
||||||
Validates the configuration, user, and returns the Config object
|
Validates the configuration, user, and returns the Config object
|
||||||
"""
|
"""
|
||||||
from arcade.core.config import config
|
from arcade_core.config import config
|
||||||
|
|
||||||
if validate_api and (not config.api or not config.api.key):
|
if validate_api and (not config.api or not config.api.key):
|
||||||
console.print(
|
console.print(
|
||||||
|
|
@ -664,8 +665,8 @@ def version_callback(value: bool) -> None:
|
||||||
Prints the version of Arcade and exit.
|
Prints the version of Arcade and exit.
|
||||||
"""
|
"""
|
||||||
if value:
|
if value:
|
||||||
version = importlib.import_module("arcade").__version__
|
version = metadata.version("arcade-ai")
|
||||||
console.print(f"[bold]Arcade[/bold] (version {version})")
|
console.print(f"[bold]Arcade CLI[/bold] (version {version})")
|
||||||
exit()
|
exit()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -751,3 +752,30 @@ def load_dotenv(path: str | Path, *, override: bool = False) -> dict[str, str]:
|
||||||
loaded[k] = v
|
loaded[k] = v
|
||||||
|
|
||||||
return loaded
|
return loaded
|
||||||
|
|
||||||
|
|
||||||
|
def require_dependency(
|
||||||
|
package_name: str,
|
||||||
|
command_name: str,
|
||||||
|
install_command: str,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
Display a helpful error message if the required dependency is missing.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
package_name: The name of the package to import (e.g., 'arcade_serve')
|
||||||
|
command_name: The command that requires the package (e.g., 'serve')
|
||||||
|
install_command: The command to install the package (e.g., "pip install 'arcade-ai[evals]'")
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
importlib.import_module(package_name.replace("-", "_"))
|
||||||
|
except ImportError:
|
||||||
|
console.print(
|
||||||
|
f"❌ The '{package_name}' package is required to run the '{command_name}' command but is not installed.",
|
||||||
|
style="bold red",
|
||||||
|
)
|
||||||
|
console.print(
|
||||||
|
f"To install it, run the following command:\n* [green]{install_command}[/green]",
|
||||||
|
style="bold",
|
||||||
|
)
|
||||||
|
raise typer.Exit(code=1)
|
||||||
|
|
@ -4,11 +4,11 @@ from arcadepy import Arcade, NotFoundError
|
||||||
from rich.console import Console
|
from rich.console import Console
|
||||||
from rich.table import Table
|
from rich.table import Table
|
||||||
|
|
||||||
from arcade.cli.constants import (
|
from arcade_cli.constants import (
|
||||||
PROD_CLOUD_HOST,
|
PROD_CLOUD_HOST,
|
||||||
PROD_ENGINE_HOST,
|
PROD_ENGINE_HOST,
|
||||||
)
|
)
|
||||||
from arcade.cli.utils import (
|
from arcade_cli.utils import (
|
||||||
OrderCommands,
|
OrderCommands,
|
||||||
compute_base_url,
|
compute_base_url,
|
||||||
validate_and_get_config,
|
validate_and_get_config,
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
from arcade.cli.main import cli
|
from arcade_cli.main import cli
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
# Supports attaching debugger to cli. Run from ../.vscode/launch.json.
|
# Supports attaching debugger to cli. Run from ../.vscode/launch.json.
|
||||||
39
libs/arcade-core/README.md
Normal file
39
libs/arcade-core/README.md
Normal file
|
|
@ -0,0 +1,39 @@
|
||||||
|
# Arcade Core
|
||||||
|
|
||||||
|
Core library for the Arcade platform providing foundational components and utilities.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Arcade Core provides the essential building blocks for the Arcade platform:
|
||||||
|
|
||||||
|
- **Tool Catalog & Toolkit Management**: Core classes for managing and organizing tools
|
||||||
|
- **Configuration & Schema Handling**: Configuration management and validation
|
||||||
|
- **Authentication & Authorization**: Auth providers and security utilities
|
||||||
|
- **Error Handling**: Comprehensive error types and handling
|
||||||
|
- **Telemetry & Observability**: Monitoring and tracing capabilities
|
||||||
|
- **Utilities**: Common helper functions and validators
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install arcade-core
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_core import ToolCatalog, Toolkit, ArcadeConfig
|
||||||
|
|
||||||
|
# Create a tool catalog
|
||||||
|
catalog = ToolCatalog()
|
||||||
|
|
||||||
|
# Load a toolkit
|
||||||
|
toolkit = Toolkit.from_directory("path/to/toolkit")
|
||||||
|
|
||||||
|
# Configure Arcade
|
||||||
|
config = ArcadeConfig.from_file("config.yaml")
|
||||||
|
```
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT License - see LICENSE file for details.
|
||||||
2
libs/arcade-core/arcade_core/__init__.py
Normal file
2
libs/arcade-core/arcade_core/__init__.py
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
from .catalog import ToolCatalog as ToolCatalog
|
||||||
|
from .toolkit import Toolkit as Toolkit
|
||||||
|
|
@ -26,10 +26,10 @@ from pydantic import BaseModel, Field, create_model
|
||||||
from pydantic.fields import FieldInfo
|
from pydantic.fields import FieldInfo
|
||||||
from pydantic_core import PydanticUndefined
|
from pydantic_core import PydanticUndefined
|
||||||
|
|
||||||
from arcade.core.annotations import Inferrable
|
from arcade_core.annotations import Inferrable
|
||||||
from arcade.core.auth import OAuth2, ToolAuthorization
|
from arcade_core.auth import OAuth2, ToolAuthorization
|
||||||
from arcade.core.errors import ToolDefinitionError
|
from arcade_core.errors import ToolDefinitionError
|
||||||
from arcade.core.schema import (
|
from arcade_core.schema import (
|
||||||
TOOL_NAME_SEPARATOR,
|
TOOL_NAME_SEPARATOR,
|
||||||
FullyQualifiedName,
|
FullyQualifiedName,
|
||||||
InputParameter,
|
InputParameter,
|
||||||
|
|
@ -46,8 +46,8 @@ from arcade.core.schema import (
|
||||||
ToolSecretRequirement,
|
ToolSecretRequirement,
|
||||||
ValueSchema,
|
ValueSchema,
|
||||||
)
|
)
|
||||||
from arcade.core.toolkit import Toolkit
|
from arcade_core.toolkit import Toolkit
|
||||||
from arcade.core.utils import (
|
from arcade_core.utils import (
|
||||||
does_function_return_value,
|
does_function_return_value,
|
||||||
first_or_none,
|
first_or_none,
|
||||||
is_strict_optional,
|
is_strict_optional,
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
from functools import lru_cache
|
from functools import lru_cache
|
||||||
|
|
||||||
from arcade.core.config_model import Config
|
from arcade_core.config_model import Config
|
||||||
|
|
||||||
|
|
||||||
@lru_cache(maxsize=1)
|
@lru_cache(maxsize=1)
|
||||||
|
|
@ -4,15 +4,15 @@ from typing import Any, Callable
|
||||||
|
|
||||||
from pydantic import BaseModel, ValidationError
|
from pydantic import BaseModel, ValidationError
|
||||||
|
|
||||||
from arcade.core.errors import (
|
from arcade_core.errors import (
|
||||||
RetryableToolError,
|
RetryableToolError,
|
||||||
ToolInputError,
|
ToolInputError,
|
||||||
ToolOutputError,
|
ToolOutputError,
|
||||||
ToolRuntimeError,
|
ToolRuntimeError,
|
||||||
ToolSerializationError,
|
ToolSerializationError,
|
||||||
)
|
)
|
||||||
from arcade.core.output import output_factory
|
from arcade_core.output import output_factory
|
||||||
from arcade.core.schema import ToolCallLog, ToolCallOutput, ToolContext, ToolDefinition
|
from arcade_core.schema import ToolCallLog, ToolCallOutput, ToolContext, ToolDefinition
|
||||||
|
|
||||||
|
|
||||||
class ToolExecutor:
|
class ToolExecutor:
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
from typing import TypeVar
|
from typing import TypeVar
|
||||||
|
|
||||||
from arcade.core.schema import ToolCallError, ToolCallLog, ToolCallOutput
|
from arcade_core.schema import ToolCallError, ToolCallLog, ToolCallOutput
|
||||||
from arcade.core.utils import coerce_empty_list_to_none
|
from arcade_core.utils import coerce_empty_list_to_none
|
||||||
|
|
||||||
T = TypeVar("T")
|
T = TypeVar("T")
|
||||||
|
|
||||||
|
|
@ -8,8 +8,8 @@ from pathlib import Path
|
||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, field_validator
|
from pydantic import BaseModel, ConfigDict, field_validator
|
||||||
|
|
||||||
from arcade.core.errors import ToolkitLoadError
|
from arcade_core.errors import ToolkitLoadError
|
||||||
from arcade.core.parse import get_tools_from_file
|
from arcade_core.parse import get_tools_from_file
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
65
libs/arcade-core/pyproject.toml
Normal file
65
libs/arcade-core/pyproject.toml
Normal file
|
|
@ -0,0 +1,65 @@
|
||||||
|
[project]
|
||||||
|
name = "arcade-core"
|
||||||
|
version = "2.0.0"
|
||||||
|
description = "Arcade Core - Core library for Arcade platform"
|
||||||
|
readme = "README.md"
|
||||||
|
license = {text = "MIT"}
|
||||||
|
authors = [
|
||||||
|
{name = "Arcade", email = "dev@arcade.dev"},
|
||||||
|
]
|
||||||
|
classifiers = [
|
||||||
|
"Development Status :: 4 - Beta",
|
||||||
|
"Intended Audience :: Developers",
|
||||||
|
"License :: OSI Approved :: MIT License",
|
||||||
|
"Programming Language :: Python :: 3",
|
||||||
|
"Programming Language :: Python :: 3.10",
|
||||||
|
"Programming Language :: Python :: 3.11",
|
||||||
|
"Programming Language :: Python :: 3.12",
|
||||||
|
"Programming Language :: Python :: 3.13",
|
||||||
|
]
|
||||||
|
requires-python = ">=3.10"
|
||||||
|
dependencies = [
|
||||||
|
"pydantic>=2.7.0",
|
||||||
|
"pyyaml>=6.0",
|
||||||
|
"loguru>=0.7.0",
|
||||||
|
"pyjwt>=2.8.0",
|
||||||
|
"toml>=0.10.2",
|
||||||
|
"packaging>=24.1",
|
||||||
|
"types-python-dateutil==2.9.0.20241003",
|
||||||
|
"types-pytz==2024.2.0.20241003",
|
||||||
|
"types-toml==0.10.8.20240310",
|
||||||
|
"opentelemetry-instrumentation-fastapi==0.49b2",
|
||||||
|
"opentelemetry-exporter-otlp-proto-http==1.28.2",
|
||||||
|
"opentelemetry-exporter-otlp-proto-common==1.28.2",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
dev = [
|
||||||
|
"pytest>=8.1.2",
|
||||||
|
"pytest-cov>=4.0.0",
|
||||||
|
"mypy>=1.5.1",
|
||||||
|
"pre-commit>=3.4.0",
|
||||||
|
"pytest-asyncio>=0.23.7",
|
||||||
|
"types-pytz>=2024.1",
|
||||||
|
"types-python-dateutil>=2.8.2",
|
||||||
|
"types-PyYAML>=6.0.0",
|
||||||
|
]
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["hatchling"]
|
||||||
|
build-backend = "hatchling.build"
|
||||||
|
|
||||||
|
[tool.hatch.build.targets.wheel]
|
||||||
|
packages = ["arcade_core"]
|
||||||
|
|
||||||
|
[tool.mypy]
|
||||||
|
files = ["arcade_core"]
|
||||||
|
python_version = "3.10"
|
||||||
|
disallow_untyped_defs = true
|
||||||
|
disallow_any_unimported = true
|
||||||
|
no_implicit_optional = true
|
||||||
|
check_untyped_defs = true
|
||||||
|
warn_return_any = true
|
||||||
|
warn_unused_ignores = true
|
||||||
|
show_error_codes = true
|
||||||
|
ignore_missing_imports = true
|
||||||
86
libs/arcade-evals/README.md
Normal file
86
libs/arcade-evals/README.md
Normal file
|
|
@ -0,0 +1,86 @@
|
||||||
|
# Arcade Evals
|
||||||
|
|
||||||
|
Evaluation toolkit for testing Arcade tools.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Arcade Evals provides comprehensive evaluation capabilities for Arcade tools:
|
||||||
|
|
||||||
|
- **Evaluation Framework**: Cases, suites, and rubrics for systematic testing
|
||||||
|
- **Critics**: Different types of comparisons (binary, numeric, similarity, datetime)
|
||||||
|
- **Tool Evaluation**: Decorators and utilities for evaluating tool performance
|
||||||
|
- **Result Analysis**: Comprehensive evaluation results and reporting
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install 'arcade-ai[evals]'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Basic Evaluation
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_evals import EvalCase, EvalSuite, tool_eval
|
||||||
|
|
||||||
|
# Create evaluation cases
|
||||||
|
case1 = EvalCase(
|
||||||
|
input={"query": "What is 2+2?"},
|
||||||
|
expected_output="4"
|
||||||
|
)
|
||||||
|
|
||||||
|
case2 = EvalCase(
|
||||||
|
input={"query": "What is the capital of France?"},
|
||||||
|
expected_output="Paris"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create evaluation suite
|
||||||
|
suite = EvalSuite(cases=[case1, case2])
|
||||||
|
|
||||||
|
# Evaluate a tool
|
||||||
|
@tool_eval(suite)
|
||||||
|
def my_calculator(query: str) -> str:
|
||||||
|
# Tool implementation
|
||||||
|
return "4" if "2+2" in query else "Unknown"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using Critics
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_evals import NumericCritic, SimilarityCritic
|
||||||
|
|
||||||
|
# Numeric comparison
|
||||||
|
numeric_critic = NumericCritic(tolerance=0.1)
|
||||||
|
result = numeric_critic.evaluate(expected=10.0, actual=10.05)
|
||||||
|
|
||||||
|
# Similarity comparison
|
||||||
|
similarity_critic = SimilarityCritic(threshold=0.8)
|
||||||
|
result = similarity_critic.evaluate(
|
||||||
|
expected="The capital of France is Paris",
|
||||||
|
actual="Paris is the capital of France"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced Evaluation
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_evals import EvalRubric, ExpectedToolCall
|
||||||
|
|
||||||
|
# Create rubric with tool calls
|
||||||
|
rubric = EvalRubric(
|
||||||
|
expected_tool_calls=[
|
||||||
|
ExpectedToolCall(
|
||||||
|
tool_name="calculator",
|
||||||
|
parameters={"operation": "add", "a": 2, "b": 2}
|
||||||
|
)
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
# Evaluate with rubric
|
||||||
|
suite = EvalSuite(cases=[case1], rubric=rubric)
|
||||||
|
```
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT License - see LICENSE file for details.
|
||||||
|
|
@ -6,7 +6,7 @@ from typing import Any, ClassVar
|
||||||
import pytz
|
import pytz
|
||||||
from dateutil import parser
|
from dateutil import parser
|
||||||
|
|
||||||
from arcade.sdk.errors import WeightError
|
from arcade_evals.errors import WeightError
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
@ -214,7 +214,7 @@ class SimilarityCritic(Critic):
|
||||||
from sklearn.metrics.pairwise import cosine_similarity
|
from sklearn.metrics.pairwise import cosine_similarity
|
||||||
except ImportError:
|
except ImportError:
|
||||||
raise ImportError(
|
raise ImportError(
|
||||||
"Use `pip install 'arcade-ai[evals]'` to install the required dependencies for similarity metrics."
|
"Use `pip install 'arcade-evals` to install the required dependencies for similarity metrics."
|
||||||
)
|
)
|
||||||
vectorizer = TfidfVectorizer()
|
vectorizer = TfidfVectorizer()
|
||||||
tfidf_matrix = vectorizer.fit_transform([expected, actual])
|
tfidf_matrix = vectorizer.fit_transform([expected, actual])
|
||||||
|
|
@ -227,7 +227,6 @@ class SimilarityCritic(Critic):
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class DatetimeCritic(Critic):
|
class DatetimeCritic(Critic):
|
||||||
"""
|
"""
|
||||||
12
libs/arcade-evals/arcade_evals/errors.py
Normal file
12
libs/arcade-evals/arcade_evals/errors.py
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
__all__ = [
|
||||||
|
"EvalError",
|
||||||
|
"WeightError",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class EvalError(Exception):
|
||||||
|
"""Base class for all evaluation errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class WeightError(EvalError):
|
||||||
|
"""Raised when the critic weights do not abide by evaluation weight constraints."""
|
||||||
|
|
@ -5,25 +5,19 @@ import json
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import TYPE_CHECKING, Any, Callable
|
from typing import TYPE_CHECKING, Any, Callable
|
||||||
|
|
||||||
from arcade.core.config_model import Config
|
import numpy as np
|
||||||
from arcade.core.schema import TOOL_NAME_SEPARATOR
|
from arcade_core.config_model import Config
|
||||||
|
from arcade_core.schema import TOOL_NAME_SEPARATOR
|
||||||
try:
|
|
||||||
import numpy as np
|
|
||||||
from scipy.optimize import linear_sum_assignment
|
|
||||||
except ImportError:
|
|
||||||
raise ImportError(
|
|
||||||
"Use `pip install 'arcade-ai[evals]'` to install the required dependencies for evaluation."
|
|
||||||
)
|
|
||||||
|
|
||||||
from openai import AsyncOpenAI
|
from openai import AsyncOpenAI
|
||||||
|
from scipy.optimize import linear_sum_assignment
|
||||||
|
|
||||||
from arcade.sdk.errors import WeightError
|
from arcade_evals.critic import NoneCritic
|
||||||
from arcade.sdk.eval.critic import NoneCritic
|
from arcade_evals.errors import WeightError
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from arcade.sdk import ToolCatalog
|
from arcade_core import ToolCatalog
|
||||||
from arcade.sdk.eval.critic import Critic
|
|
||||||
|
from arcade_evals.critic import Critic
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
70
libs/arcade-serve/README.md
Normal file
70
libs/arcade-serve/README.md
Normal file
|
|
@ -0,0 +1,70 @@
|
||||||
|
# Arcade Serve
|
||||||
|
|
||||||
|
Serving infrastructure for Arcade tools and workers.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Arcade Serve provides the infrastructure for serving Arcade tools:
|
||||||
|
|
||||||
|
- **FastAPI Worker**: High-performance FastAPI-based worker implementation
|
||||||
|
- **MCP Server**: Model Context Protocol server for tool integration
|
||||||
|
- **Core Abstractions**: Base worker classes and components
|
||||||
|
- **Authentication**: Auth utilities and routing
|
||||||
|
- **Runtime Management**: Tool execution and lifecycle management
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install arcade-serve
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### FastAPI Worker
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_serve import FastAPIWorker
|
||||||
|
|
||||||
|
# Create a FastAPI worker
|
||||||
|
worker = FastAPIWorker()
|
||||||
|
|
||||||
|
# Add tools to the worker
|
||||||
|
worker.add_toolkit("path/to/toolkit")
|
||||||
|
|
||||||
|
# Start the server
|
||||||
|
worker.start(host="0.0.0.0", port=8000)
|
||||||
|
```
|
||||||
|
|
||||||
|
### MCP Server
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_serve import StdioServer
|
||||||
|
|
||||||
|
# Create an MCP server
|
||||||
|
server = StdioServer()
|
||||||
|
|
||||||
|
# Add tools
|
||||||
|
server.add_toolkit("path/to/toolkit")
|
||||||
|
|
||||||
|
# Start the server
|
||||||
|
server.run()
|
||||||
|
```
|
||||||
|
|
||||||
|
### Custom Worker
|
||||||
|
|
||||||
|
```python
|
||||||
|
from arcade_serve import BaseWorker, WorkerComponent
|
||||||
|
|
||||||
|
class MyWorker(BaseWorker):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__()
|
||||||
|
self.add_component(MyCustomComponent())
|
||||||
|
|
||||||
|
async def handle_request(self, request):
|
||||||
|
# Custom request handling
|
||||||
|
return await super().handle_request(request)
|
||||||
|
```
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT License - see LICENSE file for details.
|
||||||
|
|
@ -4,18 +4,18 @@ import time
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from typing import Any, Callable, ClassVar
|
from typing import Any, Callable, ClassVar
|
||||||
|
|
||||||
from opentelemetry import trace
|
from arcade_core.catalog import ToolCatalog, Toolkit
|
||||||
from opentelemetry.metrics import Meter
|
from arcade_core.executor import ToolExecutor
|
||||||
|
from arcade_core.schema import (
|
||||||
from arcade.core.catalog import ToolCatalog, Toolkit
|
|
||||||
from arcade.core.executor import ToolExecutor
|
|
||||||
from arcade.core.schema import (
|
|
||||||
ToolCallRequest,
|
ToolCallRequest,
|
||||||
ToolCallResponse,
|
ToolCallResponse,
|
||||||
ToolDefinition,
|
ToolDefinition,
|
||||||
)
|
)
|
||||||
from arcade.worker.core.common import Router, Worker
|
from opentelemetry import trace
|
||||||
from arcade.worker.core.components import (
|
from opentelemetry.metrics import Meter
|
||||||
|
|
||||||
|
from arcade_serve.core.common import Router, Worker
|
||||||
|
from arcade_serve.core.components import (
|
||||||
CallToolComponent,
|
CallToolComponent,
|
||||||
CatalogComponent,
|
CatalogComponent,
|
||||||
HealthCheckComponent,
|
HealthCheckComponent,
|
||||||
|
|
@ -1,10 +1,9 @@
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from typing import Any, Callable
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from arcade_core.schema import ToolCallRequest, ToolCallResponse, ToolDefinition
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from arcade.core.schema import ToolCallRequest, ToolCallResponse, ToolDefinition
|
|
||||||
|
|
||||||
CatalogResponse = list[ToolDefinition]
|
CatalogResponse = list[ToolDefinition]
|
||||||
HealthCheckResponse = dict[str, str]
|
HealthCheckResponse = dict[str, str]
|
||||||
JSONResponse = dict[str, Any]
|
JSONResponse = dict[str, Any]
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
from opentelemetry import trace
|
from opentelemetry import trace
|
||||||
|
|
||||||
from arcade.worker.core.common import (
|
from arcade_serve.core.common import (
|
||||||
CatalogResponse,
|
CatalogResponse,
|
||||||
HealthCheckResponse,
|
HealthCheckResponse,
|
||||||
RequestData,
|
RequestData,
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
from fastapi import HTTPException
|
from fastapi import HTTPException
|
||||||
from fastapi.security import HTTPAuthorizationCredentials
|
from fastapi.security import HTTPAuthorizationCredentials
|
||||||
|
|
||||||
from arcade.worker.core.auth import validate_engine_token
|
from arcade_serve.core.auth import validate_engine_token
|
||||||
|
|
||||||
|
|
||||||
# Dependency function to validate JWT
|
# Dependency function to validate JWT
|
||||||
|
|
@ -5,13 +5,13 @@ from fastapi import Depends, FastAPI, Request
|
||||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||||
from opentelemetry.metrics import Meter
|
from opentelemetry.metrics import Meter
|
||||||
|
|
||||||
from arcade.worker.core.base import (
|
from arcade_serve.core.base import (
|
||||||
BaseWorker,
|
BaseWorker,
|
||||||
Router,
|
Router,
|
||||||
)
|
)
|
||||||
from arcade.worker.core.common import RequestData, ResponseData, WorkerComponent
|
from arcade_serve.core.common import RequestData, ResponseData, WorkerComponent
|
||||||
from arcade.worker.fastapi.auth import validate_engine_request
|
from arcade_serve.fastapi.auth import validate_engine_request
|
||||||
from arcade.worker.utils import is_async_callable
|
from arcade_serve.utils import is_async_callable
|
||||||
|
|
||||||
|
|
||||||
class FastAPIWorker(BaseWorker):
|
class FastAPIWorker(BaseWorker):
|
||||||
|
|
@ -2,6 +2,6 @@
|
||||||
MCP (Model Context Protocol) support for Arcade workers.
|
MCP (Model Context Protocol) support for Arcade workers.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from arcade.worker.mcp.stdio import StdioServer
|
from arcade_serve.mcp.stdio import StdioServer
|
||||||
|
|
||||||
__all__ = ["StdioServer"]
|
__all__ = ["StdioServer"]
|
||||||
|
|
@ -3,7 +3,7 @@ import logging
|
||||||
from enum import Enum
|
from enum import Enum
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from arcade.core.catalog import MaterializedTool
|
from arcade_core.catalog import MaterializedTool
|
||||||
|
|
||||||
# Type aliases for MCP types
|
# Type aliases for MCP types
|
||||||
MCPTool = dict[str, Any]
|
MCPTool = dict[str, Any]
|
||||||
|
|
@ -4,7 +4,7 @@ import sys
|
||||||
import time
|
import time
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from arcade.worker.mcp.types import (
|
from arcade_serve.mcp.types import (
|
||||||
JSONRPCError,
|
JSONRPCError,
|
||||||
JSONRPCRequest,
|
JSONRPCRequest,
|
||||||
JSONRPCResponse,
|
JSONRPCResponse,
|
||||||
|
|
@ -3,7 +3,7 @@ import json
|
||||||
import logging
|
import logging
|
||||||
from typing import Any, Callable, TypeVar
|
from typing import Any, Callable, TypeVar
|
||||||
|
|
||||||
from arcade.worker.mcp.types import InitializeRequest, JSONRPCRequest, MCPMessage
|
from arcade_serve.mcp.types import InitializeRequest, JSONRPCRequest, MCPMessage
|
||||||
|
|
||||||
logger = logging.getLogger("arcade.mcp")
|
logger = logging.getLogger("arcade.mcp")
|
||||||
|
|
||||||
|
|
@ -5,17 +5,17 @@ import uuid
|
||||||
from enum import Enum
|
from enum import Enum
|
||||||
from typing import Any, Callable, Union
|
from typing import Any, Callable, Union
|
||||||
|
|
||||||
|
from arcade_core.catalog import MaterializedTool, ToolCatalog
|
||||||
|
from arcade_core.executor import ToolExecutor
|
||||||
|
from arcade_core.schema import ToolAuthorizationContext, ToolContext
|
||||||
from arcadepy import ArcadeError, AsyncArcade
|
from arcadepy import ArcadeError, AsyncArcade
|
||||||
from arcadepy.types.auth_authorize_params import AuthRequirement, AuthRequirementOauth2
|
from arcadepy.types.auth_authorize_params import AuthRequirement, AuthRequirementOauth2
|
||||||
from arcadepy.types.shared import AuthorizationResponse
|
from arcadepy.types.shared import AuthorizationResponse
|
||||||
|
|
||||||
from arcade.core.catalog import MaterializedTool, ToolCatalog
|
from arcade_serve.mcp.convert import convert_to_mcp_content, create_mcp_tool
|
||||||
from arcade.core.executor import ToolExecutor
|
from arcade_serve.mcp.logging import create_mcp_logging_middleware
|
||||||
from arcade.core.schema import ToolAuthorizationContext, ToolContext
|
from arcade_serve.mcp.message_processor import MCPMessageProcessor, create_message_processor
|
||||||
from arcade.worker.mcp.convert import convert_to_mcp_content, create_mcp_tool
|
from arcade_serve.mcp.types import (
|
||||||
from arcade.worker.mcp.logging import create_mcp_logging_middleware
|
|
||||||
from arcade.worker.mcp.message_processor import MCPMessageProcessor, create_message_processor
|
|
||||||
from arcade.worker.mcp.types import (
|
|
||||||
CallToolRequest,
|
CallToolRequest,
|
||||||
CallToolResponse,
|
CallToolResponse,
|
||||||
CallToolResult,
|
CallToolResult,
|
||||||
|
|
@ -158,7 +158,7 @@ class MCPServer:
|
||||||
A user ID string
|
A user ID string
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
from arcade.core.config import config
|
from arcade_core.config import config
|
||||||
|
|
||||||
# Prefer config.user.email if available
|
# Prefer config.user.email if available
|
||||||
if config.user and config.user.email:
|
if config.user and config.user.email:
|
||||||
|
|
@ -10,7 +10,7 @@ from typing import TYPE_CHECKING, Any, TypeVar
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
from arcade.worker.mcp.server import MCPServer
|
from arcade_serve.mcp.server import MCPServer
|
||||||
|
|
||||||
logger = logging.getLogger("arcade.mcp")
|
logger = logging.getLogger("arcade.mcp")
|
||||||
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue