arcade-mcp/Makefile
Sam Partee b6b4cd0a4c
🏗️ Restructure: Multi-Package Architecture + uv Migration (#412)
### Overview
Major restructuring from monolithic `arcade-ai` package to modular
library architecture with standardized uv-based dependency management.

![arcade-ai Monorepo
(2)](https://github.com/user-attachments/assets/25f102b0-bb87-4a04-9701-d227d05664b1)

### New Package Structure
- **`arcade-tdk`** - Lightweight toolkit development kit (core
decorators, auth)
- **`arcade-core`** - Core execution engine and catalog functionality  
- **`arcade-serve`** - FastAPI/MCP server components
- **`arcade-ai`** - Meta package that includes CLI functionality.
Optionally include evals via the `evals` extra. Optionally include all
packages via the `all` extra.

### Key Benefits
- **Lighter Dependencies**: Toolkits now depend only on `arcade-tdk` (~2
deps) vs full `arcade-ai` (~30+ deps)
- **Faster Builds**: uv provides 10-100x faster dependency resolution
and installation
- **Better Modularity**: Clear separation of concerns, consumers import
only what they need
- **Standard Tooling**: Eliminates custom poetry scripts, uses standard
Python packaging

### Migration Impact
- All 20 toolkits converted from poetry → uv with `arcade-tdk`
dependencies plus `arcade-ai[evals]` and `arcade-serve` dev
dependencies. When developing locally, devs should install toolkits via
`make install-local`.
- Modern Python 3.10+ type hints throughout
- Standardized build system with hatchling backend
- Enhanced Makefile with robust toolkit management commands
- Removed `arcade dev` CLI command
- Reduce the number of files created by `arcade new` and add an option
to not generate a tests and evals folder.

This foundation enables faster development cycles and cleaner dependency
chains for the growing toolkit ecosystem.

### Todo After this PR is merged
- [ ] Post-merge workflow(s) (release & publish containers, etc)
- [ ] Release order plan. @EricGustin suggests releasing in the
following order:
    1. `arcade-core` version 0.1.0
    2. `arcade-serve` version 0.1.0 and `arcade-tdk` version 0.1.0
    3. `arcade-ai` version 2.0.0
4. Patch release for all toolkits (all changes in toolkits are internal
refactors)
- [ ] [Update docs](https://github.com/ArcadeAI/docs/pull/318)

---------

Co-authored-by: Eric Gustin <eric@arcade.dev>
Co-authored-by: Eric Gustin <34000337+EricGustin@users.noreply.github.com>
2025-06-11 16:48:17 -07:00

277 lines
9.3 KiB
Makefile

CLI_VERSION ?= "2.0.0"
TDK_VERSION ?= "2.0.0"
SERVE_VERSION ?= "2.0.0"
CORE_VERSION ?= "2.0.0"
.PHONY: install
install: ## Install the uv environment and all packages with dependencies
@echo "🚀 Creating virtual environment and installing all packages using uv workspace"
@uv sync --active --dev --extra all
@uv run pre-commit install
@echo "✅ All packages and dependencies installed via uv workspace"
.PHONY: install-toolkits
install-toolkits: ## Install dependencies for all toolkits
@echo "🚀 Installing dependencies for all toolkits"
@failed=0; \
successful=0; \
for dir in toolkits/*/ ; do \
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
echo "📦 Installing dependencies for $$dir"; \
if (cd $$dir && uv pip install -e ".[dev]"); then \
successful=$$((successful + 1)); \
else \
echo "❌ Failed to install dependencies for $$dir"; \
failed=$$((failed + 1)); \
fi; \
else \
echo "⚠️ Skipping $$dir (no pyproject.toml found)"; \
fi; \
done; \
echo ""; \
echo "📊 Installation Summary:"; \
echo " ✅ Successful: $$successful toolkits"; \
echo " ❌ Failed: $$failed toolkits"; \
if [ $$failed -gt 0 ]; then \
echo ""; \
echo "⚠️ Some toolkit installations failed. Check the output above for details."; \
exit 1; \
else \
echo ""; \
echo "🎉 All toolkit dependencies installed successfully!"; \
fi
.PHONY: check
check: ## Run code quality tools.
@echo "🚀 Linting code: Running pre-commit"
@uv run pre-commit run -a
@echo "🚀 Static type checking: Running mypy on libs"
@for lib in libs/arcade*/ ; do \
echo "🔍 Type checking $$lib"; \
(cd $$lib && uv run mypy . || true); \
done
.PHONY: check-libs
check-libs: ## Run code quality tools for each lib package
@echo "🚀 Running checks on each lib package"
@for lib in libs/arcade*/ ; do \
echo "🛠️ Checking lib $$lib"; \
(cd $$lib && uv run pre-commit run -a || true); \
(cd $$lib && uv run mypy . || true); \
done
.PHONY: check-toolkits
check-toolkits: ## Run code quality tools for each toolkit that has a Makefile
@echo "🚀 Running 'make check' in each toolkit with a Makefile"
@for dir in toolkits/*/ ; do \
if [ -f "$$dir/Makefile" ]; then \
echo "🛠️ Checking toolkit $$dir"; \
(cd "$$dir" && uv run --active pre-commit run -a && uv run --active mypy --config-file=pyproject.toml); \
else \
echo "🛠️ Skipping toolkit $$dir (no Makefile found)"; \
fi; \
done
.PHONY: test
test: ## Test the code with pytest
@echo "🚀 Testing libs: Running pytest"
@uv run pytest -W ignore -v --cov=libs/tests --cov-config=pyproject.toml --cov-report=xml
.PHONY: test-libs
test-libs: ## Test each lib package individually
@echo "🚀 Testing each lib package"
@for lib in libs/arcade*/ ; do \
echo "🧪 Testing $$lib"; \
(cd $$lib && uv run pytest -W ignore -v || true); \
done
.PHONY: test-toolkits
test-toolkits: ## Iterate over all toolkits and run pytest on each one
@echo "🚀 Testing code in toolkits: Running pytest"
@for dir in toolkits/*/ ; do \
toolkit_name=$$(basename "$$dir"); \
echo "🧪 Testing $$toolkit_name toolkit"; \
(cd $$dir && uv run --active pytest -W ignore -v --cov=arcade_$$toolkit_name --cov-report=xml || exit 1); \
done
.PHONY: coverage
coverage: ## Generate coverage report
@echo "coverage report"
@uv run coverage report
@echo "Generating coverage report"
@uv run coverage html
.PHONY: set-version
set-version: ## Set the version in all lib pyproject.toml files
@echo "🚀 Setting versions in all lib packages"
@echo "Setting arcade-ai version to $(CLI_VERSION)"
@sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(CLI_VERSION)/' pyproject.toml && rm pyproject.toml.bak
@echo "Setting libs/arcade-tdk version to $(TDK_VERSION)"
@cd libs/arcade-tdk && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(TDK_VERSION)/' pyproject.toml && rm pyproject.toml.bak
@echo "Setting libs/arcade-serve version to $(SERVE_VERSION)"
@cd libs/arcade-serve && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(SERVE_VERSION)/' pyproject.toml && rm pyproject.toml.bak
@echo "Setting libs/arcade-core version to $(CORE_VERSION)"
@cd libs/arcade-core && sed -i.bak '/^\[project\]/,/^\[/ s/^version = .*/version = $(CORE_VERSION)/' pyproject.toml && rm pyproject.toml.bak
.PHONY: unset-version
unset-version: ## Reset version to 0.1.0 in all lib pyproject.toml files
@echo "🚀 Resetting version to 0.1.0 in all lib packages"
@for lib in libs/arcade*/ ; do \
if [ -f "$$lib/pyproject.toml" ]; then \
echo "Resetting version in $$lib"; \
(cd $$lib && sed -i.bak 's/version = "[^"]*"/version = "0.1.0"/' pyproject.toml && rm pyproject.toml.bak); \
fi; \
done
.PHONY: build
build: clean-build ## Build wheel files using uv
@echo "🚀 Creating wheel files for all lib packages"
@for lib in libs/arcade*/ ; do \
if [ -f "$$lib/pyproject.toml" ]; then \
echo "🛠️ Building $$lib"; \
(cd $$lib && uv build); \
fi; \
done
.PHONY: build-toolkits
build-toolkits: ## Build wheel files for all toolkits
@echo "🚀 Creating wheel files for all toolkits"
@failed=0; \
successful=0; \
for dir in toolkits/*/ ; do \
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
toolkit_name=$$(basename "$$dir"); \
echo "🛠️ Building toolkit $$toolkit_name"; \
if (cd $$dir && uv build); then \
successful=$$((successful + 1)); \
else \
echo "❌ Failed to build toolkit $$toolkit_name"; \
failed=$$((failed + 1)); \
fi; \
else \
echo "⚠️ Skipping $$dir (no pyproject.toml found)"; \
fi; \
done; \
echo ""; \
echo "📊 Build Summary:"; \
echo " ✅ Successful: $$successful toolkits"; \
echo " ❌ Failed: $$failed toolkits"; \
if [ $$failed -gt 0 ]; then \
echo ""; \
echo "⚠️ Some toolkit builds failed. Check the output above for details."; \
exit 1; \
else \
echo ""; \
echo "🎉 All toolkit wheels built successfully!"; \
fi
.PHONY: clean-build
clean-build: ## clean build artifacts
@echo "🗑️ Cleaning build artifacts"
@for lib in libs/arcade*/ ; do \
(cd $$lib && rm -rf dist); \
done
.PHONY: publish
publish: ## publish a release to pypi.
@echo "🚀 Publishing all lib packages to PyPI"
@for lib in libs/arcade*/ ; do \
if [ -f "$$lib/pyproject.toml" ]; then \
echo "📦 Publishing $$lib"; \
(cd $$lib && uv publish --token $(PYPI_TOKEN) || true); \
fi; \
done
.PHONY: build-and-publish
build-and-publish: build publish ## Build and publish.
.PHONY: docker
docker: ## Build and run the Docker container
@echo "🚀 Building lib packages and toolkit wheels..."
@make full-dist
@echo "🚀 Building Docker image"
@cd docker && make docker-build
@cd docker && make docker-run
.PHONY: docker-base
docker-base: ## Build and run the Docker container
@echo "🚀 Building lib packages and toolkit wheels..."
@make full-dist
@echo "🚀 Building Docker image"
@cd docker && INSTALL_TOOLKITS=false make docker-build
@cd docker && INSTALL_TOOLKITS=false make docker-run
.PHONY: publish-ecr
publish-ecr: ## Publish to the ECR
# Publish the base image - <ECR_ENDPOINT>/arcadeai/worker-base
@cd docker && INSTALL_TOOLKITS=false make publish-ecr
# Publish the image with toolkits - <ECR_ENDPOINT>/arcadeai/worker
@cd docker && INSTALL_TOOLKITS=true make publish-ecr
.PHONY: publish-ghcr
publish-ghcr: ## Publish to the GHCR
# Publish the base image - ghcr.io/arcadeai/worker-base
@cd docker && INSTALL_TOOLKITS=false make publish-ghcr
# Publish the image with toolkits - ghcr.io/arcadeai/worker
@cd docker && INSTALL_TOOLKITS=true make publish-ghcr
.PHONY: full-dist
full-dist: clean-dist ## Build all projects and copy wheels to ./dist
@echo "🛠️ Building a full distribution with lib packages and toolkits"
@echo "Setting version to $(CLI_VERSION)"
@make set-version
@echo "🛠️ Building all lib packages and copying wheels to ./dist"
@mkdir -p dist
# Build all lib packages in dependency order
@for lib in arcade-core arcade-tdk arcade-serve ; do \
echo "🛠️ Building libs/$$lib wheel..."; \
(cd libs/$$lib && uv build); \
cp libs/$$lib/dist/*.whl dist/; \
done
@echo "🛠️ Building all toolkit packages and copying wheels to ./dist"
@for dir in toolkits/*/ ; do \
if [ -d "$$dir" ] && [ -f "$$dir/pyproject.toml" ]; then \
toolkit_name=$$(basename "$$dir"); \
echo "🛠️ Building toolkit $$toolkit_name wheel..."; \
(cd $$dir && uv build); \
cp $$dir/dist/*.whl dist/; \
fi; \
done
@echo "Reset version to default (0.1.0)"
@make unset-version
.PHONY: clean-dist
clean-dist: ## Clean all built distributions
@echo "🗑️ Cleaning dist directory"
@rm -rf dist
@echo "🗑️ Cleaning libs/*/dist directories"
@for lib in libs/arcade*/ ; do \
rm -rf "$$lib"/dist; \
done
@echo "🗑️ Cleaning toolkits/*/dist directory"
@for toolkit_dir in toolkits/*; do \
if [ -d "$$toolkit_dir" ]; then \
rm -rf "$$toolkit_dir"/dist; \
fi; \
done
.PHONY: setup
setup: install ## Complete development setup (same as install)
.PHONY: lint
lint: check ## Alias for check command
.PHONY: clean
clean: clean-build clean-dist ## Clean all build and distribution artifacts
.PHONY: help
help:
@echo "🛠️ Arcade Dev Commands:\n"
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.DEFAULT_GOAL := help