52d82a2789
This PR improves the Docker build process by shifting from building the
project within the Docker image to using pre-built wheels. The main
changes are:
1. **Updated Makefile:**
- **`VERSION` Variable:** Set to `0.1.0.dev0` to reflect the new default
development version.
- **`docker` Target:**
- Added steps to build the Arcade and toolkit wheels before building the
Docker image.
- Exports the required extras (`fastapi`, `evals`) to a
`requirements.txt` file.
- **`full-dist` Target:**
- Builds distributions for the main project and all toolkits.
- Copies all the built wheels to a centralized `./dist` directory.
- **`clean-dist` Target:**
- Cleans build artifacts from `./dist`, `arcade/dist`, and
`toolkits/*/dist` directories.
2. **Modified Dockerfile:**
- **Copy Pre-built Wheels:** Adjusted to copy wheels and the
`requirements.txt` from the `./dist` directory into the Docker image.
- **Installation Process:**
- Installs the Arcade wheel with the necessary extras.
- Installs toolkits from the copied wheel files, eliminating the need to
build them inside the Docker image.
- **Simplification:** Removed unnecessary commands, such as installing
build tools and copying the entire codebase, to streamline the
Dockerfile.
3. **Toolkits `pyproject.toml` Updates:**
- Changed the `arcade-ai` dependency version from `^0.1.0` to `0.1.*` in
all toolkit `pyproject.toml` files to ensure compatibility with the new
versioning scheme.
4. **Docker Makefile Adjustments:**
- Set the `VERSION` variable to `0.1.0.dev0` to align with the main
Makefile.
- Ensures consistent versioning across Docker-related build processes.
**Benefits:**
- **Efficiency:** Building wheels outside the Docker context reduces the
Docker image build time and resource consumption. overall docker image
size reduced by **1Gb**!!!
- **Reliability:** Using pre-built wheels ensures consistency across
different environments and simplifies dependency management.
- **Maintainability:** The Dockerfile and Makefiles are cleaner and more
straightforward, making them easier to understand and maintain.
**Notes:**
- Developers should run `make docker` to build and run the Docker
container using the new process.
- Ensure that any CI/CD pipelines are updated to accommodate these
changes in the build process. @sdreyer
114 lines
3.7 KiB
Makefile
114 lines
3.7 KiB
Makefile
VERSION ?= "0.1.0.dev0"
|
|
|
|
.PHONY: install
|
|
install: ## Install the poetry environment and install the pre-commit hooks
|
|
@echo "🚀 Creating virtual environment using pyenv and poetry"
|
|
@cd arcade && poetry install --all-extras
|
|
@cd arcade && poetry run pre-commit install
|
|
|
|
.PHONY: check
|
|
check: ## Run code quality tools.
|
|
@echo "🚀 Checking Poetry lock file consistency with 'pyproject.toml': Running poetry check --lock"
|
|
@cd arcade && poetry check --lock
|
|
@echo "🚀 Linting code: Running pre-commit"
|
|
@cd arcade && poetry run pre-commit run -a
|
|
@echo "🚀 Static type checking: Running mypy"
|
|
@cd arcade && poetry run mypy $(git ls-files '*.py')
|
|
|
|
.PHONY: test
|
|
test: ## Test the code with pytest
|
|
@echo "🚀 Testing code: Running pytest"
|
|
@cd arcade && poetry run pytest -v --cov --cov-config=pyproject.toml --cov-report=xml
|
|
|
|
.PHONY: test-toolkits
|
|
test-toolkits: ## Iterate over all toolkits and run pytest on each one
|
|
@echo "🚀 Testing code in toolkits: Running pytest"
|
|
@for dir in toolkits/*/ ; do \
|
|
(cd $$dir && poetry run pytest -v --cov --cov-config=pyproject.toml --cov-report=xml || exit 1); \
|
|
done
|
|
|
|
.PHONY: set-version
|
|
set-version: ## Set the version in the pyproject.toml file
|
|
@echo "🚀 Setting version in pyproject.toml"
|
|
@cd arcade && poetry version $(VERSION)
|
|
|
|
.PHONY: build
|
|
build: clean-build ## Build wheel file using poetry
|
|
@echo "🚀 Creating wheel file"
|
|
@cd arcade && poetry build
|
|
|
|
.PHONY: clean-build
|
|
clean-build: ## clean build artifacts
|
|
@cd arcade && rm -rf dist
|
|
|
|
.PHONY: publish
|
|
publish: ## publish a release to pypi.
|
|
@echo "🚀 Publishing: Dry run."
|
|
@cd arcade && poetry config pypi-token.pypi $(PYPI_TOKEN)
|
|
@cd arcade && poetry publish --dry-run
|
|
@echo "🚀 Publishing."
|
|
@cd arcade && poetry publish
|
|
|
|
.PHONY: build-and-publish
|
|
build-and-publish: build publish ## Build and publish.
|
|
|
|
.PHONY: docker
|
|
docker: ## Build and run the Docker container
|
|
@echo "🚀 Building arcade and toolkit wheels..."
|
|
@make full-dist
|
|
@echo "Writing extras [fastapi, evals] to requirements.txt"
|
|
@cd arcade && poetry export --extras "fastapi evals" --output ../dist/requirements.txt
|
|
@echo "🚀 Building Docker image"
|
|
@cd docker && make docker-build
|
|
@cd docker && make docker-run
|
|
|
|
.PHONY: full-dist
|
|
full-dist: clean-dist ## Build all projects and copy wheels to ./dist
|
|
@echo " Building a full distribution with toolkits"
|
|
|
|
@echo "Setting version to $(VERSION)"
|
|
@make set-version
|
|
|
|
@echo "🛠️ Building all projects and copying wheels to ./dist"
|
|
@mkdir -p dist/toolkits
|
|
|
|
# Build the main arcade project
|
|
@echo "🛠️ Building arcade project wheel..."
|
|
@cd arcade && poetry build
|
|
|
|
# Copy the main arcade project wheel to the dist directory
|
|
@cp arcade/dist/*.whl dist/
|
|
|
|
@echo "🛠️ Building all projects and copying wheels to ./dist"
|
|
# Build and copy wheels for each toolkit
|
|
@for toolkit_dir in toolkits/*; do \
|
|
if [ -d "$$toolkit_dir" ]; then \
|
|
toolkit_name=$$(basename "$$toolkit_dir"); \
|
|
echo "Building $$toolkit_name project..."; \
|
|
cd "$$toolkit_dir" && poetry build; \
|
|
cp dist/*.whl ../../dist/toolkits; \
|
|
cd -; \
|
|
fi; \
|
|
done
|
|
|
|
@echo "✅ All projects built and wheels copied to arcade/dist"
|
|
|
|
.PHONY: clean-dist
|
|
clean-dist: ## Clean all built distributions
|
|
@echo "🗑️ Cleaning dist directory"
|
|
@rm -rf dist
|
|
@echo "🗑️ Cleaning arcade/dist directory"
|
|
@rm -rf arcade/dist
|
|
@echo "🗑️ Cleaning toolkits/*/dist directory"
|
|
@for toolkit_dir in toolkits/*; do \
|
|
if [ -d "$$toolkit_dir" ]; then \
|
|
rm -rf "$$toolkit_dir"/dist; \
|
|
fi; \
|
|
done
|
|
|
|
.PHONY: help
|
|
help:
|
|
@echo "🛠️ Arcade AI Dev Commands:\n"
|
|
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
|
|
|
|
.DEFAULT_GOAL := help
|