This PR improves the Docker build process by shifting from building the
project within the Docker image to using pre-built wheels. The main
changes are:
1. **Updated Makefile:**
- **`VERSION` Variable:** Set to `0.1.0.dev0` to reflect the new default
development version.
- **`docker` Target:**
- Added steps to build the Arcade and toolkit wheels before building the
Docker image.
- Exports the required extras (`fastapi`, `evals`) to a
`requirements.txt` file.
- **`full-dist` Target:**
- Builds distributions for the main project and all toolkits.
- Copies all the built wheels to a centralized `./dist` directory.
- **`clean-dist` Target:**
- Cleans build artifacts from `./dist`, `arcade/dist`, and
`toolkits/*/dist` directories.
2. **Modified Dockerfile:**
- **Copy Pre-built Wheels:** Adjusted to copy wheels and the
`requirements.txt` from the `./dist` directory into the Docker image.
- **Installation Process:**
- Installs the Arcade wheel with the necessary extras.
- Installs toolkits from the copied wheel files, eliminating the need to
build them inside the Docker image.
- **Simplification:** Removed unnecessary commands, such as installing
build tools and copying the entire codebase, to streamline the
Dockerfile.
3. **Toolkits `pyproject.toml` Updates:**
- Changed the `arcade-ai` dependency version from `^0.1.0` to `0.1.*` in
all toolkit `pyproject.toml` files to ensure compatibility with the new
versioning scheme.
4. **Docker Makefile Adjustments:**
- Set the `VERSION` variable to `0.1.0.dev0` to align with the main
Makefile.
- Ensures consistent versioning across Docker-related build processes.
**Benefits:**
- **Efficiency:** Building wheels outside the Docker context reduces the
Docker image build time and resource consumption. overall docker image
size reduced by **1Gb**!!!
- **Reliability:** Using pre-built wheels ensures consistency across
different environments and simplifies dependency management.
- **Maintainability:** The Dockerfile and Makefiles are cleaner and more
straightforward, making them easier to understand and maintain.
**Notes:**
- Developers should run `make docker` to build and run the Docker
container using the new process.
- Ensure that any CI/CD pipelines are updated to accommodate these
changes in the build process. @sdreyer
On the last few PRs I have noticed two problems:
1. `ruff format` fails even though it seems OK on our local machines
(sometimes, not always)
2. Nate's and Sam's machines kept flip-flopping a specific piece of
formatting back and forth, indicating a subtle difference of config
hiding somewhere
3. This was reproducible by running `ruff format` in the terminal,
followed by `make check`. The former would edit files, and then `make
check` would edit them back!
This PR addresses both issues, and further standardizes our editor &
linter configs to be super stable.
Specifically:
1. The main fix for the above, the pre-commit hook was pinned to a super
old version of ruff.
This resulted in subtle differences in behavior between our machines,
and on CI.
2. Moved ruff settings from `pyproject.toml` to `.ruff.toml`
pyproject files in subdirectories (e.g. `toolkits/**`) were overriding
the main pyproject file and erasing the custom ruff config we set at the
root. This meant that our ruff config was applied to `arcade` but not to
any of the other packages.
By moving the config to `.ruff.toml` at the root, all projects will
inherit the same ruff linting & formatting config.
4. Un-ignored the `.vscode/` directory so that we can share
vscode/cursor workspace settings.
This is valuable for standardizing settings like the default formatter
(ruff) and default test framework (pytest).
However, it's important that going forward we _only_ commit things here
that should apply across all of our machines.
5. To avoid any conflict between prettier and ruff, prettier now
explicitly ignores *.py files
6. Finally, `ruff format` and `make check` agree. A number of files are
newly auto-formatted.
# Address Alpha Release Tweaks and Bugs
This PR addresses several issues and tweaks identified during the alpha
release:
- **Ensure `~/.arcade` directory exists before writing the config file**
In `arcade/cli/authn.py`, added code to create the `~/.arcade` directory
if it doesn't exist. This prevents errors when writing the configuration
file during the login process.
- **Fix retry logic in process management**
In `arcade/cli/launcher.py`, corrected an off-by-one error in the retry
logic within the `_manage_processes` function. This ensures that the
process management behaves as expected when retries are exhausted.
- **Allow passing environment variables to the engine process**
(technically this option isn't exposed yet)
Updated the `start_servers`, `_manage_processes`, and `_start_process`
functions in `arcade/cli/launcher.py` to accept an `engine_env`
parameter. This allows custom environment variables to be set for the
engine process. Also, set `GIN_MODE` to `"release"` by default.
- **Handle cases with no critics in evaluations**
Modified the `EvalCase` class in `arcade/sdk/eval/eval.py` to handle
scenarios where no critics are provided. This avoids potential errors
during the evaluation process when critics are absent. Should add a test
for this.
- **Adjust dependencies in `pyproject.toml`**
- Moved `uvicorn` to be an optional dependency and included it in the
`fastapi` extra.
- Removed unnecessary development dependencies (`mkdocs`,
`mkdocs-material`, `mkdocstrings`).
- Ensured that `uvicorn` is updated to version `^0.30.0`.
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
This PR includes several improvements to the Arcade client and adds
LangGraph examples:
1. Enhanced error handling in the Arcade client:
- Improved HTTP error handling in `BaseArcadeClient`
- Simplified request methods in `SyncArcadeClient` and
`AsyncArcadeClient`
2. Updated `ToolResource` class:
- Changed base path from `/v1/tool` to `/v1/tools`
- Added `tool_version` parameter to `authorize` method
3. Improved Toolkit discovery:
- Updated `find_all_arcade_toolkits` to search only in the current
Python interpreter's site-packages
5. Added LangGraph examples:
- New `langgraph_auth.py` example demonstrating Gmail authentication
- New `langgraph_with_tool_exec.py` example showing tool execution
within a LangGraph
6. Minor updates:
- Changed default `BASE_URL` to `https://api.arcade.com/`
- Updated import error message for eval dependencies
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
1. New Eval SDK (`arcade/sdk/eval.py`):
- Introduces `EvalSuite`, `EvalCase`, and `EvalRubric` classes for
structured evaluation.
- Implements various Critic classes (Binary, Numeric, Similarity) for
flexible scoring.
- Adds a `tool_eval` decorator for easy integration with existing tools.
2. CLI Integration (`arcade/cli/main.py` and `arcade/cli/utils.py`):
- Adds an `evals` command to run evaluation suites from the CLI.
- Implements result display functionality for evaluation outcomes.
3. Toolkit Updates:
- Adds evaluation scripts for Gmail
([toolkits/gmail/evals/eval_gmail_tools.py](file:///Users/spartee/Dropbox/Arcade/platform/Team/arcade-ai/toolkits/gmail/evals/eval_gmail_tools.py#1%2C1-1%2C1))
and Slack
([toolkits/slack/evals/eval_slack_messaging.py](file:///Users/spartee/Dropbox/Arcade/platform/Team/arcade-ai/toolkits/slack/evals/eval_slack_messaging.py#1%2C1-1%2C1))
toolkits.
- Demonstrates practical usage of the Eval SDK with real-world
scenarios.
4. Miscellaneous:
- Updates `arcade/cli/new.py` to optionally generate an `evals`
directory for new toolkits.
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
Two new commands to the Arcade CLI: `arcade run` and `arcade chat`.
These commands enhance the usability of the Arcade CLI by integrating
language model capabilities for running tools and engaging in chat
sessions. Users can now leverage LLMs directly from the command line
Added
- `arcade dev` - serves a simple fastapi actor
- `arcade config` - show/edit/change config in `~/.arcade`
- `arcade chat` - chat with LLM without toolcalls
Changed:
- `arcade show`, `arcade run` - can now use all installed toolkits
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
This PR makes a few sweeping changes to the actor, cli, and overall
structure of the project.
- CLI commands skeleton
- ``arcade run``, ``arcade show``, and ``arcade new``
- Working package mangement solution (``arcade_`` packages)
- Actor approach for using frameworks other than FastAPI
- Client for calling Engine within ``arcade/core``
- beginning of the config interface.
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
Start of the CI/CD workflow for the python library and actor.
Covered:
- multiple python versions with Tox (3.10+) and pytest
- Docs eventually with mkdocs (commented out for now)
- Linting and formatting with Ruff and prettier
- Codecov
- include the mypy checks when the mypy errors are resolved.
MyPy compliance for the whole codebase
- systematic way of executing tools (`executor.py`)
- support for using pydantic models in tool inputs and outputs
- mypy compliance (most of the changes)
- removal of unused code (from previous iterations)
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
