# PR Description
This PR adds 6 new tools inside the new `arcade_web` toolkit. None of
these tools require auth. They do, however, require the
`FIRECRAWL_API_KEY` API Key to be set.
The new tools implement the [Firecrawl](https://www.firecrawl.dev/) APIs
`/scrape (POST)`, `/crawl (POST)`, `/crawl/{id} (GET)`, `/crawl/{id}
(DELETE)`, and `/map (POST)`.
The six tools are:
* `Web.ScrapeUrl`:
- In the future I would like this tool to support actions (clicking,
scrolling, screenshotting, etc) and extract (specify what you want to
scrape) parameters. Firecrawl supports both of these parameters.
* `Web.CrawlWebsite`:
- If `async_crawl` is true, then the tool just returns the id of the
crawl job, which you can retrieve later with the `Web.GetCrawlData`
tool. If `async_crawl` is false, then the entire contents of the crawl
are returned.
* `Web.GetCrawlStatus`
- Works for in progress or recently finished crawl jobs (Firecrawl's
limitation)
* `Web.GetCrawlData`
- Works for in progress or recently finished crawl jobs (Firecrawl's
limitation)
* `Web.CancelCrawl`
- You can cancel an in progress async crawl job
* `Web.MapWebsite`
- This endpoint is in alpha, but it can give you all of the links of an
entire website, or optionally, you can specify in natural language what
type of links you want to map by using the `search` parameter. For
example "only map webpages that are about AI"
**New Tools Added**
- `docs.py`: Provides tools for Google Docs functionalities, including
creating documents and inserting text.
- `drive.py`: Introduces tools for Google Drive operations, such as
listing documents.
This PR also focuses on simplifying the error handling logic in the Google
toolkit, specifically within the Calendar and Gmail tools. The primary
change involves removing redundant `try-except` blocks that were
catching `HttpError` and general exceptions, and re-raising them as
`ToolExecutionError`. By removing these blocks, we allow exceptions to
propagate naturally, and be handled by the ``ToolExecutor``
---------
Co-authored-by: Eric Gustin <eric@arcade-ai.com>
## Problem
I found a bug with `Github.CountStargazers` where a stargazer count of
`0` was interpreted as a null result. In other words, 0 wasn't passed
back to the Engine correctly.
Separately, the tool function was also not authorized correctly.
## Fix
- Don't use a falsy comparison when evaluating `result` inside the
`ToolOutputFactory`
- Add unit tests for `ToolOutputFactory` to give us confidence in the
business logic
- Added `ToolContext` to pass in the authorization token correctly.
Before
```
User (nate@arcade-ai.com):
how many stars does the ArcadeAI/Docs repo have on github?
Assistant (gpt-4o):
I successfully checked the repository, but unfortunately, I cannot provide the number of stars for the ArcadeAI/Docs repository. Please try checking directly on GitHub for the most accurate information.
Called tool 'Github_CountStargazers'
Parameters:{"owner":"ArcadeAI","name":"Docs"}
'Github_CountStargazers' tool returned:Github.CountStargazers called successfully
```
After
```
User (nate@arcade-ai.com):
how many stars does the ArcadeAI/Docs repo have on github?
Assistant (gpt-4o):
The ArcadeAI/Docs repository on GitHub has 0 stars.
Called tool 'Github_CountStargazers'
Parameters:{"owner":"ArcadeAI","name":"Docs"}
'Github_CountStargazers' tool returned:0
```
authors = ["Arcade AI <dev@arcade-ai.com>"]
```
vs
```
authors = ["Arcade AI <dev@arcade-ai.com"]
```
There is also now a ``make`` command for ``make install-toolkits``
### Adds the following tools to the Github Toolkit:
1. CreateIssueComment
2. SetStarred
3. CountStargazers
4. ListOrgRepositories
5. GetRepository
6. ListRepositoryActivities
7. ListReviewCommentsInARepository
8. ListPullRequests
9. GetPullRequest
10. UpdatePullRequest
11. ListPullRequestCommits
12. CreateReplyForReviewComment
13. ListReviewCommentsOnPullRequest
14. CreateReviewComment
Adds evals for all of these tools and unit tests.
---------
Co-authored-by: Sam Partee <sam@arcade-ai.com>
This PR adds four new tools to the Google ToolKit
* `create_event`
* `list_events`
* `update_event`
* `delete_event`
I also improved an error log when tools are being registered by the
actor.
This PR also sneaks in an eval for gmail
Here is a sample conversation that shows the tools and their
capabilities and limitiations:
This PR improves the Docker build process by shifting from building the
project within the Docker image to using pre-built wheels. The main
changes are:
1. **Updated Makefile:**
- **`VERSION` Variable:** Set to `0.1.0.dev0` to reflect the new default
development version.
- **`docker` Target:**
- Added steps to build the Arcade and toolkit wheels before building the
Docker image.
- Exports the required extras (`fastapi`, `evals`) to a
`requirements.txt` file.
- **`full-dist` Target:**
- Builds distributions for the main project and all toolkits.
- Copies all the built wheels to a centralized `./dist` directory.
- **`clean-dist` Target:**
- Cleans build artifacts from `./dist`, `arcade/dist`, and
`toolkits/*/dist` directories.
2. **Modified Dockerfile:**
- **Copy Pre-built Wheels:** Adjusted to copy wheels and the
`requirements.txt` from the `./dist` directory into the Docker image.
- **Installation Process:**
- Installs the Arcade wheel with the necessary extras.
- Installs toolkits from the copied wheel files, eliminating the need to
build them inside the Docker image.
- **Simplification:** Removed unnecessary commands, such as installing
build tools and copying the entire codebase, to streamline the
Dockerfile.
3. **Toolkits `pyproject.toml` Updates:**
- Changed the `arcade-ai` dependency version from `^0.1.0` to `0.1.*` in
all toolkit `pyproject.toml` files to ensure compatibility with the new
versioning scheme.
4. **Docker Makefile Adjustments:**
- Set the `VERSION` variable to `0.1.0.dev0` to align with the main
Makefile.
- Ensures consistent versioning across Docker-related build processes.
**Benefits:**
- **Efficiency:** Building wheels outside the Docker context reduces the
Docker image build time and resource consumption. overall docker image
size reduced by **1Gb**!!!
- **Reliability:** Using pre-built wheels ensures consistency across
different environments and simplifies dependency management.
- **Maintainability:** The Dockerfile and Makefiles are cleaner and more
straightforward, making them easier to understand and maintain.
**Notes:**
- Developers should run `make docker` to build and run the Docker
container using the new process.
- Ensure that any CI/CD pipelines are updated to accommodate these
changes in the build process. @sdreyer
Included toolkits as part of the linting process.
Cleaned up any tools that needed to be updated because of this.
This portion of the PR description was added via arcade chat!
On the last few PRs I have noticed two problems:
1. `ruff format` fails even though it seems OK on our local machines
(sometimes, not always)
2. Nate's and Sam's machines kept flip-flopping a specific piece of
formatting back and forth, indicating a subtle difference of config
hiding somewhere
3. This was reproducible by running `ruff format` in the terminal,
followed by `make check`. The former would edit files, and then `make
check` would edit them back!
This PR addresses both issues, and further standardizes our editor &
linter configs to be super stable.
Specifically:
1. The main fix for the above, the pre-commit hook was pinned to a super
old version of ruff.
This resulted in subtle differences in behavior between our machines,
and on CI.
2. Moved ruff settings from `pyproject.toml` to `.ruff.toml`
pyproject files in subdirectories (e.g. `toolkits/**`) were overriding
the main pyproject file and erasing the custom ruff config we set at the
root. This meant that our ruff config was applied to `arcade` but not to
any of the other packages.
By moving the config to `.ruff.toml` at the root, all projects will
inherit the same ruff linting & formatting config.
4. Un-ignored the `.vscode/` directory so that we can share
vscode/cursor workspace settings.
This is valuable for standardizing settings like the default formatter
(ruff) and default test framework (pytest).
However, it's important that going forward we _only_ commit things here
that should apply across all of our machines.
5. To avoid any conflict between prettier and ruff, prettier now
explicitly ignores *.py files
6. Finally, `ruff format` and `make check` agree. A number of files are
newly auto-formatted.
This PR includes several improvements to the Arcade client and adds
LangGraph examples:
1. Enhanced error handling in the Arcade client:
- Improved HTTP error handling in `BaseArcadeClient`
- Simplified request methods in `SyncArcadeClient` and
`AsyncArcadeClient`
2. Updated `ToolResource` class:
- Changed base path from `/v1/tool` to `/v1/tools`
- Added `tool_version` parameter to `authorize` method
3. Improved Toolkit discovery:
- Updated `find_all_arcade_toolkits` to search only in the current
Python interpreter's site-packages
5. Added LangGraph examples:
- New `langgraph_auth.py` example demonstrating Gmail authentication
- New `langgraph_with_tool_exec.py` example showing tool execution
within a LangGraph
6. Minor updates:
- Changed default `BASE_URL` to `https://api.arcade.com/`
- Updated import error message for eval dependencies
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
In this PR:
- Handle and require fully-qualified tool names `Toolkit.ToolName` in
the actor
Also, unrelated changes/fixes:
- Cleaned up the logic around actor secrets and `$ARCADE_ACTOR_SECRET`
- Removes experimental Flask actor for now
Note: Must be merged along with
https://github.com/ArcadeAI/Engine/pull/87
* Renamed `arcade_arithmetic` to `arcade_math`
* Deleted `arcade_github` toolkit for the next release. This will be
reintroduced later.
* Added 5 tools to `arcade_x` toolkit
- post_tweet
- delete_tweet_by_id
- search_recent_tweets_by_username
- search_recent_tweets_by_keywords
- lookup_single_user_by_username
# PR Description
## Summary
Changes include renaming the `arcade_gmail` toolkit to `arcade_google`,
adding unit tests for Google toolkit, add new tools to the Google
toolkit.
## Changes
### Makefile
- Added a new `make test-toolkits` target to iterate over all toolkits
and run pytest on each one.
### Added new tools for the google toolkit
1. `send_email`
This tool sends an email using the Gmail API.
2. `write_draft_email`
This tool creates a draft email using the Gmail API.
3. `update_draft_email`
This tool updates an existing draft email using the Gmail API.
4. `send_draft_email`
This tool sends a draft email using the Gmail API.
5. `delete_draft_email`
This tool deletes a draft email using the Gmail API.
6. `list_draft_emails`
This tool retrieves a list of draft emails using the Gmail API.
7. `list_emails_by_header`
This tool searches for emails by a specific header using the Gmail API.
- `sender`: The sender's email address to search for.
- `limit`: The maximum number of emails to retrieve.
8. `list_emails`
This tool retrieves a list of emails using the Gmail API.
9. `trash_email`
This tool moves an email to the trash using the Gmail API.
1. New Eval SDK (`arcade/sdk/eval.py`):
- Introduces `EvalSuite`, `EvalCase`, and `EvalRubric` classes for
structured evaluation.
- Implements various Critic classes (Binary, Numeric, Similarity) for
flexible scoring.
- Adds a `tool_eval` decorator for easy integration with existing tools.
2. CLI Integration (`arcade/cli/main.py` and `arcade/cli/utils.py`):
- Adds an `evals` command to run evaluation suites from the CLI.
- Implements result display functionality for evaluation outcomes.
3. Toolkit Updates:
- Adds evaluation scripts for Gmail
([toolkits/gmail/evals/eval_gmail_tools.py](file:///Users/spartee/Dropbox/Arcade/platform/Team/arcade-ai/toolkits/gmail/evals/eval_gmail_tools.py#1%2C1-1%2C1))
and Slack
([toolkits/slack/evals/eval_slack_messaging.py](file:///Users/spartee/Dropbox/Arcade/platform/Team/arcade-ai/toolkits/slack/evals/eval_slack_messaging.py#1%2C1-1%2C1))
toolkits.
- Demonstrates practical usage of the Eval SDK with real-world
scenarios.
4. Miscellaneous:
- Updates `arcade/cli/new.py` to optionally generate an `evals`
directory for new toolkits.
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
In this PR:
- Rename `scope` to `scopes` so it is more understandable by humans
- DRY up provider structs, it was starting to get silly with so many
providers that just have 1 property called `scopes`
Must go along with this Engine PR:
https://github.com/ArcadeAI/Engine/pull/79
* Add new tool to the arithmetic toolkit for summation of a range.
* Add ability to attach debugger to cli. Use `.vscode/launch.json`'s
"Debug arcade dev" to do so.
* Fix issue in cli's main that used the incorrect url.
Note - This Engine PR must go first:
https://github.com/ArcadeAI/Engine/pull/65
In this PR:
- Add `client.tool.authorize` to authorize a tool by name by @Spartee
- Refactored client.auth methods to always pass around scopes (as needed
by the above Engine PR) by @nbarbettini
- Reduced the scopes needed in the Slack toolkit, which was blocked by
this until now! @nbarbettini
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
Working now:
- `arcade login` works against the Cloud
- `arcade logout` deletes your local credentials
---------
Co-authored-by: Sam Partee <sam@arcade-ai.com>
#### Summary
This PR introduces key updates to the Arcade AI codebase, focusing on
improving the CLI tool, refining the Docker build process, and enhancing
documentation within the Gmail toolkit.
#### Key Changes:
1. **Docker Build**:
- Added the `--no-cache` option to the `docker-build` target in the
Makefile to ensure fresh builds, preventing issues related to cached
layers.
2. **CLI Tool**:
- Introduced a new optional `prompt` parameter in the `chat` command,
allowing users to customize the system role's prompt. A default prompt
is now provided if none is specified.
3. **Gmail Toolkit**:
- Updated the `write_draft` function with improved docstrings for
clearer guidance and maintainability.
#### Impact:
- **Developer Workflow**: Improved Docker reliability and enhanced CLI
flexibility.
- **User Experience**: More customizable interactions in the CLI and
better documentation in the Gmail toolkit.
Please review and provide feedback.
Two new commands to the Arcade CLI: `arcade run` and `arcade chat`.
These commands enhance the usability of the Arcade CLI by integrating
language model capabilities for running tools and engaging in chat
sessions. Users can now leverage LLMs directly from the command line
A few quick fixes while testing the gmail tool with the real Engine:
- Renamed `tool.requirements.auth` to `authorization` -- Engine already
used `authorization`
- Fixed the credentials initializer in the gmail tool
Added
- `arcade dev` - serves a simple fastapi actor
- `arcade config` - show/edit/change config in `~/.arcade`
- `arcade chat` - chat with LLM without toolcalls
Changed:
- `arcade show`, `arcade run` - can now use all installed toolkits
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
