# PR Description
This PR introduces a new environment variable `ARCADE_DISABLED_TOOLS`.
Tools that are added to this env var are not added to the worker's
`ToolCatalog`. In effect, they are disabled for the worker.
## How to use the `ARCADE_DISABLED_TOOLS` environment variable
* Each tool is separated by a comma.
* For each tool, specify the toolkit name in camel case and the tool
name in camel case.
* Do not include versions. (This is a simple implementation. We can add
disabling specific versions in the future if needed)
* Separate the toolkit name and the tool name with your environment's
tool name separator. By default, the tool name separator is `.`, but you
can override this with the `ARCADE_TOOL_NAME_SEPARATOR` environment
variable.
Correct: `export
ARCADE_DISABLED_TOOLS="Math.Add,Spotify.GetAvailableDevices,Math.Sqrt"`
Incorrect: `export
ARCADE_DISABLED_TOOLS="Math.Add@0.1.0,Spotify.get_available_devices,Sqrt`
# PR Description
Well known providers (Google, X, Dropbox, etc.) can optionally have an
`id` in addition to their hardcoded `provider_id`. For non well known
providers, they must provide an `id`, and the `provider_id` is hardcoded
as `None`.
```python
OAuth2() # INVALID
OAuth2(provider_id="abc") # INVALID
OAuth2(id="abc") # VALID
OAuth2(provider_id="abc", id="def") # INVALID
```
```python
Google() # VALID
Google(provider_id="abc") # INVALID
Google(id="abc") # VALID
Google(provider_id="abc", id="def") # INVALID
```
---------
Co-authored-by: Wils Dawson <wils@arcade-ai.com>
Fixes a circular import issue where `arcade.sdk -> arcade.core` but also
`arcade.core -> arcade.sdk`. My mistake!
Moved some of the shared classes down into `core`, and re-exported them
to `sdk` to keep the expected interface for devs.
This PR ensures that `arcade.core` does not show up anywhere in "user
space". This is crucial for helping developers understand what objects
are safe to use, and helps maintain a good developer experience.
Specific changes:
- `ToolAuthorizationContext` and `ToolContext` are now visible via
`arcade.sdk`
- `ToolCatalog` is now visible via `arcade.sdk`
- `Toolkit` is now visible via `arcade.sdk`
- `config` is now visible via `arcade.sdk.config`
This PR adds four new tools to the Google ToolKit
* `create_event`
* `list_events`
* `update_event`
* `delete_event`
I also improved an error log when tools are being registered by the
actor.
This PR also sneaks in an eval for gmail
Here is a sample conversation that shows the tools and their
capabilities and limitiations:
This PR includes several improvements to the Arcade client and adds
LangGraph examples:
1. Enhanced error handling in the Arcade client:
- Improved HTTP error handling in `BaseArcadeClient`
- Simplified request methods in `SyncArcadeClient` and
`AsyncArcadeClient`
2. Updated `ToolResource` class:
- Changed base path from `/v1/tool` to `/v1/tools`
- Added `tool_version` parameter to `authorize` method
3. Improved Toolkit discovery:
- Updated `find_all_arcade_toolkits` to search only in the current
Python interpreter's site-packages
5. Added LangGraph examples:
- New `langgraph_auth.py` example demonstrating Gmail authentication
- New `langgraph_with_tool_exec.py` example showing tool execution
within a LangGraph
6. Minor updates:
- Changed default `BASE_URL` to `https://api.arcade.com/`
- Updated import error message for eval dependencies
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
In this PR:
- Handle and require fully-qualified tool names `Toolkit.ToolName` in
the actor
Also, unrelated changes/fixes:
- Cleaned up the logic around actor secrets and `$ARCADE_ACTOR_SECRET`
- Removes experimental Flask actor for now
Note: Must be merged along with
https://github.com/ArcadeAI/Engine/pull/87
Fixes 2 issues that were causing CI to fail:
- Loading `config` in `eval.py` breaks because no API key can be found
in CI
- Python 3.11+ changed `Union` to `UnionType`
1. New Eval SDK (`arcade/sdk/eval.py`):
- Introduces `EvalSuite`, `EvalCase`, and `EvalRubric` classes for
structured evaluation.
- Implements various Critic classes (Binary, Numeric, Similarity) for
flexible scoring.
- Adds a `tool_eval` decorator for easy integration with existing tools.
2. CLI Integration (`arcade/cli/main.py` and `arcade/cli/utils.py`):
- Adds an `evals` command to run evaluation suites from the CLI.
- Implements result display functionality for evaluation outcomes.
3. Toolkit Updates:
- Adds evaluation scripts for Gmail
([toolkits/gmail/evals/eval_gmail_tools.py](file:///Users/spartee/Dropbox/Arcade/platform/Team/arcade-ai/toolkits/gmail/evals/eval_gmail_tools.py#1%2C1-1%2C1))
and Slack
([toolkits/slack/evals/eval_slack_messaging.py](file:///Users/spartee/Dropbox/Arcade/platform/Team/arcade-ai/toolkits/slack/evals/eval_slack_messaging.py#1%2C1-1%2C1))
toolkits.
- Demonstrates practical usage of the Eval SDK with real-world
scenarios.
4. Miscellaneous:
- Updates `arcade/cli/new.py` to optionally generate an `evals`
directory for new toolkits.
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
In this PR:
- Rename `scope` to `scopes` so it is more understandable by humans
- DRY up provider structs, it was starting to get silly with so many
providers that just have 1 property called `scopes`
Must go along with this Engine PR:
https://github.com/ArcadeAI/Engine/pull/79
Two new commands to the Arcade CLI: `arcade run` and `arcade chat`.
These commands enhance the usability of the Arcade CLI by integrating
language model capabilities for running tools and engaging in chat
sessions. Users can now leverage LLMs directly from the command line
A few quick fixes while testing the gmail tool with the real Engine:
- Renamed `tool.requirements.auth` to `authorization` -- Engine already
used `authorization`
- Fixed the credentials initializer in the gmail tool
- Adds initial `ToolContext` to tool invocations
- This unlocks the ability to call authenticated tools (e.g. Gmail),
which works in this branch against Nate's dev engine
This PR makes a few sweeping changes to the actor, cli, and overall
structure of the project.
- CLI commands skeleton
- ``arcade run``, ``arcade show``, and ``arcade new``
- Working package mangement solution (``arcade_`` packages)
- Actor approach for using frameworks other than FastAPI
- Client for calling Engine within ``arcade/core``
- beginning of the config interface.
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
2024-07-23 16:26:54 -07:00
Renamed from arcade/arcade/tool/catalog.py (Browse further)
