Commit graph

5 commits

Author SHA1 Message Date
Eric Gustin
d5067af023
Bump toolkit versions (#194) 2025-01-07 13:32:36 -08:00
Eric Gustin
feb83c95ca
Pin poetry to 1.8.5 (#193)
# PR Description
Poetry released v2 with many breaking changes a couple days ago. The
`install-poetry` action that our workflows use default to that v2
version, so many of our workflows are failing. This PR forces that
action to use poetry version 1.8.5 and also uses 1.8.5 for toolkits

A ticket to migrate to 2.0.0 has been filed for future work
2025-01-07 13:21:55 -08:00
Eric Gustin
ab889f9f1d
Lint all toolkits (#183)
# PR Description
* Adds/updates the following files to all toolkits:
    - `.pre-commit-config.yaml`
    - `.ruff.toml`
    - `LICENSE`
    - `Makefile`
    - `pyproject.toml`
* Lint all toolkits such that they pass `make check` and `make test` (a
total doozy). This includes adding some unit tests and evals.
* Github workflow for testing toolkits before merge into main (courtesy
of @sdreyer)
* Added a QOL improvement for tool developers for when they need to get
the context's auth token.
* Minor updates to `arcade new` template.
2024-12-20 09:49:45 -08:00
Eric Gustin
7c228a59d5
Update Evals SDK (#175)
# PR Description
This PR renames `ExpectedToolCall` to `NamedExpectedToolCall` and then
creates a new dataclass called `ExpectedToolCall`. `ExpectedToolCall`
can be passed to the `EvalSuite.add_case` and `EvalSuite.extend_case`
methods.

1. Enhance `EvalSuite.add_case` and `EvalSuite.extend_case` by accepting
a list of `ExpectedToolCall` as their `expected_tool_calls` input
parameter. This helps create a scaffolding for developers. Previously,
the expected type was `list[tuple[Callable, dict[str, Any]]]`, which is
still valid for backward compatibility.
```python
# Before (still valid for backward compatibility)
expected_tool_calls=[
    (
        adjust_playback_position,
        {
            "absolute_position_ms": 10000,
        },
    )
]
        

# After
expected_tool_calls=[
    ExpectedToolCall(
        func=adjust_playback_position,
        args={"absolute_position_ms": 10000},
    )
]
```
2. Removed any references to arcade.core in toolkits directory.
3. Some linting for import organization.
2024-12-19 10:29:13 -08:00
Eric Gustin
8b46e4f7f9
Add Code Sandbox Tools (#114)
# PR Description
This PR creates a new toolkit called CodeSandbox. This toolkit has two
tools:
1. `RunCode`: Creates an E2B sandbox and runs the provided code in that
sandbox. Returns the execution logs, result, and errors. Supports
Python, JavaScript, R, Java, and Bash code.
2. `CreateStaticMatplotlibChart`: Creates a sandbox, runs the provided
python code that uses matplotlib, and returns the base64 encoded image
of the chart along with any logs or errors.
- I recommend not using `tool_choice="generate"` since the return object
contains a base64 image can be a lot of tokens that will not provide
much value to a generate's response.
    
    
    
Example of creating a pie chart:
```python
import base64
import json
import os

from openai import OpenAI


def call_tool_with_openai(client: OpenAI) -> dict:
    response = client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "There are 17 red apples, 4 green apples, and 10 yellow apples. Create a pie chart for this data.",
            },
        ],
        model="gpt-4o-mini",
        user="you@example.com",
        tools=["CodeSandbox.CreateStaticMatplotlibChart"],
        tool_choice="execute",
    )

    return response


arcade_api_key = os.environ.get("ARCADE_API_KEY")
cloud_host = "http://localhost:9099/v1"

openai_client = OpenAI(
    api_key=arcade_api_key,
    base_url=cloud_host,
)

chat_result = call_tool_with_openai(openai_client)
tool_call_id = chat_result.choices[0].message.tool_calls[0].id

content = json.loads(chat_result.choices[0].message.content)
base64_image = content[tool_call_id]["value"]["base64_image"]

image_data = base64.b64decode(base64_image)
with open("output_image.png", "wb") as image_file:
    image_file.write(image_data)

```
2024-11-15 13:29:52 -08:00