Cleanup examples and README (#8)
As the title states. Also added in a prompt/execute capabiltiy into the CLI for the ``arcade run`` command Committed by @nbarbettini Original work by @Spartee
This commit is contained in:
Sam Partee
GitHub
parent
8964111023
commit
90f1146968
7 changed files with 192 additions and 45 deletions
93
README.md
93
README.md
|
|
@ -1,39 +1,76 @@
|
|||
[](https://img.shields.io/github/v/release/spartee/arcade-ai)
|
||||
[](https://github.com/spartee/arcade-ai/actions/workflows/main.yml?query=branch%3Amain)
|
||||
[](https://codecov.io/gh/spartee/arcade-ai)
|
||||
[](https://img.shields.io/github/commit-activity/m/spartee/arcade-ai)
|
||||
[](https://img.shields.io/github/license/spartee/arcade-ai)
|
||||
[](https://img.shields.io/github/v/release/arcadeai/arcade-ai)
|
||||
[](https://github.com/arcadeai/arcade-ai/actions/workflows/main.yml?query=branch%3Amain)
|
||||
[](https://codecov.io/gh/arcadeai/arcade-ai)
|
||||
[](https://img.shields.io/github/commit-activity/m/arcadeai/arcade-ai)
|
||||
[](https://img.shields.io/github/license/arcadeai/arcade-ai)
|
||||
|
||||
# Arcade-AI
|
||||
|
||||
ToolServe is a framework specifically designed to manage and orchestrate Language Learning Models (LLMs) or "tools" with high efficiency. It distinctively separates the tools from the orchestration framework to improve dependency management, packaging, and execution.
|
||||
# Arcade AI
|
||||
|
||||
The server enhances data management capabilities, enabling LLMs to interact with structured data efficiently without the need to return the entire dataset to the context window.
|
||||
Arcade AI is the developer platform for building tools designed to be used with language models. With Arcade, developers can create, deploy, and easily integrate new tools with language models to enhance their capabilities.
|
||||
|
||||
This functionality is especially beneficial for agents tasked with performing actions or responding to queries based on extensive structured datasets.
|
||||
## `arcade-ai`
|
||||
|
||||
## Components
|
||||
|
||||
### 1. Command Line Interface (CLI)
|
||||
|
||||
The CLI component, located at `toolserve/cli/main.py`, offers commands to package, serve, and inspect LLM "tools". It utilizes the Typer library to manage command-line arguments and options.
|
||||
|
||||
### 2. Server
|
||||
|
||||
The server component, which manages the storage of artifacts, data, and logs generated by the tools, is implemented using FastAPI and can be found at `toolserve/server/main.py`. The server configuration includes routes, middleware, and database connections.
|
||||
|
||||
### 3. SDK
|
||||
|
||||
Located at `toolserve/sdk`, the SDK streamlines the development of tools by providing decorators and helper functions that abstract routine tasks, allowing developers to concentrate on the logic of the tool rather than on repetitive code.
|
||||
|
||||
### 4. Builtins
|
||||
|
||||
Built-in tools for common tasks such as SQL queries are available at `toolserve/builtin/default`. These tools are ready to use and require no additional setup.
|
||||
The `arcade-ai` package contains:
|
||||
- `arcade` CLI
|
||||
- `arcade.sdk` Tool SDK
|
||||
- `arcade.actor` serving tools with FastAPI, Flask, or Django
|
||||
|
||||
## Installation
|
||||
|
||||
To install the ToolServe package, execute the following command:
|
||||
To install the Arcade AI package, execute the following command:
|
||||
|
||||
```bash
|
||||
pip install toolserve
|
||||
pip install arcade-ai
|
||||
```
|
||||
|
||||
or install from source:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/arcadeai/arcade-ai.git
|
||||
cd arcade-ai
|
||||
pip install poetry
|
||||
poetry install
|
||||
```
|
||||
|
||||
## First steps
|
||||
|
||||
Follow these steps if you've cloned the repo and installed the package from source:
|
||||
|
||||
```bash
|
||||
cd examples/websearch
|
||||
poetry install
|
||||
|
||||
arcade show arcade_websearch
|
||||
```
|
||||
This will show an output that looks like
|
||||
|
||||
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┓
|
||||
┃ Name ┃ Description ┃ Toolkit ┃ Version ┃
|
||||
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━┩
|
||||
│ SearchGoogle │ Search Google using SerpAPI and return organic search results. │ websearch │ 0.1.0 │
|
||||
└──────────────┴────────────────────────────────────────────────────────────────┴───────────┴─────────┘
|
||||
|
||||
|
||||
Predict the parameters with a model and run the tool with the predicted parameters. Arcade adds the `execute` choice to the tool, which allows you to run the tool with the predicted parameters in a single request.
|
||||
|
||||
```bash
|
||||
> arcade run arcade_websearch "who is Sam Partee?" --choice "execute"
|
||||
Running tool: SearchGoogle with params: {'query': 'Sam Partee'}
|
||||
|
||||
[{"position": 1, "title": "Sam Partee (@SamPartee) / X", "link": "https://twitter.com/sampartee", "redirect_link":
|
||||
"https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://twitter.com/sampartee&ved=2ahUKEwjBwKiz3b6HAxV1VTABHXL8BZQQFnoECAYQAQ",
|
||||
"displayed_link": "1.5K+ followers", "thumbnail":
|
||||
.....
|
||||
.. (truncated)
|
||||
```
|
||||
|
||||
Arcade also adds the `predict` choice to the tool, which allows you to predict the parameters with a model.
|
||||
|
||||
```bash
|
||||
> arcade run arcade_websearch "who is Sam Partee?" --choice "predict" # also the default
|
||||
Running tool: SearchGoogle with params: {'query': 'Sam Partee'}
|
||||
|
||||
Sam Partee is a CTO, Co-founder of Arcade AI and former Machine Learning Engineer at companies like RedisInc and HPE_Cray. They have
|
||||
expertise in AI/ML, vector search, Python, HPC, and are a sports fan.
|
||||
```
|
||||
|
|
|
|||
|
|
@ -90,13 +90,13 @@ def show(
|
|||
|
||||
|
||||
@cli.command(help="Run a tool using an LLM to predict the arguments")
|
||||
def run(
|
||||
def run( # noqa: C901
|
||||
toolkit: str = typer.Argument(..., help="The toolkit to add to model calls"),
|
||||
prompt: str = typer.Argument(..., help="The prompt to use for context"),
|
||||
model: str = typer.Option("gpt-3.5-turbo", "-m", help="The model to use for prediction."),
|
||||
tool: str = typer.Option(None, "-t", "--tool", help="The name of the tool to run."),
|
||||
choice: str = typer.Option(
|
||||
"required", "-c", "--choice", help="The value of the tool choice argument"
|
||||
"prompt", "-c", "--choice", help="The value of the tool choice argument"
|
||||
),
|
||||
stream: bool = typer.Option(True, "-s", "--stream", help="Stream the tool output."),
|
||||
actor: Optional[str] = typer.Option(
|
||||
|
|
@ -137,7 +137,9 @@ def run(
|
|||
|
||||
# TODO put in the engine url from config
|
||||
client = EngineClient()
|
||||
calls = client.call_tool(tools, tool_choice=choice, prompt=prompt, model=model)
|
||||
# TODO better way of doing this
|
||||
tool_choice = "required" if choice in ["prompt", "execute"] else choice
|
||||
calls = client.call_tool(tools, tool_choice=tool_choice, prompt=prompt, model=model)
|
||||
|
||||
messages = [
|
||||
{"role": "user", "content": prompt},
|
||||
|
|
@ -147,6 +149,7 @@ def run(
|
|||
called_tool = catalog[tool_name]
|
||||
console.print(f"Running tool: {tool_name} with params: {parameters}", style="bold blue")
|
||||
|
||||
# TODO async.gather instead of loop.
|
||||
output = asyncio.run(
|
||||
ToolExecutor.run(
|
||||
called_tool.tool,
|
||||
|
|
@ -160,20 +163,23 @@ def run(
|
|||
if output.data:
|
||||
console.print(output.data.result, style="bold red")
|
||||
else:
|
||||
# TODO: Add the tool results to the response in a safer way
|
||||
messages += [
|
||||
{
|
||||
"role": "assistant",
|
||||
# TODO: escape the output and ensure serialization works
|
||||
"content": f"Results of Tool {tool_name}: {output.data.result!s}", # type: ignore[union-attr]
|
||||
},
|
||||
]
|
||||
if stream:
|
||||
stream_response = client.stream_complete(model=model, messages=messages)
|
||||
display_streamed_markdown(stream_response)
|
||||
else:
|
||||
response = client.complete(model=model, messages=messages)
|
||||
console.print(response.choices[0].message.content, style="bold green")
|
||||
if choice == "prompt":
|
||||
# TODO: Add the tool results to the response in a safer way
|
||||
messages += [
|
||||
{
|
||||
"role": "assistant",
|
||||
# TODO: escape the output and ensure serialization works
|
||||
"content": f"Results of Tool {tool_name}: {output.data.result!s}", # type: ignore[union-attr]
|
||||
},
|
||||
]
|
||||
if stream:
|
||||
stream_response = client.stream_complete(model=model, messages=messages)
|
||||
display_streamed_markdown(stream_response)
|
||||
else:
|
||||
response = client.complete(model=model, messages=messages)
|
||||
console.print(response.choices[0].message.content, style="bold green")
|
||||
elif choice == "execute":
|
||||
console.print(output.data.result, style="green") # type: ignore[union-attr]
|
||||
|
||||
except RuntimeError as e:
|
||||
error_message = f"❌ Failed to run tool{': '+ escape(str(e)) if str(e) else ''}"
|
||||
|
|
|
|||
0
examples/math/arcade_arithmetic/__init__.py
Normal file
0
examples/math/arcade_arithmetic/__init__.py
Normal file
43
examples/math/arcade_arithmetic/main.py
Normal file
43
examples/math/arcade_arithmetic/main.py
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
from arcade.actor.fastapi.actor import FastAPIActor
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from openai import AsyncOpenAI
|
||||
from arcade_example_nate.tools import arithmetic
|
||||
|
||||
|
||||
client = AsyncOpenAI(base_url="http://localhost:6901")
|
||||
|
||||
app = FastAPI()
|
||||
|
||||
actor = FastAPIActor(app)
|
||||
actor.register_tool(arithmetic.add)
|
||||
actor.register_tool(arithmetic.multiply)
|
||||
actor.register_tool(arithmetic.divide)
|
||||
actor.register_tool(arithmetic.sqrt)
|
||||
|
||||
|
||||
class ChatRequest(BaseModel):
|
||||
message: str
|
||||
|
||||
|
||||
@app.post("/chat")
|
||||
async def chat(request: ChatRequest):
|
||||
try:
|
||||
raw_response = await client.chat.completions.with_raw_response.create(
|
||||
messages=[
|
||||
{"role": "system", "content": "You are a helpful assistant."},
|
||||
{"role": "user", "content": request.message},
|
||||
],
|
||||
model="gpt-4o-mini",
|
||||
max_tokens=150,
|
||||
tool_choice="execute",
|
||||
)
|
||||
chat_completion = raw_response.parse()
|
||||
|
||||
return {
|
||||
"response": chat_completion.choices[0].message.content.strip(),
|
||||
"tool_call_count": raw_response.headers["arcade-tool-calls"],
|
||||
"tool_call_duration_ms": raw_response.headers["arcade-total-tool-duration"],
|
||||
}
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
0
examples/math/arcade_arithmetic/tools/__init__.py
Normal file
0
examples/math/arcade_arithmetic/tools/__init__.py
Normal file
44
examples/math/arcade_arithmetic/tools/arithmetic.py
Normal file
44
examples/math/arcade_arithmetic/tools/arithmetic.py
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
import math
|
||||
from typing import Annotated
|
||||
|
||||
from arcade.sdk.tool import tool
|
||||
|
||||
|
||||
@tool
|
||||
def add(
|
||||
a: Annotated[int, "The first number"], b: Annotated[int, "The second number"]
|
||||
) -> Annotated[int, "The sum of the two numbers"]:
|
||||
"""
|
||||
Add two numbers together
|
||||
"""
|
||||
return a + b
|
||||
|
||||
|
||||
@tool
|
||||
def multiply(
|
||||
a: Annotated[int, "The first number"], b: Annotated[int, "The second number"]
|
||||
) -> Annotated[int, "The product of the two numbers"]:
|
||||
"""
|
||||
Multiply two numbers together
|
||||
"""
|
||||
return a * b
|
||||
|
||||
|
||||
@tool
|
||||
def divide(
|
||||
a: Annotated[int, "The first number"], b: Annotated[int, "The second number"]
|
||||
) -> Annotated[float, "The quotient of the two numbers"]:
|
||||
"""
|
||||
Divide two numbers
|
||||
"""
|
||||
return a / b
|
||||
|
||||
|
||||
@tool
|
||||
def sqrt(
|
||||
a: Annotated[int, "The number to square root"],
|
||||
) -> Annotated[float, "The square root of the number"]:
|
||||
"""
|
||||
Get the square root of a number
|
||||
"""
|
||||
return math.sqrt(a)
|
||||
17
examples/math/pyproject.toml
Normal file
17
examples/math/pyproject.toml
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
[tool.poetry]
|
||||
name = "arcade_arithmetic"
|
||||
version = "0.1.0"
|
||||
description = "Nate's testing package for Arcade"
|
||||
authors = ["Nate <nate@arcade-ai.com>"]
|
||||
|
||||
|
||||
[tool.poetry.dependencies]
|
||||
python = "^3.10"
|
||||
fastapi = "^0.110.3"
|
||||
|
||||
[tool.poetry.dev-dependencies]
|
||||
pytest = "^7.4"
|
||||
|
||||
[build-system]
|
||||
requires = ["poetry-core>=1.0.0"]
|
||||
build-backend = "poetry.core.masonry.api"
|
||||
Loading…
Reference in a new issue