when testing earlier, 0.1.0 was used for a failed pypi upload. I'm
assuming this is why `This filename has already been used, use a
different version` is occuring
1. The Arcade Worker doesn't like it when the package name is different
than the directory name. This PR renames the directory from
`arcade_notion` to `arcade_notion_toolkit`
2. `Notion` is not a well-known provider name in `arcade-ai==1.0.5`,
I've updated the dep to represent this.
1. Add the following tools:
* Google Finance
- get_stock_summary
- get_stock_historical_data
* Google Flights
- search_roundtrip_flights
- search_one_way_flights
* Google Hotels
- search_hotels
2. Add some common helper functions for serpAPI tools.
Migrated all interfaces to get and return strings.
Added tests and evals for all functions (except the random generation)
Math functions are now organized into different math categories
---------
Co-authored-by: Nate Barbettini <nate@arcade-ai.com>
This tool will be useful in scenarios akin to RAG, where someone wants
to ask questions or request the production of a summary, for instance,
about a bunch of documents related to a particular topic. Currently, to
fulfill such requests, the LLM needs to first `list_documents`, then
`get_document_by_id` for each document.
We also implement a utility functions to return documents in Markdown
and HTML, since the Drive API JSON is verbose and would waste too many
tokens unnecessarily.
Limitations: the Markdown/HTML utilities do not handle table of contents
(which I think aren't really useful here), headers, footers, or
footnotes.
---
This PR deprecates `list_documents` and implements `search_documents`,
apart from `search_and_retrieve_documents`). This configuration makes it
easier for LLMs to understand when to call each tool.
Both tools had their interfaces refactored to remove Google API-specific
arguments that were confusing LLMs sometimes, such as "corpora" and
"support_all_drives". It now accepts arguments that better relate to
expected user requests.
---------
Co-authored-by: Eric Gustin <eric@arcade.dev>
~~Note: Don't merge until the correct secrets have been added to Arcade
Cloud.~~
Ready to merge, the feature is already on its way to prod.
---------
Co-authored-by: Eric Gustin <eric@arcade.dev>
Break down `search_contacts` into `search_contacts_by_name` and
`search_contacts_by_email`. The search_contacts' `query` argument was
not clear enough for LLMs.
Improved gmail toolkit. Added support for threading in draft replies,
multipart email parsing, and label management. Fixed the DateRange
parameter issue in list_emails_by_headers. Added logging and removed
print statements. Created custom exceptions for each specific google
toolkit.
-----
Summary of changes by @byrro:
- Fixed minor bug related to the `date_range` argument of
`list_emails_by_header`
- A few utility functions (`build_email_message`,
`build_reply_recipients`, `build_reply_body`) to centralize logic and
remove repeated code from email-sending tools
- New `reply_to_email` tool (apart from `write_draft_reply_email`,
implemented by Alex) to keep the toolkit consistent
- Evals and unit tests
- Handling of reply-to (only sender) and reply-to-all recipients
- Removed some unnecessary debug messages, which Alex had added to
replace print statements
- Removed HTML handling implemented by Alex in `write_draft_reply_email`
> I think we should either support HTML across all applicable tools or
not at all; I decided to remove it and leave this feature for a future
PR.
---------
Co-authored-by: Renato Byrro <rmbyrro@gmail.com>
## PR Description
Search ([see API docs
here](https://docs.x.com/x-api/posts/recent-search)) returns a 'data'
field that maps to a list of tweets returned. We've observed that the
'data' field is not present if no tweets match the search. This PR
handles that case safely.
Currently, retrieving DMs with a given username requires several
actions: first get the current user's ID; list all users and find the ID
of the username; then scan all DM conversations and find the one with
the current user's ID and the username's ID, to finally retrieve the
messages using that conversation ID.
This tool abstracts all that in a single call.
PS: we'll implement a similar tool for multi-person DM conversations in
a subsequent PR.
# PR Description
This PR adds ~~four~~ three improvements to evals.
~~## 1. Add parameterized eval cases~~
~~Adds a new method named `add_parameterized_case`. Just like pytest’s
parameterized tests, eval cases can be parameterized with multiple user
messages. Adds a case to the `EvalSuite` for each user message. All
cases have the same expected tool call(s), params, additional_messages.
This reduces duplicate code and makes it easy to observe how a model
performs based on increasingly more difficult prompts.~~
```python
""" NO LONGER IN THIS PR
user_messages = [
"Call the delete tweet by id tool with the tweet ID '148975632'.",
"Delete the tweet with ID '148975632'.",
"I don't want to have this tweet (148975632) on my account anymore.",
"do the opposite of post for https://x.com/x/status/148975632",
]
suite.add_parameterized_case(
name="Delete a tweet by ID",
user_messages=user_messages,
expected_tool_calls=[
ExpectedToolCall(
func=delete_tweet_by_id,
args={"tweet_id": "148975632"},
)
],
critics=[
BinaryCritic(
critic_field="tweet_id",
weight=1.0,
),
],
)
"""
```
~~PASSED Delete a tweet by ID (user_message 1 of 4) -- Score: 100.00%~~
~~PASSED Delete a tweet by ID (user_message 2 of 4) -- Score: 100.00%~~
~~PASSED Delete a tweet by ID (user_message 3 of 4) -- Score: 100.00%~~
~~FAILED Delete a tweet by ID (user_message 4 of 4) -- Score: 0.00%~~
~~Summary -- Total: 4 -- Passed: 3 -- Failed: 1~~
## 2. Parameters that are not explicitly criticized are assigned a
`NoneCritic`.
A NoneCritic has no effect on the evaluation results and does not
actually evaluate. Parameters that have a NoneCritic will be displayed
as ‘un-criticized’ in the evaluation summary (if `-d` flag is used).
## 3. Add a hardcoded `seed` parameter for evals.
The seed parameter aides in receiving (mostly) consistent outputs -
aiding in reproducibility for evaluations.
## 4. Disallow more than one critic for the same field.
Raises a `ValueError` if more than one critic is assigned to a field.
---------
Co-authored-by: Eric Gustin <eric@arcade-ai.com>
# PR Description
* This PR updates code in `examples/` to be compatible with version
1.0.0
* This PR removes the Spotify examples since the Arcade hosted worker
doesn't currently cataloge the Spotify toolkit. We can reintroduce these
examples when it does.
* This PR performs various renames across the codebase for
`arcade-ai.com` --> `arcade.dev` and `Arcade AI` --> `Arcade`
# PR Description
The `github.event.pull_request.author_association` in the "Prevent
Unauthorized Version Updates" workflow was returning inconsistent
results by saying that MEMBERS were CONTRIBUTORS. This PR moves away
from `author_association` in favor of a whitelist text file containing
the GitHub usernames of authorized toolkit release managers.
A toolkit release manager has the following special permissions:
* Can change the version of an existing toolkit
* Can delete an existing toolkit
* Can rename an existing toolkit
The `get_conversation_metadata_by_name` tool retrieves conversation
metadata from another tool, `list_conversations_metadata`, but was
accessing the `next_cursor` using the Slack API response dict structure,
instead of the tool response structure. As a result, in that tool, the
tool would never actually paginate to the second page. This PR fixes it
and also adjust tests to capture the issue appropriately.
`starred` is a required argument of the
`arcade_github.activity.set_starred` tool, but when it is not provided
in the tool call, the engine is somehow passing it with a falsy value,
instead of raising an error. the falsy value makes the tool unstar a
repo by default, which is not the desired behavior.
we're setting the `starred` arg to True in the tool interface to prevent
that.
implements additional tools for Slack related to retrieving
conversations metadata, list of members, history of messages, as well as
sending messages to private/public channels and DMs / multi-person DMs.
---------
Co-authored-by: Eric Gustin <eric@arcade-ai.com>
Co-authored-by: Renato Byrro <rmbyrro@gmail.com>
# PR Description
For the `Google.GetThread` tool, we had a parameter named
`metadata_headers`. This parameter only makes a difference if the format
is "metadata", but the tool will never have the format "metadata". So,
the input parameter is useless. This parameter should have never been
added to the tool and we should remove it before public beta.
# PR Description
Changes to a toolkit without changes to the toolkit's version fail the
'Publish Toolkit' workflow with `HTTP Error 400: File already exists
('arcade_zoom-0.1.7.tar.gz', with blake2_256 hash
'02183cda607f06616e7edb17e3d22bc11d1d83b074b3e44066b78ec72602fb37'). See
https://pypi.org/help/#file-name-reuse for more information.`, for
example.
This PR adds the `--skip-existing` flag to `poetry publish` to avoid
attempting to publish an existing version. Skips slack notification if
publish is skipped.
The `grep`'d string comes from
https://github.com/python-poetry/poetry/blob/main/src/poetry/publishing/uploader.py#L246-L249
Modifies X tweet tools to return metadata about media attachments
(photo, GIF or video) when retrieving a tweet by ID, username or
keywords.
The tool will always return media attachments by default. Since it's
only metadata, it shouldn't add significant network overhead to existing
implementations of the tool.
My guess is more often than not people will want this info included.
When not needed, it doesn't hurt to include by default. It'd be annoying
to have to ask the LLM to include it every time they need.
# PR Description
Poetry released v2 with many breaking changes a couple days ago. The
`install-poetry` action that our workflows use default to that v2
version, so many of our workflows are failing. This PR forces that
action to use poetry version 1.8.5 and also uses 1.8.5 for toolkits
A ticket to migrate to 2.0.0 has been filed for future work
# PR Description
* Adds/updates the following files to all toolkits:
- `.pre-commit-config.yaml`
- `.ruff.toml`
- `LICENSE`
- `Makefile`
- `pyproject.toml`
* Lint all toolkits such that they pass `make check` and `make test` (a
total doozy). This includes adding some unit tests and evals.
* Github workflow for testing toolkits before merge into main (courtesy
of @sdreyer)
* Added a QOL improvement for tool developers for when they need to get
the context's auth token.
* Minor updates to `arcade new` template.
