07c52100f3
# PR Description ## Split toolkits This PR splits the `Microsoft`, `Google`, and `Search` toolkits into multiple toolkits each. * `Microsoft` --> `OutlookCalendar`, `OutlookMail`. * `Google` -----> `GoogleCalendar`, `GoogleContacts`, `GoogleDocs`, `GoogleDrive`, `Gmail`, `GoogleSheets` * `Search` -----> `GoogleFinance`, `GoogleFlights`, `GoogleHotels`, `GoogleJobs`, `GoogleMaps`, `GoogleNews`, `GoogleSearch`, `GoogleShopping`, `Walmart`, `Youtube` > The original monolithic toolkits (`Microsoft`, `Google`, `Search`) are not removed in this PR. The plan is to keep those toolkits around while we > 1. Stop documenting the toolkits, > 2. Stop displaying the toolkits in the dashboard, and > 3. Help customers migrate over to the new split toolkits. ## Rename toolkits This PR renames the following toolkits * `Web` ------------> `Firecrawl` * `CodeSandbox` ---> `E2B` > The `Web` and `CodeSandbox` toolkits are not removed in this PR. The plan is to keep them around while we > 1. Stop documenting the toolkits, > 2. Stop displaying the toolkits in the dashboard, and > 3. Help customers migrate over to the new renamed toolkits. ## Rename tools Since toolkit names were changed, this called for some tools to be renamed as well. * `GoogleSearch.SearchGoogle` ----------------> `GoogleSearch.Search` * `GoogleShopping.SearchShoppingProducts` ---> `GoogleShopping.SearchProducts` * `Walmart.SearchWalmartProducts` ------------> `Walmart.SearchProducts` * `Walmart.GetWalmartProductDetails` ---------> `Walmart.GetProductDetails` * `Youtube.SearchYoutubeVideos` --------------> `Youtube.SearchForVideos` ## Google File Picker Improvements to the Google File Picker experience were also added in this PR. The following tools will ALWAYS provide llm_instructions in their response to "let the end-user know that they have the option to select more files via the file picker url if they want to": * `GoogleDocs.SearchDocuments` * `GoogleDocs.SearchAndRetrieveDocuments` * `GoogleDrive.GetFileTreeStructure` The following tools will only provide the file picker URL if a 404 or 403 from the Google API: * `GoogleDocs.InsertTextAtEndOfDocument` * `GoogleDocs.GetDocumentById` * `GoogleSheets.GetSpreadsheet` * `GoogleSheets.WriteToCell` Also, a standalone `GoogleDrive.GenerateGoogleFilePickerUrl` tool exists. ## Other * The `SearchDocuments` and `SearchAndRetrieveDocuments` tools used to be organized within the Drive portion of the Google toolkit, but I moved these into the new GoogleDocs toolkit because they are specific to Docs. # Progress - [x] `OutlookCalendar` - [x] `OutlookMail` - [x] `GoogleFinance` - [x] `GoogleFlights` - [x] `GoogleHotels` - [x] `GoogleJobs` - [x] `GoogleMaps` - [x] `GoogleNews` - [x] `GoogleSearch` - [x] `GoogleShopping` - [x] `Walmart` - [x] `Youtube` - [x] `GoogleCalendar` - [x] `GoogleContacts` - [x] `GoogleDocs` - [x] `GoogleDrive` - [x] `Gmail` - [x] `GoogleSheets` - [x] `Firecrawl` - [x] `E2B` - [x] File picker # Discussion * Repeated code is a consequence of splitting toolkits that use the same provider. I am open to any ideas that would allow multiple toolkits to reference common code. Comment your ideas in this PR.
68 lines
2 KiB
Python
68 lines
2 KiB
Python
import pytest
|
|
import serpapi
|
|
from arcade_tdk.errors import ToolExecutionError
|
|
|
|
from arcade_google_search.utils import call_serpapi, prepare_params
|
|
|
|
|
|
class DummyContext:
|
|
def get_secret(self, key: str) -> str | None:
|
|
if key.lower() == "serp_api_key":
|
|
return "dummy_key"
|
|
return None
|
|
|
|
|
|
@pytest.fixture
|
|
def dummy_context():
|
|
return DummyContext()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
@pytest.mark.parametrize(
|
|
"engine, kwargs, expected",
|
|
[
|
|
("google", {}, {"engine": "google"}),
|
|
(
|
|
"google",
|
|
{"q": "test", "window": 10, "time": "00:12:12"},
|
|
{
|
|
"engine": "google",
|
|
"q": "test",
|
|
"window": 10,
|
|
"time": "00:12:12",
|
|
},
|
|
),
|
|
],
|
|
)
|
|
async def test_prepare_params(engine, kwargs, expected):
|
|
params = prepare_params(engine, **kwargs)
|
|
assert params == expected
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"error_message, sanitized_message",
|
|
[
|
|
(
|
|
"You hit your rate limit",
|
|
"You hit your rate limit",
|
|
),
|
|
(
|
|
"Bad Request for url: https://serpapi.com/search?engine=google_hotels&api_key=ABC123456",
|
|
"Bad Request for url: https://serpapi.com/search?engine=google_hotels&api_key=***",
|
|
),
|
|
(
|
|
"Bad Request for url: https://serpapi.com/search?engine=google_hotels&api_key=ABC123456 make sure the api key is correct",
|
|
"Bad Request for url: https://serpapi.com/search?engine=google_hotels&api_key=*** make sure the api key is correct",
|
|
),
|
|
],
|
|
)
|
|
def test_call_serpapi_failure(monkeypatch, dummy_context, error_message, sanitized_message):
|
|
def fake_serpapi_search(self, params: dict) -> dict:
|
|
raise Exception(error_message) # noqa: TRY002
|
|
|
|
monkeypatch.setattr(serpapi.Client, "search", fake_serpapi_search)
|
|
|
|
with pytest.raises(ToolExecutionError) as excinfo:
|
|
call_serpapi(dummy_context, {})
|
|
|
|
assert excinfo.value.developer_message == sanitized_message
|