arcade-mcp/libs/arcade-core/arcade_core/schema.py
Eric Gustin f4558ef3a8
Tool Error Handling (#539)
# Improvements to Arcade TDK Error Handling
I tried my very best to not make any breaking changes in this PR. So,
you will notice various "Deprecation" notices throughout.

### Instructions for PR reviewers
1. Pull down this PR's branch
2. Pull down the Engine's tool error handling PR's branch
3. Update your installed arcadepy to have the following:
- In `arcadepy/resources/tools/tools.py`, if you want to test out
including stacktraces, then you need to update `ToolsResource.execute`
to accept a `include_error_stacktrace` argument and also include the
"include_error_stacktrace" argument to the POST to the Engine inside of
the function's execute method's body.
- In `arcadepy/types/execute_tool_response.py` add the following enum
      ```py
      class ErrorKind(str, Enum):
          """Error kind that is comprised of
          - the who (toolkit, tool, upstream)
          - the when (load time, definition parsing time, runtime)
- the what (bad_definition, bad_input, bad_output, retry,
context_required, fatal, etc.)"""
      
          TOOLKIT_LOAD_FAILED = "TOOLKIT_LOAD_FAILED"
TOOL_DEFINITION_BAD_DEFINITION = "TOOL_DEFINITION_BAD_DEFINITION"
TOOL_DEFINITION_BAD_INPUT_SCHEMA = "TOOL_DEFINITION_BAD_INPUT_SCHEMA"
TOOL_DEFINITION_BAD_OUTPUT_SCHEMA = "TOOL_DEFINITION_BAD_OUTPUT_SCHEMA"
          TOOL_RUNTIME_BAD_INPUT_VALUE = "TOOL_RUNTIME_BAD_INPUT_VALUE"
TOOL_RUNTIME_BAD_OUTPUT_VALUE = "TOOL_RUNTIME_BAD_OUTPUT_VALUE"
          TOOL_RUNTIME_RETRY = "TOOL_RUNTIME_RETRY"
TOOL_RUNTIME_CONTEXT_REQUIRED = "TOOL_RUNTIME_CONTEXT_REQUIRED"
          TOOL_RUNTIME_FATAL = "TOOL_RUNTIME_FATAL"
          UPSTREAM_RUNTIME_BAD_REQUEST = "UPSTREAM_RUNTIME_BAD_REQUEST"
          UPSTREAM_RUNTIME_AUTH_ERROR = "UPSTREAM_RUNTIME_AUTH_ERROR"
          UPSTREAM_RUNTIME_NOT_FOUND = "UPSTREAM_RUNTIME_NOT_FOUND"
UPSTREAM_RUNTIME_VALIDATION_ERROR = "UPSTREAM_RUNTIME_VALIDATION_ERROR"
          UPSTREAM_RUNTIME_RATE_LIMIT = "UPSTREAM_RUNTIME_RATE_LIMIT"
UPSTREAM_RUNTIME_SERVER_ERROR = "UPSTREAM_RUNTIME_SERVER_ERROR"
          UPSTREAM_RUNTIME_UNMAPPED = "UPSTREAM_RUNTIME_UNMAPPED"
          UNKNOWN = "UNKNOWN"
      ```
- In `arcadepy/types/execute_tool_response.py` add the following fields
to OutputError:
      ```py
      kind: ErrorKind
      status_code: Optional[int] = None
      stacktrace: Optional[str] = None
      extra: Optional[dict[str, Any]] = None
      ```
### Example Client Usage
```py
# Example of handling an upstream rate limit
error = response.output.error
if  error and error.kind == ErrorKind.UPSTREAM_RUNTIME_RATE_LIMIT:
    sleep_time = error.retry_after_ms / 1000
    time.sleep(sleep_time)
    # and then execute again
```
```py
# Examples of determining what type of runtime error it is
error = response.output.error
if error:
    is_retryable_error = error.kind == ErrorKind.TOOL_RUNTIME_RETRY
    is_a_bug_in_the_tool = error.kind == ErrorKind.TOOL_RUNTIME_FATAL
    is_additional_context_required = error.kind == ErrorKind.TOOL_RUNTIME_CONTEXT_REQUIRED
```

### Example Tool Usage
```py
# EXAMPLE 1 letting Arcade handle upstream error handling for you
reddit_client.post(params) # Arcade's httpx adapter will handle error handling for you!

# ------------------------------------

# EXAMPLE 2 handling upstream bad request yourself, but letting Arcade handle the rest
try:
    reddit_client.post(params)
except httpx.HTTPStatusError as e:
    if e.status_code == 400:
        raise UpstreamError("My extra custom message) from e
    raise
```
```py
# EXAMPLE 1 letting Arcade handle it for you
risky_element = my_risky_list[42] # Arcade will raise a FatalToolError for you

# ------------------------------------

# EXAMPLE 2 handling it yourself for extra flexibility
try:
    risky_element = my_risky_list[42]
except IndexError as e:
    raise FatalToolError("My extra custom message") from e
```
### Non-runtime Error Message Examples
Example ToolkitLoadError Messages:
```
- [TOOLKIT_LOAD_FAILED] ToolkitLoadError when loading toolkit 'sample_tool': Could not import module mock_module. Reason: Mock import error
- [TOOLKIT_LOAD_FAILED] ToolkitLoadError when loading toolkit 'test_toolkit': Tool 'ValidTool' in toolkit 'test_toolkit' already exists in the catalog.
```
Example ToolDefinitionError Messages
```
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_missing_description': Tool 'tool_missing_description' is missing a description
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_invalid_secret_type': Secret keys must be strings (error in tool ToolWithInvalidSecretType).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_empty_secret': Secrets must have a non-empty key (error in tool ToolWithEmptySecret).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_invalid_metadata_type': Metadata must be strings (error in tool ToolWithInvalidMetadataType).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_metadata_requiring_auth_without_auth': Tool ToolWithMetadataRequiringAuthWithoutAuth declares metadata key 'client_id', which requires that the tool has an auth requirement, but no auth requirement was provided. Please specify an auth requirement.
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_empty_metadata': Metadata must have a non-empty key (error in tool ToolWithEmptyMetadata).
- [TOOL_DEFINITION_BAD_DEFINITION] ToolDefinitionError in definition of tool 'tool_with_unsupported_param_type': Unsupported parameter type: <class 'test_catalog.MyFancyTestClass'>
```
Example ToolInputSchemaError Messages
```
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_missing_input_parameter_annotation': Parameter 'input_text' is missing a description
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_no_type_annotation': Parameter param has no type annotation.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_invalid_param_name': Invalid parameter name: '123invalid' is not a valid identifier. Identifiers must start with a letter or underscore, and can only contain letters, digits, or underscores.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_too_many_annotations': Parameter param: Annotated[str, 'name', 'desc', 'extra'] has too many string annotations. Expected 0, 1, or 2, got 3.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_required_union_param': Parameter param is a union type. Only optional types are supported.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_non_callable_default_factory': Default factory for parameter param: Annotated[str, 'Parameter'] = FieldInfo(annotation=NoneType, required=False, default_factory=str) is not callable.
- [TOOL_DEFINITION_BAD_INPUT_SCHEMA] ToolInputSchemaError in definition of tool 'tool_with_multiple_tool_contexts': Only one ToolContext parameter is supported, but tool tool_with_multiple_tool_contexts has multiple.
```
Example ToolOutputSchemaError Messages
```
- [TOOL_DEFINITION_BAD_OUTPUT_SCHEMA] ToolOutputSchemaError in definition of tool 'tool_missing_return_type_hint': Tool 'ToolMissingReturnTypeHint' must have a return type
- [TOOL_DEFINITION_BAD_OUTPUT_SCHEMA] ToolOutputSchemaError in definition of tool 'tool_with_unsupported_output_type': Unsupported output type '<class 'test_catalog.MyFancyTestClass'>'. Only built-in Python types, TypedDicts, Pydantic models, and standard collections are supported as tool output types.
```
### Runtime Error Message Examples
Example Tool Runtime Error Messages
```
- [TOOL_RUNTIME_FATAL] FatalToolError during execution of tool 'get_posts_in_subreddit': list index out of range
- [TOOL_RUNTIME_CONTEXT_REQUIRED] ContextRequiredToolError during execution of tool 'get_posts_in_subreddit': Ambiguous username. Please provide a more specific username
- [TOOL_RUNTIME_RETRY] RetryableToolError during execution of tool 'get_posts_in_subreddit': Retry with subreddit=learnpython or subreddit=learnprogramming
```

Example Upstream Runtime Error Messages
```
- [UPSTREAM_RUNTIME_RATE_LIMIT] UpstreamRateLimitError during execution of tool 'get_posts_in_subreddit': 429 Client Error: Too Many Requests
- [UPSTREAM_RUNTIME_BAD_REQUEST] UpstreamError during execution of tool 'get_posts_in_subreddit': 400 Client Error: Bad request. Missing 'id' parameter.
- [UPSTREAM_RUNTIME_BAD_REQUEST] UpstreamError during execution of tool 'search_files': Upstream Google API error: Invalid value '-23'. Values must be within the range: [value: 1\n, value: 1000\n]
```
2025-09-10 10:45:18 -07:00

477 lines
15 KiB
Python

import os
from dataclasses import dataclass
from enum import Enum
from typing import Any, Literal
from pydantic import BaseModel, Field
from arcade_core.errors import ErrorKind
# allow for custom tool name separator
TOOL_NAME_SEPARATOR = os.getenv("ARCADE_TOOL_NAME_SEPARATOR", ".")
class ValueSchema(BaseModel):
"""Value schema for input parameters and outputs."""
val_type: Literal["string", "integer", "number", "boolean", "json", "array"]
"""The type of the value."""
inner_val_type: Literal["string", "integer", "number", "boolean", "json"] | None = None
"""The type of the inner value, if the value is a list."""
enum: list[str] | None = None
"""The list of possible values for the value, if it is a closed list."""
properties: dict[str, "ValueSchema"] | None = None
"""For object types (json), the schema of nested properties."""
inner_properties: dict[str, "ValueSchema"] | None = None
"""For array types with json items, the schema of properties for each array item."""
description: str | None = None
"""Optional description of the value."""
class InputParameter(BaseModel):
"""A parameter that can be passed to a tool."""
name: str = Field(..., description="The human-readable name of this parameter.")
required: bool = Field(
...,
description="Whether this parameter is required (true) or optional (false).",
)
description: str | None = Field(
None,
description="A descriptive, human-readable explanation of the parameter.",
)
value_schema: ValueSchema = Field(
...,
description="The schema of the value of this parameter.",
)
inferrable: bool = Field(
True,
description="Whether a value for this parameter can be inferred by a model. Defaults to `true`.",
)
class ToolInput(BaseModel):
"""The inputs that a tool accepts."""
parameters: list[InputParameter]
"""The list of parameters that the tool accepts."""
tool_context_parameter_name: str | None = Field(default=None, exclude=True)
"""
The name of the target parameter that will contain the tool context (if any).
This field will not be included in serialization.
"""
class ToolOutput(BaseModel):
"""The output of a tool."""
description: str | None = Field(
None, description="A descriptive, human-readable explanation of the output."
)
available_modes: list[str] = Field(
default_factory=lambda: ["value", "error", "null"],
description="The available modes for the output.",
)
value_schema: ValueSchema | None = Field(
None, description="The schema of the value of the output."
)
class OAuth2Requirement(BaseModel):
"""Indicates that the tool requires OAuth 2.0 authorization."""
scopes: list[str] | None = None
"""The scope(s) needed for the authorized action."""
class ToolAuthRequirement(BaseModel):
"""A requirement for authorization to use a tool."""
# Provider ID, Type, and ID needed for the Arcade Engine to look up the auth provider.
# However, the developer generally does not need to set these directly.
# Instead, they will use:
# @tool(requires_auth=Google(scopes=["profile", "email"]))
# or
# client.auth.authorize(provider=AuthProvider.google, scopes=["profile", "email"])
#
# The Arcade SDK translates these into the appropriate provider ID (Google) and type (OAuth2).
# The only time the developer will set these is if they are using a custom auth provider.
provider_id: str | None = None
"""The provider ID configured in Arcade that acts as an alias to well-known configuration."""
provider_type: str
"""The type of the authorization provider."""
id: str | None = None
"""A provider's unique identifier, allowing the tool to specify a specific authorization provider. Recommended for private tools only."""
oauth2: OAuth2Requirement | None = None
"""The OAuth 2.0 requirement, if any."""
class ToolSecretRequirement(BaseModel):
"""A requirement for a tool to run."""
key: str
"""The ID of the secret."""
class ToolMetadataKey(str, Enum):
"""Convience enum for commonly used metadata keys."""
CLIENT_ID = "client_id"
COORDINATOR_URL = "coordinator_url"
@staticmethod
def requires_auth(key: str) -> bool:
"""Whether the key depends on the tool having an authorization requirement."""
keys_that_require_auth = [ToolMetadataKey.CLIENT_ID]
return key.strip().lower() in keys_that_require_auth
class ToolMetadataRequirement(BaseModel):
"""A requirement for a tool to run."""
key: str
"""The ID of the metadata."""
class ToolRequirements(BaseModel):
"""The requirements for a tool to run."""
authorization: ToolAuthRequirement | None = None
"""The authorization requirements for the tool, if any."""
secrets: list[ToolSecretRequirement] | None = None
"""The secret requirements for the tool, if any."""
metadata: list[ToolMetadataRequirement] | None = None
"""The metadata requirements for the tool, if any."""
class ToolkitDefinition(BaseModel):
"""The specification of a toolkit."""
name: str
"""The name of the toolkit."""
description: str | None = None
"""The description of the toolkit."""
version: str | None = None
"""The version identifier of the toolkit."""
@dataclass(frozen=True)
class FullyQualifiedName:
"""The fully-qualified name of a tool."""
name: str
"""The name of the tool."""
toolkit_name: str
"""The name of the toolkit containing the tool."""
toolkit_version: str | None = None
"""The version of the toolkit containing the tool."""
def __str__(self) -> str:
return f"{self.toolkit_name}{TOOL_NAME_SEPARATOR}{self.name}"
def __eq__(self, other: Any) -> bool:
if not isinstance(other, FullyQualifiedName):
return False
return (
self.name.lower() == other.name.lower()
and self.toolkit_name.lower() == other.toolkit_name.lower()
and (self.toolkit_version or "").lower() == (other.toolkit_version or "").lower()
)
def __hash__(self) -> int:
return hash((
self.name.lower(),
self.toolkit_name.lower(),
(self.toolkit_version or "").lower(),
))
def equals_ignoring_version(self, other: "FullyQualifiedName") -> bool:
"""Check if two fully-qualified tool names are equal, ignoring the version."""
return (
self.name.lower() == other.name.lower()
and self.toolkit_name.lower() == other.toolkit_name.lower()
)
@staticmethod
def from_toolkit(tool_name: str, toolkit: ToolkitDefinition) -> "FullyQualifiedName":
"""Creates a fully-qualified tool name from a tool name and a ToolkitDefinition."""
return FullyQualifiedName(tool_name, toolkit.name, toolkit.version)
class ToolDefinition(BaseModel):
"""The specification of a tool."""
name: str
"""The name of the tool."""
fully_qualified_name: str
"""The fully-qualified name of the tool."""
description: str
"""The description of the tool."""
toolkit: ToolkitDefinition
"""The toolkit that contains the tool."""
input: ToolInput
"""The inputs that the tool accepts."""
output: ToolOutput
"""The output types that the tool can return."""
requirements: ToolRequirements
"""The requirements (e.g. authorization) for the tool to run."""
deprecation_message: str | None = None
"""The message to display when the tool is deprecated."""
def get_fully_qualified_name(self) -> FullyQualifiedName:
return FullyQualifiedName(self.name, self.toolkit.name, self.toolkit.version)
class ToolReference(BaseModel):
"""The name and version of a tool."""
name: str
"""The name of the tool."""
toolkit: str
"""The name of the toolkit containing the tool."""
version: str | None = None
"""The version of the toolkit containing the tool."""
def get_fully_qualified_name(self) -> FullyQualifiedName:
return FullyQualifiedName(self.name, self.toolkit, self.version)
class ToolAuthorizationContext(BaseModel):
"""The context for a tool invocation that requires authorization."""
token: str | None = None
"""The token for the tool invocation."""
user_info: dict = Field(default={})
"""
The user information provided by the authorization server (if any).
Some providers can provide structured user info,
for example an internal provider-specific user ID.
For those providers that support retrieving user info,
the Engine can automatically pass that to tool invocations.
"""
class ToolSecretItem(BaseModel):
"""The context for a tool secret."""
key: str
"""The key of the secret."""
value: str
"""The value of the secret."""
class ToolMetadataItem(BaseModel):
"""The context for a tool metadata."""
key: str
"""The key of the metadata."""
value: str
"""The value of the metadata."""
class ToolContext(BaseModel):
"""The context for a tool invocation."""
authorization: ToolAuthorizationContext | None = None
"""The authorization context for the tool invocation that requires authorization."""
secrets: list[ToolSecretItem] | None = None
"""The secrets for the tool invocation."""
metadata: list[ToolMetadataItem] | None = None
"""The metadata for the tool invocation."""
user_id: str | None = None
"""The user ID for the tool invocation (if any)."""
def get_auth_token_or_empty(self) -> str:
"""Retrieve the authorization token, or return an empty string if not available."""
return self.authorization.token if self.authorization and self.authorization.token else ""
def get_secret(self, key: str) -> str:
"""Retrieve the secret for the tool invocation."""
return self._get_item(key, self.secrets, "secret")
def get_metadata(self, key: str) -> str:
"""Retrieve the metadata for the tool invocation."""
return self._get_item(key, self.metadata, "metadata")
def _get_item(
self,
key: str,
items: list[ToolMetadataItem] | list[ToolSecretItem] | None,
item_name: str,
) -> str:
if not key or not key.strip():
raise ValueError(
f"{item_name.capitalize()} key passed to get_{item_name} cannot be empty."
)
if not items:
raise ValueError(f"{item_name.capitalize()}s not found in context.")
normalized_key = key.lower()
for item in items:
if item.key.lower() == normalized_key:
return item.value
raise ValueError(f"{item_name.capitalize()} {key} not found in context.")
def set_secret(self, key: str, value: str) -> None:
"""Set a secret for the tool invocation."""
if not self.secrets:
self.secrets = []
secret = ToolSecretItem(key=str(key), value=str(value))
self.secrets.append(secret)
class ToolCallRequest(BaseModel):
"""The request to call (invoke) a tool."""
run_id: str | None = None
"""The globally-unique run ID provided by the Engine."""
execution_id: str | None = None
"""The globally-unique ID for this tool execution in the run."""
created_at: str | None = None
"""The timestamp when the tool invocation was created."""
tool: ToolReference
"""The fully-qualified name and version of the tool."""
inputs: dict[str, Any] | None = None
"""The inputs for the tool."""
context: ToolContext = Field(default_factory=ToolContext)
"""The context for the tool invocation."""
class ToolCallLog(BaseModel):
"""A log that occurred during the tool invocation."""
message: str
"""The user-facing warning message."""
level: Literal[
"debug",
"info",
"warning",
"error",
]
"""The level of severity for the log."""
subtype: Literal["deprecation"] | None = None
"""Optional field for further categorization of the log."""
class ToolCallError(BaseModel):
"""The error that occurred during the tool invocation."""
message: str
"""The user-facing error message."""
kind: ErrorKind
"""The error kind that uniquely identifies the kind of error."""
developer_message: str | None = None
"""The developer-facing error details."""
can_retry: bool = False
"""Whether the tool call can be retried."""
additional_prompt_content: str | None = None
"""Additional content to be included in the retry prompt."""
retry_after_ms: int | None = None
"""The number of milliseconds (if any) to wait before retrying the tool call."""
stacktrace: str | None = None
"""The stacktrace information for the tool call."""
status_code: int | None = None
"""The HTTP status code of the error."""
extra: dict[str, Any] | None = None
"""Additional information about the error."""
@property
def is_toolkit_error(self) -> bool:
"""Check if this error originated from loading a toolkit."""
return self.kind.name.startswith("TOOLKIT_")
@property
def is_tool_error(self) -> bool:
"""Check if this error originated from a tool."""
return self.kind.name.startswith("TOOL_")
@property
def is_upstream_error(self) -> bool:
"""Check if this error originated from an upstream service."""
return self.kind.name.startswith("UPSTREAM_")
class ToolCallRequiresAuthorization(BaseModel):
"""The authorization requirements for the tool invocation."""
authorization_url: str | None = None
"""The URL to redirect the user to for authorization."""
authorization_id: str | None = None
"""The ID for checking the status of the authorization."""
scopes: list[str] | None = None
"""The scopes that are required for authorization."""
status: str | None = None
"""The status of the authorization."""
class ToolCallOutput(BaseModel):
"""The output of a tool invocation."""
value: str | int | float | bool | dict | list | None = None
"""The value returned by the tool."""
logs: list[ToolCallLog] | None = None
"""The logs that occurred during the tool invocation."""
error: ToolCallError | None = None
"""The error that occurred during the tool invocation."""
requires_authorization: ToolCallRequiresAuthorization | None = None
"""The authorization requirements for the tool invocation."""
model_config = {
"json_schema_extra": {
"oneOf": [
{"required": ["value"]},
{"required": ["error"]},
{"required": ["requires_authorization"]},
{"required": ["artifact"]},
]
}
}
class ToolCallResponse(BaseModel):
"""The response to a tool invocation."""
execution_id: str
"""The globally-unique ID for this tool execution."""
finished_at: str
"""The timestamp when the tool execution finished."""
duration: float
"""The duration of the tool execution in milliseconds (ms)."""
success: bool
"""Whether the tool execution was successful."""
output: ToolCallOutput | None = None
"""The output of the tool invocation."""