## Summary Routes HTTP adapter exceptions to the right error class instead of shoe-horning everything into `UpstreamError`. Addresses Eric's earlier feedback that several exceptions this PR was wrapping as `UpstreamError` didn't satisfy the "something happened with the upstream" claim (local pool exhaustion, client-side request construction, local TLS failures). ### Scope - `UpstreamError` (unchanged) — upstream responded with an HTTP status code. - **`NetworkTransportError`** (new sibling in `arcade-core`) — no complete response was received. `status_code=None`. Three kinds: `NETWORK_TRANSPORT_RUNTIME_TIMEOUT`, `_UNREACHABLE`, `_UNMAPPED`. - **`FatalToolError`** (existing) — client construction bugs (`InvalidURL`, `UnsupportedProtocol`, `MissingSchema`, `InvalidHeader`, `LocalProtocolError`, …) and local TLS/cert config failures. Never retried. --- ## Before / After (per Eric's request) Shows the error payload a tool produces for each exception, before this PR vs. after. "Before" = current `main` (exceptions without real HTTP responses fall through to the generic `@tool` `FatalToolError` catch-all with `message=str(exc)`). ### No-response transport failures | Exception | Before — class / message / kind | After — class / message / kind | |---|---|---| | `httpx.PoolTimeout` | `FatalToolError` — `str(exc)` leaks raw detail — `TOOL_RUNTIME_FATAL`, not retryable | `NetworkTransportError` — `"HTTP request timed out before a complete response was received."` — `NETWORK_TRANSPORT_RUNTIME_TIMEOUT`, **retryable** | | `httpx.ConnectTimeout` | same as above | same as PoolTimeout — `TIMEOUT`, retryable | | `httpx.ConnectError` (refused / DNS) | `FatalToolError` — `str(exc)` | `NetworkTransportError` — `"HTTP request failed before reaching the upstream service."` — `UNREACHABLE`, retryable | | `httpx.RemoteProtocolError` (upstream sent bad HTTP) | `FatalToolError` — `str(exc)` | `NetworkTransportError` — same message as ConnectError — `UNREACHABLE`, retryable | | `httpx.DecodingError` | `FatalToolError` — `str(exc)` | `NetworkTransportError` — `"HTTP response from upstream could not be decoded."` — `UNMAPPED`, retryable | | `httpx.TooManyRedirects` | `FatalToolError` — `str(exc)` | `NetworkTransportError` — `"HTTP redirect limit exceeded before a final response was received."` — `UNMAPPED`, **not** retryable | ### Client construction / local env bugs | Exception | Before | After | |---|---|---| | `httpx.UnsupportedProtocol`, `httpx.InvalidURL`, `httpx.LocalProtocolError` | `FatalToolError` with `message=str(exc)` (may leak scheme / URL content) | `FatalToolError` — `"Tool constructed an invalid HTTP request — likely a tool-authoring bug."` — `TOOL_RUNTIME_FATAL`, not retryable | | `requests.MissingSchema`, `InvalidURL`, `InvalidHeader`, `InvalidSchema`, `InvalidProxyURL`, `URLRequired` | same as above | same as above | | `requests.SSLError` | `FatalToolError` — `str(exc)` often contains raw cert chain detail | `FatalToolError` — `"TLS handshake failed — likely a local certificate or trust configuration issue."` — `TOOL_RUNTIME_FATAL`, not retryable | ### Real HTTP response errors (UNCHANGED — same behavior) | Exception | Class | Message | Kind | Retryable | |---|---|---|---|---| | `httpx.HTTPStatusError` 404 | `UpstreamError` | `"Upstream HTTP request failed (Not Found, client error)."` | `UPSTREAM_RUNTIME_NOT_FOUND` | No | | `httpx.HTTPStatusError` 429 (w/ Retry-After: 60) | `UpstreamRateLimitError` | `"Upstream HTTP request failed (Too Many Requests, client error). Retry after 60 second(s)."` | `UPSTREAM_RUNTIME_RATE_LIMIT` | Yes | | `httpx.HTTPStatusError` 500 | `UpstreamError` | `"Upstream HTTP request failed (Internal Server Error, server error)."` | `UPSTREAM_RUNTIME_SERVER_ERROR` | Yes | ### What's no longer in the message - Raw exception `str(exc)` output (which frequently includes the full URL with query-string tokens, connection pool details, or cert chains) is **no longer the agent-facing `message`**. It's preserved in `developer_message` for server-side diagnostics. - The misleading "Upstream HTTP…" prefix is gone from network-transport and construction-bug messages. Those messages now honestly describe what happened on the tool side. - For 429s without a `Retry-After` header, we still show "Retry after N seconds." (pre-existing behavior; see follow-up notes). --- ## Companion PRs - [ArcadeAI/arcade-mcp#823](https://github.com/ArcadeAI/arcade-mcp/pull/823) — introduces `NetworkTransportError` in `arcade-core` - [ArcadeAI/monorepo#911](https://github.com/ArcadeAI/monorepo/pull/911) — adds the 3 `ErrorKind` constants to the Go engine and Datadog dashboards - [ArcadeAI/docs#920](https://github.com/ArcadeAI/docs/pull/920) — documents the new hierarchy and adapter routing ## Follow-ups (out of scope for this PR) A short investigation surfaced several pre-existing issues that are worth fixing separately. A full list is in `NETWORK_TRANSPORT_ERROR_FOLLOWUPS.md` (shared offline). Summary: 1. `requests.HTTPError` with `response is None` returns `None` from the adapter; should fall through to the `NetworkTransportError(UNMAPPED)` fallback instead of becoming a generic `FatalToolError`. 2. `developer_message` can leak URL query strings (and therefore tokens) since it stores raw `str(exc)`. 3. `_sanitize_uri` does not strip userinfo (credentials in URL path). 4. `_parse_retry_ms` misinterprets epoch-style `x-ratelimit-reset` headers. 5. 429 responses without `Retry-After` synthesize a fabricated "Retry after 1 second(s)." suffix. 6. `UPSTREAM_RUNTIME_VALIDATION_ERROR` is defined but never emitted. 7. `UpstreamError` silently accepts out-of-range status codes. 8. `requests.HTTPError` branch re-extracts `request_url` / `request_method` inconsistently (dead work). ## Test plan - [x] Existing `libs/tests/sdk/test_httpx_adapter.py` + `test_graphql_adapter.py` updated; every no-response / construction-bug test asserts the new class + kind + `can_retry`. - [x] Full test suite passes locally. - [x] mypy clean on `arcade-core`, `arcade-tdk`, `arcade-mcp-server`. - [x] Smoke-tested 21 exception routing cases end-to-end against real httpx / requests exceptions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes core error classification and retryability for `httpx`/`requests`/GraphQL transport failures, which can affect tool retry behavior and telemetry. Risk is mitigated by extensive new/updated tests covering the new mappings and privacy expectations. > > **Overview** > **Improves error adapter behavior to be more semantically correct and privacy-safe.** The HTTP adapter now distinguishes real HTTP responses (`UpstreamError`/`UpstreamRateLimitError`) from no-response failures (`NetworkTransportError` with `ErrorKind` + retryability) and from client construction/local TLS issues (`FatalToolError`). > > **Reduces sensitive data exposure in agent-facing messages.** Status-based errors now emit standardized messages derived from status phrase/class, while preserving raw exception detail in `developer_message`; Google/Microsoft/Slack fallback paths similarly switch to `unhandled <ExceptionType>` messages and move `str(exc)` into `developer_message`. GraphQL transport connection/protocol errors are reclassified from `UpstreamError` (502) to `NetworkTransportError`, and transport/server messages are standardized. > > Bumps `arcade-tdk` version to `3.8.0` and expands/updates the SDK test suite to assert new classes, `kind`, `can_retry`, request metadata extraction, and privacy behavior. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 1041cb1bec4fa3b0bae3e7c6b860b84cf376cf9a. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
194 lines
6.9 KiB
Python
194 lines
6.9 KiB
Python
import importlib
|
|
import logging
|
|
from functools import lru_cache
|
|
from http import HTTPStatus
|
|
from typing import Any
|
|
|
|
from arcade_core.errors import (
|
|
ErrorKind,
|
|
NetworkTransportError,
|
|
ToolRuntimeError,
|
|
UpstreamError,
|
|
)
|
|
|
|
from arcade_tdk.providers.http.error_adapter import BaseHTTPErrorMapper
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# Standard Apollo/GraphQL error codes mapped to HTTP status codes
|
|
_GQL_CODE_TO_STATUS = {
|
|
"UNAUTHENTICATED": 401,
|
|
"NOT_AUTHENTICATED": 401,
|
|
"FORBIDDEN": 403,
|
|
"ACCESS_DENIED": 403,
|
|
"NOT_FOUND": 404,
|
|
"BAD_USER_INPUT": 400,
|
|
"GRAPHQL_VALIDATION_FAILED": 400,
|
|
"GRAPHQL_PARSE_FAILED": 400,
|
|
"INTERNAL_SERVER_ERROR": 500,
|
|
}
|
|
|
|
|
|
@lru_cache(maxsize=1)
|
|
def _load_gql_transport_errors() -> (
|
|
tuple[type[Any], type[Any], type[Any], type[Any], type[Any]] | None
|
|
):
|
|
"""Import gql transport exceptions lazily and cache the result."""
|
|
try:
|
|
module = importlib.import_module("gql.transport.exceptions")
|
|
except ImportError:
|
|
logger.debug("gql not installed; GraphQL adapter disabled")
|
|
return None
|
|
else:
|
|
return (
|
|
module.TransportError,
|
|
module.TransportQueryError,
|
|
module.TransportServerError,
|
|
module.TransportConnectionFailed,
|
|
module.TransportProtocolError,
|
|
)
|
|
|
|
|
|
def _extract_error_message(message: Any) -> str:
|
|
"""Return the error message or a fallback."""
|
|
if not message:
|
|
return "Unknown GraphQL error"
|
|
try:
|
|
return str(message) or "Unknown GraphQL error"
|
|
except Exception:
|
|
return "Unknown GraphQL error"
|
|
|
|
|
|
class GraphQLErrorAdapter(BaseHTTPErrorMapper):
|
|
"""Error adapter for GraphQL clients (specifically 'gql' library)."""
|
|
|
|
slug = "_graphql"
|
|
|
|
def from_exception(self, exc: Exception) -> ToolRuntimeError | None:
|
|
"""Translate a gql exception into a ToolRuntimeError."""
|
|
gql_types = _load_gql_transport_errors()
|
|
if not gql_types:
|
|
return None
|
|
|
|
(
|
|
TransportError,
|
|
TransportQueryError,
|
|
TransportServerError,
|
|
TransportConnectionFailed,
|
|
TransportProtocolError,
|
|
) = gql_types
|
|
|
|
# GraphQL errors in response (HTTP 200 with errors array)
|
|
if isinstance(exc, TransportQueryError):
|
|
return self._handle_query_error(exc)
|
|
|
|
# HTTP-level errors (4xx, 5xx) - these can have rate limit headers
|
|
if isinstance(exc, TransportServerError):
|
|
return self._handle_transport_error(exc)
|
|
|
|
# Network/protocol errors — the upstream was never reached or never
|
|
# produced a complete response. No HTTP status is available.
|
|
if isinstance(exc, (TransportConnectionFailed, TransportProtocolError)):
|
|
return NetworkTransportError(
|
|
message=("GraphQL request failed before a complete response was received."),
|
|
developer_message=f"{type(exc).__name__}: {exc}",
|
|
kind=ErrorKind.NETWORK_TRANSPORT_RUNTIME_UNREACHABLE,
|
|
can_retry=True,
|
|
extra={"service": self.slug, "error_type": type(exc).__name__},
|
|
)
|
|
|
|
# Catch-all for unknown TransportError subclasses
|
|
if isinstance(exc, TransportError):
|
|
return self._handle_transport_error(exc)
|
|
|
|
return None
|
|
|
|
def _handle_query_error(self, exc: Any) -> UpstreamError:
|
|
"""Handle TransportQueryError (GraphQL errors in response body)."""
|
|
errors_list = exc.errors or []
|
|
logger.debug("GraphQL query errors: %s", errors_list)
|
|
|
|
messages = [_extract_error_message(e.get("message")) for e in errors_list]
|
|
joined = "; ".join(messages) if messages else "Unknown GraphQL error"
|
|
|
|
# Extract error codes and map to HTTP status
|
|
codes: list[str] = []
|
|
status = HTTPStatus.UNPROCESSABLE_ENTITY.value
|
|
|
|
for e in errors_list:
|
|
ext = e.get("extensions") if isinstance(e, dict) else None
|
|
code = ext.get("code") if isinstance(ext, dict) else None
|
|
if isinstance(code, str):
|
|
codes.append(code)
|
|
mapped = _GQL_CODE_TO_STATUS.get(code)
|
|
if mapped and mapped > status:
|
|
status = mapped
|
|
|
|
unique_codes = sorted(set(codes))
|
|
|
|
return UpstreamError(
|
|
message=f"Upstream GraphQL error: {joined}",
|
|
status_code=status,
|
|
developer_message=f"GraphQL error codes: {', '.join(unique_codes)}"
|
|
if unique_codes
|
|
else "GraphQL error",
|
|
extra={
|
|
"service": self.slug,
|
|
"error_type": "TransportQueryError",
|
|
"gql_error_codes": unique_codes,
|
|
},
|
|
)
|
|
|
|
def _handle_transport_error(self, exc: Any) -> UpstreamError:
|
|
"""Handle TransportServerError and other transport errors."""
|
|
status = getattr(exc, "code", None)
|
|
if not isinstance(status, int):
|
|
status = HTTPStatus.INTERNAL_SERVER_ERROR.value
|
|
|
|
# Extract headers for rate limit detection (check exc and __cause__)
|
|
headers = self._get_headers(exc) or self._get_headers(exc.__cause__)
|
|
|
|
# Extract URL from __cause__ (aiohttp/httpx/requests store it there)
|
|
url, method = self._get_request_info(exc.__cause__)
|
|
|
|
return self._map_status_to_error(
|
|
status=status,
|
|
headers=headers or {},
|
|
msg=f"Upstream GraphQL request failed with status code {status}.",
|
|
developer_message=str(exc),
|
|
request_url=url,
|
|
request_method=method,
|
|
)
|
|
|
|
def _get_headers(self, obj: Any) -> dict[str, str] | None:
|
|
"""Extract headers from an object if available."""
|
|
if obj and hasattr(obj, "response") and hasattr(obj.response, "headers"):
|
|
return {k.lower(): v for k, v in obj.response.headers.items()}
|
|
return None
|
|
|
|
def _get_request_info(self, cause: Any) -> tuple[str | None, str | None]:
|
|
"""Extract URL and method from the __cause__ exception."""
|
|
if not cause:
|
|
return None, None
|
|
|
|
# aiohttp: request_info.url
|
|
if hasattr(cause, "request_info"):
|
|
ri = cause.request_info
|
|
url = getattr(ri, "url", None) or getattr(ri, "real_url", None)
|
|
return (str(url), getattr(ri, "method", None)) if url else (None, None)
|
|
|
|
# httpx/requests: response.request.url
|
|
if hasattr(cause, "response") and hasattr(cause.response, "request"):
|
|
req = cause.response.request
|
|
url = getattr(req, "url", None)
|
|
return (str(url), getattr(req, "method", None)) if url else (None, None)
|
|
|
|
return None, None
|
|
|
|
def _build_extra_metadata(
|
|
self, request_url: str | None = None, request_method: str | None = None
|
|
) -> dict[str, str]:
|
|
"""Override to use GraphQL service slug."""
|
|
extra = super()._build_extra_metadata(request_url, request_method)
|
|
extra["service"] = self.slug
|
|
return extra
|