Commit graph

2 commits

Author SHA1 Message Date
Francisco Or Something
c866620435
fix(arcade-mcp-server): report missing debug stacktraces (#836)
## Summary
- Return an explicit `[DEBUG] stacktrace: unavailable ...` note when the
stacktrace debug flag is enabled but the tool error payload has no
stacktrace.
- Preserve existing behavior for real stacktraces and for developer
messages, including not leaking developer details unless the
developer-message flag is enabled.
- Clarify the toolkit-author docs around when stacktraces exist, such as
unhandled exceptions or chained `raise ... from exc` errors.

## Test plan
- `pre-commit run --files CLAUDE.md
libs/arcade-mcp-server/arcade_mcp_server/_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure_integration.py`
- `uv run --with pytest --with pytest-asyncio --with pytest-cov pytest
libs/tests/arcade_mcp_server/test_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure_integration.py -v`
- `ruff format --check
libs/arcade-mcp-server/arcade_mcp_server/_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure_integration.py`
- `ruff check
libs/arcade-mcp-server/arcade_mcp_server/_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure.py
libs/tests/arcade_mcp_server/test_debug_exposure_integration.py`

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Low risk: changes are limited to debug-only error-message augmentation
when an explicit env flag is enabled; default runtime behavior is
unchanged. Main risk is only in local debugging scenarios where the new
note could affect log parsing or expected error text.
> 
> **Overview**
> When `ARCADE_DEBUG_EXPOSE_STACKTRACE_IN_TOOL_ERROR_RESPONSES` is
enabled, tool error messages now **always include a stacktrace debug
section**: either the actual stacktrace (when present) or an explicit
`[DEBUG] stacktrace: unavailable ...` note when the tool error payload
had no stacktrace.
> 
> Adds/updates unit + integration coverage for the missing-stacktrace
case and adjusts expectations around “flag enabled but no content.”
Updates toolkit-author docs to clarify when stacktraces exist, and bumps
`arcade-mcp-server` patch version to `1.21.2`.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
7d85196a30d8d29be98ffb252a13ef2a78057742. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-04-30 20:03:53 -03:00
Francisco Or Something
70515e3356
feat(arcade-core): opt-in debug leak flags for toolkit authors (#826)
## Summary

Adds two strictly opt-in env vars that let toolkit developers see
`developer_message` / `stacktrace` content *in* the agent-facing error
message while debugging. Off by default; activation requires a specific
acknowledgement string, not a boolean — `true`/`1` is explicitly
rejected with a warning log.

- `ARCADE_UNSAFE_DEBUG_LEAK_DEVELOPER_MESSAGE_TO_AGENT`
- `ARCADE_UNSAFE_DEBUG_LEAK_STACKTRACE_TO_AGENT`
- Magic ack: `yes-i-accept-leaking-internals-to-the-agent`

Everything goes through a single funnel — `ToolOutputFactory.fail` /
`fail_retry` in `arcade_core/output.py` — so the behavior covers both
the MCP server path and the Arcade Worker path with no call-site
changes. A loud `logger.warning` fires once per process on activation,
and a big header comment in `output.py` tells future maintainers not to
add more flags of this shape (debug info belongs in `logger.debug`, not
in a field that gets shipped to the model and often to end users).

Bumps `arcade-core` 4.6.2 → 4.7.0. Non-breaking, additive.

## Why

Today the project does a lot of work to keep `developer_message` and
`stacktrace` off the agent's context. That's the right default, but it
makes iterating on a new toolkit painful — you end up adding temporary
logging or rebuilds just to see what blew up. This gives toolkit authors
a safe, ugly, loud-on-activation escape hatch.

## Safety design

- Two separate flags so you only leak what you need.
- Magic string (not a boolean) activates the flag. Boolean-style values
are rejected and log a pointer to `output.py`.
- First activation logs a `WARNING` identifying the flag and the risk.
- Flags documented only in `CLAUDE.md`, not in the public README.
- Top-of-file banner in `output.py` explicitly tells maintainers not to
add more flags of this shape.

## Test plan

- [x] Existing test suite passes (1154 tests —
`libs/tests/{core,tool,arcade_mcp_server}`).
- [x] End-to-end smoke test against the built `arcade_core-4.7.0` wheel,
driven through `ToolExecutor.run` (same path toolkits hit). Covered
cases:
  - flags off → message unchanged
- `ARCADE_UNSAFE_..._DEVELOPER_MESSAGE_TO_AGENT=true` → flag rejected,
warning logged, message unchanged
- `ARCADE_UNSAFE_..._DEVELOPER_MESSAGE_TO_AGENT=<magic>` → `[DEBUG]
developer_message: ...` appended
- both flags with magic, `ToolRuntimeError` path → developer_message
appended (stacktrace absent because `ToolRuntimeError.stacktrace()`
returned `None`, which is existing behavior)
- stacktrace flag with magic, generic `Exception` path → full
`traceback.format_exc()` appended, activation `WARNING` visible

Made with [Cursor](https://cursor.com)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Adds an opt-in path to include `developer_message` and stacktraces in
agent-facing MCP error messages, which could leak sensitive data if
misconfigured; safeguards (magic ack string + CI/pre-commit guard)
reduce but don’t eliminate risk.
> 
> **Overview**
> Adds `arcade_mcp_server/_debug_exposure.py` with two env-gated debug
flags that, only when set to a specific acknowledgement string, append
`developer_message` and/or `stacktrace` into the agent-visible MCP tool
error `message` (and logs one-shot warnings on rejection/activation).
> 
> Wires this into the MCP error path in `MCPServer._handle_call_tool`,
documents the flags in `CLAUDE.md`, bumps `arcade-mcp-server` to
`1.21.0`, and adds unit + integration tests plus a pre-commit hook and
GitHub Actions workflow (`scripts/check_debug_leak_flags_off.py`) to
ensure the magic ack string can’t be committed outside a small
allowlist.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
30e242c454128ec7cc62e169c2afd116be735cb5. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-04-25 11:40:26 -03:00