arcade-mcp/libs/arcade-cli/arcade_cli/secret.py
Eric Gustin 3424ec8219
MCP Local (#563)
Versions:
* arcade-mcp\==1.0.0rc1
* arcade-mcp-server\==1.0.0rc1
* arcade-core\==2.5.0rc1
* arcade-tdk\==2.6.0rc1
* arcade-serve\==2.2.0rc1

### Summary
Adds first-class MCP support across Arcade, introduces a new MCP server
and CLI, unifies the project under the arcade-mcp name, overhauls
templates/scaffolding, and improves developer tooling, secrets
management, and examples.

### Highlights
- **MCP Server & Core**
- New MCP server with stdio and HTTP/SSE transports, session management,
resumability, and lifecycle handling.
- FastAPI-like `MCPApp` for building servers with lazy init; integrated
worker+MCP HTTP app option.
- Middleware system (logging and error handling), robust exception
hierarchy, and Pydantic-based settings.
- Async-safe managers for tools, resources, and prompts backed by
registries and locks.
- Developer-facing, transport-agnostic runtime context interfaces (logs,
tools, prompts, resources, sampling, UI, notifications).
- Conversion from Arcade ToolDefinition to MCP tool schema; OpenAI JSON
tool schema converter.
  - Parser supports `@app.tool`/`@app.tool(...)` decorators.

- **CLI**
  - New `mcp` command to run MCP servers with stdio or HTTP/SSE.
- New `secret` command to set/list/unset tool secrets (supports .env
input, preserves original casing for lookups).
- `new` command refactored; option to create a full toolkit package with
scaffolding.
  - `chat` command removed.
- `serve.py` imports updated to `arcade_serve.fastapi.telemetry`;
version retrieval now uses `arcade-mcp`.
  - `show.py` refactor to use new local catalog utilities.
- `display_tool_details` improved: adds “Default” column and handles
nested properties.

- **Configuration & Discovery**
- New `configure.py` to set up Claude Desktop, Cursor, and VS Code to
connect to local or Arcade Cloud MCP servers.
- Discovery utilities to find/install toolkits, build `ToolCatalog`s,
analyze files for tools, load kits from directories (pyproject parsing),
and build minimal toolkits.
- Better handling of provider API key resolution and evaluation suite
loading.

- **Templates & Scaffolding**
- Reorganized template structure (minimal vs full); moved
`.pre-commit-config.yaml`, `.ruff.toml`, license, Makefile, README,
tests, and tools layout to correct paths.
  - Minimal template adds `.env.example` for runtime secret injection.
- Template pyproject updated for MCP servers; includes sample server
with greeting and secret-reveal tools.
  - Authorization flow in templates simplified.

- **Repo-wide Renaming & Examples**
- Migrates references from `arcade-ai` to `arcade-mcp` across READMEs,
scripts, and package metadata.
- Examples updated (LangChain/LangGraph/AI SDK/TypeScript) and package
name changed to `arcade-mcp-sdk`.

- **Evals & Core Utilities**
- Evals now use OpenAI tooling format (`OpenAIToolList`, `to_openai`);
`tool_eval` takes `provider_api_key`.
- Core utilities: fixed `does_function_return_value` by dedenting before
parse; version bump to `2.5.0rc1` and dependency cleanup.

- **Tooling & CI**
- `setup-uv-env` action splits toolkit vs contrib dependency
installation.
- Pre-commit: excludes `libs/arcade-mcp-server/mkdocs.yml` and
`libs/tests/` from YAML and Ruff hooks; Ruff per-file ignores (e.g.,
C901 in `libs/**/*.py`, TRY400 in server docs paths).
- Makefile updates for uv env setup, quality checks, tests, builds, and
new `shell` target.
  - Added Makefile to MCP server library to streamline dev workflow.

- **Cleanup**
  - Removed `claude.json` config.
- Simplified stdio entrypoint; removed unused imports (`arcade_gmail`,
`arcade_search`).

### Breaking Changes
- **CLI**: `chat` command removed; use `mcp`, `secret`, and updated
`new`.
- **Naming**: All users should update references from `arcade-ai` to
`arcade-mcp`.
- **Templates**: File paths moved; downstream scripts referencing old
template locations may need updates.

### Getting Started
- Run an MCP server:
  - `arcade mcp --stdio --toolkits your_toolkit`
  - `arcade mcp --http --toolkits your_toolkit`
- Manage secrets:
  - `arcade secret set your_toolkit KEY=value`
  - `arcade secret list your_toolkit`
  - `arcade secret unset your_toolkit KEY`
- Configure clients:
- `arcade configure` to set up Claude Desktop, Cursor, and VS Code for
local/Arcade Cloud MCP.

---------

Co-authored-by: Sam Partee <sam@arcade-ai.com>
Co-authored-by: Shub <125150494+shubcodes@users.noreply.github.com>
2025-09-25 15:28:15 -07:00

286 lines
8.9 KiB
Python

import httpx
import typer
from rich.console import Console
from rich.table import Table
from arcade_cli.constants import (
PROD_ENGINE_HOST,
)
from arcade_cli.utils import (
OrderCommands,
compute_base_url,
validate_and_get_config,
)
console = Console()
app = typer.Typer(
cls=OrderCommands,
add_completion=False,
no_args_is_help=True,
pretty_exceptions_enable=False,
pretty_exceptions_show_locals=False,
pretty_exceptions_short=True,
)
state = {
"engine_url": compute_base_url(
host=PROD_ENGINE_HOST, port=None, force_tls=False, force_no_tls=False
)
}
@app.callback()
def main(
host: str = typer.Option(
PROD_ENGINE_HOST,
"--host",
"-h",
help="The Arcade Engine host.",
),
port: int = typer.Option(
None,
"--port",
"-p",
help="The port of the Arcade Engine host.",
),
force_tls: bool = typer.Option(
False,
"--tls",
help="Whether to force TLS for the connection to the Arcade Engine.",
),
force_no_tls: bool = typer.Option(
False,
"--no-tls",
help="Whether to disable TLS for the connection to the Arcade Engine.",
),
) -> None:
"""
Manage tool secrets in Arcade Cloud.
Usage:
arcade secret set KEY1=value1 KEY2="value 2"
arcade secret set --from-env
arcade secret set -from-env --env-file /path/to/.env
arcade secret list
arcade secret unset KEY1 KEY2 KEY3
"""
engine_url = compute_base_url(force_tls, force_no_tls, host, port)
state["engine_url"] = engine_url
@app.command("set", help="Set tool secret(s) using KEY=VALUE pairs or from .env file")
def set_secret(
key_value_pairs: list[str] = typer.Argument(
None,
help="Key-value pairs in the format KEY=VALUE",
),
from_env: bool = typer.Option(
False,
"--from-env",
help="Load all secrets from local .env file",
),
env_file: str = typer.Option(
".env",
"--env-file",
"-f",
help="Path to .env file (default: .env)",
),
) -> None:
"""Set secrets either from .env file or KEY=VALUE pairs."""
if not from_env and not key_value_pairs:
raise typer.BadParameter(
"Either provide KEY=VALUE pairs or use --from-env to load from .env file."
)
if from_env and key_value_pairs:
raise typer.BadParameter("Cannot use both KEY=VALUE pairs and --from-env at the same time.")
config = validate_and_get_config()
if from_env:
secrets = load_env_file(env_file)
else:
secrets = {}
for pair in key_value_pairs:
if (
"=" not in pair
or pair.split("=", 1)[0].strip() == ""
or pair.split("=", 1)[1].strip() == ""
):
raise typer.BadParameter(f"Invalid format '{pair}'. Expected KEY=VALUE")
key, value = pair.split("=", 1)
key = key.strip()
if " " in key:
raise typer.BadParameter(f"Secret key '{key}' cannot contain spaces")
value = value # keep the value as is, including the whitespace
secrets[key] = value
engine_url = state["engine_url"]
for secret_key, secret_value in secrets.items():
try:
_upsert_secret_to_engine(engine_url, config.api.key, secret_key, secret_value)
except Exception as e:
console.print(f"Error setting secret '{secret_key}': {e}", style="bold red")
continue
console.print(
f"Secret '{secret_key}' with value ending in ...{secret_value[-4:]} set successfully"
)
@app.command("list", help="List all tool secrets in Arcade Cloud")
def list_secrets() -> None:
"""List all secrets (keys only, values are masked)."""
config = validate_and_get_config()
engine_url = state["engine_url"]
secrets = _get_secrets_from_engine(engine_url, config.api.key)
print_secret_table(secrets)
@app.command("unset", help="Delete tool secret(s) by key names")
def unset_secret(
keys: list[str] = typer.Argument(
...,
help="Secret keys to delete",
),
) -> None:
"""Delete tool secrets."""
config = validate_and_get_config()
engine_url = state["engine_url"]
secrets = _get_secrets_from_engine(engine_url, config.api.key)
key_to_id = {secret["key"]: secret["id"] for secret in secrets}
for key in set(keys):
secret_id = key_to_id.get(key)
if not secret_id:
console.print(f"Warning: Secret with key '{key}' not found, skipping", style="yellow")
continue
try:
_delete_secret_from_engine(engine_url, config.api.key, secret_id)
console.print(f"Secret '{key}' deleted successfully")
except Exception:
console.print(
f"Failed to delete secret '{key}'. Do you have permission to delete this secret?",
style="bold red",
)
continue
def print_secret_table(secrets: list[dict]) -> None:
"""Print a table of tool secrets (with masked values)."""
table = Table(title="Tool Secrets")
table.add_column("Key", style="cyan")
table.add_column("Type", style="green")
table.add_column("Description", style="green")
table.add_column("Hint", style="green")
table.add_column("Last Accessed", style="green")
table.add_column("Created At", style="green")
for secret in secrets:
table.add_row(
secret["key"],
secret["binding"]["type"],
secret["description"],
"..." + secret["hint"] if secret["hint"] else "-",
secret["last_accessed_at"] if secret["last_accessed_at"] else "Never",
secret["created_at"],
)
console.print(table)
def load_env_file(env_file_path: str) -> dict[str, str]:
"""Load tool secrets from a .env file."""
secrets = {}
with open(env_file_path) as file:
for line in file:
line = line.strip()
if line.startswith("#") or not line:
continue
# Split on first '=' to handle values that contain '='
if "=" not in line:
continue
key, value = line.split("=", 1)
key = key.strip()
# Remove inline comments, but respect quoted values
value = _remove_inline_comment(value)
value = value.strip()
# Skip entries with empty keys or empty values
if not key or not value:
continue
secrets[key] = value
return secrets
def _remove_inline_comment(value: str) -> str:
"""Remove inline comments from env value, respecting quoted strings."""
value = value.strip()
# Check if value starts with a quote
if value.startswith('"') or value.startswith("'"):
quote_char = value[0]
# Find the matching closing quote (not escaped)
i = 1
while i < len(value):
if value[i] == quote_char:
# Found potential closing quote
# Check if there's anything after it
remaining = value[i + 1 :]
comment_idx = remaining.find(" #")
if comment_idx != -1:
# Remove the comment part and strip quotes
quoted_value = value[: i + 1]
return quoted_value[1:-1] # Remove surrounding quotes
else:
# No comment after closing quote, strip quotes
quoted_value = value[: i + 1]
return quoted_value[1:-1] # Remove surrounding quotes
i += 1
# No closing quote, treat as unquoted
comment_idx = value.find(" #")
if comment_idx != -1:
return value[:comment_idx]
return value
else:
# For unquoted values, remove everything after ' #'
comment_idx = value.find(" #")
if comment_idx != -1:
return value[:comment_idx]
return value
def _upsert_secret_to_engine(
engine_url: str, api_key: str, secret_id: str, secret_value: str
) -> None:
response = httpx.put(
f"{engine_url}/v1/admin/secrets/{secret_id}",
headers={"Authorization": f"Bearer {api_key}"},
json={"description": "Secret set via CLI", "value": secret_value},
)
response.raise_for_status()
def _get_secrets_from_engine(engine_url: str, api_key: str) -> list[dict]:
response = httpx.get(
f"{engine_url}/v1/admin/secrets",
headers={"Authorization": f"Bearer {api_key}"},
)
response.raise_for_status()
return response.json()["items"] # type: ignore[no-any-return]
def _delete_secret_from_engine(engine_url: str, api_key: str, secret_id: str) -> None:
response = httpx.delete(
f"{engine_url}/v1/admin/secrets/{secret_id}",
headers={"Authorization": f"Bearer {api_key}"},
)
response.raise_for_status()