From 9952d2667e4de27c2d6a5744120edf1f83b68fbd Mon Sep 17 00:00:00 2001 From: Dare Date: Wed, 12 Mar 2025 04:43:50 -0700 Subject: [PATCH 01/15] Adding link to docs on Scorecard AgentSDK Support Incredibly excited to add support the agentsdk for Scorecard tracing! --- README.md | 2 +- docs/tracing.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 90fea50..e370419 100644 --- a/README.md +++ b/README.md @@ -140,7 +140,7 @@ The Agents SDK is designed to be highly flexible, allowing you to model a wide r ## Tracing -The Agents SDK automatically traces your agent runs, making it easy to track and debug the behavior of your agents. Tracing is extensible by design, supporting custom spans and a wide variety of external destinations, including [Logfire](https://logfire.pydantic.dev/docs/integrations/llms/openai/#openai-agents), [AgentOps](https://docs.agentops.ai/v1/integrations/agentssdk), and [Braintrust](https://braintrust.dev/docs/guides/traces/integrations#openai-agents-sdk). For more details about how to customize or disable tracing, see [Tracing](http://openai.github.io/openai-agents-python/tracing). +The Agents SDK automatically traces your agent runs, making it easy to track and debug the behavior of your agents. Tracing is extensible by design, supporting custom spans and a wide variety of external destinations, including [Logfire](https://logfire.pydantic.dev/docs/integrations/llms/openai/#openai-agents), [AgentOps](https://docs.agentops.ai/v1/integrations/agentssdk), [Braintrust](https://braintrust.dev/docs/guides/traces/integrations#openai-agents-sdk), and [Scorecard](https://docs.scorecard.io/docs/documentation/features/tracing#openai-agents-sdk-integration). For more details about how to customize or disable tracing, see [Tracing](http://openai.github.io/openai-agents-python/tracing). ## Development (only needed if you need to edit the SDK/examples) diff --git a/docs/tracing.md b/docs/tracing.md index da0d536..070677f 100644 --- a/docs/tracing.md +++ b/docs/tracing.md @@ -93,3 +93,4 @@ External trace processors include: - [Braintrust](https://braintrust.dev/docs/guides/traces/integrations#openai-agents-sdk) - [Pydantic Logfire](https://logfire.pydantic.dev/docs/integrations/llms/openai/#openai-agents) - [AgentOps](https://docs.agentops.ai/v1/integrations/agentssdk) +- [Scorecard] (https://docs.scorecard.io/docs/documentation/features/tracing#openai-agents-sdk-integration)) From 6010bd49cf78276004a9b8e2e5fcad94dd9ad866 Mon Sep 17 00:00:00 2001 From: Dare Date: Wed, 12 Mar 2025 04:46:05 -0700 Subject: [PATCH 02/15] Update tracing.md --- docs/tracing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tracing.md b/docs/tracing.md index 070677f..39a843f 100644 --- a/docs/tracing.md +++ b/docs/tracing.md @@ -93,4 +93,4 @@ External trace processors include: - [Braintrust](https://braintrust.dev/docs/guides/traces/integrations#openai-agents-sdk) - [Pydantic Logfire](https://logfire.pydantic.dev/docs/integrations/llms/openai/#openai-agents) - [AgentOps](https://docs.agentops.ai/v1/integrations/agentssdk) -- [Scorecard] (https://docs.scorecard.io/docs/documentation/features/tracing#openai-agents-sdk-integration)) +- [Scorecard](https://docs.scorecard.io/docs/documentation/features/tracing#openai-agents-sdk-integration)) From 9f938716d183f80a30904918f27314eb5f5dfbab Mon Sep 17 00:00:00 2001 From: Dingkang Wang Date: Wed, 12 Mar 2025 14:54:11 -0700 Subject: [PATCH 03/15] fix typo in tracing.md --- docs/tracing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tracing.md b/docs/tracing.md index fa5d522..357a873 100644 --- a/docs/tracing.md +++ b/docs/tracing.md @@ -50,7 +50,7 @@ async def main(): with trace("Joke workflow"): # (1)! first_result = await Runner.run(agent, "Tell me a joke") - second_result = await Runner.run(agent, f"Rate this joke: {first_output.final_output}") + second_result = await Runner.run(agent, f"Rate this joke: {first_result.final_output}") print(f"Joke: {first_result.final_output}") print(f"Rating: {second_result.final_output}") ``` From 5a577f962a2d4516d9a65a9967999d9764fc07e6 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Wed, 12 Mar 2025 16:46:53 -0700 Subject: [PATCH 04/15] Create pull_request_template.md. --- .../pull_request_template.md. | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE/pull_request_template.md. diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md. b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md. new file mode 100644 index 0000000..0fdeab1 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md. @@ -0,0 +1,18 @@ +### Summary + + + +### Test plan + + + +### Issue number + + + +### Checks + +- [ ] I've added new tests (if relevant) +- [ ] I've added/updated the relevant documentation +- [ ] I've run `make lint` and `make format` +- [ ] I've made sure tests pass From 5626cb19512f41339783cb31e66fe87891c30f56 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Wed, 12 Mar 2025 16:47:14 -0700 Subject: [PATCH 05/15] Rename pull_request_template.md. to pull_request_template.md --- .../{pull_request_template.md. => pull_request_template.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .github/PULL_REQUEST_TEMPLATE/{pull_request_template.md. => pull_request_template.md} (100%) diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md. b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md similarity index 100% rename from .github/PULL_REQUEST_TEMPLATE/pull_request_template.md. rename to .github/PULL_REQUEST_TEMPLATE/pull_request_template.md From 2302b478af18b9d1bdbb799309c51a9e39533d4f Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Wed, 12 Mar 2025 17:17:07 -0700 Subject: [PATCH 06/15] Fix streaming in chat completions --- src/agents/models/openai_chatcompletions.py | 20 +++++++++++++++++++- tests/test_openai_chatcompletions_stream.py | 5 +++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/src/agents/models/openai_chatcompletions.py b/src/agents/models/openai_chatcompletions.py index 50e27fc..3543225 100644 --- a/src/agents/models/openai_chatcompletions.py +++ b/src/agents/models/openai_chatcompletions.py @@ -51,8 +51,10 @@ from openai.types.responses import ( ResponseOutputText, ResponseRefusalDeltaEvent, ResponseTextDeltaEvent, + ResponseUsage, ) from openai.types.responses.response_input_param import FunctionCallOutput, ItemReference, Message +from openai.types.responses.response_usage import OutputTokensDetails from .. import _debug from ..agent_output import AgentOutputSchema @@ -405,7 +407,23 @@ class OpenAIChatCompletionsModel(Model): for function_call in state.function_calls.values(): outputs.append(function_call) - final_response = response.model_copy(update={"output": outputs, "usage": usage}) + final_response = response.model_copy() + final_response.output = outputs + final_response.usage = ( + ResponseUsage( + input_tokens=usage.prompt_tokens, + output_tokens=usage.completion_tokens, + total_tokens=usage.total_tokens, + output_tokens_details=OutputTokensDetails( + reasoning_tokens=usage.completion_tokens_details.reasoning_tokens + if usage.completion_tokens_details + and usage.completion_tokens_details.reasoning_tokens + else 0 + ), + ) + if usage + else None + ) yield ResponseCompletedEvent( response=final_response, diff --git a/tests/test_openai_chatcompletions_stream.py b/tests/test_openai_chatcompletions_stream.py index 2a15f7f..7add92a 100644 --- a/tests/test_openai_chatcompletions_stream.py +++ b/tests/test_openai_chatcompletions_stream.py @@ -107,6 +107,11 @@ async def test_stream_response_yields_events_for_text_content(monkeypatch) -> No assert isinstance(completed_resp.output[0].content[0], ResponseOutputText) assert completed_resp.output[0].content[0].text == "Hello" + assert completed_resp.usage, "usage should not be None" + assert completed_resp.usage.input_tokens == 7 + assert completed_resp.usage.output_tokens == 5 + assert completed_resp.usage.total_tokens == 12 + @pytest.mark.allow_call_model_methods @pytest.mark.asyncio From 536d7826d550ed1e295ebeca45386c6f6fd00420 Mon Sep 17 00:00:00 2001 From: Dmitry Pimenov Date: Wed, 12 Mar 2025 17:24:35 -0700 Subject: [PATCH 07/15] added a Jupyter example to clarify how to use the SDK with an existing event loop --- README.md | 4 +++- examples/basic/hello_world_jupyter.py | 11 +++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100644 examples/basic/hello_world_jupyter.py diff --git a/README.md b/README.md index d3f25fa..32984bc 100644 --- a/README.md +++ b/README.md @@ -47,9 +47,11 @@ print(result.final_output) (_If running this, ensure you set the `OPENAI_API_KEY` environment variable_) +(_For Jupyter notebook users, see [hello_world_jupyter.py](examples/basic/hello_world_jupyter.py)_) + ## Handoffs example -```py +```python from agents import Agent, Runner import asyncio diff --git a/examples/basic/hello_world_jupyter.py b/examples/basic/hello_world_jupyter.py new file mode 100644 index 0000000..6b0f275 --- /dev/null +++ b/examples/basic/hello_world_jupyter.py @@ -0,0 +1,11 @@ +from agents import Agent, Runner + +agent = Agent(name="Assistant", instructions="You are a helpful assistant") + +# Intended for Jupyter notebooks where there's an existing event loop +result = await Runner.run(agent, "Write a haiku about recursion in programming.") +print(result.final_output) + +# Code within code loops, +# Infinite mirrors reflect— +# Logic folds on self. From 65546703cbd768b411972cc06e42116aea059a5a Mon Sep 17 00:00:00 2001 From: Dmitry Pimenov Date: Wed, 12 Mar 2025 17:28:44 -0700 Subject: [PATCH 08/15] fixing lint issues --- examples/basic/hello_world_jupyter.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/examples/basic/hello_world_jupyter.py b/examples/basic/hello_world_jupyter.py index 6b0f275..5c1e302 100644 --- a/examples/basic/hello_world_jupyter.py +++ b/examples/basic/hello_world_jupyter.py @@ -3,9 +3,9 @@ from agents import Agent, Runner agent = Agent(name="Assistant", instructions="You are a helpful assistant") # Intended for Jupyter notebooks where there's an existing event loop -result = await Runner.run(agent, "Write a haiku about recursion in programming.") +result = await Runner.run(agent, "Write a haiku about recursion in programming.") # noqa: F704 print(result.final_output) -# Code within code loops, -# Infinite mirrors reflect— +# Code within code loops, +# Infinite mirrors reflect— # Logic folds on self. From 26f9cb42de9791ba43ee961722909b342f331d9d Mon Sep 17 00:00:00 2001 From: Dmitry Pimenov Date: Wed, 12 Mar 2025 17:30:59 -0700 Subject: [PATCH 09/15] fixing mypy error --- examples/basic/hello_world_jupyter.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/basic/hello_world_jupyter.py b/examples/basic/hello_world_jupyter.py index 5c1e302..bb8f14c 100644 --- a/examples/basic/hello_world_jupyter.py +++ b/examples/basic/hello_world_jupyter.py @@ -3,7 +3,7 @@ from agents import Agent, Runner agent = Agent(name="Assistant", instructions="You are a helpful assistant") # Intended for Jupyter notebooks where there's an existing event loop -result = await Runner.run(agent, "Write a haiku about recursion in programming.") # noqa: F704 +result = await Runner.run(agent, "Write a haiku about recursion in programming.") # type: ignore[top-level-await] # noqa: F704 print(result.final_output) # Code within code loops, From 25a633139dddef3e0bccd44d0553cc3d1a277072 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Wed, 12 Mar 2025 16:06:17 -0700 Subject: [PATCH 10/15] Add examples and documentation for using custom model providers --- docs/models.md | 25 +++---- examples/model_providers/README.md | 19 +++++ .../model_providers/custom_example_agent.py | 51 +++++++++++++ .../model_providers/custom_example_global.py | 55 ++++++++++++++ .../custom_example_provider.py | 73 +++++++++++++++++++ src/agents/__init__.py | 14 +++- src/agents/_config.py | 9 ++- src/agents/models/openai_provider.py | 35 ++++++--- 8 files changed, 247 insertions(+), 34 deletions(-) create mode 100644 examples/model_providers/README.md create mode 100644 examples/model_providers/custom_example_agent.py create mode 100644 examples/model_providers/custom_example_global.py create mode 100644 examples/model_providers/custom_example_provider.py diff --git a/docs/models.md b/docs/models.md index 7ad515b..a4801fe 100644 --- a/docs/models.md +++ b/docs/models.md @@ -53,21 +53,14 @@ async def main(): ## Using other LLM providers -Many providers also support the OpenAI API format, which means you can pass a `base_url` to the existing OpenAI model implementations and use them easily. `ModelSettings` is used to configure tuning parameters (e.g., temperature, top_p) for the model you select. +You can use other LLM providers in 3 ways (examples [here](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/)): -```python -external_client = AsyncOpenAI( - api_key="EXTERNAL_API_KEY", - base_url="https://api.external.com/v1/", -) +1. [`set_default_openai_client`][agents.set_default_openai_client] is useful in cases where you want to globally use an instance of `AsyncOpenAI` as the LLM client. This is for cases where the LLM provider has an OpenAI compatible API endpoint, and you can set the `base_url` and `api_key`. See a configurable example in [examples/model_providers/custom_example_global.py](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/custom_example_global.py). +2. [`ModelProvider`][agents.models.interface.ModelProvider] is at the `Runner.run` level. This lets you say "use a custom model provider for all agents in this run". See a configurable example in [examples/model_providers/custom_example_provider.py](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/custom_example_provider.py). +3. [`Agent.model`][agents.agent.Agent.model] lets you specify the model on a specific Agent instance. This enables you to mix and match different providers for different agents. See a configurable example in [examples/model_providers/custom_example_agent.py](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/custom_example_agent.py). -spanish_agent = Agent( - name="Spanish agent", - instructions="You only speak Spanish.", - model=OpenAIChatCompletionsModel( - model="EXTERNAL_MODEL_NAME", - openai_client=external_client, - ), - model_settings=ModelSettings(temperature=0.5), -) -``` +In cases where you do not have an API key from `platform.openai.com`, we recommend disabling tracing via `set_tracing_disabled()`, or setting up a [different tracing processor](tracing.md). + +!!! note + + In these examples, we use the Chat Completions API/model, because most LLM providers don't yet support the Responses API. If your LLM provider does support it, we recommend using Responses. diff --git a/examples/model_providers/README.md b/examples/model_providers/README.md new file mode 100644 index 0000000..f9330c2 --- /dev/null +++ b/examples/model_providers/README.md @@ -0,0 +1,19 @@ +# Custom LLM providers + +The examples in this directory demonstrate how you might use a non-OpenAI LLM provider. To run them, first set a base URL, API key and model. + +```bash +export EXAMPLE_BASE_URL="..." +export EXAMPLE_API_KEY="..." +export EXAMPLE_MODEL_NAME"..." +``` + +Then run the examples, e.g.: + +``` +python examples/model_providers/custom_example_provider.py + +Loops within themselves, +Function calls its own being, +Depth without ending. +``` diff --git a/examples/model_providers/custom_example_agent.py b/examples/model_providers/custom_example_agent.py new file mode 100644 index 0000000..d7519a5 --- /dev/null +++ b/examples/model_providers/custom_example_agent.py @@ -0,0 +1,51 @@ +import asyncio +import os + +from openai import AsyncOpenAI + +from agents import Agent, OpenAIChatCompletionsModel, Runner, set_tracing_disabled + +BASE_URL = os.getenv("EXAMPLE_BASE_URL") or "" +API_KEY = os.getenv("EXAMPLE_API_KEY") or "" +MODEL_NAME = os.getenv("EXAMPLE_MODEL_NAME") or "" + +if not BASE_URL or not API_KEY or not MODEL_NAME: + raise ValueError( + "Please set EXAMPLE_BASE_URL, EXAMPLE_API_KEY, EXAMPLE_MODEL_NAME via env var or code." + ) + +"""This example uses a custom provider for a specific agent. Steps: +1. Create a custom OpenAI client. +2. Create a `Model` that uses the custom client. +3. Set the `model` on the Agent. + +Note that in this example, we disable tracing under the assumption that you don't have an API key +from platform.openai.com. If you do have one, you can either set the `OPENAI_API_KEY` env var +or call set_tracing_export_api_key() to set a tracing specific key. +""" +client = AsyncOpenAI(base_url=BASE_URL, api_key=API_KEY) +set_tracing_disabled(disabled=True) + +# An alternate approach that would also work: +# PROVIDER = OpenAIProvider(openai_client=client) +# agent = Agent(..., model="some-custom-model") +# Runner.run(agent, ..., run_config=RunConfig(model_provider=PROVIDER)) + + +async def main(): + # This agent will use the custom LLM provider + agent = Agent( + name="Assistant", + instructions="You only respond in haikus.", + model=OpenAIChatCompletionsModel(model=MODEL_NAME, openai_client=client), + ) + + result = await Runner.run( + agent, + "Tell me about recursion in programming.", + ) + print(result.final_output) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/examples/model_providers/custom_example_global.py b/examples/model_providers/custom_example_global.py new file mode 100644 index 0000000..d7c293b --- /dev/null +++ b/examples/model_providers/custom_example_global.py @@ -0,0 +1,55 @@ +import asyncio +import os + +from openai import AsyncOpenAI + +from agents import ( + Agent, + Runner, + set_default_openai_api, + set_default_openai_client, + set_tracing_disabled, +) + +BASE_URL = os.getenv("EXAMPLE_BASE_URL") or "" +API_KEY = os.getenv("EXAMPLE_API_KEY") or "" +MODEL_NAME = os.getenv("EXAMPLE_MODEL_NAME") or "" + +if not BASE_URL or not API_KEY or not MODEL_NAME: + raise ValueError( + "Please set EXAMPLE_BASE_URL, EXAMPLE_API_KEY, EXAMPLE_MODEL_NAME via env var or code." + ) + + +"""This example uses a custom provider for all requests by default. We do three things: +1. Create a custom client. +2. Set it as the default OpenAI client, and don't use it for tracing. +3. Set the default API as Chat Completions, as most LLM providers don't yet support Responses API. + +Note that in this example, we disable tracing under the assumption that you don't have an API key +from platform.openai.com. If you do have one, you can either set the `OPENAI_API_KEY` env var +or call set_tracing_export_api_key() to set a tracing specific key. +""" + +client = AsyncOpenAI( + base_url=BASE_URL, + api_key=API_KEY, +) +set_default_openai_client(client=client, use_for_tracing=False) +set_default_openai_api("chat_completions") +set_tracing_disabled(disabled=True) + + +async def main(): + agent = Agent( + name="Assistant", + instructions="You only respond in haikus.", + model=MODEL_NAME, + ) + + result = await Runner.run(agent, "Tell me about recursion in programming.") + print(result.final_output) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/examples/model_providers/custom_example_provider.py b/examples/model_providers/custom_example_provider.py new file mode 100644 index 0000000..6e8af42 --- /dev/null +++ b/examples/model_providers/custom_example_provider.py @@ -0,0 +1,73 @@ +from __future__ import annotations + +import asyncio +import os + +from openai import AsyncOpenAI + +from agents import ( + Agent, + Model, + ModelProvider, + OpenAIChatCompletionsModel, + RunConfig, + Runner, + set_tracing_disabled, +) + +BASE_URL = os.getenv("EXAMPLE_BASE_URL") or "" +API_KEY = os.getenv("EXAMPLE_API_KEY") or "" +MODEL_NAME = os.getenv("EXAMPLE_MODEL_NAME") or "" + +if not BASE_URL or not API_KEY or not MODEL_NAME: + raise ValueError( + "Please set EXAMPLE_BASE_URL, EXAMPLE_API_KEY, EXAMPLE_MODEL_NAME via env var or code." + ) + + +"""This example uses a custom provider for some calls to Runner.run(), and direct calls to OpenAI for +others. Steps: +1. Create a custom OpenAI client. +2. Create a ModelProvider that uses the custom client. +3. Use the ModelProvider in calls to Runner.run(), only when we want to use the custom LLM provider. + +Note that in this example, we disable tracing under the assumption that you don't have an API key +from platform.openai.com. If you do have one, you can either set the `OPENAI_API_KEY` env var +or call set_tracing_export_api_key() to set a tracing specific key. +""" +client = AsyncOpenAI(base_url=BASE_URL, api_key=API_KEY) +set_tracing_disabled(disabled=True) + + +class CustomModelProvider(ModelProvider): + def get_model(self, model_name: str | None) -> Model: + return OpenAIChatCompletionsModel(model=model_name or MODEL_NAME, openai_client=client) + + +CUSTOM_MODEL_PROVIDER = CustomModelProvider() + + +async def main(): + agent = Agent( + name="Assistant", + instructions="You only respond in haikus.", + ) + + # This will use the custom model provider + result = await Runner.run( + agent, + "Tell me about recursion in programming.", + run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER), + ) + print(result.final_output) + + # If you uncomment this, it will use OpenAI directly, not the custom provider + # result = await Runner.run( + # agent, + # "Tell me about recursion in programming.", + # ) + # print(result.final_output) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/src/agents/__init__.py b/src/agents/__init__.py index 69c500a..79940fe 100644 --- a/src/agents/__init__.py +++ b/src/agents/__init__.py @@ -92,13 +92,19 @@ from .tracing import ( from .usage import Usage -def set_default_openai_key(key: str) -> None: - """Set the default OpenAI API key to use for LLM requests and tracing. This is only necessary if - the OPENAI_API_KEY environment variable is not already set. +def set_default_openai_key(key: str, use_for_tracing: bool = True) -> None: + """Set the default OpenAI API key to use for LLM requests (and optionally tracing(). This is + only necessary if the OPENAI_API_KEY environment variable is not already set. If provided, this key will be used instead of the OPENAI_API_KEY environment variable. + + Args: + key: The OpenAI key to use. + use_for_tracing: Whether to also use this key to send traces to OpenAI. Defaults to True + If False, you'll either need to set the OPENAI_API_KEY environment variable or call + set_tracing_export_api_key() with the API key you want to use for tracing. """ - _config.set_default_openai_key(key) + _config.set_default_openai_key(key, use_for_tracing) def set_default_openai_client(client: AsyncOpenAI, use_for_tracing: bool = True) -> None: diff --git a/src/agents/_config.py b/src/agents/_config.py index 55ded64..304cfb8 100644 --- a/src/agents/_config.py +++ b/src/agents/_config.py @@ -5,15 +5,18 @@ from .models import _openai_shared from .tracing import set_tracing_export_api_key -def set_default_openai_key(key: str) -> None: - set_tracing_export_api_key(key) +def set_default_openai_key(key: str, use_for_tracing: bool) -> None: _openai_shared.set_default_openai_key(key) + if use_for_tracing: + set_tracing_export_api_key(key) + def set_default_openai_client(client: AsyncOpenAI, use_for_tracing: bool) -> None: + _openai_shared.set_default_openai_client(client) + if use_for_tracing: set_tracing_export_api_key(client.api_key) - _openai_shared.set_default_openai_client(client) def set_default_openai_api(api: Literal["chat_completions", "responses"]) -> None: diff --git a/src/agents/models/openai_provider.py b/src/agents/models/openai_provider.py index 5194663..e6a859f 100644 --- a/src/agents/models/openai_provider.py +++ b/src/agents/models/openai_provider.py @@ -38,28 +38,41 @@ class OpenAIProvider(ModelProvider): assert api_key is None and base_url is None, ( "Don't provide api_key or base_url if you provide openai_client" ) - self._client = openai_client + self._client: AsyncOpenAI | None = openai_client else: - self._client = _openai_shared.get_default_openai_client() or AsyncOpenAI( - api_key=api_key or _openai_shared.get_default_openai_key(), - base_url=base_url, - organization=organization, - project=project, - http_client=shared_http_client(), - ) + self._client = None + self._stored_api_key = api_key + self._stored_base_url = base_url + self._stored_organization = organization + self._stored_project = project - self._is_openai_model = self._client.base_url.host.startswith("api.openai.com") if use_responses is not None: self._use_responses = use_responses else: self._use_responses = _openai_shared.get_use_responses_by_default() + # We lazy load the client in case you never actually use OpenAIProvider(). Otherwise + # AsyncOpenAI() raises an error if you don't have an API key set. + def _get_client(self) -> AsyncOpenAI: + if self._client is None: + self._client = _openai_shared.get_default_openai_client() or AsyncOpenAI( + api_key=self._stored_api_key or _openai_shared.get_default_openai_key(), + base_url=self._stored_base_url, + organization=self._stored_organization, + project=self._stored_project, + http_client=shared_http_client(), + ) + + return self._client + def get_model(self, model_name: str | None) -> Model: if model_name is None: model_name = DEFAULT_MODEL + client = self._get_client() + return ( - OpenAIResponsesModel(model=model_name, openai_client=self._client) + OpenAIResponsesModel(model=model_name, openai_client=client) if self._use_responses - else OpenAIChatCompletionsModel(model=model_name, openai_client=self._client) + else OpenAIChatCompletionsModel(model=model_name, openai_client=client) ) From 341198ff928a81c037d29244a479a64ac052a1c7 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Wed, 12 Mar 2025 18:07:16 -0700 Subject: [PATCH 11/15] Add request ID --- src/agents/models/openai_responses.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/agents/models/openai_responses.py b/src/agents/models/openai_responses.py index 86c8215..78765ec 100644 --- a/src/agents/models/openai_responses.py +++ b/src/agents/models/openai_responses.py @@ -5,7 +5,7 @@ from collections.abc import AsyncIterator from dataclasses import dataclass from typing import TYPE_CHECKING, Any, Literal, overload -from openai import NOT_GIVEN, AsyncOpenAI, AsyncStream, NotGiven +from openai import NOT_GIVEN, APIStatusError, AsyncOpenAI, AsyncStream, NotGiven from openai.types import ChatModel from openai.types.responses import ( Response, @@ -113,7 +113,8 @@ class OpenAIResponsesModel(Model): }, ) ) - logger.error(f"Error getting response: {e}") + request_id = e.request_id if isinstance(e, APIStatusError) else None + logger.error(f"Error getting response: {e}. (request_id: {request_id})") raise return ModelResponse( From a012c0d32092161d7d8c69f054a31631ef29b494 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Thu, 13 Mar 2025 11:18:40 -0400 Subject: [PATCH 12/15] v0.0.4 --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index 262ce17..8184a67 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "openai-agents" -version = "0.0.3" +version = "0.0.4" description = "OpenAI Agents SDK" readme = "README.md" requires-python = ">=3.9" From 6ab8c91d23780afc3839880c676e3a60989df8e5 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Thu, 13 Mar 2025 13:10:25 -0400 Subject: [PATCH 13/15] Update custom models to use tools --- examples/model_providers/custom_example_agent.py | 14 +++++++++----- .../model_providers/custom_example_global.py | 10 +++++++++- .../model_providers/custom_example_provider.py | 16 ++++++++++------ uv.lock | 3 +-- 4 files changed, 29 insertions(+), 14 deletions(-) diff --git a/examples/model_providers/custom_example_agent.py b/examples/model_providers/custom_example_agent.py index d7519a5..f10865c 100644 --- a/examples/model_providers/custom_example_agent.py +++ b/examples/model_providers/custom_example_agent.py @@ -3,7 +3,7 @@ import os from openai import AsyncOpenAI -from agents import Agent, OpenAIChatCompletionsModel, Runner, set_tracing_disabled +from agents import Agent, OpenAIChatCompletionsModel, Runner, function_tool, set_tracing_disabled BASE_URL = os.getenv("EXAMPLE_BASE_URL") or "" API_KEY = os.getenv("EXAMPLE_API_KEY") or "" @@ -32,18 +32,22 @@ set_tracing_disabled(disabled=True) # Runner.run(agent, ..., run_config=RunConfig(model_provider=PROVIDER)) +@function_tool +def get_weather(city: str): + print(f"[debug] getting weather for {city}") + return f"The weather in {city} is sunny." + + async def main(): # This agent will use the custom LLM provider agent = Agent( name="Assistant", instructions="You only respond in haikus.", model=OpenAIChatCompletionsModel(model=MODEL_NAME, openai_client=client), + tools=[get_weather], ) - result = await Runner.run( - agent, - "Tell me about recursion in programming.", - ) + result = await Runner.run(agent, "What's the weather in Tokyo?") print(result.final_output) diff --git a/examples/model_providers/custom_example_global.py b/examples/model_providers/custom_example_global.py index d7c293b..ae9756d 100644 --- a/examples/model_providers/custom_example_global.py +++ b/examples/model_providers/custom_example_global.py @@ -6,6 +6,7 @@ from openai import AsyncOpenAI from agents import ( Agent, Runner, + function_tool, set_default_openai_api, set_default_openai_client, set_tracing_disabled, @@ -40,14 +41,21 @@ set_default_openai_api("chat_completions") set_tracing_disabled(disabled=True) +@function_tool +def get_weather(city: str): + print(f"[debug] getting weather for {city}") + return f"The weather in {city} is sunny." + + async def main(): agent = Agent( name="Assistant", instructions="You only respond in haikus.", model=MODEL_NAME, + tools=[get_weather], ) - result = await Runner.run(agent, "Tell me about recursion in programming.") + result = await Runner.run(agent, "What's the weather in Tokyo?") print(result.final_output) diff --git a/examples/model_providers/custom_example_provider.py b/examples/model_providers/custom_example_provider.py index 6e8af42..4e59019 100644 --- a/examples/model_providers/custom_example_provider.py +++ b/examples/model_providers/custom_example_provider.py @@ -12,6 +12,7 @@ from agents import ( OpenAIChatCompletionsModel, RunConfig, Runner, + function_tool, set_tracing_disabled, ) @@ -47,16 +48,19 @@ class CustomModelProvider(ModelProvider): CUSTOM_MODEL_PROVIDER = CustomModelProvider() +@function_tool +def get_weather(city: str): + print(f"[debug] getting weather for {city}") + return f"The weather in {city} is sunny." + + async def main(): - agent = Agent( - name="Assistant", - instructions="You only respond in haikus.", - ) + agent = Agent(name="Assistant", instructions="You only respond in haikus.", tools=[get_weather]) # This will use the custom model provider result = await Runner.run( agent, - "Tell me about recursion in programming.", + "What's the weather in Tokyo?", run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER), ) print(result.final_output) @@ -64,7 +68,7 @@ async def main(): # If you uncomment this, it will use OpenAI directly, not the custom provider # result = await Runner.run( # agent, - # "Tell me about recursion in programming.", + # "What's the weather in Tokyo?", # ) # print(result.final_output) diff --git a/uv.lock b/uv.lock index 9179bd4..c3af99b 100644 --- a/uv.lock +++ b/uv.lock @@ -1,5 +1,4 @@ version = 1 -revision = 1 requires-python = ">=3.9" [[package]] @@ -783,7 +782,7 @@ wheels = [ [[package]] name = "openai-agents" -version = "0.0.3" +version = "0.0.4" source = { editable = "." } dependencies = [ { name = "griffe" }, From 4db24bdb3c29cb15210c8505d6ced11fde97ec97 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Thu, 13 Mar 2025 13:20:27 -0400 Subject: [PATCH 14/15] Update tracing docs to be correct --- src/agents/tracing/processors.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/agents/tracing/processors.py b/src/agents/tracing/processors.py index 308adf2..46df88e 100644 --- a/src/agents/tracing/processors.py +++ b/src/agents/tracing/processors.py @@ -40,7 +40,7 @@ class BackendSpanExporter(TracingExporter): """ Args: api_key: The API key for the "Authorization" header. Defaults to - `os.environ["OPENAI_TRACE_API_KEY"]` if not provided. + `os.environ["OPENAI_API_KEY"]` if not provided. organization: The OpenAI organization to use. Defaults to `os.environ["OPENAI_ORG_ID"]` if not provided. project: The OpenAI project to use. Defaults to From 8a6967b6d4a6980a44affddb3c852f1ef30891f6 Mon Sep 17 00:00:00 2001 From: Rohan Mehta Date: Thu, 13 Mar 2025 13:43:18 -0400 Subject: [PATCH 15/15] Update model docs with common issues --- docs/models.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/models.md b/docs/models.md index a4801fe..ab4cefb 100644 --- a/docs/models.md +++ b/docs/models.md @@ -64,3 +64,30 @@ In cases where you do not have an API key from `platform.openai.com`, we recomme !!! note In these examples, we use the Chat Completions API/model, because most LLM providers don't yet support the Responses API. If your LLM provider does support it, we recommend using Responses. + +## Common issues with using other LLM providers + +### Tracing client error 401 + +If you get errors related to tracing, this is because traces are uploaded to OpenAI servers, and you don't have an OpenAI API key. You have three options to resolve this: + +1. Disable tracing entirely: [`set_tracing_disabled(True)`][agents.set_tracing_disabled]. +2. Set an OpenAI key for tracing: [`set_tracing_export_api_key(...)`][agents.set_tracing_export_api_key]. This API key will only be used for uploading traces, and must be from [platform.openai.com](https://platform.openai.com/). +3. Use a non-OpenAI trace processor. See the [tracing docs](tracing.md#custom-tracing-processors). + +### Responses API support + +The SDK uses the Responses API by default, but most other LLM providers don't yet support it. You may see 404s or similar issues as a result. To resolve, you have two options: + +1. Call [`set_default_openai_api("chat_completions")`][agents.set_default_openai_api]. This works if you are setting `OPENAI_API_KEY` and `OPENAI_BASE_URL` via environment vars. +2. Use [`OpenAIChatCompletionsModel`][agents.models.openai_chatcompletions.OpenAIChatCompletionsModel]. There are examples [here](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/). + +### Structured outputs support + +Some model providers don't have support for [structured outputs](https://platform.openai.com/docs/guides/structured-outputs). This sometimes results in an error that looks something like this: + +``` +BadRequestError: Error code: 400 - {'error': {'message': "'response_format.type' : value is not one of the allowed values ['text','json_object']", 'type': 'invalid_request_error'}} +``` + +This is a shortcoming of some model providers - they support JSON outputs, but don't allow you to specify the `json_schema` to use for the output. We are working on a fix for this, but we suggest relying on providers that do have support for JSON schema output, because otherwise your app will often break because of malformed JSON.