Commit graph

1 commit

Author SHA1 Message Date
Rohan Mehta
a0254b0b74
RFC: automatically use litellm if possible (#534)
## Summary
This replaces the default model provider with a `MultiProvider`, which
has the logic:
- if the model name starts with `openai/` or doesn't contain "/", use
OpenAI
- if the model name starts with `litellm/`, use LiteLLM to use the
appropriate model provider.

It's also extensible, so users can create their own mappings. I also
imagine that if we natively supported Anthropic/Gemini etc, we can add
it to MultiProvider to make it work.

The goal is that it should be really easy to use any model provider.
Today if you pass `model="gpt-4.1"`, it works great. But
`model="claude-sonnet-3.7"` doesn't. If we can make it that easy, it's a
win for devx.

I'm not entirely sure if this is a good idea - is it too magical? Is the
API too reliant on litellm? Comments welcome.

## Test plan

For now, the example. Will add unit tests if we agree its worth mergin.

---------

Co-authored-by: Steven Heidel <steven@heidel.ca>
2025-04-21 15:03:06 -04:00