Skip to content

Keep the chosen provider on a model name that contains a slash - #617

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:keep-provider-on-a-model-name-that-contains-a-slash
Sep 21, 2026
Merged

davidmckayv merged 2 commits into
CopilotKit:mainfrom
kevin9327:keep-provider-on-a-model-name-that-contains-a-slash

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

What this changes

A model name the endpoint publishes with a slash in it — qwen/qwen3-8b, or the docs example openai/gpt-5.6-terra — was treated as already carrying a provider. LiteLLM then took the first half as the provider and sent only the rest, so a compatibility endpoint either failed with LLM Provider NOT provided or received gpt-5.6-terra instead of the namespaced name.

The ADK, Strands, Agno, LlamaIndex and CrewAI Bots now always keep the chosen provider in front. LiteLLM strips that first component when it talks to the endpoint, so the whole model name arrives.

I will rebase after merge if CHANGELOG conflicts.

Where it runs

OpenBot is deployed as several server processes behind a load balancer, serving a whole company.
Consecutive requests from the same person reach different processes, and the process that answered a
WebSocket upgrade is rarely the one that answers the next call on that conversation.

State that outlives a single request therefore has to be shared, or the change works on one machine
and stops working the moment there are two, without saying so. That failure is worse than not
shipping the feature: it passes review, passes CI, passes a local demo, and only surfaces as a Bot
that forgets, a question nobody can answer, or a boundary that never fires.

Answer these even when the answer is "none":

  • New state that outlives a request? None. Each harness still reads BOT_PROVIDER and BOT_MODEL from the environment when it starts.
  • What happens on the second replica? The same. Every replica of a harness process is started with the same two variables.
  • Anything serialised? None.
  • Anything fanned out to a browser? None.
  • New listener, port, or schedule? None.

Boundary and audit

  • Every acting call still goes through the gateway: resolve, decide, audit, then act. Untouched.
  • New refusals and new failures each write a row. None added.
  • Nothing new is trusted from the client that the server can resolve itself. The model name is still the one the setup screen wrote.

Changelog

  • A line in CHANGELOG.md under Unreleased.

Proof

On unmodified main, with the new tests present:

  • CrewAI _model(): qwen/qwen3-8b stayed qwen/qwen3-8b; openai/gpt-5.6-terra stayed openai/gpt-5.6-terra.
  • Strands and ADK roundtrips: qwen/qwen3-8b failed with LLM Provider NOT provided; openai/gpt-5.6-terra was sent to the endpoint as gpt-5.6-terra.

After the fix, those same tests pass. The fake OpenAI server records qwen/qwen3-8b and openai/gpt-5.6-terra as the model names. Existing model-choice cases still pass (Agno 5, LlamaIndex 5, ADK 5, Strands 7, CrewAI unit 10). bun run typecheck clean. Biome format:check and lint clean.

AI-assisted (Grok)

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@davidmckayv
davidmckayv force-pushed the keep-provider-on-a-model-name-that-contains-a-slash branch from 61368b8 to 1ef21ab Compare September 20, 2026 23:48
@davidmckayv
davidmckayv force-pushed the keep-provider-on-a-model-name-that-contains-a-slash branch from 1ef21ab to ec032db Compare September 21, 2026 00:11
@davidmckayv
davidmckayv merged commit 86ec571 into CopilotKit:main Sep 21, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants