Apply workspace temperature to agents and omit the parameter when unset - #6285
Conversation
timothycarambat
left a comment
There was a problem hiding this comment.
Router workspaces lose the OpenAI-compatible API temperature override.
resolveProviderConnector only honors the new temperature arg on the non-router branch; the router branch builds the delegate from workspace.openAiTemp and ignores it.
Since openaiCompatible.js no longer passes temperature into getChatCompletion, a router workspace hit via /v1/openai/chat/completions with temperature: 0 will now go out with the workspace value. Easiest fix is probably to pass temperature into AnythingLLMModelRouter (or into its getLLMProvider call) so both branches behave the same. I think that will work, but needs to be tested.
Please remove the "close live agent sessions on workspace save" change.
It's a good change feature, but it fires from every Workspace.update caller (settings save, developer API, community hub import, Telegram model picker), and it aborts an in-flight agent run and closes the invocation. Lets just uncheck it for now from at least this PR.
Then make an issue for autosave on edit of workspace fields and lets handle that UX convenience and revisit the abort handler on save later. When you leave the page it should abort the session anyway so that shouldnt change anything and you cannot change the config "on the fly" anyway.
Smaller asks:
- Add a unit test for
resolveTemperature(null, "", "0", negative, non-numeric, and one guarded model per class) - Rebase to master
timothycarambat
left a comment
There was a problem hiding this comment.
Referring to the comment from OR
Anytime we are potentially pushing a key:value and value is undefined, we should spread it via a class util so that we are not ever sending key: undefined values as undefined over REST is not good practice (RFC 8259). null is valid, undefined is not. Serialization of payload bodies also makes this a bad idea in general while spreading {} just omits it before/during serialization
We are passing potentially undefined here for this.temperature and it should have a shared utils that we can reasonable spread this on
…t request bodies - Mistral and Cerebras chat no longer force temperature 0 when unset - Generic OpenAI and Cohere untooled agent calls omit temperature when unset - Test that agent sessions, tooled/Responses helpers, chat connectors and the model router send the workspace temperature and omit it when unset
…sable The OpenAI-compatible endpoint passed Number(undefined) = NaN when a request had no temperature, which skipped the ?? fallback and dropped the workspace setting. Invalid overrides now defer to the workspace temperature.
Pull Request Type
Relevant Issues
resolves #6091
Description
NaNwhen a request omitstemperature- it now defers to the workspace settingVisuals (if applicable)
Additional Information
Developer Validations
yarn lintfrom the root of the repo & committed changes