From fd341f8d0bf37d9a81bba0f5e8fadd0818cd3328 Mon Sep 17 00:00:00 2001 From: Him188 Date: Thu, 27 Aug 2026 21:55:47 +0900 Subject: [PATCH] docs(agents): llm_extra_body / fishaudio_extra_body + custom LLM example request Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01NKJq8aWm8TSLNQfJGzTekE --- agents/build/custom-llm.mdx | 76 +++++++++++++++++++++++- agents/deploy/authenticated-sessions.mdx | 1 + agents/telephony/outbound-calls.mdx | 1 + 3 files changed, 77 insertions(+), 1 deletion(-) diff --git a/agents/build/custom-llm.mdx b/agents/build/custom-llm.mdx index bb712ae..38172ef 100644 --- a/agents/build/custom-llm.mdx +++ b/agents/build/custom-llm.mdx @@ -10,12 +10,86 @@ By default your agent generates replies with a Fish Audio platform model. With a Your endpoint speaks the standard OpenAI [chat completions](https://platform.openai.com/docs/api-reference/chat) protocol. The platform sends `POST {base_url}/chat/completions` with `stream: true` and reads the reply as server-sent events, so if your prototype already runs against another voice-agent platform through a custom LLM, the same server works here unchanged. -Each conversation turn, your endpoint receives the same fully assembled context a platform model would see. The `messages` array carries the system prompt with [dynamic variables](/agents/build/dynamic-variables) and [session overrides](/agents/deploy/authenticated-sessions#overrides) applied, the complete conversation history, and any retrieved [knowledge](/agents/build/knowledge-base), and the agent's tools are included in OpenAI function format. When your model returns `tool_calls`, the platform executes the tool and calls you again with the result. Two extra fields ride each request body so your server can look up its own state: +Each conversation turn, your endpoint receives the same fully assembled context a platform model would see. The `messages` array carries the system prompt with [dynamic variables](/agents/build/dynamic-variables) and [session overrides](/agents/deploy/authenticated-sessions#overrides) applied, the complete conversation history, and any retrieved [knowledge](/agents/build/knowledge-base), and the agent's tools are included in OpenAI function format. When your model returns `tool_calls`, the platform executes the tool and calls you again with the result. A few extra top-level fields ride each request body so your server can look up its own state: | Field | Content | |---|---| | `session_id` | The Fish Audio session id. | | `user_id` | The `end_user_id` you passed when [creating the session](/agents/deploy/authenticated-sessions). Omitted when the session has none. | +| `fishaudio_extra_body` | The `llm_extra_body` object you passed when creating the session, forwarded verbatim on every request. Use it to carry your own identifiers, such as which chat or thread the user is in. Omitted when the session has none. | + + +If your server proxies requests to an upstream provider such as OpenAI, consider removing `session_id`, `user_id`, and `fishaudio_extra_body` from the body first. Some upstream APIs reject parameters they don't recognize. + + +### Example request + +Suppose the session was created from your backend with `end_user_id: "user-42"` and `llm_extra_body: {"chat_id": "chat-9"}`, the agent has one webhook tool `lookup_order`, and the user has just asked about an order. Your endpoint receives: + +```http +POST /v1/chat/completions HTTP/1.1 +Host: llm.example.com +Authorization: Bearer sk-your-endpoint-key +Content-Type: application/json +``` + +```json +{ + "model": "persona-70b", + "stream": true, + "stream_options": { "include_usage": true }, + "messages": [ + { + "role": "system", + "content": "Today is Wednesday, August 26, 2026. Session timezone: America/New_York (UTC-4).\n\n…\n\nYou are Aria, the support assistant for Acme Shoes. Help customers with orders and returns. …" + }, + { "role": "assistant", "content": "Hi, this is Aria from Acme Shoes. How can I help you today?" }, + { "role": "user", "content": "I want to check on my order." }, + { "role": "assistant", "content": "Sure, what's the order number?" }, + { "role": "system", "content": "Current date and time: Wednesday, August 26, 2026, 14:07 (America/New_York)." }, + { "role": "user", "content": "It's A one two three four five." } + ], + "tools": [ + { + "type": "function", + "function": { + "name": "lookup_order", + "description": "Look up an order by its number.", + "parameters": { + "type": "object", + "properties": { + "order_number": { "type": "string", "description": "The order number, e.g. A12345." } + }, + "required": ["order_number"] + } + } + } + ], + "session_id": "sess_01j9x4k2m8v3q7n5p6r8t9w0y1", + "user_id": "user-42", + "fishaudio_extra_body": { "chat_id": "chat-9" } +} +``` + +A few things to note: + +- The first `system` message is the assembled prompt: your agent's [system prompt](/agents/build/configuration#system-prompt) with [dynamic variables](/agents/build/dynamic-variables) and [overrides](/agents/deploy/authenticated-sessions#overrides) applied, plus the platform's own context lines. A short `system` line carrying the current time is inserted before the latest user turn on every request. +- `messages` carries the full transcript so far. Speech recognition output arrives as plain `user` text; your earlier replies come back as `assistant` messages. +- `tools` is present only when the agent has tools configured. The `model`, `stream`, and `stream_options` fields are fixed; no `temperature` or `max_tokens` is sent, so apply your own defaults. + +When your model returns a `lookup_order` tool call, the platform executes it and immediately sends the next request with the call and its result appended to `messages`, everything else unchanged: + +```json +{ + "role": "assistant", + "tool_calls": [ + { "id": "call_1", "type": "function", "function": { "name": "lookup_order", "arguments": "{\"order_number\":\"A12345\"}" } } + ] +}, +{ "role": "tool", "tool_call_id": "call_1", "content": "{\"status\":\"shipped\",\"eta\":\"2026-08-28\"}" } +``` + +Reply to that request with the spoken answer as ordinary `delta.content` chunks ending in `finish_reason: "stop"`. Requests authenticate with `Authorization: Bearer `. The endpoint must use `https` on a publicly reachable host, and must support function calling if the agent has tools configured. diff --git a/agents/deploy/authenticated-sessions.mdx b/agents/deploy/authenticated-sessions.mdx index 9c4d3a1..f22a093 100644 --- a/agents/deploy/authenticated-sessions.mdx +++ b/agents/deploy/authenticated-sessions.mdx @@ -125,6 +125,7 @@ More on what the SDK can do once connected is in the [Web SDK](/agents/deploy/we | `tool_events` | boolean, optional | Stream tool lifecycle events (`toolCallStarted` / `toolCallCompleted` / `toolCallFailed`) to the client. Default `true`; set `false` to keep tool inputs and outputs off the client. | | `end_user_id` | string, optional | Your identifier for the end user, up to 256 characters. Stored on the session and echoed in [webhook](/agents/monitor/webhooks) payloads and [custom LLM](/agents/build/custom-llm) requests. | | `metadata` | object, optional | Your own key-value namespace. Stored and returned verbatim on session queries and webhooks, never read or interpreted by the platform. | +| `llm_extra_body` | object, optional | JSON object (at most 16 KB) forwarded verbatim to your [custom LLM](/agents/build/custom-llm) endpoint on every request as `fishaudio_extra_body`, for example the chat or thread the user is in. Not stored or returned on session reads; ignored when the agent uses a platform model. | | `record_audio` | boolean, optional | Whether to record this session's audio. Overrides the agent's [recording setting](/agents/monitor/conversation-history#what-gets-stored) for this session only; omit it to use the agent's configuration. | | `timezone` | string, optional | IANA timezone (like `Asia/Shanghai`) for the agent's sense of local time. Invalid names are rejected with `422`. See [Time & timezone](/agents/build/time-timezone). | | `client_timezone` | string, optional | The end user's browser timezone, filled automatically by the SDK in public-agent mode. A hint, not a demand: it applies only when neither `timezone` nor the agent's configured timezone is set, and invalid values are ignored. See the [resolution order](/agents/build/time-timezone). | diff --git a/agents/telephony/outbound-calls.mdx b/agents/telephony/outbound-calls.mdx index fa944dc..b7f75f8 100644 --- a/agents/telephony/outbound-calls.mdx +++ b/agents/telephony/outbound-calls.mdx @@ -69,6 +69,7 @@ This endpoint requires an API key; there is no anonymous variant. See the [API r | `dynamic_variables` | Optional: per-call values for `{{placeholders}}` in the agent's configured text, same rules as [session creation](/agents/build/dynamic-variables). Up to 50 entries. | | `overrides` | Optional: replace whole configuration fields for this call, subject to the agent's [override allowlist](/agents/deploy/authenticated-sessions#overrides). | | `metadata` | Optional: your own JSON object, returned verbatim on session reads and in webhook payloads. Never interpreted. | +| `llm_extra_body` | Optional: JSON object (at most 16 KB) forwarded to a [custom LLM](/agents/build/custom-llm) endpoint on every request as `fishaudio_extra_body`. Ignored on platform-model agents. | The session's [time and timezone context](/agents/build/time-timezone) resolves from the destination number when the agent has no fixed timezone configured, so "tomorrow morning" means the callee's morning.