diff --git a/developer-guide/self-hosting/air-gapped.mdx b/developer-guide/self-hosting/air-gapped.mdx index 6ef5ee5..653be89 100644 --- a/developer-guide/self-hosting/air-gapped.mdx +++ b/developer-guide/self-hosting/air-gapped.mdx @@ -7,7 +7,9 @@ icon: "shield-halved" Both offline delivery forms make no outbound calls at runtime. Neither one, however, installs itself out of thin air: a Helm install pulls container images and the chart from a registry, and the appliance needs its image on the host. An air-gapped -deployment is about getting those artifacts across the boundary. +deployment is about getting those artifacts across the boundary. The license bundle +has to cross it too, and unlike an image it expires — plan a recurring transfer, not +a one-off. Fish Audio does this with you as part of an air-gapped delivery, and the step-by-step @@ -18,9 +20,11 @@ deployment is about getting those artifacts across the boundary. ## Which form to choose -**The All-in-One appliance** is the straightforward answer to a strict air gap: one -image with every weight baked in, moved to the disconnected host as a file and loaded -there. There is nothing else to mirror. +**The air-gapped All-in-One appliance** is the straightforward answer to a strict air +gap: one image with every weight baked in, moved to the disconnected host as a file and +loaded there. There is nothing else to mirror, though the license bundle still has to +be carried across. Its online counterpart cannot run air +gapped: it authorises every request against Fish Audio. **The Helm chart** is more work, because the release is many images rather than one. Every image has to be mirrored into a registry the cluster can reach and the release diff --git a/developer-guide/self-hosting/all-in-one.mdx b/developer-guide/self-hosting/all-in-one.mdx index 68173d1..14c52b7 100644 --- a/developer-guide/self-hosting/all-in-one.mdx +++ b/developer-guide/self-hosting/all-in-one.mdx @@ -6,9 +6,14 @@ icon: "box" The All-in-One image packages the whole speech stack (edge API, model API layer, inference router and worker, vocoder, text normalizer, and Redis) into one container, -with every model weight baked in. Once the image is on the host it runs with no -Kubernetes and no internet access, which makes it the turnkey option for single-node -appliances and strict air gaps. +with every model weight baked in. It runs as a turnkey single-node appliance, no Kubernetes. + +The two forms, **Offline All-in-One** and **Online All-in-One**, differ only in how +usage is accounted for. The offline form records usage to a local +ledger and needs no network once the image is on the host. The online form authorises +each request against Fish Audio and charges the key that made it, so it needs outbound +access and a key with credit. The accounting is compiled in, so an image is one or the +other; the container logs which at startup. This page covers what the appliance is and what to plan for. The commands, tuning @@ -21,10 +26,9 @@ appliances and strict air gaps. The appliance runs one inference worker and one vocoder, a GPU each. It does not autoscale, does not shard across more GPUs or nodes, and does not ship the forced -aligner, so it returns no word or segment timings. It is offline-only: there is no -hosted-billing variant. For elastic or higher-throughput deployments, use the -[Kubernetes chart](/developer-guide/self-hosting/kubernetes), which scales replicas -across all GPUs and nodes. +aligner, so it returns no word or segment timings. For elastic or higher-throughput +deployments, use the [Kubernetes chart](/developer-guide/self-hosting/kubernetes), which +scales replicas across all GPUs and nodes. ## What running it involves @@ -35,8 +39,13 @@ It needs: - **Two GPUs.** The first runs the inference worker, the second the vocoder. - **One exposed port** for the API. - **One persistent volume.** Compile caches, the vocoder's built engine, reference - voice archives, and the usage ledger all live there. Model weights are in the image, - not on the volume. + voice archives, and — on the offline form — the usage ledger all live there. Model + weights are in the image, not on the volume. +- **The license bundle.** A host directory holding the certificate and key pair from + **Developer → Self Host**, mounted read-only. Without it the API never comes up; once + the certificate expires the container starts but refuses requests. Renewing is + replacing those files in place; only a bundle that carries a new key needs the + container restarted. Everything inside the container runs as a non-root user, so a reused volume or a host bind mount has to be writable by it. @@ -49,15 +58,25 @@ the GPU model, so moving to different cards rebuilds it once. ## Usage accounting and tenancy -This build records usage to a local, signed, append-only ledger on the volume instead -of calling a billing service. Two consequences worth designing around: +The form determines how usage is accounted for and what the bearer token has to be. + +**Offline.** Usage goes to a local, signed, append-only ledger on the volume. Two +consequences worth designing around: - **Any non-empty bearer token is accepted**; a missing or empty one is rejected. The appliance has nothing to validate a token against. - **The token is recorded verbatim as the billing identity.** Use a stable, distinct token per tenant. Two tenants sharing a token are indistinguishable in the ledger. -See [Offline usage accounting](/developer-guide/self-hosting/air-gapped#offline-usage-accounting). +Backups and reconciliation: [Offline usage accounting](/developer-guide/self-hosting/air-gapped#offline-usage-accounting). + +**Online.** Each request is authorised against Fish Audio and charged to the key that +made it. The token must be a [Fish Audio API key](/developer-guide/getting-started/api-key) +with credit — not the deploy token used to pull the image — and anything else is +rejected. No ledger is written; usage appears in your Fish Audio account instead. + +Authorisation happens on every request, not only at startup, and the appliance has no +local fallback. Treat outbound reachability as a production dependency. ## Capacity diff --git a/developer-guide/self-hosting/enterprise-releases.mdx b/developer-guide/self-hosting/enterprise-releases.mdx index 842a168..40a8abe 100644 --- a/developer-guide/self-hosting/enterprise-releases.mdx +++ b/developer-guide/self-hosting/enterprise-releases.mdx @@ -16,7 +16,8 @@ one you pick into the install commands on that page. | All-in-One | An image tag | The tag in the image reference | Both Helm forms are one chart, so they share a version. Which form you deploy is -decided by the values file, not by the version. +decided by the values file, not by the version. The two All-in-One forms are separate +images with independent tags. ## Upgrading diff --git a/developer-guide/self-hosting/introduction.mdx b/developer-guide/self-hosting/introduction.mdx index 7e3df1c..2a9d11d 100644 --- a/developer-guide/self-hosting/introduction.mdx +++ b/developer-guide/self-hosting/introduction.mdx @@ -23,18 +23,18 @@ audio formats, and voice behavior match what you already build against. ## Delivery forms -The same engine ships in three forms. Your enterprise agreement determines which +The same engine ships in four forms. Your enterprise agreement determines which ones your team is granted. -| | Online Helm | Offline Helm | Offline All-in-One | -| ---------------- | --------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------- | -| Platform | Kubernetes | Kubernetes | A single container, `docker run` | -| Model assets | Served in-cluster from a bundled model warehouse | Same | Baked into the image | -| Usage accounting | Validated and billed against the Fish Audio service | Local signed usage ledger | Local signed usage ledger | -| Runtime egress | The billing endpoint, plus one more host if you enable timestamps | None | None | -| Air-gap capable | No | Yes, after mirroring images and charts | Yes | -| Scaling | Scales replicas across all GPUs and nodes | Same | Single node, single inference worker | -| Best for | Managed clusters with outbound access | Isolated or regulated production clusters | Evaluation, single-node appliances, strict air gaps | +| | Online Helm | Offline Helm | Offline All-in-One | Online All-in-One | +| ---------------- | --------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------- | ------------------------------------------------ | +| Platform | Kubernetes | Kubernetes | A single container, `docker run` | A single container, `docker run` | +| Model assets | Served in-cluster from a bundled model warehouse | Same | Baked into the image | Baked into the image | +| Usage accounting | Validated and billed against the Fish Audio service | Local signed usage ledger | Local signed usage ledger | Validated and billed against the Fish Audio service | +| Runtime egress | The billing endpoint, plus one more host if you enable timestamps | None | None | The billing endpoint | +| Air-gap capable | No | Yes, after mirroring images and charts | Yes | No | +| Scaling | Scales replicas across all GPUs and nodes | Same | Single node, single inference worker | Single node, single inference worker | +| Best for | Managed clusters with outbound access | Isolated or regulated production clusters | Evaluation, single-node appliances, strict air gaps | Single-node appliances billed like the hosted service | Both Helm forms share one deployment procedure and differ only in a few values. See [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) for the @@ -43,7 +43,7 @@ for the single-container form. ## What is included -| Capability | Online Helm | Offline Helm | All-in-One | +| Capability | Online Helm | Offline Helm | All-in-One (both forms) | | ----------------------------------------- | ---------------------------- | ------------------------ | ------------------------ | | Text to speech over `POST /v1/tts` | Included | Included | Included | | WebSocket streaming | Included | Included | Included | diff --git a/developer-guide/self-hosting/kubernetes.mdx b/developer-guide/self-hosting/kubernetes.mdx index 8c89471..60452ac 100644 --- a/developer-guide/self-hosting/kubernetes.mdx +++ b/developer-guide/self-hosting/kubernetes.mdx @@ -51,12 +51,15 @@ manage that layer. | Reference voices | Requests carrying a `reference_id` resolve from a local archive, never from the network. One zip per voice, staged on shared storage before that traffic starts. | | Replica counts | Talk to your account team first. The shipped shape is what the delivery was sized and validated against. | -Three Kubernetes Secrets have to exist before the install: registry credentials, a JWT -secret for the edge API, and one shared between the in-cluster model store and the -workers that read from it. That one authenticates nothing outside the cluster, and the -value is yours to choose. Generate the JWT secret once -and keep it stable, because changing it invalidates issued tokens. For production, prefer -External Secrets, Sealed Secrets, or your cloud secret manager over plain Secrets. +Three Kubernetes Secrets have to exist before the install: registry credentials, the +license bundle, and one shared between the in-cluster model store and the workers that +read from it. The license bundle is the certificate and key pair downloaded from +**Developer → Self Host**; the services that enforce it stay down until the Secret +exists, and refuse requests once the certificate lapses — renewal is described in +[Operations](/developer-guide/self-hosting/operations#license-renewal). The model-store +secret authenticates nothing outside the cluster, so the value is yours to choose. For +production, prefer External Secrets, Sealed Secrets, or your cloud secret manager over +plain Secrets. ## What to expect diff --git a/developer-guide/self-hosting/operations.mdx b/developer-guide/self-hosting/operations.mdx index eb67288..030e3be 100644 --- a/developer-guide/self-hosting/operations.mdx +++ b/developer-guide/self-hosting/operations.mdx @@ -30,6 +30,8 @@ Prometheus is the metrics baseline. Watch: - CPU and memory usage against the configured requests and limits. - Shared storage usage and latency. - Redis availability and latency. +- License validity and time to expiry, exported as metrics; the services also log + warnings well before expiry. - Image pull failures and Kubernetes events in the release namespace. Centralize application logs and retain Kubernetes events long enough to @@ -66,6 +68,19 @@ Back up: Compile and model caches on shared storage do not need backing up. Losing them costs a slow first start, nothing more. +## License renewal + +Update the license Secret in place with the renewed bundle; no restart is needed. +Kubernetes propagates the change to the running pods, and each service re-verifies on +its own short cycle, so expect up to a couple of minutes of mixed responses before +it settles. One exception: a bundle +that carries a new key rather than a re-signed certificate. The offline form's edge API +reads the ledger-signing key only at startup, so restart that deployment after +replacing a key. + +Pods with an expired certificate keep running but refuse requests, then recover once +a renewal propagates. + ## Upgrades Commands are in the deployment runbook. Before running them, capture the release @@ -82,12 +97,14 @@ guide that ships in your documentation bundle. | ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ImagePullBackOff` or `ErrImagePull` | Missing pull secret, a deploy token that was rotated or deleted, a tag that is not in your mirror, or blocked egress to the registry. | | Pods stay `Pending` | Not enough CPU, memory, or GPU capacity; GPU node taints without matching tolerations; a node selector that matches nothing. | +| Pods stuck `ContainerCreating` | A Secret the pod mounts does not exist yet — most often the license bundle. The pods recover on their own once it is created. | | GPU not available, or `nvidia-smi` fails in a pod | Driver missing or unhealthy, device plugin or GPU Operator not ready, container runtime not configured for NVIDIA, or an unsupported GPU for the driver stack. | | hostPath volume errors, or missing model and cache paths | Shared storage is not mounted on that node, a newly added node never got the mount, the mount target is unreachable, or permissions prevent writes. | | Requests fail after reaching the edge API, with Redis errors in its logs | The Redis master or replica is not ready, the service is missing, or a network policy blocks it. | | `Reference not found` | The archive was never staged, the filename does not exactly match the reference id, the zip is malformed, or the reference root was changed without mounting the new path into every edge API pod. | | Metrics missing from Prometheus | Scrape annotations disabled in values, Prometheus not configured for annotation-based discovery, a port mismatch, or a network policy blocking scrapes. | -| `CrashLoopBackOff` | A missing secret or environment variable, model assets unreachable, or a path the container cannot write. | +| `CrashLoopBackOff` | A missing environment variable, a license file missing or unreadable inside the Secret, model assets unreachable, or a path the container cannot write. | +| Requests refused with a license error | The certificate expired, or a renewal has not finished propagating — see [License renewal](#license-renewal). | | Time-to-first-audio high, GPU utilization low | Concurrency above what the current replicas can serve, cold caches after a restart, storage latency, or CPU saturation on the API nodes. | ## Escalating to Fish Audio diff --git a/developer-guide/self-hosting/registry-access.mdx b/developer-guide/self-hosting/registry-access.mdx index 52edd52..0be4698 100644 --- a/developer-guide/self-hosting/registry-access.mdx +++ b/developer-guide/self-hosting/registry-access.mdx @@ -17,6 +17,9 @@ start is there: page reports that self-host deployment is not enabled, contact your account manager. - **The versions available to you**, and the documentation bundle for each. See [Releases](/developer-guide/self-hosting/enterprise-releases). +- **Your license bundle**: the certificate and key the deployment needs before it will + serve. Some contracts bind it to specific GPUs — have the card UUIDs ready when you + request it. - **Install commands built for your team**, with the registry host, artifact references, and the version you pick already filled in. diff --git a/developer-guide/self-hosting/requirements.mdx b/developer-guide/self-hosting/requirements.mdx index 60378c0..5cf0e40 100644 --- a/developer-guide/self-hosting/requirements.mdx +++ b/developer-guide/self-hosting/requirements.mdx @@ -76,6 +76,7 @@ on the nodes that host it, plus headroom for rewrites. | Redis | Installed by the chart into the release namespace. Do not point the release at a shared Redis without discussing it first. | | Metrics | Prometheus, scraping `prometheus.io/*` pod annotations. The chart creates no ServiceMonitors, so kube-prometheus-stack users must add an annotation-based scrape config. | | Access layer | Yours to choose. Ingress controller, DNS, TLS, and load balancing are not part of the delivery. | +| License bundle | The certificate and key from **Developer → Self Host**, loaded into the cluster as a Secret before the install. | The cluster also needs working in-cluster DNS, a dedicated namespace, node labels and taints for GPU scheduling, and a Pod Security level in that namespace that @@ -102,6 +103,8 @@ permits `hostPID` and `hostIPC`, which the inference workers require. | RAM | 128 GiB minimum, 192 GiB recommended. The container runs the whole stack in one process tree. | | CPU | 32 vCPU minimum, 48 to 64 recommended. | | Disk | 100 GB free if the host pulls the image itself; 120 GB if you move it as a file, since `docker load` needs the ~30 GB archive and the ~80 GB unpacked image at once. Plus the compile and engine caches. | +| Network | Offline form: none at runtime. Online form: outbound HTTPS to the Fish Audio authorization and billing endpoint, on every request. Ask your account team for the hostname to allowlist. | +| License bundle | The certificate and key pair from **Developer → Self Host**, in a host directory readable by the container's non-root user. | ## Next step