Repository navigation
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds a Helm chart to deploy a vLLM (TorchTPU) benchmark for the DeepSeek-R1-0528 model on tpu7x on GKE, supporting both colocated and disaggregated serving modes. The feedback highlights several template improvements and potential bugs, including a hardcoded default namespace for the ServiceAccount, missing default values for the Kubernetes service account name, unquoted environment variables, and a lack of required validation for critical parameters (such as images and bash commands) when disaggregated serving is enabled. Suggestions are also made to use tpl for dynamic bash commands and toYaml for cleaner volume attribute formatting.
| kind: ServiceAccount | ||
| metadata: | ||
| name: {{ .Values.k8s_service_account | quote }} | ||
| namespace: default |
There was a problem hiding this comment.
Hardcoding the namespace to default for the ServiceAccount will cause issues if the Helm chart is installed in a custom namespace. The ServiceAccount will be created in the default namespace, but other resources (like Deployments, Jobs, and LWS) will be created in the release namespace and will fail to find or use the ServiceAccount. It should use {{ .Release.Namespace | quote }} instead.
namespace: {{ .Release.Namespace | quote }}| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: {{ .Values.k8s_service_account | quote }} |
There was a problem hiding this comment.
If gcp_service_account is provided but k8s_service_account is left as null (which is the default in values.yaml), the ServiceAccount name will render as an empty string "", causing a Kubernetes validation failure. It is safer to provide a default value (e.g., {{ .Release.Name }}-sa) if k8s_service_account is not specified. Please also update all references to serviceAccountName in this chart to use the same default value.
name: {{ .Values.k8s_service_account | default (printf "%s-sa" .Release.Name) | quote }}| {{- end }} | ||
| command: ["/bin/bash", "-c"] | ||
| args: | ||
| - {{ .Values.server.bash_command | quote }} |
There was a problem hiding this comment.
Evaluating bash_command and worker_bash_command through the tpl function allows them to contain dynamic Helm template expressions (such as referencing .Values.gcsfuse.volumes or other values). This avoids having to manually search and replace <YOUR_BUCKET_NAME> in multiple places in values.yaml.
- {{ tpl .Values.server.bash_command $ | quote }}| {{end}} | ||
| {{if .Values.server.model_impl_type}} | ||
| - name: MODEL_IMPL_TYPE | ||
| value: {{.Values.server.model_impl_type}} |
| {{- if .volumeAttributes }} | ||
| {{- range $k, $v := .volumeAttributes }} | ||
| {{ $k }}: {{ $v | quote }} | ||
| {{- end }} | ||
| {{- end }} |
There was a problem hiding this comment.
Instead of manually looping and formatting volumeAttributes with custom indentation, you can use the idiomatic Helm toYaml function combined with nindent. This is cleaner, less error-prone, and automatically handles correct indentation.
{{- with .volumeAttributes }}
{{- toYaml . | nindent 12 }}
{{- end }}| {{- include "tpu.nodeSetup.initContainers" $ctx | nindent 8 }} | ||
| {{- end }} | ||
| containers: | ||
| - name: vllm-server |
There was a problem hiding this comment.
In disaggregated serving mode, if prefill or decode pool values (like image or bash_command) are left as null (which is the default in values.yaml), the Helm chart will render invalid YAML. Using the required function ensures that Helm fails fast with a clear error message if these mandatory fields are missing when disaggregated mode is enabled.
image: {{ required (printf "%s.image is required when serving_mode is disaggregated" .role) $pool.image }}
| {{- end }} | ||
| command: ["/bin/bash", "-c"] | ||
| args: | ||
| - {{ printf "if [ \"${LWS_WORKER_INDEX:-0}\" = \"0\" ]; then\n%s\nelse\n%s\nfi" $pool.bash_command $pool.worker_bash_command | quote }} |
There was a problem hiding this comment.
If $pool.bash_command or $pool.worker_bash_command is null, the printf function will output %!s(<nil>) into the bash script, causing syntax errors and container startup failures. Using required for bash_command and defaulting worker_bash_command to an empty string (or bash_command) prevents this.
- {{ printf "if [ \"${LWS_WORKER_INDEX:-0}\" = \"0\" ]; then\n%s\nelse\n%s\nfi" (required (printf "%s.bash_command is required" .role) $pool.bash_command) ($pool.worker_bash_command | default "") | quote }}
| {{- end }} | ||
| containers: | ||
| - name: vllm-proxy | ||
| image: {{ .Values.proxy.image }} |
There was a problem hiding this comment.
Since proxy.image is null by default in values.yaml, enabling disaggregated serving mode without configuring it will result in invalid Kubernetes manifests. Using the required function ensures that Helm fails fast with a clear error message.
image: {{ required "proxy.image is required when serving_mode is disaggregated" .Values.proxy.image }}
Adding a recipe for DeepSeek-R1-0528 inference on Ironwood using vLLM (TorchTPU).
ubench run_id: vllm_inference-torchtpu_vllm/ts_repro_deepseek_r1_0528_tp16_c64_v029-2026-11-05_221739-3b17a936-76d0-4e25-a28f-0c888672f3e4
final diff gpaste: https://paste.googleplex.com/6755399436724943?raw