Skip to content

[#recipebot] Add DeepSeek-R1-0528 inference recipe on Ironwood - #386

Draft
hjajoo-g wants to merge 1 commit into
AI-Hypercomputer:mainfrom
hjajoo-g:recipebot-deepseek-r1-inference
Draft

hjajoo-g wants to merge 1 commit into
AI-Hypercomputer:mainfrom
hjajoo-g:recipebot-deepseek-r1-inference

Conversation

@hjajoo-g

@hjajoo-g hjajoo-g commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Adding a recipe for DeepSeek-R1-0528 inference on Ironwood using vLLM (TorchTPU).

ubench run_id: vllm_inference-torchtpu_vllm/ts_repro_deepseek_r1_0528_tp16_c64_v029-2026-11-05_221739-3b17a936-76d0-4e25-a28f-0c888672f3e4
final diff gpaste: https://paste.googleplex.com/6755399436724943?raw

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a Helm chart to deploy a vLLM (TorchTPU) benchmark for the DeepSeek-R1-0528 model on tpu7x on GKE, supporting both colocated and disaggregated serving modes. The feedback highlights several template improvements and potential bugs, including a hardcoded default namespace for the ServiceAccount, missing default values for the Kubernetes service account name, unquoted environment variables, and a lack of required validation for critical parameters (such as images and bash commands) when disaggregated serving is enabled. Suggestions are also made to use tpl for dynamic bash commands and toYaml for cleaner volume attribute formatting.

kind: ServiceAccount
metadata:
name: {{ .Values.k8s_service_account | quote }}
namespace: default

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Hardcoding the namespace to default for the ServiceAccount will cause issues if the Helm chart is installed in a custom namespace. The ServiceAccount will be created in the default namespace, but other resources (like Deployments, Jobs, and LWS) will be created in the release namespace and will fail to find or use the ServiceAccount. It should use {{ .Release.Namespace | quote }} instead.

  namespace: {{ .Release.Namespace | quote }}

apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Values.k8s_service_account | quote }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If gcp_service_account is provided but k8s_service_account is left as null (which is the default in values.yaml), the ServiceAccount name will render as an empty string "", causing a Kubernetes validation failure. It is safer to provide a default value (e.g., {{ .Release.Name }}-sa) if k8s_service_account is not specified. Please also update all references to serviceAccountName in this chart to use the same default value.

  name: {{ .Values.k8s_service_account | default (printf "%s-sa" .Release.Name) | quote }}

{{- end }}
command: ["/bin/bash", "-c"]
args:
- {{ .Values.server.bash_command | quote }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Evaluating bash_command and worker_bash_command through the tpl function allows them to contain dynamic Helm template expressions (such as referencing .Values.gcsfuse.volumes or other values). This avoids having to manually search and replace <YOUR_BUCKET_NAME> in multiple places in values.yaml.

        - {{ tpl .Values.server.bash_command $ | quote }}

{{end}}
{{if .Values.server.model_impl_type}}
- name: MODEL_IMPL_TYPE
value: {{.Values.server.model_impl_type}}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The value of MODEL_IMPL_TYPE is not quoted. If it contains special characters or spaces, it can cause YAML parsing errors. It is safer to quote it or use | quote.

          value: {{ .Values.server.model_impl_type | quote }}

Comment on lines +232 to +236
{{- if .volumeAttributes }}
{{- range $k, $v := .volumeAttributes }}
{{ $k }}: {{ $v | quote }}
{{- end }}
{{- end }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of manually looping and formatting volumeAttributes with custom indentation, you can use the idiomatic Helm toYaml function combined with nindent. This is cleaner, less error-prone, and automatically handles correct indentation.

            {{- with .volumeAttributes }}
            {{- toYaml . | nindent 12 }}
            {{- end }}

{{- include "tpu.nodeSetup.initContainers" $ctx | nindent 8 }}
{{- end }}
containers:
- name: vllm-server

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In disaggregated serving mode, if prefill or decode pool values (like image or bash_command) are left as null (which is the default in values.yaml), the Helm chart will render invalid YAML. Using the required function ensures that Helm fails fast with a clear error message if these mandatory fields are missing when disaggregated mode is enabled.

          image: {{ required (printf "%s.image is required when serving_mode is disaggregated" .role) $pool.image }}

{{- end }}
command: ["/bin/bash", "-c"]
args:
- {{ printf "if [ \"${LWS_WORKER_INDEX:-0}\" = \"0\" ]; then\n%s\nelse\n%s\nfi" $pool.bash_command $pool.worker_bash_command | quote }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If $pool.bash_command or $pool.worker_bash_command is null, the printf function will output %!s(<nil>) into the bash script, causing syntax errors and container startup failures. Using required for bash_command and defaulting worker_bash_command to an empty string (or bash_command) prevents this.

          - {{ printf "if [ \"${LWS_WORKER_INDEX:-0}\" = \"0\" ]; then\n%s\nelse\n%s\nfi" (required (printf "%s.bash_command is required" .role) $pool.bash_command) ($pool.worker_bash_command | default "") | quote }}

{{- end }}
containers:
- name: vllm-proxy
image: {{ .Values.proxy.image }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since proxy.image is null by default in values.yaml, enabling disaggregated serving mode without configuring it will result in invalid Kubernetes manifests. Using the required function ensures that Helm fails fast with a clear error message.

        image: {{ required "proxy.image is required when serving_mode is disaggregated" .Values.proxy.image }}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant