fix(workflows): reject a non-string prompt in prompt-step validate()#3582
Merged
mnriem merged 1 commit intoJul 17, 2026
Merged
Conversation
`PromptStep.execute` str()-coerces `config['prompt']` and dispatches the
result to the integration CLI as the model's instructions. But its `validate`
only checked that `prompt` was *present*, not that it was a string — the exact
parity gap the sibling `ShellStep` closes for `run`.
So a YAML authoring slip like `prompt: [review, this]` or `prompt:` (null)
passed validation, then `execute` sent the Python repr (`"['review', 'this']"`,
`"None"`) to the LLM verbatim — silently wrong instructions with no error and a
COMPLETED status. The engine does not auto-validate step config
(`load_workflow` explicitly defers validation), so validation is the only place
this surfaces before dispatch.
Extend `validate` to reject any non-string `prompt` with the shell-step's
phrasing ("'prompt' must be a string, got <type>"), mirroring the shell `run`
and command `input`/`options` type checks. A `{{ ... }}` expression is still a
str, so it stays valid. Adds regression coverage for non-string prompts
(null/list/int/dict) and confirms an expression prompt still validates.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds validation to prevent non-string prompt values from being dispatched as Python representations.
Changes:
- Rejects non-string
promptvalues during validation. - Tests invalid prompt types and expression-string acceptance.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/workflows/steps/prompt/__init__.py |
Adds prompt type validation. |
tests/test_workflows.py |
Covers invalid prompt types and expressions. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Medium
Collaborator
|
Thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PromptStep.executestr()-coercesconfig['prompt']and dispatches the result to the integration CLI as the model's instructions:But
PromptStep.validateonly checked thatpromptwas present, not that it was a string — the exact parity gap the siblingShellStepalready closes for itsrunfield (which cites "the command-step input/options and gate options type checks" as precedent).So a YAML authoring slip like
prompt: [review, this]or a bareprompt:(YAML null) passed validation, thenexecutesent the Python repr ("['review', 'this']","None") to the LLM verbatim — silently wrong instructions, no error, and aCOMPLETEDstatus. The engine does not auto-validate step config (load_workflowexplicitly defers validation tovalidate_workflow()/engine.validate()), so validation is the only place this surfaces before dispatch.Fix
Extend
validateto reject any non-stringprompt, mirroring the shell-steprunand command-stepinput/optionstype checks:A
{{ ... }}expression is still astr, so it stays valid.Testing
test_validate_rejects_non_string_prompt(parametrized over null/list/int/dict), mirroringTestShellStep::test_validate_rejects_non_string_run.test_validate_accepts_expression_promptto confirm an expression prompt still validates.TestPromptStep+ validation suite passes (85 selected, 0 failures).🤖 Generated with Claude Code