Task runner: :exec-fn / :cmd CLI tasks, via a vendored babashka/cli - #1084
Conversation
host/chez/stdlib-fasl-manifest.txt pins the exact set of install-owned stdlib namespaces that get compiled to fasls and embedded. Adding vendor/cli/src and stdlib/jolt/cli.clj introduced babashka.cli, babashka.cli.exec, babashka.cli.internal and jolt.cli, so the discovered set became 83 against a manifest of 79 and build-jolt.ss failed with manifest drift. The drift check is strict by design and names the required set, so this is the edit it asks for. Verified by running the packaged build end to end and the smoke the flake workflow runs.
A bb.edn task could only be a body to run. babashka's task runner also takes a task that names a HANDLER (:exec-fn) or a tree of them (:cmd), and parses the command line for it before anything runs -- which is what gives such a task coercion, validation, subcommands and --help without the project writing a parser. That is what jolt-lang#878 said was out of reach ("jolt has no per-task option metadata today"), and the vendored babashka/cli is what puts it in reach. The semantics are babashka 1.13.219 + 1.13.221 + 1.13.223, and the reference is babashka.impl.tasks at that version. babashka assembles a program STRING and hands it to SCI, so its cli-node / -task-node / -cli-dispatch / -run-cli-dep are code generators; jolt evaluates the task map directly, so the same four shapes are ordinary function calls here. The names are kept so the two can be read side by side. What a task map may now say: serve {:exec-fn app/serve :cli {:spec {:port {:coerce :long}}}} db {:cmd {"migrate" {:exec-fn app/migrate} "seed" {:exec-fn app/seed}}} :cli holds everything babashka.cli takes, so reading a task map tells you which keys are jolt's and which are the parser's, and it may name a def instead of a literal map -- which is where options edn cannot express live, such as an :error-fn. :cmd may name a def too, for a large tree. :exec-args may sit on the task as well as under :cli, the task's winning. A spec may live on the handler var as :org.babashka/cli metadata, which is where `bb -x` reads it from, and the var's docstring becomes the node's :doc. Two things about the ORDER, which is the part that is easy to get wrong: * --help must not run the task. So the :depends walk, the body and the :enter/:leave pair all go into the dispatch as thunks, and the parser decides whether any of them is called. Asking what a task accepts now runs none of its dependencies, where before the walk happened first and the arguments were never looked at. * A CLI task named in :depends does not parse on its own (1.13.221). Its handler is called in its own place in the graph with the options the TARGET's parse produced -- only the ones actually supplied, over its own and the runner-level defaults -- and :restrict narrows that to the keys it declares. Its spec merges into the target's dispatch spec, so those options parse at the target and print in its --help. Naming a :cmd task in :depends is refused: a tree of commands has no single handler to run, and picking one for the project would be a guess. Errors here are plain ex-info, not babashka's {:babashka/exit 1}: in jolt that key means "exit with this status, the failure has already reported itself", which is right for a failed subprocess and wrong for a typo in a bb.edn -- it would make an unresolvable :exec-fn exit 1 in silence. One babashka behaviour is deliberately not here. It falls back to a handler's DOCSTRING for a task with no :doc, in `bb tasks` and in completion. Both read the project's config files alone on jolt, and deriving that doc means loading the handler's namespace -- which is exactly what a listing and a completing shell must not do. The docstring still reaches --help, where the fn is being resolved anyway. babashka.cli is required on first use, like babashka.tasks beside it: a :tasks map that names no :exec-fn and no :cmd parses nothing, and loading 2,500 lines of parser for a task that only shells out is what the lazy require in this namespace exists to avoid. Tests: test/chez/tasks/cliproj, a fixture whose handlers print the map they were called with, and 33 rows in tasks-smoke over it -- the parse and its defaults, :exec-args precedence, a spec on the var, :cli and :cmd by symbol, --help and -h, a :cmd tree and a tree with a body as its root, the :depends handling in both directions, the :cmd-in-:depends refusal, a runner-level :cli, a throwing handler, and `(run 'task)` reaching a CLI task from a body. `make taskssmoke` is 102 passed, 0 failed.
jolt's completion design is a split by how often a thing changes: its own
commands are baked into the generated snippet, and a project's tasks are
fetched once with `jolt completions tasks` and cached against the mtimes
of deps.edn and bb.edn. A warm press reads a small file and jolt does not
run, which matters because jolt's floor is ~0.22s and a quarter-second
press feels broken.
A task that PARSES its arguments cannot be cached that way: what it
accepts depends on where the cursor is. babashka answers this by calling
its binary back on every press, through a hidden
`org.babashka.cli/completions` command that babashka.cli's own stubs
know. jolt now answers that same command -- but only for the tasks that
need it:
* `completions tasks` gains a third field, `cli`, on a task with an
:exec-fn or a :cmd (and an empty doc field, so field 3 stays field 3).
* the zsh, bash and fish snippets read that marker. After a task name
they call back only when the task carries it; every other task falls
through to the shell's own file completion exactly as before, forking
nothing.
* jolt.main answers `org.babashka.cli/completions complete --shell SHELL
-- <task> <token>...` by building the task's dispatch tree and handing
the tokens to babashka.cli, and `... snippet` by printing jolt's own
snippet, since that is the one that knows how to call back.
Nothing on that path may report a failure. A completing shell discards
stderr and reads stdout as the candidate list, so a stale bb.edn or a
project whose deps don't resolve has to come back as babashka.cli's
file-completion marker: an error there would offer nothing AND suppress
the shell's fallback, which looks like completion being broken rather
than like the project being broken. The project is applied, the task's
:requires run and the tree is built inside that guard, with stdout muted
so a namespace that prints on load cannot become a candidate.
The zsh snippet's _jolt_add also had to learn that a line may have three
fields: it took everything after the first tab as the description, which
with a marker present would have rendered as "a doc<TAB>cli".
Tests: 14 rows in completions-smoke -- the marker in the task lines, the
callback itself (options, subcommands, and the file-completion answer for
a plain task, an unknown task and a project with no tasks), the bash
function actually completing a CLI task's options and subcommands, and
the two spawn-count rows that are the whole point: a press after a plain
task starts jolt not at all, and after a task that parses, exactly once.
`make completionssmoke` is 40 passed, 0 failed.
test/cli_shim_test.clj was added with the vendoring but nothing ran it: no Makefile target, no CI-GATES entry, so it could rot silently. It pins what jolt.cli re-exports -- that the surface reaches the parser, the dispatcher and the help renderer -- and that *exit-fn* is NOT among them, which is deliberate: it is a dynamic var, and import-vars can only re-export a delegating fn. `make clishim`, in CI-GATES next to the other offline source-mode gates. The task runner's own use of babashka.cli is covered by taskssmoke.
The README had no home for the task runner beyond passing references, so this adds the section the feature needs: what an :exec-fn / :cmd task map looks like, what --help does and what it does not do (run the task's :depends), how a CLI task in :depends is handled, and that the parser is vendored at vendor/cli and re-exported as jolt.cli for a program to use directly. Shell completion gains the paragraph for the third `cli` field and the one case that does call jolt back, and llms.txt records both the field and the hidden callback command.
|
Picked this up and pushed the rest of the plan to this branch — the nine
Two ordering points, which are the parts worth reviewing:
Errors are plain
Your manifest fix landed while I was working on the same thing; I dropped my One deliberate divergence from babashka, noted in the source: bb falls back to a Gates run locally against the built binary: Taking it out of draft on that basis; say the word if you would rather keep |
yogthos
changed the title
|
Answering #1083: yes — the vendored What I did to the branch:
What I checked, beyond the gates: every semantic row against real The three rows where the installed Local gates on the merged tree: One follow-up filed rather than fixed here: Thanks for opening it as a question first — that was the right way to do it. |
2 participants
Closes #1083.
A
bb.edntask that names an:exec-fn— or a:cmdtree of them — has itsarguments parsed by
babashka.clibefore anything runs, which is what gives itcoercion, validation, subcommands and
--help. That is babashka's CLI-taskfeature, and this brings it to jolt at 1.13.219 + 1.13.221 + 1.13.223
semantics.
The PR was opened as direction-seeking, with two commits and the question
"do you want a vendored
babashka/cliin this tree?" (#1083). The answer isyes, and the remaining nine tasks of the plan are now here.
What landed
vendor/cli—babashka/cliatv0.12.91, registered onldr-install-rootsplus the four other places a vendored source root has toappear (
TESTBIN-INPUTS, threeset-source-roots!*sites).stdlib/jolt/cli.clj— the public face, followingstdlib/jolt/process.clj:(import-vars babashka.cli :exclude #{*exit-fn*}). Of the 24 public vars,*exit-fn*is the only oneimport-varscannot re-export, because it isdynamic and a delegating
defnwrapper would breakbinding.jolt.tasks—cli-node,task-node,cli-dispatch!andrun-cli-dep!,keeping babashka's names so
babashka.impl.tasksand this can be read side byside. babashka assembles a program string for SCI; jolt evaluates the task map
directly, so the same semantics are wired as ordinary function calls.
jolt.completions+jolt.main— the hiddenorg.babashka.cli/completionscallback, and zsh/bash/fish snippets that useit for CLI tasks only.
llms.txt— "Tasks that parse their arguments".The semantics, measured against babashka
Every row below was run against real
bbon the samebb.edn, not assumed.jolt <task> --helpandbb <task> --helpare byte-identical apart from theprogram name, inherited options included.
:exec-argson the task as well as under:cli, the bare key winning:cli:exec-argsmerge over the:taskstop-level, not replace:clispec adds to the runner-level spec;--helplists the rest underInherited options:exec-fnruns when another task:dependson it--help:cmdtask in:dependsunless it also has a:taskbody:cmdmay be a symbol naming a var holding the tree; its namespace loads on demandtasksand--helpstill describe a task when a dependency's namespace will not load:depends:exec-fnreceives all parsed options, runner-level:clidefaults included:restrict, only the options it declares:depends:restrictfalls back to the runner-level one:depends:exec-fnreceives its own:exec-argsand spec defaults, command line winning; they do not apply to the target:exec-fntask in the:dependsof a task with a:taskbody runs, with its own:exec-argsand spec defaultsThe three rows the installed
bb(1.13.220) answers differently are exactly the1.13.221 / 1.13.223 changes, and jolt takes the newer side of each — checked
against
babashka.impl.tasksat 1.13.223 rather than against the older binary.--helpshort-circuits before the:dependswalk: the body, the dep walk andthe
:enter/:leavepair all go in as thunks the parser calls only once it haspicked a command, so asking what a task accepts runs nothing. Hook placement,
the diamond (a shared CLI dep runs once),
--parallel, andbabashka.tasks/runreaching a CLI task were each checked againstbbtoo.Two deliberate differences
ex-info, not{:babashka/exit 1}. In jolt that keymeans "exit with this status, the failure has already reported itself", which
is right for a failed subprocess and wrong for a mistake in a
bb.edn— itwould make an unresolvable
:exec-fnexit 1 in silence. A bad task map isreported the way jolt already reports
unknown command or task. Exit codesmatch babashka's.
:docinjolt tasksand completion.Both read the project's config files alone on jolt, and deriving that doc
means loading the handler's namespace — a completing shell must not be what
discovers your deps do not resolve. A docstring still reaches
--help, wherethe fn is being resolved anyway.
Completion costs a plain task nothing
jolt completions tasksmarks a task that parses with a thirdclifield, sothe snippets call jolt back only for those. What such a task accepts depends on
where the cursor is and cannot be cached as a flat list; every other task still
completes from the cache without starting jolt at all.
Gates
clishim(new, inCI-GATES),taskssmoke102,completionssmoke40,stdlibfasl11,documented,deadhost,portcheck,depssmoke,scriptsmoke, plus the full CI matrix. Fixture:test/chez/tasks/cliproj.No gate was added for the submodule itself:
vendor/fsandvendor/processareplain submodules with none, and
grenadinecheckexists only to reconcilegrenadine's separate generated-sources tree, which
clihas no equivalent of.Relationship to #878
#878 shipped
jolt completionsin v0.8.5, completing task names, and saidper-task argument completion was out of reach because "jolt has no per-task
option metadata today". This closes that gap without changing what #878 shipped.