Repository navigation
A run of experiments shows its progress and where the time went - #399
Merged
Merged
Conversation
matthiaskoenig
force-pushed
the
feat/runner-progress
branch
2 times, most recently
from
October 24, 2026 20:41
c7ebf90 to
1bb5be9
Compare
`ExperimentRunner.run_experiments` shows its progress with rich on the console: a bar over the experiments and a row per running experiment with its activity and steps, also for the experiments of the workers of a pool, which send their events through a queue the workers get from a new initializer of `parallel.start_pool`. The run ends with a table of the seconds every experiment took in every phase (models, initialize, simulations, data, figures, outputs, other), with the totals, the shares and a bar of the phases per experiment. The phases are measured where they run by a `PhaseClock` of the new module `sbmlsim.experiment.progress` and carried in `ExperimentResult.timings`, from a worker in its `_Outcome`. The `timeit` log lines of the run path are gone. `progress=None` shows the progress on an interactive terminal or in a notebook only, `False` prints nothing, the bars of the scans included.
The table of the timings was cut on a console narrower than about 80 columns: the names and the totals vanished, which the continuous integration on windows and macOS showed. A console too narrow for the seconds of every phase now gets the totals of the experiments and of the phases as two small tables. A console whose encoding is no UTF, e.g. a redirected output in cp1252 on windows, failed with a UnicodeEncodeError on the check marks; it gets ASCII marks, bars and lines instead. The tests of the progress render on a console of a fixed width.
matthiaskoenig
force-pushed
the
feat/runner-progress
branch
from
October 24, 2026 20:58
1bb5be9 to
f256777
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #391
A run of
ExperimentRunner.run_experiments(and of the functionrun_experiments) now shows its progress with rich on the console and ends with a table of where the time went. The timing log lines ofutils.timeiton the run path are gone.What a run shows
While it runs: a bar over all experiments and a row per running experiment with its activity (simulations, fit mappings, drawing figures, writing figures, writing datasets, writing JSON, loading models, initializing) and its steps, e.g.
3/7tasks. In a pool the workers report to the same display, so the rows are the experiments the workers run right now (the overall row counts an experiment when it ends, in the order they end).A frame in the middle of a run of 28 experiments on 4 workers:
At the end the bar stays as a single line and the timings follow (the HCTZ studies on 3 workers, 100 columns; in a terminal the bars are stacked in the colours of the column headers):
Before, the same run printed a
Running SimulationExperimentline and sixtimeitlines per experiment (_run_tasks <pid> 0.0379 [s], ...) through logging.Design
sbmlsim/experiment/progress.py(new):Phase(models,initialize,simulations,data,figures,outputs,other),PhaseClockwhich measures the phases where they run and reports them asEvents to the listener of the context (reporting, aContextVar),RunProgress(the rich display, thread safe) andtimings_table/show_timings.SimulationExperiment.runits simulations, fit mappings, figures and outputs;progress.advance()in the loops of_run_tasks,create_mpl_figures,save_mpl_figures, ... advances the activity. The runner addsother, the part of a run no phase measured (e.g. an overriddenrun), so the timings add up to the time an experiment took. The seconds areExperimentResult.timings(also into_dict), sent back from a worker in_Outcome, so the numbers are right for pool runs.multiprocessingqueue from a newinitializer/initargsofparallel.start_pool(a queue cannot be sent with a task) and put their events on it; a daemon thread of the parent forwards them to the display (_forwarded). A done callback of every future counts an experiment when its result arrives; an experiment whose worker died is noted in its row and counted when it ran again in the parent. A run without progress creates no queue.progress: bool | None = Noneon bothrun_experiments:Noneshows the progress on an interactive terminal (console.is_interactive, so not onTERM=dumb) or in a notebook, not when the output is redirected (no frames in files or in the generated outputs of the docs pages);Trueforces it (a non-terminal gets the last frame once),Falseprints nothing, also no bar of a scan of a task (Simulator.run(progress=scan_progress())). With progress, a scan which the simulator runs in a pool shows its bar nested below the rows (rich 15 nests live displays).models/initializecolumns are the ones of the constructor of the runner, in a pool the ones of the workers (which load the models again); a second run of the same runner has no initialization.otherwhen it is negligible), cuts long names before it drops the bars, drops the bars on a console too narrow for them, and on a console too narrow for the seconds of every phase prints the totals of the experiments and of the phases as two small tables. A console whose encoding is no UTF (e.g. a redirected output in cp1252 on windows) gets ASCII marks, bars and lines instead of failing with aUnicodeEncodeError.logger.infomessages of the run path (which experiment runs, why a run is serial) are debug now, the reason of a serial run is in the caption of the table.@timeitexposed the real signatures ofSimulationExperiment.runandcreate_mpl_figuresto ty: the return annotation ofcreate_mpl_figureswas wrong (dict[str, FigureMPL]), and the test experiments which overriderunwith an old signature on purpose carry# ty: ignore[invalid-method-override].Tests
tests/experiment/test_progress.py:progress=Falseprints and logs nothing (replacestest_a_run_prints_nothing, the new contract), no output by default when stdout is not a terminal,show_progress, the final line and table of a serial run with a failure, the phases of the timings (a model loaded once per runner,otherof an overridden run, a failed definition), the events of a serial run, the events of the workers of a pool, no queue without progress, a died worker counted once, a second run without initialization,PhaseClock, a failing listener,scan_progress, the table and its narrow fallback.Docs:
docs/experiments.mddescribes the progress and the timings, the API pageexperiment.progress, andCLAUDE.mdthe architecture.