[matthiaskoenig]

Summary

Simulations of a fit

• A group of fit mappings simulates the start and the times of its data, with the selections of its own mappings only. Before, a mapping took the selections of every earlier mapping of its experiment, and the changes between the data times were extra output times.
• The same holds for groups which read observables that are timecourses without reductions, e.g. Formula("cp", "[Cve_hctz] * Mr_hctz"). They are evaluated on that output. Before, they ran the whole simulation and interpolated linearly.
• The evaluation of a fit (residuals(complete_data=True), the report) simulates the output of the simulation for the curves. Before, the curves of these groups were straight lines between the data times.

Finite differences follow the noise of the integrator

• The least squares optimizer gets diff_step = sqrt(relative_tolerance) of the settings (1e-3 for the default 1e-6) unless it is given. This also applies to fit_cli (was 0.05) and the re-optimizations of a profile (was 0.05). The step used is recorded with each run and stated in the methods.
• fisher_information takes central differences with the cube root of the relative tolerance (1e-2), was 1e-6.

The default step of SciPy measured the noise of the integrator. Every run of the HCTZ fit stopped next to its start values (xtol). At a step of 1e-6, the Fisher matrix of the HCTZ optimum was off by a factor 143 and its standard errors were up to 30x too small.

Benchmarks

Simulations of one fit step on the HCTZ fit (pytest -m benchmark -n 0 -s tests/fit/test_benchmark.py):

Simulated output	Time
start and data times, mapping selections (what the fit uses)	5.5 ms
full timecourse, mapping selections (report curves)	12.6 ms
full timecourse, all 116 selections	21.8 ms

Default settings, 20 runs from Latin hypercube starts, develop → this branch:

Problem	One step	20 runs	Best cost	Median cost	Runs within 1 % of best
HCTZ fit	5.8 → 5.7 ms	4.8 → 8.2 s (324 → 480 evaluations)	28.70 → 28.40	190.6 → 28.40	1/20 → 17/20
Tutorial fit (cp formula)	2.0 → 1.0 ms	1.24 → 0.77 s	425.865 → 425.867	425.865 → 425.867	20/20 → 20/20

The HCTZ runs take longer because they now optimize instead of stopping at their start values.

Fisher information of the HCTZ optimum, against a reference at rtol=1e-12:

Step	Max relative error of the matrix
1e-6 (old)	143
1e-3	4.5e-3
1e-2 (new)	8.6e-4

Checklist

• the pull request targets develop
• tests were added or updated for the change
• pytest and ty check pass locally
• ruff check and ruff format are clean
• public functions and classes have type annotations and a docstring
• user visible changes are in release-notes/ (with the release commit) and, if needed, in docs/