Repository navigation
0.1.7: reuse a finished result only when its inputs match - #57
Merged
Merged
Conversation
wilcoxon_test, t_test and nb_glm_test reused an existing output after comparing a few uns fields, so relabelling the input in place or changing a filter or `perturbations` returned the old p-values. The pseudo-bulk functions likewise ignored `perturbations` and the inner `min_cells`, and nb_glm_test, sort_by_perturbation and standardize_dataset reused a sorted or standardised copy on its file name alone. Every reusable output now carries uns["crispyx"]: the run fingerprint (source path, size, mtime and every result-shaping argument), the writer's version, whether it was an editable install, and a per-kind layout schema. A file is reused only when kind, schema and fingerprint match; otherwise the notice says why it is recomputed. Files without a stamp (0.1.6 and earlier) are recomputed once rather than read under the wrong layout. Operational arguments (chunk_size, memory_limit_gb, n_jobs, ...) stay out of the result identity, so batch_process now reuses a finished result across gene chunk widths; resume still checks the width. Sorted and standardised copies are written under a partial name. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Executed from an empty tutorial_outputs/, so every cell computes its result rather than reloading one written by an earlier version. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- batch_process: resume a partial output only when its provenance matches, so a rejected finished output is recomputed instead of returned as is. - Identify the input by its resolved path, so any spelling of it matches. - Wilcoxon: stamp the result last, after rank_genes_groups, so an interrupted write is recomputed rather than reused. - scanpy_format invalidates a finished result but no longer a checkpoint. - Pseudo-bulk: check for a reusable result before reading the matrix or aggregating. - One reuse check and fingerprint helper in _provenance; drop the unused editable flag. - Update the usage guide and API page; condense the changelog entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.
Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Before 0.1.7, an existing output could be returned stale after its input file or a result-shaping argument (e.g.
perturbations) changed. Each output now records inuns["crispyx"]the input file, every result-shaping argument, the crispyx version and its layout, and is reused only when these match. The notice says why a result is recomputed.Covers the DE functions,
batch_process, the pseudo-bulk functions,sort_by_perturbationandstandardize_dataset. Execution-only arguments (chunk_size,memory_limit_gb,n_jobs, ...) don't invalidate a result. Outputs from 0.1.6 or earlier have no record and are recomputed once after upgrading.Commits
5a99406adds the provenance stamp and reuse check.3fadf37bumps the version to 0.1.7.89c6dd2re-executes the tutorials on 0.1.7.b35ac45applies fixes from a code review of the above:batch_processwithresume=Trueno longer returns a finished output that the check rejected.scanpy_formatwrite is recomputed.scanpy_formatinvalidates a finished result but not a checkpoint.editableflag is gone.Testing
tests/test_result_provenance.pypasses, including six new regression tests. Four of them fail on89c6dd2.test_wilcoxon_dispatch.pytests that fix covers. That file now passes; the full suite was not rerun after it.🤖 Generated with Claude Code