Repository navigation
Merge driver evaluation: data, figures, and a README that leads with them - #36
Open
jesusvilela wants to merge 1 commit into
Open
jesusvilela wants to merge 1 commit into
jesusvilela wants to merge 1 commit into
Conversation
…h it Four reproducible experiments under experiments/merge_driver/, each through real Git with the driver installed, judged by one test -- are the committed .semantic/ artifacts byte for byte what a rebuild produces? - replay: 150 concurrent change pairs on this repository via merge, rebase and cherry-pick. Text merge: 40 needless .semantic/ stops per operation; driver: 0, with the same 25 genuine source conflicts; 0 stale anywhere. - algorithms: git merge-file vs ort disagree on 8 of 400 random merges, 2 of them both clean with different text. - sweep: 32 Git operations and corner cases, all as expected, none stale. - scaling: batched ort merge 2.6x faster than per-file at 100-200 files, still linear (~35 ms per contested file). Trial-level CSV and JSON (with commit, Git, Python, platform) live in docs/evals/results/2026-10-15-merge-driver/, with a report labelled exploratory per the evaluation program, and light/dark SVG figures rendered from the data by a dependency-free script; a test keeps figures and data in sync. Colours were checked with the palette validator: green beside red failed colour-vision separation, so outcomes use blue / amber / red. `make merge-driver-eval` reproduces everything. The README opens with the results and the figures, and adds the driver to the quick start and the feature table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the experiment behind the merge driver as versioned, reproducible evidence, and gives the README a sharper opening built on it.
What's in it
experiments/merge_driver/: four experiments plus a dependency-free SVG renderer. Every merge runs through real Git with the driver installed, and is judged by one test: are the committed.semantic/artifacts byte for byte whataigit chunkwould produce?docs/evals/results/2026-10-15-merge-driver/:tests/test_experiment_plots.py: fails if a figure drifts from its data.make merge-driver-eval: reruns everything, in about ten minutes.Results
.semantic/stops per operation; driver: 0. The same 25 genuine source conflicts in both; 0 stale anywheregit merge-filevs ort, 400 random mergesREADME
It now opens with a "Merges that don't fight you" section: the headline tiles, the replay chart, three plain claims linked to the report, and a badge. It also adds
install-merge-driverto the quick start and a merging row to the feature table. Figures use<picture>to switch between light and dark.Colours were checked with a palette validator rather than by eye. Green beside red failed colour-vision separation (ΔE 4.1), so outcomes use blue, amber and red, always with text labels and a table view.
🤖 Generated with Claude Code