Make FLANN matching reproducible by seeding OpenCV's RNG - #1123
Conversation
|
Hi @birgerbr! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
Closing in favour of OpenSfM/OpenSfM#42, which carries the same change to the community-maintained repo. Opened here first by mistake; this repo's Meta CLA requirement is not one we need to clear for a fix that belongs upstream of the community fork. |
Make FLANN matching reproducible by seeding OpenCV's RNG
build_flann_indexbuilds a FLANN index without fixing OpenCV's RNG. FLANN'sindex construction draws from
cv::theRNG(), which is thread-local and advancesbetween builds, so two builds from identical descriptors produce different
indexes — and with
processes > 1it also depends on which worker buildswhich index and in what order. Matching is therefore not reproducible.
This pins the RNG immediately before the index is built, making the index a pure
function of
(descriptors, seed), with the seed exposed asflann_random_seed.Reproducing it without any dataset
On OpenCV 4.11.0:
The third line is the control: the seed is what determines the index, so the
agreement on line two is not simply everything being constant.
Effect on the matching stage
On a 151-image dataset with
matcher_type: FLANN,flann_algorithm: KMEANS,processes: 8, running onlymatch_featurestwice and comparing theunpickled contents of
matches/*.pkl.gz:The "before" row is one pair of runs; a second pair of the same configuration
gave 11 / 151 identical. How much differs varies, which is itself the point.
The "after" row reproduced in two independent pairs of runs.
Feature files were bit-identical across both runs and the pair graph was
identical (same pair counts by source), so matching was the only stage
diverging. Match totals are unchanged in aggregate (~0.1%), as expected from a
differently-seeded approximate search rather than a behavioural change.
Note that
matches/*.pkl.gzdiffer byte-wise between any two runsregardless, because gzip records an mtime in its header — comparing the
compressed bytes will "confirm" non-determinism whatever the state of the code.
The numbers above come from comparing unpickled contents.
Scope, and what this does not fix
This addresses matching only. Reconstruction has separate, well-known
non-determinism from Ceres and pointer-based ordering, discussed in #213 and
#566; this change does not touch it, and a run can still differ downstream of
identical tracks.
Relation to existing reports:
matching results across machines and suggests
flann_tree: 1as a partialmitigation. Seeding addresses it directly and keeps the default index
structure.
this".
cv2.setRNGSeedis such a way for the FLANN path.Trade-off worth flagging
cv2.setRNGSeedpins OpenCV's thread-local RNG rather than only FLANN's,so it also fixes the starting state of any later OpenCV randomness on the same
thread — notably
cv2.findFundamentalMat(..., FM_RANSAC)inrobust_match_fundamentalon the perspective path. That moves in the samedirection (more reproducibility), but it is wider than the title suggests, and
OpenCV exposes no
getRNGSeed, so the previous state cannot be saved andrestored around the call.
Seeding by default also means existing users get a different index than they
would have got by chance. There is no stable baseline being broken — any two
runs already differed — but it is a change in output and worth a reviewer's
attention. If you would rather it be opt-in, a sentinel value could leave the
RNG untouched; say which you prefer and I will adjust.