Key SSSOM duplicate check by searched vocabulary, not bare NoTermFound - #68
Merged
Conversation
The SSSOM duplicate-row check keyed rows on `object_id`, but every `manual-absent` row carries the same literal `sssom:NoTermFound` object -- the vocabulary actually searched is recorded in `object_source`. Five rows stating "no ORPHA term", "no DOID term", "no SNOMEDCT term", "no icd10cm term" and "no OMIM term" for one subject therefore produced one identical key, and four were reported as duplicates of the first. `sssom_key()` already resolved this correctly for the cross-file comparison; only the intra-file duplicate check missed it. Factor that resolution into `distinct_object_id()` and call it from both, so the two cannot drift apart again. The disambiguated value also feeds the `contradiction` check, which needs the same per-vocabulary comparison, and whose message now names the vocabulary (`ORPHA:NoTermFound`) instead of a bare sentinel. Counts on main are unchanged (0 errors, 17 warnings before and after). PR #67, which adds 24 manual-absent rows and trips 7 of these false errors, goes to 0 errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Validate mappingsfails on PR #67 with 7duplicate-rowerrors that are not duplicates.The SSSOM duplicate check keys each row as
(subject, object_id, modifier). But everymanual-absentrow carries the same literalsssom:NoTermFoundobject — the vocabularythat was actually searched lives in
object_source. The script says so itself:So these five distinct findings collapse to one key:
Four get reported as duplicates of the first. "We checked ORPHA and found nothing" and
"we checked OMIM and found nothing" are different curation facts about different
vocabularies, and the gate rejects the second one.
Fix
sssom_key()already resolved this correctly for the cross-file comparison — only theintra-file duplicate check missed it. This factors that resolution into
distinct_object_id()and calls it from both, so the two cannot drift apart again.sssom_key()is not reused wholesale for the duplicate check, because it also folds inauthor_id: two different curators asserting the same mapping is a duplicate worthflagging, and keying on the author would hide it.
The disambiguated value also feeds the
contradictioncheck, which needs the sameper-vocabulary comparison — and its message now names the vocabulary
(
ORPHA:NoTermFound) rather than a bare sentinel that names none.Verification
main, beforemain, afterNo-op on
main's current data, so no regression. It clears all 7 false errors blocking#67 without touching that branch's data. Warning counts are untouched in both cases.
🤖 Generated with Claude Code