Skip to content

Key SSSOM duplicate check by searched vocabulary, not bare NoTermFound - #68

Merged
KrishnaTO merged 1 commit into
mainfrom
claude/fix-sssom-duplicate-key
Aug 16, 2026
Merged

Key SSSOM duplicate check by searched vocabulary, not bare NoTermFound#68
KrishnaTO merged 1 commit into
mainfrom
claude/fix-sssom-duplicate-key

Conversation

@KrishnaTO

Copy link
Copy Markdown
Owner

Problem

Validate mappings fails on PR #67 with 7 duplicate-row errors that are not duplicates.

The SSSOM duplicate check keys each row as (subject, object_id, modifier). But every
manual-absent row carries the same literal sssom:NoTermFound object — the vocabulary
that was actually searched lives in object_source. The script says so itself:

manual-absent records "we looked and there is no term"; SSSOM spells that as the object
sssom:NoTermFound with object_source naming the vocabulary that was searched.

So these five distinct findings collapse to one key:

ARI:0001065  Chronic Lyme disease  sssom:NoTermFound  ORPHA
ARI:0001065  Chronic Lyme disease  sssom:NoTermFound  DOID
ARI:0001065  Chronic Lyme disease  sssom:NoTermFound  SNOMEDCT
ARI:0001065  Chronic Lyme disease  sssom:NoTermFound  icd10cm
ARI:0001065  Chronic Lyme disease  sssom:NoTermFound  OMIM

Four get reported as duplicates of the first. "We checked ORPHA and found nothing" and
"we checked OMIM and found nothing" are different curation facts about different
vocabularies, and the gate rejects the second one.

Fix

sssom_key() already resolved this correctly for the cross-file comparison — only the
intra-file duplicate check missed it. This factors that resolution into
distinct_object_id() and calls it from both, so the two cannot drift apart again.

sssom_key() is not reused wholesale for the duplicate check, because it also folds in
author_id: two different curators asserting the same mapping is a duplicate worth
flagging, and keying on the author would hide it.

The disambiguated value also feeds the contradiction check, which needs the same
per-vocabulary comparison — and its message now names the vocabulary
(ORPHA:NoTermFound) rather than a bare sentinel that names none.

Verification

errors warnings
main, before 0 17
main, after 0 17
PR #67, before 7 18
PR #67, after 0 18

No-op on main's current data, so no regression. It clears all 7 false errors blocking
#67 without touching that branch's data. Warning counts are untouched in both cases.

🤖 Generated with Claude Code

The SSSOM duplicate-row check keyed rows on `object_id`, but every
`manual-absent` row carries the same literal `sssom:NoTermFound` object --
the vocabulary actually searched is recorded in `object_source`. Five rows
stating "no ORPHA term", "no DOID term", "no SNOMEDCT term", "no icd10cm
term" and "no OMIM term" for one subject therefore produced one identical
key, and four were reported as duplicates of the first.

`sssom_key()` already resolved this correctly for the cross-file
comparison; only the intra-file duplicate check missed it. Factor that
resolution into `distinct_object_id()` and call it from both, so the two
cannot drift apart again.

The disambiguated value also feeds the `contradiction` check, which needs
the same per-vocabulary comparison, and whose message now names the
vocabulary (`ORPHA:NoTermFound`) instead of a bare sentinel.

Counts on main are unchanged (0 errors, 17 warnings before and after).
PR #67, which adds 24 manual-absent rows and trips 7 of these false
errors, goes to 0 errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@KrishnaTO
KrishnaTO merged commit a6fb0b8 into main Aug 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant