Skip to content

Map step reports success when an entity fails and produces no output #361

Description

@amc-corey-cox

In non-strict mode (DM_MAP_STRICT=false), an entity whose mapping fails outright is indistinguishable from one that succeeded. The pipeline prints ✓ Data mapping complete and carries on with an empty output file for that entity.

Seen on toy data: Demography's spec derives sex_derived from phv00000006, which isn't in the source schema. linkml-map raises ValueError: No such slot phv00000006 as an attribute of pht000001 ancestors or as a slot definition in the schema, exits 1, and writes a zero-byte TOY-Demography--data.yaml. Because the toy config sets DM_MAP_STRICT := false, the per-entity recipe tolerates the non-zero exit and touches the completion sentinel. The Mapping Error Summary doesn't catch it either — it greps logs for the literal string transformation error, which this failure never emits.

Non-strict mode exists to tolerate row-level errors so partial output still lands and other entities can proceed. It shouldn't also swallow a whole-entity failure that produced nothing.

This is the same shape as #337: the map step's success criteria are process-level (exit code, signal check, log grep) and never ask whether output was actually produced. #358 catches it after the fact, since the output-validation report flags empty entity files — but the map step itself should notice.

Worth deciding whether zero records is a hard failure even in non-strict mode, or a loud warning that reliably lands in the error summary.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Pipeline ImprovementImproving Automation, Integration, Testing, etc.futureFuture work — not on current development roadmap

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions