Skip to content

Invoke pipeline helpers as modules, not repo-relative paths #367

Description

@amc-corey-cox

Two pipeline recipes still shell out to source files by repo-relative path:

pipeline.Makefile:252	$(RUN) python src/dm_bip/cleaners/prepare_input.py
pipeline.Makefile:371	./src/dm_bip/cleaners/extract_conditions.sh

Everything else goes through python -m dm_bip..... Those two exceptions mean any environment running the pipeline needs the source tree laid out exactly as the repo has it, not just dm-bip installed. That is why the container does COPY . ./ and carries tests, toy data, and BDC workflow scripts along with the code it actually runs.

Intended outcome: the pipeline invokes its own code the same way everywhere, and a runtime environment needs the installed package plus the two Makefiles rather than a copy of the repo.

Worth doing on its own — it makes the image smaller and the packaging story simpler — and it is a prerequisite for splitting the container image.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Pipeline ImprovementImproving Automation, Integration, Testing, etc.futureFuture work — not on current development roadmap

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions