Our pytest plugin creates a tmt-report-results.yaml, with overall result logic: if some pass, some skip > pass; if all skip > skip.
tmt seems to ignore that and does it's own logic based on sub-results, resulting in overall result "PASS", which can mask issues.
tmt-report-results.yaml:
- name: "/vdo/module_signature/test_signatures.py::test_signatures"
result: "skip"
Please consider honoring the result in tmt-report-results.yaml, adjust the logic so if all subresults are skipped the overall result is skip, ...(or have a native pytest framework plugin :P)
Our pytest plugin creates a tmt-report-results.yaml, with overall result logic:
if some pass, some skip > pass;if all skip > skip.tmt seems to ignore that and does it's own logic based on sub-results, resulting in overall result "PASS", which can mask issues.
Please consider honoring the result in tmt-report-results.yaml, adjust the logic so if all subresults are skipped the overall result is skip, ...(or have a native pytest framework plugin :P)