Write anatomical orientation if available (RFC-4)

[melissalinkert]

Closes #329. This depends upon an open Bio-Formats pull request (ome/bioformats#4454), so is a draft for now. This will also need to make use of the 1.0-DEV version support and tests added in #330, as RFC-4 support should only be written with --ngff-version 1.0-DEV. I just didn't want to introduce conflicting commits here.

As noted in #329, the general idea is that Bio-Formats proper implements most of RFC-4, including the controlled vocabulary therein. This pull request simply consumes axis orientations provided by Bio-Formats. The API defined in Bio-Formats and consumed here should be flexible enough to handle non-anatomical orientations, if that were to be added in a future RFC.

Basic tests make use of FakeReader, but Analyze, Nifti, MINC, Bruker, Varian, and DICOM are all candidates for testing once we have suitable reference data (see #329 (comment)).

I expect the build to fail here due to the need for ome/bioformats#4454. To test locally, check out that Bio-Formats branch, then mvn clean package install, then ./gradlew clean build on this pull request's branch.

[melissalinkert]

This PR and ome/bioformats#4454 have now been updated following 0.13.0-rc1/Bio-Formats 9.0.0-rc1, so tests are now in the correct location and the orientation metadata should only be written with --ngff-version 0.9.dev1.

Build failure is still expected here, since this depends upon new API. The manual build process in the PR description should result in a usable build with passing tests. I have no further planned changes at this point, so taking out of draft.

[sbesson]

Similarly to the testing of #330, the set of samples collected and shared in #329 (comment) have been converted via a build of this PR and ome/bioformats#4454 using different versions and options

#! /bin/sh

# Generates OME-Zarr 0.5 and 0.9.dev1 (with RFC-4) from a series of
# public sample files i
# The following application are expected alongside this script and the source files
# - a build of bioformats2raw with https://github.com/glencoesoftware/bioformats2raw/pull/335/

set -e
set -x

BASE_PATH=~/Desktop/RFC-4

# Clean the generated folders
rm -rfv $BASE_PATH/ome-zarr
for format in analyze bruker dicom fdf; do
  mkdir -p $BASE_PATH/ome-zarr/0.5/default/$format
  mkdir -p $BASE_PATH/ome-zarr/0.5/compact/$format
  mkdir -p $BASE_PATH/ome-zarr/0.9.dev1/default/$format
  mkdir -p $BASE_PATH/ome-zarr/0.9.dev1/compact/$format
done

convert () {
  echo "Converting $1"

  dir="$(dirname $1)"
  file="$(basename $1)"
  zarr="$dir/${file%.*}.zarr"
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.5/default/$zarr --ngff-version 0.5 -p
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.5/compact/$zarr --ngff-version 0.5 --compact -p
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.9.dev1/default/$zarr --ngff-version 0.9.dev1 -p
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.9.dev1/compact/$zarr --ngff-version 0.9.dev1 --compact -p
}

convert analyze/avg152T1.hdr
convert bruker/rat_PV5.1/9/acqp
convert bruker/rat_PV5.1/4/acqp
convert bruker/rat_PV5.1/3/acqp
convert bruker/human_PV5.1/1/acqp
convert bruker/human_PV5.1/2/acqp
convert bruker/PV6.0_FLASH/acqp
convert dicom/real_ct_oblique.dcm
convert dicom/real_ct_small.dcm
convert dicom/TC-05_HFDR.dcm
convert dicom/TC-06_HFDL.dcm
convert fdf/synthetic_HFS_supine.fdf
# java.lang.IllegalArgumentException: Negative position
# convert fdf/test.fdf
# java.lang.NullPointerException
# convert minc/icbm152_t1_2mm_xyflip.mnc
# convert minc/icbm152_t1_2mm_zflip.mnc
# convert minc/mni_icbm152_t1.mnc
convert nifti/avg152T1_LR_nifti.nii.gz
convert nifti/avg152T1_RL_nifti.nii.gz
convert nifti/TC-10_canon_DTI_oblique_20d.nii.gz
convert nifti/TC-11_qform_sform_mismatch.nii.gz
convert nifti/TC-23_canon_DTI_axial.nii.gz
convert nrrd/MRHead.nrrd

Results have been uploaded to a public AWS S3 bucket under s3://gs-public-zarr-dev/RFC-4 and can be accessed

Source	--ngff-version=0.5	--ngff-version=0.5 --compact	--ngff-version=0.9.dev1	--ngff-version=0.9.dev1 --compact
analyze/avg152T1.hdr	validator	validator	validator	validator
bruker/rat_PV5.1/9/acqp	validator	validator	validator	validator
bruker/rat_PV5.1/4/acqp	validator	validator	validator	validator
bruker/rat_PV5.1/3/acqp	validator	validator	validator	validator
bruker/human_PV5.1/1/acqp	validator	validator	validator	validator
bruker/human_PV5.1/2/acqp	validator	validator	validator	validator
bruker/PV6.0_FLASH/acqp	validator	validator	validator	validator
dicom/real_ct_oblique.dcm	validator	validator	validator	validator
dicom/real_ct_small.dcm	validator	validator	validator	validator
dicom/TC-05_HFDR.dcm	validator	validator	validator	validator
dicom/TC-06_HFDL.dcm	validator	validator	validator	validator
fdf/synthetic_HFS_supine.fdf	validator	validator	validator	validator
nifti/avg152T1_LR_nifti.nii.gz	validator	validator	validator	validator
nifti/avg152T1_RL_nifti.nii.gz	validator	validator	validator	validator
nifti/TC-10_canon_DTI_oblique_20d.nii.gz	validator	validator	validator	validator
nifti/TC-11_qform_sform_mismatch.nii.gz	validator	validator	validator	validator
nifti/TC-23_canon_DTI_axial.nii.gz	validator	validator	validator	validator
nrrd/MRHead.nrrd	validator	validator	validator	validator

[sbesson]

Enumerating issues identified during this first round of testing.

• all MINC datasets and one of the FDF dataset fail to initialize and convert
• the NRRD and SVS reader don't implement the IAxisOrientationReader interface
• using https://huggingface.co/fideus-labs/ome-zarr-rfc4-data#source-formats--orientation-derived-from-the-native-header as the reference:
  • most of the bruker datasets have no orientation metadata except bruker/human_PV5.1/1/acqp for which the values of the orientation alongside the Y and Z axis do not match the table above
  • nifti/TC-23_canon_DTI_axial.nii.gz, nifti/TC-10_canon_DTI_oblique_20d.nii.gz and nifti/TC-11_qform_sform_mismatch.nii.gz: the X axis orientation does not match the table
  • fdf/synthetic_HFS_supine.fdf: the X axis orientation does not match the teable
    *dicom: there are several value mismatches

The majority of these issues are unrelated to the changes in this PR but come directly from Bio-Formats and the orientation implementation in ome/bioformats#4454. We might want to triage and prioritize the issues we want to tackle.

[melissalinkert]

Enumerating issues identified during this first round of testing.

• all MINC datasets and one of the FDF dataset fail to initialize and convert

Should be fixed with current state of ome/bioformats#4454.

• the NRRD and SVS reader don't implement the IAxisOrientationReader interface

SVS intentionally does not implement IAxisOrientationReader. I am not aware of any metadata in SVS files that can be used to determine the orientation, so we would need to assume the same orientation is used for all data (which I suspect is incorrect).

NRRD looks like it could implement IAxisOrientationReader, but not all NRRD data is medical data, and it is not clear how to determine if a particular dataset is medical data. See also https://teem.sourceforge.net/nrrd/format.html#space.

• most of the bruker datasets have no orientation metadata except bruker/human_PV5.1/1/acqp for which the values of the orientation alongside the Y and Z axis do not match the table above

Only the subject file is used to determine orientation, so presumably that's not sufficient. PV6.0_FLASH/acqp has:

##$ACQ_slice_orient=Arbitrary_Oblique
##$ACQ_patient_pos=Head_Prone

and rat_PV5.1/9 has:

##$ACQ_slice_orient=Arbitrary_Oblique
##$ACQ_patient_pos=Head_Supine

but since the table reports the same orientation for both datasets, those keys are probably not helpful. We'd likely need more information about how orientation is reported in this format, as we don't have a spec or acquisition software to assist.

The human_PV5.1 case at least should be fixed with current state of ome/bioformats#4454.

• nifti/TC-23_canon_DTI_axial.nii.gz, nifti/TC-10_canon_DTI_oblique_20d.nii.gz and nifti/TC-11_qform_sform_mismatch.nii.gz: the X axis orientation does not match the table

Likely that means we need to interpret https://nifti.nimh.nih.gov/nifti-1/documentation/faq.html#Q16 instead of hard-coding a single set of orientations as https://nifti.nimh.nih.gov/nifti-1/documentation/faq.html#Q14 implies.

• fdf/synthetic_HFS_supine.fdf: the X axis orientation does not match the teable

Should be fixed with the current state of ome/bioformats#4454

*`dicom`: there are several value mismatches

I think that means adjusting based on the 0020,0037 Image Orientation (Patient) value.

I'll try to sort out the dicom/nifti changes in the next day or two. Bruker/SVS/NRRD would need discussion and/or more information before making further changes.

[melissalinkert]

nifti and dicom data should now be ready to re-test with current state of ome/bioformats#4454.

[sbesson]

Using the latest state of the companion Bio-Formats PR, a new set of OME-Zarr datasets was generated from the input datasets including some sharding option for the Z dimension

#! /bin/sh

# Generates OME-Zarr 0.5 and 0.9.dev1 (with RFC-4) from a series of
# public sample files i
# The following application are expected alongside this script and the source files
# - a build of bioformats2raw with https://github.com/glencoesoftware/bioformats2raw/pull/335/

set -e
set -x

BASE_PATH=~/Desktop/RFC-4

# Clean the generated folders
rm -rfv $BASE_PATH/ome-zarr
for format in analyze bruker dicom fdf; do
  mkdir -p $BASE_PATH/ome-zarr/0.5/default/$format
  mkdir -p $BASE_PATH/ome-zarr/0.5/compact/$format
  mkdir -p $BASE_PATH/ome-zarr/0.9.dev1/default/$format
  mkdir -p $BASE_PATH/ome-zarr/0.9.dev1/compact/$format
done

convert () {
  echo "Converting $1"

  dir="$(dirname $1)"
  file="$(basename $1)"
  zarr="$dir/${file%.*}.zarr"
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.5/default/$zarr --ngff-version 0.5 -p $2
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.5/compact/$zarr --ngff-version 0.5 --compact -p $2
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.9.dev1/default/$zarr --ngff-version 0.9.dev1 -p $2
  ./bioformats2raw-0.13.0-SNAPSHOT/bin/bioformats2raw $BASE_PATH/sources/$1 $BASE_PATH/ome-zarr/0.9.dev1/compact/$zarr --ngff-version 0.9.dev1 --compact -p $2
}

convert analyze/avg152T1.hdr --shard-depth=91
convert bruker/rat_PV5.1/9/acqp --shard-depth=64
convert bruker/rat_PV5.1/4/acqp --shard-depth=64
convert bruker/rat_PV5.1/3/acqp --shard-depth=64
convert bruker/human_PV5.1/1/acqp --shard-depth=15
convert bruker/human_PV5.1/2/acqp --shard-depth=15
convert bruker/PV6.0_FLASH/acqp --shard-depth=144
convert dicom/real_ct_oblique.dcm
convert dicom/real_ct_small.dcm
convert dicom/TC-05_HFDR.dcm
convert dicom/TC-06_HFDL.dcm
convert fake/test1.fake --shard-depth=30
convert fake/test2.fake --shard-depth=30
convert fake/test3.fake --shard-depth=30
convert fake/test4.fake --shard-depth=30
convert fake/test5.fake --shard-depth=30
convert fake/test6.fake --shard-depth=30
convert fdf/synthetic_HFS_supine.fdf
convert fdf/test.fdf
convert minc/icbm152_t1_2mm_xyflip.mnc --shard-depth=97
convert minc/icbm152_t1_2mm_zflip.mnc --shard-depth=97
convert minc/mni_icbm152_t1.mnc --shard-depth=193
convert nifti/avg152T1_LR_nifti.nii.gz --shard-depth=91
convert nifti/avg152T1_RL_nifti.nii.gz --shard-depth=91
convert nifti/TC-10_canon_DTI_oblique_20d.nii.gz --shard-depth=40
convert nifti/TC-11_qform_sform_mismatch.nii.gz --shard-depth=30
convert nifti/TC-23_canon_DTI_axial.nii.gz --shard-depth=40
convert nrrd/MRHead.nrrd --shard-depth=130

The datasets have been uploaded to the same location as above. The updated table is as follows:

Source	--ngff-version=0.5	--ngff-version=0.5 --compact	--ngff-version=0.9.dev1	--ngff-version=0.9.dev1 --compact
analyze/avg152T1.hdr	validator	validator	validator	validator
bruker/rat_PV5.1/9/acqp	validator	validator	validator	validator
bruker/rat_PV5.1/4/acqp	validator	validator	validator	validator
bruker/rat_PV5.1/3/acqp	validator	validator	validator	validator
bruker/human_PV5.1/1/acqp	validator	validator	validator	validator
bruker/human_PV5.1/2/acqp	validator	validator	validator	validator
bruker/PV6.0_FLASH/acqp	validator	validator	validator	validator
dicom/real_ct_oblique.dcm	validator	validator	validator	validator
dicom/real_ct_small.dcm	validator	validator	validator	validator
dicom/TC-05_HFDR.dcm	validator	validator	validator	validator
dicom/TC-06_HFDL.dcm	validator	validator	validator	validator
fake/test1.fake	validator	validator	validator	validator
fake/test2fake	validator	validator	validator	validator
fake/test3.fake	validator	validator	validator	validator
fake/test4.fake	validator	validator	validator	validator
fake/test5.fake	validator	validator	validator	validator
fake/test6.fake	validator	validator	validator	validator
fdf/synthetic_HFS_supine.fdf	validator	validator	validator	validator
fdf/test.fdf	validator	validator	validator	validator
minc/icbm152_t1_2mm_xyflip.mnc	validator	validator	validator	validator
minc/icbm152_t1_2mm_zflip.mnc	validator	validator	validator	validator
minc/mni_icbm152_t1.mnc	validator	validator	validator	validator
nifti/avg152T1_LR_nifti.nii.gz	validator	validator	validator	validator
nifti/avg152T1_RL_nifti.nii.gz	validator	validator	validator	validator
nifti/TC-10_canon_DTI_oblique_20d.nii.gz	validator	validator	validator	validator
nifti/TC-11_qform_sform_mismatch.nii.gz	validator	validator	validator	validator
nifti/TC-23_canon_DTI_axial.nii.gz	validator	validator	validator	validator
nrrd/MRHead.nrrd	validator	validator	validator	validator

Overall, we are now in a very good place

• all source datasets convert into OME-Zarr without error
• all source datasets include some orientation metadata associated with the axes metadata when converted using --ngff-version 0.9.dev1 with the exception of the nrrd, svs, bruker/PV6.0_FLASH, bruker/rat_PV5.1 and fdf/test.fdf datasets. The latter is expected to have no patient position as per the table and all the other datasets have been discussed in the comment above
• for all OME-Zarr datasets with orientation metadata, confirmed that the orientation values now match the ground truth in https://huggingface.co/fideus-labs/ome-zarr-rfc4-data#source-formats--orientation-derived-from-the-native-header for each axis. When converting data in --compact mode, the orientation might be missing alongside the third spatial axis as the dimension is 1. When using the default 5D conversion, it is filled with the correct value.

One last outstanding issue is the conversion of the MINC with the flipped orientation. The scale vector in the coordinateTransformations include negative values which are incompatible with the assumptions of the JSON schemas - see https://ngff.openmicroscopy.org/specifications/dev/schemas/coordinate_transformations.html#coordinate-transformations-root-items-allof-item-0-allof-item-1-oneof-scale-transformation-scale-scale-items

[melissalinkert]

One last outstanding issue is the conversion of the MINC with the flipped orientation. The scale vector in the coordinateTransformations include negative values which are incompatible with the assumptions of the JSON schemas - see https://ngff.openmicroscopy.org/specifications/dev/schemas/coordinate_transformations.html#coordinate-transformations-root-items-allof-item-0-allof-item-1-oneof-scale-transformation-scale-scale-items

Should be fixed with ome/bioformats@6322349.

Note there is one additional change for DICOM (ome/bioformats@b5c903a), which should fix the failing build in https://github.com/ome/bioformats/actions/runs/36719223688. This change should not impact reported orientations for any of the DICOM test data.

[sbesson]

With the latest changes introduced in the upstream PR reconverted all sample files including a couple of syntetic .fake files testing the anatomical terms that were not covered by the other samples. The generation script and the table of validator links to the public bucket prefix in #335 (review) have been updated accordingly.

All original formats convert without issue, all OME-Zarr datasets validate against the JSON schemas and the values of the orientation terms are now matching the expectations. The only remaining change will be to bump Bio-Formats to 9.0.0-rc2 including the API and reader changes when it's available and this should be ready for inclusion.