shell-sem-analysis

This project measures dissolution damage on SEM images of shells.

Scorers trace damaged areas by hand in Fiji (ImageJ) and save each traced area as an ROI file ("region of interest"). This project reads those ROI files and works out, for each shell and each scorer:

• the area of each type of damage, in µm²
• the percentage of the shell's surface each type of damage covers
• the shell's diameter and surface area

It can also measure how closely the scorers agreed with each other.

The work happens in two steps, run one after the other:

1. ROI processing (Python and Fiji) opens every image and ROI file and saves what it finds to one data file, data/processed/rois.arrow.
2. Analysis (R) reads that file and writes the results tables.

A single run script does both steps for you.

What's in this folder

folder or file	what it is
config.yaml	settings for both steps. You will edit this once, to tell it where Fiji is.
data/raw/images/	put your SEM images here
data/raw/rois/	put your ROI files here
data/raw/images/img_sizes.csv	the size of each image, used to convert pixels to microns (see Image sizes file)
data/processed/	intermediary data file the first step creates
outputs/tables/	the results tables
run.sh, run.bat	the run scripts (run.sh for Mac and Linux, run.bat for Windows)
resources/imagej-macros/	a Fiji macro that names ROIs correctly for you
src/, analysis.R	source code

One-time setup

You need to do this once on each computer. The first run also needs an internet connection, to download the packages the code uses.

1. Install the programs

Install these three programs:

• Fiji: download from https://fiji.sc/ and unzip it somewhere you'll remember, such as your Applications folder or C:\Fiji\.
• R, version 4.6 or newer: download from https://cran.r-project.org/.
• uv, which sets up Python for you: follow the instructions at https://docs.astral.sh/uv/getting-started/installation/. You don't need to install Python separately; uv downloads the right version itself.

You don't need to install Java either. The first run downloads the version Fiji needs (Java 21).

2. Tell the project where Fiji is

Open config.yaml in any text editor (TextEdit, Notepad, RStudio, VS Code) and find the line that starts with fiji-path:. Change it to the folder where you put Fiji, for example:

    fiji-path: ~/Applications/Fiji/

on a Mac (~ means your home folder), or

    fiji-path: C:\Fiji\

on Windows. Keep the spaces at the start of the line exactly as they were.

Preparing your data

You provide three things: the images, the ROI files, and the image sizes file. The default folders are listed above. You can change them under roi-processing → input in config.yaml.

You can organise images and ROI files into subfolders (one per scorer, say); the project looks inside every subfolder. Files that aren't images or ROI files are ignored.

Images

• Images must be .tif or .tiff files.
• Each image filename must contain exactly one number. That number is how ROI files are matched to their image. img_01.tiff is image 01.
• A filename with no number, or with more than one (img_01_run2.tiff), stops the run with an error, so that ROIs are never matched to the wrong image.
• Single-digit numbers are padded to two digits: img_1.tiff is also image 01, so its ROI files must use 01.

ROI files

Every ROI file must be named like this:

<scorer>_<image number>_<region>_<type>.roi

For example, ab_01_04_ws.roi is scorer ab, image 01, region 04, a whole-shell outline (ws). The four parts are:

• scorer: the scorer's initials, with no underscores.
• image number: two digits, matching the number in the image filename.
• region: a two- or three-digit number, counting up within each image. Every ROI needs one, including s, ws, and d. A file without a region number is skipped, and the run warns you about it.
• type: one of the codes below.

code	what the scorer traces	what it's used for
1, 2, 3	damage, from least severe (1) to most severe (3)	the damage areas being measured
h	a hole: an undamaged "island" inside a damaged area	removed from any damage area it overlaps
ws	the outline of the whole shell	the shell's surface area, used to work out percent coverage
d	a line across the shell	the shell's diameter
s	a line along the image's scale bar	converting pixels to microns, when the image sizes file uses a scale bar (see below)

If you trace in Fiji, a macro can name the ROIs for you (see Naming ROIs with the Fiji macro).

The project reads individual .roi files only. If you saved your ROIs from the ROI Manager as a .zip file, unzip it into the ROI folder first.

Naming ROIs with the Fiji macro

The Rename ROIs macro, in resources/imagej-macros/RenameROIs.ijm, gives every ROI in Fiji's ROI Manager a correct name, so you don't have to type the names yourself. It works on one image at a time.

Install the macro. Do this each time you open Fiji:

1. Choose Plugins → Macros → Install....
2. Go to the project folder, open resources/imagej-macros/, and choose RenameROIs.ijm.

Rename ROIs now appears at the bottom of the Plugins → Macros menu. Fiji forgets it when you quit, so install it again next time.

Trace and name the ROIs for one image.

1. Open the image in Fiji.
2. Trace each area with a selection tool, and press T after each one to add it to the ROI Manager. The ROI Manager opens by itself the first time. Remember the whole-shell outline (ws), the diameter line (d), and, if you use the scale bar (Option B), the scale bar line (s).
3. Choose Plugins → Macros → Rename ROIs.
4. Type your initials and the image number, then click OK. For img_01.tiff, the image number is 1.
5. The macro highlights each ROI in turn and asks for its type. Choose one from the list and click OK.

choice in the macro	code in the file name
Type I, Type II, Type III	1, 2, 3
Hole	h
Whole shell	ws
Diameter	d
Scale	s

The ROIs are numbered in the order they are listed in the ROI Manager: the first becomes region 01, the second 02, and so on.

Save the ROIs.

1. In the ROI Manager, click Deselect, so that no ROI is highlighted. If one is still highlighted, only that ROI is saved.
2. Click More >>, then Save.... Fiji saves all the ROIs in one .zip file.
3. Unzip the file into data/raw/rois/. On a Mac, double-click it. On Windows, right-click it and choose Extract All.
4. Before starting the next image, clear the ROI Manager: click Deselect, then Delete, and confirm.

Things to watch for.

• Name and save all of an image's ROIs in one go. Every run of the macro starts again at region 01, so if you name some of an image's ROIs now and the rest later, the second batch overwrites the first when you unzip it. To add a forgotten ROI, drag the saved .zip onto Fiji to reload the others, add the new one, then run the macro and save again.
• Check the image number. The macro doesn't compare it with the open image. ROIs saved under the wrong number are counted towards the wrong image.
• Initials can't contain underscores (_), because underscores separate the parts of the file name. The macro removes spaces for you.

Image sizes file

Areas are reported in µm², so the project needs to know how many microns each pixel covers, for every image. It gets this from data/raw/images/img_sizes.csv, a spreadsheet saved as CSV. There are two ways to fill it in.

Option A: the image's width. Give the width of each image in microns and in pixels. The same conversion is used for every scorer.

image,width_um,width_px
img_01,1210,4186
img_02,2440,8378

Option B: the scale bar. Give the length of each image's scale bar in microns. Each scorer then traces the scale bar with an s ROI, and their own tracing is used to convert their ROIs.

image,scale_bar_um
img_01,300
img_02,500

You can include both sets of columns. If you do, Option A is used.

Rules for the file:

• One row for every image, and no extra rows. The image column holds the image filename without its extension: img_01.tiff is written img_01. If an image has no row, or a row names an image that isn't in the images folder, the run stops before doing anything else and lists the names to fix.
• Column names must be spelled exactly as shown, in lower case. Column order doesn't matter. You can add other columns (notes, magnification) and they will be ignored.
• Fill in every cell of the number columns. A blank or a non-number stops the run and names the row. If you can't fill a column for every image, remove that column and use the other option.
• If you edit the file in Excel, save it as CSV UTF-8 (comma delimited).

If you use Option B, check your s ROIs. A scorer who didn't trace the scale bar on an image gets that image's areas in pixels instead of µm². The run warns you about this (see Messages you may see), so if one scorer's numbers look far too large, check that warning first.

Running the project

You run the project by typing a command into a terminal window. Everything you need to type is shown below.

1. Open a terminal in the project folder

It is recommended to open up this project folder in an IDE such as Positron, Rstudio, or VS Code. You can then use the Terminal tab to run the scripts from the project. Otherwise a terminal emulator app such as Terminal.app for Mac or cmd.exe for Windows can be used.

2. Run the script

Mac or Linux: type this and press Return:

./run.sh

Windows: type this and press Enter:

.\run.bat

Messages will scroll past while it works. When the prompt comes back and you can type again, it has finished. The results are in outputs/tables/.

The first run sets everything up and can take several minutes or more. Later runs skip the setup.

Running one step at a time

The run script runs both steps. You can also run them separately, which is useful when only one needs redoing (see When to re-run). To run just the first step (Fiji):

uv run process_rois

To run just the second step (the analysis):

Rscript analysis.R

These are the same on every system. Run the full script at least once first, so that everything is set up.

What you'll see during a run

The first step prints many lines from Fiji as it opens each image, such as FEITiffReader initializing and Reading IFDs. These are normal, not errors.

Messages you may see

The project checks your data for five problems. If you see none of these messages, nothing was left out.

message	what it means	what to do
image(s) have no row in the image sizes CSV or CSV row(s) name no image	the image sizes file and the images folder don't match. The run stops.	fix the names it lists (see Image sizes file)
ROI file(s) were not matched to any image	these ROI files don't follow the naming pattern, usually because the region number is missing, or they name an image that isn't there. They are left out of the results.	rename the files listed
Unrecognised ROI type code(s)	an ROI file's type isn't one of the codes above, usually a typo. The run stops and names the file.	rename the file
image/scorer pair(s) are missing shell metadata ROIs	a scorer didn't trace the whole shell (ws), the diameter (d), or the scale bar (s) on an image. That measurement is left blank in the results.	add the missing ROI, if it should be there
shell/scorer pair(s) have no ROIs	a scorer drew nothing at all on an image, so that pair isn't in the results.	nothing, if that scorer wasn't meant to score that image

A few notes on these messages:

• The missing-metadata message lists missing ws, d, and s ROIs as separate groups. It's a warning rather than an error because each missing ROI only blanks one measurement: no ws means no percent coverage, and no d means no diameter.
• Missing s ROIs are only reported when you use Option B, because Option A doesn't use them.
• A scorer who drew nothing on an image appears in both of the last two messages. That is expected.

Results

outputs/tables/dissolution_stats.csv has one row per image and scorer, with these columns:

column	meaning
image_id	the image filename without its extension, such as img_01
scorer	the scorer's initials
diameter_um	length of the d ROI, in µm (for a shape other than a line, the diagonal of the smallest box around it)
surface_area_um2	area of the ws outline, in µm²
area_type_I_um2, area_type_II_um2, area_type_III_um2	area of each damage type, in µm², with holes removed
percent_coverage_type_I, _II, _III	each damage area as a percentage of surface_area_um2
dissolution_area_summed_um2	the three damage areas added together

A blank cell and a zero mean different things. A blank means the measurement couldn't be made, usually because the scorer didn't draw a ws outline (so no percent coverage) or a d line (so no diameter). A zero means the scorer found no damage of that type.

outputs/tables/scorer_agreement.csv compares every pair of scorers, for each image and damage type. Its jaccard column is the area both scorers marked, divided by the area either of them marked: 1 means they traced exactly the same area, 0 means no overlap at all. It's blank when neither scorer marked that type. This table is optional. To create it, change scorer-agreement: false to scorer-agreement: true in config.yaml.

Each run replaces the previous results tables.

When to re-run

The first step (Fiji) is the slow one, so you only need to repeat it when its inputs change.

if you changed	run
images, ROI files, img_sizes.csv, or the roi-types codes in config.yaml	both steps: ./run.sh or .\run.bat
only the analysis settings in config.yaml	only the analysis: Rscript analysis.R

If you run the analysis before the first step has ever been run, it stops and tells you to run uv run process_rois.

How overlapping ROIs are handled

When two of one scorer's ROIs of different types overlap, the shared area is counted only once, as the more severe type:

• A hole (h) beats every damage type, so a damage area with a hole inside it only counts the part outside the hole.
• Among damage types, 3 beats 2, and 2 beats 1.

When two ROIs of the same type overlap, the shared area is also counted only once, so a region traced twice by mistake doesn't double its area.

This order comes from the order the codes are listed under roi-types in config.yaml: the damage types from least to most severe, then hole.

Settings reference

config.yaml has four sections. Both steps read it.

section	what it holds
roi-types	the ROI type codes, and the order that decides overlaps
roi-dataset	where the first step saves its data file, and the second step reads it
roi-processing	where Fiji, the images, the ROI files, and the image sizes file are
analysis	where results are saved, and whether to create the scorer agreement table

Changing the type codes. If you use the ImageJ macro and change a code under roi-types, you must also change it in the Rename ROIs macro (resources/imagej-macros/RenameROIs.ijm), because Fiji macros can't read config.yaml. Then re-run both steps. If the two ever disagree, the run stops with an Unrecognised ROI type code(s) error, naming the file.

Troubleshooting

• uv, Rscript, or ./run.sh "not found" or "not recognized". Check that the terminal is in the project folder (step 1 of Running the project). If it is, the program isn't installed, or your computer can't find it: close and reopen the terminal after installing. On Windows, the R installer does not always make Rscript available to the Command Prompt; if not, add R's bin folder (for example C:\Program Files\R\R-4.6.1\bin) to your PATH.
• An error about Fiji or ImageJ at the start of the first step. Check the fiji-path line in config.yaml (see One-time setup).
• The run fails while saving results, on Windows. Close any results table you have open in Excel, then run it again. Excel locks open files.