Skip to content

Add CAP Sleep Database reader - #321

Merged
cbrnr merged 3 commits into
cbrnr:mainfrom
viranovskaya:feat/capslpdb-reader
Aug 7, 2026
Merged

cbrnr merged 3 commits into
cbrnr:mainfrom
viranovskaya:feat/capslpdb-reader

Conversation

@viranovskaya

Copy link
Copy Markdown
Contributor

Closes #35.

This adds read_capslpdb() for the CAP Sleep Database. The reader:

  • uses the REMlogic Event field for sleep stages and maps S3 and S4 to N3;
  • aligns 30-second annotations with the EDF recording and handles missing internal rows;
  • supports EKG, ECG, ECG1-ECG2, and separate ECG1/ECG2 channels;
  • warns and skips n16 because it does not contain an ECG channel;
  • supports cached heartbeats and the existing PhysioNet download options.

I checked the parser against all 108 public annotation files, containing 108,720 valid sleep-stage rows. nfle27 has one malformed row, which is skipped with a warning. I also ran the complete reader on n12, producing 990 sleep-stage epochs and 30,656 heartbeat times.

The tests cover stage mapping, time and header variants, missing rows, EDF boundaries, midnight alignment, malformed rows, irregular grids, bipolar ECG, heartbeat rebasing, n16, and downloaded EDF cleanup.

@viranovskaya
viranovskaya marked this pull request as ready for review August 5, 2026 14:03
@cbrnr

cbrnr commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Could you rebase to pull in some new Ruff rules please?

@viranovskaya
viranovskaya force-pushed the feat/capslpdb-reader branch from 2ae7ce5 to 781eff4 Compare August 6, 2026 08:14
@viranovskaya

Copy link
Copy Markdown
Contributor Author

Done, I rebased the branch onto the current main and updated the code for the new Ruff rules.

Local checks pass: Ruff, formatting, mypy, and the full test suite with 68 tests.

@cbrnr

cbrnr commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Very nice! Just one thing that should be checked: if heartbeats_source="cached", the reader should not download the EDF file(s), but instead use the cached heartbeats (like the other readers). Here's what Claude found:

heartbeats_source="cached" still re-downloads all EDFs. In read_capslpdb, download_physionet(..., extensions=[".edf", ".txt"], ...) runs unconditionally for every requested record before the per-record loop, regardless of heartbeats_source. So even in "cached" mode, read_edf(edf_filepath, ...) needs a local EDF, meaning a later run with "cached" will silently re-download the whole ~40 GB dataset just to read starttime/duration — defeating the point of caching heartbeats + keep_edfs=False.

Compare read_mesa/read_shhs: there the EDF is only downloaded inside the "ecg" branch, so "cached" never touches it.

Suggested fix: cache recording_start_time/duration alongside the heartbeats .npy when running with "ecg", so "cached" mode never needs the EDF at all.

@viranovskaya

Copy link
Copy Markdown
Contributor Author

Thanks for catching this. I updated the reader so cached mode downloads only the annotation files and never downloads or reads an EDF. The ECG path now stores the recording start time and duration alongside the cached heartbeat array, and cached mode restores them from that metadata.

I also added a regression test that fails if cached mode attempts to download or read an EDF. Ruff, formatting, mypy, and the full test suite pass locally: 69 tests.

Comment thread src/sleepecg/io/capslpdb.py Outdated
@cbrnr
cbrnr merged commit ce1533b into cbrnr:main Aug 7, 2026
8 checks passed
@cbrnr

cbrnr commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Perfect! Thanks @viranovskaya!

@viranovskaya
viranovskaya deleted the feat/capslpdb-reader branch August 10, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add functions for reading annotated sleep data

2 participants