OSDataset Registration (Optical Calibration & Cross-Modal Image Registration)

Developed with PyTorch and PyTorch Lightning, this repository addresses keypoint matching and precise optical/cross-modal image registration challenges. It integrates and benchmarks leading feature matching and geometric alignment frameworks, including LoFTR, DKM, SuperGlue, and classic SIFT baselines.

---

🛠️ Key Features & Architecture

• Matching Algorithms:
  • LoFTR: A detector-free, coarse-to-fine cross-attention feature matching network.
  • DKM: Dense Kernel Matching for global dense alignment.
  • SuperGlue: A GNN and Optimal Transport-based feature matcher.
  • SIFT / SIFT2: Classical handcrafted feature detection and description baselines.
• Custom Dataset Interface: Native support for pre-processing and loading cross-modal optical datasets via OSDataset.
• Automated Pipeline: Managed via PyTorch Lightning with built-in evaluation metrics such as AEPE (Average End-Point Error).

---

🚀 Quick Start

1. Environment Setup

Recommended setup: Python 3.10+ with PyTorch CUDA support.

pip install -r requirements.txt
2. Model Training
Run training pipelines for specific models and dataset configurations:

Bash
# Train LoFTR registration model
python train.py --config configs/model/loftr_osdataset.py

# Train SuperGlue matching model
python train.py --config configs/model/superglue_osdataset.py
3. Evaluation & Benchmarking
Evaluate checkpoints to compute AEPE and keypoint matching metrics:

Bash
# Evaluate deep learning checkpoints
python eval.py --checkpoint path/to/best_checkpoint.ckpt

# Evaluate SIFT baselines
python eval_sift2.py
📁 Directory Structure
Plaintext
├── configs/             # Hyperparameter configs for models and datasets
│   ├── data/            # Dataset configs (OSDataset, etc.)
│   └── model/           # Model architecture configs (LoFTR, SuperGlue, etc.)
├── src/
│   ├── datasets/        # Dataset interfaces and data augmentations
│   ├── losses/          # Loss functions tailored for optical registration
│   ├── models/          # Model backbones & PyTorch Lightning modules
│   └── utils/           # Geometric transforms, metrics (AEPE), and visualization
├── eval.py              # Evaluation script for deep learning models
├── eval_sift2.py        # Evaluation script for SIFT baselines
├── train.py             # Unified training entry point
└── requirements.txt     # Environment dependencies