Forams Classification

Semi-supervised 3D classification of foraminifera (forams) — single-celled marine organisms that build calcium-carbonate shells. Because different species only survive under specific environmental conditions, and their shells are preserved in sea sediment for millions of years, classifying fossil forams from CT scans gives insight into past climate and ocean conditions.

Final project for 02510 — Deep Learning for Experimental 3D Image Analysis at DTU, by Group Iora: Amir Kfir and Tomasz Niedziałkowski, May 2025. Full presentation with methodology and results: reports/Forams_Classification_2025_Presentation.pptx.

The problem

The dataset consists of 18,426 volumetric CT scans (96×96×96) of forams across 14 species, of which only 210 are labeled. With no pretrained 3D classification models available for this domain, the project explores how far semi-supervised learning can stretch such a small labeled set, using MONAI's 3D DenseNet121 as the backbone classifier.

Approaches

Three pipelines were built, each iterating on the last:

1. Naive Bootstrap — a straightforward self-training loop: train on the labeled set, pseudo-label the unlabeled set with the current model, retrain on the combined data. Simple and unoptimized, but a strong baseline given the complexity.
2. FixMatch (sweep_fixmatch.py, FixMatchDataset in data.py) — a standard semi-supervised algorithm: weak/strong augmentation pairs on unlabeled data, confidence- thresholded pseudo-labels, combined weighted supervised + unsupervised loss via separate labeled/unlabeled dataloaders. Faster to train than bootstrapping, with similar accuracy.
3. FixMatch — Multimodal (model.py, MultimodalModel) — fuses the 3D volume with a 2D visualization of the same sample through two DenseNet121 branches (a 3D branch trained from scratch, a 2D branch pretrained on ImageNet). Several fusion strategies were implemented and compared: concatenation, additive attention, gated fusion, and multi-head cross-attention (the 3D features attend to the 2D features via nn.MultiheadAttention, followed by a residual + feed-forward block). The model also exposes an energy-based out-of-distribution branch (classify_with_unknown) to flag samples that don't confidently match any of the 14 known classes.

Results

Pipeline	Best validation accuracy
Bootstrap	0.619
FixMatch	0.619
FixMatch – Multimodal (multi-head attention)	0.714

The multimodal fusion gave the clearest improvement over the sequential pipelines, at the cost of needing more training/tuning to reach that point.

The model separates most classes cleanly but confuses a handful of visually similar species — e.g. Globorotalia menardii vs. tumida, whose shells share an outer silhouette but differ internally. To understand these failures we generated 3D saliency volumes (confusion_matrix.py, evaluate.py) from the gradients of the model's prediction with respect to the input:

Saliency inspection showed the model doesn't always focus on the same regions a human would — e.g. for a species with a distinctive outer shell, the model attended more to internal structure. In confused class pairs, the misclassified samples showed a narrower saliency focus resembling the class they were confused with, rather than their true class.

Takeaways: early/multimodal fusion helps most on visually similar species; the model's learned features don't always align with human-salient cues; with only 210 labels, pretraining the 3D branch on another labeled 3D dataset would likely help more than further architecture tuning.

Repository structure

├── configs/                    # Configuration files
├── dockerfiles/                # Dockerfiles (train / API)
├── docs/                       # mkdocs documentation
├── jobs/                       # DTU HPC (LSF/BSUB) job submission scripts
├── reports/
│   ├── Forams_Classification_2025_Presentation.pptx
│   └── figures/                # Result figures used in this README
├── src/forams_classification/
│   ├── data.py                 # Dataset loading: labeled, unlabeled, FixMatch, multimodal
│   ├── model.py                 # MultimodalModel (3D+2D DenseNet121, 4 fusion strategies)
│   ├── train_densenet.py       # Main training entry point (FixMatch + multimodal + W&B)
│   ├── sweep_fixmatch.py       # W&B hyperparameter sweep script
│   ├── confusion_matrix.py     # Confusion-matrix evaluation
│   ├── evaluate.py             # 3D saliency-map generation
│   ├── generate_submission.py # Inference + Kaggle-style submission CSV, with unknown-class handling
│   ├── gen_sub_wandb.py        # Submission generation from a W&B model artifact
│   └── api.py                  # Inference API
└── tests/

Training

Model, FixMatch, and logging configuration (fusion type, epochs, batch sizes, confidence threshold, energy threshold, etc.) live at the top of train_densenet.py. Training logs to Weights & Biases.

python src/forams_classification/train_densenet.py

Hyperparameter sweeps (used to tune the multimodal fusion) are run via:

python src/forams_classification/sweep_fixmatch.py --sweep_id <id> --count <n>

Example DTU HPC (LSF) job scripts are in jobs/.

Generating predictions

python src/forams_classification/generate_submission.py --output_file submission.csv

Pulls a trained model from a W&B artifact or model registry (or a local checkpoint via --local_model) and writes per-sample class predictions, with energy/confidence-based "unknown" class detection for out-of-distribution samples.

Created using mlops_template, a cookiecutter template for getting started with Machine Learning Operations (MLOps).