Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 21 additions & 19 deletions docs/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,34 +91,36 @@ Primary Workflows
AI Surrogate Workflows (PhysicsNeMo)
=====================================

The final tier of tutorials (``tutorials/tutorial_08`` through
``tutorial_10``) turns a fitted statistical shape model into a trained AI
The final tier of tutorials (``tutorials/tutorial_08cd`` through
``tutorial_10d``) turns a fitted statistical shape model into a trained AI
physiological surrogate, replacing the explicit per-phase registration solve
with a learned model at inference time:

``tutorial_08_cardiac_fit_model.py``
``tutorial_08cd_byod_fit_model_to_patients.py``
Fits the cardiac PCA model to a patient (via
``WorkflowFitStatisticalModelToPatient``) and propagates the fitted mesh
through every gated phase using ICON-based registration
(``WorkflowReconstructHighres4DCT``), producing the per-phase mesh/surface
pairs used as AI surrogate training data.

``tutorial_09a_cardiac_train_physicsnemo_mgn.py`` /
``tutorial_09b_cardiac_train_physicsnemo_mlp.py``
Train a PhysicsNeMo surrogate — a graph-based ``MeshGraphNet``
(``physicsnemo.models.meshgraphnet``) or a fully connected MLP — on the
Tutorial 8 output to predict per-phase cardiac mesh deformation directly
from the fitted SSM coefficients. Requires the ``[physicsnemo]`` extra
(and ``torch-geometric`` for the MeshGraphNet variant); Python >= 3.11.

``tutorial_10a_cardiac_eval_physicsnemo_mgn.py`` /
``tutorial_10b_cardiac_eval_physicsnemo_mlp.py``
Load a trained MeshGraphNet or MLP checkpoint and predict/score cardiac
surfaces without running registration, i.e. the AI surrogate stands in for
``WorkflowReconstructHighres4DCT`` at inference time.

These tutorials are not wrapped in a ``physiotwin4d`` workflow class today —
they call the PhysicsNeMo model classes directly — but they follow the same
``tutorial_09c_byod_train_physicsnemo_mgn.py`` /
``tutorial_09d_byod_train_physicsnemo_mlp.py``
Train a PhysicsNeMo surrogate — a graph-based ``MeshGraphNet`` via
``WorkflowTrainPhysicsNeMoMGN`` or a fully connected MLP via
``WorkflowTrainPhysicsNeMoMLP`` — on the Tutorial 8cd output to predict
per-phase cardiac mesh deformation directly from the fitted SSM
coefficients. Requires the ``[physicsnemo]`` extra (and ``torch-geometric``
for the MeshGraphNet variant); Python >= 3.11.

``tutorial_10c_byod_eval_physicsnemo_mgn.py`` /
``tutorial_10d_byod_eval_physicsnemo_mlp.py``
Load a trained MeshGraphNet or MLP checkpoint (via
``WorkflowInferPhysicsNeMoMGN`` / ``WorkflowInferPhysicsNeMoMLP``) and
predict/score cardiac surfaces without running registration, i.e. the AI
surrogate stands in for ``WorkflowReconstructHighres4DCT`` at inference time.

These tutorials are thin drivers over the ``WorkflowTrainPhysicsNeMo`` /
``WorkflowInferPhysicsNeMo`` workflow classes; they follow the same
fit -> propagate -> train -> predict pattern the rest of the workflow layer
uses, and are the intended template for future cardiac, respiratory, and
electrophysiology AI surrogates.
Expand Down
44 changes: 22 additions & 22 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
</section>

<section class="pt4d-card-grid" aria-label="Tutorial cards">
<a class="pt4d-card" href="tutorials.html#tutorial-1-heart-gated-ct-to-animated-usd">
<span class="pt4d-card__number">01</span>
<a class="pt4d-card" href="tutorials.html#tutorial-1a-heart-gated-ct-to-animated-usd">
<span class="pt4d-card__number">1a</span>
<h2>Heart-Gated CT to Animated USD</h2>
<p>Convert cardiac 4D CT frames into registered contours and an animated OpenUSD model.</p>
<span class="pt4d-card__meta">Slicer-Heart-CT</span>
Expand All @@ -34,47 +34,47 @@
<p>Segment one CT phase and export patient anatomy as VTK PolyData surfaces.</p>
<span class="pt4d-card__meta">Slicer-Heart-CT</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-3-create-a-pca-shape-model">
<a class="pt4d-card" href="tutorials.html#tutorial-3-vtk-surface-series-to-animated-usd">
<span class="pt4d-card__number">03</span>
<h2>VTK Surface Series to Animated USD</h2>
<p>Convert VTK meshes into a time-sampled USD scene for Omniverse playback.</p>
<span class="pt4d-card__meta">Tutorial 2 output</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-4a-create-a-pca-shape-model">
<span class="pt4d-card__number">4a</span>
<h2>Create a PCA Shape Model</h2>
<p>Build a statistical shape model from aligned cardiac meshes.</p>
<span class="pt4d-card__meta">KCL-Heart-Model</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-4-fit-statistical-model-to-patient">
<span class="pt4d-card__number">04</span>
<a class="pt4d-card" href="tutorials.html#tutorial-5a-fit-statistical-model-to-patient">
<span class="pt4d-card__number">5a</span>
<h2>Fit Statistical Model to Patient</h2>
<p>Fit a PCA heart model to patient-specific anatomy for model-based reconstruction.</p>
<span class="pt4d-card__meta">Tutorial 3 output</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-5-vtk-surface-series-to-animated-usd">
<span class="pt4d-card__number">05</span>
<h2>VTK Surface Series to Animated USD</h2>
<p>Convert VTK meshes into a time-sampled USD scene for Omniverse playback.</p>
<span class="pt4d-card__meta">Tutorial 2 output</span>
<span class="pt4d-card__meta">Tutorial 4a output</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-6-reconstruct-high-resolution-4d-ct">
<span class="pt4d-card__number">06</span>
<h2>Reconstruct High-Resolution 4D CT</h2>
<p>Register respiratory CT phases and reconstruct a higher-resolution 4D volume series.</p>
<span class="pt4d-card__meta">DirLab-4DCT</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-8-fit-the-cardiac-ssm-and-propagate-through-gated-phases">
<span class="pt4d-card__number">08</span>
<a class="pt4d-card" href="tutorials.html#tutorial-8cd-fit-the-cardiac-ssm-and-propagate-through-gated-phases">
<span class="pt4d-card__number">8cd</span>
<h2>Fit the Cardiac SSM and Propagate Through Gated Phases</h2>
<p>Fit a PCA heart model to the reference phase and propagate it to every gated phase with ICON registration.</p>
<span class="pt4d-card__meta">Bring your own cardiac data</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-9a-9b-train-a-physicsnemo-cardiac-stage-model">
<span class="pt4d-card__number">09</span>
<a class="pt4d-card" href="tutorials.html#tutorial-9c-9d-train-a-physicsnemo-cardiac-stage-model">
<span class="pt4d-card__number">9cd</span>
<h2>Train a PhysicsNeMo Cardiac Stage Model</h2>
<p>Train a PhysicsNeMo MeshGraphNet (9a) or MLP (9b) to predict cardiac meshes at requested stages.</p>
<span class="pt4d-card__meta">Tutorial 8 output</span>
<p>Train a PhysicsNeMo MeshGraphNet (9c) or MLP (9d) to predict cardiac meshes at requested stages.</p>
<span class="pt4d-card__meta">Tutorial 8cd output</span>
</a>
<a class="pt4d-card" href="tutorials.html#tutorial-10a-10b-predict-and-evaluate-cardiac-surfaces">
<span class="pt4d-card__number">10</span>
<a class="pt4d-card" href="tutorials.html#tutorial-10c-10d-predict-and-evaluate-cardiac-surfaces">
<span class="pt4d-card__number">10cd</span>
<h2>Predict and Evaluate Cardiac Surfaces</h2>
<p>Load a Tutorial 9 checkpoint and predict cardiac surfaces at gated phases or caller-specified stages.</p>
<span class="pt4d-card__meta">Tutorial 9a / 9b output</span>
<p>Load a Tutorial 9c/9d checkpoint and predict cardiac surfaces at gated phases or caller-specified stages.</p>
<span class="pt4d-card__meta">Tutorial 9c / 9d output</span>
</a>
</section>

Expand Down
18 changes: 9 additions & 9 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The ``tutorials/`` directory contains eleven end-to-end scripts covering nine
major workflows (Tutorials 9 and 10 each have MeshGraphNet and MLP variants).
Each script is a ``# %%`` percent-cell Python script that exercises
the workflow classes directly. Run as a regular file
(``python tutorials/tutorial_01_...py``) or cell-by-cell in VS Code or Cursor.
(``python tutorials/tutorial_01a_...py``) or cell-by-cell in VS Code or Cursor.

See :doc:`tutorials` for the NVIDIA-styled tutorial card index, dataset
requirements, script paths, and workflow details.
Expand All @@ -41,7 +41,7 @@ tutorials:

.. code-block:: bash

python tutorials/tutorial_01_heart_gated_ct_to_usd.py
python tutorials/tutorial_01a_heart_gated_ct_to_usd.py

python tutorials/tutorial_02_ct_to_vtk.py

Expand All @@ -52,16 +52,16 @@ recommended run order.

Recommended run order:

1. Tutorials 1 and 2 first, after downloading Slicer-Heart-CT data.
2. Tutorial 5 after Tutorial 2 (consumes Tutorial 2 output).
3. Tutorial 3 after downloading KCL-Heart-Model.
4. Tutorial 4 after Tutorial 3 because it can consume Tutorial 3 output.
1. Tutorials 1a and 2 first, after downloading Slicer-Heart-CT data.
2. Tutorial 3 after Tutorial 2 (consumes Tutorial 2 output).
3. Tutorial 4a after downloading KCL-Heart-Model.
4. Tutorial 5a after Tutorial 4a because it can consume Tutorial 4a output.
5. Tutorial 6 after downloading DirLab-4DCT (manual).
6. Tutorial 8 after preparing your own cardiac gated CT, labelmaps, KCL volume
6. Tutorial 8cd after preparing your own cardiac gated CT, labelmaps, KCL volume
PCA model, and ICON weights (bring-your-own-data).
7. Tutorial 9a and/or 9b after Tutorial 8 because they train from its fitted
7. Tutorial 9c and/or 9d after Tutorial 8cd because they train from its fitted
meshes.
8. Tutorial 10a and/or 10b after Tutorial 9a / 9b because they evaluate the
8. Tutorial 10c and/or 10d after Tutorial 9c / 9d because they evaluate the
trained checkpoints.

Prerequisites
Expand Down
Loading
Loading