Workflow Classes

Workflow classes are the highest-level Python API in PhysioTwin4D. They combine segmentation, registration, contour generation, and USD conversion into repeatable pipelines. The installed CLI commands are thin wrappers around these classes.

Available Workflows

Workflow	Typical use
`WorkflowConvertImageToUSD`	Convert a 4D cardiac CT sequence into animated USD anatomy.
`WorkflowConvertImageToVTK`	Segment one CT image and export anatomy-group VTK surfaces.
`WorkflowConvertVTKToUSD`	Convert VTK/VTP/VTU meshes or time series into USD.
`WorkflowCreateMeanSurface`	Build an unbiased mean surface (atlas) from a population, for use as the reference a shape model is built against.
`WorkflowCreateStatisticalModel`	Build a PCA shape model from sample meshes aligned to a reference.
`WorkflowFitStatisticalModelToPatient`	Fit a template/statistical heart model to patient-specific surfaces.
`WorkflowReconstructHighres4DCT`	Reconstruct a high-resolution 4D CT series from phase images and a high-resolution reference.
`WorkflowFinetuneICONRegistration`	Finetune uniGradICON on your own cohort and return the weights `RegisterImagesICON` can load.

The PhysicsNeMo AI-surrogate workflows — WorkflowTrainPhysicsNeMo, WorkflowInferPhysicsNeMo, WorkflowInferMovement and WorkflowEvaluateMovement — have their own section, since they need the optional [physicsnemo] extra. See PhysicsNeMo AI Surrogates.

Convert Image to USD

class physiotwin4d.WorkflowConvertImageToUSD(time_series_images, reference_image, usd_project_name, output_directory, segmentation_method=None, registration_method=None, dynamic_labelmap_ids=None, mask_dilation_radius=10, frames_per_second=1.0, surface_reduction_rate=0.0, log_level=20, save_assets=True)[source]

Bases: PhysioTwin4DBase

Complete workflow for converting 4D CT images to dynamic USD models.

This class implements the full workflow from 4D CT images to painted USD files suitable for visualization in NVIDIA Omniverse.

segmentation_method and registration_method accept a pre-configured SegmentAnatomyBase / RegisterImagesBase instance. Configure backend-specific parameters (iteration counts, trim_branches, mass preservation, etc.) on the instance before passing it in. Defaults to SegmentChestTotalSegmentatorWithContrast / RegisterImagesGreedy when omitted.

__init__(time_series_images, reference_image, usd_project_name, output_directory, segmentation_method=None, registration_method=None, dynamic_labelmap_ids=None, mask_dilation_radius=10, frames_per_second=1.0, surface_reduction_rate=0.0, log_level=20, save_assets=True)[source]

Initialize the image-to-USD workflow.

Parameters:

time_series_images (list[itk.Image]) – List of time-series images
reference_image (itk.Image) – Reference image
usd_project_name (str) – Project name for USD file organization
output_directory (str) – Directory path where output files will be stored
segmentation_method (Optional[SegmentAnatomyBase]) – Segmentation backend instance. Defaults to a new SegmentChestTotalSegmentatorWithContrast when None.
registration_method (Optional[RegisterImagesBase]) – Registration backend instance. Defaults to a new RegisterImagesGreedy when None. A caller-supplied instance is mutated (fixed image/mask/modality) during process() - pass a fresh instance per run unless intentionally reusing state.
dynamic_labelmap_ids (Optional[list[int]]) – Labelmap ids to treat as dynamic anatomy (registered/contoured separately from the remaining static anatomy). Defaults to an empty list (no dynamic/static split; everything registers as “all”).
mask_dilation_radius (int) – Dilation radius, in voxels, applied to the dynamic/static registration masks. Defaults to 10.
frames_per_second (float) – Frames per second for animated USD time series. Defaults to 24.0, matching the underlying VTK-to-USD converter.
surface_reduction_rate (float) – Fraction in [0, 1) of triangles to remove from every extracted surface. 0.0 (default) skips decimation.
log_level (int | str) – Logging level (default: logging.INFO)
save_assets (bool) – Write registered images, transforms, and labelmaps output_directory when True

Raises:

TypeError – If segmentation_method is neither None nor a SegmentAnatomyBase instance, or registration_method is neither None nor a RegisterImagesBase instance.

process()[source]

Execute the complete workflow from 4D CT to dynamic USD models.

Returns:: Dictionary of anatomy type to filename of the final painted USD file.
Return type:: dict[str, str]

import itk

from physiotwin4d import (
    RegisterImagesICON,
    SegmentChestTotalSegmentatorWithContrast,
    WorkflowConvertImageToUSD,
)

time_series_images = [itk.imread(str(path)) for path in frame_files]

workflow = WorkflowConvertImageToUSD(
    time_series_images=time_series_images,
    reference_image=time_series_images[0],
    output_directory="./results",
    usd_project_name="patient_001",
    segmentation_method=SegmentChestTotalSegmentatorWithContrast(),
    registration_method=RegisterImagesICON(),
)

results = workflow.process()

Image to VTK

class physiotwin4d.WorkflowConvertImageToVTK(segmentation_method=None, log_level=20)[source]

Bases: PhysioTwin4DBase

Segment a CT image and produce per-anatomy-group VTK surfaces.

segmentation_method accepts a pre-configured SegmentAnatomyBase instance (e.g. SegmentChestTotalSegmentator, SegmentChestTotalSegmentatorWithContrast for contrast-enhanced studies, SegmentHeartSimpleware, or SegmentHeartSimplewareTrimmedBranches for cardiac-only segmentation with pulmonary/great-vessel branches trimmed). Defaults to a new SegmentChestTotalSegmentator when omitted.

Output anatomy groups

Determined by the active segmenter’s SegmentAnatomyBase.taxonomy (see anatomy_groups). Groups that are empty after segmentation are silently skipped. Pass extract_label_surfaces=True to process() to additionally extract one surface per individual structure (label) within each group, e.g. left_ventricle separately from right_ventricle within heart.

VTK object annotation

Each pyvista.PolyData surface returned by process() carries:

field_data['AnatomyGroup'] — anatomy group name, e.g. 'heart'.
field_data['SegmentationLabelNames'] — individual structure names within the group (e.g. ['left_ventricle', 'right_ventricle', …]).
field_data['SegmentationLabelIds'] — corresponding integer label IDs.
field_data['AnatomyColor'] — RGB float color from USDAnatomyTools.
cell_data['Color'] — RGBA uint8 array (n_cells × 4) for direct VTK rendering.

I/O contract

process() performs no file I/O. Use ContourTools’s static helpers ContourTools.save_surfaces() and ContourTools.save_combined_surfaces() — or the CLI physiotwin4d-convert-image-to-vtk — to write results to disk.

__init__(segmentation_method=None, log_level=20)[source]

Initialize the workflow.

Parameters:

segmentation_method (Optional[SegmentAnatomyBase]) – Segmentation backend instance. Defaults to a new SegmentChestTotalSegmentatorWithContrast when None.
log_level (int | str) – Logging level. Default: logging.INFO.

Raises:

TypeError – If segmentation_method is neither None nor a SegmentAnatomyBase instance.

anatomy_groups: tuple[str, ...]: Anatomy group names registered by the active segmenter’s taxonomy, in the order they were first added.

process(input_image, anatomy_groups=None, surface_reduction_rate=0.0, extract_label_surfaces=False)[source]

Segment the CT image and extract per-anatomy-group VTK surfaces.

Parameters:

input_image (Any) – Input 3D CT image (itk.Image).
anatomy_groups (Optional[list[str]]) – Subset of anatomy groups to process. None (default) processes all non-empty groups. Valid names are given by anatomy_groups, derived from the active segmenter’s taxonomy.
surface_reduction_rate (float) – Fraction in [0, 1) of surface triangles to remove via decimate_pro(surface_reduction_rate, preserve_topology=True). 0.0 (default) skips decimation. Applied to both group and (when requested) label surfaces.
extract_label_surfaces (bool) – When True, also extract one surface per individual anatomical structure (label) within each processed group — e.g. left_ventricle and right_ventricle separately within the heart group — in addition to the per-group surfaces. False (default) skips this and leaves 'label_surfaces' empty.

Returns:

'surfaces' — dict[str, pv.PolyData]: smoothed surface per group.
'label_surfaces' — dict[str, pv.PolyData]: smoothed surface per individual label, populated only when extract_label_surfaces is True.
'labelmap' — itk.Image: detailed per-structure segmentation labelmap from the segmenter.
'segmentation_masks' — dict[str, itk.Image]: per-group binary masks used to produce the VTK objects.

Return type:

dict with the following keys

Raises:

ValueError – If any name in anatomy_groups is invalid.

import itk

from physiotwin4d import (
    ContourTools,
    SegmentChestTotalSegmentatorWithContrast,
    WorkflowConvertImageToVTK,
)

image = itk.imread("chest_ct.nii.gz")
workflow = WorkflowConvertImageToVTK(
    segmentation_method=SegmentChestTotalSegmentatorWithContrast()
)
result = workflow.process(
    input_image=image,
    anatomy_groups=["heart", "major_vessels"],
)

ContourTools.save_combined_surfaces(
    result["surfaces"],
    "./output/patient01_surfaces.vtp",
)

VTK to USD

class physiotwin4d.WorkflowConvertVTKToUSD(input_meshes, usd_project_name, output_directory, *, separate_by_connectivity=True, separate_by_cell_type=False, frames_per_second=60.0, extract_surface=True, static_merge=False, time_codes=None, appearance='solid', solid_color=(0.8, 0.8, 0.8), anatomy_type=None, object_names=None, label_names=None, segmenter=None, colormap_primvar=None, colormap_name='viridis', colormap_intensity_range=None, log_level=20)[source]

Bases: PhysioTwin4DBase

Workflow to convert one or more meshes to USD with configurable splitting and appearance (solid color, anatomic material, or colormap).

__init__(input_meshes, usd_project_name, output_directory, *, separate_by_connectivity=True, separate_by_cell_type=False, frames_per_second=60.0, extract_surface=True, static_merge=False, time_codes=None, appearance='solid', solid_color=(0.8, 0.8, 0.8), anatomy_type=None, object_names=None, label_names=None, segmenter=None, colormap_primvar=None, colormap_name='viridis', colormap_intensity_range=None, log_level=20)[source]

Initialize the VTK-to-USD workflow.

Parameters:

input_meshes (Sequence[Union[DataSet, vtkDataSet]]) – One or more PyVista/VTK meshes. A single mesh, or static_merge=True, produces a static scene; multiple meshes with static_merge=False (default) are treated as ordered time-series frames, in list order.
usd_project_name (str) – Project name; used as the root USD prim name (/World/{usd_project_name}) and the output filename. A trailing USD extension (.usd, .usda, .usdc) is stripped from the prim name but preserved for the output filename; if omitted, “.usd” is used.
output_directory (Union[str, Path]) – Directory where the output USD file is written.
separate_by_connectivity (bool) – If True, split mesh into separate objects by connectivity.
separate_by_cell_type (bool) – If True, split mesh by cell type (triangle/quad/…). Cannot be True when separate_by_connectivity is True.
frames_per_second (float) – FPS for time-varying data.
extract_surface (bool) – For volumetric meshes, extract surface before conversion.
static_merge (bool) – If True, input_meshes is not a time series - each mesh is written as a separate static object with no time samples (see ConvertVTKToUSD).
time_codes (Optional[list[float]]) – Explicit time codes aligned to input_meshes, used when static_merge is False. If None, uses sequential integers [0, 1, 2, …].
appearance (Literal['solid', 'anatomy', 'colormap']) – “solid” | “anatomy” | “colormap”.
solid_color (tuple[float, float, float]) – RGB in [0,1] when appearance == “solid”.
anatomy_type (Optional[str]) – Anatomy material name applied to every mesh when appearance == “anatomy” (e.g. heart, lung, bone, soft_tissue). None (default) instead resolves a material per mesh prim from that prim’s name, so a stage whose objects are named after the structures they hold gets per-structure materials (e.g. ventricle_left vs. myocardium). A name matching no material falls back to the object’s field_data['AnatomyGroup'] (so “rib_left_3” still reaches the bone material) and then to the “other” material.
object_names (Optional[Sequence[str]]) – Prim names aligned to input_meshes, used when static_merge is True. None (default) derives them from each mesh’s field_data['SegmentationLabelNames'] when that holds exactly one name (as written by WorkflowConvertImageToVTK), and falls back to {usd_project_name}_{index} otherwise.
label_names (Optional[Mapping[int, str]]) – Mapping of label id → structure name. Each input mesh is then split on its per-cell SegmentationLabelIds (or boundary_labels) array, so every structure becomes its own prim at /World/{usd_project_name}/{group}/{structure}: time-varying across frames, or static from a single mesh. This is the only way structure identity survives a time series; without it, parts are named by connectivity-component order, which is positional per frame. None (default) reads the ids off the meshes themselves when they carry the per-cell array, and names them from segmenter’s taxonomy — so passing a mesh merged by ContourTools.save_combined_surfaces() splits by structure without further arguments. static_merge accepts only one labeled mesh: several would collide on one prim path per label.
segmenter (Optional[SegmentAnatomyBase]) – Segmenter whose taxonomy groups the labels of label_names by anatomy type. Also selects each structure’s material when appearance == “anatomy”, through USDAnatomyTools.enhance_meshes(), which falls back to the containing group for a structure with no material of its own.
colormap_primvar (Optional[str]) – Primvar name for coloring when appearance == “colormap” (e.g. vtk_point_stress_c0). If None, a candidate is auto-picked when possible.
colormap_name (str) – Matplotlib colormap name when appearance == “colormap”.
colormap_intensity_range (Optional[tuple[float, float]]) – Optional (vmin, vmax) for colormap; None = auto from data.
log_level (int | str) – Logging level.

process()[source]

Run the full workflow: convert meshes to USD, then apply the chosen appearance.

Returns:

“usd_file” (str): Path to the created USD file.

Return type:

Dict with the results of the workflow

import pyvista as pv
from physiotwin4d import WorkflowConvertVTKToUSD

input_meshes = [pv.read("heart_000.vtp"), pv.read("heart_001.vtp")]
workflow = WorkflowConvertVTKToUSD(
    input_meshes=input_meshes,
    usd_project_name="heart",
    output_directory="./output",
    appearance="anatomy",
    anatomy_type="heart",
)

result = workflow.process()
usd_file = result["usd_file"]

Statistical Shape Modeling

class physiotwin4d.WorkflowCreateMeanSurface(surfaces, template_surface=None, spatial_resolution=1.0, buffer_factor=0.25, log_level=20)[source]

Bases: PhysioTwin4DBase

Compute the mean shape of N surfaces with differing point counts.

Example

>>> workflow = WorkflowCreateMeanSurface(surfaces=[s0, s1, s2])
>>> workflow.set_number_of_iterations(3)
>>> result = workflow.process()
>>> mean_surface = result["mean_surface"]

__init__(surfaces, template_surface=None, spatial_resolution=1.0, buffer_factor=0.25, log_level=20)[source]

Initialize the mean-surface workflow.

Parameters:

surfaces (list[DataSet]) – Population of surfaces to average. Point counts and topologies may differ; only the template’s topology survives.
template_surface (Optional[DataSet]) – Surface providing the topology of the mean and the starting point of the atlas iteration. Defaults to the middle entry of surfaces. With enough iterations the result no longer depends on this choice.
spatial_resolution (float) – Voxel size of the distance-map grid used for registration. Default: 1.0
buffer_factor (float) – Padding around the template bounding box for that grid. Default: 0.25
log_level (int | str) – Logging level. Default: logging.INFO.

Raises:

ValueError – If fewer than two surfaces are supplied.

set_number_of_iterations(number_of_iterations)[source]

Set the atlas iteration count; 1 is a single template-biased pass.

Return type:: None

set_convergence_tolerance(convergence_tolerance)[source]

Set the RMS point motion (mm) below which iteration stops.

Return type:: None

set_alignment_transform_type(transform_type)[source]

Set the ICP alignment type: 'Rigid' (default) or 'Affine'.

Return type:: None

set_registration_transform_type(transform_type)[source]

Set the distance-map registration type used for correspondence.

'Deformable' (default) runs Greedy affine + ICON; 'Affine' and 'Rigid' stop after the Greedy stage.

Return type:: None

process()[source]

Run the atlas iteration and return the mean surface.

Return type:: dict[str, Any]
Returns:: Dict with mean_surface (template topology, mean shape), corresponded_surfaces and forward_transforms from the final iteration, the per-iteration iteration_rms_mm, and number_of_iterations_run.

class physiotwin4d.WorkflowCreateStatisticalModel(sample_meshes, reference_mesh, number_of_pca_components=7, reference_spatial_resolution=1.0, reference_buffer_factor=0.25, solve_for_surface_pca=True, log_level=20)[source]

Bases: PhysioTwin4DBase

Create a PCA statistical shape model from a sample of meshes aligned to a reference.

Pipeline:

Extract surfaces from sample and reference meshes, or keep as meshes
ICP alignment: align each sample surface to the reference (template) surface. Always extract surfaces for ICP alignment.
Deformable registration: establish dense correspondence via Greedy affine + ICON deformable registration. Uses either full meshes or surfaces.
Correspondence: warp reference model by each transform to get aligned shapes
PCA: compute mean and modes from corresponded shapes

sample_meshes

List of sample mesh DataSets (.vtk/.vtu/.vtp geometry)

Type:: list

reference_mesh

Reference mesh; its surface is used for alignment

Type:: pv.DataSet

number_of_pca_components

Number of PCA components to retain

Type:: int

reference_spatial_resolution

Resolution for reference image from mesh

Type:: float

reference_buffer_factor

Buffer around mesh for reference image

Type:: float

__init__(sample_meshes, reference_mesh, number_of_pca_components=7, reference_spatial_resolution=1.0, reference_buffer_factor=0.25, solve_for_surface_pca=True, log_level=20)[source]

Initialize the create-statistical-model workflow.

Parameters:

sample_meshes (list[DataSet]) – List of sample mesh DataSets (PyVista PolyData or UnstructuredGrid).
reference_mesh (DataSet) – Reference mesh; its surface is used to align all samples.
number_of_pca_components (int) – Number of PCA components. Default 7.
reference_spatial_resolution (float) – Isotropic resolution (mm) for reference image. Default 1.0.
reference_buffer_factor (float) – Buffer factor around mesh for reference image. Default 0.25.
solve_for_surface_pca (bool) – Whether to reduce the reference mesh to a surface. Default True.
log_level (int | str) – Logging level.

set_number_of_pca_components(n)[source]

Set number of PCA components to retain.

Return type:: None

process()[source]

Run the full pipeline and return a dictionary of results (no file I/O).

Returns:

pca_mean_surface: pv.PolyData mean shape surface
pca_mean_mesh: pv.UnstructuredGrid reference volume mesh, or None if reference was surface-only
pca_model: dict with “explained_variance_ratio”, “eigenvalues”, “components” (same structure as pca_model.json)
pca_fitted: fitted sklearn PCA object

Return type:

dict with keys

class physiotwin4d.WorkflowFitStatisticalModelToPatient(template_model, patient_models=None, patient_image=None, patient_labelmap=None, template_labelmap=None, labelmap_interior_object_ids=None, segmentation_method=None, log_level=20)[source]

Bases: PhysioTwin4DBase

Register anatomical models using multi-stage ICP, labelmap-based, and image-based: registration.

This class provides a flexible workflow for registering generic anatomical models (e.g., cardiac models) to patient-specific surface models and images. The registration pipeline combines: - Initial model alignment using RegisterModelsICP (centroid + affine ICP) - Labelmap-based deformable registration using RegisterModelsDistanceMaps (Greedy/ICON) - Optional final labelmap-to-image refinement using Icon registration

Registration Pipeline:

ICP Alignment: Rough affine alignment using RegisterModelsICP
PCA Registration: Performs PCA-based shape fitting using
RegisterModelsPCA
Labelmap-to-Labelmap: Deformable registration using RegisterModelsDistanceMaps
Labelmap-to-Image: Final refinement

template_model

Generic anatomical model to be registered

Type:: pv.DataSet

template_model_surface

Surface extracted from template_model_surface

Type:: pv.PolyData

template_labelmap

Multi-label labelmap for template model

Type:: itk.Image

template_mask

Binary mask for template model registration region

Type:: itk.Image

patient_models

Patient-specific models

Type:: list of pv.DataSet

patient_model_surface

Primary patient model surface (first in list)

Type:: pv.PolyData

combined_patient_model

Merged patient models before surface extraction; used when use_surface=False.

Type:: pv.PolyData

patient_image

Reference image providing coordinate frame

Type:: itk.Image

patient_labelmap

Multi-label labelmap for patient model

Type:: itk.Image

patient_mask

Binary mask for patient registration region

Type:: itk.Image

mask_dilation_mm

Dilation for binary mask generation

Type:: float

distancemap_squared_max

Saturation radius of the labelmap-to-labelmap distance maps, in squared millimeters. None means derive it from mask_dilation_mm as (1.25 * mask_dilation_mm)**2

Type:: Optional[float]

transform_tools

Transform utilities

Type:: TransformTools

registrar_ICON

ICON registration instance

Type:: RegisterImagesICON

registrar_Greedy

Greedy registration instance

Type:: RegisterImagesGreedy

use_pca_registration

Whether PCA registration is enabled (set via set_use_pca_registration)

Type:: bool

pca_model

PCA model dict when PCA enabled; same structure as WorkflowCreateStatisticalModel output

Type:: dict

number_of_pca_components

Number of PCA components when PCA enabled

Type:: int

labelmap_interior_object_ids

List of labelmap IDs corresponding to interior objects that should not be used when computing a distance map.

Type:: list

icp_forward_point_transform: ICP transforms

icp_inverse_point_transform: ICP inverse transforms

icp_template_model_surface: template model surface after ICP alignment

icp_template_model: template model (UnstructuredGrid) after ICP alignment

pca_coefficients: PCA shape coefficients (if PCA used)

pca_template_model

template model after PCA registration (if PCA used)

Type:: pv.DataSet

pca_template_model_surface: template model surface after PCA registration (if PCA used)

l2l_forward_transform: Labelmap-to-labelmap forward transform

l2l_inverse_transform: Labelmap-to-labelmap inverse transform

l2l_template_model_surface: template model surface after labelmap-to-labelmap registration

l2i_forward_transform: Labelmap-to-image forward transform

l2i_inverse_transform: Labelmap-to-image inverse transform

l2i_template_model_surface: template model surface after labelmap-to-image registration

l2i_template_labelmap: template labelmap after labelmap-to-image registration

registered_template_model: Final registered model

registered_template_model_surface: Final registered model surface

Example

>>> # Initialize with minimal parameters (no labelmap; no patient image -> reference created from patient models)
>>> registrar = WorkflowFitStatisticalModelToPatient(
...     template_model=heart_model,
...     patient_models=[lv_model, mc_model, rv_model],
... )
>>> registrar.set_mask_dilation_mm(20)
>>> # To enable PCA registration, call before process():
>>> # registrar.set_use_pca_registration(True, pca_model=pca_model_dict, number_of_pca_components=10)
>>> # To enable labelmap-to-image refinement:
>>> # registrar.set_use_labelmap_to_image_registration(True, template_labelmap, organ_mesh_ids, organ_extra_ids, background_ids)
>>> result = registrar.process()

__init__(template_model, patient_models=None, patient_image=None, patient_labelmap=None, template_labelmap=None, labelmap_interior_object_ids=None, segmentation_method=None, log_level=20)[source]

Initialize the model-to-image-and-model registration pipeline.

Parameters:

template_model (DataSet) – Generic anatomical model to be registered
patient_models (list[DataSet] | None) – List of patient-specific models extracted from imaging data. Typically 3 models for cardiac applications: LV, myocardium, RV.
patient_image (Optional[Image]) – Optional patient image providing the target coordinate frame. If None, a reference image is created from the patient model surface via create_reference_image (contour_tools).
labelmap_interior_object_ids (Optional[list]) – Optional list of labelmap IDs that should not be used when computing the distance map since they are interior (surrounded by other objects).
segmentation_method (Optional[SegmentAnatomyBase]) – Segmentation backend instance used by WorkflowConvertImageToVTK when patient_models is None and patient_image is provided. Defaults to a new SegmentHeartSimplewareTrimmedBranches (matches KCL-Heart-Model template extent) when None. Ignored when patient_models is supplied.
log_level (int | str) – Logging level (logging.DEBUG, logging.INFO, logging.WARNING). Default: logging.INFO

Raises:

TypeError – If segmentation_method is neither None nor a SegmentAnatomyBase instance.

set_mask_dilation_mm(mask_dilation_mm)[source]

Set dilation amount for binary registration masks.

Parameters:: mask_dilation_mm (float) – Dilation amount in millimeters for binary registration mask generation. Default: 10mm
Return type:: None

set_distancemap_squared_max(distancemap_squared_max)[source]

Set the saturation radius of the labelmap-to-labelmap distance maps.

The radius fixes those images’ intensity distribution, so it has to match the value the ICON weights in use were finetuned at – see tutorials/parameters_lung_ct_dirlab.py and tutorials/parameters_heart_ct_kcl.py, which carry one value per organ. Left unset, it is derived from mask_dilation_mm.

Parameters:: distancemap_squared_max (float) – Maximum squared distance, in squared millimeters, the distance maps are normalized against.
Return type:: None

set_labelmap_to_labelmap_icon_weights_path(weights_path)[source]

Set a finetuned ICON checkpoint for the labelmap-to-labelmap stage.

That stage (register_labelmap_to_labelmap()) registers distance maps rather than image intensities, so it benefits from weights finetuned on distance maps – e.g. by tutorials/tutorial_02_lung_distancemap_finetune_icon.py. The labelmap-to-image stage keeps the stock weights: it registers the patient image itself.

Parameters:: weights_path (str) – Path to an existing uniGradICON checkpoint.
Raises:: FileNotFoundError – If weights_path does not exist.
Return type:: None

set_use_pca_registration(use_pca_registration, pca_model=None, number_of_pca_components=0, use_surface=False)[source]

Set whether to use PCA-based registration and provide the PCA model.

When enabling (True), pca_model and number_of_pca_components must be provided.

Parameters:

use_pca_registration (bool) – Whether to use PCA registration after ICP.
pca_model (Optional[dict[str, Any]]) – Required when use is True. PCA model dict (e.g. from WorkflowCreateStatisticalModel result[“pca_model”]) with keys “eigenvalues” and “components”.
number_of_pca_components (int) – Required when use is True. Number of PCA components to use. Default 0 means use all components.
use_surface (bool) – Whether to use the surface of the patient model for PCA registration.

Raises:

ValueError – If use is True and pca_model is None.

Return type:

None

set_use_labelmap_to_labelmap_registration(use_labelmap_to_labelmap_registration)[source]

Set whether to use labelmap-to-labelmap registration.

Parameters:: use_labelmap_to_labelmap_registration (bool) – Whether to use labelmap-to-labelmap deformable registration. Default: True
Return type:: None

set_use_labelmap_to_image_registration(use_labelmap_to_image_registration, template_labelmap=None, template_labelmap_organ_mesh_ids=None, template_labelmap_organ_extra_ids=None, template_labelmap_background_ids=None)[source]

Set whether to use labelmap-to-image registration.

When enabling (True), a template labelmap and label IDs must be provided so the workflow can propagate and refine the labelmap to the patient image.

Parameters:

use_labelmap_to_image_registration (bool) – Whether to use labelmap-to-image registration.
template_labelmap (Optional[Image]) – Template labelmap in template model space (same geometry as template_model). Required when use is True unless one was already supplied to the constructor, in which case that value is used.
template_labelmap_organ_mesh_ids (Optional[list[int]]) – Required when use is True. Label IDs for organ mesh in the template labelmap.
template_labelmap_organ_extra_ids (Optional[list[int]]) – Required when use is True. Label IDs for organ-extra structures in the template labelmap.
template_labelmap_background_ids (Optional[list[int]]) – Required when use is True. Label IDs for background in the template labelmap.

Raises:

ValueError – If use is True and any of template_labelmap or the id lists is None or missing.

Return type:

None

register_model_to_model_icp()[source]

Perform ICP alignment of template model to patient model.

Uses RegisterModelsICP class for ICP alignment.

Returns:

Dictionary containing:

’forward_transform’: used to warp an image from model to patient space
’inverse_transform’: used to warp an image from patient to model space
’registered_template_model_surface’: Transformed model model surface

Return type:

dict

register_model_to_model_pca()[source]

Perform PCA-based registration after ICP alignment.

Uses RegisterModelsPCA to optimize shape coefficients against a distance map of the patient. The statistical model’s modes are defined in the un-aligned template frame, so the registrar is given the raw template and the ICP alignment is passed as its post_pca_transform.

Returns:

Dictionary containing:

’forward_point_transform’: DisplacementFieldTransform mapping un-aligned template points to their PCA-deformed positions. It excludes the ICP alignment, which is applied separately.
’inverse_point_transform’: its inverse
’pca_coefficients’: PCA shape coefficients
’registered_template_model_surface’: PCA-registered model surface

Return type:

dict

Raises:

ValueError – If PCA data has not been set

register_labelmap_to_labelmap()[source]

Perform labelmap-based deformable registration of template model to patient model.

Uses RegisterModelsDistanceMaps with Greedy affine followed by ICON deformable registration on distance maps derived from the model surfaces.

Returns:

Dictionary containing:

’forward_transform’: template to patient space transform
’inverse_transform’: patient to template space transform
’registered_template_model_surface’: Transformed template model surface
’registered_template_labelmap’: Transformed template labelmap

Return type:

dict

register_labelmap_to_image(use_ICON_refinement=False)[source]

Perform labelmap-to-image refinement.

Uses registration to align the propagated template labelmap to actual image intensities.

Returns:

Dictionary containing:

’inverse_transform’: patient to template space transform
’forward_transform’: template to patient space transform
’registered_template_model_surface’: Transformed template model surface
’registered_template_labelmap’: Transformed template labelmap

Return type:

dict

transform_model(base_model=None)[source]

Apply registration transforms to the model.

Transforms the model through all registration stages.

Parameters:: base_model (Optional[DataSet]) – Base model for generating the new model. If None, the template model is used.
Returns:: Registered model
Return type:: pv.DataSet

process(use_ICON_registration_refinement=False)[source]

Execute the complete multi-stage registration workflow.

Runs registration stages in sequence:

ICP alignment (RegisterModelsICP)
PCA registration (PCA data was provided)
Labelmap-to-labelmap deformable registration (RegisterModelsDistanceMaps)
Optional labelmap-to-image refinement (Icon); requires template labelmap and IDs
set via set_use_labelmap_to_image_registration(True, …).

Parameters:: use_ICON_registration_refinement (bool) – Whether to apply ICON refinement in the labelmap-to-image stage (Stage 4). The labelmap-to-labelmap stage always uses Greedy affine + ICON deformable. Default: False
Return type:: dict
Returns:: dict with registered_template_model and registered_template_model_surface

import itk
import pyvista as pv

from physiotwin4d import WorkflowFitStatisticalModelToPatient

workflow = WorkflowFitStatisticalModelToPatient(
    template_model=pv.read("template_heart.vtu"),
    patient_models=[pv.read("lv.vtp"), pv.read("rv.vtp")],
    patient_image=itk.imread("patient_ct.nii.gz"),
)

result = workflow.process()

High-Resolution 4D CT Reconstruction

class physiotwin4d.WorkflowReconstructHighres4DCT(time_series_images, reference_image, reference_time_frame=0, register_reference_time_frame_to_reference_image=True, registration_method=None, log_level=20)[source]

Bases: PhysioTwin4DBase

Reconstruct high-resolution 4D CT from time series and reference image.

This class implements a workflow for reconstructing high-resolution dynamic CT images by registering low-resolution time-series images to a high-resolution reference image using a caller-supplied registration backend.

Registration Pipeline:

Time Series Registration: Register each time-series image to the high-resolution reference using RegisterTimeSeriesImages
Reconstruction: Apply inverse transforms to reconstruct high-resolution time series
Optional Upsampling: Resample to isotropic high resolution

Input Requirements:

time_series_images: Ordered list of 3D images (typically lower resolution)
reference_image: High-resolution reference image
All images should be in the same anatomical coordinate system

registration_method accepts a pre-configured RegisterImagesBase instance. Configure backend-specific parameters (iteration counts, etc.) on the instance before passing it in. Defaults to a new RegisterImagesGreedy when omitted.

time_series_images

Ordered list of time-series images

Type:: list[itk.Image]

reference_image

High-resolution reference image

Type:: itk.Image

reference_time_frame

Index of reference time frame in time series

Type:: int

register_reference_time_frame_to_reference_image

Whether to register the reference time frame to the reference image

Type:: bool

upsample_to_fixed_resolution

Whether to upsample reconstruction

Type:: bool

registrar

Internal registration object

Type:: RegisterTimeSeriesImages

forward_transforms

one per frame; each warps its moving image onto the reference grid

Type:: list[itk.Transform]

inverse_transforms

one per frame; each warps the reference image onto that frame’s moving grid (used for reconstruction)

Type:: list[itk.Transform]

losses

Registration loss values

Type:: list[float]

reconstructed_images

Reconstructed high-resolution images

Type:: list[itk.Image]

Example

>>> # Initialize workflow with data
>>> workflow = WorkflowReconstructHighres4DCT(
...     time_series_images=lowres_images,
...     reference_image=highres_reference,
...     reference_time_frame=7,
... )
>>>
>>> # Access results
>>> reconstructed = result['reconstructed_images']
>>> transforms = result['forward_transforms']
>>> losses = result['losses']

__init__(time_series_images, reference_image, reference_time_frame=0, register_reference_time_frame_to_reference_image=True, registration_method=None, log_level=20)[source]

Initialize the high-resolution 4D CT reconstruction workflow.

Parameters:

time_series_images (list[itk.Image]) – Ordered list of 3D time-series images to be registered and reconstructed
reference_image (itk.Image) – High-resolution 3D reference image
reference_time_frame (int, optional) – Index of the reference time frame in the time series. Registration proceeds bidirectionally from this frame. Default: 0
register_reference_time_frame_to_reference_image (bool, optional) – If True, register the reference time frame to the reference image. If False, use an identity transform for that frame. Default: True
registration_method (Optional[RegisterImagesBase]) – Registration backend instance. Defaults to a new RegisterImagesGreedy when None.
log_level (int | str) – Logging level (logging.DEBUG, logging.INFO, etc.). Default: logging.INFO

Raises:

ValueError – If time_series_images is empty
ValueError – If reference_time_frame is out of range
TypeError – If registration_method is neither None nor a RegisterImagesBase instance

set_modality(modality)[source]

Set the imaging modality for registration optimization.

Parameters:: modality (str) – The imaging modality (e.g., ‘ct’, ‘mri’)
Return type:: None

set_mask_dilation(mask_dilation_mm)[source]

Set the dilation of the fixed and moving image masks.

Parameters:: mask_dilation_mm (float) – The dilation in millimeters
Return type:: None

set_fixed_mask(fixed_mask)[source]

Set a binary mask for the fixed image region of interest.

Parameters:: fixed_mask (itk.Image) – Binary mask defining ROI in fixed image
Return type:: None

set_moving_masks(moving_masks)[source]

Set binary masks for the moving images.

Parameters:: moving_masks (list[itk.Image] | None) – List of binary masks, one for each moving image. If None, no masks are used. Must have same length as time_series_images if provided.
Raises:: ValueError – If moving_masks length doesn’t match time_series_images
Return type:: None

register_time_series()[source]

Register time series images to the fixed image.

Performs sequential registration of all time-series images to the high-resolution reference image using the configured parameters.

Returns:

Dictionary containing:

’forward_transforms’ (list[itk.Transform]): one per frame; each warps its moving image onto the reference grid
’inverse_transforms’ (list[itk.Transform]): one per frame; each warps the reference image onto that frame’s moving grid (see docs/developer/transform_conventions)
’losses’ (list[float]): Registration loss value for each image

Return type:

dict

Raises:

RuntimeError – If registration fails

set_upsample_to_fixed_resolution(upsample_to_fixed_resolution)[source]

Set whether to upsample the reconstructed time series to the fixed resolution.

Parameters:: upsample_to_fixed_resolution (bool) – Whether to upsample the reconstructed time series to the fixed resolution.
Return type:: None

reconstruct_time_series()[source]

Reconstruct high-resolution time series using inverse transforms.

Applies the inverse transforms from registration to reconstruct each time-series image in the high-resolution fixed image space.

Returns:

Dictionary containing:

’reconstructed_images’ (list[itk.Image]): Reconstructed high-resolution time-series images

Return type:

dict

Raises:

RuntimeError – If reconstruction fails
ValueError – If inverse_transforms is not set (call register_time_series first)

process()[source]

Execute the complete high-resolution 4D CT reconstruction workflow.

Runs the full pipeline: 1. Register time series to high-resolution reference 2. Reconstruct high-resolution time series using inverse transforms

Returns:

Dictionary containing all results:

’forward_transforms’ (list[itk.Transform]): Registration transforms
’inverse_transforms’ (list[itk.Transform]): Inverse transforms
’losses’ (list[float]): Registration loss values
’reconstructed_images’ (list[itk.Image]): Reconstructed high-res images

Return type:

dict

Raises:

RuntimeError – If any stage of the workflow fails

import itk

from physiotwin4d import RegisterImagesGreedyICON, WorkflowReconstructHighres4DCT

time_series_images = [itk.imread(f"phase_{idx:02d}.mha") for idx in range(10)]
workflow = WorkflowReconstructHighres4DCT(
    time_series_images=time_series_images,
    reference_image=time_series_images[0],
    reference_time_frame=0,
    registration_method=RegisterImagesGreedyICON(),
)

workflow.set_modality("ct")
result = workflow.process()

Finetune ICON Registration

class physiotwin4d.WorkflowFinetuneICONRegistration(subject_image_files, output_dir, finetune_name, subject_ids=None, subject_labelmap_files=None, subject_mask_files=None, subject_landmark_files=None, epochs=500, batch_size=4, learning_rate=5e-05, input_shape=(175, 175, 175), similarity='lncc', lambda_value=1.5, dice_loss_weight=0.5, lncc_sigma=1, ct_window=(-1000.0, 1000.0), is_ct=True, gpus=None, eval_period=10, save_period=50, unigradicon_src_path=None, log_level=20)[source]

Bases: PhysioTwin4DBase

Finetune uniGradICON on paired 3D images.

Build a paired dataset JSON and YAML config from per-subject lists of image, labelmap, and landmark files, then launch unigradicon.finetuning.finetune as a subprocess. Each subject’s time-point images form one paired group (they share a subject_id).

subject_image_files

Per-subject lists of image paths. Images within one inner list share a subject_id during finetuning.

Type:: list[list[str]]

output_dir

Directory where the dataset JSON, YAML config, and the uniGradICON checkpoints/ tree are written.

Type:: Path

finetune_name

Sub-directory name for the experiment outputs.

Type:: str

subject_ids

One ID per subject (e.g. patient identifiers). Written into the dataset JSON’s subject_id field; falls back to synthetic subject_NNNN when None.

Type:: Optional[list[str]]

subject_labelmap_files

Per-subject multi-label labelmap paths aligned with subject_image_files. None (or per-image None) means no labelmap for that image. If supplied for at least one image, paired-with-seg training is enabled.

Type:: Optional[list[list[Optional[str]]]]

subject_mask_files

Per-subject binary mask paths aligned with subject_image_files, used for loss-function masking. None disables masking.

Type:: Optional[list[list[Optional[str]]]]

subject_landmark_files

Per-subject landmark CSV paths (Name,X,Y,Z format) aligned with subject_image_files. Recorded in the dataset JSON for traceability; not consumed by uniGradICON finetuning itself.

Type:: Optional[list[list[Optional[str]]]]

Example

>>> workflow = WorkflowFinetuneICONRegistration(
...     subject_image_files=[
...         ['pm0001/g000.nii.gz', 'pm0001/g050.nii.gz'],
...         ['pm0002/g000.nii.gz', 'pm0002/g050.nii.gz'],
...     ],
...     output_dir=Path('d:/PhysioTwin4D/icon_finetuned'),
...     finetune_name='duke_4d_gated_icon_ft',
...     subject_labelmap_files=[
...         ['pm0001/g000_labelmap.nii.gz', 'pm0001/g050_labelmap.nii.gz'],
...         ['pm0002/g000_labelmap.nii.gz', 'pm0002/g050_labelmap.nii.gz'],
...     ],
... )
>>> weights_path = workflow.process()

__init__(subject_image_files, output_dir, finetune_name, subject_ids=None, subject_labelmap_files=None, subject_mask_files=None, subject_landmark_files=None, epochs=500, batch_size=4, learning_rate=5e-05, input_shape=(175, 175, 175), similarity='lncc', lambda_value=1.5, dice_loss_weight=0.5, lncc_sigma=1, ct_window=(-1000.0, 1000.0), is_ct=True, gpus=None, eval_period=10, save_period=50, unigradicon_src_path=None, log_level=20)[source]

Initialize the ICON finetuning workflow.

Parameters:

subject_image_files (list[list[str]]) – Per-subject lists of image file paths. Each inner list groups frames belonging to one subject; all of those frames share a subject_id for paired training.
output_dir (Path) – Directory for the dataset JSON, YAML config, and the uniGradICON checkpoint tree.
finetune_name (str) – Sub-directory name for the experiment outputs (used as the uniGradICON experiment.name stem).
subject_ids (Optional[list[str]]) – One ID per subject, in the same order as subject_image_files. Written verbatim into the dataset JSON’s subject_id field so paired training groups frames that share an ID. None falls back to synthetic IDs of the form subject_0000, subject_0001, … Must be unique.
subject_labelmap_files (Optional[list[list[Optional[str]]]]) – Per-subject multi-label segmentation (labelmap) paths matching subject_image_files. None disables paired-with-seg training. Individual None entries inside the inner lists skip just those frames when paired-with-seg training is enabled.
subject_mask_files (Optional[list[list[Optional[str]]]]) – Per-subject binary mask paths matching subject_image_files, used for ICON loss-function masking. None disables loss-function masking. Per-image None entries skip just those frames.
subject_landmark_files (Optional[list[list[Optional[str]]]]) – Per-subject landmark CSV paths matching subject_image_files. Stored in the dataset JSON for traceability; not consumed by uniGradICON finetuning.
epochs (int) – uniGradICON training.epochs.
batch_size (int) – uniGradICON training.batch_size.
learning_rate (float) – uniGradICON training.learning_rate.
input_shape (tuple[int, int, int]) – uniGradICON training.input_shape (voxels, X/Y/Z).
similarity (str) – uniGradICON training.similarity metric (e.g. lncc).
lambda_value (float) – uniGradICON training.lambda regularization weight.
dice_loss_weight (float) – uniGradICON training.dice_loss_weight.
lncc_sigma (int) – uniGradICON training.lncc_sigma.
ct_window (tuple[float, float]) – uniGradICON dataset ct_window [low, high] in HU.
is_ct (bool) – Whether the dataset is CT (passes through to dataset config).
gpus (Optional[list[int]]) – GPU device indices for training. Defaults to [0].
eval_period (int) – uniGradICON training.eval_period.
save_period (int) – uniGradICON training.save_period.
unigradicon_src_path (Optional[Path]) – Optional path to a local uniGradICON source tree to prepend to PYTHONPATH when running finetuning. Useful for using a checked-out copy instead of the installed package.
log_level (Union[int, str]) – Logging level (logging.DEBUG, logging.INFO, …).

Raises:

ValueError – If subject_image_files is empty.
ValueError – If subject_labelmap_files, subject_mask_files, or subject_landmark_files is provided with a shape that does not match subject_image_files.

prepare_dataset(use_labelmaps=None, use_masks=None)[source]

Write the uniGradICON dataset JSON from the configured file lists.

Builds one entry per image with image, optional segmentation, optional mask, optional landmarks (path only), and a subject_id derived from the inner-list index.

Masks come from subject_mask_files only; none are derived. Frames are skipped (with a log warning) when a required companion (segmentation for paired-with-seg training, or mask for loss-function masking) is missing.

Return type:: Path
Returns:: Path to the dataset JSON written under experiment_dir.
Raises:: FileNotFoundError – If an image listed in subject_image_files does not exist on disk.

prepare_config(dataset_json_path=None)[source]

Write the uniGradICON finetuning YAML config.

Parameters:: dataset_json_path (Optional[Path]) – Path to the dataset JSON to reference. Defaults to the JSON last produced by prepare_dataset().
Return type:: Path
Returns:: Path to the YAML config written under experiment_dir.
Raises:: ValueError – If no dataset JSON path is available.

expected_weights_path()[source]

Return the path uniGradICON writes its final checkpoint to.

unigradicon.finetuning.finetune writes <experiment.name>/checkpoints/network_weights_final.trch at the end of training – its NETWORK_WEIGHTS_PREFIX plus the "final" epoch label. Also the return value of process().

The filename is hard-coded rather than imported: training runs in a subprocess so that this process never imports unigradicon (its finetuning submodule exists only on the feat-add-finetuning branch, so an import here would raise on a stock install). An upstream rename is caught by test_expected_weights_path_layout, which compares this filename against NETWORK_WEIGHTS_PREFIX wherever the submodule is installed, and by the FileNotFoundError process() raises when the checkpoint is not where this says it should be.

Return type:: Path

process()[source]

Build configs and launch unigradicon.finetuning.finetune.

Equivalent to running prepare_dataset() → prepare_config() → subprocess launch. Any existing dataset JSON or YAML in experiment_dir is overwritten.

Return type:

Path

Returns:

Path to the final checkpoint (network_weights_final.trch). The file is written by the subprocess and exists only after a successful run.

Raises:

subprocess.CalledProcessError – If the finetuning subprocess exits with a non-zero status.
FileNotFoundError – If the subprocess succeeded but the checkpoint is not where expected_weights_path() says it should be.

from physiotwin4d import RegisterImagesICON, WorkflowFinetuneICONRegistration

workflow = WorkflowFinetuneICONRegistration(
    subject_image_files=subject_image_files,
    output_dir=weights_dir,
    finetune_name="my_cohort",
    subject_ids=subject_ids,
    epochs=100,
)
weights_path = workflow.process()

registrar = RegisterImagesICON()
registrar.set_weights_path(str(weights_path))

Workflow Classes

Available Workflows

Convert Image to USD

Image to VTK

VTK to USD

Statistical Shape Modeling

High-Resolution 4D CT Reconstruction

Finetune ICON Registration

See Also