Scoring a Mesh-Stage Model Against Images

A mm error against the surfaces a registration produced says how well the network reproduces that registration. WorkflowEvaluateMovement asks the other question: how close are the size and shape of the inferred anatomy to the anatomy that was actually imaged, structure by structure. See Tutorial 11 in Tutorials.

For every gated time point it carries the reference frame’s labelmap into that time point with the network’s own deformation, and compares the result to the labelmap of the frame that was acquired: volume difference, Dice and surface RMSE per lung lobe or per heart chamber.

Per-structure scoring

class physiotwin4d.WorkflowEvaluateMovement(movement_workflow, label_names, log_level=20)[source]

Bases: PhysioTwin4DBase

Score inferred motion per anatomical structure against acquired frames.

Parameters:

  • movement_workflow (WorkflowInferMovement) – The displacement decoder whose predictions are scored.

  • label_names (dict[int, str]) – Structures to score, {label_id: name}. Ids the reference frame does not contain are dropped with a warning; ids a single acquired frame does not contain are skipped for that frame alone, since a structure can leave the field of view.

  • log_level (int | str) – Logging level. Default: logging.INFO.

__init__(movement_workflow, label_names, log_level=20)[source]

Initialize the base class with logging configuration.

Parameters:

  • class_name – Name for the class (used in log messages). If None, uses the class name. Default: None

  • log_level (int | str) – Logging level. Can be an integer (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL) or a string (‘DEBUG’, ‘INFO’, ‘WARNING’, ‘ERROR’, ‘CRITICAL’). Default: logging.INFO

  • log_to_file – Optional file path to write logs to in addition to console output. Default: None

process(case_id, shape_parameters, reference_mesh, reference_labelmap, ground_truth_labelmaps, output_directory, smoothing_sigma_mm=10.0, evaluation_spacing_mm=1.0, include_dice=True)[source]

Score every gated time point of one case.

Parameters:

  • case_id (str) – Name of the case being scored, recorded in every output.

  • shape_parameters (Path) – JSON file with the case’s PCA coefficient vector.

  • reference_mesh (Path) – The case’s fitted reference-frame SSM surface. The predicted displacements are added to its points, and its extent defines the evaluation grid.

  • reference_labelmap (Image) – Labelmap of the reference frame, the anatomy carried into every other time point.

  • ground_truth_labelmaps (dict[float, Image]) – Acquired labelmap per stage, keyed by the normalized stage in [0, 1].

  • output_directory (Path) – Directory the report, the CSV and the per-stage geometry are written to.

  • smoothing_sigma_mm (float) – Gaussian sigma, in millimeters, that turns the network’s surface-shell deformation into a continuous field.

  • evaluation_spacing_mm (float) – Isotropic pitch every metric is measured on. It sets both the voxel volume the Dice and volume figures are quantized to and the resolution of the deformation fields, whose memory grows with its cube.

  • include_dice (bool) – Report the Dice overlap. Turn it off for a structure whose motion is small against its own size: Dice is an overlap fraction, so a lung lobe scores over 0.96 undeformed and the column says more about the organ’s bulk than about the motion. The volume and surface figures still resolve it.

Return type:

dict[str, Any]

Returns:

Dict with rows (every metric row), csv_file, report_file, volume_plot_file, predicted_surfaces and warped_labelmaps.

Raises:

ValueError – If ground_truth_labelmaps is empty, none of the requested labels are in the reference frame, or no label survives scoring because the acquired frames contain none of them.

static dice(truth, predicted, label)[source]

Dice overlap of one label. nan when neither volume contains it.

Return type:

float

static volume_mm3(labels, label, voxel_volume_mm3)[source]

Volume of one label, in cubic millimeters.

Return type:

float

static surface_rmse_mm(truth, predicted)[source]

Symmetric point-to-surface RMSE, in millimeters.

Both directions are pooled before the root-mean-square. A one-sided RMSE misses a prediction that covers the truth everywhere but also bulges somewhere the truth does not reach.

Return type:

float

Example

from physiotwin4d import (
    WorkflowEvaluateMovement,
    WorkflowInferMovement,
    WorkflowInferPhysicsNeMo,
)

evaluate = WorkflowEvaluateMovement(
    movement_workflow=WorkflowInferMovement(
        WorkflowInferPhysicsNeMo(model_directory=model_dir)
    ),
    label_names={28: "lung_upper_lobe_left", 29: "lung_lower_lobe_left"},
)
result = evaluate.process(
    case_id="Case1Pack",
    shape_parameters=pca_coefficients_file,
    reference_mesh=ssm_surface_file,
    reference_labelmap=reference_labelmap,
    ground_truth_labelmaps={0.0: frame_00, 0.1: frame_10},
    output_directory=out_dir,
)
print(result["report_file"], result["csv_file"])

Notes

Why labelmaps rather than the model’s surface. The lung shape model carries its five lobes as per-cell labels, but the heart model is a single structure — the whole heart minus its chamber cavities — so its chambers exist only in the acquired labelmaps. Warping those labelmaps scores every structure the acquisition contains, whether or not the shape model represents it separately.

The evaluation grid. Everything is measured on one isotropic grid built around the reference anatomy, so a case whose gated frames carry different slice pitches is still scored on a single, stated voxel volume. Its pitch sets both that voxel volume and the memory the per-stage deformation fields take, which grows with its cube.

See Also