Available Models

Everything on this page is generated from the model registry, so it always reflects the models AIoD can actually run.

The three values you need to run a model — whether directly through Nextflow or by recognising them in the Napari plugin — are --model (the family), --model_type (the version), and --task.

Can't see a model in the plugin?

Models marked Restricted below are shared by file path rather than public download, so they only appear for people who can read that location. See Model Location for why, and adding a new location for how to share one more widely.

Tasks

A task is what a model is trying to segment. Picking one narrows the model list to those that can do it.

Task	--task	Models
Mitochondria	mito	Empanada, SEAI U-Net
Endoplasmic Reticulum	er	SEAI U-Net
Nuclear Envelope	ne	SEAI U-Net
Everything!	everything	Segment Anything, Segment Anything 2
Nuclei	nuclei	Cellpose, Cellpose-SAM, Empanada, PanSeg, StarDist
Cytoplasm	cyto	Cellpose, Cellpose-SAM
Lipid Droplets	drop	Empanada
Boundaries	boundaries	PanSeg

Model Families

Cellpose

Run with --model cellpose.

Cellpose is a generalist model for cell and nucleus segmentation.

Documentation · Repository · Cellpose: a generalist algorithm for cellular segmentation · Cellpose 2.0: how to train your own model · Cellpose3: one-click image restoration for improved cellular segmentation

Version	--model_type	Tasks	Axes	Availability
cyto3	cyto3	Cytoplasm	—	Public
nuclei	nuclei	Nuclei	—	Public
cyto1	cyto1	Cytoplasm	—	Public
cyto2	cyto2	Cytoplasm	—	Public

Parameters

Parameter	Argument	Default	Description
Diameter	diameter	0	Diameter of the cells in pixels. If None or 0, Cellpose will try to estimate it. Setting the value may improve results. See https://cellpose.readthedocs.io/en/v3.1.1.1/settings.html#diameter for details.
Segment Channel	segment_channel	0	What channel to segment on. Cellpose channel convention: 0=grayscale (no specific channel), 1=first image channel, 2=second image channel, etc. See https://cellpose.readthedocs.io/en/v3.1.1.1/settings.html#channels for details.
Nucleus Channel	nucleus_channel	0	What channel contains the nucleus. Cellpose channel convention: 0=no nucleus channel (grayscale), 1=first image channel, 2=second image channel, etc. Unused for the 'nuclei' model (nucleus is already the segment channel). See https://cellpose.readthedocs.io/en/v3.1.1.1/settings.html#channels for details.
3D Segmentation	do_3D	False	Whether to do 3D segmentation. If True, will try to segment in 3D (XY, XZ, YZ). If False, will segment in 2D and combine with stitch_threshold.
Stitch Threshold	stitch_threshold	0.0	Threshold for stitching 2D segmentations together. Value is the IoU that constitutes an overlap and thus merge. Only used if do_3D is False and >0.
Cellprob Threshold	cellprob_threshold	0.0	Threshold for flows to determine ROIs. Decrease to find more & larger ROIs, increase if too many ROIs (esp. in dim regions). See https://cellpose.readthedocs.io/en/v3.1.1.1/settings.html#cellprob-threshold for details.
Flow Threshold	flow_threshold	0.4	Threshold for flows to determine ROIs. Not used for 3D. See https://cellpose.readthedocs.io/en/v3.1.1.1/settings.html#cellprob-threshold for details.
Num Iterations	niter	0	Number of iterations to simulate dynamics for. By default (0/None), proportional to ROI diameter. See https://cellpose.readthedocs.io/en/v3.1.1.1/settings.html#number-of-iterations-niter for details.
Anisotropy	anisotropy	None (float)	Optional rescaling factor for Z (e.g. set to 2.0 if Z is sampled half as dense as X or Y). Only used if do_3D is True.
Channel axis	channel_axis	None (int)	Set the axis where channels are stored. 'None' means it tries to figure it out - useful if no/poor metadata and not loading properly.
Z axis	z_axis	None (int)	Set the axis where Z slices are stored. 'None' means it tries to figure it out - useful if no/poor metadata and not loading properly.
Batch size	batch_size	64	Number of 224x224 patches to run simultaneously on the GPU. Can make smaller or bigger depending on GPU memory usage.
Minimum size	min_size	15	Size (in pixels) of ROIs/to remove if lower. 0 will not remove any masks.

Cellpose-SAM

Run with --model cellposesam.

Cellpose-SAM is a generalist model for cell segmentation.

Documentation · Repository · Cellpose-SAM: superhuman generalization for cellular segmentation

Version	--model_type	Tasks	Axes	Availability
cpsam	cpsam	Cytoplasm, Nuclei	—	Public

Parameters

Parameter	Argument	Default	Description
Segment Channel	segment_channel	0	What channel to segment on. Cellpose-SAM is trained to be invariant to channel order and uses the first 3 channels of your image. Cellpose channel convention: 0=grayscale (no specific channel), 1=first image channel, 2=second image channel, etc. See https://cellpose.readthedocs.io/en/latest/settings.html#channels for details.
3D Segmentation	do_3D	False	Whether to do 3D segmentation. If True, will try to segment in 3D (XY, XZ, YZ). If False, will segment in 2D and combine with stitch_threshold.
Resample	resample	True	Whether to run dynamics at original image size (rather than likely smaller resized image). Slower, but more accurate boundaries.
Stitch Threshold	stitch_threshold	0.0	Threshold for stitching 2D segmentations together. Value is the IoU that constitutes an overlap and thus merge. Only used if do_3D is False and >0.
Cellprob Threshold	cellprob_threshold	0.0	Threshold for flows to determine ROIs. Decrease to find more & larger ROIs, increase if too many ROIs (esp. in dim regions). See https://cellpose.readthedocs.io/en/latest/settings.html#cellprob-threshold for details.
Flow Threshold	flow_threshold	0.4	Threshold for flows to determine ROIs. Not used for 3D. See https://cellpose.readthedocs.io/en/latest/settings.html#cellprob-threshold for details.
Num Iterations	niter	0	Number of iterations to simulate dynamics for. By default (0/None), proportional to ROI diameter. See https://cellpose.readthedocs.io/en/latest/settings.html#number-of-iterations-niter for details.
Anisotropy	anisotropy	None (float)	Optional rescaling factor for Z (e.g. set to 2.0 if Z is sampled half as dense as X or Y). Only used if do_3D is True.
Channel axis	channel_axis	None (int)	Set the axis where channels are stored. 'None' means it tries to figure it out - useful if no/poor metadata and not loading properly.
Z axis	z_axis	None (int)	Set the axis where Z slices are stored. 'None' means it tries to figure it out - useful if no/poor metadata and not loading properly.
Batch size	batch_size	64	Number of 224x224 patches to run simultaneously on the GPU. Can make smaller or bigger depending on GPU memory usage.
Minimum size	min_size	15	Size (in pixels) of ROIs/to remove if lower. 0 will not remove any masks.
Max size fraction	max_size_fraction	0.4	Maximum size of ROIs as fraction of image size. Masks larger than this are removed.
Tile Overlap	tile_overlap	0.1	Fraction of overlap of tiles when computing flows.

Empanada

Run with --model empanada.

Empanada contains segmentation models for mitochondria, nuclei, and lipid droplets in EM images.

Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model trained on a diverse dataset

Version	--model_type	Tasks	Axes	Availability
MitoNet v1	mitonet_v1	Mitochondria	—	Public
MitoNet Mini v1	mitonet_mini_v1	Mitochondria	—	Public
NucleoNet v1	nucleonet_v1	Nuclei	—	Public
DropNet v1	dropnet_v1	Lipid Droplets	—	Public

Parameters

Parameter	Argument	Default	Description
Plane	plane	XY (of XY, XZ, YZ, All)	Whether to use all planes (XY, XZ, YZ) or a single plane
Downsampling	downsampling	1 (of 1, 2, 4, 8, 16, 32, 64)	Downsampling factor for the input image
Segmentation threshold	conf_threshold	0.5	Confidence threshold for the segmentation
Center threshold	center_threshold	0.1	Confidence threshold for the center
Minimum distance	min_distance	3	Minimum distance between object centers
Maximum objects	max_objects	10000	Maximum number of objects to segment per class
Semantic only	semantic_only	False	Only run semantic segmentation for all classes
Fine boundaries	fine_boundaries	False	Finer boundaries between objects
Fill holes	fill_holes_in_segmentation	False	Whether to fill holes in the segmentation masks
Median Filter Size	median_slices	3 (of 1, 3, 5, 7, 9, 11)	Number of image slices over which to apply a median filter to semantic segmentation probabilities. Only applies to 3D/'All'.
Erode labels (px; 3D)	label_erosion	0	Will erode objects in the masks by specified number of pixels before final consensus. Only applies to 3D/'All'.
Dilate labels (px; 3D)	label_dilation	0	Will dilate objects in the masks by specified number of pixels before final consensus. Only applies to 3D/'All'.
Voxel Vote	pixel_vote_thr	2	Number of stacks from ortho-plane inference in which a voxel must be labelled in order to end up in the consensus segmentation. Only applies to 3D/'All'.
Allow single-plane objects	allow_one_view	False	Whether to allow objects that are only segmented in a single plane (XY, XZ, or YZ) to be included in the final consensus segmentation. Only applies to 3D/'All'.

PanSeg

Run with --model panseg.

PanSeg: Advanced Interactive User-Friendly Tissue Segmentation

Documentation · Repository · PanSeg: Advanced Interactive User-Friendly Tissue Segmentation · Accurate and versatile 3D segmentation of plant tissues at cellular resolution

Version	--model_type	Tasks	Axes	Availability
generic_confocal_3D_unet	generic_confocal_3D_unet	Boundaries	ZYX	Public
generic_light_sheet_3D_unet	generic_light_sheet_3D_unet	Boundaries	ZYX	Public
confocal_3D_unet_ovules_ds1x	confocal_3D_unet_ovules_ds1x	Boundaries	ZYX	Public
confocal_3D_unet_ovules_ds2x	confocal_3D_unet_ovules_ds2x	Boundaries	ZYX	Public
confocal_3D_unet_ovules_ds3x	confocal_3D_unet_ovules_ds3x	Boundaries	ZYX	Public
confocal_2D_unet_ovules_ds2x	confocal_2D_unet_ovules_ds2x	Boundaries	YX	Public
lightsheet_3D_unet_root_ds1x	lightsheet_3D_unet_root_ds1x	Boundaries	ZYX	Public
lightsheet_3D_unet_root_ds2x	lightsheet_3D_unet_root_ds2x	Boundaries	ZYX	Public
lightsheet_3D_unet_root_ds3x	lightsheet_3D_unet_root_ds3x	Boundaries	ZYX	Public
lightsheet_2D_unet_root_ds1x	lightsheet_2D_unet_root_ds1x	Boundaries	YX	Public
lightsheet_3D_unet_root_nuclei_ds1x	lightsheet_3D_unet_root_nuclei_ds1x	Nuclei	ZYX	Public
lightsheet_2D_unet_root_nuclei_ds1x	lightsheet_2D_unet_root_nuclei_ds1x	Nuclei	YX	Public
confocal_2D_unet_sa_meristem_cells	confocal_2D_unet_sa_meristem_cells	Boundaries	YX	Public
confocal_3D_unet_sa_meristem_cells	confocal_3D_unet_sa_meristem_cells	Boundaries	ZYX	Public
lightsheet_3D_unet_mouse_embryo_cells	lightsheet_3D_unet_mouse_embryo_cells	Boundaries	ZYX	Public
confocal_3D_unet_mouse_embryo_nuclei	confocal_3D_unet_mouse_embryo_nuclei	Nuclei	ZYX	Public
PlantSeg_3Dnuc_platinum	PlantSeg_3Dnuc_platinum	Nuclei	ZYX	Public

Parameters

Parameter	Argument	Default	Description
Patch Size	patch	None (list)	Patch size for prediction [Z, Y, X]. If null, will be auto-computed. Example: [80, 160, 160]
Patch Halo	patch_halo	None (list)	Halo size for patches [Z, Y, X]. If null, will be auto-computed. Example: [8, 16, 16]
Single Batch Mode	single_batch_mode	True	Whether to process patches one at a time to reduce memory usage.
Model Update	model_update	False	Whether to check for and download model updates.
Disable Progress Bar	disable_tqdm	False	Whether to disable the progress bar during prediction.
Watershed Threshold	ws_threshold	0.5	Threshold for watershed segmentation. Higher values create fewer, larger segments.
Watershed Sigma Seeds	ws_sigma_seeds	1.0	Sigma for Gaussian smoothing of seeds in watershed.
Watershed Stacked	ws_stacked	False	Whether to apply watershed slice-by-slice (2D) instead of in 3D.
Watershed Sigma Weights	ws_sigma_weights	2.0	Sigma for Gaussian smoothing of boundary map weights in watershed.
Watershed Min Size	ws_min_size	100	Minimum size (in voxels) for watershed superpixels. Smaller regions will be removed.
Watershed Alpha	ws_alpha	1.0	Alpha parameter for distance transform watershed.
Watershed Pixel Pitch	ws_pixel_pitch	None (list)	Pixel pitch for anisotropic data [Z, Y, X]. If null, assumes isotropic.
Watershed Non-Max Suppression	ws_apply_nonmax_suppression	False	Whether to apply non-maximum suppression in watershed.
GASP Linkage Criteria	gasp_linkage_criteria	average	Linkage criteria for GASP: 'average' or 'mutex_watershed'.
GASP Beta	gasp_beta	0.5	Beta parameter for GASP segmentation. Controls merging aggressiveness.
GASP Post Min Size	gasp_post_minsize	100	Minimum size (in voxels) for final segments after GASP. Smaller regions will be merged.
Number of Threads	n_threads	6	Number of threads to use for watershed and GASP segmentation.
Voxel Size	voxel_size	None (list)	Voxel size [Z, Y, X] for saving TIFF metadata. Example: [1.0, 0.5, 0.5]
Voxel Size Unit	voxel_size_unit	um	Unit for voxel size (e.g., 'um', 'nm', 'mm').

SEAI U-Net

Run with --model seai_unet.

SEAI U-Net developed on internal Crick EM data

Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations (2021) · Automatic instance segmentation of mitochondria in electron microscopy data (2021)

Version	--model_type	Tasks	Axes	Availability
U-Net	u-net	Mitochondria	—	Restricted
Finetuned HU-Net	finetuned_hu-net	Endoplasmic Reticulum	—	Restricted
Attention U-Net	attention_u-net	Mitochondria, Nuclear Envelope	—	Restricted
Finetuned Attention U-Net	finetuned_attention_u-net	Nuclear Envelope	—	Restricted

Restricted versions

Some versions above are shared by file path rather than public download, so they are only visible to people who can read that location — see Model Location.

Segment Anything

Run with --model sam.

Segment Anything is a vision foundation model with flexible prompting.

Documentation · Repository · Segment Anything

Version	--model_type	Tasks	Axes	Availability
default	default	Everything!	—	Public
vit_h	vit_h	Everything!	—	Public
vit_l	vit_l	Everything!	—	Public
vit_b	vit_b	Everything!	—	Public
MedSAM	medsam	Everything!	—	Public
MicroSAM-Boundaries	microsam-boundaries	Everything!	—	Public
MicroSAM-Organelles	microsam-organelles	Everything!	—	Public

Usage guidance

Please refer to the tooltips to provide an indication of how to adjust parameters to change the output. Given SAM is a 2D model, it is advised to first run SAM on a few representative slices and adjust parameters to get the desired output before running on the full volume. Postprocessing (i.e. relabelling) can take a while, so only enable this when suitable parameters have been found.

Parameters

Parameter	Argument	Default	Description
Points per side	points_per_side	32	Number of point prompts per side, controlling density of point grid. Higher values will capture more objects (rightly or wrongly), but take longer to run.
Points per batch	points_per_batch	64	Number of points to process per batch. Higher values will be faster but will require more (GPU) memory.
Pred IoU threshold	pred_iou_thresh	0.88	Threshold (range [0,1]) for model-predicted IoU, for filtering out low-confidence masks. Higher values will remove more masks.
Stability score threshold	stability_score_thresh	0.95	Threshold (range [0,1]) for mask stability score. Higher values will remove more masks, in conjunction with stability score offset that controls level of stability measured.
Stability score offset	stability_score_offset	1.0	Amount to shift the cutoff for stability score. Higher values will remove less stable masks.
Box NMS IoU threshold	box_nms_thresh	0.7	The IoU threshold (range [0,1]) for non-maximum suppression of boxes. Lower values will merge more masks, useful to reduce mask 'halos' and speed-up postprocessing, but could lead to undersegmentation/agglomeration.
Crop N layers	crop_n_layers	0	Values >0 will crop the image into N layers, which will be processed separately, giving more detail but taking longer. Each deeper layer will have more crops (2^N). Balance number of point prompts with 'Crop N points downscale factor'.
Crop NMS IoU threshold	crop_nms_thresh	0.7	Same as Box NMS IoU threshold, but for each crop.
Crop overlap ratio	crop_overlap_ratio	0.34133	The amount which crops overlap. Higher values may help identify more objects, but duplicates computation.
Crop N points downscale factor	crop_n_points_downscale_factor	1	Value to be raised to the power of the crop layer will be used to scale down number of points per side in each crop. Higher values will reduce point grid density in deeper layers.
Min mask region area	min_mask_region_area	3	Size (in pixels) of masks to remove if identified as a disconnected region or hole. 0 will not remove any masks.
Max mask region area	max_mask_region_area	0	Size (in pixels) of masks to remove if at least this size. 0 will not remove any masks.

Segment Anything 2

Run with --model sam2.

Segment Anything 2 is a vision foundation model with flexible prompting for images and videos.

Documentation · Repository · SAM 2: Segment Anything in Images and Videos

Version	--model_type	Tasks	Axes	Availability
default	default	Everything!	—	Public
hiera_base	hiera_base	Everything!	—	Public
hiera_small	hiera_small	Everything!	—	Public
hiera_large	hiera_large	Everything!	—	Public
hiera_tiny	hiera_tiny	Everything!	—	Public

Usage guidance

Usage guidance is coming soon once this new model is tested.

Parameters

Parameter	Argument	Default	Description
Points per side	points_per_side	32	Number of point prompts per side, controlling density of point grid. Higher values will capture more objects (rightly or wrongly), but take longer to run.
Points per batch	points_per_batch	64	Number of points to process per batch. Higher values will be faster but will require more (GPU) memory.
Pred IoU threshold	pred_iou_thresh	0.88	Threshold (range [0,1]) for model-predicted IoU, for filtering out low-confidence masks. Higher values will remove more masks.
Stability score threshold	stability_score_thresh	0.95	Threshold (range [0,1]) for mask stability score. Higher values will remove more masks, in conjunction with stability score offset that controls level of stability measured.
Stability score offset	stability_score_offset	1.0	Amount to shift the cutoff for stability score. Higher values will remove less stable masks.
Box NMS IoU threshold	box_nms_thresh	0.7	The IoU threshold (range [0,1]) for non-maximum suppression of boxes. Lower values will merge more masks, useful to reduce mask 'halos' and speed-up postprocessing, but could lead to undersegmentation/agglomeration.
Crop N layers	crop_n_layers	0	Values >0 will crop the image into N layers, which will be processed separately, giving more detail but taking longer. Each deeper layer will have more crops (2^N). Balance number of point prompts with 'Crop N points downscale factor'.
Crop NMS IoU threshold	crop_nms_thresh	0.7	Same as Box NMS IoU threshold, but for each crop.
Crop overlap ratio	crop_overlap_ratio	0.34133	The amount which crops overlap. Higher values may help identify more objects, but duplicates computation.
Crop N points downscale factor	crop_n_points_downscale_factor	1	Value to be raised to the power of the crop layer will be used to scale down number of points per side in each crop. Higher values will reduce point grid density in deeper layers.
Min mask region area	min_mask_region_area	0	Size (in pixels) of masks to remove if identified as a disconnected region or hole. 0 will not remove any masks.
Use previous mask refinement	use_m2m	False	Tick to use one-step refinement using previous mask predictions.
Multimask output	multimask_output	False	Tick to output multiple masks at each grid point prompt.

StarDist

Run with --model stardist.

StarDist is a deep learning method for star-convex object detection in 2D and 3D microscopy images, particularly suited for cell and nucleus segmentation.

Documentation · Repository · Cell Detection with Star-convex Polygons · Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy · Nuclei Instance Segmentation and Classification in Histopathology Images with Stardist

Version	--model_type	Tasks	Axes	Availability
2D_versatile_fluo	2d_versatile_fluo	Nuclei	YX	Public
2D_versatile_he	2d_versatile_he	Nuclei	YXC	Public
2D_paper_dsb2018	2d_paper_dsb2018	Nuclei	YX	Public
2D_demo	2d_demo	Nuclei	YX	Public
3D_demo	3d_demo	Nuclei	ZYX	Public

Parameters

Parameter	Argument	Default	Description
Probability Threshold	prob_thresh	None (float)	Threshold for pixel-wise object probability mask. Higher values lead to fewer detected objects with higher confidence.
NMS Threshold	nms_thresh	None (float)	Intersection over Union (IoU) threshold for non-maximum suppression. Higher values allow for more overlapping objects.
Normalize Min Percentile	normalize_pmin	1	Lower percentile for image normalization. Images are normalized to this percentile value before prediction.
Normalize Max Percentile	normalize_pmax	99.8	Upper percentile for image normalization. Images are normalized to this percentile value before prediction.
Number of Tiles	n_tiles	None (tuple)	Number of tiles for tiled prediction to reduce memory consumption for large images. Specify as tuple (e.g., (2, 2) for 2x2 tiling).
Normalize Image	normalize_img	True	Whether to normalize the input image before prediction. Recommended for most cases.
Scale	scale	None (float)	Optional scaling factor for the input image. Can be used to match the resolution of training data.
Channel Index	channel_idx	0	Select which channel StarDist should use. -1=use original image as-is, 0=first channel, 1=second channel, etc. Ignored for model variants that consume multi-channel images directly.

Generated from aiod_registry 0.2.0.