shinestacker

Alignment and registration

Automatically align, scale, tanslate and rotate or apply full perspective correction.

All constants described below are contained in the object constants, imported as:

from shinestacker.config.constants import constants

Alignment actions can be added to a job as follows:

job.add_action(Actions("align", [AlignFrames(*options)])

Arguments for the constructor AlignFrames of are:

• feature_config (optional, default: None): a dictionary specifying the following parameters, with the corresponding default values:

  feature_config = {
    'detector': constants.DETECTOR_SIFT,
    'descriptor': constants.DESCRIPTOR_SIFT
  }
  

• name (optional, default: empty): only used in the GUI as identifier
• enabled (optiona, default: True): enable/disable sub-action
• detector (optional, default: DETECTOR_SIFT): the feature detector is used to find matches. See Feature Detection and Description for more details. Possible values are:
  • DETECTOR_SIFT (default): Scale-Invariant Feature Transform]
  • DETECTOR_ORB: Oriented FAST and Rotated BRIEF
  • DETECTOR_SURF: Speeded-Up Robust Features
  • DETECTOR_AKAZE: AKAZE local features matching
  • DETECTOR_BRISK: Binary Robust Invariant Scalable Keypoints
• descriptor (optional, default: DESCRIPTOR_SIFT): the feature descriptor is used to find matches. Possible values are:
  • DESCRIPTOR_SIFT (default)
  • DESCRIPTOR_ORB
  • DESCRIPTOR_AKAZE
  • DESCRIPTPR_BRISK

  For a more quantitative comparison of performances of the different methods, consult the publication: S. A. K. Tareen and Z. Saleem, “A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK”, doi:10.1109/ICOMET.2018.8346440

matching_config (optional, default; None): a dictionary specifying the following parameters, with the corresponding default values:

matching_config= {
    'match_method': constants.MATCHING_KNN,
    'flann_idx_kdtree': 2,
    'flann_trees': 5,
    'flann_checks': 50,
    'threshold': 0.75
}

• match_method (optional, default: MATCHING_KNN): the method used to find matches. See Feature Matching for more details. Possible values are:
  • MATCHING_KNN (default): Feature Matching with FLANN
  • MATCHING_NORM_HAMMING: Use Hamming distance
• flann_idx_kdtree (optional, default: 2): parameter used by the FLANN matching algorithm.
• flann_tree (optional, default: 5): parameter used by the FLANN matching algorithm.
• flann_checks (optional, default: 50): parameter used by the FLANN matching algorithm.

• threshold (optional, default: 0.75): parameter used to select good matches. See Feature Matching for more details.

• alignment_config (optional, default; None): a dictionary specifying the following parameters, with the corresponding default values:

  alignment_config = {
    'transform': constants.ALIGN_RIGID,
    'align_methid': constants.RANSAC,
    'rans_threshold': 5.0,
    'refine_iters': 100,
    'align_confidence': 99.9,
    'max_iters': 2000,
    'border_mode': constants.BORDER_REPLICATE_BLUR,
    'border_value': (0, 0, 0, 0),
    'border_blur': 50,
    'subsample': 0,
    'fast_subsampling': False,
    'min_good_matches': 100,
    'phase_corr_fallback': False,
    'abort_abnormal': False
  }
  

• transform (optional, default: ALIGN_RIGID): the transformation applied to register images. Possible values are:
  • ALIGN_RIGID (default): allow scale, tanslation and rotation correction. This should be used for image acquired with tripode or microscope.
  • ALIGN_HOMOGRAPHY: allow full perspective correction. This should be used for images taken with hand camera.
• align_method (optional, default: RANSAC): the method used to find matches. Valid options are:
  • RANSAC (Random Sample Consensus, default)
  • LMEDS (Least Medians of Squares)
• rans_threshold (optional, default: 5.0): parameter used if ALIGN_HOMOGRAPHY is choosen as tansformation, see Feature Matching + Homography to find Objects for more details.
• refine_iters (optional, default: 100): refinement iterations. Used only if transform=ALIGN_RIGID.
• align_confidence (optional, default: 99.9): alignment algorithm confidence (%). Used only if transform=ALIGN_RIGID.
• max_iters (optional, default: 2000): maximum number of iterations. Used only if transform=ALIGN_HOMOGRAPHY.
• subsample (optional, default: 0=automatic): subsample image for faster alignment. Faster, but alignment could be less accurate if only a small portion of the image is focused. It can save time, in particular for large images.
• fast_subsampling (optional, default: False): perform fast image subsampling without interpolation. Used if subsample is set to True.
• min_good_matches (optional, default: 100): if subsample>1 and the number of good matches is below min_good_matches, the alignment is retried without subsampling. This improbes robustness in case a too large subsampling factor is specified.
• phase_corr_fallback (optional, default: False): use phase correlation alignment in case the number of matches is too low to allow to estimate the transformation. This algorithm is not very precise, and may help only in case of blurred images.
• abort_abnormal (optional, default: False): abort in case of anormal tansformation. By default, frames where abnormal transformation is found are skipped, and processing continunes to next frame.
• border_mode (optional, default: BORDER_REPLICATE_BLUR): border mode. See Adding borders to your images for more details. Possible values are:
  • BORDER_CONSTANT: pad the image with a constant value. The border value is specified with the parameter border_value.
  • BORDER_REPLICATE: the rows and columns at the very edge of the original are replicated to the extra border.
  • BORDER_REPLICATE_BLUR (default): same as above, but the border is blurred. The amount of blurring is specified by the parameter border_blur.
• border_value (optional, default: (0, 0, 0, 0)): border value. See Adding borders to your images for more details.
• border_blur (optional, default: 50): amount of border blurring, in pixels. Only applied if border_mode is set to BORDER_REPLICATE_BLUR, which is the default option.
• plot_summary (optional, default: False): if True, plot a summary histogram with number of matches in each frame. May be useful for inspection and debugging.
• plot_matches (optional, default: False): if True, for each image matches with reference frame are drawn. May be useful for inspection and debugging.
• enabled (optional, default: True): allows to switch on and off this module.

Parallel processing

A class AlignFramesParallel implements alignment using parallel processing. This class has extra parameters, in addition to the above ones:

• max_threads (optional, default: 2): number of parallel processes allowed. The number of actual threads will not be greater than the number of available CPU cores.
• chunk_submit (optional, default: True): submit at most max_threads parallel processes. If chunk_submit is greater than max_threads a moderate performance gain is achieved at the cost of a possibly large memory occupancy.
• bw_matching (optional, default: False): perform matches on black and white version of the images in order to save memory. Preliminary tests indicate that the gain with this option is marginal, and this option may be dropped in the future.
• delta_max (optional, defaut: 2): maximum consecutive frames skipped in case of too few matches found.

Automatic selection of processing strategy

A class AlignFramesAuto implements alignment with either sequential or parallel processing, and automatically tunes parallel processing parameters. This class has extra parameters, in addition to the above ones:

• mode (optional, default: auto): can be auto, sequential or parallel.
• memory_limit (optional, default: 8×1024³sup>): memory limit to determine optimal running parameters

Allowed configurations

⚠️ Not all combinations of detector, descriptor and match methods are allowed. Combinations that are not allowed give raise to an exception. This is automatically prevented if one works with the GUI, but may occur when using python scripting. Below the table of the allowed combination with a comparison of CPU performances.

CPU performances

⏳ Below the time performances for alignment of two of the sample images with 2000×1300 resolution for allowed configuration combinations, from fastest to slowest. Note that slower may be more precise.

Time (s)	Detector	Descriptor	Match method
0.0250	SURF	ORB	NORM_HAMMING
0.0347	SURF	BRISK	NORM_HAMMING
0.0469	ORB	ORB	NORM_HAMMING
0.0471	ORB	BRISK	NORM_HAMMING
0.1001	BRISK	BRISK	NORM_HAMMING
0.1199	BRISK	ORB	NORM_HAMMING
0.1604	SURF	SIFT	KNN
0.1966	BRISK	SIFT	KNN
0.2121	ORB	SIFT	KNN
0.2738	AKAZE	AKAZE	NORM_HAMMING
0.2863	AKAZE	ORB	NORM_HAMMING
0.2887	AKAZE	BRISK	NORM_HAMMING
0.4075	AKAZE	SIFT	KNN
0.4397	SIFT	SIFT	KNN

References

For a detailed review of the various image registration methods, see the publication below:

• A Review of Keypoints’ Detection and Feature Description in Image Registration, Scientific Programming 2021, 8509164, doi:10.1155/2021/8509164

This site is open source. Improve this page.