Automatically align, scale, tanslate and rotate or apply full perspective correction.
All constants described below are contained in the object constants, imported as:
from shinestacker.config.constants import constants
Alignment actions can be added to a job as follows:
job.add_action(Actions("align", [AlignFrames(*options)])
Arguments for the constructor AlignFrames of are:
feature_config (optional, default: None): a dictionary specifying the following parameters, with the corresponding default values:
feature_config = {
'detector': constants.DETECTOR_SIFT,
'descriptor': constants.DESCRIPTOR_SIFT
}
name (optional, default: empty): only used in the GUI as identifierenabled (optiona, default: True): enable/disable sub-actiondetector (optional, default: DETECTOR_ORB): the feature detector is used to find matches. See Feature Detection and Description for more details. Possible values are:
DETECTOR_SIFT: Scale-Invariant Feature Transform]DETECTOR_ORB (default): Oriented FAST and Rotated BRIEFDETECTOR_SURF: Speeded-Up Robust FeaturesDETECTOR_AKAZE: AKAZE local features matchingDETECTOR_BRISK: Binary Robust Invariant Scalable Keypointsdescriptor (optional, default: DESCRIPTOR_ORB): the feature descriptor is used to find matches. Possible values are:
DESCRIPTOR_SIFT (default)DESCRIPTOR_ORBDESCRIPTOR_AKAZEDESCRIPTPR_BRISKFor a more quantitative comparison of performances of the different methods, consult the publication: S. A. K. Tareen and Z. Saleem, “A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK”, doi:10.1109/ICOMET.2018.8346440
matching_config (optional, default; None): a dictionary specifying the following parameters, with the corresponding default values:
matching_config= {
'match_method': constants.MATCHING_KNN,
'flann_idx_kdtree': 2,
'flann_trees': 5,
'flann_checks': 50,
'threshold': 0.75
}
match_method (optional, default: MATCHING_KNN): the method used to find matches. See Feature Matching for more details. Possible values are:
MATCHING_KNN (default): Feature Matching with FLANNMATCHING_NORM_HAMMING: Use Hamming distanceflann_idx_kdtree (optional, default: 2): parameter used by the FLANN matching algorithm.flann_tree (optional, default: 5): parameter used by the FLANN matching algorithm.flann_checks (optional, default: 50): parameter used by the FLANN matching algorithm.threshold (optional, default: 0.75): parameter used to select good matches. See Feature Matching for more details.
alignment_config (optional, default; None): a dictionary specifying the following parameters, with the corresponding default values:
alignment_config = {
'transform': constants.ALIGN_RIGID,
'align_methid': constants.RANSAC,
'rans_threshold': 5.0,
'refine_iters': 100,
'align_confidence': 99.9,
'max_iters': 2000,
'border_mode': constants.BORDER_REPLICATE_BLUR,
'border_value': (0, 0, 0, 0),
'border_blur': 50,
'interpolation': constants.INTER_CUBIC,
'subsample': 0,
'fast_subsampling': False,
'min_good_matches': 100,
'phase_corr_fallback': False,
'abort_abnormal': False
}
transform (optional, default: ALIGN_RIGID): the transformation applied to register images. Possible values are:
ALIGN_RIGID (default): allow scale, tanslation and rotation correction. This should be used for image acquired with tripode or microscope.ALIGN_HOMOGRAPHY: allow full perspective correction. This should be used for images taken with hand camera.align_method (optional, default: RANSAC): the method used to find matches. Valid options are:
RANSAC (Random Sample Consensus, default)LMEDS (Least Medians of Squares)rans_threshold (optional, default: 5.0): parameter used if ALIGN_HOMOGRAPHY is choosen as tansformation, see Feature Matching + Homography to find Objects for more details.refine_iters (optional, default: 100): refinement iterations. Used only if transform=ALIGN_RIGID.align_confidence (optional, default: 99.9): alignment algorithm confidence (%). Used only if transform=ALIGN_RIGID.max_iters (optional, default: 2000): maximum number of iterations. Used only if transform=ALIGN_HOMOGRAPHY.subsample (optional, default: 0=automatic): subsample image for faster alignment. Faster, but alignment could be less accurate if only a small portion of the image is focused. It can save time, in particular for large images.resolution_target (optional, default: 2): with subsample=0, the image is subsampled for feature matching to about this many megapixels. Larger values match finer detail and take longer.fast_subsampling (optional, default: False): perform fast image subsampling without interpolation. Used if subsample is set to True.min_good_matches (optional, default: 100): if subsample>1 and the number of good matches is below min_good_matches, the alignment is retried without subsampling. This improbes robustness in case a too large subsampling factor is specified.phase_corr_fallback (optional, default: False): use phase correlation alignment in case the number of matches is too low to allow to estimate the transformation. This algorithm is not very precise, and may help only in case of blurred images.abort_abnormal (optional, default: False): abort in case of anormal tansformation. By default, frames where abnormal transformation is found are skipped, and processing continunes to next frame.border_mode (optional, default: BORDER_REPLICATE_BLUR): border mode. See Adding borders to your images for more details. Possible values are:
BORDER_CONSTANT: pad the image with a constant value. The border value is specified with the parameter border_value.BORDER_REPLICATE: the rows and columns at the very edge of the original are replicated to the extra border.BORDER_REPLICATE_BLUR (default): same as above, but the border is blurred. The amount of blurring is specified by the parameter border_blur.border_value (optional, default: (0, 0, 0, 0)): border value. See Adding borders to your images for more details.border_blur (optional, default: 50): amount of border blurring, in pixels. Only applied if border_mode is set to BORDER_REPLICATE_BLUR, which is the default option.interpolation (optional, default: INTER_CUBIC): interpolation used to warp the frames onto the reference frame. Possible values are:
INTER_LINEAR: bilinear. Fast, but a sub-pixel shift softens fine detail by up to a factor two at the highest frequencies.INTER_CUBIC (default): bicubic. Keeps more fine detail; recommended for focus stacking.INTER_LANCZOS4: Lanczos over 8x8 pixels. Slightly sharper than bicubic and slower.plot_summary (optional, default: False): if True, plot a summary histogram with number of matches in each frame. May be useful for inspection and debugging.plot_matches (optional, default: False): if True, for each image matches with reference frame are drawn. May be useful for inspection and debugging.enabled (optional, default: True): allows to switch on and off this module.Some detector/descriptor combinations — most notably ORB/ORB, which is the default — can occasionally fail to find enough matches, or to estimate a reliable transformation. When this happens, alignment is automatically retried with a more robust configuration (SIFT/SIFT with KNN matching). The fallback is built into the code: it is not configurable and it is not exposed in the GUI.
A fallback attempt is logged as a warning for the affected frame.
If plot_matches is enabled, the match plot of the fallback attempt is saved
with a -fb-sift suffix (e.g. align-matches-0042-fb-sift.png), alongside the
plot of the primary attempt.
If the fallback also fails, the frame is handled as usual: it is skipped, or
recovered by phase correlation if phase_corr_fallback is enabled. No
success/failure message is emitted for the fallback itself: the existing
warnings cover both cases.
A class AlignFramesParallel implements alignment using parallel processing.
This class has extra parameters, in addition to the above ones:
max_threads (optional, default: 2): number of parallel processes allowed. The number of actual threads will not be greater than the number of available CPU cores.chunk_submit (optional, default: True): submit at most max_threads parallel processes. If chunk_submit is greater than max_threads a moderate performance gain is achieved at the cost of a possibly large memory occupancy.bw_matching (optional, default: False): perform matches on black and white version of the images in order to save memory. Preliminary tests indicate that the gain with this option is marginal, and this option may be dropped in the future.delta_max (optional, defaut: 2): maximum consecutive frames skipped in case of too few matches found.A class AlignFramesAuto implements alignment with either sequential or parallel processing, and automatically tunes parallel processing parameters.
This class has extra parameters, in addition to the above ones:
mode (optional, default: auto): can be auto, sequential or parallel.memory_limit (optional, default: 8×10243sup>): memory limit to determine optimal running parameters⚠️ Not all combinations of detector, descriptor and match methods are allowed. Combinations that are not allowed give raise to an exception. This is automatically prevented if one works with the GUI, but may occur when using python scripting. Below the table of the allowed combination with a comparison of CPU performances.
⏳ Below the time performances for alignment of two of the sample images with 2000×1300 resolution for allowed configuration combinations, from fastest to slowest. Note that slower may be more precise.
| Time (s) | Detector | Descriptor | Match method |
|---|---|---|---|
| 0.0250 | SURF | ORB | NORM_HAMMING |
| 0.0347 | SURF | BRISK | NORM_HAMMING |
| 0.0469 | ORB | ORB | NORM_HAMMING |
| 0.0471 | ORB | BRISK | NORM_HAMMING |
| 0.1001 | BRISK | BRISK | NORM_HAMMING |
| 0.1199 | BRISK | ORB | NORM_HAMMING |
| 0.1604 | SURF | SIFT | KNN |
| 0.1966 | BRISK | SIFT | KNN |
| 0.2121 | ORB | SIFT | KNN |
| 0.2738 | AKAZE | AKAZE | NORM_HAMMING |
| 0.2863 | AKAZE | ORB | NORM_HAMMING |
| 0.2887 | AKAZE | BRISK | NORM_HAMMING |
| 0.4075 | AKAZE | SIFT | KNN |
| 0.4397 | SIFT | SIFT | KNN |
For a detailed review of the various image registration methods, see the publication below: