Petascale pipeline for precise alignment of images from serial section electron microscopy

Sergiy Popovych, Thomas Macrina, Nico Kemnitz, Manuel Castro, Barak Nehoran, Zhen Jia, J. Alexander Bae, Eric Mitchell, Shang Mu, Eric T. Trautman, Stephan Saalfeld, Kai Li and H. Sebastian Seung ()
Additional contact information
Sergiy Popovych: Princeton University
Thomas Macrina: Princeton University
Nico Kemnitz: Princeton University
Manuel Castro: Princeton University
Barak Nehoran: Princeton University
Zhen Jia: Princeton University
J. Alexander Bae: Princeton University
Eric Mitchell: Princeton University
Shang Mu: Princeton University
Eric T. Trautman: HHMI Janelia Research Campus
Stephan Saalfeld: HHMI Janelia Research Campus
Kai Li: Princeton University
H. Sebastian Seung: Princeton University

Nature Communications, 2024, vol. 15, issue 1, 1-15

Abstract: Abstract The reconstruction of neural circuits from serial section electron microscopy (ssEM) images is being accelerated by automatic image segmentation methods. Segmentation accuracy is often limited by the preceding step of aligning 2D section images to create a 3D image stack. Precise and robust alignment in the presence of image artifacts is challenging, especially as datasets are attaining the petascale. We present a computational pipeline for aligning ssEM images with several key elements. Self-supervised convolutional nets are trained via metric learning to encode and align image pairs, and they are used to initialize iterative fine-tuning of alignment. A procedure called vector voting increases robustness to image artifacts or missing image data. For speedup the series is divided into blocks that are distributed to computational workers for alignment. The blocks are aligned to each other by composing transformations with decay, which achieves a global alignment without resorting to a time-consuming global optimization. We apply our pipeline to a whole fly brain dataset, and show improved accuracy relative to prior state of the art. We also demonstrate that our pipeline scales to a cubic millimeter of mouse visual cortex. Our pipeline is publicly available through two open source Python packages.

Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.nature.com/articles/s41467-023-44354-0 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-023-44354-0

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-023-44354-0

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().