WARNING: alpha software. We are sharing this early in case it is useful to other groups. Please expect breaking changes. Examples of running the package are found in /Examples
.
GPU accelerated post-processing for 3D MERFISH data generated using qi2lab microscopes. This package currently Linux only due to RAPIDS.AI package availabilty.
Create a python 3.10 environment using your favorite package manager, e.g.
mamba create -n wf-merfish python=3.10
Activate the environment and install the GPU dependencies.
mamba activate wf-merfish
mamba install -c conda-forge -c nvidia -c pytorch -c rapidsai cupy cucim=24.08 cudadecon cuda-version=12.5 cudnn cutensor nccl onnx onnxruntime pytorch torchvision 'pytorch=*=*cuda*'
Finally, repository using git clone https://github.com/QI2lab/wf-merfish
and then install using pip install .
or for interactive editing use pip install -e .
.
file format
- /project_root
- /calibrations.zarr (calibration information)
- .zattrs
- <exp_codebook> (with blank codes)
- <exp_order> (e.g. round 0 -> codebook bits 0,1)
- (objective NA, channels used, etc...)
- <camera_noise_map> (used for hotpixel correction)
- <psf_data> (calculated based on experiment metadata)
- .zattrs
- /polyDT (raw and processed image data for polyDT label)
- /tile000
- /round0000.zarr
- .zattrs
- <stage_zyx_um> (z,y,x in microns)
- <wavelengths_um> (excitation, emission)
- <voxel_size_zyx_um> (z,y,x in microns)
- <bit_linker> (what bits are linked to this polyDT image)
- <affine_zyx_um> (affine transform for world coordinate warping)
- <origin_zyx_um> (origin rigid translation transform for world coordinate warping)
- <spacing_zyx_um> (spacing scale transform for world coordinate warping)
- <camera_data> (optional: raw camera data)
- <corrected_data> (hot pixel, offset, and gain corrected data)
- <registered_decon_data> (deconvolved data)
- .zattrs
- /round0001.zarr
- .zattrs
- <stage_zyx_um> (z,y,x in microns)
- (excitation, emission)
- <voxel_size_zyx_um> (z,y,x in microns)
- <bit_linker> (what bits are linked to this polyDT image)
- <rigid_xyz_px> (rigid registration for round 0 alignment in pixels)
- <camera_data> (optional: raw camera data)
- <corrected_data> (hot pixel, offset, and gain corrected data)
- <of_xyz_3x_downsample> (optical flow field for round 0 alginment in pixels)
- <registered_decon_data> (deconvolved, then rigid and deformable registration applied to warp back to round 0)
- .zattrs
- /roundNNNN.zarr
- /round0000.zarr
- /tile001
- ...
- /tileNNN
- /tile000
- /readouts (raw and processed image data for readout bits)
- /tile0000
- /bit00.zarr
- .zattrs
- <wavelengths_um> (excitation, emission)
- <voxel_size_zyx_um> (z,y,x in microns)
- <round_linker> (what polyDT round is bit corresponds to)
- <camera_data> (optional: raw camera data)
- <corrected_data> (hot pixel, offset, and gain corrected data)
- <registered_corrected_data> (corrected, then rigid and defromable registration applied to warp back to round 0)
- <registered_decon_data> (deconvolved, then rigid and deformable registration applied to warp back to round 0)
- <registered_ufish_data> (ufish prediction applied to registered_decon_data)
- .zattrs
- /bit01.zarr
- ...
- /bitNN.zarr
- /bit00.zarr
- /tile0000
- /ufish_localizations (ufish spot predictions, can be useful for diagnostics)
- /tile0000
- bit01.parquet
- bitNN.parquet
- /tile0001
- ....
- /tileNNNN
- /tile0000
- /decoded (decoding results)
- tile0000_decoded_features.parquet
- ...
- tileNNNN_decoded_features.parquet
- all_tiles_filtered_decoded_features.parquet
- /decoded_optimization (temporary files used during iterative optimizing for normalization factors)
- tileNNNN_decoded_features.parquet
- /fused (fused, down-sampled polyDT image)
- /fused.zarr
- Either one of:
- <fused_iso_all_zyx> (polyDT and readouts deconvolved, registered data fused at isotropic voxel spacing)
- <fused_iso_polyDT_zyx> (polyDT deconvolved, registered data fused at isotropic voxel spacing)
- Either one of:
- /fused.zarr
- /segmentation (segmentation and spots-to-cell assignment results)
- /cellpose
- cell_centroids.parquet (yx centroid for each cell in maximum Z projection of cellpose mask prediction)
- cell_outlines.json (yx polygons for each cell in maximum Z projection of cellpose mask prediction)
- /cellpose.zarr
- <masks_iso_zyx> (cellpose masks generated from maximum Z projection data in fused.zarr)
- /baysor
- baysor_filtered_genes.parquet (gene ID, global zyx position, cell assignment, and confidence for all tiles)
- segmentation_polygons.json (yx polygons for each cell determined by Baysor. GeoJSON format.)
- diagnostic outputs (see Baysor repository for explanations)
- /cellpose
- /mtx_output (mtx formatted output of genes assigned to cells)
- /calibrations.zarr (calibration information)