unravel.allen_institute.abca.merfish.merfish_expression_to_nii module

Use abca_merfish_expression_to_nii or me from UNRAVEL to make 3D .nii.gz images of ABCA MERFISH expression data.

Inputs:

• MERFISH expression .h5ad (or use -b to find it in the ABCA download cache)

• MERFISH cell metadata CSV (or use -b to find it in the ABCA download cache)

• Reference .nii.gz for affine/header/shape info (e.g., image_volumes/MERFISH-C57BL6J-638850-CCF/20230630/resampled_annotation.nii.gz)

• Optional: -g <gene> … to select specific genes. If omitted, all genes in the selected MERFISH dataset are processed.

• Optional: -i path/cell_metadata_filtered.csv to use a pre-filtered cell metadata CSV instead of the default ABCA MERFISH metadata.

Outputs:

• <gene>.nii.gz for each requested gene, with the same affine and header as the reference .nii.gz

• Dir: MERFISH[_neuronal|_nonneuronal]_expression_maps/

• Dir with -c and -val: MERFISH[_neuronal|_nonneuronal]_expression_maps_<filter_column>-<filter_value>/

• Dir with -i: MERFISH[_neuronal|_nonneuronal]_expression_maps_<input_stem>/

Notes:

• -w uses threads, not processes, to avoid duplicating the loaded expression matrix.

• Each worker still creates one full 3D output image in memory, so keep -w modest.

Usage:

abca_merfish_expression_to_nii -b <abc_download_root> -r <ref_nii> [-g <gene> …] [-i path/cell_metadata_filtered.csv] [–imputed] [–neurons | –nonneurons] [-o <output>] [-w N] [–dense] [–no-csc] [-dt float32|float64] [-f] [-v]

Usage for one gene:

abca_merfish_expression_to_nii -b <abc_download_root> -g <gene> –no-csc

Usage for selected genes:

abca_merfish_expression_to_nii -b <abc_download_root> -g <gene> <gene> …

Usage for all genes:

abca_merfish_expression_to_nii -b <abc_download_root>

class unravel.allen_institute.abca.merfish.merfish_expression_to_nii.GeneJobResult(gene: 'str', output_path: 'Path', nonzero_cells: 'int', expression_sum: 'float')

Bases: object

Parameters:

• gene (str) –

• output_path (Path) –

• nonzero_cells (int) –

• expression_sum (float) –

gene: str

output_path: Path

nonzero_cells: int

expression_sum: float

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.parse_args()

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.unique_preserve_order(values)

Return unique strings while preserving first occurrence order.

Parameters:

values (Iterable[str]) –

Return type:

list[str]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.get_requested_genes_from_args(args)

Return requested genes and whether the user omitted -g/–gene.

When -g is omitted, use the curated ABCA MERFISH gene-list helpers from merfish.py. These lists let us build output paths and skip existing files before loading the large expression matrix. The final list is still validated against adata.var after AnnData is loaded.

Return type:

tuple[list[str], bool]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.format_gene_preview(genes, max_genes=20)

Return a compact gene-list preview for console output.

Parameters:

• genes (list[str]) –

• max_genes (int) –

Return type:

str

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.find_expression_h5ad(download_base, imputed, verbose=False)

Try to find the ABCA MERFISH expression h5ad without using backed subsetting.

This lets the script bypass mf.load_expression_data(), which may subset a CSRDataset-backed h5ad before loading it into memory. The selected AnnData is opened backed/read-only, then adata.X is loaded fully into memory later.

Parameters:

• download_base (Path) –

• imputed (bool) –

• verbose (bool) –

Return type:

Path | None

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.default_output_path(gene, args, n_requested_genes)

Return the default output path for one gene.

Parameters:

• gene (str) –

• n_requested_genes (int) –

Return type:

Path

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.get_var_symbols(adata)

Return gene symbols aligned to adata.var columns.

Return type:

ndarray

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.get_gene_to_cols(adata, requested_genes)

Map requested gene symbols to integer column positions in adata.X.

Parameters:

requested_genes (list[str]) –

Return type:

dict[str, list[int]]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.load_and_prepare_cell_metadata(download_base, input_path, neurons, nonneurons, filter_column=None, filter_value=None)

Load and optionally filter MERFISH cell metadata.

Parameters:

• download_base (Path) –

• input_path (str | None) –

• neurons (bool) –

• nonneurons (bool) –

• filter_column (str | None) –

• filter_value (str | None) –

Return type:

DataFrame

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.infer_cell_id_column(cell_df, obs_names)

Infer which cell metadata field matches AnnData obs_names best.

The original ABCA metadata generally uses cell labels as the index, but this keeps the script robust if future exports expose them as a column instead.

Parameters:

• cell_df (DataFrame) –

• obs_names (Index) –

Return type:

tuple[Series, str, int]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.align_cells_to_adata(cell_df, adata)

Align cell metadata to AnnData row order without subsetting backed AnnData.

Returns:

• aligned_cell_df (DataFrame) – Metadata rows in the same order as adata.X[row_indices, :].

• row_indices (ndarray[int]) – Integer AnnData row indices to keep.

• id_source (str) – Metadata field used for matching.

Parameters:

cell_df (DataFrame) –

Return type:

tuple[DataFrame, ndarray, str]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.precompute_linear_indices(cell_df, shape, pixel_size_um)

Convert reconstructed MERFISH coordinates to flattened image indices once.

This mirrors mf.points_to_img_sum style accumulation, but does it in 3D:

img[int(x / pixel_size), int(y / pixel_size), z_index] += expression

Parameters:

• cell_df (DataFrame) –

• shape (tuple[int, int, int]) –

• pixel_size_um (float) –

Return type:

tuple[ndarray, ndarray]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.load_X_fully_into_memory(adata)

Load adata.X into memory once, avoiding backed slicing/subsetting.

For AnnData CSRDataset/CSCDataset, to_memory() should return a scipy sparse matrix. Fallbacks cover older anndata/scipy combinations.

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.subset_matrix_after_memory_load(X, row_indices, valid_cell_mask, gene_to_cols, genes_to_process, dense, convert_to_csc)

Subset rows/columns after full in-memory load and return local gene-column mapping.

Parameters:

• row_indices (ndarray) –

• valid_cell_mask (ndarray) –

• gene_to_cols (dict[str, list[int]]) –

• genes_to_process (list[str]) –

• dense (bool) –

• convert_to_csc (bool) –

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.get_gene_values(X, cols, dtype)

Return a dense 1D expression vector for one gene from the working matrix.

Parameters:

• cols (list[int]) –

• dtype (dtype) –

Return type:

ndarray

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.expression_values_to_img(values, linear_idx, shape, dtype)

Accumulate per-cell expression values into a 3D image.

Parameters:

• values (ndarray) –

• linear_idx (ndarray) –

• shape (tuple[int, int, int]) –

• dtype (dtype) –

Return type:

ndarray

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.save_nii(img, affine, header, output_path, dtype)

Save image with reference affine/header and a floating output dtype.

Parameters:

• img (ndarray) –

• affine (ndarray) –

• output_path (Path) –

• dtype (dtype) –

Return type:

None

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.process_gene_job(gene, X, gene_to_cols, linear_idx, shape, dtype, affine, header, output_path)

Build and save one gene image from the shared in-memory expression matrix.

Parameters:

• gene (str) –

• gene_to_cols (dict[str, list[int]]) –

• linear_idx (ndarray) –

• shape (tuple[int, int, int]) –

• dtype (dtype) –

• affine (ndarray) –

• output_path (Path) –

Return type:

GeneJobResult

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.run_gene_jobs(genes_to_process, X, gene_to_cols, linear_idx, shape, dtype, affine, header, output_paths, workers, verbose)

Run gene image construction/saving sequentially or in a thread pool.

Parameters:

• genes_to_process (list[str]) –

• gene_to_cols (dict[str, list[int]]) –

• linear_idx (ndarray) –

• shape (tuple[int, int, int]) –

• dtype (dtype) –

• affine (ndarray) –

• output_paths (dict[str, Path]) –

• workers (int) –

• verbose (bool) –

Return type:

list[GeneJobResult]

unravel.allen_institute.abca.merfish.merfish_expression_to_nii.main()