multiprocessing_ndfilter

grizli.utils.multiprocessing_ndfilter(data, filter_func, filter_args=(), size=None, footprint=None, cutout_size=256, n_proc=4, timeout=90, mask=None, verbose=True, **kwargs)

Cut up a large array and send slices to scipy.ndimage filters

Parameters

dataarray-like

Main image array

filter_funcfunction

Filtering function, e.g., scipy.ndimage.median_filter

filter_argstuple

Arguments to pass to filter_func

size, footprintint, array-like

Filter size or footprint, see, e.g., scipy.ndimage.median_filter

cutout_sizeint

Size of subimage cutouts

n_procint

Number of multiprocessing processes to use

timeoutfloat

multiprocessing timeout (seconds)

maskarray-like

Array multiplied to data that can zero-out regions to ignore

verbosebool

Print status messages

kwargsdict

Keyword arguments passed through to filter_func

Returns

filteredarray-like

Filtered version of data

Examples

>>> import time
>>> import numpy as np
>>> import scipy.ndimage as nd
>>> from grizli.utils import multiprocessing_ndfilter
>>> rnd = np.random.normal(size=(512,512))
>>> t0 = time.time()
>>> f_serial = nd.median_filter(rnd, size=10)
>>> t1 = time.time()
>>> f_mp = multiprocessing_ndfilter(rnd, nd.median_filter, size=10,
>>>                                 cutout_size=256, n_proc=4)
>>> t2 = time.time()
>>> np.allclose(f_serial, f_mp)
True
>>> print(f'  serial: {(t1-t0)*1000:.1f} ms')
>>> print(f'parallel: {(t2-t1)*1000:.1f} ms')
  serial: 573.9 ms
parallel: 214.8 ms