0.4 El pirata

Handling large datasets and local feature equalization

In case something is not working perfectly, consider using the updated demo files:

https://github.com/agiovann/Constrained_NMF/blob/master/demo.py
https://github.com/agiovann/Constrained_NMF/blob/master/demo_patches.py
https://github.com/agiovann/Constrained_NMF/blob/master/demoCNMF.ipynb

New features

• The main new feature with this release is the ability to handle large datasets through memory mapping and partitioning the FOV in different spatially overlapping patches that are processed in parallel. For more details please read here. There is a new demo file (demo_patches.py) that illustrates the use of such tool. If you need also motion correction please check this repository and/or contact me.
• added travis/docker support (experimental)

Important functions that have been introduced are:

save_memmap: Save a tiff stack file as a matlab file that can be memory mapped. If the dataset is too large to load in memory, it can be read/saved in pieces. And of course the counterpart load_memmap.py. One can also decide to downsample and/or use portion of each of the files on the fly, thus allowing easily to have quick approximation of data analysis of very large files.

Another new feature is an unsupervised classifier that orders the component according to a metric that counts fluctuations in the traces that are statistically exceptional given an estimate of baseline and noise. Exceptional events are assumed to be spikes. The function implementing it is utilities.evaluate_components.

Deprecated functions

• The SPGL module has been deprecated and removed from the repository

Bug fixing

• the number of processes is decided at a single point in the code now
• the number of processes has been decreased because it seems to affect performance when more than 80% of cores where in use
• now the pfft is computed only on part of the data to speed up computation