Merge with devleoper branch (#2)

* add get_nd2_list method

* remove comment

* support pipelining nd2 files

* support pipelining nd2 files

* fix docstring and whitespaces

* bug fix

* final touch

* Modified README to contain updated suite2p installation instructions for mac Ventura OS. Addresses issue MouseLand#897.

* Fixed issue MouseLand#908 regarding badframes for register.py. Detection module was fixed to properly bin movies AFTEr taking into account bad frames.

* Fix blocks issue for nonrigid registration in MouseLand#907

* Update test_and_deploy.yml

* Removed whitespace from display xauthority line in tox.ini.

* Update test_and_deploy.yml

Add apt-get update for linux runner and remove dvc requirement.

* Update test_and_deploy.yml

Added py import to workflow file and added debugging only during job failures

* Update test_and_deploy.yml

* Update tox.ini

* Update test_and_deploy.yml

Remove the up-term step.

* Updated TwophotonSeries initialization for save_nwb in io

* Bump torch from 1.11.0 to 1.13.1

Bumps [torch](https://github.com/pytorch/pytorch) from 1.11.0 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](pytorch/pytorch@v1.11.0...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update issue templates

* Update installation-issues.yml for issues

* Rename installation-issues.yml to installation-issue.yml

* Update installation issue.yml

* Update bug_report.yml

* Added bug_report.yml issue template

* Update all issue templates

* Update feature_request issue

* Update feature_request.yml

* Fixes issue MouseLand#914 for saving news ops parameters

* Update documentation_issue.yml

* Update documentation_issue.yml

* Fixes multiplane issue MouseLand#905 with save_nwb

* Fixes issue MouseLand#904 for 2 channel manual labeling

* Update .readthedocs.yml

* remove spaces

* conform to function signature and usage pattern a few lines above

* add nd2 in requirements

* minor revision

* new logic

* refactor dim sorting

* revert to previous code

* Fixes the from_stat_dict bug mentioned in MouseLand#935.

* Update run suite2p jupyter notebook to fix issue MouseLand#935

* Addresses MouseLand#930 in which bin_data has trouble dealing with cases in which the current batch has fewer frames than the bin_size

* removed ops assignment

* removed deprecated np.bool

* Update the version of the dependencies for suite2p

* Update np.bool to 'bool' to account for numpy deprecation with latest version

* Add Sparse detection to gui as a setting for running suite2p

* Addresses MouseLand#951 and adds nd2 as an input to gui

* Fix issue with np.asanyarray being changed in numpy 1.24.4

* addressed IndexError

* Update README.md

* Update README.md

* fixing skip torch if version 2.0

* adding pep8 style to repo

* fixing dictionary splitting

* modifying imports

* converting to "" from ''

* converting binaryfile to memmap

* rewriting binarycombined

* removing scanimage-tiff-reader for now

* fixing dragging in gui and linewidth=88

* removing old testing yml

* adding py39, updating readme

* adding option for MouseLand#955

* fixing bug in docstring  (MouseLand#902)

* adding iplane to saved manual ROIs (MouseLand#789)

* fixing MouseLand#747 saving to mat

* making nd2 and scanimage-tiff-reader optional (MouseLand#950)

* removing bruker dependency on scanimage-tiff-reader

* Create codecov.yml

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update BinaryRWFile name usage to BinaryFile

* Add instructions on how to use test_data generation script

* Added instructions on generating new test data

* Add method for writing tiff for binaryFile class

* Update write_tiff for binary.py to crop frames, x, and y too

* fix frame range issue with write_Tiff for binary

* added debugging logs

* removed debugging logs

* removed print statement

* removed unnecessary space

* Fixes MouseLand#943 to address documentation issue with high_pass

* adding copyright to every py file

* Delete .dvc directory

* fixing import error with h5py and paramiko MouseLand#987

* Update README.md

* Add pynwb package to setup.py as tests needs pynwb for one of the io_tests

* Update readme.md to be for python version 3.9

* Addresses MouseLand#944  leading zero issue with tiff filename.

* Addresses MouseLand#944  leading zero issue with tiff filename.

* Update save_path0 to close MouseLand#991.

* Update save_path0 to close MouseLand#991.

* update visualize.py to import rastermap properly

* Re-add connected to GUI options

* Update README.md

Remove note about scanimage-tiff-reader mac incompatibility

* Update README.md

Simplify mac os instructions

* Update setup.py

Update rastermap requirement based on issue ticket suggestions

* Update setup.py

* Updated jupyter notebook to work with new binaryFile class in suite2p.io

* Updated jupyter notebook name

* Update colab 2023 to fully fix MouseLand#1011 and update some comments on notebook

* Updated jupyter notebook to work with new binaryFile class in suite2p.io

* Updated jupyter notebook name

* Update colab 2023 to fully fix MouseLand#1011 and update some comments on notebook

* adding pyqt6 and switching to qtpy

* updating readme and fixing spelling error in copyright

* adding license notice

* adding check in setup.py for installation of a pyqt version

* adding tutorial

* Update tutorial.md with additional comments

* Update suite2p gui to work with QFileDialog.Options() not working with pyqt6

* Update binary.py write_tiff to add all frames to a single series

* Updated tiff.py to use contiguous parameter

* Update README.md

* checking h5py key to open binaries before data_path

* fixing bug in code

* adding new data type

* adding support for binary and mp4/avi files

* enabling ops.json loading in reggui and binary-only processing

* increase tolerance on classification regression test

* add more tolerance

* increasing tolerance again

* increasing tolerance again

* Update README.md

* Add explanation for how to apply classifiers

* CheckState Fix

* Fix conda environment creation for developers

* Update utils.py

* Update README.md to fix MouseLand#1060.

Updates badge links for tests and docs

* Update merge.py of gui subpackage to not use deprecated np.bool

* Update merge.py of gui subpackage to not use deprecated np.bool

* Add import to gui/io.py

* Update gui.rst to have clearer instructions for MouseLand#1002.

* bug in h5py key option

* adding support for multipage bruker ome tiffs

* Update inputs.rst to include multipage bruker instructions

* Update inputs.rst

* Added support for Thorlabs RAW files (io.raw)

* Added xmltodict

* Update raw.py- todo verified

* safe import for xmltodict

* fixing bug with multiplane registration

* allowing reg_file input to compute_zpos

* fixing bug with multi-plane nonrigid ops

* add support for Hamamatsu DCIMG format

* add support for Hamamatsu DCIMG format

* add dcimg_deps to all_deps and restrict cellpose to <3.0

* Addresses MouseLand#1093 with cellpose method having too many arguments

* Addresses MouseLand#1093 with cellpose method having too many arguments

* remove cellpose<3.0 restriction

* Fixes issue MouseLand#1101 with older version of cellpose

* Update zalign.py

Made sure that if a reg_file is given to the zalign function it uses it, rather than the ops reg_file

* Update setup.py

* adding dcimg

* removing dcimg extra

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: tdincer <[email protected]>
Co-authored-by: Chris Ki <[email protected]>
Co-authored-by: Chris Ki <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: carsen-stringer <[email protected]>
Co-authored-by: Kabilar Gunalan <[email protected]>
Co-authored-by: itsb <[email protected]>
Co-authored-by: Sidharth Hulyalkar <[email protected]>
Co-authored-by: Ahmed Morsi <[email protected]>
Co-authored-by: Marius Pachitariu <[email protected]>
Co-authored-by: Heather Ratigan <[email protected]>
Co-authored-by: Ryan Ly <[email protected]>
Co-authored-by: Arielle Leon <[email protected]>
Co-authored-by: Andrew Landau <[email protected]>
Co-authored-by: arielleleon <[email protected]>
Co-authored-by: yaelpri <[email protected]>
Co-authored-by: yaelpri <[email protected]>
Co-authored-by: Yael Prilutski <[email protected]>
Co-authored-by: nguyemi5 <[email protected]>
Co-authored-by: nguyemi5 <[email protected]>
Co-authored-by: liadJB <[email protected]>

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,60 @@
+name: Bug report
+description: Report a bug.
+title: "BUG: <Please write a comprehensive title after the 'BUG: ' prefix>"
+
+body:
+- type: markdown
+  attributes:
+    value: >
+      Thank you for taking the time to file a bug report. Before creating a new
+      issue, please make sure to take a few minutes to check if this issue has been 
+      brought up before.
+    
+- type: textarea
+  attributes: 
+    label: "Describe the issue:"
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Reproduce the code example:"
+    description: >
+      A short code example that reproduces the problem/missing feature. It
+      should be self-contained, i.e., can be copy-pasted into the Python
+      interpreter or run as-is via `python myproblem.py`. Please include as much 
+      detail you can about the ops.npy that was used.
+    placeholder: |
+      import suite2p
+      << your code here >>
+    render: python
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Error message:"
+    description: >
+      Please include full error message, if any.
+    placeholder: |
+      << Full traceback starting from `Traceback: ...` >>
+    render: shell
+
+- type: textarea
+  attributes:
+    label: "Version information:"
+    description: >
+      Output from running `suite2p --version` in your command line.
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Context for the issue:"
+    description: |
+      Please explain how this issue affects your work or why it should be prioritized.
+    placeholder: |
+      << your explanation here >>
+  validations:
+    required: false
+

.github/ISSUE_TEMPLATE/documentation_issue.yml

@@ -0,0 +1,22 @@
+name: Documentation
+description: Report an issue related to suite2p documentation.
+title: "DOC: <Please write a comprehensive title after the 'DOC: ' prefix>"
+
+body:
+- type: textarea
+  attributes: 
+    label: "Issue with current documentation:"
+    description: >
+      Please make sure to leave a reference to the document/code you're
+      referring to. Please report where in https://suite2p.readthedocs.io/ you see this issue. 
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Idea or request for content:"
+    description: >
+      Please describe as clearly as possible what topics you think are missing
+      from the current documentation. Make sure to check
+      https://colab.research.google.com/github/MouseLand/suite2p/blob/main/jupyter/run_suite2p_colab_2021.ipynb 
+      and see if this documentation should be added there.

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,27 @@
+name: Feature request
+description: Suggest an additional feature you'd like to see in suite2p.
+title: "FEATURE: <Please write a comprehensive title after the 'FEATURE: ' prefix>"
+
+body:
+- type: textarea
+  attributes: 
+    label: "Feature you'd like to see:"
+    description: >
+        Provide a clear and concise description of what problem you'd like this feature to address. 
+        Then, provide a clear description of the solution you'd like to see.
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Attempted alternative approaches:"
+    description: >
+       Provide a description of alternative approaches you've tried. 
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Additional Context"
+    description: >
+        Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/installation_issue.yml

@@ -0,0 +1,22 @@
+name: Installation issue
+description: Report an issue with installation for suite2p.
+title: "<Please write a comprehensive title.>"
+
+body:
+
+- type: textarea
+  attributes: 
+    label: "Describe the issue:"
+    description: >
+      Let us know what issues you are having with installation.
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: "Provide environment info:"
+    description: >
+      Please run `conda info` in your suite2p environment in your terminal/anaconda prompt to let us know the versions of your packages.
+  validations:
+    required: true
+

.github/workflows/test_and_deploy.yml

@@ -22,7 +22,7 @@ jobs:
       fail-fast: false
       matrix:
         platform: [ubuntu-latest, windows-latest, macos-latest]
-        python-version: [3.8]
+        python-version: [3.8, 3.9]
 
     steps:
       - uses: actions/checkout@v2
@@ -32,14 +32,17 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
 
+
       # these libraries, along with pytest-xvfb (added in the `deps` in tox.ini),
       # enable testing on Qt on linux
       - name: Install Linux libraries
         if: runner.os == 'Linux'
         run: |
+          sudo apt-get update
           sudo apt-get install -y libdbus-1-3 libxkbcommon-x11-0 libxcb-icccm4 \
             libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 \
-            libxcb-xinerama0 libxcb-xinput0 libxcb-xfixes0 pkg-config libhdf5-103 libhdf5-dev
+            libxcb-xinerama0 libxcb-xinput0 libxcb-xfixes0 pkg-config libhdf5-103 libhdf5-dev \
+            libegl1
       # strategy borrowed from vispy for installing opengl libs on windows
       - name: Install Windows OpenGL
         if: runner.os == 'Windows'
@@ -54,25 +57,24 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install wheel setuptools tox tox-gh-actions
-          pip install dvc==1.11.0 pydrive2
-      # For debugging purposes, allows one to ssh into host machine. 
-      # Follow instructions in  https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account 
-      # to add your ssh keys.
-      # MAKE SURE TO COMMENT OUT IF NOT DEBUGGING!
-#       - name: Setup upterm session 
-#         if: runner.os == 'macOS'
-#         uses: lhotari/action-upterm@v1
-#         with:
-#           ## limits ssh access and adds the ssh public key for the user which triggered the workflow
-#           limit-access-to-actor: true
-#           ## limits ssh access and adds the ssh public keys of the listed GitHub users
-#           limit-access-to-users: chriski777, carsen-stringer
+          pip install pydrive2 py # Added py due to pytest tox issues requiring py module.
 
       - name: Test with tox
         run: tox
         env:
-          PLATFORM: ${{ matrix.platform }}
-
+          PLATFORM: ${{ matrix.platform }}    
+      # ONLY UNCOMMENT SECTION BELOW FOR DEBUGGING PURPOSES: allows one to ssh into host machine. 
+      # Follow instructions in  https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account 
+      # to add your ssh keys.
+#      - name: Job failed. Activating debugging mode via up-term.
+#        if: ${{ failure() }}
+#        uses: lhotari/action-upterm@v1
+#        with:
+#           ## limits ssh access and adds the ssh public key for the user which triggered the workflow
+#            limit-access-to-actor: true
+#           ## limits ssh access and adds the ssh public keys of the listed GitHub users
+#            limit-access-to-users: chriski777, carsen-stringera
+
       - name: Coverage
         # Only run coverage once
         if: runner.os == 'Linux'

.readthedocs.yml

@@ -18,7 +18,7 @@ formats:
 
 # specify dependencies
 python:
-   version: 3.7
+   version: 3.8
    install:
       - method: pip
         path: .

.style.yapf

@@ -0,0 +1,4 @@
+[style]
+based_on_style = google
+split_before_named_assigns = false
+column_limit = 88