quick fix, adding cfg.progressBar logic, fixed another issue with the…

… loading bar. (#821)

* Updated logic, bug fix

cfg.progressBar = 2 will display all progress bars
cfg.progressBar = 1 will call tqdm with progress bars w/ leave = 0
cfg.progressBar = 0 will disable the progress bar.

CHANGES.md

@@ -2,7 +2,7 @@
 
 **New features**
 
-- Added progress-bar indicating network creation progress
+- Added progress-bar indicating network creation progress. Toggle the progress bar with cfg.progressBar
 
 - cfg.connRandomSecFromList and cfg.distributeSynsUniformly can now be overriden in individual conn rule
 

doc/source/user_documentation.rst

@@ -2548,7 +2548,7 @@ The code for neural network optimization through evolutionary algorithm used in
 .. --------------------
 
 Running a Batch Job (Beta)
-===================
+
 
 The NetPyNE batchtools subpackage provides a method of automating job submission and reporting::
 
@@ -2571,7 +2571,7 @@ The NetPyNE batchtools subpackage provides a method of automating job submission
 
 
 1. Setting up batchtools
------
+
 Beyond the necessary dependency installations for NetPyNE and NEURON, several additional `pip` installations are required.
 
 The NetPyNE installation should be handled as a development installation of the repository branch `batch`::
@@ -2596,13 +2596,13 @@ Ray is a dependency for batchtools, and should be installed with the following c
     pip install -u ray[default]
 
 2. Examples
------
+
 Examples of NetPyNE batchtools usage can be found in the ``examples`` directory `here <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples>`_.
 
 Examples of the underlying batchtk package can be in the ``examples`` directory `here <https://github.com/jchen6727/batchtk/tree/release/examples>`_.
 
 3. Retrieving batch configuration values through the ``specs`` object
------
+
 Each simulation is able to retrieve relevant configurations through the ``specs`` object, and communicate with
 the dispatcher through the ``comm`` object.
 
@@ -2636,7 +2636,6 @@ This replaces the previous idiom for updating the SimConfig object with mappings
 
 
 4. Communicating results to the ``dispatcher`` with the ``comm`` object
------
 
 Prior batched simulations relied on ``.pkl`` files to communicate data. The ``netpyne.batch`` subpackage uses a specific ``comm`` object to send custom data back
 The ``comm`` object determines the method of communication based on the batch job submission type.
@@ -2651,7 +2650,7 @@ In terms of the simulation, the following functions are available to the user:
 * **comm.close()**: closes and cleans up the connection with the batch ``dispatcher``
 
 5. Specifying a batch job
------
+
 Batch job handling is implemented with methods from ``netpyne.batchtools.search``
 
 **search**::
@@ -2787,7 +2786,7 @@ The basic search implemented with the ``search`` function uses ``ray.tune`` as t
 * **algorithm_config**: additional configuration for the search algorithm (see the `optuna docs <https://docs.ray.io/en/latest/tune/api/suggestion.html>`_)
 
 6. Performing parameter optimization searches (CA3 example)
------
+
 The ``examples`` directory `here <https://github.com/suny-downstate-medical-center/netpyne/tree/batch/netpyne/batchtools/examples>`_ shows both a ``grid`` based search as well as an ``optuna`` based optimization.
 
 In the ``CA3`` example, we tune the ``PYR->BC`` ``NMDA`` and ``AMPA`` synaptic weights, as well as the ``BC->PYR`` ``GABA`` synaptic weight. Note the search space is defined::

netpyne/network/conn.py

@@ -406,9 +406,10 @@ def fullConn(self, preCellsTags, postCellsTags, connParam):
 
     if sim.cfg.verbose:
         print('Generating set of all-to-all connections (rule: %s) ...' % (connParam['label']))
-    if sim.rank == 0 and not sim.cfg.verbose: pbar = tqdm(total=len(postCellsTags.items()), ascii=True,
-                                  desc='  ' + connParam['label'], position=0, leave=True,
-                                  bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} postsynaptic cells on node %i (all-to-all connectivity)' % sim.rank)
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar:
+        pbar = tqdm(total=len(postCellsTags.items()), ascii=True,
+                    desc='  ' + connParam['label'], position=0, leave=True,
+                    bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} postsynaptic cells on node %i (all-to-all connectivity)' % sim.rank)
 
     # get list of params that have a lambda function
     paramsStrFunc = [param for param in [p + 'Func' for p in self.connStringFuncParams] if param in connParam]
@@ -427,11 +428,11 @@ def fullConn(self, preCellsTags, postCellsTags, connParam):
         }
 
     for postCellGid in postCellsTags:  # for each postsyn cell
-        if sim.rank == 0 and not sim.cfg.verbose: pbar.update(1)
+        if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.update(1)
         if postCellGid in self.gid2lid:  # check if postsyn is in this node's list of gids
             for preCellGid, preCellTags in preCellsTags.items():  # for each presyn cell
                 self._addCellConn(connParam, preCellGid, postCellGid, preCellsTags)  # add connection
-    if sim.rank == 0 and not sim.cfg.verbose: pbar.close()
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.close()
 
 
 # -----------------------------------------------------------------------------
@@ -510,9 +511,11 @@ def probConn(self, preCellsTags, postCellsTags, connParam):
 
     if sim.cfg.verbose:
         print('Generating set of probabilistic connections (rule: %s) ...' % (connParam['label']))
-    if sim.rank == 0 and not sim.cfg.verbose: pbar = tqdm(total=len(postCellsTags.items()), ascii=True,
-                                  desc='  ' + str(connParam['label']), position=0, leave=True,
-                                  bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} postsynaptic cells on node %i (probabilistic connectivity)' % sim.rank)
+
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar:
+        pbar = tqdm(total=len(postCellsTags.items()), ascii=True,
+                    desc='  ' + str(connParam['label']), position=0, leave=(sim.cfg.progressBar == 2),
+                    bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} postsynaptic cells on node %i (probabilistic connectivity)' % sim.rank)
 
     allRands = self.generateRandsPrePost(preCellsTags, postCellsTags)
 
@@ -546,22 +549,22 @@ def probConn(self, preCellsTags, postCellsTags, connParam):
             probMatrix, allRands, connParam['disynapticBias'], prePreGids, postPreGids
         )
         for preCellGid, postCellGid in connGids:
-            if sim.rank == 0 and not sim.cfg.verbose: pbar.update(1)
+            if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.update(1)
             for paramStrFunc in paramsStrFunc:  # call lambda functions to get weight func args
                 connParam[paramStrFunc + 'Args'] = {
                     k: v if isinstance(v, Number) else v(preCellsTags[preCellGid], postCellsTags[postCellGid])
                     for k, v in connParam[paramStrFunc + 'Vars'].items()
                 }
             self._addCellConn(connParam, preCellGid, postCellGid, preCellsTags)  # add connection
-        if sim.rank == 0 and not sim.cfg.verbose: pbar.close()
+        if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.close()
     # standard probabilistic conenctions
     else:
         # print('rank %d'%(sim.rank))
         # print(connParam)
         # calculate the conn preGids of the each pre and post cell
         # for postCellGid,postCellTags in sorted(postCellsTags.items()):  # for each postsyn cell
         for postCellGid, postCellTags in postCellsTags.items():  # for each postsyn cell  # for each postsyn cell
-            if sim.rank==0: pbar.update(1)
+            if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.update(1)
             if postCellGid in self.gid2lid:  # check if postsyn is in this node
                 for preCellGid, preCellTags in preCellsTags.items():  # for each presyn cell
                     probability = (
@@ -578,7 +581,7 @@ def probConn(self, preCellsTags, postCellsTags, connParam):
                                 )
                             # connParam[paramStrFunc+'Args'] = {k:v if isinstance(v, Number) else v(preCellTags,postCellTags) for k,v in connParam[paramStrFunc+'Vars'].items()}
                         self._addCellConn(connParam, preCellGid, postCellGid, preCellsTags)  # add connection
-        if sim.rank == 0 and not sim.cfg.verbose: pbar.close()
+        if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.close()
 
 # -----------------------------------------------------------------------------
 # Generate random unique integers
@@ -651,9 +654,10 @@ def convConn(self, preCellsTags, postCellsTags, connParam):
 
     if sim.cfg.verbose:
         print('Generating set of convergent connections (rule: %s) ...' % (connParam['label']))
-    if sim.rank == 0 and not sim.cfg.verbose: pbar = tqdm(total=len(postCellsTags.items()), ascii=True,
-                                  desc='  ' + connParam['label'], position=0, leave=True,
-                                  bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} postsynaptic cells on node %i (convergent connectivity)' % sim.rank)
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar:
+        pbar = tqdm(total=len(postCellsTags.items()), ascii=True,
+                    desc='  ' + connParam['label'], position=0, leave=(sim.cfg.progressBar == 2),
+                    bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} postsynaptic cells on node %i (convergent connectivity)' % sim.rank)
 
     # get list of params that have a lambda function
     paramsStrFunc = [param for param in [p + 'Func' for p in self.connStringFuncParams] if param in connParam]
@@ -673,7 +677,7 @@ def convConn(self, preCellsTags, postCellsTags, connParam):
     hashPreCells = sim.hashList(preCellsTagsKeys)
 
     for postCellGid, postCellTags in postCellsTags.items():  # for each postsyn cell
-        if sim.rank == 0 and not sim.cfg.verbose: pbar.update(1)
+        if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.update(1)
         if postCellGid in self.gid2lid:  # check if postsyn is in this node
             convergence = (
                 connParam['convergenceFunc'][postCellGid]
@@ -706,7 +710,7 @@ def convConn(self, preCellsTags, postCellsTags, connParam):
 
                 if preCellGid != postCellGid:  # if not self-connection
                     self._addCellConn(connParam, preCellGid, postCellGid, preCellsTags)  # add connection
-    if sim.rank == 0 and not sim.cfg.verbose: pbar.close()
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.close()
 
 
 # -----------------------------------------------------------------------------
@@ -739,9 +743,10 @@ def divConn(self, preCellsTags, postCellsTags, connParam):
 
     if sim.cfg.verbose:
         print('Generating set of divergent connections (rule: %s) ...' % (connParam['label']))
-    if sim.rank == 0 and not sim.cfg.verbose: pbar = tqdm(total=len(preCellsTags.items()), ascii=True,
-                                  desc='  ' + connParam['label'], position=0, leave=True,
-                                  bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} presynaptic cells on node %i (divergent connectivity)' % sim.rank)
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar:
+        pbar = tqdm(total=len(preCellsTags.items()), ascii=True,
+                    desc='  ' + connParam['label'], position=0, leave=(sim.cfg.progressBar == 2),
+                    bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} presynaptic cells on node %i (divergent connectivity)' % sim.rank)
 
     # get list of params that have a lambda function
     paramsStrFunc = [param for param in [p + 'Func' for p in self.connStringFuncParams] if param in connParam]
@@ -761,7 +766,7 @@ def divConn(self, preCellsTags, postCellsTags, connParam):
     hashPostCells = sim.hashList(postCellsTagsKeys)
 
     for preCellGid, preCellTags in preCellsTags.items():  # for each presyn cell
-        if sim.rank == 0 and not sim.cfg.verbose: pbar.update(1)
+        if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.update(1)
         divergence = (
             connParam['divergenceFunc'][preCellGid] if 'divergenceFunc' in connParam else connParam['divergence']
         )  # num of presyn conns / postsyn cell
@@ -788,7 +793,7 @@ def divConn(self, preCellsTags, postCellsTags, connParam):
 
             if preCellGid != postCellGid:  # if not self-connection
                 self._addCellConn(connParam, preCellGid, postCellGid, preCellsTags)  # add connection
-    if sim.rank == 0 and not sim.cfg.verbose: pbar.close()
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.close()
 
 
 # -----------------------------------------------------------------------------
@@ -821,10 +826,10 @@ def fromListConn(self, preCellsTags, postCellsTags, connParam):
 
     if sim.cfg.verbose:
         print('Generating set of connections from list (rule: %s) ...' % (connParam['label']))
-    if sim.rank == 0 and not sim.cfg.verbose: pbar = tqdm(total=len(connParam['connList']), ascii=True,
-                                  desc='  ' + connParam['label'], position=0, leave=True,
-                                  bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} pairs of neurons on node %i (from list)' % sim.rank)
-
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar:
+        pbar = tqdm(total=len(connParam['connList']), ascii=True,
+                    desc='  ' + connParam['label'], position=0, leave=(sim.cfg.progressBar == 2),
+                    bar_format= '{l_bar}{bar}| Creating synaptic connections for {n_fmt}/{total_fmt} pairs of neurons on node %i (from list)' % sim.rank)
     orderedPreGids = sorted(preCellsTags)
     orderedPostGids = sorted(postCellsTags)
 
@@ -864,7 +869,7 @@ def fromListConn(self, preCellsTags, postCellsTags, connParam):
         connParam['preSecFromList'] = list(connParam['preSec'])
 
     for iconn, (relativePreId, relativePostId) in enumerate(connParam['connList']):  # for each postsyn cell
-        if sim.rank == 0 and not sim.cfg.verbose: pbar.update(1)
+        if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.update(1)
         preCellGid = orderedPreGids[relativePreId]
         postCellGid = orderedPostGids[relativePostId]
         if postCellGid in self.gid2lid:  # check if postsyn is in this node's list of gids
@@ -885,7 +890,7 @@ def fromListConn(self, preCellsTags, postCellsTags, connParam):
             # TODO: consider cfg.allowSelfConns?
             if preCellGid != postCellGid:  # if not self-connection
                 self._addCellConn(connParam, preCellGid, postCellGid, preCellsTags)  # add connection
-    if sim.rank == 0 and not sim.cfg.verbose: pbar.close()
+    if sim.rank == 0 and not sim.cfg.verbose and sim.cfg.progressBar: pbar.close()
 
 
 # -----------------------------------------------------------------------------

netpyne/specs/simConfig.py

@@ -72,7 +72,7 @@ def __init__(self, simConfigDict=None):
         self.printPopAvgRates = False  # print population avg firing rates after run
         self.printSynsAfterRule = False  # print total of connections after each conn rule is applied
         self.verbose = False  # show detailed messages
-
+        self.progressBar = 2 # (0: no progress bar; 1: progress bar w/ leave = False; 2: progress bar w/ leave = True)
         # Recording
         self.recordCells = []  # what cells to record traces from (eg. 'all', 5, or 'PYR')
         self.recordTraces = {}  # Dict of traces to record