00_bq3d_setup.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BrainQuant3D Setup\n",
    "## 1. Download necessary applications/packages\n",
    "Ensure you have these downloaded on your computer:\n",
    "1. git (for setting up BrainQuant3D as a package): https://git-scm.com/downloads\n",
    "2. anaconda/miniconda (for isolating packages in its own Python environment): https://www.anaconda.com/download\n",
    "\n",
    "Then, open your terminal (mac, linux) or anaconda prompt (windows) and execute these commands:\n",
    "```\n",
    "cd <path to directory where this notebook and the .yml file is in> #e.g. cd mnt/ssd\n",
    "conda env create -f bq3d_env.yml\n",
    "source activate bq3d_env\n",
    "pip install git+https://github.com/sunilgandhilab/brainquant3d.git\n",
    "```\n",
    "then, choose the newly created \"bq3d\" conda environment as the kernel to run these Jupyter Notebooks and code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#run this code block to ensure everything has been imported correctly.\n",
    "import bq3d\n",
    "\n",
    "import os\n",
    "from pathlib import Path\n",
    "import yaml\n",
    "import bq3d\n",
    "import glob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Download Allen Brain Atlas datasets\n",
    "Download Warping.zip, which contains the Allen Brain Atlas datasets (4.04 GB zipped, 39.79 GB unzipped):\n",
    "\n",
    "Option 1 - Download from Google Drive:\n",
    "https://drive.google.com/drive/u/2/folders/1JjUxW3k2foIB2LBOa5LrtnCTofHToVho\n",
    "\n",
    "Option 2 - Download using the following command:\n",
    "```\n",
    "$ wget --no-check-certificate https://glams.bio.uci.edu/Warping.zip\n",
    "```\n",
    "Unzip and move the downloaded Warping directory to a location you will remember (e.g. SSD drive).\n",
    "```\n",
    "$ unzip Warping.zip\n",
    "$ mv Warping /mnt/ssd\n",
    "```\n",
    "Update this path in the parameter_template.py file in the same folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "replace this variable in parameter_template.py (line 37):\n",
      "PathReg = Path('/Users/ytcao/Documents/Warping')\n"
     ]
    }
   ],
   "source": [
    "brain_atlas_directory = \"/Users/ytcao/Documents/Warping\" #!!! modify the path to the downloaded brain atlas directory on your computer\n",
    "\n",
    "try:\n",
    "    if not any(Path(brain_atlas_directory).iterdir()):\n",
    "        raise Exception(\"No files found in the brain atlas directory. Make sure that the path is correct!\")\n",
    "except Exception as e: print(e)\n",
    "\n",
    "print(f\"\\nreplace this variable in parameter_template.py (line 37):\\nPathReg = Path('{brain_atlas_directory}')\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Set up configuration file\n",
    "Now you need to set up the configuration file. This file will tell BrainQuant3D where to place temp data and where to find the atlas data. The configuration template file, default.conf, can be found in the package install location.\n",
    "\n",
    "Temp_path: This is the location where temp data will be stored. It should be on a high-performance SSD. Preferably an NVMe. If the path does not exist, it will be created.\n",
    "\n",
    "Rigid_default, Affine_default, BSpline_default, Labeled_default, Annotations_default: These are the paths to the Allen Brain Atlas datasets downloaded earlier.\n",
    "\n",
    "Processing_cores, Thread_ram_max_Gb: These parameters specify how many processing cores to use and how much RAM to user per core. For example, if your system has 10 cores and 256 GB of RAM, you would set \"Processing_cores\" to 10 and \"Thread_ram_max_Gb\" to a value such as 22. This would use 10 x 22 = 220 GB of total RAM. It's a good idea to leave a little available RAM so as to not overload the system.\n",
    "\n",
    "Adjust variables as necessary below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "temporary directory: /var/folders/47/6bm96nxd0ks5x370bsvkpghh0000gn/T/\n",
      "hw.ncpu: 8\n",
      "8589934592\n",
      "8.0 GB ram\n"
     ]
    }
   ],
   "source": [
    "### locate temporary directory path on your computer\n",
    "!echo \"temporary directory:\" $TMPDIR\n",
    "\n",
    "### uncomment to find the number of cores and amount of RAM supported by your computer (linux)\n",
    "# !lscpu | egrep 'Model name|Socket|Thread|NUMA|CPU\\(s\\)' #look for number of CPUs\n",
    "# !free -h #max RAM\n",
    "\n",
    "### for windows:\n",
    "# !wmic cpu get caption, deviceid, name, numberofcores, maxclockspeed, status #look for number of cores\n",
    "# !systeminfo #look for max RAM\n",
    "\n",
    "### for mac:\n",
    "!sysctl -a | grep hw.ncpu #number of cores\n",
    "!sysctl -n hw.memsize #max RAM in bytes; divide by (1024**3) for GB\n",
    "print(8589934592/(1024**3), \"GB RAM\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### replace temp_directory variable with above output (optional), or at any location you choose.\n",
    "\n",
    "temp_directory = '' #!!! change this accordingly-- can be temporary directory path of your computer, the scratch folder, any directory you'd like. for example, on mac: temp_directory = '/var/folders/47/6bm96nxd0ks5x370bsvkpghh0000gn/T/'\n",
    "\n",
    "processing_cores = 1 #!!! change this accordingly\n",
    "thread_ram_max_GB = 4 #!!! change this accordingly\n",
    "\n",
    "#find installation address of bq3d package\n",
    "bq3d_package_directory = Path(bq3d.__file__).parent\n",
    "assert Path(bq3d_package_directory, \"default.conf\").exists(), f\"cannot find BrainQuant3D configuration file in the directory {bq3d_package_directory}\"\n",
    "print(\"BQ3D_PACKAGE_DIRECTORY:\", bq3d_package_directory)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "run the following code blocks to automatically create a configuration file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### create updated config file\n",
    "try:\n",
    "    parigid_path = Path(brain_atlas_directory, \"ParRigid.txt\")\n",
    "    paraffine_path = Path(brain_atlas_directory, \"ParAffine.txt\")\n",
    "    parbspline_path = Path(brain_atlas_directory, \"ParBSpline.txt\")\n",
    "    annotation_path = Path(brain_atlas_directory, \"ARA2\", \"annotation_25_right.tif\")\n",
    "    collapse_path = Path(brain_atlas_directory, \"ARA2_annotation_info_collapse.csv\")\n",
    "except Exception as e: print(e, f\"file does not exist in the provided directory {brain_atlas_directory}\")\n",
    "\n",
    "config_data = {\n",
    "    'user': {\n",
    "        'default': {\n",
    "            #paths\n",
    "            ## path for mac only\n",
    "            ## 'Ilastik_path': f'{str(bq3d_package_directory)}/.external/ilastik-1.3.3/Contents/ilastik-release', \n",
    "            'Ilastik_path': f'{str(bq3d_package_directory)}/.external/ilastik-1.3.3',\n",
    "            'Elastix_path': f'{str(bq3d_package_directory)}/.external/elastix-5.0.0',\n",
    "            'Temp_path':    temp_directory, \n",
    "\n",
    "            #brain atlas\n",
    "            'Rigid_default':       f'{str(parigid_path)}',\n",
    "            'Affine_default':      f'{str(paraffine_path)}',\n",
    "            'BSpline_default':     f'{str(parbspline_path)}',\n",
    "            'Labeled_default':     f'{str(annotation_path)}',\n",
    "            'Annotations_default': f'{str(collapse_path)}',\n",
    "            'Console_level':       'verbose',\n",
    "\n",
    "            #processing\n",
    "            'Processing_cores':    processing_cores, \n",
    "            'Thread_ram_max_Gb':   thread_ram_max_GB, \n",
    "        }\n",
    "    }\n",
    "}\n",
    "print(config_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### save to the bq3d folder\n",
    "with open(Path(bq3d_package_directory,\"brainquant3d.conf\"), mode=\"w+\") as f:\n",
    "    yaml.dump(config_data, f, sort_keys=False, indent=4, allow_unicode=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now move onto the tutorial Jupyter Notebook (01_bq3d_tutorial.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "bq3d",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}