Merge branch 'main' of github.com:michaelgroeger/boxshrink

README.md

@@ -1,18 +1,44 @@
 # BoxShrink: From Bounding Boxes to Segmentation Masks
+
 This is the repository for the corresponding MICCAI-MILLanD Workshop 2022 paper [BoxShrink: From Bounding Boxes to Segmentation Masks](https://arxiv.org/abs/2208.03142)
 
-## Introduction
+## Overview
+
 We present two algorithms how one can process bounding boxes to pseudo segmentation masks in a binary class segmenetation setting:
+
 1. rapidBoxshrink: Works by a simple thresholding and overlapping strategy between the initial bounding box and
 the generated superpixels. This algorithm will reject superpixels that don't overlap to a certain percentage with the bounding box and then run a F-CRF on the pseudomask. ![rapidBoxshrink-Overview](/images/rapidBoxshrink_overview.png)
+
 2. robustBoxshrink: Compares the superpixels on the boundary with the mean foreground and background embedding of the training dataset. Those whose cosine distance is closer to the background embedding are being rejected. Finally, a F-CRF is being run on the pseudomask. ![robustBoxshrink-Overview](/images/robustBoxshrink_Overview.png)
 
+## Installation 
+
+Please follow these steps to install the environment.
+
+```zsh
+# Creates new conda environment
+conda create -n boxshrink python=3.10.8 ipython
+# Activates conda environment
+conda activate boxshrink
+# Makes script folder callable as module in python scripts
+conda install conda-build
+conda develop ./scripts
+# Installs dependencies
+pip install -r requirements.txt
+# Make conda available in jupyter notebook
+conda install ipykernel
+python -m ipykernel install --user --name=boxshrink
+```
+
 ## Usage
+
 Please check the config file in `scripts/config` to set paths and hyperparameters. Please have a look at the notebook files if you want to generate bounding boxes from masks, run rapidBoxshrink or robustBoxshrink. After you generated the masks feel free to use them as training input as shown in `train.ipynb`. Have fun!
 
 ## Citation
+
 If you use this work please cite:
-```
+
+```latex
 @inproceedings{groger2022boxshrink,
   title={BoxShrink: From Bounding Boxes to Segmentation Masks},
   author={Gröger, Michael and Borisov, Vadim and Kasneci, Gjergji},

requirements.txt

@@ -0,0 +1,17 @@
+# Automatically generated by https://github.com/damnever/pigar.
+grad-cam==1.4.6
+matplotlib==3.6.2
+numpy==1.23.4
+opencv-python==4.7.0.72
+pandas==1.5.2
+Pillow==9.2.0
+pytorch-text-crf==0.1
+scikit-image==0.20.0
+scikit-learn==1.2.1
+segmentation-models-pytorch==0.3.2
+tifffile==2023.3.21
+torch==1.12.1
+torchmetrics==0.11.0
+torchvision==0.13.1
+tqdm==4.64.1
+git+https://github.com/lucasb-eyer/pydensecrf.git

robust_boxshrink.ipynb

@@ -2,18 +2,9 @@
  "cells": [
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/michaelgroeger/miniconda3/envs/boxshrink/lib/python3.8/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
-      "  from .autonotebook import tqdm as notebook_tqdm\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from sklearn.model_selection import train_test_split\n",
     "from scripts.tools import return_files_in_directory, human_sort\n",
@@ -23,12 +14,12 @@
     "from torchvision.models import resnet50\n",
     "from scripts.dataset import Colonoscopy_Dataset\n",
     "from torch.utils.data import DataLoader\n",
-    "from scripts.embeddings import ResnetFeatureExtractor,generate_embedding_masks_for_dataset"
+    "from scripts.embeddings import ResnetFeatureExtractor, generate_embedding_masks_for_dataset"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -37,10 +28,8 @@
     "# Ensure files are in correct order\n",
     "human_sort(image_files)\n",
     "human_sort(box_files)\n",
-    "X_train, X_test, y_train, y_test = train_test_split(image_files, box_files, test_size=0.1, random_state=1)\n",
-    "X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.11111, random_state=1) # 0.1111 x 0.9 = 0.1\n",
     "\n",
-    "dataset = Colonoscopy_Dataset(X_val, y_val)\n",
+    "dataset = Colonoscopy_Dataset(image_files, box_files)\n",
     "data_loader = DataLoader(dataset, batch_size=64, shuffle=False, num_workers=0)\n",
     "\n",
     "TESTING_DIR = DATA_DIR + \"/testing/robust_boxshrink\"\n",
@@ -54,7 +43,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -67,17 +56,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Generating Embedding Masks: 100%|██████████| 1/1 [13:15<00:00, 795.02s/batch]\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "resnet = resnet50(weights=\"ResNet50_Weights.IMAGENET1K_V2\")\n",
     "resnet.eval()\n",

scripts/crf.py

@@ -20,7 +20,7 @@
     RGB_STD,
 )
 
-jaccard_crf = JaccardIndex(num_classes=len(CLASSES), average="none", ignore_index=0).to(
+jaccard_crf = JaccardIndex(num_classes=len(CLASSES), average="none", ignore_index=0, task="binary").to(
     DEVICE
 )
 

scripts/embeddings.py

@@ -393,7 +393,7 @@ def generate_embedding_masks_for_dataset(
         with tqdm(
             data_loader, unit="batch", desc="Generating Embedding Masks"
         ) as tepoch:
-            for train_inputs, train_labels, train_org_images in tepoch:
+            for _, train_labels, train_org_images in tepoch:
                 for idx in range(train_labels.shape[0]):
                     mask_name = dataset.X[filecounter]
                     pseudomask = get_embedding_mask_or_box(