CNNVis is a high-level convolutional neural network (CNN) visualization API built on top of Keras. The intention behind this project aligns with the intention of Keras: "Being able to go from idea to result with the least possible delay is key to doing good research".
Use CNNVis if you need to visualize the following aspects of a CNN. Of course, your CNN model needs to be a keras.models.Sequential or keras.models.Model instance.
Feel free to email me at [email protected] about any additional features that you would like to visualize!
The main resources that helped the development of this project tremendously:
- Chapter 5: Computer Vision, of book Deep Learning with Python by Francis Chollet
- The official Keras documentation: https://keras.io/
Important Note: This library only supports your Keras if your backend is TensorFlow.
First, make sure that all dependencies are installed:
- prettytable
- Numpy
- matplotlib
- PIL
- Keras
To instantiate a Visualizer instance for a vgg16 network:
import keras
from cnnvis import Visualizer
vgg16_model = keras.applications.VGG16(weights='imagenet', include_top=True)
visualizer = Visualizer(model=vgg16_model, image_shape=(224, 224, 3), batch_size=1, preprocess_style='vgg16')To print the default summary:
visualizer.summary(style='default')To print the "cnn style" summary:
visualizer.summary(style='cnn')CNN Style Model Summary
+--------------+--------------+------------+-------------+----------------+---------------------+
| Layer Name | Layer Type | Kernel Num | Kernel Size | Kernel Padding | Output Shape |
+--------------+--------------+------------+-------------+----------------+---------------------+
| block1_conv1 | Conv2D | 64 | (3, 3) | same | (1, 224, 224, 64) |
| block1_conv2 | Conv2D | 64 | (3, 3) | same | (1, 224, 224, 64) |
| block1_pool | MaxPooling2D | / | (2, 2) | / | (1, 112, 112, 3) |
| block2_conv1 | Conv2D | 128 | (3, 3) | same | (1, 224, 224, 128) |
| block2_conv2 | Conv2D | 128 | (3, 3) | same | (1, 224, 224, 128) |
| block2_pool | MaxPooling2D | / | (2, 2) | / | (1, 112, 112, 3) |
| block3_conv1 | Conv2D | 256 | (3, 3) | same | (1, 224, 224, 256) |
| block3_conv2 | Conv2D | 256 | (3, 3) | same | (1, 224, 224, 256) |
| block3_conv3 | Conv2D | 256 | (3, 3) | same | (1, 224, 224, 256) |
| block3_pool | MaxPooling2D | / | (2, 2) | / | (1, 112, 112, 3) |
| block4_conv1 | Conv2D | 512 | (3, 3) | same | (1, 224, 224, 512) |
| block4_conv2 | Conv2D | 512 | (3, 3) | same | (1, 224, 224, 512) |
| block4_conv3 | Conv2D | 512 | (3, 3) | same | (1, 224, 224, 512) |
| block4_pool | MaxPooling2D | / | (2, 2) | / | (1, 112, 112, 3) |
| block5_conv1 | Conv2D | 512 | (3, 3) | same | (1, 224, 224, 512) |
| block5_conv2 | Conv2D | 512 | (3, 3) | same | (1, 224, 224, 512) |
| block5_conv3 | Conv2D | 512 | (3, 3) | same | (1, 224, 224, 512) |
| block5_pool | MaxPooling2D | / | (2, 2) | / | (1, 112, 112, 3) |
| flatten | Flatten | / | / | / | (1, 150528) |
| fc1 | Dense | / | / | / | (1, 224, 224, 4096) |
| fc2 | Dense | / | / | / | (1, 224, 224, 4096) |
| predictions | Dense | / | / | / | (1, 224, 224, 1000) |
+--------------+--------------+------------+-------------+----------------+---------------------+
Training set: 1000 Categories of ImageNet
Number of Conv2D layers: 13
Number of MaxPooling2D layers: 5
Number of Dense layers: 3
To plot saliency maps:
img_paths = ['fish.jpg', 'bird.jpg', 'elephants.jpg']
saliency_maps = visualizer.get_saliency_map(img_paths) import numpy as np
from matplotlib import pyplot as plt
fig = plt.figure()
fig.add_subplot(1, 3, 1)
plt.axis('off')
plt.imshow(saliency_maps[0])
fig.add_subplot(1, 3, 2)
plt.axis('off')
plt.imshow(saliency_maps[1])
fig.add_subplot(1, 3, 3)
plt.axis('off')
plt.imshow(saliency_maps[2])
plt.show()
To plot the feature map of a specific layer to a specific image (e.g. giraffe):
feature_map = visualizer.get_feature_maps(['block5_conv3'], ['giraffe.png'])import numpy as np
from matplotlib import pyplot as plt
plt.matshow(np.mean(feature_map[0, 0], axis=-1))
plt.show()
To plot mean activations of multiple layers to multiple images (e.g. a cat image and a dog image):
- https://github.com/zhihanyang2022/pngs/blob/master/cat.png
- https://github.com/zhihanyang2022/pngs/blob/master/dog.jpg
mean_activation = visualizer.get_mean_activations(['block5_conv2', 'block5_conv3'], [img_path_cat, img_path_dog])from matplotlib import pyplot as plt
plt.plot(mean_activation[0, 0], label='Cat', alpha=0.6)
plt.plot(mean_activation[0, 1], label='Dog', alpha=0.6)
plt.xlabel('Kernel Index')
plt.ylabel('Mean Activation')
plt.title('Mean activations of block5_conv2')
plt.legend()
plt.show()
from matplotlib import pyplot as plt
plt.plot(mean_activation[1, 0], label='Cat', alpha=0.6)
plt.plot(mean_activation[1, 1], label='Dog', alpha=0.6)
plt.xlabel('Kernel Index')
plt.ylabel('Mean Activation')
plt.title('Mean activations of block5_conv3')
plt.legend()
plt.show()
To plot max activation to specific kernels in a specific layer:
max_activations = visualizer.get_max_activations('block3_conv1', [12, 123], 2)from matplotlib import pyplot as plt
plt.imshow(max_activation[0])
plt.axis('off')
plt.show()
from matplotlib import pyplot as plt
plt.imshow(max_activation[1])
plt.axis('off')
plt.show()
To plot kernels
kernels = visualizer.get_kernels('block2_conv1')from matplotlib import pyplot as plt
import numpy as np
plt.matshow(np.mean(kernels[:, :, :, 1], axis=-1))
plt.show()
