Bases: XAImethod_Base
XAI Method for Gradient-weighted Class Activation Mapping (Grad-CAM).
Normally, this class is used internally in the aucmedi.xai.decoder.xai_decoder in the AUCMEDI XAI module.
Reference - Implementation #1
Author: François Chollet
Date: April 26, 2020
https://keras.io/examples/vision/grad_cam/
Reference - Implementation #2
Author: Adrian Rosebrock
Date: March 9, 2020
https://www.pyimagesearch.com/2020/03/09/grad-cam-visualize-class-activation-maps-with-keras-tensorflow-and-deep-learning/
Reference - Publication
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra. 7 Oct 2016.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization.
https://arxiv.org/abs/1610.02391
This class provides functionality for running the compute_heatmap function,
which computes a Grad-CAM heatmap for an image with a model.
Source code in aucmedi/xai/methods/gradcam.py
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129 | class GradCAM(XAImethod_Base):
""" XAI Method for Gradient-weighted Class Activation Mapping (Grad-CAM).
Normally, this class is used internally in the [aucmedi.xai.decoder.xai_decoder][] in the AUCMEDI XAI module.
??? abstract "Reference - Implementation #1"
Author: François Chollet <br>
Date: April 26, 2020 <br>
[https://keras.io/examples/vision/grad_cam/](https://keras.io/examples/vision/grad_cam/) <br>
??? abstract "Reference - Implementation #2"
Author: Adrian Rosebrock <br>
Date: March 9, 2020 <br>
[https://www.pyimagesearch.com/2020/03/09/grad-cam-visualize-class-activation-maps-with-keras-tensorflow-and-deep-learning/](https://www.pyimagesearch.com/2020/03/09/grad-cam-visualize-class-activation-maps-with-keras-tensorflow-and-deep-learning/) <br>
??? abstract "Reference - Publication"
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra. 7 Oct 2016.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization.
<br>
[https://arxiv.org/abs/1610.02391](https://arxiv.org/abs/1610.02391)
This class provides functionality for running the compute_heatmap function,
which computes a Grad-CAM heatmap for an image with a model.
"""
def __init__(self, model, layerName=None):
""" Initialization function for creating a Grad-CAM as XAI Method object.
Args:
model (keras.model): Keras model object.
layerName (str): Layer name of the convolutional layer for heatmap computation.
"""
# Cache class parameters
self.model = model
self.layerName = layerName
# Try to find output layer if not defined
if self.layerName is None : self.layerName = self.find_output_layer()
#---------------------------------------------#
# Identify Output Layer #
#---------------------------------------------#
def find_output_layer(self):
""" Internal function. Applied if `layerName==None`.
Identify last/final convolutional layer in neural network architecture.
This layer is used to obtain activation outputs / feature map.
"""
# Iterate over all layers
for layer in reversed(self.model.layers):
# Check to see if the layer has a 4D output -> Return layer
if len(layer.output.shape) == 4:
return layer.name
# Otherwise, throw exception
raise ValueError("Could not find 4D layer. Cannot apply Grad-CAM.")
#---------------------------------------------#
# Heatmap Computation #
#---------------------------------------------#
def compute_heatmap(self, image, class_index, eps=1e-8):
""" Core function for computing the Grad-CAM heatmap for a provided image and for specific classification outcome.
???+ attention
Be aware that the image has to be provided in batch format.
Args:
image (numpy.ndarray): Image matrix encoded as NumPy Array (provided as one-element batch).
class_index (int): Classification index for which the heatmap should be computed.
eps (float): Epsilon for rounding.
The returned heatmap is encoded within a range of [0,1]
???+ attention
The shape of the returned heatmap is 2D -> batch and channel axis will be removed.
Returns:
heatmap (numpy.ndarray): Computed Grad-CAM for provided image.
"""
# Gradient model construction
gradModel = tf.keras.models.Model(inputs=[self.model.inputs],
outputs=[self.model.get_layer(self.layerName).output,
self.model.output])
# Compute gradient for desierd class index
with tf.GradientTape() as tape:
inputs = tf.cast(image, tf.float32)
(conv_out, preds) = gradModel(inputs)
loss = preds[:, class_index]
grads = tape.gradient(loss, conv_out)
# Averaged output gradient based on feature map of last conv layer
pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
# Normalize gradients via "importance"
heatmap = conv_out[0] @ pooled_grads[..., tf.newaxis]
heatmap = tf.squeeze(heatmap).numpy()
# Intensity normalization to [0,1]
numer = heatmap - np.min(heatmap)
denom = (heatmap.max() - heatmap.min()) + eps
heatmap = numer / denom
# Return the resulting heatmap
return heatmap
|
__init__(model, layerName=None)
Initialization function for creating a Grad-CAM as XAI Method object.
Parameters:
| Name |
Type |
Description |
Default |
model |
keras.model
|
Keras model object. |
required
|
layerName |
str
|
Layer name of the convolutional layer for heatmap computation. |
None
|
Source code in aucmedi/xai/methods/gradcam.py
55
56
57
58
59
60
61
62
63
64
65
66 | def __init__(self, model, layerName=None):
""" Initialization function for creating a Grad-CAM as XAI Method object.
Args:
model (keras.model): Keras model object.
layerName (str): Layer name of the convolutional layer for heatmap computation.
"""
# Cache class parameters
self.model = model
self.layerName = layerName
# Try to find output layer if not defined
if self.layerName is None : self.layerName = self.find_output_layer()
|
compute_heatmap(image, class_index, eps=1e-08)
Core function for computing the Grad-CAM heatmap for a provided image and for specific classification outcome.
Attention
Be aware that the image has to be provided in batch format.
Parameters:
| Name |
Type |
Description |
Default |
image |
numpy.ndarray
|
Image matrix encoded as NumPy Array (provided as one-element batch). |
required
|
class_index |
int
|
Classification index for which the heatmap should be computed. |
required
|
eps |
float
|
Epsilon for rounding. |
1e-08
|
The returned heatmap is encoded within a range of [0,1]
Attention
The shape of the returned heatmap is 2D -> batch and channel axis will be removed.
Returns:
| Name | Type |
Description |
heatmap |
numpy.ndarray
|
Computed Grad-CAM for provided image. |
Source code in aucmedi/xai/methods/gradcam.py
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129 | def compute_heatmap(self, image, class_index, eps=1e-8):
""" Core function for computing the Grad-CAM heatmap for a provided image and for specific classification outcome.
???+ attention
Be aware that the image has to be provided in batch format.
Args:
image (numpy.ndarray): Image matrix encoded as NumPy Array (provided as one-element batch).
class_index (int): Classification index for which the heatmap should be computed.
eps (float): Epsilon for rounding.
The returned heatmap is encoded within a range of [0,1]
???+ attention
The shape of the returned heatmap is 2D -> batch and channel axis will be removed.
Returns:
heatmap (numpy.ndarray): Computed Grad-CAM for provided image.
"""
# Gradient model construction
gradModel = tf.keras.models.Model(inputs=[self.model.inputs],
outputs=[self.model.get_layer(self.layerName).output,
self.model.output])
# Compute gradient for desierd class index
with tf.GradientTape() as tape:
inputs = tf.cast(image, tf.float32)
(conv_out, preds) = gradModel(inputs)
loss = preds[:, class_index]
grads = tape.gradient(loss, conv_out)
# Averaged output gradient based on feature map of last conv layer
pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
# Normalize gradients via "importance"
heatmap = conv_out[0] @ pooled_grads[..., tf.newaxis]
heatmap = tf.squeeze(heatmap).numpy()
# Intensity normalization to [0,1]
numer = heatmap - np.min(heatmap)
denom = (heatmap.max() - heatmap.min()) + eps
heatmap = numer / denom
# Return the resulting heatmap
return heatmap
|
find_output_layer()
Internal function. Applied if layerName==None.
Identify last/final convolutional layer in neural network architecture.
This layer is used to obtain activation outputs / feature map.
Source code in aucmedi/xai/methods/gradcam.py
71
72
73
74
75
76
77
78
79
80
81
82
83 | def find_output_layer(self):
""" Internal function. Applied if `layerName==None`.
Identify last/final convolutional layer in neural network architecture.
This layer is used to obtain activation outputs / feature map.
"""
# Iterate over all layers
for layer in reversed(self.model.layers):
# Check to see if the layer has a 4D output -> Return layer
if len(layer.output.shape) == 4:
return layer.name
# Otherwise, throw exception
raise ValueError("Could not find 4D layer. Cannot apply Grad-CAM.")
|