Bases: XAImethod_Base
XAI Method for Gradient-weighted Class Activation Mapping (Grad-CAM).
Normally, this class is used internally in the aucmedi.xai.decoder.xai_decoder in the AUCMEDI XAI module.
Reference - Implementation #1
Author: François Chollet
Date: April 26, 2020
https://keras.io/examples/vision/grad_cam/
Reference - Implementation #2
Author: Adrian Rosebrock
Date: March 9, 2020
https://www.pyimagesearch.com/2020/03/09/grad-cam-visualize-class-activation-maps-with-keras-tensorflow-and-deep-learning/
Reference - Publication
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra. 7 Oct 2016.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization.
https://arxiv.org/abs/1610.02391
This class provides functionality for running the compute_heatmap function,
which computes a Grad-CAM heatmap for an image with a model.
Source code in aucmedi/xai/methods/gradcam.py
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138 | class GradCAM(XAImethod_Base):
""" XAI Method for Gradient-weighted Class Activation Mapping (Grad-CAM).
Normally, this class is used internally in the [aucmedi.xai.decoder.xai_decoder][] in the AUCMEDI XAI module.
??? abstract "Reference - Implementation #1"
Author: François Chollet <br>
Date: April 26, 2020 <br>
[https://keras.io/examples/vision/grad_cam/](https://keras.io/examples/vision/grad_cam/) <br>
??? abstract "Reference - Implementation #2"
Author: Adrian Rosebrock <br>
Date: March 9, 2020 <br>
[https://www.pyimagesearch.com/2020/03/09/grad-cam-visualize-class-activation-maps-with-keras-tensorflow-and-deep-learning/](https://www.pyimagesearch.com/2020/03/09/grad-cam-visualize-class-activation-maps-with-keras-tensorflow-and-deep-learning/) <br>
??? abstract "Reference - Publication"
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra. 7 Oct 2016.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization.
<br>
[https://arxiv.org/abs/1610.02391](https://arxiv.org/abs/1610.02391)
This class provides functionality for running the compute_heatmap function,
which computes a Grad-CAM heatmap for an image with a model.
"""
def __init__(self, model, layerName=None):
""" Initialization function for creating a Grad-CAM as XAI Method object.
Args:
model (keras.model): Keras model object.
layerName (str): Layer name of the convolutional layer for heatmap computation.
"""
# Cache class parameters
self.model = model
self.layerName = layerName
# Try to find output layer if not defined
if self.layerName is None : self.layerName = self.find_output_layer()
#---------------------------------------------#
# Identify Output Layer #
#---------------------------------------------#
def find_output_layer(self):
""" Internal function. Applied if `layerName==None`.
Identify last/final convolutional layer in neural network architecture.
This layer is used to obtain activation outputs / feature map.
"""
# Iterate over all layers
for layer in reversed(self.model.layers):
# Check to see if the layer has a 4D output -> Return layer
if len(layer.output.shape) >= 4:
return layer.name
# Otherwise, throw exception
raise ValueError("Could not find 4D layer. Cannot apply Grad-CAM.")
#---------------------------------------------#
# Heatmap Computation #
#---------------------------------------------#
def compute_heatmap(self, image, class_index, eps=1e-8):
""" Core function for computing the Grad-CAM heatmap for a provided image and for specific classification outcome.
???+ attention
Be aware that the image has to be provided in batch format.
Args:
image (numpy.ndarray): Image matrix encoded as NumPy Array (provided as one-element batch).
class_index (int): Classification index for which the heatmap should be computed.
eps (float): Epsilon for rounding.
The returned heatmap is encoded within a range of [0,1]
???+ attention
The shape of the returned heatmap is 2D or 3D -> batch and channel axis will be removed.
Returns:
heatmap (numpy.ndarray): Computed Grad-CAM for provided image.
"""
# Gradient model construction
layer_output = self.model.get_layer(self.layerName).output
model_output = self.model.output
if isinstance(model_output, list):
outputs = [layer_output] + model_output
else:
outputs = [layer_output, model_output]
gradModel = tf.keras.models.Model(inputs=self.model.inputs,
outputs=outputs)
# Compute gradient for desired class index
with tf.GradientTape() as tape:
inputs = tf.cast(image, tf.float32)
(conv_out, preds) = gradModel(inputs)
loss = preds[:, class_index]
grads = tape.gradient(loss, conv_out)
# Identify pooling axis
if len(image.shape) == 4 : pooling_axis = (0, 1, 2)
else : pooling_axis = (0, 1, 2, 3)
# Averaged output gradient based on feature map of last conv layer
pooled_grads = tf.reduce_mean(grads, axis=pooling_axis)
# Normalize gradients via "importance"
heatmap = conv_out[0] @ pooled_grads[..., tf.newaxis]
heatmap = tf.squeeze(heatmap).numpy()
# Intensity normalization to [0,1]
numer = heatmap - np.min(heatmap)
denom = (heatmap.max() - heatmap.min()) + eps
heatmap = numer / denom
# Return the resulting heatmap
return heatmap
|
__init__(model, layerName=None)
Initialization function for creating a Grad-CAM as XAI Method object.
Parameters:
| Name |
Type |
Description |
Default |
model |
keras.model
|
Keras model object. |
required
|
layerName |
str
|
Layer name of the convolutional layer for heatmap computation. |
None
|
Source code in aucmedi/xai/methods/gradcam.py
55
56
57
58
59
60
61
62
63
64
65
66 | def __init__(self, model, layerName=None):
""" Initialization function for creating a Grad-CAM as XAI Method object.
Args:
model (keras.model): Keras model object.
layerName (str): Layer name of the convolutional layer for heatmap computation.
"""
# Cache class parameters
self.model = model
self.layerName = layerName
# Try to find output layer if not defined
if self.layerName is None : self.layerName = self.find_output_layer()
|
compute_heatmap(image, class_index, eps=1e-08)
Core function for computing the Grad-CAM heatmap for a provided image and for specific classification outcome.
Attention
Be aware that the image has to be provided in batch format.
Parameters:
| Name |
Type |
Description |
Default |
image |
numpy.ndarray
|
Image matrix encoded as NumPy Array (provided as one-element batch). |
required
|
class_index |
int
|
Classification index for which the heatmap should be computed. |
required
|
eps |
float
|
Epsilon for rounding. |
1e-08
|
The returned heatmap is encoded within a range of [0,1]
Attention
The shape of the returned heatmap is 2D or 3D -> batch and channel axis will be removed.
Returns:
| Name | Type |
Description |
heatmap |
numpy.ndarray
|
Computed Grad-CAM for provided image. |
Source code in aucmedi/xai/methods/gradcam.py
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138 | def compute_heatmap(self, image, class_index, eps=1e-8):
""" Core function for computing the Grad-CAM heatmap for a provided image and for specific classification outcome.
???+ attention
Be aware that the image has to be provided in batch format.
Args:
image (numpy.ndarray): Image matrix encoded as NumPy Array (provided as one-element batch).
class_index (int): Classification index for which the heatmap should be computed.
eps (float): Epsilon for rounding.
The returned heatmap is encoded within a range of [0,1]
???+ attention
The shape of the returned heatmap is 2D or 3D -> batch and channel axis will be removed.
Returns:
heatmap (numpy.ndarray): Computed Grad-CAM for provided image.
"""
# Gradient model construction
layer_output = self.model.get_layer(self.layerName).output
model_output = self.model.output
if isinstance(model_output, list):
outputs = [layer_output] + model_output
else:
outputs = [layer_output, model_output]
gradModel = tf.keras.models.Model(inputs=self.model.inputs,
outputs=outputs)
# Compute gradient for desired class index
with tf.GradientTape() as tape:
inputs = tf.cast(image, tf.float32)
(conv_out, preds) = gradModel(inputs)
loss = preds[:, class_index]
grads = tape.gradient(loss, conv_out)
# Identify pooling axis
if len(image.shape) == 4 : pooling_axis = (0, 1, 2)
else : pooling_axis = (0, 1, 2, 3)
# Averaged output gradient based on feature map of last conv layer
pooled_grads = tf.reduce_mean(grads, axis=pooling_axis)
# Normalize gradients via "importance"
heatmap = conv_out[0] @ pooled_grads[..., tf.newaxis]
heatmap = tf.squeeze(heatmap).numpy()
# Intensity normalization to [0,1]
numer = heatmap - np.min(heatmap)
denom = (heatmap.max() - heatmap.min()) + eps
heatmap = numer / denom
# Return the resulting heatmap
return heatmap
|
find_output_layer()
Internal function. Applied if layerName==None.
Identify last/final convolutional layer in neural network architecture.
This layer is used to obtain activation outputs / feature map.
Source code in aucmedi/xai/methods/gradcam.py
71
72
73
74
75
76
77
78
79
80
81
82
83 | def find_output_layer(self):
""" Internal function. Applied if `layerName==None`.
Identify last/final convolutional layer in neural network architecture.
This layer is used to obtain activation outputs / feature map.
"""
# Iterate over all layers
for layer in reversed(self.model.layers):
# Check to see if the layer has a 4D output -> Return layer
if len(layer.output.shape) >= 4:
return layer.name
# Otherwise, throw exception
raise ValueError("Could not find 4D layer. Cannot apply Grad-CAM.")
|