Skip to content

Gradcam pp

GradCAMpp ¤

Bases: XAImethod_Base

XAI Method for Grad-CAM++.

Normally, this class is used internally in the aucmedi.xai.decoder.xai_decoder in the AUCMEDI XAI module.

Reference - Implementation

Author: Samson Woof
GitHub Profile: https://github.com/samson6460
Date: May 21, 2020
https://github.com/samson6460/tf_keras_gradcamplusplus

Reference - Publication

Aditya Chattopadhay; Anirban Sarkar; Prantik Howlader; Vineeth N Balasubramanian. 07 May 2018. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks.
https://ieeexplore.ieee.org/document/8354201

This class provides functionality for running the compute_heatmap function, which computes a Grad-CAM++ heatmap for an image with a model.

Source code in aucmedi/xai/methods/gradcam_pp.py
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
class GradCAMpp(XAImethod_Base):
    """ XAI Method for Grad-CAM++.

    Normally, this class is used internally in the [aucmedi.xai.decoder.xai_decoder][] in the AUCMEDI XAI module.

    ??? abstract "Reference - Implementation"
        Author: Samson Woof <br>
        GitHub Profile: [https://github.com/samson6460](https://github.com/samson6460) <br>
        Date: May 21, 2020 <br>
        [https://github.com/samson6460/tf_keras_gradcamplusplus](https://github.com/samson6460/tf_keras_gradcamplusplus) <br>

    ??? abstract "Reference - Publication"
        Aditya Chattopadhay; Anirban Sarkar; Prantik Howlader; Vineeth N Balasubramanian. 07 May 2018.
        Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks.
        <br>
        [https://ieeexplore.ieee.org/document/8354201](https://ieeexplore.ieee.org/document/8354201)

    This class provides functionality for running the compute_heatmap function,
    which computes a Grad-CAM++ heatmap for an image with a model.
    """
    def __init__(self, model, layerName=None):
        """ Initialization function for creating a Grad-CAM++ as XAI Method object.

        Args:
            model (keras.model):               Keras model object.
            layerName (str):                   Layer name of the convolutional layer for heatmap computation.
        """
        # Cache class parameters
        self.model = model
        self.layerName = layerName
        # Try to find output layer if not defined
        if self.layerName is None : self.layerName = self.find_output_layer()

    #---------------------------------------------#
    #            Identify Output Layer            #
    #---------------------------------------------#
    def find_output_layer(self):
        """ Internal function. Applied if `layerName==None`.

        Identify last/final convolutional layer in neural network architecture.
        This layer is used to obtain activation outputs / feature map.
        """
        # Iterate over all layers
        for layer in reversed(self.model.layers):
            # Check to see if the layer has a 4D output -> Return layer
            if len(layer.output_shape) == 4:
                return layer.name
        # Otherwise, throw exception
        raise ValueError("Could not find 4D layer. Cannot apply Grad-CAM++.")

    #---------------------------------------------#
    #             Heatmap Computation             #
    #---------------------------------------------#
    def compute_heatmap(self, image, class_index, eps=1e-8):
        """ Core function for computing the Grad-CAM++ heatmap for a provided image and for specific classification outcome.

        ???+ attention
            Be aware that the image has to be provided in batch format.

        Args:
            image (numpy.ndarray):              Image matrix encoded as NumPy Array (provided as one-element batch).
            class_index (int):                  Classification index for which the heatmap should be computed.
            eps (float):                        Epsilon for rounding.

        The returned heatmap is encoded within a range of [0,1]

        ???+ attention
            The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

        Returns:
            heatmap (numpy.ndarray):            Computed Grad-CAM++ for provided image.
        """
        # Gradient model construction
        gradModel = tf.keras.models.Model(inputs=[self.model.inputs],
                         outputs=[self.model.get_layer(self.layerName).output,
                                  self.model.output])

        # Compute gradient for desierd class index
        with tf.GradientTape() as gtape1:
            with tf.GradientTape() as gtape2:
                with tf.GradientTape() as gtape3:
                    inputs = tf.cast(image, tf.float32)
                    (conv_output, preds) = gradModel(inputs)
                    output = preds[:, class_index]
                    conv_first_grad = gtape3.gradient(output, conv_output)
                conv_second_grad = gtape2.gradient(conv_first_grad, conv_output)
            conv_third_grad = gtape1.gradient(conv_second_grad, conv_output)
        global_sum = np.sum(conv_output, axis=(0, 1, 2))

        # Normalize constants
        alpha_num = conv_second_grad[0]
        alpha_denom = conv_second_grad[0]*2.0 + conv_third_grad[0]*global_sum
        alpha_denom = np.where(alpha_denom != 0.0, alpha_denom, eps)
        alphas = alpha_num / alpha_denom
        alpha_normalization_constant = np.sum(alphas, axis=(0,1))
        alphas /= alpha_normalization_constant

        # Deep Linearization weighting
        weights = np.maximum(conv_first_grad[0], 0.0)
        deep_linearization_weights = np.sum(weights*alphas, axis=(0,1))
        heatmap = np.sum(deep_linearization_weights*conv_output[0], axis=2)

        # Intensity normalization to [0,1]
        numer = heatmap - np.min(heatmap)
        denom = (heatmap.max() - heatmap.min()) + eps
        heatmap = numer / denom

        # Return the resulting heatmap
        return heatmap

__init__(model, layerName=None) ¤

Initialization function for creating a Grad-CAM++ as XAI Method object.

Parameters:

Name Type Description Default
model keras.model

Keras model object.

required
layerName str

Layer name of the convolutional layer for heatmap computation.

None
Source code in aucmedi/xai/methods/gradcam_pp.py
51
52
53
54
55
56
57
58
59
60
61
62
def __init__(self, model, layerName=None):
    """ Initialization function for creating a Grad-CAM++ as XAI Method object.

    Args:
        model (keras.model):               Keras model object.
        layerName (str):                   Layer name of the convolutional layer for heatmap computation.
    """
    # Cache class parameters
    self.model = model
    self.layerName = layerName
    # Try to find output layer if not defined
    if self.layerName is None : self.layerName = self.find_output_layer()

compute_heatmap(image, class_index, eps=1e-08) ¤

Core function for computing the Grad-CAM++ heatmap for a provided image and for specific classification outcome.

Attention

Be aware that the image has to be provided in batch format.

Parameters:

Name Type Description Default
image numpy.ndarray

Image matrix encoded as NumPy Array (provided as one-element batch).

required
class_index int

Classification index for which the heatmap should be computed.

required
eps float

Epsilon for rounding.

1e-08

The returned heatmap is encoded within a range of [0,1]

Attention

The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

Returns:

Name Type Description
heatmap numpy.ndarray

Computed Grad-CAM++ for provided image.

Source code in aucmedi/xai/methods/gradcam_pp.py
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
def compute_heatmap(self, image, class_index, eps=1e-8):
    """ Core function for computing the Grad-CAM++ heatmap for a provided image and for specific classification outcome.

    ???+ attention
        Be aware that the image has to be provided in batch format.

    Args:
        image (numpy.ndarray):              Image matrix encoded as NumPy Array (provided as one-element batch).
        class_index (int):                  Classification index for which the heatmap should be computed.
        eps (float):                        Epsilon for rounding.

    The returned heatmap is encoded within a range of [0,1]

    ???+ attention
        The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

    Returns:
        heatmap (numpy.ndarray):            Computed Grad-CAM++ for provided image.
    """
    # Gradient model construction
    gradModel = tf.keras.models.Model(inputs=[self.model.inputs],
                     outputs=[self.model.get_layer(self.layerName).output,
                              self.model.output])

    # Compute gradient for desierd class index
    with tf.GradientTape() as gtape1:
        with tf.GradientTape() as gtape2:
            with tf.GradientTape() as gtape3:
                inputs = tf.cast(image, tf.float32)
                (conv_output, preds) = gradModel(inputs)
                output = preds[:, class_index]
                conv_first_grad = gtape3.gradient(output, conv_output)
            conv_second_grad = gtape2.gradient(conv_first_grad, conv_output)
        conv_third_grad = gtape1.gradient(conv_second_grad, conv_output)
    global_sum = np.sum(conv_output, axis=(0, 1, 2))

    # Normalize constants
    alpha_num = conv_second_grad[0]
    alpha_denom = conv_second_grad[0]*2.0 + conv_third_grad[0]*global_sum
    alpha_denom = np.where(alpha_denom != 0.0, alpha_denom, eps)
    alphas = alpha_num / alpha_denom
    alpha_normalization_constant = np.sum(alphas, axis=(0,1))
    alphas /= alpha_normalization_constant

    # Deep Linearization weighting
    weights = np.maximum(conv_first_grad[0], 0.0)
    deep_linearization_weights = np.sum(weights*alphas, axis=(0,1))
    heatmap = np.sum(deep_linearization_weights*conv_output[0], axis=2)

    # Intensity normalization to [0,1]
    numer = heatmap - np.min(heatmap)
    denom = (heatmap.max() - heatmap.min()) + eps
    heatmap = numer / denom

    # Return the resulting heatmap
    return heatmap

find_output_layer() ¤

Internal function. Applied if layerName==None.

Identify last/final convolutional layer in neural network architecture. This layer is used to obtain activation outputs / feature map.

Source code in aucmedi/xai/methods/gradcam_pp.py
67
68
69
70
71
72
73
74
75
76
77
78
79
def find_output_layer(self):
    """ Internal function. Applied if `layerName==None`.

    Identify last/final convolutional layer in neural network architecture.
    This layer is used to obtain activation outputs / feature map.
    """
    # Iterate over all layers
    for layer in reversed(self.model.layers):
        # Check to see if the layer has a 4D output -> Return layer
        if len(layer.output_shape) == 4:
            return layer.name
    # Otherwise, throw exception
    raise ValueError("Could not find 4D layer. Cannot apply Grad-CAM++.")