Skip to content

Occlusion sensitivity

OcclusionSensitivity ¤

Bases: XAImethod_Base

XAI Method for Occlusion Sensitivity.

Normally, this class is used internally in the aucmedi.xai.decoder.xai_decoder in the AUCMEDI XAI module.

Reference - Implementation

Author: Raphael Meudec
GitHub Profile: https://gist.github.com/RaphaelMeudec
Date: Jul 18, 2019
https://gist.github.com/RaphaelMeudec/7985b0c5eb720a29021d52b0a0be549a

This class provides functionality for running the compute_heatmap function, which computes a Occlusion Sensitivity Map for an image with a model.

Source code in aucmedi/xai/methods/occlusion_sensitivity.py
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
class OcclusionSensitivity(XAImethod_Base):
    """ XAI Method for Occlusion Sensitivity.

    Normally, this class is used internally in the [aucmedi.xai.decoder.xai_decoder][] in the AUCMEDI XAI module.

    ??? abstract "Reference - Implementation"
        Author: Raphael Meudec <br>
        GitHub Profile: [https://gist.github.com/RaphaelMeudec](https://gist.github.com/RaphaelMeudec) <br>
        Date: Jul 18, 2019 <br>
        [https://gist.github.com/RaphaelMeudec/7985b0c5eb720a29021d52b0a0be549a](https://gist.github.com/RaphaelMeudec/7985b0c5eb720a29021d52b0a0be549a) <br>

    This class provides functionality for running the compute_heatmap function,
    which computes a Occlusion Sensitivity Map for an image with a model.
    """
    def __init__(self, model, layerName=None, patch_size=16):
        """ Initialization function for creating a Occlusion Sensitivity Map as XAI Method object.

        Args:
            model (keras.model):            Keras model object.
            layerName (str):                Not required in Occlusion Sensitivity Maps, but defined by Abstract Base Class.
        """
        # Cache class parameters
        self.model = model
        self.patch_size = patch_size

    #---------------------------------------------#
    #             Heatmap Computation             #
    #---------------------------------------------#
    def compute_heatmap(self, image, class_index, eps=1e-8):
        """ Core function for computing the Occlusion Sensitivity Map for a provided image and for specific classification outcome.

        ???+ attention
            Be aware that the image has to be provided in batch format.

        Args:
            image (numpy.ndarray):              Image matrix encoded as NumPy Array (provided as one-element batch).
            class_index (int):                  Classification index for which the heatmap should be computed.
            eps (float):                        Epsilon for rounding.

        The returned heatmap is encoded within a range of [0,1]

        ???+ attention
            The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

        Returns:
            heatmap (numpy.ndarray):            Computed Occlusion Sensitivity Map for provided image.
        """
        # Utilize only image matrix instead of batch
        image = image[0]
        # Create empty sensitivity map
        sensitivity_map = np.zeros((image.shape[0], image.shape[1]))
        # Iterate the patch over the image
        for top_left_x in range(0, image.shape[0], self.patch_size):
            for top_left_y in range(0, image.shape[1], self.patch_size):
                patch = apply_grey_patch(image, top_left_x, top_left_y,
                                         self.patch_size)
                prediction = self.model.predict(np.array([patch]))[0]
                confidence = prediction[class_index]

                # Save confidence for this specific patch in the map
                sensitivity_map[
                    top_left_y:top_left_y + self.patch_size,
                    top_left_x:top_left_x + self.patch_size,
                ] = 1 - confidence
        # Return the resulting sensitivity map (automatically a heatmap)
        return sensitivity_map

__init__(model, layerName=None, patch_size=16) ¤

Initialization function for creating a Occlusion Sensitivity Map as XAI Method object.

Parameters:

Name Type Description Default
model keras.model

Keras model object.

required
layerName str

Not required in Occlusion Sensitivity Maps, but defined by Abstract Base Class.

None
Source code in aucmedi/xai/methods/occlusion_sensitivity.py
45
46
47
48
49
50
51
52
53
54
def __init__(self, model, layerName=None, patch_size=16):
    """ Initialization function for creating a Occlusion Sensitivity Map as XAI Method object.

    Args:
        model (keras.model):            Keras model object.
        layerName (str):                Not required in Occlusion Sensitivity Maps, but defined by Abstract Base Class.
    """
    # Cache class parameters
    self.model = model
    self.patch_size = patch_size

compute_heatmap(image, class_index, eps=1e-08) ¤

Core function for computing the Occlusion Sensitivity Map for a provided image and for specific classification outcome.

Attention

Be aware that the image has to be provided in batch format.

Parameters:

Name Type Description Default
image numpy.ndarray

Image matrix encoded as NumPy Array (provided as one-element batch).

required
class_index int

Classification index for which the heatmap should be computed.

required
eps float

Epsilon for rounding.

1e-08

The returned heatmap is encoded within a range of [0,1]

Attention

The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

Returns:

Name Type Description
heatmap numpy.ndarray

Computed Occlusion Sensitivity Map for provided image.

Source code in aucmedi/xai/methods/occlusion_sensitivity.py
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
def compute_heatmap(self, image, class_index, eps=1e-8):
    """ Core function for computing the Occlusion Sensitivity Map for a provided image and for specific classification outcome.

    ???+ attention
        Be aware that the image has to be provided in batch format.

    Args:
        image (numpy.ndarray):              Image matrix encoded as NumPy Array (provided as one-element batch).
        class_index (int):                  Classification index for which the heatmap should be computed.
        eps (float):                        Epsilon for rounding.

    The returned heatmap is encoded within a range of [0,1]

    ???+ attention
        The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

    Returns:
        heatmap (numpy.ndarray):            Computed Occlusion Sensitivity Map for provided image.
    """
    # Utilize only image matrix instead of batch
    image = image[0]
    # Create empty sensitivity map
    sensitivity_map = np.zeros((image.shape[0], image.shape[1]))
    # Iterate the patch over the image
    for top_left_x in range(0, image.shape[0], self.patch_size):
        for top_left_y in range(0, image.shape[1], self.patch_size):
            patch = apply_grey_patch(image, top_left_x, top_left_y,
                                     self.patch_size)
            prediction = self.model.predict(np.array([patch]))[0]
            confidence = prediction[class_index]

            # Save confidence for this specific patch in the map
            sensitivity_map[
                top_left_y:top_left_y + self.patch_size,
                top_left_x:top_left_x + self.patch_size,
            ] = 1 - confidence
    # Return the resulting sensitivity map (automatically a heatmap)
    return sensitivity_map

apply_grey_patch(image, top_left_x, top_left_y, patch_size) ¤

Internal function.

Replace a part of the image with a grey patch.

Parameters:

Name Type Description Default
image numpy.ndarray

Input image

required
top_left_x int

Top Left X position of the applied box

required
top_left_y int

Top Left Y position of the applied box

required
patch_size int

Size of patch to apply

required

Returns:

Name Type Description
patched_image numpy.ndarray

Patched image

Source code in aucmedi/xai/methods/occlusion_sensitivity.py
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
def apply_grey_patch(image, top_left_x, top_left_y, patch_size):
    """ Internal function.

    Replace a part of the image with a grey patch.

    Args:
        image (numpy.ndarray):                  Input image
        top_left_x (int):                       Top Left X position of the applied box
        top_left_y (int):                       Top Left Y position of the applied box
        patch_size (int):                       Size of patch to apply

    Returns:
        patched_image (numpy.ndarray):          Patched image
    """
    patched_image = np.array(image, copy=True)
    patched_image[top_left_y:top_left_y + patch_size, top_left_x:top_left_x + patch_size, :] = 127.5
    return patched_image