Occlusion sensitivity

OcclusionSensitivity

Bases: XAImethod_Base

XAI Method for Occlusion Sensitivity.

Normally, this class is used internally in the aucmedi.xai.decoder.xai_decoder in the AUCMEDI XAI module.

Reference - Implementation

Author: Raphael Meudec
GitHub Profile: https://gist.github.com/RaphaelMeudec
Date: Jul 18, 2019
https://gist.github.com/RaphaelMeudec/7985b0c5eb720a29021d52b0a0be549a

This class provides functionality for running the compute_heatmap function, which computes a Occlusion Sensitivity Map for an image with a model.

Source code in aucmedi/xai/methods/occlusion_sensitivity.py

class OcclusionSensitivity(XAImethod_Base):
    """ XAI Method for Occlusion Sensitivity.

    Normally, this class is used internally in the [aucmedi.xai.decoder.xai_decoder][] in the AUCMEDI XAI module.

    ??? abstract "Reference - Implementation"
        Author: Raphael Meudec <br>
        GitHub Profile: [https://gist.github.com/RaphaelMeudec](https://gist.github.com/RaphaelMeudec) <br>
        Date: Jul 18, 2019 <br>
        [https://gist.github.com/RaphaelMeudec/7985b0c5eb720a29021d52b0a0be549a](https://gist.github.com/RaphaelMeudec/7985b0c5eb720a29021d52b0a0be549a) <br>

    This class provides functionality for running the compute_heatmap function,
    which computes a Occlusion Sensitivity Map for an image with a model.
    """
    def __init__(self, model, layerName=None, patch_size=16):
        """ Initialization function for creating a Occlusion Sensitivity Map as XAI Method object.

        Args:
            model (keras.model):            Keras model object.
            layerName (str):                Not required in Occlusion Sensitivity Maps, but defined by Abstract Base Class.
        """
        # Cache class parameters
        self.model = model
        self.patch_size = patch_size

    #---------------------------------------------#
    #             Heatmap Computation             #
    #---------------------------------------------#
    def compute_heatmap(self, image, class_index, eps=1e-8):
        """ Core function for computing the Occlusion Sensitivity Map for a provided image and for specific classification outcome.

        ???+ attention
            Be aware that the image has to be provided in batch format.

        Args:
            image (numpy.ndarray):              Image matrix encoded as NumPy Array (provided as one-element batch).
            class_index (int):                  Classification index for which the heatmap should be computed.
            eps (float):                        Epsilon for rounding.

        The returned heatmap is encoded within a range of [0,1]

        ???+ attention
            The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

        Returns:
            heatmap (numpy.ndarray):            Computed Occlusion Sensitivity Map for provided image.
        """
        # Utilize only image matrix instead of batch
        image = image[0]
        # Create empty sensitivity map
        sensitivity_map = np.zeros((image.shape[0], image.shape[1]))
        # Iterate the patch over the image
        for top_left_x in range(0, image.shape[0], self.patch_size):
            for top_left_y in range(0, image.shape[1], self.patch_size):
                patch = apply_grey_patch(image, top_left_x, top_left_y,
                                         self.patch_size)
                prediction = self.model.predict(np.array([patch]))[0]
                confidence = prediction[class_index]

                # Save confidence for this specific patch in the map
                sensitivity_map[
                    top_left_y:top_left_y + self.patch_size,
                    top_left_x:top_left_x + self.patch_size,
                ] = 1 - confidence
        # Return the resulting sensitivity map (automatically a heatmap)
        return sensitivity_map

__init__(model, layerName=None, patch_size=16)

Initialization function for creating a Occlusion Sensitivity Map as XAI Method object.

Parameters:

Name	Type	Description	Default
model	keras.model	Keras model object.	required
layerName	str	Not required in Occlusion Sensitivity Maps, but defined by Abstract Base Class.	None

Source code in aucmedi/xai/methods/occlusion_sensitivity.py

def __init__(self, model, layerName=None, patch_size=16):
    """ Initialization function for creating a Occlusion Sensitivity Map as XAI Method object.

    Args:
        model (keras.model):            Keras model object.
        layerName (str):                Not required in Occlusion Sensitivity Maps, but defined by Abstract Base Class.
    """
    # Cache class parameters
    self.model = model
    self.patch_size = patch_size

compute_heatmap(image, class_index, eps=1e-08)

Core function for computing the Occlusion Sensitivity Map for a provided image and for specific classification outcome.

Attention

Be aware that the image has to be provided in batch format.

Parameters:

Name	Type	Description	Default
image	numpy.ndarray	Image matrix encoded as NumPy Array (provided as one-element batch).	required
class_index	int	Classification index for which the heatmap should be computed.	required
eps	float	Epsilon for rounding.	1e-08

The returned heatmap is encoded within a range of [0,1]

Attention

The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

Returns:

Name	Type	Description
heatmap	numpy.ndarray	Computed Occlusion Sensitivity Map for provided image.

Source code in aucmedi/xai/methods/occlusion_sensitivity.py

def compute_heatmap(self, image, class_index, eps=1e-8):
    """ Core function for computing the Occlusion Sensitivity Map for a provided image and for specific classification outcome.

    ???+ attention
        Be aware that the image has to be provided in batch format.

    Args:
        image (numpy.ndarray):              Image matrix encoded as NumPy Array (provided as one-element batch).
        class_index (int):                  Classification index for which the heatmap should be computed.
        eps (float):                        Epsilon for rounding.

    The returned heatmap is encoded within a range of [0,1]

    ???+ attention
        The shape of the returned heatmap is 2D -> batch and channel axis will be removed.

    Returns:
        heatmap (numpy.ndarray):            Computed Occlusion Sensitivity Map for provided image.
    """
    # Utilize only image matrix instead of batch
    image = image[0]
    # Create empty sensitivity map
    sensitivity_map = np.zeros((image.shape[0], image.shape[1]))
    # Iterate the patch over the image
    for top_left_x in range(0, image.shape[0], self.patch_size):
        for top_left_y in range(0, image.shape[1], self.patch_size):
            patch = apply_grey_patch(image, top_left_x, top_left_y,
                                     self.patch_size)
            prediction = self.model.predict(np.array([patch]))[0]
            confidence = prediction[class_index]

            # Save confidence for this specific patch in the map
            sensitivity_map[
                top_left_y:top_left_y + self.patch_size,
                top_left_x:top_left_x + self.patch_size,
            ] = 1 - confidence
    # Return the resulting sensitivity map (automatically a heatmap)
    return sensitivity_map

apply_grey_patch(image, top_left_x, top_left_y, patch_size)

Internal function.

Replace a part of the image with a grey patch.

Parameters:

Name	Type	Description	Default
image	numpy.ndarray	Input image	required
top_left_x	int	Top Left X position of the applied box	required
top_left_y	int	Top Left Y position of the applied box	required
patch_size	int	Size of patch to apply	required

Returns:

Name	Type	Description
patched_image	numpy.ndarray	Patched image

Source code in aucmedi/xai/methods/occlusion_sensitivity.py

def apply_grey_patch(image, top_left_x, top_left_y, patch_size):
    """ Internal function.

    Replace a part of the image with a grey patch.

    Args:
        image (numpy.ndarray):                  Input image
        top_left_x (int):                       Top Left X position of the applied box
        top_left_y (int):                       Top Left Y position of the applied box
        patch_size (int):                       Size of patch to apply

    Returns:
        patched_image (numpy.ndarray):          Patched image
    """
    patched_image = np.array(image, copy=True)
    patched_image[top_left_y:top_left_y + patch_size, top_left_x:top_left_x + patch_size, :] = 127.5
    return patched_image