_focal_loss

`focal_loss` ¶

Calculate the focal loss between two tensors.

Original implementation from https://github.com/facebookresearch/fvcore/blob/master/fvcore/nn/focal_loss.py . Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.

This method can be used with TensorFlow tensors:

true = tf.constant([[1], [1], [1], [0], [0], [0]])
pred = tf.constant([[0.97], [0.91], [0.73], [0.27], [0.09], [0.03]])
b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=None, alpha=None) #0.1464
b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=2.0, alpha=0.25) #0.00395

This method can be used with PyTorch tensors:

true = torch.tensor([[1], [1], [1], [0], [0], [0]])
pred = torch.tensor([[0.97], [0.91], [0.73], [0.27], [0.09], [0.03]])
b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=None, alpha=None) #0.1464
b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=2.0, alpha=0.25) #0.004

Parameters:

Name	Type	Description	Default
`y_true`	`Tensor`	Ground truth class labels with shape([batch_size, d0, .. dN]), which should take values of 1 or 0.	required
`y_pred`	`Tensor`	Prediction score for each class, with a shape like y_true. dtype: float32 or float16.	required
`alpha`	`float`	Weighting factor in range (0,1) to balance positive vs negative examples or (-1/None) to ignore. Default = 0.25	`0.25`
`gamma`	`float`	Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples.	`2.0`
`normalize`	`bool`	Whether to normalize focal loss along samples based on number of positive classes per samples.	`True`
`from_logits`	`bool`	Whether y_pred is logits (without sigmoid).	`False`
`reduction`	`str`	'none' \| 'mean' \| 'sum' 'none': No reduction will be applied to the output. 'mean': The output will be averaged. 'sum': The output will be summed.	`'mean'`

Returns:

Type	Description
`Tensor`	The Focal loss between `y_true` and `y_pred`

Raises:

Type	Description
`ValueError`	If `y_pred` or 'y_true' is an unacceptable data type.

Source code in fastestimator\fastestimator\backend\_focal_loss.py

def focal_loss(y_true: Tensor,
               y_pred: Tensor,
               gamma: float = 2.0,
               alpha: float = 0.25,
               from_logits: bool = False,
               normalize: bool = True,
               reduction: str = "mean") -> Tensor:
    """Calculate the focal loss between two tensors.

    Original implementation from https://github.com/facebookresearch/fvcore/blob/master/fvcore/nn/focal_loss.py .
    Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.

    This method can be used with TensorFlow tensors:
    ```python
    true = tf.constant([[1], [1], [1], [0], [0], [0]])
    pred = tf.constant([[0.97], [0.91], [0.73], [0.27], [0.09], [0.03]])
    b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=None, alpha=None) #0.1464
    b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=2.0, alpha=0.25) #0.00395
    ```

    This method can be used with PyTorch tensors:
    ```python
    true = torch.tensor([[1], [1], [1], [0], [0], [0]])
    pred = torch.tensor([[0.97], [0.91], [0.73], [0.27], [0.09], [0.03]])
    b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=None, alpha=None) #0.1464
    b = fe.backend.focal_loss(y_pred=pred, y_true=true, gamma=2.0, alpha=0.25) #0.004
    ```

    Args:
        y_true: Ground truth class labels with shape([batch_size, d0, .. dN]), which should take values of 1 or 0.
        y_pred: Prediction score for each class, with a shape like y_true. dtype: float32 or float16.
        alpha: Weighting factor in range (0,1) to balance
                positive vs negative examples or (-1/None) to ignore. Default = 0.25
        gamma: Exponent of the modulating factor (1 - p_t) to
               balance easy vs hard examples.
        normalize: Whether to normalize focal loss along samples based on number of positive classes per samples.
        from_logits: Whether y_pred is logits (without sigmoid).
        reduction: 'none' | 'mean' | 'sum'
                 'none': No reduction will be applied to the output.
                 'mean': The output will be averaged.
                 'sum': The output will be summed.
    Returns:
        The Focal loss between `y_true` and `y_pred`

    Raises:
        ValueError: If `y_pred` or 'y_true' is an unacceptable data type.
    """
    if gamma is None or gamma < 0:
        raise ValueError("Value of gamma should be greater than or equal to zero.")

    if alpha is None or (alpha < 0 or alpha > 1):
        raise ValueError("Value of alpha can either be -1 or None or within range (0, 1)")

    if tf.is_tensor(y_true):
        y_true = tf.cast(y_true, dtype=y_pred.dtype)
        focal_loss = SigmoidFocalCrossEntropy(from_logits=from_logits,
                                              alpha=alpha,
                                              gamma=gamma,
                                              reduction=tf.keras.losses.Reduction.NONE)(y_pred=y_pred, y_true=y_true)
    elif isinstance(y_true, torch.Tensor):
        y_true = y_true.to(y_pred.dtype)
        focal_loss = pytorch_focal_loss(y_pred=y_pred, y_true=y_true, alpha=alpha, gamma=gamma, from_logits=from_logits)
    else:
        raise ValueError("Unsupported tensor type.")

    # normalize along the batch size based on number of positive classes
    if normalize:
        focal_reduce_axis = [*range(len(focal_loss.shape))][1:]
        focal_loss = reduce_sum(focal_loss, axis=focal_reduce_axis)

        gt_reduce_axis = [*range(len(y_true.shape))][1:]
        gt_count = clip_by_value(reduce_sum(y_true, axis=gt_reduce_axis), min_value=1)
        focal_loss = focal_loss / gt_count

    if reduction == "mean":
        focal_loss = reduce_mean(focal_loss)
    elif reduction == "sum":
        focal_loss = reduce_sum(focal_loss)

    return focal_loss

`pytorch_focal_loss` ¶