how to implement squared hinge loss in pytorch

does anyone have any advice on how to implement this loss in order to use it with a convolutional neural network? Also, how should I encode the labels of my training data? We were using one hot encoding with bce loss before and I was wandering if I should keep it that way also for the hinge loss, since the label itself is not used in the formula of the loss other than for indicating which one is the true class. The dataset is CIFAR100 by the way. Thank you in advance!

EDIT: I implemented a version of this loss, the problem is that after the first epoch the loss is always zero and so the training doesn't go further. Here is the code:

class MultiClassSquaredHingeLoss(nn.Module):
    def __init__(self):
        super(MultiClassSquaredHingeLoss, self).__init__()

    def forward(self, output, y): #output: batchsize*n_class
        n_class = y.size(1)
        #margin = 1 
        margin = 1
        #isolate the score for the true class
        y_out = torch.sum(torch.mul(output, y)).cuda()
        output_y = torch.mul(torch.ones(n_class).cuda(), y_out).cuda()
        #create an opposite to the one hot encoded tensor
        anti_y = torch.ones(n_class).cuda() - y.cuda()
        
        loss = output.cuda() - output_y.cuda() + margin
        loss = loss.cuda()
        #remove the element of the loss corresponding to the true class
        loss = torch.mul(loss.cuda(), anti_y.cuda()).cuda()
        #max(0,_)
        loss = torch.max(loss.cuda(), torch.zeros(n_class).cuda())
        #squared hinge loss
        loss = torch.pow(loss, 2).cuda()
        #sum up
        loss = torch.sum(loss).cuda()
        loss = loss / n_class        
        
        return loss

Topic pytorch cnn torch hinge-loss loss-function

Category Data Science


One option is to use the existing torch.nn.MultiMarginLoss. For squared loss, set p=2.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.