does anyone have any advice on how to implement this loss in order to use it with a convolutional neural network? Also, how should I encode the labels of my training data? We were using one hot encoding with bce loss before and I was wandering if I should keep it that way also for the hinge loss, since the label itself is not used in the formula of the loss other than for indicating which one is the true class. The dataset is CIFAR100 by the way. Thank you in advance!
EDIT: I implemented a version of this loss, the problem is that after the first epoch the loss is always zero and so the training doesn't go further. Here is the code:
class MultiClassSquaredHingeLoss(nn.Module):
def __init__(self):
super(MultiClassSquaredHingeLoss, self).__init__()
def forward(self, output, y): #output: batchsize*n_class
n_class = y.size(1)
#margin = 1
margin = 1
#isolate the score for the true class
y_out = torch.sum(torch.mul(output, y)).cuda()
output_y = torch.mul(torch.ones(n_class).cuda(), y_out).cuda()
#create an opposite to the one hot encoded tensor
anti_y = torch.ones(n_class).cuda() - y.cuda()
loss = output.cuda() - output_y.cuda() + margin
loss = loss.cuda()
#remove the element of the loss corresponding to the true class
loss = torch.mul(loss.cuda(), anti_y.cuda()).cuda()
#max(0,_)
loss = torch.max(loss.cuda(), torch.zeros(n_class).cuda())
#squared hinge loss
loss = torch.pow(loss, 2).cuda()
#sum up
loss = torch.sum(loss).cuda()
loss = loss / n_class
return loss
