I created a neural network with multi-label classification using MSE. Now, I would like to use Mixup. Do I need two loss functions (for each target one) or is the result the same if I just combine the two targets like this?
target = t * target1 + (1-t)* target2
Topic data-augmentation tensorflow loss-function
Category Data Science
In their paper, mixup: Beyong Empirical Risk Mininization, the authors provide a piece of code how to train an epoch in PyTorch:
# y1, y2 should be one-hot vectors
for (x1, y1), (x2, y2) in zip(loader1, loader2):
lam = numpy.random.beta(alpha, alpha)
x = Variable(lam * x1 + (1. - lam) * x2)
y = Variable(lam * y1 + (1. - lam) * y2)
optimizer.zero_grad()
loss(net(x), y).backward()
optimizer.step()
As you can see, they only use a single loss function defined on the new combined target variable y or, in your terminology, target.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.