How to force a NN to ouput the same output given a reverse input?

I want to choose an architecture that can deal with an input symmetry.

As input, I have a sequence of zeros and ones, like [1, 1, 1, 0, 1, 0] and at the output layer I have N neurons that outputs a categorical distribution like [0.3, 0.4, 0.3].

How can I force a NN to ouput the same distribution when I feed its reverse copy, i.e [1, 1, 1, 0, 1, 0]?

A simple way just to learn twice:

feed straight [1, 1, 1, 0, 1, 0] - [0.3, 0.4, 0.3]
feed reverse  [0, 1, 0, 1, 1, 1] - [0.3, 0.4, 0.3]

Or maybe, there are more elegant ways? What type of architecture I should use or maybe I need to play with loss functuions?

Topic weight-initialization machine-learning-model deep-learning

Category Data Science

1
answered at

The most general and robust way is to double the training set using the "mirror image" of each sample. Also it is sound approach from a statistical learning perspective.

There can be some approaches to symmetry where one hacks the NN architecture (eg by symmetrising the network) but these are usualy very fragile and do not generalise well, while the above approach is robust and lets the network adjust itself as it needs.

A similar but cheaper approach is to sort the samples before feeding them to the network. Eg for your example:

If you have this input [1, 1, 1, 0, 1, 0] then the sorted related mirror input (in lexicographic order) is the [0, 1, 0, 1, 1, 1] (reverse). So you train the NN only the sorted lexicographic inputs and before testing each sample you convert it to the sorted lexicographic related input (by reversing it) if it is not already in that form.

So effectively you normalise the samples (by reversing them if necessary) both before training and before testing. This is similar approach to doubling the train set, only it is cheaper (and in practice exactly accurate, while doubling the training set is only theoreticaly accurate, in practice it may have discrepancies).

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.