Can the size of a pooling layer be learned?

As far as I understood it, the pooling layer doesn't learn anything. It has several parameters, most important its pool_size and stride (see Lasagne documentation for more), but none of those is learned.

Is it possible to learn these two (or one of those) parameters? Are there papers about it?

(I would guess that it is not possible to add this to the objective function in a meaningful way... but I'd like to see if people thought about it.)

Topic convnet neural-network machine-learning

Category Data Science

1
answered at

No it is not possible to learn those meta parameters from a data set using a learning algorithm. There is no way to calculate a gradient of any objective function with respect to pool_size and stride params. Even if there were, they are typically discrete values and relatively small integers, so cannot be updated in the same way as e.g. the weights are.

What you can do is search through different values looking for optimal ones, using cross-validation to pick the best. The size and stride meta parameters can also have a large effect on memory use and speed of learning, so it may be the case you are looking to find a good compromise. With stride in lower layers for instance, it may be more effective (in terms of training time required for similar level of accuracy) to have a higher stride value but increase the overall training set size by adding random x/y translations to the training set images.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.