Example of convex activation function

For a particular task, I need a convex activation function with the following properties:

  • f''(x) > 0
  • 0 = f(x) = 1
  • f(x) is monotonic
  • f(x) is not "exploding" i.e. avoiding functions such as f(x) = x²

The only example I have in mind for this is the softplus activation function. Would you have anything else in mind?

Topic activation-function neural-network machine-learning

Category Data Science

1
answered at

There is no such function.

A convex function defined on all of $\mathbb{R}$ with bounded range is constant.

Softplus is not bounded above as required by your second bullet.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.