Currently, I am learning about Bellman Operator in Dynamic Programming and Reinforcement Learning. I would like to know why is Bellman operator contraction with respect to infinity norm? Why not another norm e.g. Euclidian norm?
Topic dynamic-programming reinforcement-learning
Category Data Science
The proofs I have seen for contraction of the bellman operator (this page has a really nice run through, explicitly utilize the fact that an infinity norm is being used. This does not necessarily mean that contraction doesn't occur under other norms, but it does suggest one of two possibilities:
When showing that the Bellman Operator converges to a fixed point it is satisfactory to simply show that it is a contraction, it doesn't matter what sort of contraction it is, so we would typically prove the contraction that is easiest to show.
That being said, intuitively, I would imagine that the question of whether or not this contraction holds under other norms would impact questions surrounding the rate of convergence to the fixed point (ie., how many times we must apply the bellman operator for convergence). After a cursory search I was not able to find any theory explicitly proving or disproving the contraction under other norms. This might be an interesting question to raise in the mathematics stack exchange.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.