On the Principle of Optimality for Nonstationary Deterministic Dynamic Programming


This note studies a general nonstationary infinite-horizon optimization problem in discrete time. We allow the state space in each period to be an arbitrary set, and the return function in each period to be unbounded. We do not require discounting, and do not require the constraint correspondence in each period to be nonempty-valued. The objective function is defined as the limit superior or inferior of the finite sums of return functions. We show that the sequence of time-indexed value functions satisfies the Bellman equation if and only if its right-hand side is well defined, i.e., it does not involve -∞+∞.

Keywords: Bellman equation, dynamic programming, principle of optimality, value function
JEL Classification: C61, O41

Research Institute for Economics and Business Administration
Kobe University
Rokkodai-cho, Nada-ku, Kobe
657-8501 Japan
Phone: (81) 78 803 7036
Fax: (81) 78 803 7059