Both of the cost models presented in this section were designed to force the cumulative cost to become finite, even though there is an infinite number of stages. Each can be considered as a minor adaptation of cost functional used in Formulation 10.1.

The following formulation will be used throughout Section 10.3.

- A nonempty, finite
*state space*. - For each state , a finite
*action space*(there is no termination action, contrary to Formulation 10.1). - A finite
*nature action space*for each and . - A
*state transition function*that produces a state, , for every , , and . - A set of
*stages*, each denoted by , that begins at and continues indefinitely. - A stage-additive cost functional, , in which , , and are infinite state, action, and nature histories, respectively. Two alternative forms of will be given shortly.

As a possible application, imagine a robot that delivers materials in a factory from several possible source locations to several destinations. The robot operates over a long work shift and has a probabilistic model of when requests to deliver materials will arrive. Formulation 10.2 can be used to define a problem in which the goal is to minimize the average amount of time that materials wait to be delivered. This strategy should not depend on the length of the shift; therefore, an infinite number of stages is reasonable. If the shift is too short, the robot may focus only on one delivery, or it may not even have enough time to accomplish that.