This paper investigates the solutions to the functional equations that arise inter alia in Undiscounted Markov Renewal Programming. We show that the solution set is a connected, though possibly nonconvex set whose members are unique up to the n* constants, characterize n* and show that some of these n* degrees of freedom are locally rather than globally independent.
Our results generalize those obtained in Romanovsky where another approach is followed for a special class of discrete time Markov Decision Processes. Basically our methods involve the set of randomized policies. We first study the sets of pure and randomized maximal gain policies, as well as the set of states that are recurrent under some maximal gain policy.
Schweitzer, Paul, and Awi Federgruen. "The functional equations of undiscounted Markov renewal programming." Mathematics of Operations Research 3, no. 4 (November 1978): 308-321.
Each author name for a Columbia Business School faculty member is linked to a faculty research page, which lists additional publications by that faculty member.
Each topic is linked to an index of publications on that topic.