Preface Notation 1 Introduction 1.1 Motivational Examples 1.2 Related Literature 1.3 Outline 2 Control Models and Ergodicity 2.1 The Controlled System 2.2 Control Policies and Extended Generator 2.3 Positive Recurrence and Ergodicity 2.4 Conclusions 3 Basic Optimality Criteria 3.1 Discounted Reward Criterion 3.2 An LQ Example:Discounted Case 3.3 A Pollution Accumulation Example 3.4 Ergodic Reward Criterion 3.5 An LQ System Example:Average Case 3.6 A Pollution Accumulation Example 3.7 Conclusions 4 Bias Optimality 4.1 Preliminaries 4.2 Existence of Bias Optimal Policies 4.3 Overtaking Optimality 4.4 Asymptotic Rate of Average Optimal Policies 4.5 A Pollution Accumulation Example 4.6 Conclusions 5 Sensitive-Discount Optimality 5.1 m-Discount Optimality 5.2 Poisson Equation and Average Reward HJB Equation 5.3 Laurent Series 5.4 Existence and Characterizations of m-Discount Optimal Policies 5.5 Special Cases:m=-1 and m=0 5.6 Blackwell Optimality 5.7 A Pollution Accumulation Example 5.8 Conclusions 6 Concluding Remarks References A Technical Complements Index