Contents Preface Chapter 1 Generation of Random Variables 1 1.1 The Inversion Method 3 1.1.1 Generating samples from continuous distributions 3 1.1.2 Generating samples from discrete distributions 7 1.2 The Grid Method 12 1.3 The Rejection Method 15 1.3.1 Generating samples from continuous distributions 15 1.3.2 The efficiency of the rejection method 18 1.3.3 Several examples 20 1.3.4 Log-concave densities 24 1.4 The Sampling/Importance Resampling (SIR) Method 27 1.4.1 The SIR without replacement 28 1.4.2 Theoretical justification 30 1.5 The Stochastic Representation (SR) Method.32 1.5.1 The‘d=’operator 32 1.5.2 Many-to-one SR for univariate case 34 1.5.3 SR for multivariate case 36 1.5.4 Mixture representation 39 1.6 The Conditional Sampling Method 42 Exercise 1 47 Chapter 2 Optimization 53 2.1 A Review of Some Standard Concepts 54 2.1.1 Order relations 54 2.1.2 Stationary points 57 2.1.3 Convex and concave functions 60 2.1.4 Mean value theorem 61 2.1.5 Taylor theorem 63 2.1.6 Rates of convergence 64 2.1.7 The case of multiple dimensions 64 2.2 Newton’s Method and Its Variants 66 2.2.1 Newton’s method and root finding 67 2.2.2 Newton’s method and optimization 71 2.2.3 The Newton–Raphson algorithm 72 2.2.4 The Fisher scoring algorithm 75 2.2.5 Application to logistic regression 76 2.3 The Expectation–Maximization (EM) Algorithm 80 2.3.1 The formulation of the EM algorithm 81 2.3.2 The ascent property of the EM algorithm 89 2.3.3 Missing information principle and standard errors 92 2.4 The ECM Algorithm 95 2.5 Minorization–Maximization (MM) Algorithms 100 2.5.1 A brief review of MM algorithms 100 2.5.2 The MM idea 101 2.5.3 The quadratic lower–bound algorithm 103 2.5.4 The De Pierro algorithm 106 Exercise 2 115 Chapter 3 Integration 125 3.1 Laplace Approximations 126 3.2 Riemannian Simulation 129 3.2.1 Classical Monte Carlo integration 129 3.2.2 Motivation for Riemannian simulation 132 3.2.3 Variance of the Riemannian sum estimator 133 3.3 The Importance Sampling Method 135 3.3.1 The formulation of the importance sampling method 135 3.3.2 The weighted estimator 138 3.4 Variance Reduction 141 3.4.1 Antithetic variables 141 3.4.2 Control variables 145 Exercise 3 146 Chapter 4 Markov Chain Monte Carlo Methods 149 4.1 Bayes Formulae and Inverse Bayes Formulae (IBF) 151 4.1.1 The point,function- and sampling-wise IBF 152 4.1.2 Monte Carlo versions of the IBF 160 4.1.3 Generalization to the case of three random variables 163 4.2 The Bayesian Methodology 163 4.2.1 The posterior distribution 165 4.2.2 Nuisance parameters 167 4.2.3 Posterior predictive distribution 169 4.2.4 Bayes factor 172 4.2.5 Estimation of marginal likelihood 173 4.3 The Data Augmentation (DA) Algorithm 175 4.3.1 Missing data mechanism 175 4.3.2 The idea of data augmentation 177 4.3.3 The original DA algorithm 178 4.3.4 Connection with the IBF 180 4.4 The Gibbs sampler 181 4.4.1 The formulation of the Gibbs sampling 182 4.4.2 The two–block Gibbs sampling 184 4.5 The Exact IBF Sampling 187 4.6 The IBF sampler 191 4.6.1 Background and the basic idea 191 4.6.2 The formulation of the IBF sampler 192 4.6.3 Theoretical justification for choosing θ0 =.θ 194 Exercise 4 196 Chapter 5 Bootstrap Methods 203 5.1 Bootstrap Confidence Intervals 203 5.1.1 Parametric bootstrap 203 5.1.2 Non-parametric bootstrap 213 5.2 Hypothesis Testing with the Bootstrap 219 5.2.1 Testing equality of two unknown distributions 219 5.2.2 Testing equality of two group means 223 5.2.3 One–sample problem 228 Exercise 5 231 Appendix A Some Statistical Distributions and Stochastic Processes 233 A.1 Discrete Distributions 233 A.1.1 Finite discrete distribution 233 A.1.2 Hypergeometric distribution 234 A.1.3 Binomial and related distributions 235 A.1.4 Poisson and related distributions 237 A.1.5 Negative–binomial and related distributions 240 A.1.6 Generalized Poisson and related distributions 242 A.1.7 Multinomial and related distributions 243 A.2 Continuous Distributions 245 A.2.1 Uniform, beta and Dirichlet distributions 245 A.2.2 Logistic and Laplace distributions 248 A.2.3 Exponential, gamma and inverse gamma distributions 249 A.2.4 Chi-square, F and inverse chi-square distributions 251 A.2.5 Normal, lognormal and inverse Gaussian distributions 252 A.2.6 Multivariate normal distribution 254 A.2.7 Student’s t and multivariate t distributions 255 A.2.8 Wishart and inverse Wishart distributions 256 A.3 Stochastic Processes 258 A.3.1 Homogeneous Poisson process 258 A.3.2 Nonhomogeneous Poisson process 259 Appendix B R Programming 261 B.1 Basic Commands 262 B.1.1 Expressions 262 B.1.2 Assignment operator 266 B.2 Vectors and Matrices 268 B.2.1 Vectors 268 B.2.2 Matrices 276 B.3 Lists, Data Frames and Arrays 284 B.3.1 Lists 284 B.3.2 Data frames 287 B.3.3 Arrays 291 B.4 Flow Control 292 B.5 User Functions 294 B.6 Some Commonly–Used R Functions for Data Analysis 295 Appendix C Introduction of Latent Variables Methods 301 C.1 MLEs of Parameters in t Distribution 301 C.2 MLEs of Parameters in the Poisson Additive Model 305 C.3 MLEs of Parameters in Constrained Normal Models 308 C.4 Binormal Model with Missing Data 312 List of Figures 315 List of Tables 317 List of Acronyms 319 List of Symbols 321 References 325 Subject Index 333
京公网安备 11010102004214号