Probability and Random Processes with Applications to Signal Processing

<p>Preface </p> <p>1 Introduction to Probability 1</p> <p>1.1 Introduction: Why Study Probability? 1</p> <p>1.2 The Different Kinds of Probability 2</p> <p>Probability as Intuition 2</p> <p>Probability as the Ratio of Favorable to Total Outcomes (Classical Theory) 3</p> <p>Probability as a Measure of Frequency of Occurrence 4</p> <p>Probability Based on an Axiomatic Theory 5</p> <p>1.3 Misuses, Miscalculations, and Paradoxes in Probability 7</p> <p>1.4 Sets, Fields, and Events 8</p> <p>Examples of Sample Spaces 8</p> <p>1.5 Axiomatic Definition of Probability 15</p> <p>1.6 Joint, Conditional, and Total Probabilities; Independence 20</p> <p>Compound Experiments 23</p> <p>1.7 Bayes’ Theorem and Applications 35</p> <p>1.8 Combinatorics 38</p> <p>Occupancy Problems 42</p> <p>Extensions and Applications 46</p> <p>1.9 Bernoulli Trials–Binomial and Multinomial Probability Laws 48</p> <p>Multinomial Probability Law 54</p> <p>1.10 Asymptotic Behavior of the Binomial Law: The Poisson Law 57</p> <p>1.11 Normal Approximation to the Binomial Law 63</p> <p>Summary 65</p> <p>Problems 66</p> <p>References 77</p> <p>&nbsp;</p> <p>2 Random Variables 79</p> <p>2.1 Introduction 79</p> <p>2.2 Definition of a Random Variable 80</p> <p>2.3 Cumulative Distribution Function 83</p> <p>Properties of FX(x) 84</p> <p>Computation of FX(x) 85</p> <p>2.4 Probability Density Function (pdf) 88</p> <p>Four Other Common Density Functions 95</p> <p>More Advanced Density Functions 97</p> <p>2.5 Continuous, Discrete, and Mixed Random Variables 100</p> <p>Some Common Discrete Random Variables 102</p> <p>2.6 Conditional and Joint Distributions and Densities 107</p> <p>Properties of Joint CDF FXY (x, y) 118</p> <p>2.7 Failure Rates 137</p> <p>Summary 141</p> <p>Problems 141</p> <p>References 149</p> <p>Additional Reading 149</p> <p>&nbsp;</p> <p>3 Functions of Random Variables 151</p> <p>3.1 Introduction 151</p> <p>Functions of a Random Variable (FRV): Several Views 154</p> <p>3.2 Solving Problems of the Type Y = g(X) 155</p> <p>General Formula of Determining the pdf of Y = g(X) 166</p> <p>3.3 Solving Problems of the Type Z = g(X, Y ) 171</p> <p>3.4 Solving Problems of the Type V = g(X, Y ), W = h(X, Y ) 193</p> <p>Fundamental Problem 193</p> <p>Obtaining fVW Directly from fXY 196</p> <p>3.5 Additional Examples 200</p> <p>Summary 205</p> <p>Problems 206</p> <p>References 214</p> <p>Additional Reading 214</p> <p>&nbsp;</p> <p>4 Expectation and Moments 215</p> <p>4.1 Expected Value of a Random Variable 215</p> <p>On the Validity of Equation 4.1-8 218</p> <p>4.2 Conditional Expectations 232</p> <p>Conditional Expectation as a Random Variable 239</p> <p>4.3 Moments of Random Variables 242</p> <p>Joint Moments 246</p> <p>Properties of Uncorrelated Random Variables 248</p> <p>Jointly Gaussian Random Variables 251</p> <p>4.4 Chebyshev and Schwarz Inequalities 255</p> <p>Markov Inequality 257</p> <p>The Schwarz Inequality 258</p> <p>4.5 Moment-Generating Functions 261</p> <p>4.6 Chernoff Bound 264</p> <p>4.7 Characteristic Functions 266</p> <p>Joint Characteristic Functions 273</p> <p>The Central Limit Theorem 276</p> <p>4.8 Additional Examples 281</p> <p>Summary 283</p> <p>Problems 284</p> <p>References 293</p> <p>Additional Reading 294</p> <p>&nbsp;</p> <p>5 Random Vectors 295</p> <p>5.1 Joint Distribution and Densities 295</p> <p>5.2 Multiple Transformation of Random Variables 299</p> <p>5.3 Ordered Random Variables 302</p> <p>Distribution of area random variables 305</p> <p>5.4 Expectation Vectors and Covariance Matrices 311</p> <p>5.5 Properties of Covariance Matrices 314</p> <p>Whitening Transformation 318</p> <p>5.6 The Multidimensional Gaussian (Normal) Law 319</p> <p>5.7 Characteristic Functions of Random Vectors 328</p> <p>Properties of CF of Random Vectors 330</p> <p>The Characteristic </p>