UP | HOME

MATH 5312: Mathematical Statistics I
Fall 2016

Table of Contents


Course Syllabus

Course Schedule

The due date of HW 10 has changed to 11/29.

The instructor reserves the right to adjust this schedule in any way that serves the educational needs of the students enrolled in this course.

Week Date Topics Reading Notes
Week1 8/25 Introduction    
Week2 8/30 Probability set function HMC 1.2-1.3 HW 1 - Due 9/6 in class
Week2 9/1 No class    
Week3 9/6 Random variable, distribution function and density HMC 1.5 HW 2 - Due 9/13 in class
Week3 9/8 Distribution function, conditional distribution HMC 1.4,6  
Week4 9/13 Transformation of random variables HMC 1.7-8 HW 3 - Due 9/22 in class
Week4 9/15 Moment generating function, important inequalities HMC 1.9-10  
Week5 9/20 Review    
Week5 9/22 Midterm 1    
Week6 9/27 No Class   HW 4 - Due 10/4 in class
Week6 9/29 Distribution of two random variables HMC 2.1  
Week7 10/4 Transformation of bivariate random variables HMC 2.2 HW 5 - Due 10/11 in class
Week7 10/6 Conditional distribution, correlation coefficient HMC 2.3-4  
Week8 10/11 Independence, linear combination of random variables HMC 2.5,8 HW 6 - Due 10/20 in class
Week8 10/13 Binomial distribution HMC 3.1  
Week9 10/18 Poisson distribution HMC 3.2  
Week9 10/20 Midterm 2    
Week10 10/27 \(\Gamma\), \(\chi^{2}\), and \(\beta\) distributions HMC 3.3 HW 7 - Due 11/1 in class
Week10 10/25 Normal distribution HMC 3.4  
Week10 11/1 Multivariate normal distribution HMC 3.5 HW 8 - Due 11/8 in class
Week11 11/3 t/F-distribution HMC 3.6  
Week11 11/8 Statistical model   HW 9 - Due 11/15 in class
Week12 11/10 Statistical model & inference, introduction to estimation HMC 4.1  
Week12 11/15 Likelihood, MLE, posterior distribution HMC 4.1, 6.1 HW 10 - Due 11/29 in class
Week13 11/17 Mean, variance, MSE HMC 11.1,2  
Week13 11/22 Midterm 3    
Week14 11/24 No class   Thanksgiving holidays
Week14 11/29 Loss and risk functions HMC 7.1 HW 11 - Due 12/6 in class
Week15 12/1 Order statistics HMC 4.4  
Week15 12/6 Review    
Week16 12/13 Final exam   11-1:30pm, PKH 302

Homework

HW 1

Do Exercise 1.3.2-5,10,12


HW 2

  • Do Exercise 1.4.11,12,25
  • Do Exercise 1.5.3,5,8. Sketch graphs of CDFs.
  • Do Exercise 1.7,9,12,14
  • Prove or disprove: two events are mutually exclusive if and only if they are independent.

HW 3

  • Do Exercise 1.8.2,8,11
  • Do Exercise 1.9.4,6-8,17,18,25,26
  • Do Exercise 1.10.2,3,4,5,6

HW 4

  • Do Exercise 2.1.6,7,9,10,13-16.
  • Two random variables \(X_{1}\) and \(X_{2}\) has a joint pdf, \[ f(x_{1},x_{2}) = 4 x_{1} x_{2}, \, 0 < x_{1} < 1, \, 0 < x_{2} < 1. \] Let \(Y = X_{1}+X_{2}\). Find the CDF of \(Y\).

HW 5

  • Do Exercise 2.2.3-7.
  • Using Example 2.2.2, find the CDF of \(Y\).
    1. Use the transformation.
    2. Use the direct integration.
  • Do Exercise 2.3.1-3,5,12
  • Do Exercise 2.4.3,4

HW 6

  • Do Exercise 2.5.1,2,4,8
  • Do Exercise 2.8.2-7,10,18
  • Do Exercise 3.1.2,3.1.28
  • Do Exercise 3.1.4
    • Find \(P(\min X \ge 0.5)\) and \(P(\max X \ge 0.5)\).
  • Do Exercise 3.2.1-2,5,11-12,14

HW 7

  • Do Exercise 3.3.1-3,6,9,16-17,23-24.
  • Do Exercise 3.4.2,4,6,10,12-13,16,19,28-29,32.
  • Do Exercise 3.5.1-2,5,8,10.

HW 8

  • Do Exercise 3.6.8-10,12-14.
  • Let \(X_{1},\ldots,X_{n}\) are iid samples from N(\(\mu,\sigma^2\)). Prove \(\bar X \sim \text{N}(\mu,\sigma^2 / n)\).

HW 9

  1. Suppose \(X_{1},\ldots,X_{n}\) are iid Uniform(\(\theta\),\(\theta+1\)), \(\theta \in \mathbb{R}\).
    1. Find the method of moment estimator of \(\theta\).
    2. Consider the following data: \(3.2, 3.5, 4.0, 3.8, 3.3\). What is the estimate of \(\theta\)?
  2. Suppose \(X_{1},\ldots,X_{n}\) are iid Gamma(\(\alpha\),\(\beta\)), \((\alpha,\beta) \in (0,\infty) \times (0,\infty)\). The MGF of Gamma distribution is \(M(t) = (1-\beta t) ^{-\alpha}, \, t < 1/\beta\).
    1. Find the mean and variance of the Gamma distribution.
    2. Find the method of moments estimator of \(\alpha\) and \(\beta\).

HW 10

  1. Let \(X_{1}, X_{2}, \ldots , X_{n}\) are iid samples from the distribution with probability density function \[ f(x|\lambda)=2\lambda^{2} x^{3}e^{-\lambda x^{2}}, x>0, λ>0. \]
    1. Find the log-likelihood function (you can drop a constant term).
    2. Find the MLE of \(\lambda\).
  2. Suppose \(X_{1}, \ldots, X_{n}\) are iid Uniform(\(\theta\),\(\theta+1\)), \(\theta \in \mathbb{R}\).
    1. What is the space of \(X_{i}\)?
    2. Find the likelihood function.
    3. Consider the following data: 3.2, 3.5, 4.0, 3.8, 3.3. The likelihood is maximized for θ in an interval (so the MLE is not unique). What is the interval?
  3. Suppose that number of spam emails you received follows a Poisson process with a rate of \(\lambda\) per minute, where \(\lambda \in (0,\infty)\). That is, if \(X\) is the number of spams coming in over the course of t minutes, then \[X \sim \text{Poisson}(t\lambda).\]
    1. Assuming t is known, what is the MLE of \(\lambda\)?
    2. Assuming t is known, is \(\hat \lambda\) (MLE) unbiased? (i.e. \(E(\hat \lambda) = \lambda\)?)
    3. (Hint: First find \(\theta\) in terms of \(\lambda\).) Assuming t is known, what is the MLE of the parameter \[\theta = P_{\lambda}[\text{no spams in the next two minutes}]?\]
  4. Suppose \(X_{1}, \ldots , X_{n}\) are iid with pdf \[ f(x_{i}; \alpha, \lambda) = λ\exp[−λ(x_{i}−α)]I_{x_{i}≥α}(x_{i}), \] where \((α,λ) ∈ \mathbb{R}×(0,∞)\). Find the MLE of \((α,λ)\) when \(n\) = 4 and the data are 10,7,12,15. (Hint: First find the MLE of \(α\) for fixed \(λ\), and note that it does not depend \(λ\).)
  5. Suppose \(X | \theta \sim \text{Binomial}(100,\theta)\) and \(\theta \sim \text{Uniform(0,1)}\).
    1. Find the prior mean of \(\theta\).
    2. Find the posterior mean of \(\theta\) (Bayes estimator).
    3. Find the MSE of the posterior mean.
  6. Let \(X_{1},...,X_{n}\) be iid N\((μ,1)\), with \(μ ∈ \mathbb{R}\). A modification to least squares uses the estimator \(δ_{c}(\underline x)\) that is the value of \(μ\) that minimizes \[ l(μ; x_{1}, \ldots, x_{n})= \sum_{i=1}^{n} (x_{i} − μ)^{2} +c μ^{2}\] over \(μ\), where \(c ≥ 0\) is some fixed constant. (Adding that extra term to the sum of squares is a form of regularization that is popular in machine learning. This particular function is a special case of the objective function in ridge regression.)
    1. What is \(\delta_{c} (\underline x)\)? Is this unbiased for \(\mu\)?
    2. For which value of \(c\) is \(δ_{c}\) the MLE?
    3. For \(c > 0\), \(δ_{c} = E [ μ | \underline X = \underline x ] = \frac{n\bar x + \mu_{0} / \sigma_{0}^{2}}{n + 1/\sigma_{0}^{2}}\) is the Bayes posterior mean using the \(N(μ_{0},σ_{0}^{2})\) prior. What are \(μ_{0}\) and \(σ_{0}^{2}\)?
    4. Obatin the MSE for \(\delta_{c}\).
  7. Let \(X_{1},...,X_{n}\) be iid Poisson\((\theta)\), \(\theta >0\).

    1. Find the MLE of \(\theta\).
    2. Find the MSE of the MLE.
    3. Suppose the prior distribution on \(\theta\) is Gamma(\(\alpha, lambda\)), \(\alpha,\lambda >0\) with pdf

    \[ \pi (\theta) = \frac{\lambda^{\alpha}}{\Gamma(\alpha)} \theta^{\alpha - 1} \exp( - \lambda \theta), \, \theta >0. \] . Show that the posterior distribution of \(\theta\) is Gamma(\(\sum_{i}^{n}x_{i} + \alpha, n+\lambda\)).


HW 11

  1. Do Exercise 6.1.1, 6.1.5.
  2. Let \(X_{1},...,X_{n}\) be iid Uniform\((0,\theta)\), \(\theta \in (0, \infty)\).
    1. What is the space of \(X_{i}\)?
    2. Find the likelihood function of \(\theta\).
    3. Show that the MLE of \(\theta\) is \(\hat \theta = \max X_{i}\).
    4. Find \(E_{\theta}(\hat \theta)\). Is it unbiased? (Hint: find the probability density of \(\hat \theta\) using \(P(\max X_{i} \le x)\).)
    5. Find the risk of \(\hat \theta\) under squared-error loss when \(\theta = 1\).
  3. Suppose \(X\) and \(Y\) are independent, with \(X \sim\) Poisson(\(1 + θ\)) and \(Y \sim\) Poisson(\(1 − θ\)), where \(θ ∈ (−1, 1)\).
    1. Find constants \(a\) and \(b\) so that \(δ_{1}(x) = a + bx\) is an unbiased estimator of \(θ\). What is \(Var[δ_{1}(X)]\)?
    2. Find constants \(c\) and \(d\) so that \(δ_{2}(x,y)\) = \(cx+dy\) is an unbiased estimator of \(θ\). What is \(Var[δ_{2}(X,Y)]\)?
    3. Sketch \(MSE(θ,δ_{i})\)’s for the estimators found in parts 1) and 2). Is either one always better than the other?
  4. Suppose \(X_{1},...,X_{n}\) are iid N(\(μ,σ^{2}\)), where \((μ,σ^{2})∈R×(0,∞)\). Let \(U= \sum (X_{i} - \bar X)^{2}\), so that \(\frac{U}{\sigma^{2}} \sim \chi^{2}(n−1)\).
    1. Let \(δ_{c}(u) = cu\) be an estimator of \(σ^{2}\) for some constant c. Find the mean and variance of \(δ_c\), and show that the mean square error of \(δ_{c}\) is \[ MSE(σ^{2};δ_{c})= σ^{4} ((c(n−1)−1)^{2} +2c^{2}(n−1)). \]
    2. Find the \(c\) that minimizes the MSE in part (a).
    3. For \(c\) in part 2), find \(δ_{c}\) and \(E[δ_{c}(U)]\). Is this estimator unbiased?
  5. Suppose \(X_{1},...,X_{n}\) are iid Bernoulli(\(\theta\)), \(0 < \theta < 1\). The goal is to estimate \(g(\theta) = \theta^{2}\).
    1. Let \(\delta_{1} = X_{1} X_{2}\). Is this unbiased for \(g(\theta)\)?
    2. Let \(Y = \sum_{i=1} ^{n} X_{i}\). What is the distribution of \(Y\)?
    3. Let \(\delta_{2} = (Y^{2} - Y) / ( n(n-1))\). Is this unbiased for \(g(\theta)\)?
    4. Find the variance of \(\delta_{1}\) and \(\delta_{2}\).
    5. In the light of MSE, which estimator is better?
  6. Let \(X_{1},...,X_{n}\) be iid Poisson\((\theta)\). The prior on \(\theta\) is Gamma(\(\alpha, \lambda\)) with pdf \[ \pi (\theta) = \frac{\lambda^{\alpha}}{\Gamma(\alpha)} \theta^{\alpha - 1} \exp( - \lambda \theta), \, \theta >0. \] The posterior \(\pi(\theta | \underline{x})\) is given in the last HW. Suppose the loss function is \(L_{2}(\theta ,\delta) = (\theta - \delta)^{2}/ \theta\). This loss function may be more appropriate for the estimation of a variance of \(X_{i}\).
    1. Given \(\underline{X} = \underline{x}\), \(\delta(\underline{X})\) becomes \(\delta(\underline{x})\) (constant). The posterior risk is \(E[L_{2}(\theta ,\delta(\underline{X})) | \underline{X} = \underline{x}] = A - B \delta(\underline{x}) + C \delta(\underline{x}) ^{2}\). What are \(A\),\(B\), and \(C\)?
    2. Find the Bayes estimator that minimizes the risk above.

Exams

Exams will be closed book, open calculators. Calculators may not be shared on exams. Cell phone calculators are not permitted on exams. A single page (on US letter size paper), hand-written cheat-sheet is permitted for each exam.