\chapter{General results for multi-parameter problems}
In this chapter we will study some general results for multi-parameter
problems.

\section{Different levels of  prior knowledge}
We have {\it substantial prior information} for $\thetavec$ when the
prior distribution {\it dominates} the posterior distribution, that is
$\pi(\thetavec|\xvec)\sim\pi(\thetavec)$. 

When prior information about $\thetavec$ is limited, this is usually
represented through the use of a conjugate prior distribution, with
vague prior knowledge represented by making the conjugate distribution
as diffuse as possible.

If we represent prior ignorance for a single parameter $\theta$ by using uniform or improper priors then we have seen (MAS2903, section 3.4) that, in general, the prior for $g(\theta)$ is not constant and so we are not ignorant about $g(\theta)$.  The same problem occurs when we have more than one parameter.  

Suppose we represent prior ignorance about $\thetavec = (\theta_1, \theta_2, \ldots, \theta_p)^{T}$ using $\pi(\thetavec) = \textit{constant}$.  Let $\phi_{i} = g_{i}(\thetavec)$, $i=1, \ldots, p$ and $\phivec = (\phi_1, \ldots, \phi_{p})^{T}$ be a 1--1  transformation.  Then, in general, the prior density for $\phivec$ is not constant and this suggests that we are not ignorant about $\phivec$.  However, if we are ignorant about $\thetavec$ then we must also be ignorant about $g(\thetavec)$.  

This contradiction makes it impossible to use this representation of prior ignorance.  

%%Vspace hack
\newpage
\begin{example}~\\
Suppose $0<\theta_1<1$ and $0<\theta_2<1$.  If we are ignorant about $\thetavec = (\theta_1, \theta_2)^{T}$ then show that $\theta_1 \theta_2$ does not have a constant prior density.  


\framebox[17cm]{ \parbox[t]{16cm}{

\vspace{0.4cm}

\huge
\ding{46} \normalsize 

\textcolor{blue}{
\vspace{18cm}
}
%\begin{soln}
}}
%\end{soln}
\end{example}

\newpage

\begin{example}~\\
Suppose we have a random sample from a $N(\mu, 1/\tau)$ distribution (with $\tau$ unknown).  Determine the Jeffreys prior for this model.

Hint: We have already seen that the likelihood function can be written as
\[
f(\xvec|\mu, \tau) = \left(\frac{\tau}{2\pi}\right)^{n/2}\exp\left[-\frac{n\tau}{2}\left\{s^2 + (\bar{x}-\mu)^2\right\}\right]
\]
where
\[
s^2 = \frac{1}{n} \sum_{i=1}^{n}(x_{i}-\bar{x})^2.
\]
%\begin{soln}
%\vspace{10cm}
%\end{soln}
\framebox[17cm]{ \parbox[t]{16cm}{

\vspace{0.4cm}

\huge
\ding{46} \normalsize 

\vspace{17cm}
}}

\newpage

\framebox[17cm]{ \parbox[t]{16cm}{

\vspace{0.4cm}

\huge
\ding{46} \normalsize \textit{...Solution to Example 3.2}

\vspace{23cm}

}}

\newpage
\framebox[17cm]{ \parbox[t]{16cm}{

\vspace{0.4cm}

\huge
\ding{46} \normalsize \textit{...Solution to Example 3.2}

\vspace{23cm}

}}


\end{example}
\newpage


\section{Asymptotic posterior distribution}
Suppose we have a statistical model for data with likelihood function
$f(\xvec|\thetavec)$, where $\xvece$ and $\thetavece$, together with a
prior distribution with density $\pi(\thetavec)$ for $\thetavec$. Then
\begin{align*}
&J(\thetavechat)^{1/2}(\thetavec-\thetavechat)|\xvec
\stackrel{\cal D} \longrightarrow N_p(\vec{0},I_p)
\quad\quad\text{as }n\to\infty, \\
\intertext{where $\thetavechat$ is the likelihood mode, $I_p$ is the
$p\times p$ identity matrix and $J(\thetavec)$ is the observed
information matrix, with $(i,j)^{th}$ element}
&J_{ij}=-\frac{\partial^2}{\partial\theta_i\partial\theta_j}\,
\log f(\xvec|\thetavec), \\
\intertext{and $A^{1/2}$ denotes the square root matrix of $A$.}
\end{align*}

\subsection*{Comments}
%\vspace*{-2cm}~
%\begin{ggap}

\textcolor{blue}{
\begin{enumerate}
\item This asymptotic result can give us a useful approximation to the
posterior distribution for $\thetavec$ when $n$ is large:
\[
\thetavec|\xvec\sim N_p\left(\thetavechat,J(\thetavechat)^{-1}\right)
\quad\quad\text{approximately}.
\]
\item This limiting result is similar to one for the maximum
likelihood estimator in Frequentist Statistics:
\[
I(\thetavec)^{1/2}(\thetavechat-\thetavec)
\stackrel{\cal D} \longrightarrow N_p(\vec{0},I_p)
\quad\quad\text{as }n\to\infty, 
\]
where $I(\thetavec)=E_{\Xvec|\thetavec}[J(\thetavec)]$ is Fisher's
information matrix. Note that this statement about the distribution of
$\thetavechat$ for fixed (unknown) $\thetavec$, whereas the results
above is a statement about the distribution of $\thetavec$ for fixed
(known)~$\thetavechat$.
\end{enumerate}
%\end{ggap}
}
%%Hack for vspace
\newpage

\begin{example}~\\
Suppose we now have a random sample from a $N(\mu,1/\tau)$
\label{ex:asymprs}
distribution (with unknown precision). Determine the asymptotic
posterior distribution for~$(\mu,\tau)$.

Hint: we have already seen that the likelihood function can be written
as
\[
f(\xvec|\mu,\tau)=\left(\frac{\tau}{2\pi}\right)^{n/2}
\exp\left[-\frac{n\tau}{2}
\left\{s^2+(\bar x-\mu)^2\right\}\right]
\]
where $s^2=\sum_{i=1}^n (x_i-\bar x)^2/n$.
\end{example}

\framebox[17cm]{ \parbox[t]{16cm}{
\vspace{0.4cm}
\huge
\ding{46} \normalsize 
\textcolor{blue}{
%\begin{soln} 
We have
\begin{align*}
\log f(\xvec|\mu,\tau)&=\frac{n}{2}\log\tau-\frac{n}{2}\log(2\pi)
-\frac{n\tau}{2}\left\{s^2+(\bar x-\mu)^2\right\} \\
\Longrightarrow&\frac{\partial}{\partial\mu} \log
f(\xvec|\mu,\tau)=n\tau(\bar x -\mu)\\
&\frac{\partial}{\partial\tau} \log
f(\xvec|\mu,\tau)=\frac{n}{2\tau}
-\frac{n}{2}\left\{s^2+(\bar x-\mu)^2\right\} \\
\Longrightarrow&\frac{\partial^2}{\partial\mu^2} \log
f(\xvec|\mu,\tau)=-n\tau\\
&\frac{\partial^2}{\partial\mu\partial\tau} \log
f(\xvec|\mu,\tau)=n(\bar x -\mu)\\
&\frac{\partial^2}{\partial\tau^2} \log
f(\xvec|\mu,\tau)=-\frac{n}{2\tau^2}.
\end{align*}
\vspace{8.5cm}}
}}

\newpage

\framebox[17cm]{ \parbox[t]{16cm}{

\vspace{0.4cm}

\huge
\ding{46} \normalsize  \textit{...Solution to Example 3.3}
\textcolor{blue}{
Now
\begin{align*}
\frac{\partial}{\partial\mu} \log &f(\xvec|\mu,\tau)=0
\qquad\Longrightarrow\quad\quad
\hat\mu=\bar x \\
\frac{\partial}{\partial\tau} \log
&f(\xvec|\mu,\tau)=0
\qquad\Longrightarrow\quad\quad
\hat\tau=\frac{1}{s^2}
\intertext{Therefore}
J_{11}(\hat\mu,\hat\tau)&=-\left.\frac{\partial^2}{\partial\mu^2} \log
f(\xvec|\mu,\tau)\right|_{(\hat\mu,\hat\tau)}
=n\hat\tau=\frac{n}{s^2}\\
J_{12}(\hat\mu,\hat\tau)&=-\left.\frac{\partial^2}{\partial\mu\partial\tau} \log
f(\xvec|\mu,\tau)\right|_{(\hat\mu,\hat\tau)}
=-n(\bar x-\hat\mu)=0\\
J_{22}(\hat\mu,\hat\tau)&=-\left.\frac{\partial^2}{\partial\tau^2} \log
f(\xvec|\mu,\tau)\right|_{(\hat\mu,\hat\tau)}
=\frac{n}{2{\hat\tau}^2}=\frac{ns^4}{2}
\intertext{and so}
J(\hat\mu,\hat\tau)
&=\begin{pmatrix} \frac{n}{s^2} & 0 \\ 0 & \frac{ns^4}{2} \\
\end{pmatrix},
\intertext{whence}
J(\hat\mu,\hat\tau)^{-1}
&=\begin{pmatrix} \frac{s^2}{n} & 0 \\ 0 & \frac{2}{ns^4}\\ \end{pmatrix}.
\end{align*}
Therefore, for large $n$, the (approximate) posterior distribution for
$(\mu,\tau)$ is
\[
\left.\begin{pmatrix}\mu \\ \tau\end{pmatrix}\right|\xvec\sim 
N_2\left\{
\begin{pmatrix}\bar x\\ \frac{1}{s^2}\end{pmatrix},\,
\begin{pmatrix} \frac{s^2}{n} & 0 \\ 0 & \frac{2}{ns^4} \\ \end{pmatrix}
\right\}.
\]
Putting this another way, for large $n$
\[
\mu|\xvec\sim N(\bar x,s^2/n), \qquad \tau|\xvec\sim
N\{1/s^2,2/(ns^4)\},
\]
independently.}
%\end{soln}
\vspace{4cm}
}}
\newpage

\section{Learning objectives}
By the end of this chapter, you should be able to:
\begin{itemize}
\item understand different levels of prior information and have an appreciation for the difficulty of specifying ignorance priors in multi-parameter problems
\item determine the asymptotic posterior distribution when the data
  are a large random sample from \textbf{any} distribution
\item explain the similarities and differences between the asymptotic
  posterior distribution and the asymptotic distribution of the
  maximum likelihood estimator
\end{itemize}