最大似然(likelihood)原理

假设一个随机试验，有若干可能结果A1,A2,A3,...$A_1,A_2,A_3,...$<script type="math/tex" id="MathJax-Element-476">A_{1},A_{2},A_{3},...
如果只进行一次实验，而结果Ak$A_k$<script type="math/tex" id="MathJax-Element-477">A_{k}出现了，那么我们就认为实验的条件对结果Ak$A_k$<script type="math/tex" id="MathJax-Element-478">A_{k}的出现最有利。即实验出现的结果Ak$A_k$<script type="math/tex" id="MathJax-Element-479">A_{k}的概率最大

最大似然法的基本思想

对于已经出现的样本值x1,x2,x3,...$x_1,x_2,x_3,...$<script type="math/tex" id="MathJax-Element-153">x_{1},x_{2},x_{3},...，适当的选取参数θ$\theta$<script type="math/tex" id="MathJax-Element-154">\theta，使实验得出结果 X1=x1,X2=x2,X3=x3,...$X_1=x_1,X_2=x_2,X_3=x_3,...$<script type="math/tex" id="MathJax-Element-155">X_{1}=x_{1},X_{2}=x_{2},X_{3}=x_{3},...的概率最大

最大似然估计法的模型

设总体X为离散型随机变量，分布律为

<script type="math/tex; mode=display" id="MathJax-Element-1154">P\left \{ X=x \right \}=p\left ( x;\theta \right )
其中θ$\theta$<script type="math/tex" id="MathJax-Element-1155">\theta是未知参数，X1,X2,X3,...$X_1,X_2,X_3,...$<script type="math/tex" id="MathJax-Element-1156">X_{1},X_{2},X_{3},...是来自总体X$X$<script type="math/tex" id="MathJax-Element-1157">X的样本，x1,x2,x3,...$x_1,x_2,x_3,...$<script type="math/tex" id="MathJax-Element-1158">x_{1},x_{2},x_{3},...是一组样本值。记
<script type="math/tex; mode=display" id="MathJax-Element-1159">L\left ( \theta \right ) = P\left \{ X_{1}=x_{1},X_{2}=x_{2},...,X_{n}=x_{n}\right \}
满足独立同分布的情况下上式等于
<script type="math/tex; mode=display" id="MathJax-Element-1160">L\left ( \theta \right )=\prod_{i=1}^{n}P\left \{ X_{i}=x_{i} \right \}= \prod_{i=1}^{n}P\left \{ x_{i};\theta \right \}
称L(θ)$L\left(\theta \right)$<script type="math/tex" id="MathJax-Element-1161">L\left ( \theta \right )为样本x1,x2,x3,...$x_1,x_2,x_3,...$<script type="math/tex" id="MathJax-Element-1162">x_{1},x_{2},x_{3},...的似然函数。
由于x1,x2,x3,...$x_1,x_2,x_3,...$<script type="math/tex" id="MathJax-Element-1163">x_{1},x_{2},x_{3},...是已经知道的，因此上述L(θ)$L\left(\theta \right)$<script type="math/tex" id="MathJax-Element-1164">L\left ( \theta \right )是关于θ$\theta$<script type="math/tex" id="MathJax-Element-1165">\theta的一元函数。
由于 L(θ)$L\left(\theta \right)$<script type="math/tex" id="MathJax-Element-1166">L\left ( \theta \right ) 是事件 {X1=x1,X2=x2,...,Xn=xn}$\left\{X_1=x_1,X_2=x_2,...,X_n=x_n\right\}$<script type="math/tex" id="MathJax-Element-1167">\left \{ X_{1}=x_{1},X_{2}=x_{2},...,X_{n}=x_{n}\right \}的概率，由最大似然函数的思想，希望求出的这样的θ$\hat{\theta }$<script type="math/tex" id="MathJax-Element-1168">\hat{\theta},使得L(θ)$L\left(\hat{\theta }\right)$<script type="math/tex" id="MathJax-Element-1169">L\left ( \hat{\theta}\right )达到L(θ)$L\left(\theta \right)$<script type="math/tex" id="MathJax-Element-1170">L\left ( \theta \right )的最大值