6 1. The Fisher Eﬃciency

Example 1.5. Let X be a Binomial(n , θ

2)

observation, that is, a random

number of successes in n independent Bernoulli trials with the probability of

a success p = θ

2

, 0 θ 1. An unbiased estimator of the parameter θ does

not exist. In fact, if

ˆ

θ =

ˆ(X)

θ were such an estimator, then its expectation

would be an even polynomial of θ,

Eθ

ˆ(X)

θ =

n

k = 0

ˆ(k)

θ

n

k

θ

2k

(1 − θ

2)n−k,

which cannot be identically equal to θ.

1.2. The Fisher Information

Introduce the Fisher score function as the derivative of the log-likelihood

function with respect to θ,

l (Xi , θ) =

∂ ln p(Xi , θ)

∂θ

=

∂p(Xi , θ)/∂θ

p(Xi , θ)

.

Note that the expected value of the Fisher score function is zero. Indeed,

Eθ l (Xi , θ) =

R

∂p(x , θ)

∂θ

dx =

∂

R

p(x , θ) dx

∂θ

= 0.

The total Fisher score function for a sample X1 , . . . , Xn is defined as the

sum of the score functions for each individual observation,

Ln(θ) =

n

i = 1

l (Xi , θ).

The Fisher information of one observation Xi is the variance of the Fisher

score function l (Xi , θ),

I(θ) = Varθ l (Xi , θ) = Eθ

(

l (Xi , θ)

)2

= Eθ

∂ ln p (X, θ)

∂ θ

2

=

R

∂ ln p(x , θ)

∂θ

2

p(x , θ) dx

=

R

(

∂p(x , θ)/∂θ

)2

p(x , θ)

dx.

Remark 1.6. In the above definition of the Fisher information, the density

appears in the denominator. Thus, it is problematic to calculate the Fisher

information for distributions with densities that may be equal to zero for

some values of x; even more so, if the density turns into zero as a function of

x on sets that vary depending on the value of θ. A more general approach to

the concept of information that overcomes this diﬃculty will be suggested

in Section 4.2.