Theory of Statistics - George Mason University

800 0 Statistical Mathematics
For a function f : S ⊆ IR n → IR m , we define ∂f T /∂x to be the n × m matrix,
which is the natural extension of ∂/∂x applied to a scalar function, and
∂f/∂x T to be its transpose, the m×n matrix. Although the notation ∂f T /∂x
is more precise because it indicates that the elements of f correspond to the
columns of the result, we often drop the transpose in the notation. We have
∂f
∂x
∂fT
= by convention
∂x
∂f1 ∂fm
= . . .
∂x ∂x
⎡
⎤
⎢
= ⎢
⎣
∂f1
∂x1
∂f1
∂x2
∂f1
∂xn
∂f2
∂x1
∂f2
∂x2
· · ·
∂f2
∂xn
· · · ∂fm
∂x1
· · · ∂fm
∂x2
· · · ∂fm
∂xn
⎥
⎦
(0.3.48)
if those derivatives exist. This derivative is called the matrix gradient and
is denoted by Gf or ∇f for the vector-valued function f. (Note that the ∇
symbol can denote either a vector or a matrix, depending on whether the
function being differentiated is scalar-valued or vector-valued.)
The m × n matrix ∂f/∂x T = (∇f) T is called the Jacobian of f and is
denoted by Jf:
Jf = G T f = (∇f) T . (0.3.49)
The absolute value of the determinant of the Jacobian appears in integrals
involving a change of variables. (Occasionally, the term “Jacobian” is used
to refer to the absolute value of the determinant rather than to the matrix
itself.)
To emphasize that the quantities are functions of x, we sometimes write
∂f(x)/∂x, Jf(x), Gf(x), or ∇f(x).
0.3.3.14 Derivatives of Matrices with Respect to Vectors
The derivative of a matrix with respect to a vector is a three-dimensional
object that results from applying equation (0.3.45) to each of the elements of
the matrix. For this reason, it is simpler to consider only the partial derivatives
of the matrix Y with respect to the individual elements of the vector x; that
is, ∂Y/∂xi. The expressions involving the partial derivatives can be thought
of as defining one two-dimensional layer of a three-dimensional object.
Using the rules for differentiation of powers that result directly from the
definitions, we can write the partial derivatives of the inverse of the matrix Y
as
∂
∂x Y −1 = −Y −1
∂
∂x Y
Theory of Statistics c○2000–2013 James E. Gentle
Y −1 . (0.3.50)