06.06.2013 Views

Theory of Statistics - George Mason University

Theory of Statistics - George Mason University

Theory of Statistics - George Mason University

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

798 0 Statistical Mathematics<br />

0.3.3.8 Derivatives <strong>of</strong> Matrices with Respect to Scalars<br />

The derivative <strong>of</strong> the matrix Y (x) = (yij) with respect to the scalar x is the<br />

matrix<br />

∂Y (x)/∂x = (∂yij/∂x). (0.3.43)<br />

The second or higher derivative <strong>of</strong> a matrix with respect to a scalar is<br />

likewise a matrix <strong>of</strong> the derivatives <strong>of</strong> the individual elements.<br />

0.3.3.9 Derivatives <strong>of</strong> Functions with Respect to Scalars<br />

Differentiation <strong>of</strong> a function <strong>of</strong> a vector or matrix that is linear in the elements<br />

<strong>of</strong> the vector or matrix involves just the differentiation <strong>of</strong> the elements, followed<br />

by application <strong>of</strong> the function. For example, the derivative <strong>of</strong> a trace <strong>of</strong><br />

a matrix is just the trace <strong>of</strong> the derivative <strong>of</strong> the matrix. On the other hand,<br />

the derivative <strong>of</strong> the determinant <strong>of</strong> a matrix is not the determinant <strong>of</strong> the<br />

derivative <strong>of</strong> the matrix (see below).<br />

0.3.3.10 Higher-Order Derivatives with Respect to Scalars<br />

Because differentiation with respect to a scalar does not change the rank<br />

<strong>of</strong> the object (“rank” here means rank <strong>of</strong> an array or “shape”), higher-order<br />

derivatives ∂ k /∂x k with respect to scalars are merely objects <strong>of</strong> the same rank<br />

whose elements are the higher-order derivatives <strong>of</strong> the individual elements.<br />

0.3.3.11 Differentiation with Respect to a Vector<br />

Differentiation <strong>of</strong> a given object with respect to an n-vector yields a vector<br />

for each element <strong>of</strong> the given object. The basic expression for the derivative,<br />

from formula (0.3.39), is<br />

Φ(x + ty) − Φ(x)<br />

lim<br />

t→0 t<br />

(0.3.44)<br />

for an arbitrary conformable vector y. The arbitrary y indicates that the<br />

derivative is omnidirectional; it is the rate <strong>of</strong> change <strong>of</strong> a function <strong>of</strong> the<br />

vector in any direction.<br />

0.3.3.12 Derivatives <strong>of</strong> Scalars with Respect to Vectors; The<br />

Gradient<br />

The derivative <strong>of</strong> a scalar-valued function with respect to a vector is a vector<br />

<strong>of</strong> the partial derivatives <strong>of</strong> the function with respect to the elements <strong>of</strong> the<br />

vector. If f(x) is a scalar function <strong>of</strong> the vector x = (x1, . . ., xn),<br />

<br />

∂f<br />

∂f<br />

∂x =<br />

, . . .,<br />

∂x1<br />

∂f<br />

∂xn<br />

<strong>Theory</strong> <strong>of</strong> <strong>Statistics</strong> c○2000–2013 James E. Gentle<br />

<br />

, (0.3.45)

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!