Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3981 questions
229
votes
6 answers

Why does this matrix give the derivative of a function?

I happened to stumble upon the following matrix: $$ A = \begin{bmatrix} a & 1 \\ 0 & a \end{bmatrix} $$ And after trying a bunch of different examples, I noticed the following remarkable pattern. If $P$ is a polynomial,…
ASKASK
  • 9,221
  • 4
  • 30
  • 49
212
votes
7 answers

How could we define the factorial of a matrix?

Suppose I have a square matrix $\mathsf{A}$ with $\det \mathsf{A}\neq 0$. How could we define the following operation? $$\mathsf{A}!$$ Maybe we could make some simple example, admitted it makes any sense, with $$\mathsf{A} = \left(\begin{matrix} 1…
user266764
94
votes
5 answers

Derivative of the inverse of a matrix

In a scientific paper, I've seen the following $$\frac{\delta K^{-1}}{\delta p} = -K^{-1}\frac{\delta K}{\delta p}K^{-1}$$ where $K$ is a $n \times n$ matrix that depends on $p$. In my calculations I would have done the following $$\frac{\delta…
Sara
  • 1,157
83
votes
6 answers

How to take the gradient of the quadratic form?

It's stated that the gradient of: $$\frac{1}{2}x^TAx - b^Tx +c$$ is $$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$ How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?
victor
  • 953
  • 1
  • 7
  • 6
82
votes
4 answers

Factorial of a matrix: what could be the use of it?

Recently on this site, the question was raised how we might define the factorial operation $\mathsf{A}!$ on a square matrix $\mathsf{A}$. The answer, perhaps unsurprisingly, involves the Gamma function. What use might it be to take the factorial of…
Oliphaunt
  • 953
51
votes
5 answers

Vector derivative w.r.t its transpose $\frac{d(Ax)}{d(x^T)}$

Given a matrix $A$ and column vector $x$, what is the derivative of $Ax$ with respect to $x^T$ i.e. $\frac{d(Ax)}{d(x^T)}$, where $x^T$ is the transpose of $x$? Side note - my goal is to get the known derivative formula $\frac{d(x^TAx)}{dx} =…
Asaf R
  • 621
49
votes
6 answers

Not understanding derivative of a matrix-matrix product.

I am trying to figure out a the derivative of a matrix-matrix multiplication, but to no avail. This document seems to show me the answer, but I am having a hard time parsing it and understanding it. Here is my problem: We have $\mathbf{D} \in \Re^{m…
Spacey
  • 1,117
47
votes
6 answers

Integral of matrix exponential

Let $A$ be an $n \times n$ matrix. Then the solution of the initial value problem \begin{align*} \dot{x}(t) = A x(t), \quad x(0) = x_0 \end{align*} is given by $x(t) = \mathrm{e}^{At} x_0$. I am interested in the following matrix \begin{align*} …
samsa44
  • 491
45
votes
5 answers

Derivative of Quadratic Form

For the Quadratic Form $X^TAX; X\in\mathbb{R}^n, A\in\mathbb{R}^{n \times n}$ (which simplifies to $\Sigma_{i=0}^n\Sigma_{j=0}^nA_{ij}x_ix_j$), I tried to take the derivative wrt. X ($\Delta_X X^TAX$) and ended up with the following: The $k^{th}$…
Praveen
  • 551
43
votes
2 answers

Taking a derivative with respect to a matrix

I'm studying about EM-algorithm and on one point in my reference the author is taking a derivative of a function with respect to a matrix. Could someone explain how does one take the derivative of a function with respect to a matrix...I don't…
jjepsuomi
  • 8,979
37
votes
4 answers

How to calculate the gradient of log det matrix inverse?

How to calculate the gradient with respect to $X$ of: $$ \log \mathrm{det}\, X^{-1} $$ here $X$ is a positive definite matrix, and det is the determinant of a matrix. How to calculate this? Or what's the result? Thanks!
pluskid
  • 1,159
30
votes
7 answers

Derivative of matrix exponential w.r.t. to each element of the matrix

I have $x= \exp(At)$ where $A$ is a matrix. I would like to find derivative of $x$ with respect to each element of $A$. Could anyone help with this problem?
darwin rajpal
  • 413
29
votes
5 answers

Gradient of squared Frobenius norm of a matrix

In linear regression, the loss function is expressed as $$ W \mapsto \frac1N \left\| X W - Y \right\|_{\text{F}}^2 $$ where the matrices $X$ and $Y$ are given. Taking the gradient yields $$ W \mapsto \frac 2N \, X^T( X W - Y ) $$ Why is this so?
wrek
  • 489
28
votes
2 answers

Derivatives of eigenvalues

Say I have a hermitian matrix A with elements $A_{ij}$, given the eigenvalues $e_p$, (and optional eigenvectors $v_p$) Is there an easy(ish) way to calculate $\frac{\partial e_p}{\partial A_{ij}}$? The elements represent potentials between…
byo
  • 383
28
votes
3 answers

Derivative of a vector with respect to a matrix

let $W$ be a $n\times m$ matrix and $\textbf{x}$ be a $m\times1$ vector. How do we calculate the following then? $$\frac{dW\textbf{x}}{dW}$$ Thanks in advance.
arindam mitra
  • 1,481
1
2 3
…
99 100