Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.

The Hessian matrix or Hessian is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field. It describes the local curvature of a function of many variables.

Specifically, suppose $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is a function taking as input a vector $x \in \mathbb{R}^n$ and outputting a scalar $f(x) \in \mathbb{R}$; if all second partial derivatives of $f$ exist and are continuous over the domain of the function, then the Hessian matrix $H$ of $f$ is a square $n \times n$ matrix

889 questions
53
votes
2 answers

Are there any Hessian matrices that are asymmetric on a large set?

Are there any functions, $f:U\subset \mathbb{R}^n \to \mathbb{R}$, with Hessian matrix which is asymmetric on a large set (say with positive measure)? I'm familiar with examples of functions with mixed partials not equal at a point, and I also know…
AppliedSide
  • 1,414
35
votes
3 answers

The connection between the Jacobian, Hessian and the gradient?

In this Wikipedia article they have this to say about the gradient: If $m = 1$, $\mathbf{f}$ is a scalar field and the Jacobian matrix is reduced to a row vector of partial derivatives of $\mathbf{f}$—i.e. the gradient of $\mathbf{f}$. As well…
kjQtte
  • 682
32
votes
4 answers

Geometric interpretation of $\frac {\partial^2} {\partial x \partial y} f(x,y)$

Is there any geometric interpretation for the following second partial derivative? $$f_{xy} = \frac {\partial^2 f} {\partial x \partial y}$$ In particular, I'm trying to understand the determinant from second partial derivative test for determining…
Lie Ryan
  • 1,231
28
votes
2 answers

Lipschitz smoothness, strong convexity and the Hessian

I'm working with the following two concepts: Lipschitz Smoothness - a function $f$ is Lipschitz smooth with constant $L$ if its derivatives are Lipschitz continuous with constant $L$, in other words if for any $x$ and $y$, $$ \| \nabla f(x) -…
Daniel
  • 281
25
votes
3 answers

Lipschitz-constant gradient implies bounded eigenvalues on Hessian

I've read in a few places that if we have a Lipschitz gradient $$\|\nabla f(x) - \nabla f(y)\|\leq L\|x-y\|,\, \forall x,y, $$ we can equivalently say $\nabla^2f\preceq LI.$ But I'm having a hard time showing this. (Equivalently, I want to show $z^T…
Leland Stirner
  • 755
  • 1
  • 6
  • 16
23
votes
2 answers

Why is the approximation of Hessian$= J^TJ$ reasonable?

I met this equation frequently in Guass-Newton optimizations. But I dont understand why the left and right side of the equation can be equal. Lets say the Jacobian is $2$ by $2$ and Hessian is $$\begin{bmatrix}\frac{\partial^2f_1}{\partial^2 x_1 }…
user7487638
  • 333
21
votes
3 answers

Relation between the Hessian matrix and curvature

According to Hessian matrix, It describes the local curvature of a function. AFAIK, for one-variable function $f(x)$, its local curvature is $$\kappa = \frac{|f''|}{(1 + f'^2)^{3/2}},$$ and its Hessian matrix is $$\mathcal{Hess}(f) =…
avocado
  • 1,269
20
votes
2 answers

Convexity, Hessian matrix, and positive semidefinite matrix

I would like to ask whether my understanding of convexity, Hessian matrix, and positive semidefinite matrix is correct. For a twice differentiable function $f$, it is convex iff its Hessian $H$ is positive semidefinite. The Hessian matrix $H$ can be…
Dylan Lan
  • 347
16
votes
2 answers

Calculate the Hessian of a Vector Function

I'm working with optimisation. I am trying to obtain the hessian of a vector function: $$ \mathbf{F(X) = 0} \quad \text{or} \quad \begin{cases} f_1(x_1,x_2,\dotsc,x_n) = 0,\\ f_2(x_1,x_2,\dotsc,x_n) = 0,\\ \vdots\\ …
Abraham Alvarez
  • 173
15
votes
1 answer

Hessian of log-sum-exp $f(z) = \operatorname{log} \sum_{i=1}^n z_i$, find $\nabla^2f(z)$

Let $f(z) = \operatorname{log} \sum_{i=1}^n z_i = \operatorname{log} 1^Tz$. This problem comes from the following famous theorem: My work: The following step is just consider one entry of the Hessian matrix: $$\nabla^2 f(z)_{ij} =…
sleeve chen
  • 8,576
14
votes
2 answers

Why does the Hessian work?

I am working through Leonard Susskind's The Theoretical Minimum (on physics, but it also includes some maths). In particular, there is an interlude for which he discusses partial differentiation. He discusses a surface, a function of two…
lagrange103
  • 701
12
votes
3 answers

Relating condition number of hessian to the rate of convergence

While minimizing a Lipschitz continuous strongly convex functions, the rate of convergence of the gradient descent method depends on the condition number of the hessian of the function, where a high condition number leads to slow convergence. Can…
user2684957
  • 397
10
votes
2 answers

Second directional derivative and Hessian matrix

I am reading the following from the book Deep Learning, and I have the following questions. I don't quite understand second directional derivatives. The first directional derivative of a function $f:\mathbb{R}^m\to\mathbb{R}$ in the direction $u$…
Supreeth Narasimhaswamy
  • 1,272
10
votes
7 answers

Hessian matrix of a quadratic form

Prove that the Hessian matrix of a quadratic form $f(x)=x^TAx$ is $f^{\prime\prime}(x) = A + A^T$. I am not even sure what the Jacobian looks like (I never did one for $x \in \Bbb R^n$). Please help.
Smajl
  • 716
10
votes
3 answers

Eigenvectors and eigenvalues of Hessian matrix

Because the Hessian matrix is real and symmetric, we can decompose it into a set of real eigenvalues and an orthogonal basis of eigenvectors. The second derivative in a specific direction represented by a unit vector d is given by $d^T Hd$.…
Abhishek Bhatia
  • 1,758
  • 7
  • 23
  • 38
1
2 3
…
59 60