Let $V: \mathbb{R}^3 \rightarrow \mathbb{R}$ be a potential function.
The force is now $F = -\nabla V$, that is, it is minus the gradient of $V$. The gradient is the direction of steepest ascent. Hence, minus the gradient is the direction of steepest descent.
Now there are two claims:
- Following the force (or minus the gradient) infinitesimally will give you a local minimum, provided that the potential is bounded below$^1$.
- The gradient need not point in the direction of a global or local minimum.
For the second point I invite you to think about a hilly landscape, here we can define a function $h: \mathbb{R}^2 \rightarrow \mathbb{R}$ to be the height of the current position. Now, a river will flow along the direction of steepest descent (that is $-\nabla h$), but if you are standing on the bank of a river and looking downstream there is no guarantee that you are looking in the direction of the lake where the river ends up. Indeed the river may meander back and forth, and may even stream in the opposite direction before doubling back on itself!
The first point is the justification for the method of gradient descent for finding minima/maxima of functions.
A proof would presumably use the gradient theorem, but I don't have time to write anything up right now.
(1): A local minimum could still be located "at infinity", if this bothers you, you could ask for the domain of $V$ to be compact.