Is essence of Lagrangian mechanics just independence of $\frac{\partial}{\partial q}$ and $\frac{\partial}{\partial \dot{q}}$?

Question

It has always been confusing learning Lagrangian mechanics from physicists, since their notation $\dot{q}$ is so abusive (it can mean both an independent variable, tangent vector on the $T_{q}M$ or $\frac{d}{dt}(q(t))$ and this makes you think if the E-L equations must have some chain rules that takes the potential implicit dependence of $\dot{q}(t)$ on $q(t)$ as some chain rule expression)(Related: Why does $\frac{dq}{dt}$ not depend on $q$? Why does the calculus of variations work?)

Anyway, I learned the meaning of the results, but one thing upsets me is that it seems the main tool to deduce Newton's law is the fact that $\frac{\partial}{\partial q}$ and $\frac{\partial}{\partial \dot{q}}$ act as operators that can ignore the other variable to turn your expression, in this case Lagrangian $L(q, \dot{q}, t)$ (all variables independent), to match with the terms of the Newton's law $\frac{d}{dt}(m\dot{q})(s) = -\frac{\partial U}{\partial q}(s)$ and $\frac{d q}{dt}(s) = \dot{q}(s)$ (Here functions $q(s), \dot{q}(s)$ have potential dependence of course)

The simplest example is $L = \frac{1}{2}mv_y^2 - mgy$ for an object that is projected vertically to get the usual equations : $\frac{d}{dt}(mv_y) = mg$ and $v_y = \frac{d}{dt}y$ .

My question is that, this approach seems so "cheap" in the following sense. Ofc I can turn many expressions with two variables into two equations if I have operators that kills the other one and take a single derivative of my desired variable. To put into a more concrete question, why is the following function on the tangent bundle (say for a one-dimensional manifold) not as important as the Lagrangian for the purpose of re-formulating Newtonian mechanics:

$$M(a, b, t) = \frac{1}{6}mb^3 - U(a)$$ and in the $M-$theoric reformulation of mechanics, Newton's law reads: $\frac{d}{dt}(\frac{\partial^2}{\partial b^2 }M) = \frac{\partial M}{\partial a}$ and $\frac{da}{dt} = b$ along with the boundary conditions. Is it the case that all functions of tangent bundle, call it $M$ again, that can admit its own $M-$theoric " Newton's law " are equivalent logically for the purpose of formulating Newtonian mechanics, but the main importance of Lagrangian , i.e $M = L$, is that it has lowest degree (unlike my example which required taking taking second derivatives ) and has interpretations like being the derivative of some functionals? Note that, degree of derivatives required also shows that there is not a proper dimension for $M$ (but this doesn't seem to cause mathematical difficulties)

If so, in practice no one calculates those functionals I think, unless they are easy to interpret. Moreover, if degree of derivatives is the main reason, why not consider $M = T + V$ (which is a function of $(q, \dot{q}, t)$ not $(q, p, t)$ unlike Hamiltonian) with its own $"T+V"-$theoric formulation of Newton's law. In this case, such an $M$ also has interpretation as the "energy" of a point on the tangent bundle.

Answer 1

"Independent" partial derivatives. You seem to think that the partial derivatives with respect to coordinates and the associated velocities behave independently in order to recover Newton's law by applying a somewhat artificial recipe to the Lagrangian of the system, but that is not the case. How so ?

The Lagrangian $L(q,\dot{q},t)$ is indeed a function of several variables. However, as any multivariate function $f(x,y,z)$ for instance, nothing prevents you to consider variables depending on each others. Then, the variation of $f$ with respect to its arguements is usually dealt with the help of its differential, i.e. $$ \mathrm{d}f(x,y,z) = \frac{\partial f}{\partial x}\mathrm{d}x + \frac{\partial f}{\partial y}\mathrm{d}y + \frac{\partial f}{\partial z}\mathrm{d}z. $$ Now, let's assume that $y$ is a function of $x$, in contrary to $z$. Then, the total derivative of $f$ with respect to the variable $x$ is given by $$ \frac{\mathrm{d}}{\mathrm{d}x}f(x,y(x),z) = \frac{\partial f}{\partial x} + \frac{\partial f}{\partial y}\frac{\mathrm{d}y}{\mathrm{d}x}. $$ One sees that the variable $x$ contributes to $\frac{\mathrm{d}f}{\mathrm{d}x}$ through its partial derivatives $-$ nothing surprising : it means simply that $f$ varies when $x$ does $-$, while $y$ acts through the second term, where you have recognized the chain rule. The latter measures the indirect effect on $f$ of a variation of $x$ through the intermediary of $y$. Actually, the chain rule permits to divide this indirect effect in two steps : firstly, $f$ is varied as a function of $y$, hence the partial derivative $\frac{\partial f}{\partial y}$, whatever $y$ depends on, and after that, the variation of $y$ with respect to $x$ is taken into account through the derivative $\frac{\mathrm{d}y}{\mathrm{d}x}$. Finally, let's notice that an independent variable, such as $z$ in the present case, doesn't contribute to the final result, because $\frac{\mathrm{d}z}{\mathrm{d}x} = 0$.

Now, coming back to the Lagrangian mechanics, let's recall that the equation of motion, and thus ultimately Newton's law, is deduced from the first variation of the action functional $S[q] = \int L(q,\dot{q},t) \,\mathrm{d}t$, i.e. $\delta S[q] = \int \delta L(q,\dot{q},t) \,\mathrm{d}t$. Note that the functional differential acts basically as the usual differential, except that the variation is not considered locally (i.e. not at a unique time $t$), but it takes into account the contribution of $q(t)$ to $S[q]$ as a whole. Nonetheless, it works as before, hence $$ \delta L(q,\dot{q},t) = \frac{\partial L}{\partial q}\delta q + \frac{\partial L}{\partial \dot{q}}\delta\dot{q} + \frac{\partial L}{\partial t}\delta t. $$ The contribution of time alone can be ignored, because it produces a constant term after integration, which cannot vary and thus will disappear after differentiating the action. Next, given that $\frac{\delta q(t)}{\delta q(t')} = \delta(t-t')$, with the last $\delta$ being the Dirac delta function $-$ this relation only states that $q$ depends on itself $-$, one gets : $$ \frac{\delta S[q]}{\delta q(t')} = \int \frac{\partial L}{\partial q}\frac{\delta q(t)}{\delta q(t')} + \frac{\partial L}{\partial \dot{q}}\frac{\delta \dot{q}(t)}{\delta q(t')} \mathrm{d}t, $$ with $$ \frac{\delta \dot{q}(t)}{\delta q(t')} = \frac{\delta}{\delta q(t')} \frac{\mathrm{d}(t)}{\mathrm{d}t} = \frac{\mathrm{d}}{\mathrm{d}t} \frac{\delta q(t)}{\delta q(t')} = \frac{\mathrm{d}}{\mathrm{d}t}\delta(t-t') \equiv -\delta(t-t')\frac{\mathrm{d}}{\mathrm{d}t}, $$ where the last equality is obtained implicitly after an integration by parts inside the action. In the end, the Euler-Lagrange equation arises as $$ \frac{\delta S[q]}{\delta q(t')} = \int \delta(t-t')\left(\frac{\partial}{\partial q} - \frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial}{\partial \dot{q}}\right) L(q,\dot{q},t) \,\mathrm{d}t = \frac{\partial L}{\partial q} - \frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial L}{\partial \dot{q}} = 0. $$ Two conluding remarks have to done thus : 1° even if the partial derivatives are independent, the chain rule ensures that the contribution of the velocity $\dot{q}$ is not omitted when the coordinate $q$ varies $-$ with the formal equivalence $\frac{\delta}{\delta q} \equiv -\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial}{\partial \dot{q}}$ for an expression involving the said velocity $-$; and 2° this derivation doesn't follow an arbitrary recipe and has not been constructed in order to "match" Newton's law, it corresponds to the functional derivative of a physical quantity, namely the action.

Higher-order Lagrangians. My last remark answers partially why "other recipes" (such as the $M$-model you propose) are rarely considered; those would be actually more arbitrary and couldn't be interpreted as easily as the usual Lagrangian formalism. However, there exist various refomulations of classical mechanics, but they are all linked together with different changes of coordinates in the end $-$ otherwise we would need several theories in order to describe natural phenomena $-$, so that the usual setup is usually preferred for convenience and by force of habit. In a way, historical developments have selected the most practical formalisms and alternative formulations would require to adapt all the results obtained so far (such as Noether's theorem for instance). Nevertheless, nothing prevents you to use less common formalisms, in particular when they lead to easier equations of motions, but they are not different in nature to Lagrangian/Hamiltonian mechanics.