First of all, there are some possible ways to generalize higher order derivatives for smooth manifolds: Jets, Double/Triple/... Tangent bundle, exterior derivative, Connections.
Jets usually generalize Taylor expansion, and are useful for discussing partial differential equations on manifolds. But they produce abstract polynomials instead of real functions over the manifold.
If you think of the tangent bundle $TM$ as a manifold itself, the derivative of a smooth path $\gamma$ can be thought as a lift of $\gamma$ to a path on $TM$. In this sense double/triples/... tangent bundles can indeed generalize higher order derivatives. If you think of a class $C^r$ path on $M$, in this sense there exists a lift of this path to $T^rM$.
I think one of the reasons you may be felling uncomfortable with the above constructions is the following: they are either abstract or coordinate dependent. If I am correct the problem is that if you wish for a coordinate independent way to talk about higher order derivatives, such that it is a map from concrete function/vector-like fields. For this you need to choose a connection.
To illustrate this point, I will take an example from John Lee's introduction to riemannian manifolds. Consider a path $\gamma(t)=(\cos t,\sin t)$ in $\mathbb{R}^2$.
This path traces out a circle with constant velocity. You can truly calculate it's velocity $\gamma'(t)$. And if you were to change coordinates, say to Polar coordinates using a map $F$ getting a different path $\tilde{\gamma}(t)=F\circ\gamma(t)=(0,t)$, you could then map the velocity of this smooth path using $dF$.
This is in some sense what the derivative is, a map $dF$ that locally looks linear and tells you how to map velocities of smooth paths. But then what is the second derivative? It should be the acceleration of this path!
But while our path in the original coordinates clearly has an nonzero acceleration (centripetal) $\gamma''(t)=(-\cos t,-\sin t)$, in our second path it is zero $\tilde{\gamma}''(t)=(0,0)$. So there is no linear map relating both of them! The map $D^2$ you desired doesn't exist naturally!
The first order derivative is always coordinate independent but the higher order ones aren't. In a informal sense, this happens because the second derivative would be:
$$
"\gamma''(t)=\lim_{\epsilon\rightarrow 0}\frac{1}{\epsilon}(\gamma'(t+\epsilon)+\gamma'(t))"
$$
But $\gamma'(t+\epsilon)$ and $\gamma'(t)$ are vectors from different tangent spaces. We need some way to connect different tangent spaces, this is precisely what a connection is.
An affine connection can be defined as a $C^\infty(M)$ linear map:
$$
\nabla: \Gamma(TM)\mapsto \Gamma(\text{End}(TM))
$$
Where $\Gamma(TM)$ is the space of vector fields and $\Gamma(\text{End}(TM))$ is the space of fields of linear transformations on the tangent space.
Usually you can define the exterior derivative as a map:
$$
d:C^{\infty}(M)\rightarrow \Omega^{1}(M)\\
d:\Omega^{n}(M)\rightarrow \Omega^{n+1}(M)
$$
From smooth functions to $1-forms$ (which is like the 1st order derivative), or $n$-forms to $(n+1)$-form (as two Lie derivatives doesn't satisfy Leibnitz rule, to obtain an higher order operator you need to antissymetrize it). This is similar to how you can canonically define first order derivatives.
Then equipped with a connection $\nabla$, you can extend the exterior derivative to a de Rham operator:
$$
d_{\nabla}:\Omega^{n}(M,TM)\rightarrow\Omega^{n+1}(M,TM)
$$
From vector-valued $n$-forms to vector-valued $(n+1)$-forms.
So this enables you concretely calculate higher order derivatives, not only of real functions, but also vector fields and differential forms. Then if you have a smooth function $F$, you can think of the 0th order as a pullback of functions, 1st order as a pullback of differential forms (given by $dF$), and higher order as a pullback of these higher tensor objects $\Omega^n(M)$ or $\Omega^n(M,TM)$.
