The folklore is true: $K$, a subset of a subgroup $H$ of group $G$ and a subgroup of $G$, is a subgroup of $H$!
One hint we know it's true is that the above linked Tower Law for Subgroups (the difference there is that $K$ is a subgroup of $H$ is assumed and that the folklore there is that $K$ is a subgroup of $G$) has similar proof to Artin for Prop 2.8.14 (and these: 1, 2, 3).
Proof that $K$, a subset of a subgroup $H$ of group $G$ and a subgroup of $G$, is a subgroup of $H$:
Subset: $K \subseteq H$ by assumption.
Closure: Let $k_1,k_2 \in K$. Because $K \subseteq G$ is a subgroup of $G$, $k_1k_2 \in K$, which is the same requirement of closure for $K \subseteq H$ to be a subgroup of $H$.
Existence of Identity: Because $K \subseteq G$ is a subgroup of $G$, $K$ has an identity $1_K$, and, by Exer 2.2.5, $1_K$ is the identity of $1_G$, i.e. $1_K=1_G$. Because $H \subseteq G$ is a subgroup of $G$, $H$ has an identity $1_H$, and, by Exer 2.2.5, $1_H$ is the identity of $1_G$, i.e. $1_H=1_G$. Therefore, $1_K=1_H$, i.e. $K$ has an identity, and it is the identity in $H$.
Existence of Inverse: Let $k_1 \in K$. Because $K \subseteq G$ is a subgroup of $G$, there exists a $k_3 \in K$ s.t. $k_3k_1=k_1k_3=1$, which is the same requirement of existence of inverses for $K \subseteq H$ to be a subgroup of $H$.
QED
Note: Unlike in the analogues below, we used that $H$ is a subgroup of $G$.
I'm open other proofs that do not use that $H$ is a subgroup of $G$.
Proof of Prop 2.8.14:
By twice application of Counting Formula (Formula 2.8.8) with what we just proved, we have that for finite orders
$$[G:K] = \frac{|G|}{|K|}, [H:K] = \frac{|H|}{|K|}, [G:H] = \frac{|G|}{|H|}$$
Therefore, the result follows.
For infinite orders, it looks like we'll have to use the kind of proof with listing the cosets.
QED