How can I de-mystify the trick of multiplying by $\frac{\cos x}{\cos x}$ to integrate $\sec x \equiv \frac{1}{\cos x}{\frac{\cos x}{\cos x}}$?

Question

Every year, several of my 17-year-old calculus students always ask how they could have come up with this trick – that is, multiplying by ${\frac{\cos x}{\cos x}}$ to get $$\int \frac1{\cos x} \, \mathrm dx = \int \frac{\cos x}{\cos^2 x} \, \mathrm dx =\int \frac{\mathrm d(\sin x)}{1 - \sin^2 x}$$ (from this answer).

How can I demystify "the way I used to integrate $\frac{1}{\cos x}$ is to multiply with $\frac{\cos x}{\cos x}$" (from this answer)?

If one were developing the theory from scratch, how would one find this solution (other than blind luck)?" I seek "good explanations and motivations for everything (as opposed to just pulling out ready-made solutions like what was done to me when I was learning this exact thing)." (Taken from this question.)

I'm NOT asking about other methods like integrating any function of the form $\sin^n(x)\cos^m(x)$ (like this), the tangent half-angle substitution, or multiplying by ${\sec x} \dfrac{\sec x+ \tan x}{\sec x+ \tan x}$ (like this).

Answer 1

I think it applies to a lot of areas of mathematics, but not everything necessarily has an intuitive approach from first principles that someone might be expected to notice. Sometimes people stumble upon things by experimentation, or by being very clever and noticing patterns/relations, and so on. Once you "know the trick" it becomes a new tool you can use even if it doesn't have an obvious motivation.

This particular integral even gets mentioned as a specific example of accidental discoveries in Spivak's calculus:

There is another expression for the integral of sec x dx, which is less cumbersome than $\log(\sec x + \tan x)$; using Problem 15-9, we obtain the integral as $\log(\tan(x/2 + \pi/4))$. This last expression was actually the one first discovered, and was due, not to any mathematician’s cleverness, but to a curious historical accident: In 1599 Wright computed nautical tables that amounted to definite integrals of $\sec$. When the first tables for the logarithms of tangents were produced, the correspondence between the two tables was immediately noticed (but remained unexplained until the invention of calculus).

So in this case, even the first discovered answer is still useful information as it now gives you an endpoint for other transformations and substitution experiments in terms of direction. But sometimes discoveries are just accidents and any "intuitive" explanation is going to come after the fact with that pre-knowledge driving it.