Similar to mathematics, Lie groups appear in physics as symmetry groups. They help us solve differential equations, as Lie first studied them. I will answer this question from three different perspectives, where the Lie group SO(3) and SU(2) appear in physics.
First part: Classical Mechanics From a mathematician's point of view, a Hamiltonian systems is a pair $(M,\omega,H)$, where $(M,\omega)$ is a symplectic manifold and $H$ is a smooth function on $M$, often called the hamiltonian. The observables are the smooth functions on $M$, i.e. $f \in C^{\infty}(M)$ is an observable. They form a Poisson algebra if equipped with the Poisson bracket $\{ \cdot, \cdot \}$. Here is where symmetry comes in. If we have a Lie group $G$, which acts in a certain way (you can check the details in the reference Ratiu Marsden book) on the manifold $M$, then you can quotient out by the group action and obtain a reduced symplectic manifold of lower dimension. This is what physicists call reducing degrees of freedom via constants of motion. In particular, this plays an important role for systems with rotational symmetry, as there the group $SO(3)$ will indeed satisfy the conditions needed for the theorem. Thus, we can quotient out and obtain a lower dimensional manifold, i.e. a system with less degrees of freedom, which is easier to solve.
Note: in case of Harmonic oscillator, this is doable repeatedly- one can go down even to a $1$-dimensional manifold- when this is the case: we call the Hamiltonian system $(M,\omega,H)$ completely integrable.
NB: Up to my knowledge, $SU(2)$ does not play a very important role in Classical Mechanics, but it does in quantum theory.
Second part: Quantum Theory Here is where $SU(2)$ appears for the first time and plays a really important role. Once again, as in classical mechanics, one is interested in studying symmetries of Quantum systems. Formally, here observables are self-adjoint operators (possibly unbounded) on a complex, separable Hilbert space $H$. The states of the quantum system are elements of the Projective Hilbert space $\mathcal{P}(H)$, in physics we call them rays. Physically we require this to hold, since probabilities $|\psi|^2$ are not affected by multiplying with a phase, and probabilities are what we can measure in the lab. Here is where $SU(2)$ comes in. We still have that in case of rotational symmetry $SO(3)$ is the group of interest. However, as we specified we want representations 'up to a phase', this is what mathematicians call Projective Representations, $SU(2)$ appears. This holds, as the projective representations of $SO(3)$ are precisely the representations of $SU(2)$. This gives rise to the spin representations in Quantum Mechanics. More details about this part and obstructions of lifting representations can be found in Schottenloher's book, which will be given in the references.
A really nice answer for this is also given on the physics SE:
https://physics.stackexchange.com/questions/203944/why-exactly-do-sometimes-universal-covers-and-sometimes-central-extensions-feat
Part Three: Gauge Theories In gauge theories, like Electrodynamics, Yang-Mills theories, the gauge fields, which we use in physics, are connection one-forms on some Principal $SU(n)$ bundle over spacetime. So roughly speaking, the Lie groups $SO(3)$ and $SU(2)$ appear as the fibres of the Principal Bundles in discussion. The representation theory is important, as given a representation, we can build an associated bundle from this representation. Moreover, the Field Strengths $F_{\mu \nu}$ are the curvature $2$-forms with values in the Lie algebra of the structure group of principal $G$-bundle. In particular, the standard model is a Gauge theory, where the things said above apply: the gauge bosons are described by the adjoint representation of the structure group $G$, say $SU(2)$. For further references how these concretely appear, consider the reference by Hamilton.
References:
Representation Theory, coadjoint orbits of $SO(3),SU(2)$ in Classical Mechanics: Ratiu,Marsden: Mechanics and Symmetry;
Complete integrability of the harmonic oscillator with detailed proof: chapter 1.3.2 in the recently developed notes: https://www.mathematik.uni-muenchen.de/~schotten/GEQ/GEQ.pdf ;
Projective Representations in Quantum Mechanics, relation between $SU(2),SO(3)$: Martin Schottenloher: A mathematical introduction to Conformal Field Theory : chapters 3 and 4;
Gauge Theories, $SU(n)$-principal bundles and the Standard model: Mark Hamilton: Mathematical Gauge Theory: chapter 2,7 and 8.