To show that a subgroup of a group is normal, I typically construct a homomorphism whose kernel is that subgroup. Are there any general principles or tests that I can use to determine that a subgroup is not normal? Or are such claims usually proved by ad hoc methods?
In particular, I would like to show that $\textbf{SL}_n(\mathbb{Z})$ is not a normal subgroup of $\textbf{SL}_n(\mathbb{R})$.
Any help is appreciated!
Paul's comment is the way to go here; you want to pick an element $h$ in the subgroup and find some other element $g$ such that $ghg^{-1}$ is not in the subgroup.
Conjugating a matrix essentially amounts to changing the basis. Pick some simple nonidentity element $h$ of $SL_n(\mathbb{Z})$ and find an orthonormal basis in which the corresponding matrix representation has at least one entry that is not an integer. This will give you an orthogonal matrix $g$ such that $g^{-1}hg$ is not in $SL_n(\mathbb{Z})$.
Well, one approach is just good ol' fashion conjugation. Here I'd just choose the easiest thing I possibly can to conjugate by, since I'm lazy! In this case, it turns out conjugating $\begin{pmatrix}1 & 2 \\ 2 & 5 \end{pmatrix}$ by $\begin{pmatrix}0 & 1/2 \\ -2 & 0 \end{pmatrix}$ will work.
I don't have a good reason for believing this, but I would suspect in most cases normality should fail pretty badly: it ought to be fairly likely that just choosing randomly would show you that the subgroup isn't normal (it worked first try, for me). That is to say, I expect the normalizer of $\mathbf{SL}_n(\Bbb Z)$ to be not much (if at all!) larger than $\mathbf{SL}_n(\Bbb Z)$ itself, although that's pure speculation. At least, if this weren't the case, the problem would be significantly more difficult!
More generally, it's often hard to pinpoint exactly why something nice doesn't happen; it's rare that we disprove a "theorem" by any means other than a counterexample. Some things just aren't meant to be...
Given any subgroup $H$ of a group $G$, you can describe its conjugate $gHg^{-1}$ conceptually as the stabilizer of the coset $gH$ with respect to the action of $G$ on $G/H$. The intersection $\bigcap_{g \in G} gHg^{-1}$ of these conjugates is then the kernel of this action. You can sometimes identify in fairly explicit terms the action of $G$ on $G/H$, and from there it's sometimes easy to compute this intersection (and verify that it's smaller than $H$) even if it's hard to describe the conjugates of $H$ explicitly.
In your case, we can say the following. In terms of the action of $SL_n(\mathbb{R})$ on $\mathbb{R}^n$, you can define $SL_n(\mathbb{Z})$ to be the subgroup stabilizing the standard lattice $\mathbb{Z}^n$ in $\mathbb{R}^n$. $SL_n(\mathbb{R})$ acts transitively on all lattices in $\mathbb{R}^n$ with stabilizer $SL_n(\mathbb{Z})$, and so the quotient $X = SL_n(\mathbb{R})/SL_n(\mathbb{Z})$ can be identified with the space of lattices in $\mathbb{R}^n$ (not up to rotation or scaling or anything). The nontrivial conjugates of $SL_n(\mathbb{Z})$ can then be identified with the stabilizer of other lattices in $\mathbb{R}^n$.
This action is very close to being faithful: I think when $n$ is even the kernel is generated by $-I$ and when $n$ is odd the kernel is just trivial. This is just the statement that any matrix in $SL_n(\mathbb{R})$ that isn't $\pm I$ fails to stabilize at least one lattice. If you believe that, it follows that the intersection of the conjugates of $SL_n(\mathbb{Z})$ in $SL_n(\mathbb{R})$ is either $\pm I$ or trivial, and so $SL_n(\mathbb{Z})$ is very far from normal.
