How to identify this group generated by these reflections in $\mathbb R^2$?

Question

This question was asked in the TIFR 2025 GS1 test (question number 16 in true-false section).

Check whether the given statement is true or false:

Let $\ell_1$ be the line in $\mathbb R^2$ joining $(0,0)$ and $(\frac{1}{2},\frac{\sqrt 3}2)$, and $\ell_2$ be the line in $\mathbb R^2$ joining $(0,0)$ and $(\frac{\sqrt 3}2, \frac{1}{2})$. Consider the group of bijections $\mathbb R^2\to \mathbb R^2$ under composition, and its subgroup $G$ generated by the reflections about $\ell_1$ and $\ell_2$. Then $G$ has exactly $12$ elements.

My attempt: I could not make much progress by thinking geometrically so here's what I planned. I'd find the matrix of the reflection operators and then decipher relations between them to identify $G$.

For $i=1,2$, let $T_i\colon\,\mathbb R^2\to\mathbb R^2$ denote the linear operator associated with reflection about $\ell_i$.

We know that: $$T_{i} = 2\cdot\mathrm{Proj}_{\ell_i}-\mathrm{Id}$$ where $\mathrm{Proj}_{\ell_i}$ denotes the operator associated with orthogonal projection onto $\ell_i$ and $\mathrm{Id}$ is the identity operator.

Now,
$T_1(1,0)= 2\cdot\frac{1}{2}(\frac{1}{2},\frac{\sqrt 3}2)-(1,0)=(-\frac12,\frac{\sqrt 3}2)$
$T_1(0,1)=2\cdot\frac{\sqrt 3}{2}(\frac{1}{2},\frac{\sqrt 3}2)-(0,1)=(\frac{\sqrt 3}{2},\frac12)$.

Thus, matrix of $T_1$ is: $$ A=\pmatrix{-\frac{1}{2}&\frac{\sqrt3}2\\\frac{\sqrt3}{2}&\frac{1}{2}}$$

Similarly, working out, I find that matrix of $T_2$ is: $$ B=\pmatrix{\frac{1}{2}&\frac{\sqrt3}2\\\frac{\sqrt3}{2}&-\frac{1}{2}}$$

Now, I want to identify the group generated by $A$ and $B$. Since $A$ and $B$ are matrices of reflection operators, we already know that $A^2=B^2=I$.

By computation, we have: $$AB=\pmatrix{\frac{1}{2}&-\frac{\sqrt3}2\\\frac{\sqrt3}{2}&\frac{1}{2}}$$ which can be identified as the matrix associated with anti-clockwise rotation by $60^{\circ}$ so it has multiplicative order of $\frac{360^{\circ}}{60^{\circ}}=6$.

So I'm left to identify this group: $\langle x,y\mid x^2=y^2=e, (xy)^6=e\rangle$. I'm not sure how to do this or do I need to find more relations between $A$ and $B$?

Answer 1

Generalizations..

Result 1:

Let $\ell_{\theta}$ be the line joining $(0,0)$ and $(\cos\theta,\sin\theta)$ i.e., the subspace $\text{span}\{(\cos\theta,\sin\theta)\}$ of $\mathbb R^2$ then the matrix of the linear operator associated with reflection about $\ell_\theta$ is: $$\pmatrix{\cos2\theta&\sin2\theta\\ \sin 2\theta&-\cos2\theta}$$ w.r.t. the standard ordered basis of $\mathbb R^2$.

The proof is straight forward. Let's call the linear operator $T_\theta$.

Let $\mathrm{Proj}_{\ell_\theta}$ denote the linear operator associated with orthogonal projection onto $\ell_\theta$ and $\mathrm{Id}$ be the identity operator.

Given an arbitrary point $(x,y)$, to obtain its reflection about $\ell_\theta$, we first project it onto $\ell_\theta$. This projection $\mathrm{Proj}_{\ell_\theta}(x,y)$ is the mid-point of the line joining $(x,y)$ and its reflected image $T_\theta(x,y)$. From this geometric interpretation, we get an equation to obtain $T_\theta(x,y)$ in terms of the projected image $\mathrm{Proj}_{\ell_\theta}(x,y)$ and it follows that:

$$ T_\theta = 2\cdot\mathrm{Proj}_{\ell_\theta}-\mathrm{Id}$$

Evaluating $T_\theta(1,0)$ and $T_\theta(0,1)$ gives the result.

---

Result 2:

Reflecting a point about $\ell_{\varphi}$ and then about $\ell_{\theta}$ is same as rotating its associated vector anti-clockwise by an angle of $2(\theta-\varphi)$. In other words, $T_{\theta}\circ T_\varphi$ is the linear operator associated with anti-clockwise rotation by $2(\theta-\varphi)$.

Is there a visual or geometric way to see why this is true? I am not sure. However, multiplying the matrices of $T_\theta$ and $T_\varphi$ does confirm this.

$$\pmatrix{\cos2\theta&\sin2\theta\\ \sin 2\theta&-\cos2\theta}\pmatrix{\cos2\varphi&\sin2\varphi\\ \sin 2\varphi&-\cos2\varphi} = \pmatrix{\cos2(\theta-\varphi)&-\sin2(\theta-\varphi)\\ \sin2(\theta-\varphi)&\cos2(\theta-\varphi)}$$

---

Result 3:

The group $G$ generated by $T_{\theta}$ and $T_\varphi$ under function composition is finite iff $2(\theta-\varphi)$ is a rational multiple of $2\pi$. Moreover, $G\cong D_{2n}$ where $n$ is the order of the rotation induced by $T_\theta\circ T_\varphi$.

The generators $T_\theta$ and $T_\varphi$ satisfy $T_\theta^2=T_\varphi^2=(T_\theta\circ T_\varphi)^n=\mathrm{Id}$. Thus, $G\cong \langle x,y\mid x^2=y^2=(xy)^n=e\rangle$ which is a standard presentation of $D_{2n}$. (I'll add more details later as I learn...)