Just work backwards. You know that $539 a_{11} = a_{12}^2$, but $539=7^2\cdot 11$, hence $a_{11}=11 n^2$ and $a_{12}=77 n$. The sequence has to be strictly increasing, hence $n\leq 6$. What happens if we take $n=6$?
It follows that $a_{11}=11\cdot 6^2 = 396$ and $a_{12}=462$, so $a_{10}=2\cdot a_{11}-a_{12}=330$ and $a_9=a_{10}^2/a_{11}=275$. It follows that $a_8=2a_9-a_{10}=220$ and $a_7=a_8^2/a_9 = 176$. Then $a_6=2a_7-a_8=132$ and
$$ a_5 = \frac{a_6^2}{a_7} = \frac{132^2}{176} = \color{red}{99}.$$
The whole sequence then is: $11,22,44,66,99,132,176,220,275,330,396,462,539,616,704,792,891,990,1100,1210,\ldots $
It is left to check that smaller values of $n$ do not really work.