Let $n$ be a positive integer. Define the Gauss sum $$g(\ell,n):=\sum_{k=0}^{n-1}\,\omega_n^{k^2\ell}=\sum_{k=0}^{n-1}\,\exp\left(\frac{2\pi\text{i}k^2\ell}{n}\right)\,,$$ for every integer $\ell$. Here, $\omega_n$ is the primitive $n$-th root of unity $\exp\left(\frac{2\pi\text{i}}{n}\right)$, and $\text{i}$ is the imaginary unit $\sqrt{-1}$. From my observation (using Mathematica), for every integer $n>0$, we have $$\frac{g(1,n)}{\sqrt{n}}=\begin{cases} 1+\text{i}\,,&\mbox{if }n\equiv 0\pmod{4}\,, \\ 1\,,&\mbox{if }n\equiv 1\pmod{4}\,, \\ 0\,,&\mbox{if }n\equiv 2\pmod{4}\,, \\ \text{i}\,,&\mbox{if }n\equiv 3\pmod{4}\,. \end{cases}\tag{*}$$ The question is to compute $g(\ell,n)$ for general $\ell$ and $n$.
Note also that, for every $\ell\in\mathbb{Z}$, $g(\ell+n,n)=g(\ell,n)$ and $g(-\ell,n)=\overline{g(\ell,n)}$, where $\bar{z}$ is the complex conjugate of $z\in\mathbb{C}$. Thus, it suffices to evaluate $g(\ell,n)$ for integers $\ell$ with $0\leq \ell\leq \frac{n}{2}$. The value of $g(\ell,n)$ is of great interest for integers $\ell$ with $\gcd(\ell,n)=1$.
If the claim (*) is true, then I can find the multiplicities of the eigenvalues of the (inverse) discrete Fourier transform matrix $\mathbf{A}$ in my answer here with $\omega:=\omega_n$. That is, $+\sqrt{n}$ will have multiplicity $\left\lfloor\frac{n+4}{4}\right\rfloor$, $-\sqrt{n}$ will be of multiplicity $\left\lfloor\frac{n+2}{4}\right\rfloor$, $+\sqrt{n}\text{i}$ will be of multiplicity $\left\lfloor\frac{n+1}{4}\right\rfloor$, and $-\sqrt{n}\text{i}$ will be of multiplicity $\left\lfloor\frac{n-1}{4}\right\rfloor$. I am sure there are different approaches of getting these multiplicities, but I would like to see how the sum above is evaluated.
Notes:
The only case in (*) I am able to prove is when $n\equiv 2\pmod{4}$. That is because $$\omega^{(k+n/2)^2}=-\omega^{k^2}$$ for any primitive $n$-th root of unity $\omega$ and for all $k=0,1,2,\ldots,n-1$. Consequently, if $n\equiv 2\pmod{4}$, then $g(\ell,n)=0$ for all $\ell\in\mathbb{Z}$ with $\gcd(\ell,n)=1$.
i707107 has provided me with a great reference. Hence, I no longer need a proof for (*), except a different proof. However, I will still greatly appreciate if anybody can determine $g(\ell,n)$ for $\ell\not\equiv\pm1\pmod{n}$.
As arthur's link shows, if $n:=p$ is an odd prime and $p\nmid \ell$, then $$g(\ell,p)=\left(\frac{\ell}{p}\right)\,g(1,p)\,,$$ where $\left(\frac{\ell}{p}\right)$ denotes the Legendre symbol of $\ell$ modulo $p$.