Using a theorem of Masser it is possible to show that the quantities $(A_N)^6$ listed are rational numbers. It seems reasonable to show using his arguments that they must be integral, but this would involve digging deeper into coefficients of modular polynomials.
The reference to look up is the monograph by Masser:
D. Masser, Elliptic Functions and Transcendence, Springer Lecture Notes in Math., vol. 477, 1975.
In Appendix I, he considers the function,
$$\Psi(\tau) := \frac{3E_4(\tau)}{2E_6(\tau)} \biggl(E_2(\tau) - \frac{3}{\pi\, \mathrm{Im}(\tau)} \biggr),$$
for $\tau \in \mathbb{H} = \{ \tau \in \mathbb{C} \mid \mathrm{Im}(\tau) > 0 \}$. Now $\Psi$ is invariant under the action of $\mathrm{SL}_2(\mathbb{Z})$, but it is not holomorphic on $\mathbb{H}$. Here $E_4(\tau)$ and $E_6(\tau)$ are normalized Eisenstein series: for $q = e^{2\pi i \tau}$,
$$E_4(\tau) = 1 + 240 \sum_{n=1}^{\infty} \sigma_3(n)q^n,$$
$$E_6(\tau) = 1 - 504 \sum_{n=1}^{\infty} \sigma_5(n)q^n.$$
We also have the discriminant function,
$$\Delta(\tau) = \frac{E_4(\tau)^3-E_6(\tau)^2}{1728} = \eta(\tau)^{24} = q \prod_{m=1}^{\infty} (1 - q^m)^{24},$$
and the $j$-invariant,
$$j(\tau) = \frac{E_4(\tau)^3}{\Delta(\tau)},$$
which is a modular function. If $\tau \in \mathbb{H}$ is a quadratic irrationality, then $j(\tau)$ is an algebraic integer.
Theorem (Masser, 1975): Let $\tau \in \mathbb{H}$. If $\tau$ is a quadratic irrationailty not equivalent to $i=\sqrt{-1}$, then $\Psi(\tau) \in \mathbb{Q}(j(\tau))$.
Now for $\tau \in \mathbb{H}$, set
$$\Phi(\tau) := \left( \frac{E_2(\tau) - \dfrac{3}{\pi\,\mathrm{Im}(\tau)}}{\eta(\tau)^4} \right)^6 = \frac{ \biggl( E_2(\tau) - \dfrac{3}{\pi\,\mathrm{Im}(\tau)} \biggr)^6}{\Delta(\tau)}.$$
Claim: Suppose that $\tau \in \mathbb{H}$ is a quadratic irrationality not equivalent to $i$ or $e^{2\pi i/3}$. Then $\Phi(\tau) \in \mathbb{Q}(j(\tau))$ and so then is algebraic.
To prove this multiply top and bottom by $E_4(\tau)^6 E_6(\tau)^6$ (why we avoid $i$ and $e^{2\pi i /3}$), we have
$$\Phi(\tau) = \Psi(\tau)^6 \cdot \frac{2^6 E_6(\tau)^6}{3^6 \Delta(\tau) E_4(\tau)^6}.$$
Now $E_6^6/(\Delta E_4^6)$ is a modular function and so is an element of $\mathbb{Q}(j)$. In fact it is simply
$$\frac{E_6(\tau)^6}{\Delta(\tau) E_4(\tau)^6} = \frac{(j(\tau)-1728)^3}{j(\tau)^2}.$$
In any event this, together with Masser's theorem, implies that $\Phi(\tau) \in \mathbb{Q}(j(\tau))$, which proves the claim.
On a final note, Masser shows more explicitly (see equation (106) in his App. I) that when $\tau$ is a quadratic irrationality not equivalent to $i$,
$$\Psi(\tau) = 9\gamma_{\tau}\cdot j(\tau) + \frac{3}{2} \cdot \frac{7j(\tau)-6912}{j(\tau) - 1728},$$
where $\gamma_\tau \in \mathbb{Q}$ is an expression in terms of the coefficients of modular polynomials. Since $j(\tau)$ is an algebraic integer, determining more information about $\gamma_{\tau}$ may lead to showing that $D^3 \Phi(\tau)$ is an algebraic integer in general ($D$ the fundamental discriminant for $\tau$).
In the case of class number 1, Masser includes a table of the values of $\Psi(\tau_N)$, so showing that $(A_N)^6$ is an integer in these cases can likely be deduced from his table (I did not have time to verify it).
