I was working on this problem, I tried to do it without using the homeomorphism, I'm not sure at all if the solution is rigorously correct (it seems to me, correct me otherwise):
I will use that $X_K$ is connected iff the only closed and open sets are $X_K$ and $\emptyset$ (in its relative topology)
\begin{equation}
\text{By definition: $X_K = \prod_\alpha A_\alpha$ such that}
\begin{cases}
A_\alpha = \{a_\alpha\} &\alpha \notin K \\
A_\alpha = X_\alpha &\alpha \in K \\
\end{cases}
\end{equation}
Let $P \subseteq X_K$ an open and closed set, then $P = X_K \cap G$ with $G$ an open set of the product topology in $X$, and using that
\begin{equation}
\prod_{i\in I}M_i \cap \prod_{i\in I}N_i = \prod_{i\in I}M_i \cap N_i
\end{equation}
\begin{equation}
\text{$\Rightarrow P = \prod_\alpha B_\alpha$ such that}
\begin{cases}
B_\alpha = \{a_\alpha\} &\alpha \notin K \\
B_\alpha \subseteq X_\alpha \text{(an open set of $X_\alpha$)} &\alpha \in K \\
\end{cases}
\end{equation}
Then $X_K - P = \prod_\alpha C_\alpha$ which is open (because P is closed), and using that
\begin{equation}
\prod_{i\in I}M_i - \prod_{i\in I}N_i = \prod_{i\in I}M_i-N_i
\end{equation}
\begin{equation}
\prod_\alpha C_\alpha =
\begin{cases}
C_\alpha = \{a_\alpha\} &\alpha \notin K\\
C_\alpha = X_\alpha - B_\alpha \text{(an open set of $X_\alpha$)} &\alpha \in K\\
\end{cases}
\end{equation}
from here we get that $C_\alpha = X_\alpha - B_\alpha$ is an open and closed set in $X_\alpha \forall \alpha \in K$
Every $X_\alpha$ is connected, so we have 2 options for each $C_\alpha$: $C_\alpha = \emptyset$ or $C_\alpha = X_\alpha$
If there is at least one $\alpha \in K$ such that $C_\alpha = X_\alpha$, then $B_\alpha = \emptyset$ and we have that $P = \prod_\alpha B_\alpha = \emptyset$
If not, $C_\alpha = \emptyset, \forall \alpha \in K$, so $B_\alpha = X_\alpha, \forall \alpha \in K$, and then $P = \prod_\alpha B_\alpha = X_K$
That is to say that if P is an open and closed set, is either the empty set or $X_K$, so $X_K$ is connected.
