Let $p,q$ be in $B$. Connect $p$ and $q$ by a path $\lambda$ in $A$. We will modify $\lambda$ in order to make it a path in $B$, i.e. make sure all of its image lies in $B$.
First, cover $\lambda(I)$ by open cubes $\{U_i\}_{i\in I}$ contained in $A$ (which is possible since $A$ is open). Since $I$ is compact, we can take a Lebesgue number $\delta$ for the open cover $\{\lambda ^{-1}(U_i)\}_{i \in I}$. Taking $N>1/\delta$ we have that for every $0 \leq i \leq N$, $\lambda([i/N,(i+1)/N]) \subset U_{j}$ for some $j$ .
We do the modifications inductively. Suppose we have modified $\lambda$ over $[(i-1)/N,i/N]$ already such that it avoids $A\backslash B$. We then have that $\lambda(i/N)$ and $\lambda((i+1)/N)$ are in some open cube $Q\subset A$. Take the straight line connecting these two points. On the middle of this line, consider variations by small amounts, obtained by changing the middle of the line perpendicularly to the line, resulting in broken lines (more specifically, in concatenations of two segments). This can be done in a (perhaps small) continuum since the open cube is open and convex. In any case we get uncountably many (different) such paths. Hence one of them must miss the countable set. Define $\hat{\lambda}$ on $[i/N, (i+1)/N]$ by that broken line. Because at the endpoints, this part is glued continuously to previous part, we retain the continuity of the path. This way, inductively, we finish our construction of a path from $p$ to $q$ that lies entirely in $B$.