You need to limit the category a little more (e.g. to Hausdorff spaces) for this to be true.
If, for instance, we are allowed to equip $Y$ with the indiscrete topology, then $j,h$ can be any functions that agree on $A$, and it should be apparent that if $A\neq X$, then we could extend these functions arbitrarily.
That is, in the category of topological spaces, then $$\text{the inclusion } i : A \to X \text{ is an epimorphism } \iff A=X$$
If you work in the category of Hausdorff spaces, then the inclusion of a dense set into a space will always be an epimorphism. This likely is possible with less powerful separation axioms (e.g. $T_1$), but I don't know precisely how weak we can make them.