While you're a crypto newbie, I hope that you know something about mathematics (because you've stumbled something that needs to be described in mathematical terms). What you've described here is actually polynomial multiplication (over polynomials in $GF(2)$ (that is, we do our computations modulo 2).
That is, you can interpret the numbers as specifying a polynomial (where each bit indicates whether the coefficient of the polynomial is either 0 or 1; for example, 1011 would correspond to the polynomial $x^3 + x + 1$, and 1010 would correspond to the polynomial $x^3 + x$. When you perform your operation, you generate the product polynomial, in this case $(x^3 + x + 1) \times (x^3 + x) = x^6 + 2x^4 + x^3 + x^2 + x = x^6 + x^3 + x^2 + x$ (this last step of dropping $x^4$ happens because we're computing everything in $GF(2)$
With that in mind, the problem of reversing the operation is actually a factorization problem (not factorization over the integers, which is what people generally think about when factorization is brought up, but its analogue over polynomials).
This implies that:
The values $A$ and $B$ need not be unique. For one, the operation is actually symmetric (even if your algorithm to perform it is not); swapping $A$ and $B$ will always result in the same product, and so another solution in your case is $A = 1010$ and $B = 1011$.
More generally, as yyyyyyy referenced, the factorization process can reduce the product to its prime factors, but there might be multiple ways to reassemble these prime factors (for example, the prime factorization of $1010 = 11 \times 11 \times 10$). It terms out that $1011$ is prime (and hence if we limit our multiplicands to 4 bits, $1011$ and $1010$ are the only two possible factors), however this won't be the case in general.
And if you're asking whether factorization is a hard problem, it is not. There are known polynomial time factorization methods over polynomials (this is one way where factorization over the integers differs from factorization over polynomials). However, those techniques are a bit beyond what a newbie can be expected to understand.