Using 3 A's, 5 B's, 7 C's, how many 4-letter word can you arrange them into? Note that here 3+5+7>4.
We can't use the formula: $\frac{n!}{p!q! \cdots}$ because $\frac{4!}{3!4!7!}$ will be a fraction.
What is the general formula for this type of problem?
One way to answer this question would be to consider how many 4 letter words contain a certain number of A's.
For words containing 3 A's, there are two choices for the final letter, and 4 different reorderings, so 8 such words.
For words containing 2 A's , we can take the final two letters as 4 ordered choices, and then there are 6 ways of reordering the A's, so 24 such possibilities.
For words containing 1 A, there are 8 ordered choices for the final three letters, and 4 different choices for the position of A. So 32 choices.
And for words containing 0 A's , there are 16 ordered choices for the final four letters.
So overall, 8+24+32+16=80 words
A much nicer way to see this is that you can make all the words you would be able to make without the restriction apart from AAAA , so if you could use every letter as many times as you would like, then there would be 3^4=81 possible ways to form words. As you can't use AAAA, take away one , and you get 80.
J.Dmaths has already explained how you can solve such a question.
The only thing I would add is that you can't use the formula $\frac{n!}{p!q!r! \cdots}$ $\big [$ as you have, $\frac{4!}{3!4!7!}$ $\big ]$ because $n$ must be the total number of elements ($p+q+r + \cdots$). That is not the case here, and so you end up with a fraction.
Since you asked for a general way to count labeled enumeration problems: use exponential generating functions.
If a word has $a$ As, $b$ Bs, and $c$ Cs, then the number of permutations that can be formed is $$\frac{4!}{a!b!c!}$$
Then we can use counting combinations similar to ordinary generating functions: for $a \le 3$, find the coefficient of $x^4/4!$ in
$$\underbrace{\left(\frac{x^0}{0!} + \frac {x^1} {1!} + \frac{x^2}{2!} + \frac{x^3}{3!} \right)}_{\text{select } a} \underbrace{\left(\frac{x^0}{0!} + \frac {x^1} {1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \cdots \right)^2 }_{\text{select } b,c} = \left(1 + x + x^2/2 + x^3/6 \right) e^{2x}$$
(Since $b$ and $c$ are effectively unlimited, we can use the EGF of $e^x$)
P.S. I think for extremely large polynomials, FFT polynomial multiplication can be used, but obviously that isn't needed here.
