Why must it be either true or false that the empty set is a subset of all sets?
Your questions is equivalent to "Why do we accept the empty set everywhere any non-empty set is allowed?" or "Why do we consider the empty set to be a set?".
Practically, it is this way because excluding the empty set from set-theoretical considerations would make any set theory incredibly complicated, while solving no problem.
Any and all definitions and operations we have would become more complicated if we had no $\emptyset$: while we would not mention it per se (as we would not have the notation or the concept), we still would to, for example, need to talk about the case of two sets not intersecting. Instead of saying $A \cap B = \emptyset$ we would need to write "$A$ and $B$ do not intersect" or "$A$ and $B$ have no element in common" or $\not \exists x: x \in A \land x \in B$ and so on and forth.
Much worse, in any composite formula where a sub-expression could evaluate to the empty set, we would need step out of the world of mathematical formulation, and would have to speak in human language about it (or in explicit $\exists, \forall$ notation). Even just a way to formulate $(A \cap B) \subset C$ in a way that never lets the case $A \cap B = \emptyset$ occur would be complicated, or at least laborious.
In short, it would be a nightmare - for no gain whatsoever. At no point does $\emptyset$ lead to any contradictions or complications; all definitions and axioms can be and easily are formulated in a way that our conventional usage of $\emptyset$ just follows automatically. Not allowing $\emptyset$ would be like arbitrarily introducing "division-by-zero-style" exceptions to all set theoretical statements.
It is important enough that the usual set theory has an Axiom of the Empty Set.
It is equivalent of allowing addition and subtraction of 0, or multiplication and division by 1. On the face of it doing so may be nonsensical, but if you do not consider it as part of the mathematical structures you very quickly get into deep trouble, indeed.
Formally, the empty set is still a set. It has properties (like $|\emptyset|=0$). In mathematical terms, it can be seen as part of the category of sets and plays an important role there; in Computer Science terms it would be an object of type "set" and plays an important part in all "typed" systems.
Philosophically, the empty set is not nothing. It is like an empty bag or an empty bank account: such a thing certainly exists and we have no issues imagining or working with it, it just does not contain anything.
As to your other question regarding "vacuous truth": statements like $\forall Set(X): \emptyset \subset X$ is in itself not vacuous; it is simply true on this level of formulation.
"Vacuous" means that any statement of the form $\forall x \in \emptyset : P(x)$ is true (by convention or definition). It is relatively natural to set this value to true if you look at the representation $\neg (\exists x \in \emptyset : \neg P(x))$ which is equivalent to the previous statement. Intuitively, $\exists x \in \emptyset$ is immediately "false" (it is not a statement, so it is "false in quotes"); so to assign the true value to the opposite (our original vacuous statement) makes the most sense.
Note that "vacuous truth" is not something weird or illogical, it just is a label for something which is true but gives no additional meaning or semantic power - it's useless, but harmless; but not having (the concept of) it would be harmful.