How do these two category-theoretic descriptions of set difference differ?

Question

Most of the set operations are easy to transcribe into the category-theoretic language. Formally, let $A$ be a set, then:

• A subset $B \subset A$ is a category-theoretic subobject, whose accompanied monomorphism (or "inclusion map") $i_B: B \to A$, is an isomorphism when its codomain is restricted into its image. (This restriction is needed under the category $\mathbf{Top}$.)

• Given two subsets $B, C \subset A$, their intersection $B \cap C$ is the subset of $A$ such that, for every subset $D \subset A$ whose $i_D$ factors through $i_B$ and through $i_C$, $i_D$ factors through $i_{B \cap C}$. (It follows that $B \cap C \subset B$ and $B \cap C \subset C$.)

• Given two subsets $B, C \subset A$, their union $B \cup C$ is the subset of $A$ such that, for every subset $D \subset A$ where both $i_B$ and $i_C$ factor through $i_D$, $i_{B \cup C}$ factors through $i_D$. (It follows that $B \subset B \cup C$ and $C \subset B \cup C$.)

But what about the notion of set difference? Formally, given a set $A$ and its subset $B$, I attempted to come up with how to define the complement $A \setminus B$.

I see two ways. One "bottom-up", the other "top-down":

1. $A \setminus B$ is a subset of $A$ such that, for every subset $C \subset A$ such that $B \cap C = \emptyset$ (where $\emptyset$ is the initial object), $i_C$ factors through $i_{A \setminus B}$.

2. $A \setminus B$ is a subset of $A$ such that, for every subset $C \subset A$ such that $B \cup C = A$ (that is, $i_{B \cup C}$ is an isomorphism), $i_{A \setminus B}$ factors through $i_C$.

I won't judge which is more plausible as a definition, so here's my question: How do these two definitions differ, when generalized into other categories such as $\mathbf{Top}$ or $\mathbf{Grp}$?

Answer 1

Your definition 1 can be summarised in words as "the largest subobject whose intersection with the given one is empty". This is called the pseudocomplement. In a category in which subobject lattices are complete and (possibly infinitary) unions distribute over (binary) intersections, pseudocomplements automatically exist: just take the union of all subobjects disjoint from the given one!

The real difficulty in applying this definition is that (ordinary) subobjects are not always nice. For example, in $\textbf{Top}$, the monomorphisms are the continuous maps that are injective on points, so (generally) a topological space will have subobjects other than just subspaces. Thus it is often more useful to relativise the definition and only consider subobjects that are in a certain class of monomorphisms. For example, in $\textbf{Top}$, the regular monomorphisms are the subspace embeddings, so applying this definition with "regular subobject" instead of "subobject" yields the desired notion, namely the set-theoretic complement with the subspace topology. That said, the definition still works with "subobject", but one has to be more careful when establishing the universal property.

Your definition 2 is the lattice-theoretic dual of your definition 1: after all, it is "the smallest subobject whose union with the given one is everything". I am not aware of a standard name for it, but it corresponds to the negation operator in dual-intuitionistic logic. In $\textbf{Top}$, if we apply this definition with "regular subobject" instead of "subobject", we again recover the desired notion, i.e. the set-theoretic complement with the subspace topology. However, the definition with "subobject" behaves poorly: if $X$ is a connected space and $U$ is any open subspace other than $\emptyset$ or $X$, then there is no subobject $X'$ of $X$ such that $U \cap X' = \emptyset$ and $U \cup X' = X$, because $U \cup X'$ would be topologised with the disjoint union topology when computed in the subobject lattice of $X$.

Definitions 1 and 2 are not particularly applicable to $\textbf{Grp}$ even though there are non-trivial examples of groups $G$ with subgroups $H$ and $K$ such that $H \cap K = 1$ and $H \vee K = G$ (considered in the lattice of subgroups of $G$). The reason is that, fixing $G$ and $H$, there may be many subgroups $K$ that satisfy both conditions.