Most common elliptic curves in use today (Edwards, Montgomery and the likes) have small cofactors. The reason we seem to want these curves in practice is because they facilitate fast scalar multiplication. On the other hand, much of public key cryptography relies on groups of prime order.
Of course, there are ways to clear cofactors and solutions like Decaf and Ristretto offer ways to perform prime-order group operations efficiently. However, at the heart of these techniques, we still rely on curves of non-prime order.
What are the precise mathematical properties that makes these curves easier to do computation over?