22

Java 6's mergesort implementation in Arrays.java uses an insertion-sort if the array length is less than some threshold. This value is hard-coded to 7. As the algorithm is recursive, this eventually happens many times for a large array. The canonical merge-sort algorithm does not do this, just using merge-sort all the way down until there is only 1 element in the list.

Is this an optimisation? If so, how is it supposed to help? And why 7? The insertion sort (even of <=7 things) increases the number of comparisons required to sort a large array dramatically - so will add cost to a sort where compareTo() calls are slow.

array-size vs #-of-comparisons for different values of INSERTIONSORT_THRESHOLD

(x-axis is size of array, y-axis is # of comparisons, for different values of INSERTIONSORT_THRESHOLD)

Matthew Gilliard
  • 9,298
  • 3
  • 33
  • 48
  • 1
    What is the source for this graph? You seem to be presenting it without any comment – matt b Jul 11 '11 at 14:25
  • I made this graph by sorting an array of objects which count how many times compareTo is called and varying INSERTIONSORT_THRESHOLD. – Matthew Gilliard Jul 11 '11 at 14:44
  • 1
    It is worth noting that Java7 also has the Timsort, which is a hybrid merge-insert developed by Tim Peters for python. http://download.java.net/jdk7/docs/api/java/util/Arrays.html#sort%28java.lang.Object[] – Tremmors Jul 11 '11 at 14:49

3 Answers3

18

Yes this is intentional. While the Big-O of mergesort is less than that of quadratic sorts such as insertion sort, the operations it does are more complex and thus slower.

Consider sorting an array of length 8. Merge sort makes ~14 recursive calls to itself in addition to 7 merge operations. Each recursive call contributes some non-trivial overhead to the run-time. Each merge operation involves a loop where index variables must be initialized, incremented, and compared, temporary arrays must be copied, etc. All in all, you can expect well over 300 "simple" operations.

On the other hand, insertion sort is inherently simple and uses about 8^2=64 operations which is much faster.

Think about it this way. When you sort a list of 10 numbers by hand, do you use merge sort? No, because your brain is much better at doing simple things like like insertion sort. However if I gave you a year to sort a list of 100,000 numbers, you might be more inclined to merge sort it.

As for the magic number 7, it is empirically derived to be optimal.

EDIT: In a standard insertion sort of 8 elements, the worst case scenario leads to ~36 comparisons. In a canonical merge sort, you have ~24 comparisons. Adding in the overhead from the method calls and complexity of operations, insertion sort should be faster. Additionally if you look at the average case, insertion sort would make far fewer comparisons than 36.

tskuzzy
  • 35,812
  • 14
  • 73
  • 140
  • This explanation of the complexity makes a lot of sense intuitively - although I was unable to prove any advantage of 7 specifically, >25 did make a difference. – Matthew Gilliard Jul 11 '11 at 12:46
  • Edited my answer. I'm not 100% sure what your benchmarks show since your axes aren't really labeled. – tskuzzy Jul 11 '11 at 12:52
  • +1 this makes a very big difference if, for example, you need to sort lots of small arrays. I've felt this on my own skin. – Gabi Purcaru Jul 11 '11 at 14:57
4

Insertion sort is n(n-1)/2 and merge sort is n*(log n with base 2 ).

Considering this -

  1. For Array of Length 5 => Insetion sort = 10 and merge sort is 11.609
  2. For Array of Length 6 => Insetion sort = 15 and merge sort is 15.509
  3. For Array of Length 7 => Insetion sort = 21 and merge sort is 19.651
  4. For Array of Length 8 => Insetion sort = 28 and merge sort is 24

From above data it is clear, till length 6, insetion sort is faster and after 7, merge sort is efficient.

That explains why 7 is used.

user1289117
  • 115
  • 1
  • 6
3

My understanding is that this is an empirically derived value, where the time required for an insertion sort is actually lower, despite a (possible) higher number of comparisons required. This is so because near the end of a mergesort, the data is likely to be almost sorted, which makes insertion sort perform well.

dlev
  • 48,024
  • 5
  • 125
  • 132
  • I guessed so, too. But when I ran some benchmarks I found that it isn't the case. For cheap compareTo operations, any number less than about 20 was roughly equivalent, and for expensive compareTos the comparison time dominates. – Matthew Gilliard Jul 11 '11 at 12:36
  • 1
    Matthew: Note that expensive `compareTo` implementations are probably not the most frequent case (remember, Java's base class library is pretty general-purpose and not specifically catered to exactly your use case) and by using Insertion sort on small sublists you can also save the overhead for repeatedly applying the D&C algorithm or Merge sort. – Joey Jul 11 '11 at 12:39
  • @Matthew Joey is right about the "general purposeness" of the Java BCL. Another point to note is that truly expensive `compareTo()` methods should probably be fixed, since comparing two objects shouldn't need to take a long time. If there's no way to avoid that (maybe because the objects really are that complex) it might be worth sorting a set of proxy object on the relevant criteria (since it's rare that *every* facet of an object will be taken into account when sorting.) – dlev Jul 11 '11 at 12:45
  • Understood - I don't have a real-life problem that this is trying to address ;) But this algorithm is only used to sort general objects - primitive arrays are handled differently - so *unknown* complexity of comparisons ought to be a consideration, didn't it? – Matthew Gilliard Jul 11 '11 at 12:48
  • 1
    How is the data *almost sorted* when calling the insertion sort? Mergesort sorts only in the merge phase, and the recursive insertion sort call occurs before that. – Paŭlo Ebermann Jul 11 '11 at 14:29