4

Suppose I have two algorithms A() and B() such that algorithm A() takes exactly O(3n^2) while algorithm B() takes O(n^2). Although both algorithms run in quadratic time, can we say algorithm B runs faster than?

I understand that we ignore constants when analyzing the running time of an algorithm but I want to ask about the case when we need to consider the constants while analyzing algorithms.

Thank you

Kristofer
  • 1,457
  • 2
  • 19
  • 27
  • By "exactly O(3n^2)", perhaps you mean "exactly 3n^2 steps (using a particular implementation on a particular type of computer that's of interest to us)"? If every step of the implemented program running on this computer takes the same amount of time (e.g., because every step is of the same type) then for n >= 1, the 3n^2-step algorithm will always be slower than the n^2-step algorithm *using these implementations on this computer*. But many of these assumptions fail in practice: e.g., an algorithm often requires both "compare" steps and "swap" steps, and these might take different times. – j_random_hacker Oct 13 '16 at 11:40
  • On almost any modern computer, even steps of the same basic "type" can take different amounts of time -- e.g., a "copy this byte to that byte" step will be faster if some of its operands happen to be in cache. Either you can include details like this in your model, or not; if you do, you'll get more *accurate* conclusions (on computers that fit the model closely), but less *general* conclusions (they won't apply to other models). Big-O worst-case bounds are near the most general end of the spectrum: given large enough inputs, they give correct conclusions for *any* computer. – j_random_hacker Oct 13 '16 at 11:53

2 Answers2

2

You may want to take a look at this SO answer.

From that answer:

In summary - since big-O only talks about relative classes of growth rates, it ignores the constant factor. However, those constants are absolutely significant; they just aren't relevant to an asymptotic analysis.

So it may not make a difference in terms of Big O notation, but in real life your algorithm B will indeed run faster.

Community
  • 1
  • 1
Michele Ricciardi
  • 2,107
  • 14
  • 17
  • You brought something to me now, do you think we should consider the constants when I already know that the input size of my algorithm will be very large? – Kristofer Oct 12 '16 at 21:21
  • I think that if you are making a choice between C*n or G*n^2 (where G and C are constants and G is much smaller than C) you should ignore the constants and focus on whether it's an exponential or linear. If instead you have the case (like your example) of 2 algorithms that grow at the same rate (ignoring the constants), then you should take your constants in consideration. – Michele Ricciardi Oct 12 '16 at 21:24
  • np. also just to clarify: my previous comment is valid given the assumption of an infinitively big data set, however if you know how big your dataset is, you may be able to estimate that C*n may be quicker than G*n^2 but what you need to bear in mind is that there is a point at which the lines will cross and past that point your linear algorithm will do much better – Michele Ricciardi Oct 12 '16 at 21:33
  • so if you are looking for something that scale (even if you know the size of your dataset today), my initial comment is very much what you should be looking at – Michele Ricciardi Oct 12 '16 at 21:34
  • I suggest you have a play around and visualise those http://fooplot.com/#W3sidHlwZSI6MCwiZXEiOiIoMS8xMDAwMCkqeF4yIiwiY29sb3IiOiIjMDAwMDAwIn0seyJ0eXBlIjowLCJlcSI6IjIqeCIsImNvbG9yIjoiIzAwMDAwMCJ9LHsidHlwZSI6MTAwMCwid2luZG93IjpbIi0xMjY2NTAuMjg2MzE5NDU4NDgiLCIxMTIxMjIuMDAzNjkxMzE4MDQiLCItNjE0NjMuNjM3MzU4NzMzMTU1IiwiODU0NzMuMTU2NDk0MDUyMzUiXSwic2l6ZSI6WzY0OSwzOTldfV0- – Michele Ricciardi Oct 12 '16 at 21:39
  • 1
    This link is really helpful .. I appreciate your effort – Kristofer Oct 12 '16 at 21:43
2

Your two algorithms have the same asymptotic complexity, but one may definitely be faster than the other.

In this case, A has a larger constant so it is probably slower, but there may be other factors at play (such as implementation details, both in the algorithm implementation and the hardware it is running on) that may sway the balance either way.

dkasak
  • 2,651
  • 17
  • 26