12

I tried to test bad_alloc exception by passing some negative arguments to new[]. When passing small negative numbers I get what I hoped for - a bad_alloc. However, when passing -1, I can see that my object is constructed thousands of times (I print static counter in constructor) and the application terminates with segfault.

new[] converts signed integer to size_t, so -1 is the max of size_t and -2 is the maximum - 1 and so on.

So why new[] throws exception when receiving some huge number, but tries to allocate when receiving the max of size_t? What is the difference between 1111...1 and 1111...0 for new[]? :)

Thanks in advance!

Mysticial
  • 464,885
  • 45
  • 335
  • 332
flyjohny
  • 273
  • 1
  • 2
  • 11
  • 2
    Which compiler/compiler's version/OS/... – xanatos Mar 13 '12 at 09:09
  • That time I missed the answers because StackOverflow doesn't send me any notifications, and now I cannot reproduce the problem... I regret I hadn't generated core dumps and didn't investigate it... Now I cannot reproduce the problem - passing -1 to 'new' seems to work and throws exception, while passing max int allocates a lot of objects but doesn't segfault. I'm starting to think that the problem was somewhere else... I give a plus to Mystical for his try. – flyjohny Sep 07 '12 at 10:40

1 Answers1

18

Here's my wild guess:

In lot of implementations, the allocator will place some meta-data next to the allocated region.
(For example, the size of the allocation.) So you are, in effect, allocating more than what you asked for.

Let's assume size_t is 32-bits. Compiled for 32-bits.

When you do:

int *array = new int[-1];

The -1 becomes -1 * 4 bytes = 4294967292 (after overflow). But if the allocator implementation puts 4-bytes of meta-data next to the allocated region. The actual size becomes:

4294967292 + 4 bytes = 0 bytes (after overflow)

So 0 bytes is actually allocated.

When you try to access the memory, you segfault since you go out-of-bounds immediately.

Now let's say you do:

int *array = new int[-2];

The -2 becomes -2 * 4 bytes = 4294967288 (after overflow). Append 4-bytes of meta-data and you get 4294967288 + 4 = 4294967292.

When the allocator requests 4294967292 bytes from the OS, it is denied. So it throws bad_alloc.

So basically, it's possible that -1 and -2 makes the difference between whether or not it will overflow after the allocator appends its meta-data.

Mysticial
  • 464,885
  • 45
  • 335
  • 332
  • I have never seen such a buggy implementation in the wild though – PlasmaHH Mar 13 '12 at 09:33
  • +1. "When you try to access the memory" - "you" being `new[]` moving on to loop over the elements it thinks it allocated space for, calling placement `new` to invoke the constructor on each.... – Tony Delroy Mar 13 '12 at 09:34
  • @PlasmaHH That's what I'm thinking too. I'm trying to find a source that tells me whether any of this is undefined behavior. But I haven't found anything yet. – Mysticial Mar 13 '12 at 09:37
  • @TonyDelroy: Which makes me think... the OP said that he can see thousands of objects being constructed. So the question is how likely it is that allocating 0 bytes will lead to a pointer pointing to something where you can allocate thousands of objects. It would have been useful, had the OP mentioned his platform. – PlasmaHH Mar 13 '12 at 09:40
  • `operator new` is required to return a non-null pointer, even if the requested size is 0. Then you are free to overwrite the rest of the heap... – Bo Persson Mar 13 '12 at 10:01
  • @BoPersson, you're not really free to overwrite the rest of the heap. Writing beyond the end of a memory buffer allocated with new results in undefined behaviour. If you're returned a pointer to a zero byte buffer, you basically can't read or write anything there. All you can do is use the pointer as a unique value (amongst other pointers) and delete it. –  Mar 13 '12 at 11:22
  • @james - I missed the smiley after the dots. :-) It was supposed to be an "explanation" on how the compiler could construct "thousands of objects" in zero allocated bytes. – Bo Persson Mar 13 '12 at 11:41
  • @James: "you're not really free" / "undefined behaviour" - absolutely true that the behaviour is not guaranteed - the discussion is just about whether the likely implementation on real systems is reconcilable with the observed behaviour. It's entirely believable that on the OP's system a heap allocation for 0 bytes returned an address at which thousands of objects could be constructed before the segfault, as described in the question. – Tony Delroy Mar 13 '12 at 12:09
  • @BoPersson Ah. Fair enough. :) –  Mar 13 '12 at 13:47
  • @TonyDelroy: Actually, the question is how big is the object being constructed by the OP's test code? If it's the same size as size_t then I would expect exactly the behaviour described by Mystical. The new array operator needs to store the size of the array next to the array itself so it knows how many objects to destroy when deleting the array. `new o[-1]` where `sizeof(o)==sizeof(size_t)` will attempt to allocate exactly zero bytes - something most memory allocators can do. `new o[-2]` will attempt to allocate 2^32-4 bytes, which will fail on a 32-bit system and thus throw bad_alloc. –  Mar 13 '12 at 14:07
  • If it manages to allocate 0 bytes but assumes there are 2^32-1 objects, it will most probably crash after initialising ~1000 objects. Ie when the next page boundary is met. –  Mar 13 '12 at 14:08
  • @James: I agree the size is a question and factor - though not "the question". Also agree that while the new operator doesn't *need* to store the array size next to the array, that's the implementation choice Mystical's exploring. Anyway, there's absolutely nothing to say that it should crash at the first page boundary - it's common for an allocation library to use `sbreak()` to allocate large numbers of pages to heap in each call, and unspecified whether the heap grows upward or downward in a way that means earlier memory allocation may be overwritten by the run-away constructions.... – Tony Delroy Mar 13 '12 at 14:20