A callee only needs to save/restore callee saved (nonvolatile, call-preserved) registers that it needs to change the value of momentarily (some of which might not be used by any caller in the stack chain / stacktrace, but the callee doesn't know this), and a caller only needs to save/restore caller saved (volatile, call-clobbered) registers that it needs after the call (which a callee in the future stack chain might not actually modify anyway, but the caller doesn't know this).
Typically, at least on Microsoft x64 calling convention, you will see a lot of explicitly saved nonvolatile registers on the stack but not explicitly saved volatile registers -- I think the idea is that compilers never get to the stage where a caller needs to explicitly save a register right before a call, particularly an expression that isn't a variable in the program in itself; instead, it can plan ahead and avoid using those registers entirely, use the register but not optimise the variable backing store off of the stack, use the registers for parameters passed to a callee function that are dead after calling a callee function because they aren't defined as variables in the program, or use a volatile register.
A callee explicitly pushes any nonvolatile register it needs to keep modified around a call that it makes to the stack in the function prologue and restores them in the epilogue. It can save them in volatile registers but must restore them to the nonvolatile register or save them to the stack (in which case the save/store is called a spill) if the callee function makes a call itself, and it cant store it in another non-volatile register because then that register will need to be saved as well.
I agree that caller saved implies that register needs to be saved regardless of whether the caller uses it or not. This isn't true, and may not even have to save the register even if it does use the register, because it knows that it does not need it after the call, or might not make a call at all.
It is good to have an even balance. It is only a disadvantage to have all of one and none of the other, but sometimes it might be optimal to have a bias towards one type, for example nonvolatile, where that register might be predominantly used in a callee function and not in a caller function, like Peter suggested with xmm registers.
I think having all nonvolatile registers would hurt more than having all of them volatile registers, because you'd be saving parameters that might be dead in the caller after the call (which is why parameters are volatile; furthermore, preserving the return value register is impossible, so you'd have to have at least one volatile register for that or return values on the stack, which is slower), and you also would not be able to modify a register momentarily without saving the value to the stack because there are only nonvolatile registers available, whereas if they were all volatile registers, you'd be able to store values in registers until a call is made or if there is no call at all. There will always be a caller function (unless it's the base frame), but there are far more leaf functions than base frames, and the base frame would have to not adhere to the calling convention in order to optimise out the saving of nonvolatile registers, and might not optimise them out if adhering to it strictly, whereas a leaf function not saving volatile registers is defined in the calling convention.
If all the registers were volatile, this is still a disadvantage because nonvolatile registers can make it easier for compiling your own application, because the burden is on the callee function, which might be in some library compiled separately. Furthermore, seeing as all volatile registers are saved when making a trap frame and not nonvolatile registers (this is the case on Microsoft x64 calling convention at least, unless there is an exception or a context switch), there will be more time / space penalty for regular system calls if all the registers were volatile.
