As mentioned in the comment, you need to make a selection to use those shortcuts. This is easy enough with Shift and the arrow keys. I use this way to:
a) Ensure that I'm filling down even if there are blanks in the column to the left
b) Ensure I'm only filling down the range I want to fill down (if you use the "Go To">"Special">"Last Cell" method, you might end up in the wrong place in complex worksheets)
It may seem long-winded, but I'm trying to offer as much detail as possible. It's actually quite quick when you get used to it.
E.g.
From your starting cell, if you want to fill-down to the last used cell in the column to the left, you can do this:
Ctrl+↓ to take you to the bottom of the sheet in the current column
← to go to the column to the left
Ctrl+↑ to get to the last used cell in the column to the left of your fill-down column
→ to get back to your fill-down column
Ctrl+Shift+↑ to select from the bottom of your fill-down column to the first value in the fill-down area.
Adjust the fill down selection by keeping Shift pressed and tapping up or down.
Press Ctrl+D
If you already know the address of the bottom of the column you want to fill-down, you can skip steps 1 through 4, just use Ctrl+G for "Go to" and enter the address, press enter, then continue from step 5.