He is indeed talking about constructing an injection. However, it's even better than that: $2^\mathbb{N}$ carries a natural topology - even a metric! - and according to that, the injection Tao mentions is continuous.
You ask about using binary vs. decimal - there is a slight issue in using binary: reals with multiple binary expansions. E.g. in binary $0.0111111...=0.100000...$, so the "obvious" map is not injective. Things are easiest in base $>2$ - decimal, or ternary, or whatever you want.
One way to do things via decimal notation would be as follows:
Take your binary sequence,
change each $1$ to a $5$,
and put a decimal point at the end.
E.g. "$0, 1, 1, 0, 1, 0, 0, . . .$" becomes "$0.0550500...$". This is indeed injective, as is easily checked. And there are lots of other ways to do this, too. Tao says to use decimal notation just for simplicity - if you prefer to work in a different base you can, but if you work in binary you need to be careful to ensure injectivity.
One way to do this in binary is to use a different replacement scheme - e.g. replace each "$0$" with "$01$", and each "$1$" with "$10$", so you're using more than one digit. Then e.g. the sequence "$01001101...$" turns into the number $0.0110010110100110...$. This is less elegant, though, than just using a different base.