=== a shorter cleaner tl;dr answer ====
All three answers more or less do the following.
If you have a set $S$ and a group $<G, \cdot>$ and you can have a bijective mapping from $f:S \to G$, then you can create a group operation in the follow way.
For any $a,b \in S$, you will have $f(a), f(b) \in G$. You can do the group operation on $f(a)$ and $f(b)$ to get $f(a)\cdot f(b) = \gamma \in G$. And then you can map from $c\in G$ back to $S$ but $f^{-1} (\gamma) = c\in S$.
Thus we can define a group operation on $S$ as $a\odot b = f^{-1}(f(a)\cdot f(b))$
This will satisfy conditions of a group:
$\cdot$ is clearly binary and as $\cdot$ is associative:
$(a\odot b)\odot c=$
$f^{-1}(f(a\odot b)\cdot f(c))=$
$f^{-1}(f(f^{-1}(f(a)\cdot f(b)))\cdot f(c))=$
$f^{-1}((f(a)\cdot f(b))\cdot f(c))=$
$f^{-1}(f(a)\cdot (f(b)\cdot f(c)))=$
$f^{-1}(f(a)\cdot f(f^{-1}(f(b)\cdot f(c))))=$
$f^{-1}(f(a) \cdot f(b\odot c))=$
$a\odot (b\odot c)$
so $\odot$ is a associative.
If $e$ is the identity in $G$ the $f^{-1}(e)$ is clear the $\odot$ identity:
$f^{-1}(e) \odot a = f^{-1}(f(f^{-1}e)\cdot f(a)=f^{-1}(e\cdot f(a))=f^{-1}(f(a)) = a$.
And for any $a \in S$ if $f(a) =\alpha\in G$ and $\alpha^{-1}$ is the inverse of $\alpha$ in $G$ then $f^{-1}(\alpha^-1)$ will be the $\odot$ inverse of $a$:
$a\cdot f^{-1}(\alpha^{-1}) = f^{-1}(f(a)\cdot f(f^{-1}(\alpha^{-1})))=f^{-1}(\alpha \cdot \alpha^{-1})=f^{-1}(e)=$ the $\odot$ identity.
....
So as $(0,1) \sim \mathbb R \sim (0,\infty)$ any bijection for $(0,1)$ to $<\mathbb R, +>$ or to $<\mathbb R_{>0}, \times>$ will work.
In my original answer I showed (in excrutiating detail) have $f(x) = \frac {1-x}x=\frac 1x -1$, a bijection from $(0,1)\to \mathbb R_{>0}$ will yield a group operation:
$a\odot b =f^{-1}(f(a)\times f(b))= \frac {ab} {1-a-b+2b}$ will yield a group operation with group identity $f^{-1}(1) = \frac 12$ and for any $a\in (0,1)$ then $f^{-1}(\frac 1{f(a)}) = 1-a$ is the inverse of $a$.
Read below if you want more.
You can of course do something equivalent with the $\arctan$ function or then $\log (\frac x{1-x})$ to map $(0,1)$ to $\mathbb R$ which is a group with addition. (I don't know if they also yield $\frac 12$ and $1-a$ as identity and $1-a$ as inverse but I bet they do. In fact I'm pretty certain $\log\frac x{1-x}$ yields the exact same group element by element as mine does.)
=== original answer below ====
Step 1: Think of any other group that has the same cardinality of $(0,1)$. Two obvious examples are $< \mathbb R, +>$ and $<(0,\infty), \times>$. There are of course others but these are the only two that I believe everyone familiar with with.
Step 2: Think of a bijective mapping from $(0,1)$ two the other group. It doesn't have to be sophisticated or clean. it can be anything so long as it is bijective.
A simple way could noting we stretch $(0,1)$ to $(1,\infty)$ via $h(x) = \frac 1x$ and we go then shift $(1,\infty)$ down to $(0,\infty)$ via $g(c) = x -1$ so we can map $(0,1)\to (0, \infty)$ via $f(x)= \frac 1x - 1=\frac {1-x}x$. (I'll leave it as an exercise that that is one to one and onto.)
We have that $<(0,\infty), \times >$ is a group with identity equal to $1$ and $f$ maps $(0,1)$ to it.
Step three: We can then define the $\cdot$ operation by saying the product of two elements is the pre-image of the product of the mapped elements.
If other words... as $f(x) = \frac {1-x}x$ will map $(0,1)$ to $(0, \infty)$ then we can define $a \dot b$ as $f^{-1}( f(a) \times f(b))$.
If we want to spell it out specifically: $f(x) = \frac {x-1}x$ so $f^{-1}(x) = \frac 1{x+1}$ and so $a\cdot b = \frac 1{\frac {1-a}a\frac {1-b}b +1}=\frac {ab}{(1-a)(1-b) +ab}=\frac {ab}{1-a-b+2ab}$
The identity is $f^{-1}(1) = \frac 1{1+1} = \frac 12$.
And the inverse of $a$ would be: Well $f(a) = \frac {1-a}a$ and the multiplicative inverse of that is $\frac a{1-a}$ so $a^{-1} = f^{-1}(\frac a{1-a})=\frac 1{\frac a{1-a} + 1}=\frac {1-a}{a+(1-a)}=1-a$
If as an excercise we want to prove this is a group:
- $\cdot$ is a binary operation:
If $a,b \in (0,1)$ then $\frac 1a, \frac 1b \in (1,\infty)$ and $\frac 1a -1, \frac 1b - 1 \in (0, \infty)$ and so $(\frac 1a -1)(\frac 1b-1)\in (0, \infty)$ and $\frac 1{(\frac 1a -1)(\frac 1b-1)}\in (0, \infty)$.
- $\cdot$ is associative.
$(a\cdot b)\cdot c = \frac {ab}{1-a-b+2ab}\cdot c=$
$\frac {\frac {ab}{1-a-b+2ab}c}{1-\frac {ab}{1-a-b+2ab}-c + 2c\frac {ab}{1-a-b+2ab}}=$
$\frac {abc}{(1-a-b+2ab)-ab -c(1-a-b-2ab) +2abc}=$
$\frac {abc}{1-a-b-c+ab +ac +bc + 4abc}$.
and similarly $a\cdot (b\cdot c)$ can be evaluated to equal the same.
- $\frac 12$ is an identity:
$\frac 12 \cdot a = \frac {\frac 12 a}{1-\frac 12 -a +2\cdot \frac 12a}=$
$\frac {\frac 12 a}{\frac 12 -a+a}=a$.
- $a^{-1} = 1-a$ is the inverse:
$(1-a)\cdot a = \frac {a(1-a)}{1 - (1-a) -a + 2a(1-a)}= \frac{a(1-a)}{2a(1-a)}=\frac 12$