The assumption
\begin{equation}
|\langle x,y\rangle '| \leq |\langle x,y\rangle | \tag{1}
\end{equation}
for all $x,y\in H$ is much stronger than what is needed for this problem. The following argument shows why. For each $x\in H\setminus \{0\}$ the maps $\psi_{x}\colon H\to \mathbb{F}$ and $\phi_{x}\colon H\to \mathbb{F}$ defined by
\begin{equation}
\psi_{x}(y) := \overline{\langle x,y\rangle '} \tag{2}
\end{equation}
and
\begin{equation}
\phi_{x}(y) := \overline{\langle x,y\rangle } \tag{3}
\end{equation}
respectively define continuous linear functionals on $H$. By $(1)$ we have $\ker \phi_{x} \subseteq \ker \psi_{x}$, so by this result, there is some $c_{x}\in \mathbb{F}$ such that $c_{x} \psi_{x} = \phi_{x}$. It follows that
\begin{equation}
\overline{c_{x}} \langle x, y\rangle ' = \langle x,y\rangle \tag{4}
\end{equation}
for all $y\in H$. By taking $y = x$ in $(4)$ and applying $(1)$, we see that $c_{x}$ is uniquely determined with $c_{x} \geq 1$, so for each $x\in H\setminus\{0\}$ there is a unique $c_{x} \geq 1$ such that
\begin{equation}
c_{x} \langle x, y\rangle ' = \langle x,y\rangle
\end{equation}
for all $y\in H$.
We claim that $c_{x} = c_{x'}$ for all $x, x'\in H\setminus\{0\}$. Suppose for a contradiction this is not the case. Since the equality clearly holds if one of $x$ or $x'$ is a scalar multiple of the other, we can assume that $x$ and $x'$ are linearly independent. Hence using the notation from $(2)$ and $(3)$, the linear functionals $\psi_{x}$ and $\psi_{x'}$ are linearly independent. Note that we can write
\begin{equation}
c_{x+x'}(\psi_{x} + \psi_{x'}) = \phi_{x+x'} = c_{x}\psi_{x} + c_{x'} \psi_{x'} ,
\end{equation}
so that $\phi_{x+x'}$ can be expressed as a linear combination of $\psi_{x}$ and $\psi_{x'}$ in two distinct ways. But this contradicts the linear independence of $\psi_{x}$ and $\psi_{x'}$. Consequently, we have $c_{x} = c_{x'}$ for all $x, x' \in H\setminus \{0\}$. Therefore, the assumption $(1)$ implies that there is some $c\geq 1$ such that
\begin{equation}
c \langle x, y\rangle ' = \langle x, y\rangle
\end{equation}
for all $x,y\in H$, or equivalently
\begin{equation}
\langle x, y\rangle ' = \langle (c^{-1} I)(x), y\rangle
\end{equation}
for all $x,y\in H$. In particular, the bounded self-adjoint operator associated with $\langle \cdot , \cdot \rangle '$ is in fact a scalar multiple of the identity.
A more natural setting for this problem is to start with a sesquilinear form $\mathfrak{a}\colon H\times H \to \mathbb{F}$ satisfying the following assumptions. There is a $C\geq 0$ such that
\begin{equation}
|\mathfrak{a}(x,y)| \leq C \|x\| \|y\| \tag{5}
\end{equation}
for all $x,y\in H$ and
\begin{equation}
\mathfrak{a}(x,y) = \overline{\mathfrak{a}(y,x)} \tag{6}
\end{equation}
for all $x,y\in H$. Note that the inner product $\langle \cdot , \cdot \rangle '$ satisfies $(5)$ by the Cauchy-Schwarz inequality and $(6)$ by the conjugate symmetry of the inner product. From $(5)$ and this result we obtain a unique $T\in \mathscr{B}(H)$ such that
\begin{equation}
\mathfrak{a}(x,y) = \langle T(x), y\rangle \tag{7}
\end{equation}
for all $x,y\in H$. We now combine $(6)$ with this result to deduce that the operator $T$ from $(7)$ is self-adjoint.
With regards to your attempt, it looks like you were on the right track for the first part, although a few of the details are not quite right. For example, if you associate each $y\in H$ with the unique $T(y)\in H$ such that
\begin{equation}
\langle x,y \rangle ' = \langle x, T(y) \rangle
\end{equation}
for all $x\in H$, then the map $y\mapsto T(y)$ on $H$ is conjugate linear instead of linear.
