Here is how to build such a perfect set: take a union $X$ of disjoint open intervals covering $\mathbb{Q}$, having irrational bounds, so that the distance between two such intervals is positive. Take $P = {\mathbb R}\setminus X$. It is a perfect set containing only irrational points.
More precisely, let $(q_1, q_2, ...)$ be an enumeration of ${\mathbb Q}$ Let us construct a sequence of non empty open intervals $I_k = (\alpha_k, \beta_k)$,where $\alpha_i, \beta_i\in {\mathbb R}\setminus{\mathbb Q}$ by the following method: let $q$ be the first rational in the enumeration that does not belong to $A_k=\bigcup_{i<k} I_i$ (we start with $A_1=\emptyset$) We choose $I_k$ such that $q\in I_k$ and $A_k\cap I_k = \emptyset$. We can choose $\alpha_k<\beta_k$ irrational so that none of them is equal to one of the previously chosen $\alpha_i$ or $\beta_i$.
Let $A_\infty=\bigcup_i I_i$. Clearly, $A_\infty$ is an open set (as a union of open sets) that contains ${\mathbb Q}$. Let $P = {\mathbb R}\setminus A_\infty$. It is a closed set containing only irrational numbers.
Suppose that $x\in P$ is an isolated point. It means that there are non empty intervals $(a,x)$ and $(x, b)$ contained in $A_\infty$. But as $A_\infty$ is a disjoint union of non empty open intervals it means that $(a, x)$ must be included in one of the intervals $I_k$ and similarly for $(x, b)\subset I_\ell$. This is a contradiction because the upper bound of $I_k$ would be equal to the lower bound of $I_\ell$.
Finally, $P$ is a perfect set of irrational numbers.
Conversely, let us prove that every perfect set of irrational numbers has this structure, provided we also allow the bounds $-\infty$ and $\infty$ for the intervals $I_k$.
For this, let $P$ be a perfect set of irrational numbers and $A={\mathbb R}\setminus P$, an open subset of ${\mathbb R}$. We can decompose $A$ as the disjoint union of its connected components $A=\bigcup_k I_k$. If $A$ is not empty, the number of connected components is finite or countable, each $I_k=(\alpha_k, \beta_k)$ is an open interval which bounds belong to $P\cup\{-\infty, \infty\}$ (because $A$ is open). When $\ell\not = k$, one has $\beta_k\not = \alpha_l$ otherwise $\beta_k$ would be an isolated point in $P$, contradicting the perfection of $P$. QED