Let $1 \leq p,q \leq \infty$ and $A= (a_{ij})$ be a scalar matrix. Suppose for every $x= (x_j)\in l^p$, the series $\sum_{1}^{\infty}a_{ij}x_j$ is convergent for every $i$ and that $y=(y_i) \in {l}^q$ where $y_i = \sum_{1}^{\infty}a_{ij}x_j = (Ax)_{j}$. I need to show that the map $A : l^p \rightarrow l^q $ is a bounded linear map.
The hint given to me was to use Closed graph theorem but I absolutely don't know how to do it !
As Daniel Fischer suggested: consider a sequence $\left(x^{(n)}, y^{(n)}\right)$ in the graph (that is, $y^{(n)}=Ax^{(n)}$), and such that $\left(x^{(n)}, y^{(n)}\right)\to (x,y)$ in $\ell^p\times \ell^q$. We have to show that $y=Ax$. Considering the sequence $\left(x^{(n)}, y^{(n)}\right)-\left(x,Ax\right)$ instead of the original one, we can assume without loss of generality that $x=0$ and we have to prove that $y=0$. It suffices to prove that for any $i$, $\left(Ax^{(n)}\right)_i\to 0$ as $n$ goes to infinity, that is, $$\tag{*} \lim_{n\to +\infty}\sum_{j=1}^{+\infty}a_{i,j}x_j^{(n)}=0. $$ It is possible to prove (for example using Baire's theorem) that for any $i$, the sequence $\left(a_{i,j}\right)_{j\geqslant 1}$ belongs to $\ell^q$ hence (*) follows by an application of Hölder's inequality.
Here, the advantage of the closed graph theorem is that in order to prove the continuity at $0$, we only have to consider sequences $\left(x^{(n)}\right)$ which go to $0$ such that $\left(Ax^{(n)}\right)$ converges to something, which is much easier to do than treating all the sequences going to $0$.
To elaborate on @DavideGiraudo's solution, here we prove that for each $i\in\mathbb{N}$, $\{a_{ij}:j\in\mathbb{N}\}$ belonged to $\ell_q$, where $\frac1p+\frac1q=1$, $1\leq p$.
Lemma: Suppose $\alpha:\mathbb{N}\rightarrow\mathbb{C}$ is a sequence such that for all $x\in\ell_p$, $\alpha x\in\ell_1$, i.e., $\sum_j|\alpha_j x_j|<\infty$. Then, $\alpha\in\ell_q$.
If $p=\infty$, take $x\equiv\mathbb{1}\in\ell_\infty$. Then $\alpha=\alpha\mathbb{1}\in\ell_1$.
Suppose $1\leq p<\infty$. For any $m\in\mathbb{N}$ define the linear functional $\Lambda_m:x\mapsto \sum^m_{j=1}\alpha_jx_j$. It is clear that $\Lambda_m\in\ell^*_p$. For any $x\in\ell_p$, the orbit $\{\Lambda_mx:m\in\mathbb{N}\}$ is a bounded set in $\mathbb{N}$ since $$|\Lambda_mx|\leq\sum^m_{j=1}|\alpha_j||x_j|\leq \sum^\infty_{j=1}|\alpha_j||x_j|=\|\alpha x\|_1<\infty$$ for all $m$. Since $\ell_p$ is a Banach space, an application of the Banach-Steinhaus theorem (a.k.a. uniform boundedness principle, which is based on Baire's category theorem) shows that $c:=\sup_m\|\Lambda_m\|<\infty$. Hence, for any $y\in\ell_p$ $$\Big|\sum^\infty_{j=1}\alpha_jy_j\Big|=\lim_{m\rightarrow\infty}\Big|\sum^m_{j=1}\alpha_jy_j\Big|=\lim_m|\Lambda_m y|\leq c\|y\|_p$$ that is, the linear map $\Lambda:y\mapsto\sum^\infty_{j=1}\alpha_jy_j$ is an element of $\ell^*_p\cong\ell_q$ To conclude, notice that from the Riesz representation theorem, there is $a\in\ell_q$ such that $\Lambda y=\sum^\infty_{j=1}a_jy_j$. Applying $\Lambda$ to $e_i=\mathbb{1}_{\{i\}}$, we obtain that $\alpha_i=a_i$ for all $I\in\mathbb{N}$. This shows that $\alpha\in\ell_q$.