Let A be your first statement "For all $\varphi \in L^p$, if $(\lambda - m)\varphi = 0$ a.e. then $\varphi = 0$ a.e.", and B your second statement "$\lambda - m \ne 0$ a.e.".
To show A implies B, we would like to simply say: suppose A holds, and let $C = \{x : m(x) = \lambda\}$, and take $\varphi = 1_C$. Then certainly $(\lambda - m) 1_C = 0$ everywhere, so by A we have $1_C = 0$ almost everywhere, meaning $\mu(C) = 0$, which proves B.
But this proof only works if we start by assuming that $\mu(C) < \infty$. Without this assumption, we do not know that $1_C \in L^p$.
To rule out the possibility that $\mu(C) = \infty$, suppose to the contrary that we do have $\mu(C) = \infty$. By semifiniteness there is a measurable set $D \subset C$ with $0 < \mu(D) <\infty$. Then $1_D \in L^p$ and we also have $(\lambda - m) 1_D = 0$ everywhere. So as before we have $\mu(D) = 0$, a contradiction.
Showing B implies A is easier: if B holds and $(\lambda - m) \varphi = 0$ almost everywhere, then for almost every $x$ we have either $\lambda - m(x) = 0$ or $\varphi(x) = 0$. The former holds only on a null set so the latter must hold almost everywhere.
More formally, let $C = \{x : m(x) = \lambda\}$ and $D = \{x : \varphi(x) = 0\}$. B is asserting that $\mu(C) = 0$. Now for $x \in C \cup D$ we have $(\lambda - m(x)) \varphi(x) = 0$. So if $(m - \lambda) \varphi = 0$ almost everywhere, this means that $\mu((C \cup D)^c) = 0$. In other words, $\mu(D^c \cap C^c) = 0$. Since $D^c \subset (D^c \cap C^c) \cup C$, we have $\mu(D^c) \le \mu(D^c \cap C^d) + \mu(C) = 0+0$. So $\mu(D^c) = 0$ which is to say that $\varphi = 0$ almost everywhere.