Let $X \subset \mathbb{R}^{n}$ be closed and $f$ be a continuous real valued function on $X$.
Consider now the sets \begin{align} S_{<} = \{ x \in X : f(x) < 0 \}, && S_{\le} = \{ x \in X : f(x) \le 0 \}. \end{align}
Is it generally true that if $ S_{<} \ne \emptyset$, then $\overline{S_{<}} = S_{\le}$?
In general this is not true as madnessweasley's counterexample shows. If $S_{<}$ is non-empty, a counterexample is given by $f(x) = -1$ for $x \in A \subseteq X$ which is not dense in $X$ and $f(x) = 0$ when $x \in X \setminus A$.
In general the result is true if $f$ is continuous and closed:
Note that $S_{<} := f^{-1}((- \infty, 0))$ and $S_{<} := f^{-1}((- \infty, 0])$ and $\overline{(- \infty, 0)} = (-\infty, 0]$. If $f$ is continuous, we have $f^{-1}(\overline{(- \infty, 0)}) \subseteq \overline{f^{-1}(-\infty, 0))} = \overline{S_{<}}$. If in addition $f$ is closed, then this is an equality.
The question is whether $\{x \ | \ f(x)=0\}$ is contained in the closure of $f^{-1}( (-\infty, 0))$. Note that points in the interior of the zero set cannot be in the closure of $\{x \ \ f(x)\ne 0\}$. So now we have very simple counterexamples to the statement posted. For instance: $f(x) =\max (0, |x|-1)$.
