$\newcommand{\un}{\mathrm{un}}$$\newcommand{\Z}{\mathbb{Z}}$$\newcommand{\Q}{\mathbb{Q}}$ $\newcommand{\Gal}{\mathrm{Gal}}$$\newcommand{\Hom}{\mathrm{Hom}}$$\newcommand{\Spec}{\mathrm{Spec}}$Here's a supplement to Adrien Morin's answer. His answer handles the case when $S$ is the spectrum of a non-perfect field. This post will discuss the more subtle case of $S=\mathrm{Spec}(\mathbb{Z})$.
We make the following claim:
Claim 1: Let $n$ be an integer such that there exists a prime $p>2$ such that $p\mid n$. Then the sequence
$$0\to \mu_n\to\mathbb{G}_m\to\mathbb{G}_m\to 0$$
of abelian presheaves on $\mathrm{Spec}(\mathbb{Z})$ is exact in the fppf topology but not in the etale topology. In particular, exactness fails at second $\mathbb{G}_m$ term.
The fact that it's a short exact sequence of fppf sheaves is classical. Namely, exactness at all points except the second $\mathbb{G}_m$-term is clear. If $X\to\mathrm{Spec}(\mathbb{Z})$ is any object of the fppf site of $\mathrm{Spec}(\mathbb{Z})$ and $u\in \mathcal{O}_X(X)^\times$ then the affine morphism $\underline{\mathrm{Spec}}(\mathcal{O}_X[t]/(t^p-u))\to X$ is an fppf cover over which $u$ obtains a $p^\text{th}$ root, which shows exactness at the second $\mathbb{G}_m$ in the fppf topology. It's also clear that the only place that exactness of this sequence can fail as sheaves on etale site of $\mathrm{Spec}(\mathbb{Z})$ is at the second $\mathbb{G}_m$.
To proceed further recall that the sequence of presheaves
$$0\to\mu_n\to\mathbb{G}_m\to\mathbb{G}_m\to 0$$
is exact on the etale site of $\mathrm{Spec}(\mathbb{Z})$ if and only if for all geometric points $\overline{s}$ of $\mathrm{Spec}(\mathbb{Z})$ the sequence
$$0\to \mu_n(\mathcal{O}_{S,\overline{s}})\to \mathcal{O}_{\mathrm{Spec}(\Z),\overline{s}}^\times\xrightarrow{x\mapsto x^n}\mathcal{O}_{\mathrm{Spec}(\Z),\overline{s}}^\times\to 0\qquad (1)$$
is exact. This follows from Tag03PU and the fact that if $\mathcal{F}$ is a representable presheaf on $\mathrm{Spec}(\mathbb{Z})$ given $\mathrm{Hom}(-,X)$ with $X\to\mathrm{Spec}(\mathbb{Z})$ finite type then
$$\begin{aligned} \mathcal{F}_{\overline{s}} &=\varinjlim_{(U,\overline{u})}\mathcal{F}(U)\\ &=\varinjlim_{(U,\overline{u})}\Hom(U,X)\\ &\overset{(*)}{=}\Hom(\varprojlim_{(U,\overline{u})}U,X)\\ &= \Hom(\Spec(\varinjlim_{(U,\overline{u})}\mathcal{O}_U(U)),X)\\ &= \Hom(\mathrm{Spec}(\mathcal{O}_{\mathrm{Spec}(\mathbb{Z}),\overline{s}}),X)\\ &= X(\mathcal{O}_{\mathrm{Spec}(\mathbb{Z}),\overline{s}})\end{aligned}$$
where $(\ast)$ follows from Tag01ZC.
Now, let's suppose that $\overline{s}:\mathrm{Spec}(k(\overline{s}))\to S$ is a geometric point of $S$ rooted at the point $p$. Let $\overline{\mathbb{F}_p}$ denote the algebraic (=separable) closure of $\mathbb{F}_p$ in $k(\overline{s})$. Then, $\mathcal{O}_{S,\overline{s}}$ is nothing but the strict Henselization of $\mathbb{Z}$ at $(p)$ which we denote $R$.
Thus, we see to show that the Kummer sequence is not exact it suffices to show that the sequence of abelian groups
$$0\to \mu_n(R)\to R^\times\xrightarrow{x\mapsto x^n} R^\times\to 0$$
is not exact, or, really that it's not exact at the second $R^\times$ or that not every element of $R^\times$ has an $n^\text{th}$-root. We will do this by showing that there are elements of $R^\times$ that don't have $p^\text{th}$ roots.
Let us make the following claim:
Lemma 1: The ring $R$ is the integral closure of $\mathbb{Z}_{(p)}$ in the ring of integers $\mathbb{Z}_p^\un$ in $\Q_p^\un$ the maximal unramified extension of $\Q_p$.
Proof: Let $S$ denote the henselization of $\Z_{(p)}$ at $(p)$. Then, $S$ is the integral closure of $\Z_{(p)}$ in $\Z_p$. There are several ways to see this. One is to use Artin Approximation. One can also use the construction of henselizations in Nagata's book Local Rings (middle of page 180). There, since $\mathbb{Z}_{(p)}$ is normal (with perfect residue field) he constructs $S$ as follows. Let $\widetilde{S}$ be the integral closure of $\mathbb{Z}_{(p)}$ in $\overline{\mathbb{Q}}$. Let $\mathfrak{p}$ be a prime of $\widetilde{S}$ dividing $p$. Let $D:=D(\mathfrak{p}\mid p)\subseteq \mathrm{Gal}(\overline{\mathbb{Q}}/\Q)$ denote the decomposition group of $\mathfrak{p}$--the subgroup of Galois group elements fixing $\mathfrak{p}$. Then, he describes $S$ as $\widetilde{S}^D$.
Note though that completing $\overline{\Q}$ at $\mathfrak{p}$ one obtains the field $\mathbb{C}_p$ of complex numbers and one can identify $D$ with $\mathrm{Gal}(\overline{\Q_p}/\Q_p)$ or the norm-preserving automorphisms of $\mathbb{C}_p/\Q_p$. Note then that $S$ embeds into the $\Gal(\overline{\Q_p}/\Q_p)$-invariants of $\mathbb{C}_p$. But, these are nothing more than $\Q_p$ (e.g. see Proposition 2.1.2 of
this). Thus, $S$ is evidently contained in the integral closure of $\Z_{(p)}$ in $\Z_p$. But, the reverse inclusion is clear.
To deduce now that $R$ is the integral closure of $\Z_{(p)}$ in $\Z_p^\un$ we proceed as follows. Note that the residue field $S$ is $\mathbb{F}_p$ and so by Tag0BSL it suffices to for each $n\geqslant 1$ construct the unique finite connected etale cover $\mathrm{Spec}(R_n)$ of $S$ with residue field $\mathbb{F}_{p^n}$. Note though that this can be easily seen $R_n=S[\zeta_{p^n-1}]$ which is clearly the integral closure of $S$ in $\Z_{p^n}$ (the ring of integers of the unique unramified extension of degree $n$) since $\Z_{p^n}=\Z[\zeta_{p^n-1}]$. Letting $n$ tend to infinity gives the desired result. $\blacksquare$
From this, we obtain the following corollary:
Corollary 1: An element $u\in R^\times$ has a $p^\text{th}$ root in $R^\times$ if and only if there exists a root $\alpha$ of $x^p-u$ such that $\Q_p(\alpha)/\Q_p$ is unramified.
Proof: Note that since the equation $x^p-u=0$ is monic and $R$ is integrally closed in $\Q_p^\un$ we see that this equation has a solution in $R$ (if and only if it has a solution $R^\times$ for obvious reasons) if and only if it has a solution in $\Q_p^\un$. Moreover, such a solution $\alpha$ in $\overline{\Q_p}$ will lie in $\Q_p^\un$ if and only if $\Q_p(\alpha)/\Q_p$ is unramified. $\blacksquare$
Thus, to finish the proof of the claim it suffices to produce elements of $u\in R^\times$ such that for any root $\alpha$ of $x^p-u$ in $\overline{\mathbb{Q}_p}$ we have that $\Q_p(\alpha)/\Q_p$ is ramified.
To wit, we have the following:
Lemma 2: Let $u\in R^\times$ be such that $u=1\mod p$ but $u\ne 1\mod p^2$. Then, for all roots $\alpha$ of $x^p-u$ in $\overline{\Q_p}$ we have that $\Q_p(\alpha)/\Q_p$ is totally ramified of degree $p$.
Proof: Set $f(x)=x^p-u$ then substituting $x=t+1$ gives the polynomial
$$f(t)=t^p+p q(t)-(u-1)$$
where $q(t)\in \mathbb{Z}[t]$ (which follows from the Binomoial Theorem). This shows that $f$ is Eisenstein and thus irreducible. The ramifiedness claim then follows from classical algebraic number theory (e.g. see Theorem 3.1 of this). $\blacksquare$
So, for instance, we see that $1+p\in R^\times$ does not have a $p^\text{th}$ root.
As another example, let $q\in\Z$ be as in the answer of Minseon Shin. Note then that since $q^{p-1}\ne 1\mod p^2$ implies that its order in $(\Z/p^2\Z)^\times\cong \mathbb{Z}/p(p-1)\Z$ is divisible by $p$. Thus, we see that $q':=q^{p-1}$ satisfies $(q')^{p-1}\ne 1\mod p^2$ and thus $q'$ has no $p^\text{th}$ root in $R^\times$ as in the answer of Minseon Shin. But, the above gives an alternative proof that $q'$ has no root in $R^\times$ since $q'=1\mod p$ but $q'\ne 1\mod p^2$. Conversely, the above actually shows that for any $q$ as in Minseon Shin's answer and any root $\alpha$ of $x^p-q$ in $\overline{\mathbb{Q}_p}$ we have that $\Q_p(\alpha)/\Q_p)$ is ramified.
In fact, we can actually explicitly describe the cokernel of the $p^\text{th}$ power map on $R^\times$:
Claim 2: The cokernel of $R^\times\xrightarrow{x\mapsto x^p}R^\times$ can be identified with $\overline{\mathbb{F}_p}$.
Proof: Let us note that every element of $f\in R^\times$ can be written uniquely in the form $f=\zeta u$ with $\zeta\in \mu^p(R)$ and $u=1\mod p$ where
$$\mu^p(R):=\bigcup_{(m,p)=1}\mu_m(R)$$
Indeed, note that we have a multiplicative Teichmuller map $[-]:\overline{\mathbb{F}_p}^\times\to R^\times$ which follows from Henselianess. If one is more familiar with the Witt vector construction another way to see the existence of this map is to note that $R\subseteq \Z_p^\mathrm{un}\subseteq W(\overline{\mathbb{F}_p})$ because this latter ring is the completion of the former. By classical theory we have a multiplicative Teichmuller map $[-]:\overline{\mathbb{F}_p}^\times\to W(\overline{\mathbb{F}_p})^\times$. But, since the image of $[-]$ lands in $\Z_p^\mathrm{un}$ (since this already contains all prime-to-$p$ roots of unity) and these are integral over $\mathbb{Z}_{(p)}$ they lie in $\mu^p(R)\subseteq R^\times$.
So then, note that for any $f\in R^\times$ we can write
$$f=[f\mod p]([f\mod p]^{-1} f)$$
which gives a decomposition $f=\zeta u$ as desired. The unicity is clear.
Note that the $p^\text{th}$-power map is surjective on $\mu^p(R)$ and thus we see that $R^\times/(R^\times)^p$ can be identified with $(1+pR)/(1+pR)^p$. We claim that this is isomorphic to $\overline{\mathbb{F}_p}$. To do this, we first note the following lemma:
Lemma 3: Let $u\in R^\times$ be such that $u=1\mod p$. Then, $u$ has $p^\text{th}$ root in $R^\times$ if and only if $u=1\mod p^2$.
Proof: The necessity of the claim is clear from Lemma 2. So, it suffices to show that if $u=1\mod p^2$ then $u$ has a $p^\text{th}$ root. Note that since $u\in R_n:= S[\zeta_{p^n-1}]$ (i.e. $R_n$ is the integral closure of $\Z_{(p)}$ in $\Z_{p^n}=\Z_p[\zeta_{p^n-1}]$) it suffices to show that $u$ has a $p^\text{th}$ root in $R_n$. But, since $R_n$ is integrally closed in $\Z_{p^n}$ it suffices to show that $u$ has a $p^\text{th}$ root in $\Z_{p^n}$. Note though that since $|u-1|\leqslant p^{-2}$ we have that $\log(u)$ exists in $\Z_{p^n}^\times$ and $|\log(u)|=|u-1|$ (e.g. see Theorem 8.7 of this). Since $|\log(u)|\leqslant p^{-2}$ we know that $p^{-1}\log(u)\in \Z_{p^n}^\times$ and, in fact, $|p^{-1}\log(u)|\leqslant p^{-1}$. Thus, we see that $\exp(p^{-1}\log(u)$ is a well-defined element of $\Z_{p^n}^\times$. Noting then that $\exp(p^{-1}\log(u))^p=u$ finishes the claim. $\blacksquare$
From this it's easy to see that $(1+p R)/(1+pR)^p$ can be identified with $(1+pR)/(1+p^2 R)$. To see that this is isomorphic to $\overline{\mathbb{F}_p}$ we may proceed as follows. Note that we have an isomorphism
$$(1+p R)/(1+p^2 R)\to pR/p^2 R$$
given by taking $1+px$ to $px$. Thus, it suffices to explain why $pR/p^2 R\cong \overline{\mathbb{F}_p}$. But, this is easy since $\overline{\mathbb{F}_p}\cong R/pR$ and we have a natural isomorphism $R/pR\to pR/p^2 R$ given by $x\mapsto px$. $\blacksquare$
As a last point, here's something to point out. A priori you might believe that what is really play here is the fact $p$ is not invertible in $\mathbb{Z}$. But, note the following:
Observation 1: If $S$ is a scheme over $\mathbb{F}_p$ which is perfect (i.e. the absolute Frobenius $F_S:S\to S$ is an isomorphism) then the same is true for all etale maps $X\to S$.
Proof: This follows from the fact that since $X/S$ is etale we have that $F_{X/S}:X\to X^{(p)}$ is an isomorphism. Indeed, note that $F_{X/S}$ is a universal homeomorphism by Tag0CCB. But, in addition, we see that the composition $X\to X^{(p)}\to S$ is etale since it coincides with $X\to S$. But, in addition the diagonal of $X^{(p)}\to S$ is etale (in fact an open embedding) since $X^{(p)}$ is etale (being the pullback of the map $X\to S$ along $F_S$). Thus, by the cancellation lemma $F_{X/S}$ is etale. But, by Tag025G that fact that $F_{X/S}$ is an etale homeomorphism implies it's an isomomorphism. Finally, note that $F_X$ is the composition of $F_{X/S}$ and the map $X^{(p)}\to X$ (which is a base change of $F_S$ and thus an isomorphism) and thus an isomorphism $\blacksquare$
Corollary 2: If $S$ is a perfect $\mathbb{F}_p$-scheme then the $p$-Kummer sequence
$$0\to\mu_p\to\mathbb{G}_m\to\mathbb{G}_m\to 0$$
is exact on the etale site of $S$.
Proof: Let $X\to S$ be an object of the etale site of $S$ and let $u\in \mathcal{O}_X(X)^\times$. We want to show that etale locally on $X$ we have that $u$ is a $p^\text{th}$ power. But, since $X\to S$ is etale by the above observation we see that $X$ is perfect so, in fact, $u$ is a $p^\text{th}$ power already in $X$. $\blacksquare$
So,the $p$-Kummer sequence is an exact sequence on all residue fields of $\mathrm{Spec}(\mathbb{Z})$ since these residue fields are either characteristic not $p$ or perfect of characteristic $p$, but is not an etale sequence integrally on $\mathrm{Spec}(\mathbb{Z})$. Something very subtle and interesting is happening integrally!