The problem is related to but not an exact match with linear complexity of sequences. I try and reuse some of the algebraic machinery from that theory as I recall it.
My observation. The example sequence suggested by @xxxxxxxxx, namely $s_{k+1}=1$, $s_i=0$ whenever $i\neq k+1$, is a counterexample unless $k+1$ is a factor of $p-1$.
The matrix $H$ has then all ones on the wrong diagonal and zeros elsewhere, so it has full rank.
It is worth noting that in the case $k=2$, $p=7$, where Youzhe Heng verified the existence of a solution, we do have $k+1=3$, $p-1=6$ and therefore also $k+1\mid p-1$.
I think of this result only as a beginning of the full study of this problem. As an answer it is thus unsatisfactory, but them's the breaks. On with the story. Obviously
everything needs to be double-checked.
We turn a sequence $(a_1,a_2,\ldots)$ of entries from $\Bbb{F}_p$ into a power series
$$A(T)=a_1+a_2T+a_3T^2+\cdots\in \Bbb{F}_p[[T]].$$
We apply this only to sequences of length $2k+1$ that appear as columns of the matrix equation in the question. What this means is that we only know the terms up to degree $T^{2k}$. So we need to keep in mind that we only know the resulting power series modulo the ideal $I=T^{2k+1}\Bbb{F}_p[[T]]$. Effectively, I will be doing the arithmetic in the quotient ring $R:=\Bbb{F}_p[[T]]/I$.
With a fixed element $n\in\Bbb{F}_p$ we associate the sequence $(1,n,n^2,n^3,\ldots)$. This then yields the geometric series
$$G_n(T)=1+nT+n^2T^2+n^3T^3+\cdots=\frac1{1-nT}\in\Bbb{F}_p[[T]].$$
Let us fix a subset $M=\{n_1,n_2,\ldots,n_{k+1}\}\subseteq\Bbb{F}_p$ of $k+1$
distinct elements. The question is whether we can choose the set $M$ in such a way that the series
$$
S(T)=s_1+s_2T+\cdots+s_{2k+1}T^{2k}
$$
or rather, its coset in the quotient ring $R$, lies in the $\Bbb{F}_p$ span
$$
V(M)=\langle G_n(T)\mid n\in M\rangle_{\Bbb{F}_p}\subseteq R.
$$
I will need the following idea from the theory of linear complexity (in the context of the Berlekamp-Massey algorithm it is called the key equation), characterizing the elements of the space $V(M)$.
Lemma. Denote
$$F_M(T)=\prod_{n\in M}(1-nT).$$
If $R(T)\in V(M)$, then there exists a polynomial $Q(T)$ of degree $\le k$ such that
$$
F_M(T)R(T)\equiv Q(T)\pmod I.\tag{1}
$$
Proof. The set of polynomials of degree $\le k$ is closed under linear combinations, so it suffices to check (1) for the generating elements $G_n(T), n\in M$. But, if we fix an element $n'\in M$, then
$$G_{n'}(T)F_M(T)=\frac{F_M(T)}{1-n'T}=\prod_{n\in M, n\neq n'}(1-nT),$$
which is a polynomial of degree $\le k=|M|-1.$ QED
At long last we are ready for the proof of my observation. In the example situation we have
$$S(T)=T^k.$$
Assume that we can find a set $M\subseteq \Bbb{F}_p$ of size $k+1$ such that the OP's equation has a solution. In other words, we assume that $S(T)\in V(M)$. We
have $\deg F_M(T)\le k+1$ (this is actually an equality unless $0\in M$). The key equation $(1)$ thus promises the existence of a polynomial $Q(T)$ of degree $\le k$
such that
$$T^k F_M(T)\equiv Q(T)\pmod{T^{2k+1}}.$$
Remembering that the constant term of $F_M(T)$ is equal to $1$, the special shape of the factor $T^k$ then forces $F_M(T)$ to be of the form
$$
F_M(T)=1+a T^{k+1}\tag{2}
$$
for some $a\in\Bbb{F}_p$. The case $a=0$ is uninteresting (then $M=\{0\}$ and $k=1$), so we can conclude that $0\notin M$. As $1/n$ is a zero of $F_M(T)$ for every $n\in M$, $(2)$ implies that the equation
$$
1+a x^{k+1}=0\equiv x^{k+1}=-1/a\tag{3}
$$
has $k+1$ distinct solutions in the prime field $\Bbb{F}_p$. The ratio of any two solutions $x,x'$ of $(3)$ satisfies $(x/x')^{k+1}=1$. So for $(3)$ to have $k+1$
solutions in the prime field, we need the said prime field to contain a primitive root of unity of order $k+1$. Such a primitive root is well known to exist if and only if $k+1\mid p-1$.
Some concluding remarks:
- In Youzhe Heng's solution for the special $s$-sequence in the case $k=2$, $p=7$, the variables $n_i$ were exactly the third roots of unity $1,2,4$ in $\Bbb{F}_7$. This aligns well enough with my findings.
- A small test case to verify my observation would be to brute force $k=2$, $p=11$. Again with the $s$-sequence $(0,0,1,0,0)$. I claim that no solutions exist.
- If $k+1\mid p-1$, then our special example $s$-sequence is NOT a counterexample. We can use the prescribed roots of unity as the set $M$. The first $k+1$ equations then have a unique solution to $b_i$s (Vandermonde, or Discrete Fourier Transform). In other words, if $\zeta\in\Bbb{F}_p$ is a primitive root of unity of order $k+1$,
we can use $n_j=\zeta^{j-1}$ for all $j, 1\le j\le k+1$. Then the first $k+1$ equations look like
$$
\left(\begin{array}{ccccc}
1&1&1&\cdots&1\\
1&\zeta&\zeta^2&\cdots&\zeta^k\\
1&\zeta^2&\zeta^{2\cdot2}&\cdots&\zeta^{2k}\\
\vdots&\vdots&\vdots&\ddots&\vdots\\
1&\zeta^k&\zeta^{2k}&\cdots&\zeta^{k\cdot k}
\end{array}\right)
\left(\begin{array}{c}
b_1\\ b_2\\ b_3 \\ \vdots \\ b_{k+1}
\end{array}\right)=
\left(\begin{array}{c}
s_1\\ s_2\\ s_3 \\ \vdots \\ s_{k+1}
\end{array}\right).
$$
As this time the coeffiecient matrix is Vandermonde type square matrix, this smaller system has a unique solution. A key ingredient is that when we extend the system to $2k+1$ equations, the next $k$ equations will simply repeat the top $k$ equations. Due to powers of $\zeta$ as well as the $s_i$s repeating with period $k+1$.
