The answer is yes, and in fact your theory should be mutually interpretable with Morse-Kelley set theory. The relevant proofs are rather elaborate however, so I'll start by outlining the proof strategy I found.
The basic idea behind our strategy is to build the constructible universe embedded within the ordinals. Each constructible set can be coded using its definition, which is nothing more than a list containing: an ordinal marking its place in the hierarchy, a formula describing the set, and a list of parameters to the formula which must be (representations of) constructible sets appearing strictly earlier in the hierarchy. The membership relationship for the structure can then be coded as a proper class of ordered pairs, which can be constructed by transfinite recursion. To get Extensionality, we can define a notion of membership isomorphism between our representations of constructible sets, and then define $L$ as containing only the smallest representation from each isomorphism class. This approach gives the Foundation axiom for free, simply because the ordinals are already wellfounded, and we get Infinity from the existence of $\omega$. If the order type of the class of all ordinals is regular, then we can prove Replacement (every function admits a constructible set which contains its image) simply by selecting a sufficiently large layer of the constructible hierarchy. Relying on the large cofinality, we can also prove the Reflection principle for every first order formula, which can then be used to prove Specification for every first order formula. By a similar technique, we can show that any initial segment of $L$ which is bounded by a regular ordinal will be closed under subset formation, so we can construct Powersets by appealing to regular ordinals. The remaining axioms of $\sf{ZFC}$ are trivial, so we get our interpretation.
This construction makes a few demands on the ordinal numbers. To get the axiom of Infinity, we need to prove the existence of the infinite ordinal $\omega$. To get Replacement and Specification, we need the class of all ordinals to have an order type which is like a regular ordinal. Finally, to get Powersets, we need to prove that the class of regular ordinals is cofinal in the class of all ordinals (it's enough to have a cofinal class of ordinals such that each ordinal is regular). Your axioms allow us to meet all three of these requirements, and these are the only requirements made of the ordinals. To clarify, when I say we need an order type to be regular, I mean in the second order sense: for $\kappa$ to be regular, it's required that $\kappa$ is a limit ordinal, and that for any formula-defined function $F:\alpha\to\kappa$, the image $\{F(x) : x<\alpha\}$ has a supremum strictly less than $\kappa$.
This construction also makes a few demands on what sorts of structures we can encode. The membership relation on $L$ requires that we already have proper classes of ordered pairs of representations, so these ordered pairs need to be coded as ordinals (since in the beginning, we can only make sets of ordinals). Each representation also needs to be coded as a list, containing: an ordinal, a formula, and some other smaller representations. The formula can be coded as a finite list of symbols, with the symbols being finite ordinals, and so any given representation is just a finitely nested list of lists and ordinals.
To solve the encoding problem, we can encode ordered pairs via $(x,y)=(x+y)^2 + x$, which is easily shown to obey the fundamental property $(x,y)=(a,b) \iff [x=a\land y=b]$. We can then encode finite lists of ordinals by nesting ordered pairs, which identifies each finite list of ordinals with some ordinal. The indexing operation for such lists is easily defined via finite recursion, which can be defined using ordered pairs and the pre-existing architecture of sets. Since the lists are identified with ordinals, and lists can contain ordinals, then lists can contain lists, so we end up having nested lists of ordinals as desired. All of this only requires ordinal arithmetic: addition, multiplication, and exponentiation. Because ordinal arithmetic is strictly monotonic in the right argument (except for a few edge cases), these operations can be defined using only the operation which sends a set to its order type (your $\#$ function allows us to calculate the order type). I will demonstrate how to define addition, and leave multiplication and exponentiation to you.
$$\alpha+1 := \min\{\beta : \alpha<\beta\}$$
$$1+\alpha := \begin{cases}
\alpha+1 &: \alpha<\omega \\
\alpha &: \text{otherwise}
\end{cases}$$
$$\begin{align}
\alpha+\beta=\gamma \iff \exists Y, \min(Y)=\alpha &\land [\forall(y\in Y), y=\min(Y) \lor y=\lim\{z+1 : z\in Y \land z<y\}] \\
&\land \operatorname{order}(Y)=\beta+1 \land \lim(Y)=\gamma
\end{align}$$
Assuming the class of all ordinals has unlimited cofinality, we can prove via transfinite induction on $\beta$ that there always exists a unique $\gamma$ satisfying $\alpha+\beta=\gamma$ as defined above, and we clearly see how this is equivalent to the typical recursive definition of ordinal addition. A similar technique allows us to define multiplication and exponentiation, since those are also strictly monotonic outside of the edge cases, and the edge cases ($0\cdot \beta$ and $0^\beta$ and $1^\beta$) can be handled individually. Later I'll clarify how to define the order type of a set using your $\#$ function, so we will see that everything works out.
In summary, we only have three objectives: find $\omega$, prove the class of all ordinals has a regular order type, and prove that the class of regular ordinals is itself cofinal in the ordinals. Below, I have included proofs which handle all three of these objectives. I'll leave the construction of $L$ (and the proofs of its properties) as a matter you can handle yourself, since it's unrelated to the specifics of your theory. If you have trouble proving the properties of $L$ using the tools I've laid out, leave a comment or ask another question and I'll try to address it.
Definition: $\mathbb{N}=\{n : \forall(x\leq n), x=0\lor \exists(y<x), x=y+1\}$
Lemma: $0\in\mathbb{N}\land\forall(n\in\mathbb{N}), n+1\in \mathbb{N} \land \forall(m<n), m\in\mathbb{N}$
Lemma: $\forall(S\subseteq \mathbb{N}), S=\emptyset \lor [\lim(S)\in\mathbb{N}\implies \lim(S)\in S]$
Theorem: $\mathbb{N}$ has a limit.
To prove this theorem, first notice that whenever $\lim(S)\in S$ then $S$ is accessible from $\{1\}$. Indeed, let $Z=\{\lim(S)\}$, and since $\lim(S)\in S$ then $\lim(S)=\max\{S\}$, so we clearly see that $Z\text{ cofinal }S$. Since $\operatorname{count}(Z)=\{1\}$, then indeed $S$ is accessible from $\{1\}$. Next, given any $x\in\mathbb{N}$, the set $\{z : z<x\}$ is hyperaccessible from $\{1\}$. Indeed, by our lemma any subset $C\subseteq\{z : z<x\}$ either has $\lim(C)\in C$ and thus $C\text{ accessible from }\{1\}$ as previously established, or else $C=\emptyset$ and thus $C\text{ accessible from } \{1\}$ trivially, therefore $\{z : z<x\}\text{ hyperaccessible from }\{1\}$. This holds for all $x\in\mathbb{N}$, so we have $\mathbb{N}\subseteq\{x | \{z : z<x\}\text{ hyperaccessible from }\{1\}\}$, which has a limit due to your hyeraccessibility axiom.
Definition: $\omega = \lim(\mathbb{N})$
Using your $\#$ function to enumerate the members of sets, we can easily define a function which maps each set to its order type, using some placeholder value in case the order type cannot be assigned to any ordinal in our domain of discourse. We're going to use $\Omega$ as if it were an ordinal, to assign cofinalities and order types. We extend the order relation so that $x<\Omega$ for all ordinals $x$, and whenever $X$ is unbounded we may write $\lim(X)=\Omega$. We assign sets their order types as follows.
Definition: $\Omega = \{x : x=x\}$
Definition: Let $\operatorname{order}(X) = \min(\{\alpha : \forall(y\in \operatorname{count}(X)), y\neq 1+\alpha\})$, taking the value $\operatorname{order}(X)=\Omega$ whenever no such $\alpha$ exist.
I had to use the $1+x$ function in the generic case, because your $\#$ function starts counting from $1$ instead of from $0$. We have a few easily proven lemmas about order types.
Lemma: $\operatorname{order}(X)\leq \lim(X)+1$
Lemma: $X\subseteq Y \implies \operatorname{order}(X)\leq \operatorname{order}(Y)$
Lemma: $\operatorname{count}(\operatorname{count}(X))=\operatorname{count}(X)$, and consequently $\operatorname{order}(\operatorname{count}(X))=\operatorname{order}(X)$.
Lemma: $\operatorname{count}(X) = \{1+x : x<\operatorname{order}(X)\}$
Definition: Let $\operatorname{cf}(X) := \min\{\alpha : \exists(Z\text{ cofinal }X), \alpha=\operatorname{order}(Z)\}$, taking the value $\operatorname{cf}(X) = \Omega$ whenever no such $\alpha$ exist. We also define $\operatorname{cf}(\alpha) = \operatorname{cf}(\{x : x<\alpha\})$ when $\alpha$ is an ordinal.
Lemma: $\operatorname{cf}(X)\leq \operatorname{order}(X)$, and similarly $\operatorname{cf}(\alpha)\leq \alpha$.
Lemma: if $X\text{ cofinal }Y$, and $Y\text{ cofinal }Z$, then $X\text{ cofinal }Z$.
Theorem: $X_0\text{ cofinal }X_1 \implies \operatorname{cf}(X_0)=\operatorname{cf}(X_1)$
We prove the above theorem in case $X_1$ is nonempty and contains no maximum, since the other cases are trivial. For $j=0,1$, find sets $Z_j\text{ cofinal }X_j$ such that $\operatorname{order}(Z_j)=\operatorname{cf}(X_j)$. Since $X_0\text{ cofinal }X_1$ then $Z_0\text{ cofinal }X_1$, so immediately $\operatorname{cf}(X_1)\leq\operatorname{order}(Z_0)=\operatorname{cf}(X_0)$. For the other direction, define the function symbol $F(x_1)=\min\{x_0 : x_0\in X_0 \land x_0\geq x_1\}$, and we easily infer that $\forall(x_1\in X_1), F(x_1)\in X_0$, which works because $X_1$ has no maximum. Letting $Z=\{y : \exists(x_1\in Z_1), y=F(x_1)\}$, we quickly prove that $Z\text{ cofinal }X_0$ so that $\operatorname{cf}(X_0)\leq \operatorname{order}(Z)$, and also $\operatorname{order}(Z')\leq \operatorname{order}(Z_1)=\operatorname{cf}(X_1)$, therefore $\operatorname{cf}(X_0)\leq \operatorname{cf}(X_1)$, proving equality.
Theorem: $\operatorname{order}(X_0)=\operatorname{order}(X_1) \implies \operatorname{cf}(X_0)=\operatorname{cf}(X_1)$
Find $Z_0\text{ cofinal }X_0$ such that $\operatorname{order}(Z_0)=\operatorname{cf}(X_0)$, and let $Z_1=\{x_1\in X_1 : \exists(x_0\in X_0), \#^{X_0}(x_0)=\#^{X_1}(x_1)\}$. We very easily prove that $\operatorname{order}(Z_0)=\operatorname{order}(Z_1)$, and that $Z_1\text{ cofinal }X_1$, and thus $\operatorname{cf}(X_1)\leq\operatorname{order}(Z_1)=\operatorname{order}(Z_0)=\operatorname{cf}(Z_0)$. The converse is simlar, therefore $\operatorname{cf}(X_0)=\operatorname{cf}(X_1)$.
corollary: $\operatorname{cf}(\operatorname{order}(X))=\operatorname{cf}(X)$
Theorem: $\operatorname{cf}(\operatorname{cf}(X))=\operatorname{cf}(X)$
Let $\alpha=\operatorname{cf}(X)$ and find $Z\text{ cofinal }X$ such that $\operatorname{order}(Z)=\alpha$. The previous corollary proves $\operatorname{cf}(\alpha)=\operatorname{cf}(Z)$, and the theorem before that then proves $\operatorname{cf}(Z)=\operatorname{cf}(X)$, therefore $\operatorname{cf}(\alpha)=\alpha$ as required.
Theorem: Given $\alpha\geq \omega$, we have $X\text{ accessible from } \{y : y<\alpha\}$ if and only if $\operatorname{cf}(X)\leq \alpha$
For convenience label $Y=\{y : y<\alpha\}$. In the case where $X$ is accessible from $Y$, by definition there's $Z\text{ cofinal }X$ such that $\operatorname{count}(Z)\subseteq Y$ and thus $\operatorname{order}(Z)\leq \operatorname{order}(Y)=\alpha$, therefore $\operatorname{cf}(X)\leq\operatorname{order}(Z)\leq\alpha$. Conversely take the case where $\operatorname{cf}(X)\leq\alpha$, then we can find $Z\text{ cofinal }X$ such that $\operatorname{order}(Z)\leq \alpha$. Now all $x\in\operatorname{count}(Z)$ admit $y<\operatorname{order}(Z)$ with $x=1+y$, so either $x<\omega$ and thus $x=1+y<\omega\leq \alpha$, or else $x\geq \omega$ and so $x=1+y=y<\alpha$. In any case we see $x<\alpha$, hence $\operatorname{count}(Z)\subseteq Y$.
Corollary: $X\text{ has a limit }\iff \operatorname{cf}(X)\neq\Omega$.
Corollary: $\operatorname{cf}(\Omega)=\Omega$.
To prove the first corollary, notice that if $\operatorname{cf}(X)\neq\Omega$ then $X$ is accessible from $\{x : x<\operatorname{cf}(X)\}$ since $\operatorname{cf}(\operatorname{cf}(X))=\operatorname{cf}(X)$, and that set clearly has a limit, therefore $X$ also has a limit due to the Accessibility axiom. Conversely if $\operatorname{cf}(X)=\Omega$ then necessarily $X\text{ cofinal }\Omega$, so $X$ cannot have any limit since $\Omega$ does not have a limit. Our second corollary then follows from the first: since $\Omega$ does not have a limit, then $\operatorname{cf}(\Omega)=\Omega$.
Definition: We say $\alpha$ is a regular ordinal whenever $\alpha\geq \omega$ and $\operatorname{cf}(\alpha)=\alpha$.
Definition: We say $\alpha$ is a cardinal whenever $\alpha$ is finite, regular, or a limit of regular ordinals.
Definition: $|X|=\lim\{\alpha : \alpha\leq \operatorname{order}(X) \land \alpha\text{ is a cardinal}\}$. Similarly define $|\alpha|=|\{x : x<\alpha\}|$.
Lemma: $\operatorname{cf}(\omega)=\omega$
Theorem: $|X|$ is always a cardinal.
We have $|X|$ being the limit of a set of cardinals, and so either $|X|$ is a limit of regular ordinals, or it's a limit of finite ordinals. In the former case $|X|$ is clearly a cardinal, and in the latter case either $|X|<\omega$ so that $|X|$ is a finite cardinal, or else $|X|=\omega$. Our lemma shows that $\omega$ is regular, and thus a cardinal, so $|X|$ is a cardinal in every case.
Lemma: $\operatorname{cf}(X)\leq |X|\leq\operatorname{order}(X)$
Lemma: $X\subseteq Y \implies |X|\leq|Y|$
Lemma: $|\alpha|=\alpha$ whenever $\alpha$ is a cardinal.
Lemma: $|\operatorname{order}(X)|=|X|$
Lemma: $\forall X, \forall(\alpha\leq\operatorname{order}(X)), \exists(C\subseteq X), \operatorname{order}(C)=\alpha$
Theorem: Given $\alpha\geq \omega$, we have $X\text{ hyperaccessible from }\{y : y<\alpha\}$ if and only if $|X|\leq \alpha$
Letting $Y=\{y : y<\alpha\}$ for convenience, by definition we have $X$ hyperaccessible from $Y$ if and only if $\forall(C\subseteq X), C\text{ accessible from } Y$, which in turn holds if and only if $\forall(C\subseteq X), \operatorname{cf}(C)\leq \alpha$, which is finally equivalent to having $\alpha \geq \lim\{\beta : \exists(C\subseteq X), \beta=\operatorname{cf}(C)\}$. To get our conclusion, simply notice that that limit is precisely $|X|$. Indeed, every $C\subseteq X$ has $\operatorname{cf}(C)\leq |C|\leq |X|$ so the limit is at most $|X|$, and conversely by our lemma the subsets $C$ of $X$ cover every possible order type below $\operatorname{order}(X)$, and thus we have sets $C$ of every possible cofinality below $|X|$, the limit of which is exactly $|X|$.
Theorem: $\forall \alpha, \exists \beta, \alpha<|\beta|$
The case where $\alpha$ is finite is trivial, so assume $\alpha\geq\omega$. Letting $Y=\{x : x<\alpha\}$, we have the set $\{x | \{z : z<x\}\text{ hyperaccessible from }Y\} = \{x : |X|\leq \alpha\}$, which has a limit by the hyperaccessibility axiom. Label $\beta = \lim(\{X : |X|\leq\alpha\})$, then since $|\alpha|=\alpha$ necessarily $\beta\geq \alpha$. If we had $|\beta|=\alpha$ however, then it would follow that $\beta<\alpha+1$ and thus $|\alpha|<|\alpha+1|$. This necessitates that $|\alpha+1|$ is either: finite (impossible since $\omega\leq\alpha$), or it's regular, or it's a limit of regular ordinals. In either of the latter cases we would find that $\alpha+1$ is regular, but this is not possible. Indeed, clearly $\{\alpha\}\text{ cofinal }\{x : x<\alpha+1\}$, and also $\operatorname{order}(\{\alpha\})=1$, therefore $\operatorname{cf}(\alpha+1)=1\leq\omega<\alpha+1$. From this contradiction, infer $\alpha<|\beta|$.
Corollary: $\operatorname{cf}(\{\alpha : \alpha\text{ is a cardinal}\})=\Omega$, and thus $\operatorname{cf}(\{\alpha : \alpha=\operatorname{cf}(\alpha)\})=\Omega$
The first corollary follows immediately from our theorem and the fact that $\operatorname{cf}(\Omega)=\Omega$. Similarly, the second claim follows since the regular ordinals are cofinal in the cardinals. Indeed, every infinite cardinal is a regular ordinal or a limit of regular ordinals, and since every cardinal admits another cardinal above it, then every infinite cardinal admits some regular ordinal above it.
To complete our work, we just need to clarify that our cofinality operation $\operatorname{cf}(X)$ does what we expect it to. In particular, we need to prove that for any ordinal $\kappa$ with $\operatorname{cf}(\kappa)=\kappa$, any $\alpha<\kappa$, and any formula-defined function $F:\alpha\to\kappa$, that the image $\{F(x) : x<\alpha\}$ admits a limit strictly less than $\kappa$. We also need an analogous result when using $\Omega$ in place of $\kappa$. We prove these results below, and with that we'll have completed all of our objectives.
Theorem: For any formula $\phi(x,y)$ obeying $\forall x, \exists y, \phi(x,y)$, we have $\forall\alpha, \exists\beta, \forall(x<\alpha), \exists(y<\beta), \phi(x,y)$
Define a function symbol $F(x)=\min\{y : \phi(x,y)\}$, which is well defined by premise on $\phi$, and define another function symbol $Y_\alpha=\{y : \exists(x\leq \alpha), y=F(x)\}$ to be the image of $F$ over the segment $\{x : x<\alpha\}$. We prove by transfinite induction on $\alpha$ that $Y_\alpha$ has a limit. Suppose our claim holds for all $x<\alpha$, then define $G(x)=\lim(Y_x)$, and notice that $G$ is well defined and monotonic over the domain $x<\alpha$. Finally, define $Z=\{y : \exists(x<\alpha), G(x)=y\}$ to be the image of $G$, and since $G$ is monotonic, we easily prove by transfinite induction on $x$ that $\#^X(G(x))\leq 1+x$ for all $x<\alpha$. It follows that $\operatorname{order}(Z)\leq \alpha$, and since $\operatorname{cf}(Z)\leq \operatorname{order}(Z)\leq \alpha$ then $\operatorname{cf}(Z)<\Omega$, therefore $Z$ has a limit. Let $\beta=\max\{\lim(Z),F(\alpha)\}$, then all $x<\alpha$ have $F(x)\leq G(x)\in Z$ and therefore $F(x)\leq\beta$, hence all $x\leq \alpha$ have $F(x)\leq \beta$, proving that $\lim(Y_\alpha)=\beta$ and completing our induction. Circling back around to our theorem, and letting $\beta=\lim(Y_\alpha)$ as previously, we see that all $x<\alpha$ admit $y<\beta$ such that $\phi(x,y)$, namely by using $y=F(x)$.
Corollary: If $\kappa$ is regular and $\forall x, \exists(y<\kappa), \phi(x,y)$, then $\forall\alpha, \exists(\beta<\kappa), \forall(x<\alpha), \exists(y<\beta), \phi(x,y)$
The proof of the above corollary is almost identical to the proof of our theorem but we use $\kappa$ as the upper bound to our ordinals instead of $\Omega$. After asserting $F$ is well defined, we should clarify that we always have $F(x)<\kappa$, and instead of proving $Y_\alpha$ has a limit we instead prove $\lim(Y_\alpha)<\kappa$. We define $G$ and $Z$ the same, prove $\operatorname{cf}(Z)<\kappa$ as expected, and finally infer $\lim(Z)<\kappa$ since $\kappa$ is regular. When we construct $\beta$ near the end, we need only clarify that $\beta<\kappa$, and the rest of the proof is identical.
