If you assume associativity, then the generalization would probably be the Clifford Algebras. These then contain e.g. the Real Numbers $\mathbb{R}$, the Complex Numbers $\mathbb{C}$, the Quaternions $\mathbb{H}$ and the Dual Quaternions. However, they do not contain the non-associative algebras like the Octanions $\mathbb{O}$ or Cayley Algebras. This algebra is also often referred to / summarized as the Algebra of Hypercomplex Numbers, which I personally don't find so nice, but more on that later.
If you are interested in it I can recommend the book Hypercomplex Analysis: New Perspectives and Applications by Swanhild Bernstein, Uwe Kähler, ... (and so on)
Another generalization are the Cayley–Dickson Algebras (using the Cayley–Dickson Construction), which also include non-associative algebras. With the Cayley–Dickson Construction we can create an algebra with $2 \cdot n$ dimensions ($\mathbb{R}^{\left( 2 \cdot n \right)}$) from an algebra with $n$ dimensions ($\mathbb{R}^{n}$). The algebras generated by this process are known as Cayley-Dickson Algebras. For example, the Complex Numbers $\mathbb{C}$, the Qzuaternions $\mathbb{H}$, Octanions $\mathbb{O}$ and Sedenions $\mathbb{S}$ can be formed from them, all of which belong to these Cayley–Dickson Algebras.
"The Cayley–Dickson construction can be modified by inserting an extra sign at some stages. It then generates the "split algebras" in the collection of composition algebras instead of [] division algebras."
E.g. they also contain the Split Complex Numbers, which have the imaginary unit $j$ with $j^{2} = 1 ~\wedge~ j \ne \pm1$.
Another popular generalization of these algebras is the Algebra of Hypercomplex Numbers:
[...] a hypercomplex number is a number having properties departing from those of the real and complex numbers. The most common examples are biquaternions, exterior algebras, group algebras, matrices, octonions, and quaternions. - Weisstein, Eric W. "Hypercomplex Number." From MathWorld--A Wolfram Web Resource. https://mathworld.wolfram.com/HypercomplexNumber.html
With this definition, you could write a number of your searched generalization as $a = \sum\limits_{n = 0}^{k}\left[ a_{n} \cdot \gamma_{k} \right]$ where $\left( a_{1},\, a_{2},\, a_{3},\, \dots,\, a_{n} \right) \in \mathbb{R}^{n}$, $\gamma_{n}^{b} \in \mathbb{R}$, $\gamma_{0} = 0$ (where $b$ is some constant) and $k \in \mathbb{Z}$. Usually one would use $\mathrm{i}_{n}$ instead of $\gamma_{n}$ as the imaginary unit where $\left| \mathrm{i}_{n} \right| = 1$. Usually $\mathrm{i}$ is chosen such that $\mathrm{i}^{b} = \left\{ -1,\, \pm0,\, +1 \right\}$ holds (where $b$ is some constant). In this algebra you can also consider products of different imaginary units as their own imaginary unit. E.G. this algebra inlcudes the Complex Numbers $\mathbb{C}$, the Qzuaternions $\mathbb{H}$, Octanions $\mathbb{O}$, Sedenions $\mathbb{S}$, Dual Numbers, ...
For example, you can write the Dual Quaternions $$\hat{A} = \left( A,\, B \right) = a_{0} + a_{1} \cdot i + a_{2} \cdot j + a_{3} \cdot k + b_{1} \cdot \epsilon \cdot i + b_{2} \cdot \epsilon \cdot j + b_{3} \cdot \epsilon \cdot k$$ as $$\hat{A} = \sum\limits_{n = 0}^{6}\left[ a_{n} \cdot \mathrm{i}_{k} \right]$$ in the hypercomplex sense, where $i,\, j,\, k$ are the imaginary units of the Quaternions $\mathbb{H}$ with $i^{2} = j^{2} = k^{2} = i \cdot j \cdot k = -1$, $\epsilon$ is the imaginary unit of the Dual Numbers with $\epsilon^{2} = 0 ~\wedge~ \epsilon \ne 0$, $b_{n} = a_{2 \cdot n + 1}$ and $\mathrm{i}_{0} = 0,\, \mathrm{i}_{1} = i,\, \mathrm{i}_{2} = j,\, \mathrm{i}_{3} = k,\, \mathrm{i}_{4} = \epsilon \cdot i,\, \mathrm{i}_{5} = \epsilon \cdot j,\, \mathrm{i}_{6} = \epsilon \cdot k$.
Most hypercomplex numbers can be divided into a scalar part (often the real part) and a vector part (often the imaginary parts). The hypercomplex numbers also do not have to be commutative and aassociative with respect to multiplication and can contain Zero Divisors e.g. some Sedenions $\mathbb{S}$.
I myself have always considered this algebra to be the alegbra of imaginary units. This is the only algebra I have listed that includes all the algebras you mentioned, generalized and expanded.
Another useful generalization is the Composition Algebras or Real Normed Algebra. They need not be commutative or associative.
A real normed algebra [...] is a multiplication $*$ on $\mathbb{R}^{n}$ that respects the length of vectors, i.e., $\left| x \cdot y \right| = \left| x \right| \cdot \left| y \right|$ for $x,\, y \in \mathbb{R}^{n}$. - Garibaldi, Skip; Rowland, Todd; and Weisstein, Eric W. "Real Normed Algebra." From MathWorld--A Wolfram Web Resource. https://mathworld.wolfram.com/RealNormedAlgebra.html
It applies to these that they have no Zero Divisors and can only have $n$ dimensions where $n \in \left\{ 1,\, 2,\, 4,\, 8 \right\}$.
These then contain e.g. the Real Numbers $\mathbb{R}$, the Complex Numbers $\mathbb{C}$, the Quaternions $\mathbb{H}$ and the Octanions $\mathbb{O}$.
Some square matrices can also be viewed as a generalization of such algebras:
The $\mathbb{R}^{n \times n}$ ($n \in \mathbb{N}$) is an algebra with the $n \times n$-identity matrix as the unit element – i.e. also a hypercomplex algebra. More precisely, it is an associative hypercomplex algebra and thus also a ring and as such also unitary.$^{\left[ 1 \right]}$
