Vector spaces and Affine spaces are abstractions of different properties of Euclidean space. Like many abstractions, once abstracted they become more general.
A Vector space abstracts linearity/linear combinations. This involves the concept of a zero, scaling things up and down, and adding them to each other.
An Affine space abstracts the affine combinations. You can think of an affine combination as a weighted average, or a convex hull (if you limit the coefficients to be between 0 and 1).
As it turns out, you do not need a zero, nor do you need the concept of "scaling", nor do you need full on addition, in order to have a concept of weighted average and convex hull within a space.
Now, you can take your affine space $\mathbb {A}$ , pick any point $o$ from it, and talk about ${\mathbb A}-o$ as a vector space.
Mapping your $n$ dimensional affine space over $\mathbb {R}$ to $\mathbb{R}^n$ is in effect picking a point, and mapping it to a space with more structure than your original affine space. So you end up with the origin $o$ appearing special, but that is an artifact of your mapping.
If you look at the Earth, the lines of longitude have a zero point, but that zero point is arbitrary -- it has no meaning. The lines of longitude are an affine space. We measure them in degrees (or radians), and we have picked a zero, but other than it being useful to agree where the zero is, it isn't a special line.
The space of rotations around a circle, on the other hand, have a zero that is meaningful -- zero means you don't rotate. We measure them as a vector space.
The lines of longitude are measured as rotations away from our arbitrary point we assigned zero. But what matters about them is the ability to say how far apart two longitude are from each other, not any one line's absolute value.
If we where doing some math and it would be useful to move the zero of longitude, we are free to do so. But if we want to move the zero in the space of rotation (to say bending things 90 degrees) we are not nearly as free.
In general, your location is an affine space, as there is no special place, and scaling your location by a factor of 3 makes no sense, and adding two locations makes no sense -- but taking the average of two locations makes sense.
The (directed) distance between locations is a vector space. Saying something is twice as far as another distance makes sense, the "same place" (distance zero) makes sense, and adding two directed distances together makes sense.
And you can pick a spot and describe locations as the directed distance from that particular spot, but the spot picked was arbitrary, and if it would be useful to pick a different spot, you are free to.