To get the perpendicular distance of a point Q to a line defined by two points P_1 and P_2 calculate this:
d = DOT(Q, CROSS(P_1, P_2) )/MAG(P_2 - P_1)
where DOT is the dot product, CROSS is the vector cross product, and MAG is the magnitude (sqrt(X*X+Y*Y+..))
Using Fig 5. You calculate d_1 the distance from sj to line (si->ei) and d_2 the distance from ej to the same line.
I would establish a coordinate system based on three points, two (P_1, P_2) for a line and the third Q for either the start or the end of the other line segment. The three axis of the coordinate system can be defined as such:
e = UNIT(P_2 - P_1) // axis along the line from P_1 to P_2
k = UNIT( CROSS(e, Q) ) // axis normal to plane defined by P_1, P_2, Q
n = UNIT( CROSS(k, e) ) // axis normal to line towards Q
where UNIT() is function to return a unit vector (with magnitude=1).
Then you can establish all your projected lengths with simple dot products. So considering the line si-ei and the point sj in Fig 5, the lengths are:
(l || 1) = DOT(e, sj-si);
(l |_ 1) = DOT(n, sj-si);
ps = si + e * (l || 1) //projected point
And with the end of the second segment ej, new coordinate axes (e,k,n) need to be computed
(l || 2) = DOT(e, ei-ej);
(l |_ 1) = DOT(n, ej-ei);
pe = ei - e * (l || 1) //projected point
Eventually the angle distance is
(d th) = ATAN( ((l |_ 2)-(L |_ 1))/MAG(pe-ps) )
PS. You might want to post this at Math.SO where you can get better answers.