The problem is similar to this, are there any geometric methods that can measure the distance between two homographies and tell whether these homographies can describe the same or similar transformations?
Since homography is up to scale, one simple and straightforward algebraic method is to scale the homographies and align their 9th element to 1, and use these measures. But I am afraid that this algebraic method is not appropriate to compare two homographies.
