More realistic precision for cosine distance results

Chris Veness' discussion of using the spherical law of cosines (http://www.movable-type.co.uk/scripts/latlong.html) suggests it gives well-conditioned results "around 1 metre" in JavaScript (this assumes a sphere with 6,371km radius).

In Chrome 33, Math.pow(Math.cos(Math.PI / 4), 2) yields 0.4999999999999999.  When we take the arccosine of twice this, we get something significantly different than zero.  Multiplying by 6371 means we can't assert that this is within 1e-9 of zero.
This commit is contained in:
Tim Schaub
2014-02-24 15:54:17 -07:00
parent ac83bb78f1
commit bfc42ffa62

View File

@@ -379,7 +379,7 @@ describe('ol.Sphere', function() {
for (i = 0; i < expected.length; ++i) {
e = expected[i];
expect(sphere.cosineDistance(e.c1, e.c2)).to.roughlyEqual(
e.cosineDistance, 1e-9);
e.cosineDistance, 1e-3);
}
});