Fast coordinate matching with k3match

Pim Schellart
LOFAR Transients Key Project Meeting, Meudon, December 2011

Transcript

1. Fast coordinate matching with k3match Pim Schellart M. v.d. Akker, A. Corstanje, E. Enriquez, H. Falcke, J. R. Hörandel, A. Nelles, S. Thoudam, S. ter Veen Cosmic Rays Key Science Project Radboud University Nijmegen
2. The problem • Which of the objects in catalog A are within 1” of the objects in catalog B? • What object from catalog A is closest to my search coordinates? • How do we answer these questions fast?
3. K3Match • Stand alone library written in C • Python bindings • idx_a, idx_b, d = k3match.celestial(ra_a, dec_a, ra_b, dec_b, r) • input and output using numpy arrays • Source code available under open source GPL license
4. KD-Trees J.L. Bentley, “Multidimensional binary search trees used for associative searching,” Communications of the ACM, 1975. Image credit: wikipedia
5. Range search
6. Range search
7. Range search
8. Range search
9. Range search
10. Range search
11. Range search
12. Range search
13. Optimized KD-Trees • J.H. Friedman. J.L. Bentley R.A. Finkel, “An algorithm for finding best matches in logarithmic expected time,” ACM Transactions on Mathematical Software. • Tree needs to be balanced, e.g. choose median as discriminator value at each partition. • With O(N) median, tree building takes O(k N log N) time. • Single query takes O(log N) time on average, total running time is O(N log N) when Ns ~ N.
14. Scaling
15. Wrong scaling due to unexpected behavior of quickselect median algorithm (solved) before after
16. RPM DEB Source Simple installation from source or binary package http://pschella.github.com/k3match
17. Conclusion • Easy to install • Easy to use • Fast • Get it now at http://pschella.github.com/k3match