Fast coordinate matching with k3match

Fast coordinate matching with k3match

Pim Schellart
LOFAR Transients Key Project Meeting, Meudon, December 2011

Ab44292d7d6f032baf342a98230a6654?s=128

transientskp

June 23, 2012
Tweet

Transcript

  1. Fast coordinate matching with k3match Pim Schellart M. v.d. Akker,

    A. Corstanje, E. Enriquez, H. Falcke, J. R. Hörandel, A. Nelles, S. Thoudam, S. ter Veen Cosmic Rays Key Science Project Radboud University Nijmegen
  2. The problem • Which of the objects in catalog A

    are within 1” of the objects in catalog B? • What object from catalog A is closest to my search coordinates? • How do we answer these questions fast?
  3. K3Match • Stand alone library written in C • Python

    bindings • idx_a, idx_b, d = k3match.celestial(ra_a, dec_a, ra_b, dec_b, r) • input and output using numpy arrays • Source code available under open source GPL license
  4. KD-Trees J.L. Bentley, “Multidimensional binary search trees used for associative

    searching,” Communications of the ACM, 1975. Image credit: wikipedia
  5. Range search

  6. Range search

  7. Range search

  8. Range search

  9. Range search

  10. Range search

  11. Range search

  12. Range search

  13. Optimized KD-Trees • J.H. Friedman. J.L. Bentley R.A. Finkel, “An

    algorithm for finding best matches in logarithmic expected time,” ACM Transactions on Mathematical Software. • Tree needs to be balanced, e.g. choose median as discriminator value at each partition. • With O(N) median, tree building takes O(k N log N) time. • Single query takes O(log N) time on average, total running time is O(N log N) when Ns ~ N.
  14. Scaling

  15. Wrong scaling due to unexpected behavior of quickselect median algorithm

    (solved) before after
  16. RPM DEB Source Simple installation from source or binary package

    http://pschella.github.com/k3match
  17. Conclusion • Easy to install • Easy to use •

    Fast • Get it now at http://pschella.github.com/k3match