Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast coordinate matching with k3match

Fast coordinate matching with k3match

Pim Schellart
LOFAR Transients Key Project Meeting, Meudon, December 2011



June 23, 2012


  1. Fast coordinate matching with k3match Pim Schellart M. v.d. Akker,

    A. Corstanje, E. Enriquez, H. Falcke, J. R. Hörandel, A. Nelles, S. Thoudam, S. ter Veen Cosmic Rays Key Science Project Radboud University Nijmegen
  2. The problem • Which of the objects in catalog A

    are within 1” of the objects in catalog B? • What object from catalog A is closest to my search coordinates? • How do we answer these questions fast?
  3. K3Match • Stand alone library written in C • Python

    bindings • idx_a, idx_b, d = k3match.celestial(ra_a, dec_a, ra_b, dec_b, r) • input and output using numpy arrays • Source code available under open source GPL license
  4. KD-Trees J.L. Bentley, “Multidimensional binary search trees used for associative

    searching,” Communications of the ACM, 1975. Image credit: wikipedia
  5. Range search

  6. Range search

  7. Range search

  8. Range search

  9. Range search

  10. Range search

  11. Range search

  12. Range search

  13. Optimized KD-Trees • J.H. Friedman. J.L. Bentley R.A. Finkel, “An

    algorithm for finding best matches in logarithmic expected time,” ACM Transactions on Mathematical Software. • Tree needs to be balanced, e.g. choose median as discriminator value at each partition. • With O(N) median, tree building takes O(k N log N) time. • Single query takes O(log N) time on average, total running time is O(N log N) when Ns ~ N.
  14. Scaling

  15. Wrong scaling due to unexpected behavior of quickselect median algorithm

    (solved) before after
  16. RPM DEB Source Simple installation from source or binary package

  17. Conclusion • Easy to install • Easy to use •

    Fast • Get it now at http://pschella.github.com/k3match