Towards a Dynamic LOFAR Catalogue

Transcript

1. Towards a Dynamic LOFAR Catalogue Bart Scheers Astronomical Institute Anton

   Pannekoek, University of Amsterdam Centre for Mathematics & Informatics (CWI) LOFAR TKP Meeting, API, 2011–06–29

2. 1 LOFAR TKP Meeting – 2011-06-29 LOFAR Catalogue ▸ List

   of all sources detected by LOFAR ▸ Multiple observations per source ▹ Adding time domain → dynamic ▹ Light-curves ▸ Keep track of 'meta-data' ▹ image properties ▹ observation characteristics ▸ Make available for data mining/discovery ▸ Scalable

3. Data Stream of Images 2 LOFAR TKP Meeting – 2011-06-29

4. What can we expect? 3 LOFAR TKP Meeting – 2011-06-29

   Scheers, PhD Thesis, 2011

5. What can we expect? ▸ Full operation: 50 TB/yr ▸

   Peaks: 10,000 sources per second ▸ Distinct sources: ~107 ▸ Source is about 300 Bytes in DB 3 LOFAR TKP Meeting – 2011-06-29

6. How fast will it grow? 4 LOFAR TKP Meeting –

   2011-06-29 Full-NL Zenith Monitor

7. Inside the Database ▸ Appending all detected sources ▸ Associate

   detected sources ▹ running catalogue ◦ i.e. LOFAR sources ▹ external catalogues ◦ VLSS, WENSS, NVSS, exoplanets, ... ▸ Run variability checks ▸ Move observations to light-curve database 5 LOFAR TKP Meeting – 2011-06-29

8. Inside the Database 5 LOFAR TKP Meeting – 2011-06-29

9. Source Association 6 LOFAR TKP Meeting – 2011-06-29

10. Source Association 7 LOFAR TKP Meeting – 2011-06-29 (1) distance

    on sky (2) dimensionless distance (3) likelyhood ratio

11. Association Parameter, r 8 LOFAR TKP Meeting – 2011-06-29

12. Source Variability 9 LOFAR TKP Meeting – 2011-06-29 (1) absolute

    flux change (2) weighted flux change Maintain 6 parameters

13. Association & Variability Probabilities ▸ Rayleigh distribution ▸ η ν

    behaves as chi square probability 10 LOFAR TKP Meeting – 2011-06-29

14. Simulated Data 11 LOFAR TKP Meeting – 2011-06-29 Non-variable sources

15. WSRT GRB030329 Observations 12 LOFAR TKP Meeting – 2011-06-29

16. WSRT GRB030329 Observations 12 LOFAR TKP Meeting – 2011-06-29

17. More Light-curve Analysis ▸ Detecting trends ▹ n sequential data

    points mσ above average ▸ Systematic structure of light curve ▹ Ratio of the mean square successive difference to the sample variance ▸ Array Query Language: SciQL 13 LOFAR TKP Meeting – 2011-06-29

18. Data mining Light-curve Database ▸ Extending SQL → SciQL 14

    LOFAR TKP Meeting – 2011-06-29

19. SciLens Experimentation Platform 15 LOFAR TKP Meeting – 2011-06-29 ▸

    Computational top tier (1 node) ▸ High-end tier (16 nodes) ▸ Cloud-oriented tier (64 nodes) ▸ Energy-conservative tier (256 nodes)

20. Distributed Light-curve Database 16 LOFAR TKP Meeting – 2011-06-29