Building LOFAR in 2009

Michael Wise
LOFAR and the Transient Radio Sky, Amsterdam, December 2008

Transcript

1. LOFAR and the Transient Radio Sky 15 December 2008 Michael Wise LOFAR / ASTRON RO / UvA LOFAR Status Update
2. LOFAR Status Update • Overview and Rollout • Pipeline Development • Testing and Commissioning • Archives and Timelines Outline
3. ASTRON LOFAR Transients Workshop, 15 Dec 2008 3 LOFAR Radio Telescope  Telescope the size of the Netherlands-Europe  Frequencies: 30 – 240 MHz  Interferometer baselines ~100 km  European expansion to 1000 km  Square Kilometer Array (SKA) pathfinder Phased aperture array design No dishes ⇒
4. ASTRON LOFAR Transients Workshop, 15 Dec 2008 4 Low band antenna: 30 – 80 MHz 48/96 antennas per station • 36 NL + 8 EU stations of dipoles • Replace big dishes by many cheap dipoles • No moving parts: electronic beam steering • Flexible digital beam forming LOFAR Antennas High band tiles:120 – 240 MHz 96 tiles/station, 4x4 antennas/tile
5. ASTRON LOFAR Transients Workshop, 15 Dec 2008 5 Antenna- Beam Digital Beam-Forming Dipole
6. ASTRON LOFAR Transients Workshop, 15 Dec 2008 5 Antenna- Beam Digital Beam-Forming Station-Beam Station Dipole (Tied) Array-Beam Array
7. ASTRON LOFAR Transients Workshop, 15 Dec 2008 6 • Station level processing Amplification, digitization, filtering, beam-forming, transient ram buffers (TBB) • Central processing Delay compensation, correlation, calibration, data reduction pipelines (BG/P, storage, offline cluster) Data Flow ⇒ 36 stations @ 2 Gbps/station
8. ASTRON LOFAR Transients Workshop, 15 Dec 2008 7 Central Processing • BG/P Data reception, transpose, correlation, beam-forming, de-dispersion • Storage system Short term storage of data, ~1 PByte, >100Gbps I/O • Offline cluster Calibration, data products, off-line analysis, ~10 TFLOPS Blue Gene/P Blue Gene/P Blue Gene/P
9. ASTRON LOFAR Transients Workshop, 15 Dec 2008 8
10. ASTRON LOFAR Transients Workshop, 15 Dec 2008 9 Blue Gene/P Installed
11. ASTRON LOFAR Transients Workshop, 15 Dec 2008 10 Remote Operation
12. ASTRON LOFAR Transients Workshop, 15 Dec 2008 11 International LOFAR • Netherlands • Germany • UK • France • Sweden • Italy • Poland Members
13. ASTRON LOFAR Transients Workshop, 15 Dec 2008 11 International LOFAR • Netherlands • Germany • UK • France • Sweden • Italy • Poland Members Effelsberg (IS-DE01)
14. ASTRON LOFAR Transients Workshop, 15 Dec 2008 12 © ASTRON CS010 CS001 CS008 CS016 Rollout Timeline Dec 08 CEP hardware tender closed Jan 08 First NL station operational Feb 08 Partial CEP storage and offline cluster Apr 09 5 Remote stations and superstation Apr 09 Standard imaging pipeline ready Jun 09 20 NL + 3-4 EU stations completed Aug 09 Global Sky Model survey begins Dec 09 36 NL + 8 EU stations online
15. Michiel
van
Haarlem 50+ supernova remnants, Many z>2 radio galaxies, A2255, A2256, M82 ... Recent LOFAR imaging results 45 Mhz, 24 hrs (courtesy S. Yatawatta)
16. ASTRON LOFAR Transients Workshop, 15 Dec 2008 Science Pipelines 14 Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Standard Imaging Known Pulsars VHECR Transient Detection EoR UHEP Pulsar Surveys HECR Thunder-Storm Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Variable Polyphase Filter Archiving Create and Store Single Pulse Data Single Pulse Analysis RFI Rejection Apply Coherent De-dispersion BlueGene/L Fold with Pulse Period Pulse Parameters Create and Store Frequency/ Phase Cube Calculate Dynamic Spectra Integrated Profiles MAC Ephemeris Data Most Recent Calibration Tied Array Beam Forming Single Pulse Gating Flagging Calibration & RFI Rejection Imaging Source finding Event Parameter Reconstruction Archiving TBB Dataset Creation + Storage Input section Aux. processing section BlueGene/L Storage system Off-line processing Event Parameter List Creation TBB Data MAC Coincidence Check Initiate Dump of TBB Data Send Trigger Messages & Parameters Receive Trigger Messages & Parameters Trigger Messages & Parameters Off-line Storage On-line Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage RFI Detection and Flagging De-dispersion Time and Frequency Averaging Correction Imaging Source Detection Source Classification Archiving BlueGene/L Image Cube Creation and Storage Determine Source Properties Update Transient Source Database MAC Initiate TBB Dump Reconfigure Array Generate External Alert Transient Database Most Recent Calibration Receive External Alert Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Delay Compensation Polyphase Filter Subsample Delay Compensation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Signal Extraction Transpose Correlation Input section Aux. processing section BlueGene/L Storage system Off-line processing Tied Array Beam PPF Inversion Peak Detection Ionos. Correction Peak Detection Tied Array Beam PPF Inversion Peak Detection Ionos. Correction TBB Dataset Creation & Storage TBB Data Flagging Calibration & RFI Rejection Imaging Source Finding Event Parameter Reconstruction Transport & Archiving Transpose & Delay Compensation Transpose & Delay Compensation MAC Receive Alert Message Initiate Dump of TBB Data Factors for Ionospheric Correction Off-line Storage On-line Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check RM Synthesis Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Solar Imaging ⇒ Derived from combined KSP science Flagging Calibration & RFI Rejection Imaging Source finding Event Parameter Reconstruction Archiving TBB Dataset Creation + Storage Input section Aux. processing section BlueGene/L Storage system Off-line processing Event Parameter List Creation TBB Data MAC Coincidence Check Initiate Dump of TBB Data Send Trigger Messages & Parameters Receive Trigger Messages & Parameters Trigger Messages & Parameters Off-line Storage On-line Solar Spectral Monitor
17. ASTRON LOFAR Transients Workshop, 15 Dec 2008 Science Pipelines 14 Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Standard Imaging Known Pulsars VHECR Transient Detection EoR UHEP Pulsar Surveys HECR Thunder-Storm Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Variable Polyphase Filter Archiving Create and Store Single Pulse Data Single Pulse Analysis RFI Rejection Apply Coherent De-dispersion BlueGene/L Fold with Pulse Period Pulse Parameters Create and Store Frequency/ Phase Cube Calculate Dynamic Spectra Integrated Profiles MAC Ephemeris Data Most Recent Calibration Tied Array Beam Forming Single Pulse Gating Flagging Calibration & RFI Rejection Imaging Source finding Event Parameter Reconstruction Archiving TBB Dataset Creation + Storage Input section Aux. processing section BlueGene/L Storage system Off-line processing Event Parameter List Creation TBB Data MAC Coincidence Check Initiate Dump of TBB Data Send Trigger Messages & Parameters Receive Trigger Messages & Parameters Trigger Messages & Parameters Off-line Storage On-line Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage RFI Detection and Flagging De-dispersion Time and Frequency Averaging Correction Imaging Source Detection Source Classification Archiving BlueGene/L Image Cube Creation and Storage Determine Source Properties Update Transient Source Database MAC Initiate TBB Dump Reconfigure Array Generate External Alert Transient Database Most Recent Calibration Receive External Alert Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Delay Compensation Polyphase Filter Subsample Delay Compensation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Signal Extraction Transpose Correlation Input section Aux. processing section BlueGene/L Storage system Off-line processing Tied Array Beam PPF Inversion Peak Detection Ionos. Correction Peak Detection Tied Array Beam PPF Inversion Peak Detection Ionos. Correction TBB Dataset Creation & Storage TBB Data Flagging Calibration & RFI Rejection Imaging Source Finding Event Parameter Reconstruction Transport & Archiving Transpose & Delay Compensation Transpose & Delay Compensation MAC Receive Alert Message Initiate Dump of TBB Data Factors for Ionospheric Correction Off-line Storage On-line Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check RM Synthesis Off-line Storage On-line Input section Aux. processing section Storage system Off-line processing Transpose and Delay Compensation Polyphase Filter Subsample Delay Compensation Correlation Measurement Set Creation and Storage Archiving BlueGene/L Image Quality Check Correction Calibration Imaging Source Extraction Create Mosaics Source Cross Check Solar Imaging ⇒ Derived from combined KSP science Available in 2009 Flagging Calibration & RFI Rejection Imaging Source finding Event Parameter Reconstruction Archiving TBB Dataset Creation + Storage Input section Aux. processing section BlueGene/L Storage system Off-line processing Event Parameter List Creation TBB Data MAC Coincidence Check Initiate Dump of TBB Data Send Trigger Messages & Parameters Receive Trigger Messages & Parameters Trigger Messages & Parameters Off-line Storage On-line Solar Spectral Monitor
18. ASTRON LOFAR Transients Workshop, 15 Dec 2008 15 Standard Imaging Pipeline
19. ASTRON LOFAR Transients Workshop, 15 Dec 2008 16 Imaging Development • BG/P correlator working • First versions of all major components exist • CEP cluster and storage tender under way Current status Upcoming work • BBS improvements – Ionosphere models – Beam models – More source types • Imager testing • Source detection • Global Sky Model • Major cycle pipeline Streaming bandpass correction (courtesy J. Romein) • 2 single dipoles • 58.6 MHz • 30 min integration
20. ASTRON LOFAR Transients Workshop, 15 Dec 2008 17 Imaging Development • BG/P correlator working • First versions of all major components exist • CEP cluster and storage tender under way Current status Upcoming work • BBS improvements – Ionosphere models – Beam models – More source types • Imager testing • Source detection • Global Sky Model • Major cycle pipeline New LOFAR distributed imager (from J. Romein)
21. ASTRON LOFAR Transients Workshop, 15 Dec 2008 18 Data Preprocessing (DP3) Unflagged Flagged Time Frequency • Automatic flagging Multiple algorithms • Data compression Time and frequency averaging • Simple cal. corrections Clock phases, A-team subtraction
22. ASTRON LOFAR Transients Workshop, 15 Dec 2008 BBS prediction using 3C sources 19 BBS Calibration Engine • Simulate model visibilities – Sources (point, Gaussian) – Bandpasses – Directional, complex gains – Dipole beam models • Fit model parameters to data • Subtract simulated visibilities • Apply corrections from solve • Distributed execution • SPAM ionosphere model • Station beam models • Residual based flagging Amplitude and phase solutions (16 HBA tiles, 4 hrs) (courtesy R.Nijboer, J. van Zwieten)
23. ASTRON LOFAR Transients Workshop, 15 Dec 2008 20 Source Detection • Detect sources in image cubes • Determine source parameters • Update LSM for next major cycle loop • Merge final results into GSM database O-Pipeline Detection O-Bower Steady Sources O-Bower Transient Bower VLA deep field (courtesy M.Bell) MS3 Options • Transient module (fast) • PyBDSM (more features) • DuChamp (cubes) Procedure
24. ASTRON LOFAR Transients Workshop, 15 Dec 2008 21 Global Sky Model • GSM stored as a database (PostgreSQL) • Many predictive functions implemented in database • Python interfaces provided for database access • Prototype already exists (VLSS, WENSS, NVSS, 3C, 8C, .....) • Simulated maps created using external tools (ORGSM, N. Mohan)
25. ASTRON LOFAR Transients Workshop, 15 Dec 2008 22 Figure 1: Figure 2: 4 Transient Pipeline with WSRT (courtesy M.Bell)
26. ASTRON LOFAR Transients Workshop, 15 Dec 2008 23 Known Pulsar Mode Current Status • Beam-formed data format defined • Manual tool to dump raw BF data • Connection to PRESTO pulsar analysis package made (Stappers, Jun 07) HBA 170-230 Mhz Next steps • Tied-array beam forming • Streaming version of BF data writer • Available in Q3 2009 LOFAR CS-1 detection of Pulsar B0329+54
27. ASTRON LOFAR Transients Workshop, 15 Dec 2008 24 Pulsar Data B0329+54 (courtesy J. Hessels)
28. ASTRON LOFAR Transients Workshop, 15 Dec 2008 25 Dynamic Spectra Jupiter bursts (courtesy J.-M. Grießmeier)
29. ASTRON LOFAR Transients Workshop, 15 Dec 2008 26 Dynamic Spectra Jupiter bursts (courtesy J.-M. Grießmeier) • Real-time monitoring (sec-msec) • Full frequency coverage (10-240 MHz) • Does not require full array • Can trigger alerts • Can be triggered by alerts ⇒ Available starting Q2 2009
30. ASTRON LOFAR Transients Workshop, 15 Dec 2008 27 Commissioning Tasks • Station acceptance procedures − Hardware installation (LOFAR, M. Norden) − CEP connection (LOFAR, A. Gunst) − Astronomical readiness (RO, M. Brentjens) • Software testing – Regression tests (simulations, standard datasets) – Configuration, build, release tests – Pipeline integration testing – Profiling and benchmarking • Astronomical − Testing of observing modes − Calibration of 20-200 MHz flux density scale − Determine astrometric accuracy − Determine performance figures (image quality, dynamic range) − .....and many more • Research topics (courtesy J. Hessels) HBA tracking error
31. ASTRON LOFAR Transients Workshop, 15 Dec 2008 27 Commissioning Tasks • Station acceptance procedures − Hardware installation (LOFAR, M. Norden) − CEP connection (LOFAR, A. Gunst) − Astronomical readiness (RO, M. Brentjens) • Software testing – Regression tests (simulations, standard datasets) – Configuration, build, release tests – Pipeline integration testing – Profiling and benchmarking • Astronomical − Testing of observing modes − Calibration of 20-200 MHz flux density scale − Determine astrometric accuracy − Determine performance figures (image quality, dynamic range) − .....and many more • Research topics (courtesy J. Hessels) Solved! HBA tracking error
32. ASTRON LOFAR Transients Workshop, 15 Dec 2008 28 Commissioning Observations • Testing of observing modes − Global Sky Model survey − Timing survey − Deep Survey KSP fields − Deep EoR KSP fields − Deep RM fields • KSP teams will prepare observing requests • Not scientific observing proposals • Describe specific commissioning goals • Describe expected results and deliverables • Storage and processing estimates • Personnel available (courtesy E. Carretti and G. Bernardi) RM cube from WSRT data
33. ASTRON LOFAR Transients Workshop, 15 Dec 2008 29 Data Processing School • Feb 10-12, 2009 • Hosted by ASTRON • Expect 30-40 students • System overview • Pipeline products • Hands-on sessions • Commissioners • Station owners ⇒ Registration open in next few days
34. ASTRON LOFAR Transients Workshop, 15 Dec 2008 30 LOFAR Storage Estimates Estimated growth rate ~ 2 Pb/yr
35. ASTRON LOFAR Transients Workshop, 15 Dec 2008 31 Requirements and Design • Long-term, public archive of LOFAR data products • Support storage and processing capabilities • Based on KSPs (storage, processing, usage patterns) • Support distributed resources (GRID) Current Activities and Status • LOFAR major application in the BigGRID project – Multi PB storage – Several hundreds of computing nodes • LOFAR/BigGRID working group active • Recommendation for hardware infrastructure in Dec 08 • Initial hardware expected in Q3 09 • Commissioning prototype in Q1 09 (100-200 TB) LOFAR Archive
36. ASTRON LOFAR Transients Workshop, 15 Dec 2008 32 Archive Prototype Collaboration with ASTRO-WISE
37. ASTRON LOFAR Transients Workshop, 15 Dec 2008 32 Archive Prototype (courtesy W.-J. Friend and J. McFarland) Collaboration with ASTRO-WISE
38. ASTRON LOFAR Transients Workshop, 15 Dec 2008 33 Archive Prototype (courtesy W.-J. Friend and J. McFarland) Collaboration with ASTRO-WISE
39. ASTRON LOFAR Transients Workshop, 15 Dec 2008 33 Archive Prototype (courtesy W.-J. Friend and J. McFarland) Collaboration with ASTRO-WISE
40. ASTRON LOFAR Transients Workshop, 15 Dec 2008 34 Dec 08 LTA hardware design complete Jan 09 First NL station operational Jan 09 LOFAR prototype archive in place Feb 09 Partial CEP storage and offline cluster Mar 09 VHECR pipeline ready Apr 09 5 Remote stations and superstation Apr 09 Standard imaging pipeline ready May 09 First LTA hardware arrives May 09 Known Pulsar pipeline ready Jun 09 20 NL + 3-4 EU stations completed Aug 09 LOFAR GSM survey begins Dec 09 36 NL + 8 EU stations online Timeline
41. ASTRON LOFAR Transients Workshop, 15 Dec 2008 35 LOFAR News Mailing List We will be using this list to communicate project status updates, meeting announcements, and other LOFAR-related news items to the community. To subscribe: Send mail to majordomo@astron.nl with the following command in the body of your email message: subscribe lofar-news To unsubscribe: Send mail to majordomo@astron.nl with the following command in the body of your email message: unsubscribe lofar-news