AARTFAAC Project Update

John Swinbank

Transcript

1. AARTFAAC Project Update John Swinbank swinbank@aartfaac.org LOFAR TKP Project Meeting 3 December 2012, Amsterdam
2. Amsterdam-ASTRON Radio Transients Facility and Analysis Centre Low Band Field of view ~10000 sq. deg. RMS 1 Jy (1 s) Resolution 40 ’ High Band Field of view ~300 sq. deg. RMS 0.05 Jy (1 s) Resolution 16 ’
3. Correlator Outputs Number of baselines 41616 Dual polarization Total Bandwidth 13.8 MHz 8 bit mode Subbands 575 24 kHz each Dump Time 0.1 s Data Rate ~30 Gb s-1 166176×575×32×10 • Low-latency (“real time”) processing • Significant computational challenge
4. Distributed Correlator B CS002 B/6 data exchange with CS003 UniBoards Correlator Products RCU RSP Filter Bank Correlator URI Packet Organizer B/6 data exchange with CS004 B/6 data exchange with CS005 B/6 data exchange with CS006 B/6 data exchange with CS007 • FPGA based “Uniboard” developed at ASTRON • 2 Uniboards per station • Each station correlates ⅙ of total bandwidth
5. Hardware URI-Board taps data off at station with no impact on LOFAR URI- and Uniboards being tested at ASTRON last month
6. Software overview Antenna Antenna Antenna Antenna Antenna Monitoring and control Pelican Server Image cube Distributed Correlator Transients Pipeline Inspect Calibrate Postprocess Write to image cube Image Pelican Pipeline Inspect Calibrate Postprocess Write to image cube Image Inspect Calibrate Postprocess Write to image cube Image Pelican Pipeline Pelican Pipeline • New imaging & calibration algorithms being developed (Prasad, Wijnholds) • Pelican-based high- performance pipeline (Huizinga) • Transients search performed by interface to LOFAR Trap (Molenaar)
7. Bad Data Mitigation Antenna Mean. Amplitude • Reject bad dipoles based on amplitudes of cross-correlations • Reject outlier visibilities based on rolling average • Simple strategies are adequate for now
8. Calibration Requirements • Capability for standalone operation, without the need for separate calibration observations • Accuracy adequate to generate consistent images for image plane transient detection • Latency low enough to enable quasi-real time operation • Amenable to a distributed approach • Ability to deal with problematic (RFI affected, inaccurate sky model etc.) timeslices
9. Calibration Algorithm Periodic multi-unit recalibration Initial calibration over several units Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Time ... Tracking calibration per unit Flag RFI and bad dipoles Calibrate based on catalogue- derived model Update model based on WSF position fitting Recalibrate based on refined positions Subtract bright sources ... • Development of LOFAR Station Calibration • Iterative weighted alternating least-squares estimation of calibration parameters • Weighted subspace fitting based direction of arrival estimation • Fast per-sensor complex gain estimation (Salvini, OeRC)
10. Imager • Snapshot imaging • Currently: naive FFT implemented • Also: dump calibrated visibilities to disk, investigation algorithms with CASA, awimager, etc • Development & prototyping of facetted imaging ongoing
11. Data Output • I thought it would never happen, but... • Thanks, Adriaan, Anastasia, Ger, the LOFAR Data Formats WG, Antonia, ...! • I thought it would never happen, but... • Thanks, Adriaan, Anastasia, Ger, the LOFAR Data Formats WG, Antonia, ...!
12. Transients Pipeline MonetDB Database Imaging Pipeline Image Cube Source Finding Lightcurve Storage Transient & Variability Analysis Source Association Archive Database Classification & Analysis Send External Alert Real-time Processing Off-line & External Systems • “Borrowed” from LOFAR: see tomorrow! • Performance, performance, performance...
13. Experimental data • Data taken Autumn 2011, Summer 2012 • Correlation in software on clusters at ASTRON and in Amsterdam • A correlator world record?!
14. First images...
15. Recent Images 3C295 3C310 3C348 3C353 3C392 3C390 3C398 3C400 3C274 3C409 3C433 3C461 3C405 3C410 3C380 Galactic Centre LBA OUTER 90 kHZ BW 2 Seconds 3x3 Facets
16. Imaging pipeline status • In place: • C++/Pelican pipeline structure • Simple bad data rejection • Simple imager • Matlab calibration code • Integration of Matlab code with C++ • MeasurementSet & image output • In progress: • Matlab prototype of advanced imager • Porting Matlab calibration code to C++ • To come: • Port imager to C++ • Command and control • Integration with LOFAR systems + further upgrades, refinements, etc to all the above!
17. Summary • Uniboard correlator making progress • Temporary alternative correlator may be considered for early deployment • Data-dumps & offline correlation enable rapid progress with the imaging system • Combination of Matlab for experimentation & C++/Pelican for deployment working well • Still a lot of work to make the Trap meet our performance requirements • Targeting regular operations during 2013