Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AARTFAAC Project Update

Ab44292d7d6f032baf342a98230a6654?s=47 transientskp
December 03, 2012

AARTFAAC Project Update

John Swinbank

Ab44292d7d6f032baf342a98230a6654?s=128

transientskp

December 03, 2012
Tweet

Transcript

  1. AARTFAAC Project Update John Swinbank swinbank@aartfaac.org LOFAR TKP Project Meeting

    3 December 2012, Amsterdam
  2. Amsterdam-ASTRON Radio Transients Facility and Analysis Centre Low Band Field

    of view ~10000 sq. deg. RMS 1 Jy (1 s) Resolution 40 ’ High Band Field of view ~300 sq. deg. RMS 0.05 Jy (1 s) Resolution 16 ’
  3. Correlator Outputs Number of baselines 41616 Dual polarization Total Bandwidth

    13.8 MHz 8 bit mode Subbands 575 24 kHz each Dump Time 0.1 s Data Rate ~30 Gb s-1 166176×575×32×10 • Low-latency (“real time”) processing • Significant computational challenge
  4. Distributed Correlator B CS002 B/6 data exchange with CS003 UniBoards

    Correlator Products RCU RSP Filter Bank Correlator URI Packet Organizer B/6 data exchange with CS004 B/6 data exchange with CS005 B/6 data exchange with CS006 B/6 data exchange with CS007 • FPGA based “Uniboard” developed at ASTRON • 2 Uniboards per station • Each station correlates ⅙ of total bandwidth
  5. Hardware URI-Board taps data off at station with no impact

    on LOFAR URI- and Uniboards being tested at ASTRON last month
  6. Software overview Antenna Antenna Antenna Antenna Antenna Monitoring and control

    Pelican Server Image cube Distributed Correlator Transients Pipeline Inspect Calibrate Postprocess Write to image cube Image Pelican Pipeline Inspect Calibrate Postprocess Write to image cube Image Inspect Calibrate Postprocess Write to image cube Image Pelican Pipeline Pelican Pipeline • New imaging & calibration algorithms being developed (Prasad, Wijnholds) • Pelican-based high- performance pipeline (Huizinga) • Transients search performed by interface to LOFAR Trap (Molenaar)
  7. Bad Data Mitigation Antenna Mean. Amplitude • Reject bad dipoles

    based on amplitudes of cross-correlations • Reject outlier visibilities based on rolling average • Simple strategies are adequate for now
  8. Calibration Requirements • Capability for standalone operation, without the need

    for separate calibration observations • Accuracy adequate to generate consistent images for image plane transient detection • Latency low enough to enable quasi-real time operation • Amenable to a distributed approach • Ability to deal with problematic (RFI affected, inaccurate sky model etc.) timeslices
  9. Calibration Algorithm Periodic multi-unit recalibration Initial calibration over several units

    Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Calibration Unit Time ... Tracking calibration per unit Flag RFI and bad dipoles Calibrate based on catalogue- derived model Update model based on WSF position fitting Recalibrate based on refined positions Subtract bright sources ... • Development of LOFAR Station Calibration • Iterative weighted alternating least-squares estimation of calibration parameters • Weighted subspace fitting based direction of arrival estimation • Fast per-sensor complex gain estimation (Salvini, OeRC)
  10. Imager • Snapshot imaging • Currently: naive FFT implemented •

    Also: dump calibrated visibilities to disk, investigation algorithms with CASA, awimager, etc • Development & prototyping of facetted imaging ongoing
  11. Data Output • I thought it would never happen, but...

    • Thanks, Adriaan, Anastasia, Ger, the LOFAR Data Formats WG, Antonia, ...! • I thought it would never happen, but... • Thanks, Adriaan, Anastasia, Ger, the LOFAR Data Formats WG, Antonia, ...!
  12. Transients Pipeline MonetDB Database Imaging Pipeline Image Cube Source Finding

    Lightcurve Storage Transient & Variability Analysis Source Association Archive Database Classification & Analysis Send External Alert Real-time Processing Off-line & External Systems • “Borrowed” from LOFAR: see tomorrow! • Performance, performance, performance...
  13. Experimental data • Data taken Autumn 2011, Summer 2012 •

    Correlation in software on clusters at ASTRON and in Amsterdam • A correlator world record?!
  14. First images...

  15. Recent Images 3C295 3C310 3C348 3C353 3C392 3C390 3C398 3C400

    3C274 3C409 3C433 3C461 3C405 3C410 3C380 Galactic Centre LBA OUTER 90 kHZ BW 2 Seconds 3x3 Facets
  16. Imaging pipeline status • In place: • C++/Pelican pipeline structure

    • Simple bad data rejection • Simple imager • Matlab calibration code • Integration of Matlab code with C++ • MeasurementSet & image output • In progress: • Matlab prototype of advanced imager • Porting Matlab calibration code to C++ • To come: • Port imager to C++ • Command and control • Integration with LOFAR systems + further upgrades, refinements, etc to all the above!
  17. Summary • Uniboard correlator making progress • Temporary alternative correlator

    may be considered for early deployment • Data-dumps & offline correlation enable rapid progress with the imaging system • Combination of Matlab for experimentation & C++/Pelican for deployment working well • Still a lot of work to make the Trap meet our performance requirements • Targeting regular operations during 2013