Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AARTFAAC Project Update

transientskp
December 03, 2012

AARTFAAC Project Update

John Swinbank

transientskp

December 03, 2012
Tweet

More Decks by transientskp

Other Decks in Science

Transcript

  1. AARTFAAC
    Project Update
    John Swinbank
    [email protected]
    LOFAR TKP Project Meeting
    3 December 2012, Amsterdam

    View Slide

  2. Amsterdam-ASTRON Radio Transients
    Facility and Analysis Centre
    Low Band Field of view ~10000 sq. deg.
    RMS 1 Jy (1 s)
    Resolution 40 ’
    High Band Field of view ~300 sq. deg.
    RMS 0.05 Jy (1 s)
    Resolution 16 ’

    View Slide

  3. Correlator Outputs
    Number of baselines 41616 Dual polarization
    Total Bandwidth 13.8 MHz 8 bit mode
    Subbands 575 24 kHz each
    Dump Time 0.1 s
    Data Rate ~30 Gb s-1 166176×575×32×10
    • Low-latency (“real time”) processing
    • Significant computational challenge

    View Slide

  4. Distributed Correlator
    B
    CS002
    B/6 data exchange with CS003
    UniBoards
    Correlator Products
    RCU RSP
    Filter Bank
    Correlator
    URI
    Packet
    Organizer
    B/6 data exchange with CS004
    B/6 data exchange with CS005
    B/6 data exchange with CS006
    B/6 data exchange with CS007
    • FPGA based “Uniboard”
    developed at ASTRON
    • 2 Uniboards per station
    • Each station correlates ⅙ of
    total bandwidth

    View Slide

  5. Hardware
    URI-Board taps data off at station
    with no impact on LOFAR
    URI- and
    Uniboards
    being
    tested at
    ASTRON
    last month

    View Slide

  6. Software overview
    Antenna
    Antenna
    Antenna
    Antenna
    Antenna
    Monitoring and control
    Pelican Server
    Image cube
    Distributed Correlator
    Transients Pipeline
    Inspect
    Calibrate
    Postprocess
    Write to
    image cube
    Image
    Pelican Pipeline
    Inspect
    Calibrate
    Postprocess
    Write to
    image cube
    Image
    Inspect
    Calibrate
    Postprocess
    Write to
    image cube
    Image
    Pelican Pipeline
    Pelican Pipeline
    • New imaging & calibration
    algorithms being developed
    (Prasad, Wijnholds)
    • Pelican-based high-
    performance pipeline
    (Huizinga)
    • Transients search performed
    by interface to LOFAR Trap
    (Molenaar)

    View Slide

  7. Bad Data Mitigation
    Antenna
    Mean. Amplitude
    • Reject bad dipoles
    based on amplitudes
    of cross-correlations
    • Reject outlier visibilities
    based on rolling
    average
    • Simple strategies are
    adequate for now

    View Slide

  8. Calibration Requirements
    • Capability for standalone operation, without
    the need for separate calibration observations
    • Accuracy adequate to generate consistent
    images for image plane transient detection
    • Latency low enough to enable quasi-real time
    operation
    • Amenable to a distributed approach
    • Ability to deal with problematic (RFI affected,
    inaccurate sky model etc.) timeslices

    View Slide

  9. Calibration Algorithm
    Periodic multi-unit recalibration
    Initial calibration over several units
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Calibration Unit
    Time
    ...
    Tracking calibration per unit
    Flag RFI and
    bad dipoles
    Calibrate based
    on catalogue-
    derived model
    Update model
    based on WSF
    position fitting
    Recalibrate
    based on refined
    positions
    Subtract bright
    sources
    ...
    • Development of LOFAR Station Calibration
    • Iterative weighted alternating least-squares estimation of
    calibration parameters
    • Weighted subspace fitting based direction of arrival
    estimation
    • Fast per-sensor complex gain estimation (Salvini, OeRC)

    View Slide

  10. Imager
    • Snapshot imaging
    • Currently: naive FFT
    implemented
    • Also: dump calibrated
    visibilities to disk,
    investigation algorithms
    with CASA, awimager, etc
    • Development &
    prototyping of facetted
    imaging ongoing

    View Slide

  11. Data Output
    • I thought it would
    never happen, but...
    • Thanks, Adriaan,
    Anastasia, Ger, the
    LOFAR Data Formats
    WG, Antonia, ...!
    • I thought it would
    never happen, but...
    • Thanks, Adriaan,
    Anastasia, Ger, the
    LOFAR Data Formats
    WG, Antonia, ...!

    View Slide

  12. Transients Pipeline
    MonetDB Database
    Imaging
    Pipeline
    Image
    Cube
    Source
    Finding
    Lightcurve
    Storage
    Transient &
    Variability
    Analysis
    Source
    Association
    Archive
    Database
    Classification
    & Analysis
    Send
    External
    Alert
    Real-time Processing
    Off-line & External Systems
    • “Borrowed” from LOFAR: see tomorrow!
    • Performance, performance, performance...

    View Slide

  13. Experimental data
    • Data taken Autumn 2011, Summer 2012
    • Correlation in software on clusters at ASTRON
    and in Amsterdam
    • A correlator world record?!

    View Slide

  14. First images...

    View Slide

  15. Recent Images
    3C295
    3C310
    3C348
    3C353
    3C392
    3C390
    3C398
    3C400
    3C274
    3C409
    3C433
    3C461
    3C405
    3C410
    3C380
    Galactic Centre
    LBA OUTER
    90 kHZ BW
    2 Seconds
    3x3 Facets

    View Slide

  16. Imaging pipeline status
    • In place:
    • C++/Pelican pipeline structure
    • Simple bad data rejection
    • Simple imager
    • Matlab calibration code
    • Integration of Matlab code with C++
    • MeasurementSet & image output
    • In progress:
    • Matlab prototype of advanced imager
    • Porting Matlab calibration code to C++
    • To come:
    • Port imager to C++
    • Command and control
    • Integration with LOFAR systems
    +
    further upgrades,
    refinements, etc
    to
    all the above!

    View Slide

  17. Summary
    • Uniboard correlator making progress
    • Temporary alternative correlator may be
    considered for early deployment
    • Data-dumps & offline correlation enable
    rapid progress with the imaging system
    • Combination of Matlab for experimentation
    & C++/Pelican for deployment working well
    • Still a lot of work to make the Trap meet our
    performance requirements
    • Targeting regular operations during 2013

    View Slide