Algorithm development Pipeline Implementation ASM Deployment AARTFAAC: A Status Update Peeyush Prasad∗,†, Stefan Wijnholds∗, Folkert Huizinga† ∗ASTRON, †UvA January 08, 2014 LOFAR TKP Meeting, Amsterdam TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Outline Overview Algorithm development Pipeline Implementation Deployment Conclusions TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment AARTFAAC ASM 24/7 Radio Sky Monitor, uses the 6 Superterp LOFAR stations Low resolution, low sensitivity, All-sky FoV, all the time (Piggyback) Low latency, near real-time response TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment AARTFAAC ASM Calibration and Imaging blocks 24/7 Radio Sky Monitor, uses the 6 Superterp LOFAR stations Low resolution, low sensitivity, All-sky FoV, all the time (Piggyback) Low latency, near real-time response TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment AARTFAAC ASM System level blocks TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Summary of calibration scheme Point src model sky based multisource direction dependent SelfCal. Iterative weighted LS estimators for model src flux, and position, antenna complex gains, system noise. WSF based model source position estimation. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Spatial filtering and flux estimation Spatial filtering of diffuse and extended emission prior to modeling. Leads to overestimation of model source flux with weighted LS estimator. Reformulation of Weighting matrix and optimization problem: unbiased estimate. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Spatial filtering and flux estimation Spatial filtering of diffuse and extended emission prior to modeling. Leads to overestimation of model source flux with weighted LS estimator. Reformulation of Weighting matrix and optimization problem: unbiased estimate. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Handling Ionospheric Scintillation Scintillation can cause model sources to change flux ratios, couples to gain TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Handling Ionospheric Scintillation TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Handling Ionospheric Position errors TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Tracking calibration: Bounded latency TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Tracking calibration: Bounded latency TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment SelfCal residual phase errors TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Extracted Light curve stability TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Difference Imaging beats confusion limit IN diff = N i=0 Ii+N − N i=0 Ii TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Software Architecture Pelican framework for real-time data streaming, with flow management. CPU based implementation with low coupling, dedicated core per calibration time/freq. unit. Service data lane for tracking solution dispersal. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Performance Eigen3 LinAlg. library for optimized matrix operations. Typical latency of ˜250ms for full convergence, ˜2sec on ’difficult’ data. Further optimization of the WSF component may be warranted. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Control and Monitoring Interface with LOFAR via observation parset parsing. Universal controller for various subcomponents. Cherimoya: python based production mode pipeline monitoring on the network. Database of monitor products for history generation. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Deployment: Station Hardware URI +Uniboards installed at all 6 stations, stress tested with LOFAR hardware (˜Nov 2013). 8bit raw sampling, 7 MHz bandwidth@16bit complex subbands. Initial commissioning data over 1Gbps FPGA control links: 1 subband available. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Deployment: GPU correlator 2xXeon-E5 2690 CPU2 corner turner, 2xAMD FirePro S10000 FX engine. Hardware installation in progress. Expected bandwidth: ∼2.5MHz@16kHz/1sec resolution per GPU (currently limited by CPU packet reorder). TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Deployment: Imaging server 2xOpteron 6376 dual proc servers, 128 cores. 16kHz/1sec target resolution, subband collapsed images. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment Conclusions Basic RFI mitigation via low order statistics seems adequate. Pipeline routinely achieves images with a dynamic range of ˜2000:1, limited by ionosphere and confusion. Difference imaging beats the confusion limit to achieve close to thermal noise limited imaging. Meeting fundamental latency requirements: Tracking calibration necessary for challenging data. Single subband pipeline deployment by mid-Feb. Future Work Interfacing with TraP: Image generation with proper WCS. Offline tuning of TraP on AARTFAAC lightcurves. Continuous sky monitoring for generation of reference sky. TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC, 

institution-logo Algorithm development Pipeline Implementation ASM Deployment ‘ TKP Meeting, Amsterdam Prasad, Wijnholds, Huizinga UvA/AARTFAAC,