(NWO) Netherlands Institute for Radio Astronomy LOFAR LOFAR Transient KSP Plenary Meeting December 3, 2012 Current Array Status and Upgrades Developments Plans for 2013 Start of Cycle 0 Operations MSSS Update
LOFAR LOFAR 3 Current Array Status §44 operational stations - 24 core + 12 remote in NL - Remaining 4 NL stations by early-2013 - 8 international stations online - New stations in Poland (3), Germany (1) §IBM BlueGene/P supercomputer - Operating nominally (uptime of 99%) - Bandwidth to CEP2 upgraded to 80 Gbit/s - Upgrade studies underway §Post-processing cluster - CEP2 stable, CEP1 software upgraded - All development work moved to CEP1 - CEP3 upgrade plan under discussion §Additional hardware upgrades - Synoptics boards installed in all NL stations - TBB (4GB) upgrades under test
LOFAR LOFAR 4 Rollout of Final NL Stations 3.2 Asynchronous RSP Boards As reported in a previous progress report, a problem has been identified with the start-up of the RSP boards on a given station at the beginning of an observation. This problem can cause significantly (~30%) reduced signal-to-noise ratios compared to baselines with ‘good’ stations. The problem was already traced in part to behaviour where one or more RSP boards on a station can start 5 nsec earlier or later than the remaining boards. This difference gives 180 degrees phase error at 100MHz (200MHz clock). The net effect of the problem is an error in the pointing and a net reduction in the sensitivity of the station, ranging from 30% to 100%. Further investigation traced the problem down to the Rb- clocks in the stations: the clock signal inside the Rb clock is distributed to several signal outlets, but a temperature dependent phase difference between these outlets to which the RSP boards are connected causes unpredictable phase relations. To remedy this effect, clock distribution was moved to a SyncOptic board (already under development for the core single clock project, see section 3.5). This board was successfully tested on CS401 in May. The next step was to produce a series of these SyncOptic boards for both core stations and remote stations. The boards were delivered in the last week of August, and underwent integration and final testing at ASTRON in early September. All Dutch stations, core and remote, were outfitted with SyncOptic boards in September and early October. SyncOptic boards were installed as an experiment in the Effelsberg station, DE601, in the week of November 12 to 16. This station was selected because a) the clock system at DE601 showed erratic behaviour, and b) the station is sufficiently close to ASTRON to permit traveling to the station in case of complications. 3.3 Station Rollout Schedule At the time of writing, construction of three Dutch stations, RS409, RS305, and RS407, is completed, with the stations being on-line (RS305) or being brought on-line (RS407 and RS409). Of the remaining four NL stations, two, RS310 and RS210 are expected to be completed before the end of 2012. The location of the Buinen station (RS404) has become uncertain due to archaeological finds nearby. The remaining station, RS410, is expected to come online in 2013 due to delays associated with obtaining the necessary building permits. Table 1 gives the current expected rollout schedule for the remaining NL stations. Table 1: Estimated rollout schedule for remaining LOFAR stations. Station ID Location Status Completion RS404 Buinen Location uncertain 2013 RS310 Onna Construction started Q4 2012 RS210 Weerselo Construction started Q4 2012 RS410 Wolvega Location uncertain 2013 (courtesy M. Gerbers & N. Ebbendorf) § Four remaining NL remote stations under construction § RS310 and RS210 are expected to be completed before the end of 2012 § RS404 location uncertain due to archaeological ﬁnds nearby § RS410 delayed due to building permit issues LOFAR station RS310 (Zuidveen/Onna)
LOFAR LOFAR 5 Installation of Synoptics Boards Mid-September 2012 Mid-November 2012 Synchronization issue between station RSPs identiﬁed early 2012 Causes out-of-focus, mispointed beams and loss of sensitivity Prototype synoptics board tested in May, rolled out to all NL stations by October Second batch of boards ordered for remaining international LOFAR stations (courtesy W. Frieswijk)
LOFAR LOFAR 6 Single Core Clock Distribution (courtesy J. Hessels) Central clock at Concentrator Node Distributed using existing ﬁbers All core stations now use single clock Improved S/N for coherent TABs
LOFAR LOFAR 7 Uniformity of Station Beams (courtesy M. Brentjens & W. Frieswijk) §HBA NL core and remote stations have differing FoVs §HBA Inner mode selects only innermost 24 tiles for remote stations §Yields HBA beam sizes that are now the same for core and remote stations FWHM ~3.8 deg (150 MHz)
LOFAR LOFAR 8 LOFAR Version 1.0 Deployed § Standard Imaging Observations § LBA and HBA MSSS-like imaging observations § Imaging observations with up to 8 independent beams § Automated execution of Standard Imaging Pipeline (SIP) § Speciﬁcation, planning, and management of observations § Beam-formed Observations § Tied array mode observing with up to 127 pencil beams § Four tied-array modes: complex voltages, coherent stokes, incoherent stokes, ﬂy's eye § BG/P processing and kickoff of post-processing pipeline § Speciﬁcation, planning, and management of observations § Initial Long Term Archive (LTA) § Ingest of imaging data products (visibilities and images) § Ingest of raw beam-formed data products § Search and retrieve functionality for LTA data products (courtesy J. Broderick) Available for Cycle 0 Operations, but 6 months late!
LOFAR LOFAR 9 Post-1.0 Developments § 8-bit Mode Observing § Updates to station ﬁrmware (8-bit and 16-bit) § Updates to MAC/SAS station control software § Updates to Navigator, MoM, Scheduler for operations § Testing of LCU capacity, network, CEP write speed § Propagation of broken HBA tile metadata § Testing of downstream pipeline processing § Availability of sufﬁcient LTA ingest capacity § Standard Imaging Pipeline § Updates to beam model § Direction dependent corrections § Initial source ﬁnding and GSM integration § Long Baseline Imaging § Superterp baseline addition in NDPPP § Fringe ﬁtting preprocessing (developed by LB working group) (courtesy S. Yatawatta) 30o x 30o 96 MHz bandwidth 2 arcmin FWHM PSF 100 μJy noise NCP ﬁeld 8-bit mode test data
LOFAR LOFAR 11 Central Processing Upgrade GbE GbE Post-processing CPU Cluster Storage Section BlueGene/P Supercomputer I/O Nodes I/O Nodes GbE GbE GbE GbE GbE LTA Staging Area LOFAR Long-term Archive (LTA) Station Station Station Station WAN Real-time Processing Ofﬂine Processing § Support contract on BG/P will expire end of 2013 § CEP compute cluster is insufﬁcient for full operations § Report on hardware options currently being prepared § Large GPU+CPU cluster is one favored scenario § Requires signiﬁcant porting effort throughout 2013
LOFAR LOFAR 12 GbE GbE Storage Section GbE GbE GbE GbE GbE LTA Staging Area LOFAR Long-term Archive (LTA) Station Station Station Station WAN Real-time and OfﬂineProcessing CEP3 GPU+CPU Cluster (Integrated correlator and post-processing) Central Processing Upgrade § Support contract on BG/P will expire end of 2013 § CEP compute cluster is insufﬁcient for full operations § Report on hardware options currently being prepared § Large GPU+CPU cluster is one favored scenario § Requires signiﬁcant porting effort throughout 2013
LOFAR LOFAR 13 Development Overview §Current development priorities - Freeze of LOFAR version 1.0 operational system - Clean separation between development and operational system - Testing and bug ﬁxing for Cycle 0 operations - Post-1.0 additions (8-bit mode, LB additions, etc.) §Development priorities for 2013 - Replacement of the BG/P and CEP2 upgrade - A fully functional Long Term Archive (LTA) - System redesign to incorporate Apertif control and operations - System redesign to support a fully responsive LOFAR telescope - System redesign to facilitate a maintainable and expandable system New system functionality will be very limited in 2013
LOFAR LOFAR 14 LTA Data Accumulation 0" 500" 1000" 1500" 2000" 2500" 3000" 2011/01" 2011/04" 2011/07" 2011/10" 2012/01" 2012/04" 2012/07" 2012/10" Volume'(Terabyte)' Date' Data'Stored'in'the'Long'Term'Archive' NIKHEF" TARGET" FZJ" SARA" 2.65 PBytes as of Nov 2012 Comparable to expected NRAO aggregate archive by start of 2013 Current data volume on Tier 1 sites Includes visibilities, images, and BF data Estimated growth of LTA data volume Does not include raw visibility data Does not include derived products
LOFAR LOFAR 16 Multi-band Source Detection § Detections in most/all bands used for cross-id’s § Spectral indices determined for all sources § Spectral index distributions seem reasonable based on limited number of ﬁelds § First steps to ﬁlling the broadband catalog (Pietka et al. 2012) Spectral Index <α > = -0.66 L070+69
LOFAR LOFAR 20 Cycle 0 Proposals and Operations §Cycle 0 Proposal Call § 43 Cycle 0 proposals were submitted § 247 authors from 17 countries § 29 associated with Reserved Access proposals § Over-subscription ratios of 1.4 - 3.0 § LOFAR PC met on Nov 28-29, 2012 § Cycle 1 deadline tentatively March 15, 2013 §Regular Cycle 0 Operations § Began on December 1, 2012 (ﬁrst EoR ﬁelds) § Modest operational efﬁciency through January § EoR and MSSS dominate early schedule § Long-term schedule available mid-December Commissioning time will be very constrained during Cycle 0!
LOFAR LOFAR 21 § Hardware status of the LOFAR array is excellent § Initial system software system is operational § Software development will be constrained in 2013 § MSSS and commissioning making excellent progress § Regular Cycle 0 operations have begun Summary New system functionality will be very limited in 2013