Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Square Kilometre Array: Overview and Engineering Update

The Square Kilometre Array: Overview and Engineering Update

The Square Kilometre Array is currently undergoing the Preliminary Design Reviews for its composing elements, and is thus at a critical point on its way to becoming ready for construction starting in 2018. In this talk we will provide an overview of the SKA, its composing elements, and their status, with emphasis on the Telescope Manager and the Science Data Processor, respectively the Monitoring & Control system, and Pipeline. We will see how do they compare with their ALMA equivalents, and how is the SKA similar/different from ALMA, and what are the main issues to be solved regarding the SKA computing infrastructure.

Talk given on Dec 10, 2014, at the Centre for Mathematical Modelling (CMM) of the University of Chile.

D6c83d5d20c63b8e421a7966b04cfedb?s=128

Juande Santander-Vela

December 10, 2014
Tweet

More Decks by Juande Santander-Vela

Other Decks in Technology

Transcript

  1. The Square Kilometre Array: Overview and Engineering Update CMM, Dec

    10 2014 Juande Santander-Vela
  2. • Who am I? • SKA Science • Telescopes, Sites,

    Technologies • SKA Organisation • Engineering Progress • SKA as an Exaflop Tomograph • Distributed SKA Control • What’s Next? Outline CMM U. Chile, December 10, 2014
  3. Who am I? CMM U. Chile, December 10, 2014

  4. CMM U. Chile, December 10, 2014 • Systems Engineer for

    the SDP/TM elements • Former VIA-SKA Project Manager • VLT Archive Applied Scientist, later ALMA Science Archive Front-/Back-end Developer at ESO • Ph.D. in Radio Astronomical data archives and Virtual Observatory • Software Analyst/Manager Who am I?
  5. Great observatories of the present CMM U. Chile, December 10,

    2014 ALMA SCO HQ, Dec 11 2014
  6. …and future CMM U. Chile, December 10, 2014 ALMA SCO

    HQ, Dec 11 2014
  7. …and future CMM U. Chile, December 10, 2014 ALMA SCO

    HQ, Dec 11 2014
  8. …and (near) future CMM U. Chile, December 10, 2014 ALMA

    SCO HQ, Dec 11 2014
  9. SKA Science CMM U. Chile, December 10, 2014

  10. Fundamental Forces & Particles • Gravity • Radio pulsar tests

    of General Relativity • Gravitational Waves • Dark Energy • Magnetism • Origin and evolution of Cosmic Magnetism
 Origins • Galaxies and the Universe • Cosmic Dawn • First Galaxies • Galaxy Assembly & Evolution • Stars, Planets & Life • Protoplanetary Disks • Biomolecules • SETI SKA Science Goals CMM U. Chile, December 10, 2014 XX Century: We discovered our place in the Universe XXI Century: We understand the Universe we inhabit
  11. • Initial SKA Science Book (2004): 5 KSPs, 43 additional

    ones • Updated SKA Science Book (2014): 13 Phase 1 Headline Projects, 130 Chapters • Projects reduced, parameterised from ~43 projects, through science prioritisation exercise with SKAO and SWGs SKA Science Goals CMM U. Chile, December 10, 2014
  12. • Astrobiology (“The Cradle of Life”) • Project Scientist: Tyler

    Bourke • WG Chair: Melvin Hoare • Galaxy Evolution – Continuum • Project Scientist: Jeff Wagg • WG Chairs: Nick Seymour & Isabella Prandoni • Cosmic Magnetism • Project Scientist: Jimi Green • WG Chairs: Melanie Johnston- Hollitt & Federica Govoni • Cosmology • Project Scientist: Jeff Wagg • WG Chair: Roy Maartens • Epoch of Reionisation & Cosmic Dawn • Project Scientist: Jeff Wagg • WG Chair: Leon Koopmans • Galaxy Evolution – HI • Project Scientist: Jimi Green • WG Chairs: Lister Staveley-Smith & Tom Osterloo • Pulsars (“Strong field tests of gravity”) • Project Scientist: Jimi Green • WG Chairs: Ben Stappers & Michael Kramer • Transients • Project Scientist: Tyler Bourke • WG Chairs: Rob Fender & J.-P. MacQuart SKA Science Goals: WGs CMM U. Chile, December 10, 2014
  13. SKA Science Goals CMM U. Chile, December 10, 2014 Year

    in the life of SKA1: Mock-up 3-years scheduling to constrain operations, computation
  14. Telescopes, Sites, Technologies CMM U. Chile, December 10, 2014

  15. SKA Telescopes CMM U. Chile, December 10, 2014 SKA1-Low

  16. SKA Telescopes CMM U. Chile, December 10, 2014 SKA1-Mid

  17. SKA Telescopes CMM U. Chile, December 10, 2014 SKA1-Survey

  18. CMM U. Chile, December 10, 2014

  19. CMM U. Chile, December 10, 2014

  20. SKA Organisation CMM U. Chile, December 10, 2014

  21. • Australia (DoI) • Canada (NRCZHIA) • China (MOST) •

    Germany (BMBF) • India (NCRA) • Italy (INAF) • Netherlands (NWO) • New Zealand (MED) • South Africa (DST) • Sweden (Chalmers) • UK (STFC) SKA Organisation CMM U. Chile, December 10, 2014 Currently UK Company Limited by Guarantee → Treaty Organisation Other partners in negotiation, expected
  22. SKA Governance: Pre-Construction CMM U. Chile, December 10, 2014 *

    * * * * * * * * * * * * * * * * * SKA Governance Members$ Board$of$Directors$ $ SKA$Office$ ~50$staff$ $ Director;General$ 11$Design$ConsorXa$ ~500$ScienXsts$&$Engineers$ Strategy$&$$ Business$Dev$ Cmte$ Finance$ Cmte$ ExecuXve$Cmte$ Science$&$Eng$ Advisory$Cmte$ €23.4M$ (cash)$ ~$€125M$ (in;kind)$
  23. SKA International Design Consortia CMM U. Chile, December 10, 2014

    Project Management and System Engineering Team based at JBO (UK) ~500 scientists & engineers in institutes & industry in 11 Member countries
  24. Engineering Progress CMM U. Chile, December 10, 2014

  25. 64 x 13.5m offset Gregorian antennas 8km maximum baseline length

    First receivers: 0.9 – 1.67 GHz (L-band) 0.58 – 1 GHz (UHF) 770 MHz bandwidth Early operations 2016/7 SKA1 Precursors: MeerKAT CMM U. Chile, December 10, 2014 First light with L-band receiver and digitiser erKAT
  26. SKA1 Precursors: MeerKAT CMM U. Chile, December 10, 2014 Lot’s

    more of infrastructure already in place: dish assembly shed; bunkered and RFI shielded data processor room and power system
  27. SKA1 Precursors: MeerKAT CMM U. Chile, December 10, 2014 Lot’s

    more of infrastructure already in place: dish assembly shed; bunkered and RFI shielded data processor room and power system
  28. SKA1 Precursors: ASKAP CMM U. Chile, December 10, 2014 36

    x 12m antennas 3-axis movement 30m – 6km baselines Novel PAF receiver 700MHz – 1800MHz Wide field of view Fast survey speed
  29. BETA • 6 Mk I PAFs and digital systems •

    Commissioning, learning about beamforming 
 Lesson: deploy early, test, ensure adequate gap to production commitment ASKAP • First Mk II PAF installed • Results expected in next few weeks • Production of Mk II PAFs install through to Feb 2016 SKA1 Precursors: ASKAP CMM U. Chile, December 10, 2014
  30. SKA1 Precursors: MWA CMM U. Chile, December 10, 2014 Murchison

    Widefield Array, Operational 17 refereed papers published, more coming
  31. 3 dish prototypes in testing CMM U. Chile, December 10,

    2014
  32. 3 dish prototypes in testing CMM U. Chile, December 10,

    2014
  33. 3 dish prototypes in testing CMM U. Chile, December 10,

    2014
  34. SKA as an Exaflop Tomograph CMM U. Chile, December 10,

    2014
  35. • Dishes, feeds, receivers (N=250 → 2500) • Low and

    mid aperture arrays (n=250k → 1000k) • Signal transport (~1 Pb/s → 10 Pb/s) • Signal processing (exa-MACs) • Software engineering & algorithm development • High performance computing (exa-flop capability) • Data storage (exa-byte capacity) • (Distributed) power requirements (10 → 50MW) Engineering Challenges CMM U. Chile, December 10, 2014
  36. SKA High-Level Architecture CMM U. Chile, December 10, 2014 SKA-TEL-SKO-DD-001

    Revision : 1 Figure 1 A schematic diagram of the SKA Observatory, showing the geographical locations of site entities (telescopes), the entities at regional centres (Host Country Headquarters and Science Data Processing), and Regional Science & Engineering Centre(s) Regional Science & Engineering Centre(s) SKA Observatory Global Headquarters Remote stations on spiral arms Science Computing Remote Station Remote Station Remote Station Host Country Headquarters Central Signal Processing SKA1-low SKA1-survey Core Arrays Australia Remote stations on spiral arms Science Computing Remote Station Remote Station Remote Station Host Country Headquarters Central Signal Processing Mid-Freq Aperture Array (SKA2) SKA1-mid Core Arrays South Africa
  37. CMM U. Chile, December 10, 2014 SDP F2F Meeting, April

    7th 2014, Cambridge SKA Elements & Interfaces INFRA DSH LFAA CSP SDP TM Command control flow Data flow SaDT Timing Conceptual figure: it should be split by telescope
  38. CMM U. Chile, December 10, 2014 SDP F2F Meeting, April

    7th 2014, Cambridge SKA Elements & Interfaces INFRA DSH/LFAA CSP SDP TM Command control flow Data flow SaDT Timing Conceptual figure: it should be split by telescope
  39. • Software Defined Networking • Allows easy partitioning, sub-arraying, quality

    of service, commensality… • Software Defined Beamforming (-Low, - Survey) • Software Defined Computing • Data-Flow Driven Science Data Processing SKA is a Software-Defined Telescope CMM U. Chile, December 10, 2014 Lots of computing challenges
  40. CMM U. Chile, December 10, 2014 Science Data Processor Local

    M&C Science Data Processor Telescope Manager Correlator / Beamformer Data Routing Ingest Visibility processing Multiple Reads Time Series Search Multiple Reads Data$Buffer Data Routing Time Series Processing Image Plane Processing Data Prodcuts Sky Models, Calibration Parameters ... Meta Data Master Controller Master Controller Local M&C Database Tiered Data Delivery
  41. CMM U. Chile, December 10, 2014 Performance Modelling •  Imaging

    and calibration determines system sizing •  Data is Buffered after Ingest •  Detailed analysis building on Cornwell: •  Imaging o  AW-projection o  Plus w-snapshots o  Plus faceting (working memory size) •  Continuum imaging always required for calibration o  Field of view to second zero of beam •  Maximal case includes “science discovery” mode of forming high spectral-resolution image cube over primary beam •  Commensal fast-imaging mode for slow transients Processing maximal (Pflop) Ingest (GB/s) Use Case examples (GB/s) LFAA! 100! 7300! 245! Survey! 135! 4600! 995! Mid! 360! 3300! 255! Detailed analysis is much more complicated
  42. CMM U. Chile, December 10, 2014

  43. CMM U. Chile, December 10, 2014

  44. SDP Parallelism: Frequency CMM U. Chile, December 10, 2014 x

    Time: accumulate visibilities into separate grids and combine later x Beams Simplified examples are illustrated in the following figures. Figure 5: Illustrating data parallelism based on frequency selection only (c.f. LOFAR)
  45. SDP Parallelism: uv-space partition CMM U. Chile, December 10, 2014

    Figure 6: Managing memory bandwidth by distributing regions of target uv-space over nodes followed by a gather and FFT to image space.
  46. SDP Parallelism: spatial partition CMM U. Chile, December 10, 2014

    Figure 6: Managing memory bandwidth by distributing regions of target uv-space over nodes followed by a gather and FFT to image space. Figure 7: Managing IO bandwidth from local storage by distributing source data over multiple nodes. Target gridded visibility plane is duplicated then an accumulation step is performed over the data island over which the source data were distributed 3.5 Data flow programming As discussed in Section 4 a data-flow approach is adopted for the top-level SDP architecture. In this section we provide some background discussion to the established concept of data-flow
  47. >1 EXAFLOP/S Correlation Data Product Generation Temporary Storage Long Term

    Storage High Availability Storage / DB On-Demand Processing CAN’T STORE IT! 1 DAY STREAM = 150 DAYS GLOBAL INTERNET TRAFFIC PROCESSING NEEDS 109 TOP RANGE PCS Antenna & Fronted-End Systems 30-300 PETAFLOP/S 30 PETAFLOP/S SKA Data Flow and Processing CMM U. Chile, December 10, 2014
  48. Distributed SKA Control CMM U. Chile, December 10, 2014

  49. • Need to find COTS-based solution • Move away from

    expensive hard real-time solutions • Switched network inside products • Orchestration instead of deterministic control • Certain amount of product independence SKA Control Problem CMM U. Chile, December 10, 2014
  50. CMM U. Chile, December 10, 2014 Figure 1. ALMA Software

    Subsystems interaction diagram (source: ALMA Software documentation)
  51. CMM U. Chile, December 10, 2014 TM.TELMGT TM.OBSMGT DSH.LMC CSP.LMC

    SDP.INGEST SDP.PIP.* SDP.ARCH SDP.LMC Commands Data Monitoring/Status CSP.CORR DSH.RECEIVER SKA Observation interaction
  52. CMM U. Chile, December 10, 2014 Ethernet switching Middleware TBD

    (EPICS/TANGO/OPC UA) DSH.LMC SADT.DDBH SADT.TM CSP.LMC SADT.SAT
  53. CMM U. Chile, December 10, 2014 TM.TELMGT TM.OBSMGT TM.TELMOD SDP.PIP.

  54. What’s Next? CMM U. Chile, December 10, 2014

  55. CMM U. Chile, December 10, 2014 2016 2015 2017 2018

    2019 2020 2021 2022 2023 2014 High/level/design Rebaselining/submissions/(RBS) Preliminary/Design/Review/(elements) Detailed/design Prototype/systems/deployed Critical/Design/Review SKA1/construction/approved Procurement SKA1/early/science SKA1/construction SKA2/detailed/design Design/consortia/start SKA2/procurement SKA2/construction/starts SKA2/concept/development Board/RBS/approval 3K5/Mar/15 15K22/Sep/14 1/Nov/13 PreKconstruction/Stage/1 PreKconstruction/Stage/2 Production*Readiness*Review*(4*dish*array) Construction*Readiness*Review* (1*dish*line9up) New/SKA/Organisation/in/place Key/Doc/Set/&/Prospectus Formal/negotiations Ratification/of/Agreements Organisation/has/minimum/no./Members/with/Agreements/ratified Agreements/in/Key/Doc/Set/signed Andrea/Casson,/SKAO/Project/Controller,/Sept/2014 System/PDR Integration/testing/on/site Advanced/Instrumentation/Prog. KEY:/Blue/=/SKA1/science/&/engineering;/orange/=/policy;/green/=/SKA2;/milestones*in*italics*=/proposals/under/discussion The SKA1 timeline
  56. CMM U. Chile, December 10, 2014 Spain SKA Day, October

    23rd, Granada The SKA1 timeline Issue%Baseline% Design Science% Assessment% Workshops Proposal% Feedback Level%1% Requirements% Feedback Science% Conference Engineering% Meeting Engineering%Change%Proposals Designs%&% Costs SKAO%Review Technical% Advice (working% groups) SEAC Board Consortium%Design%Activities Level%0% Requirements Brief% SEAC 2013 2015 Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar 2014 PDR Science Review%Panel Science Review% Panel !! !! !! !! !! !! !!(43) !! !! !! !! Issue%Baseline% Design Science% Assessment% Workshops Proposal% Feedback Level%1% Requirements% Feedback Science% Conference Engineering% Meeting Engineering%Change%Proposals Designs%&% Costs SKAO%Review Technical% Advice (working% groups) SEAC Board Consortium%Design%Activities Level%0% Requirements Brief% SEAC 2013 2015 Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar 2014 PDR Science Review%Panel Science Review% Panel
  57. CMM U. Chile, December 10, 2014 The SKA1 timeline Issue%Baseline%

    Design Science% Assessment% Workshops Proposal% Feedback Level%1% Requirements% Feedback Science% Conference Engineering% Meeting Engineering%Change%Proposals Designs%&% Costs SKAO%Review Technical% Advice (working% groups) SEAC Board Consortium%Design%Activities Level%0% Requirements Brief% SEAC 2013 2015 Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar 2014 PDR Science Review%Panel Science Review% Panel Level%1% Requirements% Feedback Engineering% Meeting Engineering%Change%Proposals Designs%&% Costs SKAO%Review Technical% Advice (working% groups) SEAC Boar Consortium%Design%Activities Level%0% Requirements Brief% SEAC Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar PDR
  58. CMM U. Chile, December 10, 2014 • Formed by External

    Reviewers, AG representative, SE representative, PM, Observers • Chaired by a selected External Reviewer • SKAO Project Manager acts as Secretary • SKAO personnel contributes to OAR • Consortia respond to Consolidated OAR before PDR review • Face-to-face for outstanding actions, 1-2 days PDR panels and process
  59. CMM U. Chile, December 10, 2014 • Element overall design

    report. • Specific design reports for all second tier subsystems (the Element being the first tier), including manufacturing/procurement approach, transport, installation, verification, support… • Finite element models (if needed). • Software architecture models and use cases. • Parametric models. • Analysis reports: Finite element modelling (including all mechanical and thermal load cases), thermal analysis, fluid dynamics, RFI/EMC analysis, safety analysis, hazard analysis… • Cost analysis (development, construction, verification, commissioning & steady state ops). • Preliminary construction plans. • Configuration items list. • Compliance matrix. • Preliminary maintenance plan. • Preliminary verification plan. • Schedule estimate for construction. • High level risk register. • Draft Statement of work for the construction of the Element. • Draft technical specifications for the construction of the Element. PDR documentation
  60. CMM U. Chile, December 10, 2014 • Interface Control Documents

    (ICDs) • Owned by the SKAO, developed between the Consortia • Technology Readiness Level Assessment • Controlled through Risk Register right now PDR documentation (in parallel)
  61. Questions? Juande Santander-Vela j.santander-vela@skatelescope.org