Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Square Kilometre Array: Overview and Engineering Update

The Square Kilometre Array: Overview and Engineering Update

The Square Kilometre Array is currently undergoing the Preliminary Design Reviews for its composing elements, and is thus at a critical point on its way to becoming ready for construction starting in 2018. In this talk we will provide an overview of the SKA, its composing elements, and their status, with emphasis on the Telescope Manager and the Science Data Processor, respectively the Monitoring & Control system, and Pipeline. We will see how do they compare with their ALMA equivalents, and how is the SKA similar/different from ALMA, and what are the main issues to be solved regarding the SKA computing infrastructure.

Talk given on Dec 10, 2014, at the Centre for Mathematical Modelling (CMM) of the University of Chile.

Juande Santander-Vela

December 10, 2014
Tweet

More Decks by Juande Santander-Vela

Other Decks in Technology

Transcript

  1. The Square Kilometre Array:
    Overview and Engineering Update
    CMM, Dec 10 2014
    Juande Santander-Vela

    View Slide

  2. • Who am I?

    • SKA Science

    • Telescopes, Sites, Technologies

    • SKA Organisation

    • Engineering Progress

    • SKA as an Exaflop Tomograph

    • Distributed SKA Control

    • What’s Next?
    Outline
    CMM U. Chile, December 10, 2014

    View Slide

  3. Who am I?
    CMM U. Chile, December 10, 2014

    View Slide

  4. CMM U. Chile, December 10, 2014
    • Systems Engineer for the SDP/TM elements

    • Former VIA-SKA Project Manager

    • VLT Archive Applied Scientist, later ALMA
    Science Archive Front-/Back-end
    Developer at ESO

    • Ph.D. in Radio Astronomical data archives
    and Virtual Observatory

    • Software Analyst/Manager
    Who am I?

    View Slide

  5. Great observatories of the present
    CMM U. Chile, December 10, 2014
    ALMA SCO HQ, Dec 11 2014

    View Slide

  6. …and future
    CMM U. Chile, December 10, 2014
    ALMA SCO HQ, Dec 11 2014

    View Slide

  7. …and future
    CMM U. Chile, December 10, 2014
    ALMA SCO HQ, Dec 11 2014

    View Slide

  8. …and (near) future
    CMM U. Chile, December 10, 2014
    ALMA SCO HQ, Dec 11 2014

    View Slide

  9. SKA Science
    CMM U. Chile, December 10, 2014

    View Slide

  10. Fundamental Forces &
    Particles
    • Gravity

    • Radio pulsar tests of
    General Relativity

    • Gravitational Waves

    • Dark Energy

    • Magnetism

    • Origin and evolution of
    Cosmic Magnetism

    Origins
    • Galaxies and the Universe

    • Cosmic Dawn

    • First Galaxies

    • Galaxy Assembly &
    Evolution

    • Stars, Planets & Life

    • Protoplanetary Disks

    • Biomolecules

    • SETI
    SKA Science Goals
    CMM U. Chile, December 10, 2014
    XX Century: We discovered our place in the Universe
    XXI Century: We understand the Universe we inhabit

    View Slide

  11. • Initial SKA Science Book (2004): 5 KSPs,
    43 additional ones

    • Updated SKA Science Book (2014): 13
    Phase 1 Headline Projects, 130 Chapters

    • Projects reduced, parameterised from ~43
    projects, through science prioritisation
    exercise with SKAO and SWGs
    SKA Science Goals
    CMM U. Chile, December 10, 2014

    View Slide

  12. • Astrobiology (“The Cradle of Life”)
    • Project Scientist: Tyler Bourke

    • WG Chair: Melvin Hoare

    • Galaxy Evolution – Continuum
    • Project Scientist: Jeff Wagg

    • WG Chairs: Nick Seymour &
    Isabella Prandoni

    • Cosmic Magnetism
    • Project Scientist: Jimi Green

    • WG Chairs: Melanie Johnston-
    Hollitt & Federica Govoni

    • Cosmology
    • Project Scientist: Jeff Wagg

    • WG Chair: Roy Maartens

    • Epoch of Reionisation & Cosmic Dawn
    • Project Scientist: Jeff Wagg

    • WG Chair: Leon Koopmans

    • Galaxy Evolution – HI
    • Project Scientist: Jimi Green

    • WG Chairs: Lister Staveley-Smith
    & Tom Osterloo

    • Pulsars (“Strong field tests of gravity”)
    • Project Scientist: Jimi Green

    • WG Chairs: Ben Stappers &
    Michael Kramer

    • Transients
    • Project Scientist: Tyler Bourke

    • WG Chairs: Rob Fender & J.-P.
    MacQuart
    SKA Science Goals: WGs
    CMM U. Chile, December 10, 2014

    View Slide

  13. SKA Science Goals
    CMM U. Chile, December 10, 2014
    Year in the life of SKA1: Mock-up
    3-years scheduling to constrain
    operations, computation

    View Slide

  14. Telescopes, Sites,
    Technologies
    CMM U. Chile, December 10, 2014

    View Slide

  15. SKA Telescopes
    CMM U. Chile, December 10, 2014
    SKA1-Low

    View Slide

  16. SKA Telescopes
    CMM U. Chile, December 10, 2014
    SKA1-Mid

    View Slide

  17. SKA Telescopes
    CMM U. Chile, December 10, 2014
    SKA1-Survey

    View Slide

  18. CMM U. Chile, December 10, 2014

    View Slide

  19. CMM U. Chile, December 10, 2014

    View Slide

  20. SKA Organisation
    CMM U. Chile, December 10, 2014

    View Slide

  21. • Australia (DoI)

    • Canada (NRCZHIA)

    • China (MOST)

    • Germany (BMBF)

    • India (NCRA)

    • Italy (INAF)

    • Netherlands (NWO)

    • New Zealand (MED)

    • South Africa (DST)

    • Sweden (Chalmers)

    • UK (STFC)
    SKA Organisation
    CMM U. Chile, December 10, 2014
    Currently UK Company Limited by Guarantee → Treaty Organisation
    Other partners in negotiation, expected

    View Slide

  22. SKA Governance: Pre-Construction
    CMM U. Chile, December 10, 2014
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    *
    SKA Governance
    Members$
    Board$of$Directors$
    $
    SKA$Office$
    ~50$staff$
    $
    Director;General$
    11$Design$ConsorXa$
    ~500$ScienXsts$&$Engineers$
    Strategy$&$$
    Business$Dev$
    Cmte$
    Finance$
    Cmte$
    ExecuXve$Cmte$
    Science$&$Eng$
    Advisory$Cmte$
    €23.4M$
    (cash)$
    ~$€125M$
    (in;kind)$

    View Slide

  23. SKA International Design Consortia
    CMM U. Chile, December 10, 2014
    Project Management and System Engineering Team based at JBO (UK)
    ~500 scientists & engineers in institutes & industry in 11 Member countries

    View Slide

  24. Engineering
    Progress
    CMM U. Chile, December 10, 2014

    View Slide

  25. 64 x 13.5m offset Gregorian
    antennas

    8km maximum baseline length

    First receivers:

    0.9 – 1.67 GHz (L-band)

    0.58 – 1 GHz (UHF)

    770 MHz bandwidth

    Early operations 2016/7
    SKA1 Precursors: MeerKAT
    CMM U. Chile, December 10, 2014
    First light with L-band
    receiver and digitiser
    erKAT

    View Slide

  26. SKA1 Precursors: MeerKAT
    CMM U. Chile, December 10, 2014
    Lot’s more of infrastructure already in place: dish assembly shed; bunkered and RFI
    shielded data processor room and power system

    View Slide

  27. SKA1 Precursors: MeerKAT
    CMM U. Chile, December 10, 2014
    Lot’s more of infrastructure already in place: dish assembly shed; bunkered and RFI
    shielded data processor room and power system

    View Slide

  28. SKA1 Precursors: ASKAP
    CMM U. Chile, December 10, 2014
    36 x 12m antennas
    3-axis movement
    30m – 6km baselines
    Novel PAF receiver
    700MHz – 1800MHz
    Wide field of view
    Fast survey speed

    View Slide

  29. BETA

    • 6 Mk I PAFs and digital systems

    • Commissioning, learning about beamforming 

    Lesson: deploy early, test, ensure adequate
    gap to production commitment

    ASKAP

    • First Mk II PAF installed

    • Results expected in next few weeks

    • Production of Mk II PAFs install through to Feb 2016
    SKA1 Precursors: ASKAP
    CMM U. Chile, December 10, 2014

    View Slide

  30. SKA1 Precursors: MWA
    CMM U. Chile, December 10, 2014
    Murchison Widefield Array, Operational
    17 refereed papers published, more coming

    View Slide

  31. 3 dish prototypes in testing
    CMM U. Chile, December 10, 2014

    View Slide

  32. 3 dish prototypes in testing
    CMM U. Chile, December 10, 2014

    View Slide

  33. 3 dish prototypes in testing
    CMM U. Chile, December 10, 2014

    View Slide

  34. SKA as an Exaflop
    Tomograph
    CMM U. Chile, December 10, 2014

    View Slide

  35. • Dishes, feeds, receivers (N=250 → 2500)

    • Low and mid aperture arrays (n=250k → 1000k)

    • Signal transport (~1 Pb/s → 10 Pb/s)

    • Signal processing (exa-MACs)

    • Software engineering & algorithm development

    • High performance computing (exa-flop
    capability)

    • Data storage (exa-byte capacity)

    • (Distributed) power requirements (10 → 50MW)
    Engineering Challenges
    CMM U. Chile, December 10, 2014

    View Slide

  36. SKA High-Level Architecture
    CMM U. Chile, December 10, 2014
    SKA-TEL-SKO-DD-001
    Revision : 1
    Figure 1 A schematic diagram of the SKA Observatory, showing the geographical locations of site entities
    (telescopes), the entities at regional centres (Host Country Headquarters and Science Data Processing), and
    Regional
    Science &
    Engineering
    Centre(s)
    Regional
    Science &
    Engineering
    Centre(s)
    SKA Observatory Global
    Headquarters
    Remote stations
    on spiral arms
    Science
    Computing
    Remote
    Station
    Remote
    Station
    Remote
    Station
    Host Country
    Headquarters
    Central Signal
    Processing
    SKA1-low
    SKA1-survey
    Core Arrays
    Australia
    Remote stations
    on spiral arms
    Science
    Computing
    Remote
    Station
    Remote
    Station
    Remote
    Station
    Host Country
    Headquarters
    Central Signal
    Processing
    Mid-Freq
    Aperture Array
    (SKA2)
    SKA1-mid
    Core Arrays
    South Africa

    View Slide

  37. CMM U. Chile, December 10, 2014
    SDP F2F Meeting, April 7th 2014, Cambridge
    SKA Elements & Interfaces
    INFRA
    DSH
    LFAA
    CSP SDP
    TM
    Command control flow
    Data flow
    SaDT
    Timing
    Conceptual figure:
    it should be split by
    telescope

    View Slide

  38. CMM U. Chile, December 10, 2014
    SDP F2F Meeting, April 7th 2014, Cambridge
    SKA Elements & Interfaces
    INFRA
    DSH/LFAA CSP SDP TM
    Command control flow
    Data flow
    SaDT
    Timing
    Conceptual figure:
    it should be split by
    telescope

    View Slide

  39. • Software Defined Networking

    • Allows easy partitioning, sub-arraying,
    quality of service, commensality…

    • Software Defined Beamforming (-Low, -
    Survey)

    • Software Defined Computing

    • Data-Flow Driven Science Data
    Processing
    SKA is a Software-Defined Telescope
    CMM U. Chile, December 10, 2014
    Lots of computing challenges

    View Slide

  40. CMM U. Chile, December 10, 2014
    Science Data Processor Local M&C
    Science Data Processor
    Telescope Manager
    Correlator /
    Beamformer
    Data Routing Ingest
    Visibility
    processing
    Multiple
    Reads
    Time Series
    Search
    Multiple
    Reads
    Data$Buffer
    Data Routing
    Time Series
    Processing
    Image Plane
    Processing
    Data
    Prodcuts
    Sky Models,
    Calibration
    Parameters ...
    Meta Data
    Master Controller
    Master Controller
    Local M&C
    Database
    Tiered Data
    Delivery

    View Slide

  41. CMM U. Chile, December 10, 2014
    Performance Modelling
    •  Imaging and calibration determines
    system sizing
    •  Data is Buffered after Ingest
    •  Detailed analysis building on Cornwell:
    •  Imaging
    o  AW-projection
    o  Plus w-snapshots
    o  Plus faceting (working memory size)
    •  Continuum imaging always required for
    calibration
    o  Field of view to second zero of beam
    •  Maximal case includes “science discovery”
    mode of forming high spectral-resolution image
    cube over primary beam
    •  Commensal fast-imaging mode for slow
    transients
    Processing
    maximal
    (Pflop)
    Ingest
    (GB/s)
    Use Case
    examples
    (GB/s)
    LFAA! 100! 7300! 245!
    Survey! 135! 4600! 995!
    Mid! 360! 3300! 255!
    Detailed analysis is much
    more complicated

    View Slide

  42. CMM U. Chile, December 10, 2014

    View Slide

  43. CMM U. Chile, December 10, 2014

    View Slide

  44. SDP Parallelism: Frequency
    CMM U. Chile, December 10, 2014
    x Time: accumulate visibilities into separate grids and combine later
    x Beams
    Simplified examples are illustrated in the following figures.
    Figure 5: Illustrating data parallelism based on frequency selection only (c.f. LOFAR)

    View Slide

  45. SDP Parallelism: uv-space partition
    CMM U. Chile, December 10, 2014
    Figure 6: Managing memory bandwidth by distributing regions of target uv-space over nodes followed by a
    gather and FFT to image space.

    View Slide

  46. SDP Parallelism: spatial partition
    CMM U. Chile, December 10, 2014
    Figure 6: Managing memory bandwidth by distributing regions of target uv-space over nodes followed by a
    gather and FFT to image space.
    Figure 7: Managing IO bandwidth from local storage by distributing source data over multiple nodes. Target
    gridded visibility plane is duplicated then an accumulation step is performed over the data island over which
    the source data were distributed
    3.5 Data flow programming
    As discussed in Section 4 a data-flow approach is adopted for the top-level SDP architecture. In this
    section we provide some background discussion to the established concept of data-flow

    View Slide

  47. >1 EXAFLOP/S
    Correlation
    Data Product
    Generation Temporary
    Storage
    Long Term
    Storage
    High Availability
    Storage / DB
    On-Demand
    Processing
    CAN’T STORE IT!
    1 DAY STREAM = 150 DAYS
    GLOBAL INTERNET TRAFFIC
    PROCESSING NEEDS
    109 TOP RANGE PCS
    Antenna &
    Fronted-End
    Systems
    30-300 PETAFLOP/S
    30 PETAFLOP/S
    SKA Data Flow and Processing
    CMM U. Chile, December 10, 2014

    View Slide

  48. Distributed SKA
    Control
    CMM U. Chile, December 10, 2014

    View Slide

  49. • Need to find COTS-based solution

    • Move away from expensive hard real-time
    solutions

    • Switched network inside products

    • Orchestration instead of deterministic control

    • Certain amount of product independence
    SKA Control Problem
    CMM U. Chile, December 10, 2014

    View Slide

  50. CMM U. Chile, December 10, 2014
    Figure 1. ALMA Software Subsystems interaction diagram (source: ALMA Software documentation)

    View Slide

  51. CMM U. Chile, December 10, 2014
    TM.TELMGT
    TM.OBSMGT
    DSH.LMC CSP.LMC
    SDP.INGEST SDP.PIP.* SDP.ARCH
    SDP.LMC
    Commands
    Data
    Monitoring/Status
    CSP.CORR
    DSH.RECEIVER
    SKA Observation interaction

    View Slide

  52. CMM U. Chile, December 10, 2014
    Ethernet switching
    Middleware TBD
    (EPICS/TANGO/OPC UA)
    DSH.LMC
    SADT.DDBH
    SADT.TM
    CSP.LMC
    SADT.SAT

    View Slide

  53. CMM U. Chile, December 10, 2014
    TM.TELMGT
    TM.OBSMGT
    TM.TELMOD
    SDP.PIP.

    View Slide

  54. What’s Next?
    CMM U. Chile, December 10, 2014

    View Slide

  55. CMM U. Chile, December 10, 2014
    2016
    2015 2017 2018 2019 2020 2021 2022 2023
    2014
    High/level/design
    Rebaselining/submissions/(RBS)
    Preliminary/Design/Review/(elements)
    Detailed/design
    Prototype/systems/deployed
    Critical/Design/Review
    SKA1/construction/approved
    Procurement
    SKA1/early/science
    SKA1/construction
    SKA2/detailed/design
    Design/consortia/start
    SKA2/procurement
    SKA2/construction/starts
    SKA2/concept/development
    Board/RBS/approval 3K5/Mar/15
    15K22/Sep/14
    1/Nov/13
    PreKconstruction/Stage/1
    PreKconstruction/Stage/2
    Production*Readiness*Review*(4*dish*array)
    Construction*Readiness*Review*
    (1*dish*line9up)
    New/SKA/Organisation/in/place
    Key/Doc/Set/&/Prospectus
    Formal/negotiations
    Ratification/of/Agreements Organisation/has/minimum/no./Members/with/Agreements/ratified
    Agreements/in/Key/Doc/Set/signed
    Andrea/Casson,/SKAO/Project/Controller,/Sept/2014
    System/PDR
    Integration/testing/on/site
    Advanced/Instrumentation/Prog.
    KEY:/Blue/=/SKA1/science/&/engineering;/orange/=/policy;/green/=/SKA2;/milestones*in*italics*=/proposals/under/discussion
    The SKA1 timeline

    View Slide

  56. CMM U. Chile, December 10, 2014
    Spain SKA Day, October 23rd, Granada
    The SKA1 timeline
    Issue%Baseline%
    Design
    Science%
    Assessment%
    Workshops
    Proposal%
    Feedback
    Level%1%
    Requirements%
    Feedback
    Science%
    Conference
    Engineering%
    Meeting
    Engineering%Change%Proposals
    Designs%&%
    Costs
    SKAO%Review
    Technical%
    Advice
    (working%
    groups)
    SEAC
    Board
    Consortium%Design%Activities
    Level%0%
    Requirements
    Brief%
    SEAC
    2013 2015
    Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
    2014
    PDR
    Science
    Review%Panel
    Science
    Review%
    Panel
    !! !!
    !! !!
    !!
    !!
    !!(43)
    !!
    !!
    !!
    !!
    Issue%Baseline%
    Design
    Science%
    Assessment%
    Workshops
    Proposal%
    Feedback
    Level%1%
    Requirements%
    Feedback
    Science%
    Conference
    Engineering%
    Meeting
    Engineering%Change%Proposals
    Designs%&%
    Costs
    SKAO%Review
    Technical%
    Advice
    (working%
    groups)
    SEAC
    Board
    Consortium%Design%Activities
    Level%0%
    Requirements
    Brief%
    SEAC
    2013 2015
    Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
    2014
    PDR
    Science
    Review%Panel
    Science
    Review%
    Panel

    View Slide

  57. CMM U. Chile, December 10, 2014
    The SKA1 timeline
    Issue%Baseline%
    Design
    Science%
    Assessment%
    Workshops
    Proposal%
    Feedback
    Level%1%
    Requirements%
    Feedback
    Science%
    Conference
    Engineering%
    Meeting
    Engineering%Change%Proposals
    Designs%&%
    Costs
    SKAO%Review
    Technical%
    Advice
    (working%
    groups)
    SEAC
    Board
    Consortium%Design%Activities
    Level%0%
    Requirements
    Brief%
    SEAC
    2013 2015
    Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
    2014
    PDR
    Science
    Review%Panel
    Science
    Review%
    Panel
    Level%1%
    Requirements%
    Feedback
    Engineering%
    Meeting
    Engineering%Change%Proposals
    Designs%&%
    Costs
    SKAO%Review
    Technical%
    Advice
    (working%
    groups)
    SEAC
    Boar
    Consortium%Design%Activities
    Level%0%
    Requirements
    Brief%
    SEAC
    Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
    PDR

    View Slide

  58. CMM U. Chile, December 10, 2014
    • Formed by External Reviewers, AG
    representative, SE representative, PM,
    Observers

    • Chaired by a selected External Reviewer

    • SKAO Project Manager acts as Secretary

    • SKAO personnel contributes to OAR

    • Consortia respond to Consolidated OAR
    before PDR review

    • Face-to-face for outstanding actions, 1-2 days
    PDR panels and process

    View Slide

  59. CMM U. Chile, December 10, 2014
    • Element overall design report.

    • Specific design reports for all
    second tier subsystems (the
    Element being the first tier), including
    manufacturing/procurement
    approach, transport, installation,
    verification, support…

    • Finite element models (if needed).

    • Software architecture models and
    use cases.

    • Parametric models.

    • Analysis reports: Finite element
    modelling (including all mechanical
    and thermal load cases), thermal
    analysis, fluid dynamics, RFI/EMC
    analysis, safety analysis, hazard
    analysis…

    • Cost analysis (development,
    construction, verification,
    commissioning & steady state ops).

    • Preliminary construction plans.
    • Configuration items list.

    • Compliance matrix.

    • Preliminary maintenance plan.

    • Preliminary verification plan.

    • Schedule estimate for construction.

    • High level risk register.

    • Draft Statement of work for the
    construction of the Element.

    • Draft technical specifications for
    the construction of the Element.
    PDR documentation

    View Slide

  60. CMM U. Chile, December 10, 2014
    • Interface Control Documents (ICDs)

    • Owned by the SKAO, developed
    between the Consortia

    • Technology Readiness Level Assessment

    • Controlled through Risk Register right
    now
    PDR documentation (in parallel)

    View Slide

  61. Questions?
    Juande Santander-Vela
    [email protected]

    View Slide