Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SLATE: Services Layer Platform at the Edge

SLATE: Services Layer Platform at the Edge

Introducing our new NSF funded project (@slateci) to develop a services layer to advance construction of a national research platform for collaborative science

Rob Gardner

August 12, 2017
Tweet

Other Decks in Research

Transcript

  1. SLATE Services Layer at the Edge Rob Gardner Shawn McKee

    Joe Breen University of Chicago University of Michigan University of Utah First Meeting of the National Research Platform Montana State University August 7-8, 2017
  2. What is SLATE? • New NSF DIBBs award, "SLATE and

    the Mobility of Capability" • Equip the SciDMZ with a service orchestration platform, potentially federated to create scalable, multi-campus science platforms • Underlayment-as-a-service (UaaS) for platform builders & science gateway developers 2
  3. Early 'platforms': ivdgl, grid3, open science grid, wlcg Deployments of

    globus software for gatekeepers and storage interfaces, condor for job submission and routing Since 2003 we've led deployments of integration testbeds and supported production services across ~100 institutions 5
  4. CERN LHC computing: global grid of data centers ATLAS Tier2

    centers led by UMich and UChicago • Multi-site grid federations of distributed storage, job routing, software and data caches in production > 10 years • Each supports workflows from >1k users from the international collaboration 6
  5. perfSONAR meshes & data federations Helping campuses deploy perfSONAR instances

    to optimize data throughput (OSG and WLCG) Managed deployment of xrootd federation network of 66 LHC data centers xrootd data federation 7
  6. Friction in deployment of distributed services A broken DevOps cycle!

    - Latency in updates across sites make it extremely difficult to rapidly innovate platform services VO (domain & application) centric - Requires distributed expertise in specialized software stacks across many institutions - Resource providers need to become experts in a range of VO software services 8
  7. 10

  8. Services layer at the edge • A ubiquitous "underlayment" --

    the missing shim ◦ A generic CI substrate optimized for hosting edge services ◦ Programmable ◦ Easy & natural for HPC and IT professionals ◦ Tool for creating "hybrid" platforms • Potentially federate ◦ Centralized deployment & operations possible • DevOps friendly (for platform and science gateway developers) ◦ quick patches, release iterations, fast track new capabilities 11
  9. Concepts & components • Containerized services in managed clusters •

    Standalone or federated • Federated clusters support multi-site platform systems • Widely used open source technologies for growth and sustainability 12
  10. WP2: Node Specification & Deployment (SPN) • API for local

    and (potentially) remote admin • Monitoring & analytics • SLATE SciDMZ node -- to be compatible with emerging national specs • "Big", "Small", "Central" types 13
  11. Two important challenges (among many) Cybersecurity and policy • On

    premise services, potentially managed by third parties, present additional risks that need to be understood ◦ Engagements with CISO from our campuses ◦ Engagement with Center for Trustworthy Scientific Cyberinfrastructure Scientific outreach • Underlayment to Science ◦ Embedded with science collaborations with multi-institution cyberinfrastructure - distributed data services, software, job routing ◦ Developers associated with the Science Gateways Community Institute 15
  12. SLATE and the NRP • A service for science platform

    builders ◦ Easily compatible with FIONA box and current PRP setups • We need partners & volunteers: ◦ to help inform technology choices ◦ to confront and solve policy & cybersecurity issues ◦ focus on usability & integration in HPC environments ◦ usability for science gateway developers & distributed service deployment teams ◦ align with emergent national SciDMZ patterns 16
  13. Timeline - first year • Testbed spec and deployment •

    Service orchestration technology evaluations • Early multi-site pilot demonstrator with: ◦ stateless services ◦ stateful services ◦ non-trivial HPC resource integration • Early federated platform tools • Recruit volunteers and build community 17
  14. Summary • Reduce barriers to supporting collaborative science • Give

    science platform developers a ubiquitous "CI substrate" • Change distributed cyberinfrastructure opertional practice by mobilizing capabilities in the edge • Achieve scale through multi-site service meshes: ensembles of federated, orchestrated services 18 SLIDES: http://bit.ly/slateci-nrp