Nodeum - IT Press Tour #45 - Paris, France - Sept. 2022

Transcript

1. September 6th, 2022 IT PRESS TOUR
2. 2 | © 2022 NODEUM Hi, I’m Valery, CEO of Nodeum
3. 3 | © 2022 NODEUM MY VISION MY VISION Transform the traditional storage ecosystem and leverage Data Management as one solution for Hybrid Storage solution. Therefore, to answer these strategic challenges, I design Nodeum. Aiming to accelerate the combinations and the relationships between machines and humans to get values out of the data to serve new business models. Eradicate time spent in searching data. Empower talented end-users to achieve more – to solve more of their data management challenges.
4. 4 | © 2022 NODEUM We started from a paper and ideas
5. 5 | © 2022 NODEUM DATA DELUGE • World is changing • The way on how information are generated is shifting drastically and this started at least from 5-10 years. • The result is an unprecedented and exponential increase of volumes of (unstructured) data that need to be managed.
6. 6 | © 2022 NODEUM NAS A NAS B PUBLIC CLOUD A OBJECT STORAGE TAPE STORAGE PUBLIC CLOUD B I’m comfortable to manage this single system. I’m feeling lost to manage safety, quickly, easily and cost- effectively such large volumes of data MY STORAGE
7. 7 | © 2022 NODEUM Data Management Complexity Data Size VOLUME NEW CHALLENGES 5
8. 8 | © 2022 NODEUM Plusieurs Millions de fichiers Milliers de fichiers Seconde(s) Heure(s) ou même Jour(s) 1xx TBs TB Nombre de fichiers Temps pour obtenir un résultat Billions of files Millions of files Files # Time to get a result Hours or even days seconds TRACEABILITY
9. 9 | © 2022 NODEUM PROTECTION Availability
10. 10 | © 2022 NODEUM FREEDOM Lock-In Freedom Convenience Discounts Innovation Cost Flexibility
11. 11 | © 2022 NODEUM COMPLEXITY
12. 12 | © 2022 NODEUM BACKUP Structured Data oriented solution to backup Databases, Applications, …. HSM Hierarchical storage management moves unused data to cost- effective storage media DATA ARCHIVING & PRESERVATION Filter on metadata to archive less demanded content. Leverage storage for long term preservation. DATA PROTECTION Fast and highly efficient file-based workflow to protect unstructured data DATA MOBILITY WORKFLOW DRIVEN High-performance, date mover which enables line-of- business creativity, productivity, and efficiency IT LINE OF BUSINESS OWNERSHIP
13. 13 | © 2022 NODEUM Hardware Training Maintenance Complexity Lock-in Time Volume CAPEX BUT ALSO OPEX
14. Big data expert Bernard Marr writes … : “ From a technological standpoint, we already have everything we need to leverage big data to quickly and effectively develop vaccines for new viruses such as Zika… Now what we need are platforms and systems to get this data into the hands of those who can develop solutions before a public health emergency develops. ”
15. BUT IT IS NOW DATA MANAGEMENT IS NOT ANYMORE MYTH REALITY
16. IT IS WHY WE BUILT NODEUM DATA MANAGEMENT SOFTWARE FOR HYBRID STORAGE Data Management Workflow Scalability Easy to access Catalog (OPEN) API No lock in
17. 17 | © 2022 NODEUM THE MARKETS WE SERVE RESEARCH GENOMICS DIGITAL PATHOLOGY UNIVERSITY MEDIA BROADCAST PRODUCTION POST-PRODUCTION GEOSPATIAL
18. 18 | © 2022 NODEUM ARCHIVE, MIGRATE AND PROTECT Data Migration Automate secure migration of your data between your different storage systems. Individual file reporting for audit requirements. Data Archiving Data Protection Move selected data to secondary storage tier to decrease the capacity of expensive primary storage while keeping online access to your data. Individual file-level search and access. User-centric backup approach to eliminate disruption and ensure business continuity with a reliable and easily accessible offline archive. Individual file-level search and retrieve.
19. 19 | © 2022 NODEUM High-performance workflow 01 Checksum integrity 02 Verification and Monitoring 03 TO EXECUTE AND CONTROL ANY DATA MOVEMENT A POWERFUL DATA MOVER
20. 20 | © 2022 NODEUM NAS Storage It supports file-based NAS storage through NFS – SMB protocols Cloud Storage Public Cloud Storage and Object Storage are supported through the 2 main standard protocols: S3 & Swift. Tape Storage Tape Library Manager brings the capability to address any SCSI driven Tape Library, ACSLS Tape Library and LTO Tape Drives. Most known systems are supported. The open format LTFS is used as a standard solution for writing to tapes. NAS, CLOUD, S3 COMPATIBLE OBJECT STORAGE AND TAPE ONE SOFTWARE TO MANAGE ALL TYPE OF STORAGE
21. 21 | © 2022 NODEUM CONTENT TRACEABILITY – DATA LIFECYCLE
22. 22 | © 2022 NODEUM REDUCE COMPLEXITY MANSUETO LIBRARY, UNIVERSITY OF CHICAGO, Book capacity 3.5 million, Number of book bins 24,000, Height of retrieval cranes 55 feet, Delivery time 5 minutes NODEUM, a single pane of glass of all storage movement across any type of storage type. With data mobility and cost management. 5 minutes to execute a data movement.
23. 23 | © 2022 NODEUM Commodity Hardware Open Format ZERO VENDOR LOCKIN
24. 24 | © 2022 NODEUM KEY BENEFITS ONE SOFTWARE TO MANAGE ALL OF YOUR STORAGE Achieve a high-performance workflow to gain control of your exponential data growth. Improve operations and save time Intuitive storage and process management software Handle single namespace across multiple storage silos.
25. USE CASES ARCHIVE, PROTECT, AND MIGRATE VIDEO AND UNSTRUCTURED DATA ACROSS ANY TYPE OF STORAGE (NAS, CLOUD, OBJECT AND TAPE)
26. 26 | © 2022 NODEUM SOME OF OUR KEY CUSTOMERS
27. 27 | © 2022 NODEUM MPC EPISODIC & FILM NEED: Integrate a data management solution into their post-production workflows which are very complex. SOLUTION : Nodeum automate their archiving and delivery workflows. They can work on more show – movies in accelerating the archive of rushes and masters with the same team. They are able to deliver on-time content to Netflix.
28. 28 | © 2022 NODEUM GRAZ MEDICAL UNIVERSITY NEED: The aim of the project is to continue to build up a central archive with digitalized pathological data. SOLUTION: With Nodeum Data Mover, Medical University of Graz can analyze a large scale of data generated by the high throughput scanner. They are able to determine the type of disease faster (such as carcinoma of the thyroid). Nodeum reduces the time to analyze each slide allowing for increased usage of expensive medical hardware. NODEUM, DATA MOVEMENT PLATFORM FOR RESEARCH FROM 10 minutes TO 1 minute
29. 29 | © 2022 NODEUM UNIVERSITY OF ROSTOCK “ We consider it important, in addition to the optimized data structure within and between our heterogeneous system landscapes, to implement more security and an additional media break to protect our data as part of our data storage mechanisms. The use of the convenient and technically mature data mover NODEUM® ensures us efficiency and data security through the individual linking of our data sources, also via APIs and taking into account all role and authorization models. ” Malte Willert, University of Rostock, IT and Media Center, Department 2: Systems and services, project manager for the introduction of NODEUM®. KEEP CONTROL OF DATA AND LEVERAGING LTFS TAPE STORAGE TECHNOLOGY AND MEDIA BREAK EXPAND THE 3-2-1 RULE IN ARCHIVING
30. 30 | © 2022 NODEUM SPACE INDUSTRY (CONFIDENTIAL) 2 TYPE OF MEDIA : OBJECT STORAGE & TAPE STORAGE NEED: Integrate a data management to protect contents into 2 copies – 2 different vendors – 2 different media 2 TYPE OF STORAGE FORMAT TO BATTLE TO REDUCE RISK OF DUST - HUMIDITY - HEAT. ...) SEPARATED GEO LOCALISATION
31. 31 | © 2022 NODEUM JESUISBIENCONTENT NEEDS: Keep control of your data when leveraging LTFS Tape Storage. Transparent access of any Tape storage technology Store content in ‘offline’ medium. SOLUTION: Enables in-house users who might be skeptical about the tape opportunity to try out Organize data movement directly to Tape and keep local disk caching for most used data Accelerate the adoption of tape storage and enables an elastic storage offering when capacity requirements scale.
32. NEXT GEN NODEUM ARCHITECTURE IS COMING Data Management Highest throughput IDP Bash Client Docker Ready Interface with Workload Manager (e.g. SLURM) Hook & Trigger Released in January 2023
33. 33 | © 2022 NODEUM NEXT GEN SOFTWARE ARCHITECTURE COMING SOON – January 2023 HIGHEST DATA MOVEMENT THROUGHPUT - READY FOR EXASCALE COMPUTING COMPATIBLE WITH IDENTITY PROVIDER (SERVICE AND STORAGE) BASH CLIENT TO ALLOW DATA MOVEMENT DIRECTLY FROM COMPUTE NODE SUSTAIN 5,000 REQUESTS PER SECOND AND HANDLE 10 MILLION SIMULTANEOUS TRANSFER REQUESTS DOCKER AND APP MARKETPLACE READY
34. 34 | © 2022 NODEUM NEXT GEN SOFTWARE ARCHITECTURE • Allows event-based hooks registering • Bash Command Line interfaces to communicate in REST with Nodeum • Full integration with AAI infrastructure: • local accounts for accessing file system-based storage • token-based access for the object storage What’s new ? • BANKING • HPC • And also for our current existing verticals: • Media, Post Production, Broadcasting, ... • LifeScience, Research, Genomics, ... • Geospacial For who ?
35. 35 | © 2022 NODEUM www.humanbrainproject.eu – www.fenix-ri.eu KEEP A DIRECT ACCESS BY THE USERS TO ACTIVE AND ARCHIVAL DATA REPOSITORIES INTEGRATION WITH HPC WORKLOAD MANAGERS LIKE SLURM ORGANIZE THE MOVEMENT OF THE DATA FROM THE ACTIVE TO THE ARCHIVAL DATA REPOSITORY Today, supercomputing systems are so performant and scalable that the speed of data generation has never been so fast. Research centers have to store the generated content in “data repositories” that are located close to each other and that are well integrated. Two different categories of data repositories are used as storage tiers: • Active Data Repositories which provide the performance when data is written by supercomputing systems • Archival Data Repositories with interfaces used in Cloud systems, which are more suitable for data sharing Research is becoming increasingly collaborative. Sharing of data and FAIR data management is becoming mandatory. This adds to the requirements for modern data infrastructures. PROVIDE A PUBLIC API AND SDK TO FACILITATE INTEGRATION WITH SPECIFIC RESEARCH APPLICATIONS deployed deployed under deployment October 2022 November 2022 #11 top500.org #17 #82 #23 #21
36. 36 | © 2022 NODEUM NEXT GEN SOFTWARE ARCHITECTURE • Allows event-based hooks registering • Bash Command Line interfaces to communicate in REST with Nodeum • Full integration with AAI infrastructure: • local accounts for accessing file system-based storage • token-based access for the object storage What’s new ? • S3 – SWIFT – Mountable File Systems storage systems are available day 1. • Tape Storage is in the roadmap and will be released in a second step. Roadmap « The multifunctional Data Mover NODEUM® simplifies the data transfer between cloud object stores with a Swift interface and fast parallel file systems with a POSIX interface. » Fenix-ri project owner Customer Satisfaction • BANKING • HPC • And also for our current existing verticals: • Media, Post Production, Broadcasting, ... • LifeScience, Research, Genomics, ... • Geospatial For who ?
37. PAST GIVES THE IDENTITY AND FUTURE HOLDS THE PROMISE OF SALVATION Eckhart Tolle, The Power of Now: A Guide to Spiritual Enlightenment
38. 38 | © 2022 NODEUM VENDORS PARTNER WITH US S3 SWARM S3 ActiveScale S3 Glacier ActiveScale Cold Storage Tape Scalar File StorNext DATA PROTECTION DATA ARCHIVE DATA PROTECTION DATA ARCHIVE SCALAR LTFS REPLACEMENT Nodeum helps vendors in data archiving, protection and data migration scenarios
39. 39 | © 2022 NODEUM MILESTONES & STRATEGY 2017 2022 2018 - 2021 2023 - …. Achieved 2015 - 2016 R&D Phase Scale—up Phase Pilot Phase Product Design Made our first customer Market validation with key references and uses cases New Channel in UK and DACH Release Nodeum v2.0 Market and Channel Expansion
40. THANK YOU TO LISTENING