PoINT Software & Systems - IT Press Tour #45 - Paris, France - Sept. 2022

Transcript

1. © 2022 PoINT Software & Systems GmbH All rights reserved. www.point.de PoINT Software & Systems PoINT Software & Systems GmbH Company and Product Overview The IT Press Tour – Paris Edition September 6 -7, 2022
2. © 2022 PoINT Software & Systems GmbH All rights reserved. 2 PoINT Software & Systems www.point.de Roots in storage and archiving business (1985) • World-Wide Competence Center of Philips • European Project Organization of Digital Equipment (DEC) Foundation of PoINT Software & Systems GmbH in 1994 • Independent software vendor (ISV) located in Germany • Privately held company
3. © 2022 PoINT Software & Systems GmbH All rights reserved. 3 PoINT Software & Systems www.point.de PoINT Jukebox Manager (market launch 1997) • Native file system access to optical jukeboxes • CD, DVD, BD and MO/WORM, UDO support • World-wide sold licenses >2,500 • Leading software product for management of optical storage
4. © 2022 PoINT Software & Systems GmbH All rights reserved. 4 PoINT Software & Systems www.point.de PoINT Storage Manager File Tiering and Archiving
5. © 2022 PoINT Software & Systems GmbH All rights reserved. 5 PoINT Software & Systems www.point.de 80% of corporate data is unstructured file data Hot Active 10% Warm Less Active 20% Cold Inactive 70% Exponential data growth Archiving required for legal and compliance reasons PoINT Storage Manager Motivation
6. © 2022 PoINT Software & Systems GmbH All rights reserved. 6 PoINT Software & Systems www.point.de PoINT Storage Manager (market launch 2007) • File Tiering and Archiving • Supporting all storage technologies and storage systems • Certified by leading storage vendors • 200+ installations worldwide Flash Disk Tape Optical Supported Storage Media Supported Storage Systems Optical Libraries SONY, HIT, … Tape Libraries IBM, Spectra, … Public Cloud AWS, Microsoft, … Object Storage Cloudian, Scality, … NAS NetApp, EMC, …
7. © 2022 PoINT Software & Systems GmbH All rights reserved. 7 PoINT Software & Systems www.point.de Dell EMC FileMover Protocol PoINT NTFS/ReFS Service NetApp FAS API PoINT Web Link Service Storage Manager Services Object Store Connectors HDD / NAS Connectors Tape Library Connectors Optical Library Connectors PoINT Storage Manager Appliance Connectors Dell EMC Unity Windows File Servers NetApp FAS Generic NAS Share „WORM“ File System PoINT Archive File System PoINT Symbolic Link Service e.g. Dell EMC Isilon PoINT Storage Manager Software Architecture
8. © 2022 PoINT Software & Systems GmbH All rights reserved. 8 PoINT Software & Systems www.point.de PoINT Storage Manager Conceptual Overview Primary Storage - NAS Systems File Tiering & Archiving Policy-based archiving of inactive files from primary to secondary/archive storage CIFS / NFS Storage Systems ”off premises“ LAN Primary Storage Client and application transparency - Archived files are visible through primary storage - Multiple options to access archived data Secondary/Archive Storage - HDD / NAS - Appliances (CIFS, NFS) - Private and Public Object Stores (S3, Azure) - Tape (LTO, IBM 3592, LTFS, ACSLS) - Optical Storage Systems ”on premises“ Active Approach
9. © 2022 PoINT Software & Systems GmbH All rights reserved. 9 PoINT Software & Systems www.point.de PoINT Storage Manager File Access Methods Active Approach • Independent of operating system • HTTP/Internet • Search • Functionality • Versions • Search • Requires installation • Any NAS system • Indirect file system access • HTTP/Internet • Looks “similar” to original file • Reference to original file • Dependent of operating system Web Links File System • Transparent file system access • “Pass through“ • Previous versions in snapshots • Primary storage support Stubbing File System Application Client System Web Client Browser Symbolic Links File System Application Client System
10. © 2022 PoINT Software & Systems GmbH All rights reserved. 10 PoINT Software & Systems www.point.de PoINT Storage Manager Workflow Active Approach Primary Storage Secondary / Archive Storage Step 1: Copy (scheduled) Step 3: Stubbing (scheduled) Step 4: Pass Through on Read PoINT Storage Manager Step 5: Copy new version (scheduled) Edit Step 2: Optionally set retention (e.g. 10 years) Retention Management V1 V1 V1 V2 Read Create
11. © 2022 PoINT Software & Systems GmbH All rights reserved. 11 PoINT Software & Systems www.point.de PoINT Storage Manager Archive File System Passive Approach • Instant archiving New files are immediately archived • Transparent access Based on standard protocol and “Pass Through on Read” feature • Versioning Retains original file version when editing • WORM Protects archived files from any changes • PoINT File Retention Lock Application driven retention period by setting read-only flag and access-time Minimum / default period configured by Retention Management PoINT Archive File System Secondary / Archive Storage PoINT Storage Manager Copy / Create Read Modify / Delete Specify A-Time (e.g. 12/31/2031) Standard File System Operations
12. © 2022 PoINT Software & Systems GmbH All rights reserved. 12 PoINT Software & Systems www.point.de PoINT Storage Manager Archive Volumes Files are collected and Archive Volumes in UDF format are created • Volume size is appropriately adjustable to archive device/media • Universal Disc Format (UDF) is well-known and industry standard • Standardized format and platform-independent • Windows, Linux and MacOS can mount and read UDF volumes Files to be archived Creation of UDF Archive Volumes Write finalized Archive Volumes to archive device Vendor- Lock-In
13. © 2022 PoINT Software & Systems GmbH All rights reserved. 13 PoINT Software & Systems www.point.de PoINT Storage Manager Case Study - Daimler AG Active Approach Location 1 WAN LAN LAN Primary Storage Private Cloud Secondary / Archive Storage • Policy Based Tiering and Archiving • Stubbing / “Pass Through” • Transparent Access through Primary Storage • Optimized Caching • Archive Volumes (“Containerization”) • Strict multi-tenancy Capability by “Storage Vaults” • WORM and Versioning • Encryption • Authentication • Replication • Migration • Standard Formats and Protocols (UDF, RESTful HTTP, …) Location n PoINT Storage Manager PoINT Storage Manager Primary Storage
14. © 2022 PoINT Software & Systems GmbH All rights reserved. 14 PoINT Software & Systems www.point.de PoINT Archival Gateway Tape-based Object Storage
15. © 2022 PoINT Software & Systems GmbH All rights reserved. 15 PoINT Software & Systems www.point.de PoINT Archival Gateway Motivation How secure is cloud/object storage? • Hard drive failure • Node failure • Rack failure • Site failure • Software errors • Accidental or malicious deletion • Ransomware • The human factor Data loss scenarios ü Erasure coding ü Replication, CRR ü Multiple sites ü Backup Solutions
16. © 2022 PoINT Software & Systems GmbH All rights reserved. 16 PoINT Software & Systems www.point.de Storage classes at AWS, Google etc. AWS Google Cloud S3 Standard Multi-Regional / Regional S3 Standard-IA Nearline S3 Glacier Coldline S3 Glacier Deep Archive "Ice cold archive storage" Access frequency Price per TB Tape ! PoINT Archival Gateway Motivation
17. © 2022 PoINT Software & Systems GmbH All rights reserved. 17 PoINT Software & Systems www.point.de Advantages of Object Storage Approach for Tape • Designed for massive amounts of data without decrease of efficiency • Scalability and system security by adding nodes • Ingest of complete objects which perfectly supports sequential character of tape • Support for rich custom metadata (in contrast to file systems) • Prevention of alteration to assure integrity (Versioning, Object Lock) – fits well with tape • Redundancy by Erasure Coding – can be realized with multiple tape media • Standardized S3/HTTP application interface • HTTP can easily be routed and connected to any networked system • HTTP concept simplifies handling of tape-specific challenges like high access times and timeouts DON’T • Don’t tier or replicate data from object storage to file system • Don’t put S3 service on top of file system
18. © 2022 PoINT Software & Systems GmbH All rights reserved. 18 PoINT Software & Systems www.point.de PoINT Archival Gateway (market launch 2018) • Tape-based S3 Object Storage • Scalable solution for high data volume (10+ PB) and high throughput (1PB/day) • Certified by leading tape library vendors
19. © 2022 PoINT Software & Systems GmbH All rights reserved. 19 PoINT Software & Systems www.point.de PoINT Archival Gateway Features 8 Tape Libraries 256 Tape Drives Up to 230 GB/s throughput 50 Billion objects per S3 bucket Scales by Nodes Native S3 API Object Versioning Erasure Coding for tape storage High Availability Tape-based Object Storage High Performance S3 Gateway for Tape Storage No Hard Disk Cache/Buffer required !
20. © 2022 PoINT Software & Systems GmbH All rights reserved. 20 PoINT Software & Systems www.point.de Interface Nodes (IFN) • S3 REST API • Library Drives Control • Up to 8 drives • Multiple active nodes Database Nodes (DBN) • Databases and Logs • HTML admin interface • Library Robotics Control • Up to 4 nodes possible PoINT Archival Gateway System Components
21. © 2022 PoINT Software & Systems GmbH All rights reserved. 21 PoINT Software & Systems www.point.de PoINT Archival Gateway Compact Edition • DBN and IFN services on one node • Easy deployment • Failover cluster (optional) FC or iSCSI S3
22. © 2022 PoINT Software & Systems GmbH All rights reserved. 22 PoINT Software & Systems www.point.de Archival Storage Device - Group of up to 8 tape loaders or library units Archival Storage Volume - Medium Protected Volume Array (PVA) - 1 - 4 volumes - Protected by Erasure Codes Archival Storage Partition - Space provided by PVAs - Expands automatically Object Repository (Bucket) PoINT Archival Gateway Architecture
23. © 2022 PoINT Software & Systems GmbH All rights reserved. 23 PoINT Software & Systems www.point.de PoINT Archival Gateway Erasure Coding Code rate 3/4 (or 3+1)
24. © 2022 PoINT Software & Systems GmbH All rights reserved. 24 PoINT Software & Systems www.point.de PoINT Archival Gateway Erasure Coding Code rate 3/4 (or 3+1)
25. © 2022 PoINT Software & Systems GmbH All rights reserved. 25 PoINT Software & Systems www.point.de PoINT Archival Gateway Use Cases PoINT Archival Gateway Any S3 compatible App File Tiering & Archiving to S3 ILM and CloudMirror Auto-Tiering and CRR Tiering and Replication Cloud Backup HDFS S3a backend PDR Replication S3 API
26. © 2022 PoINT Software & Systems GmbH All rights reserved. 26 PoINT Software & Systems www.point.de PoINT Archival Gateway Use Case – Object/Cloud Backup to Tape Native S3 Restore S3-to-S3 Replication (based on change tracking) Cross Region Replication - CRR (if supported by source) Tape Library
27. © 2022 PoINT Software & Systems GmbH All rights reserved. 27 PoINT Software & Systems www.point.de PoINT Archival Gateway Workflow Continuous S3-to-S3 Replication Object Storage Tape Library Record consumer 2 Write to tape media 6 Change tracking and logging 5 Multi-threaded replication Send object Multi-threaded replication 3 Request object Notification: Object created 1 Send object 4
28. © 2022 PoINT Software & Systems GmbH All rights reserved. 28 PoINT Software & Systems www.point.de PoINT Archival Gateway Case Study – EMBL European Bioinformatics Institute The challenges • Archiving workloads from Kubernetes Cluster via S3 • Storage capacity in high three-digit PB range • Long-term preservation and high availability • Multiple vendor support incl. support for LTO and 3592 drives • High performance for read and write access (1 PB per week) • Low TCO The solution – PoINT Archival Gateway • Scale-out architecture and unlimited storage space • Native S3 interface including versioning • Logical partitioning for workload requirements • WORM and retention management • Read and write performance up to 3 PB per week The benefits • Compatibility with any S3 client • Workload specific configuration on bucket or partition level • Scalable by nodes, drives, slots and media • Software-defined for independence and sustainable planning • No vendor and technology lock-in Kubernetes Cluster S3 Clients Network Infrastructure PoINT Archival Gateway SAN Infrastructure IP Connectivity IP Connectivity Node 1 Cluster Network 40 GbE direct Fibre Channel Fibre Channel Node 2 Tape Libraries
29. © 2022 PoINT Software & Systems GmbH All rights reserved. 29 PoINT Software & Systems www.point.de PoINT Archival Gateway Unified Object Storage
30. © 2022 PoINT Software & Systems GmbH All rights reserved. 30 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage (market launch 10/2022) Positioning PoINT Archival Gateway - Unified Object Storage Storage Classes Disk S3 Lifecycle Policies Intelligent Tiering Tape S3 Software-defined Secondary and Archive Storage
31. © 2022 PoINT Software & Systems GmbH All rights reserved. 31 PoINT Software & Systems www.point.de PoINT Archival Gateway - Unified Object Storage Storage Classes Disk S3 Lifecycle Policies Intelligent Tiering Tape S3 PoINT Archival Gateway – Unified Object Storage Features • Software-defined • Multiple storage classes • Single namespace across disk and tape • AWS S3 compatible • Free choice of storage hardware • Flexible configurations • Automatic replication
32. © 2022 PoINT Software & Systems GmbH All rights reserved. 32 PoINT Software & Systems www.point.de Storage Class Purpose Disk Warm, frequently accessed data (low access times, within msec) Tape Cold, long-lived, archive data (direct access, within sec to minutes) PoINT Archival Gateway – Unified Object Storage Storage Classes • Disk and Tape • AWS S3 compatible • Transparent access to disk and tape storage class
33. © 2022 PoINT Software & Systems GmbH All rights reserved. 33 PoINT Software & Systems www.point.de S3 PoINT Archival Gateway - Unified Object Storage Storage Classes Disk Tape PoINT Archival Gateway – Unified Object Storage Storage Classes • Disk and Tape • AWS S3 compatible • Transparent access to disk and tape storage class No S3 Object Restore to disk necessary, but possible
34. © 2022 PoINT Software & Systems GmbH All rights reserved. 34 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage Supported Storage Hardware PoINT Archival Gateway - Unified Object Storage Storage Classes Disk S3 Lifecycle Policies Intelligent Tiering Tape S3 Supported systems and products • JBOD • HDD Object Storage Supported systems and products • LTO Libraries • 3592 Libraries
35. © 2022 PoINT Software & Systems GmbH All rights reserved. 35 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage Lifecyle Policies • AWS S3 compatible • Transition policies • E. g. migrate objects from disk to tape 30 days after creation • Expiration policies • E. g. delete objects one year after creation • Example • Storage of periodic logs in disk storage class for one month for analysis purposes • After one month migrate to tape storage class for archiving purposes • After further 24 months delete objects Disk Tape S3 30 days 2 years
36. © 2022 PoINT Software & Systems GmbH All rights reserved. 36 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage Restoring Objects from Tape to Disk • AWS S3 compatible command: Restore Object (Glacier API) • Objects can temporarily be restored on disk • Number of days how long a restored object is stored on disk can be specified S3: Restore Object S3: Get Object from Disk Disk Tape
37. © 2022 PoINT Software & Systems GmbH All rights reserved. 37 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage Automatic Replication • Automatic replication to disk and tape Disk Tape Automatic replication to disk and tape S3
38. © 2022 PoINT Software & Systems GmbH All rights reserved. 38 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage Intelligent Tiering1) • Automatic tiering of objects between disk and tape storage classes • Based on a cognitive AI that learns which objects are stored where and when it makes sense • Research project at PoINT Disk Tape Automatic Tiering based on learning AI 1) Release planned for Q2/2023
39. © 2022 PoINT Software & Systems GmbH All rights reserved. 39 PoINT Software & Systems www.point.de PoINT Archival Gateway – Unified Object Storage Benefits • Savings in storage costs (compared to all data on HDD) • No ingest, egress and transaction costs (compared to public cloud) • No vendor lock-in regarding storage hardware • Data sovereignty (on-premises private cloud) • Fulfillment of archiving and compliance requirements (by long-term data preservation) • Protection from ransomware and cybercrime (“Air-gap” by tape) • Investment protection by standardized interface (AWS S3) • Sustainability by less energy consumption and lower CO2 emissions (compared to all data on HDD)
40. © 2022 PoINT Software & Systems GmbH All rights reserved. 40 PoINT Software & Systems www.point.de www.point.de Software Products for Data & Storage Management “Made in Europe“