Disk Archive - IT Press Tour 52 Dec. 2023 Madrid

© Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.
Archive Innovation & Global Standards for Data Preservation Disk Archive Corporation Ltd E.TV Cape Town Megahertz Broadcast

Disk Archive Corporation Company Background Headquarters in UK, founded in 2008 Fourteen years of innovation in Film & TV Preservation Open Systems - Standards-based, No Proprietary Technology No licenses, no Vendor lock-in, Local Disk Purchasing Futureproof storage in the 100TB to 100 PB range Vault Storage & Airgap Security Lifetime cost avoidance, power reduction & disk life in decades Lowest carbon footprint We Make Archiving Easy, Secure and Affordable © Disk Archive Corporation 2023 All Rights Reserved. Commercially Confidential.

Demand in hours driven by growth in production Files are becoming massive driven by higher resolutions Film scanning in ultra-high deﬁnition for re-purposing Content may be locked into unplayable Video Tapes Content may be locked into unreadable Data Tapes Video Tapes & Optical Disks - disappearing Migrate to futureproof Media or face obsolescence Hybrid Systems, Forensic Recovery, Anti Malware Challenges in Film & TV Archive and Preservation © Disk Archive Corporation 2023 All Rights Reserved. Commercially Confidential.

Legal processes are becoming increasingly complex Taking longer to investigate & prosecute - years New “Smart Cities” capturing more content on CCTV High Resolution & High Frame Rates for facial recognition Multiple camera angles in many more courtrooms Remote network access for Investigators over networks Shares many of the characteristics of a Television Archive Longevity is paramount but long-term cost is also critical Challenges in Legal Evidence Preservation © Disk Archive Corporation 2023 All Rights Reserved. Commercially Confidential.

A High proportion of data is at-rest & accessed infrequently Online spinning-disk storage is expensive to own and operate Flash Storage Unlikely to replace HDD on cost and capacity Disk in Cloud is unaffordable and impractical for Archives Data Tape storage perceived as the only low-cost medium The reality is many hidden costs & difﬁcult to manage well Unmanaged Tape in Cloud is impractical for active Archives Investment in Technology may be amortized over 3,4 or 5 I. nvestment in Archive technology – 10, 15, 20 years Archive & Preservation Characteristics © Disk Archive Corporation 2023 All Rights Reserved. Commercially Confidential.

Archive Aspiration No 1. Lower Lifetime Cost Archive owners want the lowest lifetime of cost of ownership Tape Migration is an unpopular recurring cost No more Robotics to maintain Dust and humidity-controlled environments are expensive Low power consumption - 240 W per Petabyte Extended media life in decades – less recycling No Vendor Lock-in or licenses Local media purchasing policy, no expensive egress costs. High density storage, cold & green © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Archive Aspiration No 2. Faster Content Access Clients want faster access to archive Content for Fast turnaround television productions without breaking the bank buying massive Tier 2 Storage Volumes Restore times in seconds and Minutes, rather than hours for heavily contended Tape libraries Lower media contention delays. Each disk to be an independently managed reader. No restore caching, and random content access withing each individual disk No more waiting for cloud egress which can commonly be 5-10 hours for affordable service levels. © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Archive Aspiration No 3. Low Carbon Footprint The lowest carbon footprint achievable Dramatically lower power consumption due to fully spun down disks, Less air conditioning load. Less disks to be replaced and recycled. No special environmental provisions for dust extraction or humidity control © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Archive Aspirations No 4. Total Content Security An archiving environment with user selectable security for irreplaceable and strategically valuable content Geographically Distributed Systems across the enterprise between the premises and a Data Center Removable Media for Vault storage with Airgap security Eliminate the risk of content loss resulting from concurrent Disk and long rebuild times. Direct Read from externalised disks Disk with Windows, Mac and Linux Computers Hybrid Conﬁgurations with Cloud for copy of last resort © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Archive Challenges & Principal Technologies

ALTO Position in the Storage Hierarchy © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential. Media Asset Management Tier 1 RAID & FLASH Production & Distribution Workflow Tier 2 CLUSTER, GRID, OBJECT Tier 3 ALTO & TAPE ON PREM Tier 4 TAPE IN CLOUD

The Challenges with using Robotic Tape Libraries Low-density storage with a large physical footprint . High capital outlay to build a clean and humidity-controlled environment with specialist project disciplines. Expensive HSM and Archive Management requirements to optimise Tape to Drive Contention, which is the major issue. Unmanaged contention in Cloud Tape limits suitability for Archiving. First come ﬁrst served principal results in 5-10 HOURS latency! Studies have indicated Tape Libraries have the highest lifetime cost due to mandatory Tape Migration, Service costs, Environmental and other hidden costs. LTO Tape duopoly has already caused supply issues. Tape Drive monopoly could pose a future threat. © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

The Challenges with Optical Disk Archives Derived from Consumer technology, Highly Mechanical Juke-box approach with limited capacity and throughput Low storage density and limited Read / Write speed Made this format unsuitable for demanding applications at scale. 5 TB Cartridge capacity lagged behind 20 TB HDD capacities Passable for SD & HD, unsuitable for 2K & 4K content Linear writing principal did not allow space recovery on media. Essentially a write-once format Single Vendor Format. Withdrawn in 2023 © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Cloud Archives Cloud is a Location not a Technology High Performance Cloud is simply Object storage at a secure location. Typically, suitable for transactional data and collaborative media production - but unaffordable for Archive Volumes. Low-cost offerings are based on unmanaged Tape Libraries with 5 -10 hours restore latency. Suitable for “Copy of Last Resort” for disaster recovery but not for Archives A Hybrid system combining ALTO for fast On-Prem access with Cloud Tape tiers for Disaster Recover gives a good combination of Performance, Cost and Security “De-Clouding” is easy and affordable with an ALTO Hybrid. Risk of total loss if annual Cloud Budget is not totally secure © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Spinning Disk Storage Vulnerable OR Expensive Single Parity RAID systems are cheap and protect against a single disk failure. The disk rebuild time is proportional to the disk capacity and operational demand. For 20 TB Disks it can 4 weeks or more. A second concurrent failure will result in catastrophic loss. Dual Parity RAID systems are more expensive to own and operate, they protect against a second concurrent disk failure, but a third concurrent failure will result in catastrophic loss With high-capacity disks there is an unacceptable risk of loss with RAID. Spinning disk storage is also vulnerable to Malware © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Higher Security increases Outlay & Lifetime Cost. Duplicated RAID System offer good protection, but at twice the capital cost, twice the power consumption and with many more disks to wear out and replace. Clustered Storage offers good protection, but again with a much higher capital outlay, higher power consumption, and more disk drives to replace – SLA may leverage high disk cost. No SLA usually means no disks. Object Storage increasingly used for on-premise private clouds. Secure but with high capital cost and running cost. Typically, too expensive for Film & TV Archives © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

Reverse the Cost vs Security trend with Replication - ALTO is a Massive Array of Independent Disks. No RAID. - Files are written Sequentially on disk pairs, not Segmented. - Disks started & stopped INDIVIDUALLY to Read & Write - Only disks which are Reading or Writing are spinning - Massive power reduction, Consuming less than 210 Watts per PB - Disk lifetime in Decades… ﬁfteen years and counting! - No Migration, no Compaction, no Fragmentation - Designed for Distributed Systems and Hybrids - Disks may be externalized to Shelf for Vault Storage - Direct File Restore from a Vault Disk with any computer © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

ALTO-III Characteristics - An “IT solution”, simple to integrate, no special environment - Disks are sealed and ﬁlled with inert Helium 24 TB Today - 10 -15 years proven disk life, Compatible with SSD - Systems may be Geographically distributed for DR - Un-mounted disks provide Virtual Airgap Security from Malware - No proprietary technology, Linux platform - No proprietary File System – uses EXT- 4 - No vendor lock-in or media obsolescence - Local Disk purchasing, Any size disk from any Vendor - Buy and add disks as needed without Migration - Delete ﬁles and recover storage space without compaction © Disk Archive Corporation 2021 All Rights Reserved. Commercially Confidential.

© Disk Archive Corporation 2019 All Rights Reserved. Commercially Confidential. ALTO The Alternative to LTO ALTO is a scalable Cold Storage solution for infrequently-accessed media storage optimised for the needs of Television and Film production, preservation and archival. The ALTO Technology is equally applicable for any structured or unstructured data, adapted at the Application Layer for best-of-breed solutions ALTO replaces outdated Data Tape Libraries and replaces or complements Optical or Cloud offerings in open, standards-based systems with the lowest lifetime cost of ownership, superior performance and outstanding environmental credentials. ALTO – The Alternative to LTO

Cold Storage Architecture implemented on High Capacity Commodity Disks

Data Replication and Removable Media The ALTO Principle, MAID-III writes non-segmented data files with user-selectable Replication The disk drives are individually managed in software, allowing disks to be spun-up and spun-down on Read / Write demand massively reducing power consumption and extending disk lifetime for the most cost-effective long-term storage with the lowest carbon footprint The Replication technique offers user-configurable security, removable media and flexibility to create hybrid archives, distributed archives and vault storage.

Replication Factor 2:1 Replication by default Replication factor 2 is the default replication for an Archive Two non-segmented copies of every file are created and stored on different Disks for protection. The two disks may be in the same enclosure, where media preservation is the primary concern, or in separate enclosures for additional resilience and high-availability. Since MAID-III uses Replication not RAID, the file Replicas can be in geographically separated locations, or one Replica can be on the shelf in a vault Individual disks can be opened for forensic file recovery off the disks with a standard Windows, Mac or Linux Computer

Make More Replicas for for Additional Security Replication factor 3 offers a third copy of your media for even higher security. This can apply to all of the Archive or selectively to content of strategic value ALTO systems can be distributed between different premises or Data centres to create a Disaster Recovery Archive, or the third Replica may be in a vault for “Airgap Security” Media may be restored from the Vault at ﬁle-level to eliminate the risk of transporting disks. Open the disk at the Vault site and send the ﬁle over a network instead

Make Single Replicas… 1:1 Replication for Hybrids Replication Factor 1 is used to create Hybrid Archives with other Replication-based systems including Tape or Optical storage, to eliminate Technology obsolescence and vendor Lock-In Hybrid solutions with ALTO on the Premises & a second replica in a Cloud provides a combination of affordable high performance, with outstanding DR capabilities Reducing the Replication Factor to 1 allows all, or just part of the ALTO to be used as a highly cost-effective backup for other storage Devices

Off-the-Shelf Disks. Any Size any Vendor any Slot, any Time ALTO uses commodity disk drives that can be purchased locally from suppliers or online and added in batches to provide additional capacity when needed as an operating expense. Buying disks locally in batches allows users to take advantage of falling disk prices and increasing disk capacities over time, with no vendor lock-in. Disk capacity is increasing, driven by economies of scale in social media storage. Price per currently around $15 per TB Use multiple disk vendors, disk types, and performance to increase diversity and optimise performance against cost.

The ALTO Product Family

ALTO-III System Enclosure A single ALTO III System Enclosure holds up to 60 disk drives. 1440 TB using 24TB disks in a 4U unit and provides an increment of throughput. Linux Operating System – no Windows Licenses Linux File System - EXT-4 – no vendor lock-in Includes the low-level software to drive the ALTO System and up to 10 Expansions. Front panel LCD screen for admin and monitoring. Adding more ALTO System Enclosures increases the throughput as well as capacity and availability

ALTO-EX 60 Expansion The ALTO Expansion Chassis units scale a single ALTO System Enclosure into a Node with 660 disk slots Each ALTO Expansion Chassis adds an extra 60 slots. 1440 TB with 24TB disks, Up to 10 Expansions per ALTO Node, each with up to 15 PB High density cold storage Scalable to 200 PB and over

ALTO-SX Compact Archive A single ALTO-SX System Enclosure holds up to 24 disk drives. 576 TB using 24TB disks in a 4U unit. Short Enclosure and Front-Loading Disks for applications where Rack Depth is limited, and Rack Aisles are narrow Up to 4 x 24 Slot Expansions to create a 2.8 PB node. Centos OS & EXT- 4 File System – no Windows Licenses System Software & API are fully compatible with ALTO-III Disk Drives are fully Compatible with ALTO-III Disk Externalisation & Vault Management options available

ALTO-EX 24 Expansion Each ALTO-EX 24 Expansion Chassis adds 24 front-loading Disk Drive Slots Any size disk from any vendor listed on the DAC AVL may be used in any slot at any time An SX Expansion fully populated with 22TB disks will add 576 TB Up to 4 Expansions can be added to an ALTO-SX to provide up to 2.8 PB high density cold storage May be used on Prem and managed in a Hybrid Conﬁguration with low-cost Long Latency Cloud Tier to provide the Copy of Last Resort for a 2.8 PM Fully Redundant & Distributed Archive, supporting Partial File Restore

ALTO Manager ALTO Manager software provides monitoring and control of the disks in ALTO units. It drives the LCD on the front of an ALTO III head and provides a web server, making this monitoring and control functionality available for local and remote system administrators ALTOCare Support provides two years inclusive Hardware, Software and Online Operational Support with every ALTO system Optional unlimited renewals. No mandatory Technology Refresh or Media Replacement schedule. No Migration & Fragmentation free

ALTO Vault Storage With Shelf Management ALTO can manage externalized Replica disks held in a physical vault. Use ALTO Connect software to QR Code label externalised disks, and physical shelves then the ALTO Connect Database will track the X-Y coordinates of disks on the shelves down to ﬁle level Use the tablet-based Scanner to verify the content on a disk before restoring the disk to an ALTO Optionally restore just the needed ﬁles with an external computer to avoid physical transportation from a remote warehouse.

Integration Options for Best of Breed Solutions ALTO API – Offered free of charge to specialist Application Partners ALTO has a low-level API Integration with many MAM Applications and Customer in-house software developments ALTO Protocol Emulation - Middleware Emulation pseudo-standard Protocol for Archive Management ALTO Connect Virtual File System – An embeddable Virtual File System which presents ALTO as a CIFS / SMB File System, offering a simple way to Push and Pull Media Files to the Archive. Using one of these three approaches allows ALTO to be integrated with more than 50 enterprise class applications for best of breed solutions with Escrow options for total future-prooﬁng

Application Partnerships for Media Platforms Best of breed solutions ensure that the Archive is not locked in a closed system 60+ World class Technology Partners have proven Asset Management, Production Management, Video Management and Backup Management applications deployed with DAC. Standards based approach offers the widest range of Archive applications Partial File Restore Options avoid the need to move massive ﬁles over the Media Network

ALTO Application Partners 2023

Scalable & Modular Archive Conﬁgurations. In Film; Television; Legal Evidence and IT Backup Markets.

ALTO Solution for Content Preservation & High Availability 38

ALTO Transitional Solution from LTO Tape to Disk & Hybrid 39

ALTO Hybrid Solution with Object Storage & S3 Cloud 40

Use Case 1. Rwanda Broadcasting Agency

RBA – Rwanda Broadcasting A 2 Petabyte Platform to preserve a National Cultural Archive Ingest “as a service” over 2-3 years period Budget made allowance for continued use of the Archive Platform and Media Asset Management system to remain for browsing, searching & restoring as well as archiving newly produced content Two Replicas (copies) of every ﬁle are created on disk pairs for protection. 10 TB Disks – the biggest at the time! Future option to externalize one disk from each pair to a secure Vault, creating free space for more disks The Ingest process was managed by EMAM based on ALTO Connect CIFS / SMB VFS

Use Case 2. National Film Archive of Thailand

National Film Archive of Thailand Thailand has a very proud Film Heritage and a strong focus on preservation. This ALTO system features an ultra-high security scalable approach. This 1.5 PB Platform is designed to expand over time to house the Royal Heritage Collection in Thailand The Digitised Films and other Content are Triple Replicated in three ALTO Nodes and Distributed in two geographically separated locations for Disaster Recovery. The EMAM Media Asset Management system integrates with the ALTO Cold Storage using ALTO Connect, a Virtual CIFS / SMB File System hosted on dual IT Servers with automatic failover

Use Case 3. TV2 Denmark, National Television Content Archive

TV2 Denmark A 30 Petabyte Platform to preserve a TV Channel Archive based on ALTO and DIVA Content was most recently stored on T10K Data Tapes which have been discontinued by the manufacturer, used in a Robotic Tape Library, also end-of-life meant there was an urgent need to migrate to a futureproof storage strategy Storage policy one Replica of every ﬁle in the TV Station on ALTO for fast access and no Egress Cost with a Second Replica in a remote Tape Cloud as the Copy of Last Resort for Disaster Recovery. There is no Egress cost with all restores coming from ALTO. Cloud ingress and at-rest costs are affordable Disaster Recovery layer

Use Case 4. Radio & Television Malaysia Media City Project

RTM Media City A 20 Petabyte fully redundant, high availability Platform to preserve Legacy and newly produced content at 2K Resolution for a new Media City project in Kuala Lumpur The house format is AVC Ultra 4:4:4:4 at 220 Mbit/sec 5 x Primary ALTO nodes and 5 x Secondary ALTO nodes provide bandwidth for 20 Concurrent Ingest or 30 concurrent Archive Restore operations. Ingest and Archive Management will use Blue Lucy Media BLM with API Integration supporting Partial File Restore

Use Case 5. Super-Scale National Film Archive

National Film Archive Currently with 80+ PB storage, and planned expansion to 100 PB, the archive is based on a single Replica of each File. The system relies on ALTOs greatly extended disk life where we have real world examples of original disk drives still functional after10-15 years without replacement Recognising that failures can occur, even if rarely the Organisation retains the original Films which can be re-scanned if necessary to recover from a disk failure. A second Disk Replica is planned for the future, but the priority was on creating the ﬁrst Digital Replica The system took a Factory Approach” using multiple scanners, working 16 hours per day, managed by Custom Software developed in house using the ALTO API

Use Case 6. Saudi Arabia Oil Company Media Archive

Saudi Aramco Media Archive Saudi Aramco has an extensive Archive of Cultural content including Film Television and Images which contribute to new productions as well as legacy preservation Initially commissioned in 2016 as a centralised resource for the Aramco Media center, the system was expanded in 2021 to create a Geographically Distributed Disaster Recovery capability The system is used to archive a wide variety of Television and Digital Film Formats including ARRI and RED formats managed by a Media Alliance MediaXplorer MAM and PAM system

Use Case 7. Distributed Deep Archive For a Military Customer

Deep Archive for Military IT Backup. ALTO can be used for any form of Data Archive, not just unstructured data for Film Television and Media. The Platform is adapted to speciﬁc user requirements at the Application Layer. Application software managing ALTO for IT Backup includes well known solutions from Veeam, Commvault and Retrospect In this use case, the customer, an International Military wanted to replace ageing LTO Tape Archives ALTO in a Headquarters location and 5 Regional Locations with a modern Replication-based Cold Storage solution, with Fully Redundant Hardware and Software

Use Case 8. High Court Evidence Archive with Long Term Retention

A 14 PB Fully Redundant High Court Evidence Archive. Increasingly Video, Audio and Image Evidence is a core element Legal cases. This evidence will often include high-deﬁnition television courtroom coverage, which must be preserved retained and easily accessible for many years In conventional LTO Tape Archives, the storage media must be regularly migrated to avoid obsolescence. The archive itself is very low density and must be housed in an ultra-clean, dust free, humidity-controlled environment and incorporates with an extremely expensive Management layer to manage linear tape storage and a high support overhead for complex robotics. The world's largest High Court Archives are turning to ALTO for a secure, affordable, maintenance free Archive Platform with the lowest Lifetime Cost and Lowest Carbon footprint

Thank You Alan Hoggarth [email protected] www.diskarchive.com

Disk Archive - IT Press Tour 52 Dec. 2023 Madrid

Disk Archive - IT Press Tour 52 Dec. 2023 Madrid

More Decks by The IT Press Tour

Other Decks in Technology

Featured

Transcript