• Archive • Migration DATA PROTECTION for ENTERPRISE • Backup • Archive • Migration DATA PROTECTION for SMB • Backup Appliances • New Gen AntiMalware 2010 AI LAB for STORAGE MANAGEMENT 2019
AT E E V E N T S C O R P O R AT E E V E N T S B U S I N E S S D E V. B U S I N E S S D E V. Creation in Paris 1991 Acquisition of Storactive: CDP pioneer (USA) 2006 2011 ATEMPO acquired by ASG (USA) Aug. 2017 Aug. 2017 ATEMPO acquired by KICKSTART (FR) Oct. 2017 Oct. 2017 Acquisition of SYNERWAY(FR) 1992 2008 Tina V 1.0 Time Navigation Lina V 1.0 2019 2019 Miria 2019 Backstone V1.0 2018 2018 Opening of subsidiaries 3
Workstations MEDIUM & LARGE BUSINESS Software for: Physical, Virtual Servers and Workstations BRANCH OFFICE Hardware Software Hosting BRANCH OFFICE Hardware Software Hosting APPLIANCES LINA TINA ADA MEDIA MEDIA MEDIA Software to: Tier, Archive, Backup Media Assets MEDIA Software to: Tier, Archive, Backup Media Assets ENTERPRISE DATA PROTECTION ENTERPRISE DATA PROTECTION 8 Product Portfolio: Backup & Archive
data integrity and giving quick and easy archive access to end-users to search and retrieve productions from our collections.” -- Luisa Frate, COO and CFO, NFB 10 Years Partnership Renewed! 2009 2019 2029 14 The National Film Board of Canada(NFB) Needs • Preserve the cinematography legacy of Canada 13,000 films, +6000 h. of content • Needed the solution to be integrated with their homemade MAM • Make new and old movies of the NFB accessible to the broader audience • Automating archive audits to perform continuous data integrity checks of NFB’s long term digital archives • Moving large amounts of audiovisual data between geographically separate sites and proactively define best storage & movement strategies Benefits • 70 years of movie legacy preserved with a simple and open archiving system • Seamless integration with the NFB workflows to archive data and metadata “Atempo’s storage agnostic solutions have been key to successfully managing our archiving goals. Miria is reliable, scalable and flexible and ensures that our assets are completely safe for the long term.” Jimmy Fournier, Director of R&D, NFB
1 Zettabyte is one 1 million Petabytes or 1 billion Terabytes 61% Annual data growth from now until 2025 2025 175 2018 33 ZB Zetta bytes Unstructured data will account for more than 80% of the data collected by organizations
Industry • Energy • Security • Media & Entertainment • Health • Smart city • Government • Cloud providers DIGITAL TRANSFORMATION: DRIVING ALL SECTORS GENERATES BILLIONS OF FILES AND ZETTABYTES OF DATA Meeting The Challenge Of The Data Tsunami
lasting protection all from a single centralized platform Migrate very large data volumes and billions and files efficiently between heterogeneous storages and file systems Moving data where necessary with direct and shared access for remote teams while maintaining a high level of security Free up storage space on high performing primary storages and manage storage growth requirements Miria engine delivers performance and manages petabytes of data and billions of small files on site and in hybrid environments Backup, Archive, Move and Synchronize Unstructured Data at Scale 22
and Performance MIRIA Admin Interface MIRIA User Interface • Open on Storages and Platforms • Scale Out Scalability • Optimized Data Flow MIRIA Server MIRIA Data Movers
Data Movers perform • Backup/Archiving/Copy/Move • Multiple copies to different storages • Parallel data flows Data Movers for Performance and Scalability • Fail over • Load-balancing • Process Multithreading • Data path optimization
Metadata Users • Role based • Access & Delegation • LDAP Integration 26 Software Defined Architecture Manage Objects & Activity From A Central Console Storage • Manager & Containers • Media • Interface Data Movers • Data mover roles • Data mover pools Activity • Task management • Jobs monitoring • Reports, Capacity management
lasting protection all from a single centralized platform Migrate very large data volumes and billions and files efficiently between heterogeneous storages and file systems Moving data where necessary with direct and shared access for remote teams while maintaining a high level of security Free up storage space on high performing primary storages and manage storage growth requirements Miria engine delivers performance and manages petabytes of data and billions of small files on site and in hybrid environments 28 Introducing Miria for Migration
change while primary storage is in production Cannot have an outage Limited periods of time to perform data transfers Must limit the impact of migration on storage & network Challenges: • Migration solution has to support heterogeneous storages • Complexity of monitoring the changes and associated transfers • Maintaining ACLs & folders structure between platforms & shares 29 Challenges & Constrains Petabyte-Scale Data Migration
data from NAS Cluster to Qumulo • Approximately 20 million files • Minimum user downtime: last synchronization cannot exceed 12 hours • Limit performance scanning to avoid any issue on their production environment Technical Environment • Source: Cluster of ten nodes • Qumulo K144T • 2 Data Movers 32GB 2CPU(20cores) running on Windows 2016 • Having 10Gbe network connection on each DM for both source NAS reading and Qumulo writing operations QF2 CIFS data migration path 32 Leading Scale-Out NAS Cluster Miria for Migration USE CASE #1: F O R T U N E 5 0 0 T E C H N O L O G Y C O Confidential – Do not distribute
to not disturb production • Limit the number of parallel synchronization jobs to 4 • Creation of a job for: 1TB or 500,000 files Migration Performance • 6 source volumes were sequentially synchronized: • The total 1,25PB volume was scanned in 9h and sync’ed in 12 days (100TB/day); incremental sync duration was 30 minutes only! QF2 CIFS data migration path 33 Miria for Migration Leading Scale-Out NAS Cluster Windows 2016 MIRIA Server USE CASE #1: F O R T U N E 5 0 0 T E C H N O L O G Y C O Confidential – Do not distribute Confidential – Do not distribute Confidential – Do not distribute
from NAS Cluster to Qumulo • Approximately 970 million of files, 29 NFS shares (75% are small files <8kb) • Minimum user downtime • Limit performance scanning to avoid any issue on their NFS environment • Cutover planned 3-weeks after 1st synchronization Technical Environment • Source storage x410 of six nodes • Qumulo K144T Cluster • 4* Data Movers 32GB 2CPU (20cores) - Linux • Having a 10Gbe network connection on each DM for both source NAS reading and Qumulo writing operations NFS data migration path 34 MIRIA Server Linux QF2 Leading Scale-Out NAS Cluster USE CASE #2: F O R T U N E 5 0 0 D E F E N S E I N D U S T RY Confidential – Do not distribute
to 4 and then to 16 • Creation of a job for: 1024GB or 250,000 files Migration Performance • 10 source NAS volumes were simultaneously synchronized: • Daily synchronization performance: around 20 million files 35 Miria for Migration NFS data migration path MIRIA Server Linux QF2 Leading Scale-Out NAS Cluster USE CASE #2: F O R T U N E 5 0 0 D E F E N S E I N D U S T RY Confidential – Do not distribute
powerful NAS • Migrate large file volumes without stopping or impacting user production activity • Migrate NAS user shares and maintain user access rights after migration • Migrate millions of files, folders and links between 2 different storage platforms and respect data integrity Market Requirements Miria Benefits Miria for Migration Migrate files incrementally and automatically between different storage platforms Storages remain operational Very low production impact Automatic verification of the integrity and accessibility of migrated files Adjust performances through simple addition or removal of an Atempo Data Mover Complete solution comprising Software and Professional Services. Initial audit, constant monitoring and final checks Possibility to continue protecting the new storage with the same solutions (Miria for Backup) M AR K E T R E Q U I R E M E N T S M I R I A B E N E F I T S Requirements & Benefits 36
lasting protection all from a single centralized platform Migrate very large data volumes and billions and files efficiently between heterogeneous storages and file systems Moving data where necessary with direct and shared access for remote teams while maintaining a high level of security Free up storage space on high performing primary storages and manage storage growth requirements Miria engine delivers performance and manages petabytes of data and billions of small files on site and in hybrid environments Miria For Backup High Performance 38
• Backup engines were designed for tape • Exploding backup windows • Difficulties handling billions of files • Transfer speeds heavily impacted by the number of objects • Low API implementation levels limit the possibilities for outside interaction • Proprietary storage formats • Migration between different storages is complex and costly • No simple view of all data assets • Complicated access right and role management for moved data • Impossible to achieve the mantra of hybrid infrastructures: “Your data when you need it, where you need it" • Hidden costs impact the true TCO Historical leaders in the data movement and data protection market are finding it increasingly difficult to manage growing data levels with the risk of cornering their clients into technology which is less and less well adapted.
vendor lock-in Support issues New projects lacking security endangering the business License costs and functional evolutions Complicated and slow compliance with tighter European legislation Require I/O and performance optimization. - restricted bandwidth use Confront the challenge of many new and modified files including numerous very small files Need to be able to migrate data simply between heterogeneous storages Handle complicated migration between difficult storages. Long and risky processes with file loss risks Restore to another storage FRUSTRATED CLIENTS MANAGING EVER-GROWING DATA VOLUMES AND FILE NUMBERS WITH TRADITIONAL BACKUP SOLUTIONS
storages • Powerful and reliable backups for hundreds of terabytes or even petabytes of unstructured data along with millions/billions of folders/files • Protect file storage where current incremental backups exceed the backup window • Rapid recovery of very large data volumes • Restore to another type of storage M AR K E T R E Q U I R E M E N T S Minimize the length of the backup window Perform regular and efficient backups with successive increments Recover a few files or complete volumes including to a different storage Adjust performances through simply adding or removal of an Atempo Data Mover Open backup and multi-storage solution (cloud, disk, object, tape). M I R I A B E N E F I T S Miria for Backup Requirements & Benefits 41
Leading university located in Switzerland 7 facilities 15,300 students from 187 countries 3,000 researchers Many academic disciplines : Medicine, Life, Sciences, Geosciences, Environment, Business, Humanities and Social Sciences and ... 53 sheep to mow the lawn 42
to find a solution • UNIL already tested all the EMC² solutions (Networker, Avamar, SourceOne) Customer environment • Research department had 2 replicated EMC Isilon NAS Storages with a capacity of 1PB • They needed a third copy on another storage to secure the data • Secondary storage was an EMC² ECS (Object Storage) Request for proposal • Miria succeeded the PoC • UNIL issued a Request For Proposal • Unexpectedly, a competitor won. However it failed the performance PoC Back to Miria • UNIL published an official note to select the second winner (Miria) • Miria succeeded the performance PoC and was installed in production 43 Project History Miria Use Case
Miria Use Case • 1 Miria Server • 6 Data Movers S E R V E R S • No NDMP • FastScan with ISILON integration • Scan & copy in parallel • Cross platform support (ACL) • Enhanced support for S3 • Incremental forever • Scalability to 4 PB • File integrity check • Roaming for throttling • Advanced Data Management
Architecture Miria Use Case Cluster ECS 8 Nodes 1 PB Replication Miria Server Windows Data Movers Linux Data Movers Isilon #1 18 Nodes 1 PB Isilon #2 10 Nodes 1 PB Incr. Backup from CIFS Incr. Backup from NFS Confidential – Do not distribute
lasting protection all from a single centralized platform Migrate very large data volumes and billions and files efficiently between heterogeneous storages and file systems Moving data where necessary with direct and shared access for remote teams while maintaining a high level of security Free up storage space on high performing primary storages and manage storage growth requirements Miria engine delivers performance and manages petabytes of data and billions of small files on site and in hybrid environments Miria for Backup DR – Business Continuity 48
Restore at original location or in another location • Restore Last Version & Restore on specific point of time • ACLs are retrieved • Data IO done by Data Mover • Retrieval Parallelization cross Data Movers GPFS 6 PB Isilon 2 PB 40GbE Infiniband 56Gb/s 10 PB GPFS MIRIA Server Confidential – Do not distribute
Disaster Primary Storage Volume User Access to Backup Target using NFS & CIFS protocol on Read / Write Volume Primary Storage issue fixed Incremental re-synchronization between Target Storage GPFS to Primary Storage using ADA Data Movers Primary Storage Volume is back Online Switch to a Regular Workflow Backup from Source NAS to GPFS Target GPFS 6 PB Isilon 2 PB 40GbE Infiniband 56Gb/s 10 PB GPFS MIRIA Server GPFS Protocol Node GPFS Protocol Node Confidential – Do not distribute
lasting protection all from a single centralized platform Migrate very large data volumes and billions and files efficiently between heterogeneous storages and file systems Moving data where necessary with direct and shared access for remote teams while maintaining a high level of security Free up storage space on high performing primary storages and manage storage growth requirements Miria engine delivers performance and manages petabytes of data and billions of small files on site and in hybrid environments Miria for Archiving 53
DiRAC, an integrated HPC-based computing facility for cosmology and particle physics modeling • 452 nodes & 12,656 cores of computing power • 20 PB of data by 2022 • 4 UK universities including Cambridge & Durham Edinburgh and Leicester Needs: • Archiving research projects to LTO7 and LTO8 tape for 100s of projects worldwide 54
lasting protection all from a single centralized platform Migrate very large data volumes and billions and files efficiently between heterogeneous storages and file systems Moving data where necessary with direct and shared access for remote teams while maintaining a high level of security Free up storage space on high performing primary storages and manage storage growth requirements Miria engine delivers performance and manages petabytes of data and billions of small files on site and in hybrid environments Miria for Data Moving 57
position data to match workflow and resource requirements • One copy site or many, with flexibility to change as required • Automated and manual data copy policies • Copy synchronization utilizes FastScan • Ability to enable disaster recovery • Full security control and as all data movements are logged • Ability to parallelize data synchronization across some or all Data Movers • Support copy modes; Echo, Subscribe (no FastScan), Contribute, Combine (no FastScan) • Any storage to storage, enables usage of dissimilar storages enabling heterogeneous workflows while preserving ACLs 58 Use Case: Copy On Going Replication – CLI / Task Need Company having teams in multiple distant locations each with local storages, and needing periodic synchronization of partial content of remote storage locally (for local access to important content). Primary B Primary A
position data to match workflow and resource requirements • Automated and manual data movement policies • Movement utilizing FastScan • Ability by policy to remove data on source or remote locations • Full security control and as all data movements are logged • Ability to parallelize data movement • Supported modes; Echo, Subscribe (no FastScan), Contribute, Combine (no FastScan) • Any storage to storage, enables usage of dissimilar storages enabling heterogeneous workflows while preserving ACLs 59 Use Case: Copy Data Placement – On Going – CLI / Task Need Company having remote team members with a need to centralize data and ensure all resources have latest updates, While maintaining content control and storage utilization in all locations. Primary B Primary A
Data Mover Auto-Tuning V1 • Lustre Fastscan Q 4 - 2 0 1 9 61 Miria 2019 First Miria Release With AI Impact AI LAB for STORAGE MANAGEMENT Confidential – Do not distribute
Backup-Achiving-Data Moving • Software solution available as perpetual and term license • Free of charge unlimited number of Data Mover • License scoped on the volume of the target destination storages • 8*5 Standard support or 24*7 • Solution starts at $17,500 • Miria for Migration • Software solution available as term license covering the duration of the migration • Packaged with Professional Services offering a turn-key solution • Solution starts at $10,000 for 100TB migrated within 1 month
for data growth which other solutions were unable to offer. Live and archived data for finished projects use the solution to perform secure data movement for completed projects using a simple drag and drop to a chosen storage target.” Sarah F. Architecte solution - UK Film Post- Production Arnaud Caréo Technical Director “I wanted a solution in which I could entrust all my customers’ data I researched the market and selected Atempo. The solution can automate all the features that I require and it is reassuring that Miria is used by so many American media and entertainment leaders.” Film Production Chuck D. Architecte stockage USA ”We tested several free and paying migration solutions and only Atempo enabled us to move our data reliably and respect our two major imperatives: zero impact of migration on production and the storage switch in under 4 hours.” “We have backed up our unstructured NAS data sets based on multi-OS sources while respecting ACLs. Thanks to ADA, our NFS and CIFS file systems are working in perfect harmony with S3- type storage.” Michel Ruffieux Resp. Stockages & Sauvegardes High Education & Research Our Clients Talk about Atempo Solutions 64
itpresstour
shift_evolve
rakus_dev
fossamagna
nari_ex
miyukki
genda
moongift
nttcom
bitkey
kaonavi
ynisqa1988
bcantrill
gr2m
ivargrimstad
szymonslowik
aleyda
rocio
mfonobong
minecr
denniskardys
iamctodd
rmw
mraible
