Backing up with Laravel (Tokyo)

by Freek Van der Herten

Slide 1

Slide 1 text

Backing up   with   Laravel

Slide 2

Slide 2 text

こんばんは

Slide 3

Slide 3 text

About me Freek Van der Herten Partner & developer at SPATIE @freekmurze https://murze.be

Slide 4

Slide 4 text

About me phpantwerp.be Dries Vints @driesvints Frederick Vanbrabant @TheEdonian

Slide 5

Slide 5 text

Since 2003 Websites, applications and webshops Team 4 developers, 1 manager Specialization Laravel development

Slide 6

Slide 6 text

OPEN SOURCE SOFTWARE

Slide 7

Slide 7 text

Open source software We use a lot of it Nginx, Laravel, Ubuntu, Elasticsearch, Memcached, Redis, Beanstalkd, MySQL, Composer, Glide, Homestead, Flysystem, Fractal, Carbon, PHPUnit, NPM, React, jQuery, Java, SSH, Sequel Pro, BackupPC … Basically everything listed in composer.json and package.json

Slide 8

Slide 8 text

± 80 packages on Packagist + 900 000 downloads A lot of beneﬁts: learning, forced to write documentation and tests, showing the quality of our work, eat our own dogfood We create a lot of it Open source software

Slide 9

Slide 9 text

Laravel Permission Laravel Analytics Laravel Reponsecache Laravel Medialibrary Laravel Fractal Open source software

Slide 10

Slide 10 text

Associate users with roles and permissions Laravel Permission $role = Role::create(['name' => 'writer']);  $role->givePermissionTo('edit articles');  $user->assignRole('writer');  $user->can('edit articles’);

Slide 11

Slide 11 text

Retrieve data from Google Analytics Laravel Analytics Analytics::fetchMostVisitedPages(); Analytics::fetchVisitorsAndPageViews();   Analytics::getTopReferrers();

Slide 12

Slide 12 text

Speed up your application   by caching the entire response Laravel Responsecache RESPONSE CACHE REQUESTS APP 1 2

Slide 13

Slide 13 text

Associate ﬁles with Eloquent Models Laravel Medialibrary $newsItem->addMedia($file) ->toCollection('images');   $newsItem->getMedia('images')->first()->getUrl();   $newsItem->getMedia('images')->first()->getUrl('thumb');    $newsItem->addMedia($bigFile) ->toCollectionOnDisk('downloads', 's3');

Slide 14

Slide 14 text

Transformation layer for complex output Fractal $manager = new Manager();  $resource = new Collection($books, new BookTransformer());  $manager->parseIncludes('characters');  $manager->createData($resource)->toArray();

Slide 15

Slide 15 text

Easy integration with Fractal in Laravel Laravel Fractal fractal()  ->collection($books)  ->transformWith(new BookTransformer())  ->includeCharacters()  ->toArray();

Slide 16

Slide 16 text

https://spatie.be/opensource/laravel

Slide 17

Slide 17 text

Not free!

Slide 18

Slide 18 text

POSTCARDWARE

Slide 19

Slide 19 text

Spatie  Samberstraat 69D 2060 Antwerp  Belgium

Slide 20

Slide 20 text

LET’S TALK BACKUPS

Slide 21

Slide 21 text

There is no   one size ﬁts all solution for backups

Slide 22

Slide 22 text

Small sites Shared hosting Backups by hosting provider Did not do anything regarding backups ourselves Hosting Up until a few year ago

Slide 23

Slide 23 text

Bigger applications Learned basic server management through resources like serversforhackers.com Digital Ocean Forge / Ansible Hosting How we are currently doing it

Slide 24

Slide 24 text

HORROR STORIES

Slide 25

Slide 25 text

Horror Story #1 Earlier today, our Cloud Operations team was alerted to some performance issues aﬀecting the physical server that hosts your droplet. … The damage was serious enough that this droplet was lost and not able to be restored. One ﬁne morning I got this mail from Digital Ocean Full story: https://murze.be/2016/02/today-digitalocean-lost-our-entire-server/

Slide 26

Slide 26 text

Horror Story #2 This message is a follow-up to our previous emails. Despite the very best eﬀorts of our engineering and datacenter operations teams, unfortunately the host machine that your cloud server resides on was not able to be recovered. At this point, there is complete data loss on the server. Horror can occur everywhere. A mail from Rackspace: Source: https://news.ycombinator.com/item?id=3982571

Slide 27

Slide 27 text

Ok, I’ll just use  the backup service   from my cloud provider

Slide 28

Slide 28 text

Horror Story #3

Slide 29

Slide 29 text

Horror Story #3

Slide 30

Slide 30 text

Relying on DO backups   is not enough! Only weekly snapshots Potentially a lot of data could get lost All the backups in the same datacenter If Digital Ocean is down backups can’t be accessed

Slide 31

Slide 31 text

SOLUTIONS

Slide 32

Slide 32 text

DIY: A bash script Hosted service: https://ottomatik.io/ Open source: BackupPC, Bacula Solutions Do not put your eggs in one basket

Slide 33

Slide 33 text

https://serversforhackers.com/video/backup-to-s3 interface Dumps the database with mysqldump Uploads dump + ﬁles using was cli tool to S3 Use cron to frequently run script Downside: no notiﬁcations when something goes wrong    A bash script

Slide 34

Slide 34 text

Backups as a service Not free Install Ottomatik Server Agent on every server Easy to use interface Bring your own storage Ottomatik.io

Slide 35

Slide 35 text

Open source software Installed onto a control server SSH’s into servers to be backed up Uses hard links to save disk space Some system administration knowledge is required to setup up BackupPC SERVERS BackupPC SSH

Slide 36

Slide 36 text

LARAVEL BACKUP

Slide 37

Slide 37 text

Backup files and databases to one or more filesystems Cleanup old backups Sending notifications Easily installed into any Laravel application Laravel Backup

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Backup ﬁles & databases FILES DATABASE SERVER S3 DROPBOX DB DUMP .ZIP .SQL

Slide 40

Slide 40 text

Backups use a lot of storage Artisan task to clean up old backups Fully conﬁgurable Will never delete the youngest backup Grandfather-father-son rotation scheme Cleanup old backups

Slide 41

Slide 41 text

Grandfather-father-son rotation scheme FEW TIMES A DAY DAILY (16) WEEKLY (8) MONTHLY (4) YEARLY(2) KEEP  EVERYTHING KEEP DAILY BACKUPS KEEP WEEKLY BACKUPS KEEP MONTHLY BACKUPS KEEP YEARLY BACKUPS

Slide 42

Slide 42 text

Detects when no backups were made in a certain amount of days Detects if there is too much storage used Fully conﬁgurable Monitor backups

Slide 43

Slide 43 text

On (un)successful events (backup failed, youngest backup is too old, backup uses too much storage …) Mail, Slack, Telegram, Pushover Fully conﬁgurable Easy to extend Sending notiﬁcations

Slide 44

Slide 44 text

DEMO

Slide 45

Slide 45 text

Your Laravel app may break Server may be down Install the package in a Laravel app on a separate server Monitor the backups of all your apps from there Monitoring best practices

Slide 46

Slide 46 text

Monitoring best practices APPS BACKUPS MONITOR

Slide 47

Slide 47 text

Install a Laravel app on the same server Point the conﬁguration the the database/directories you which to backup Proﬁt! Backing up a non Laravel app

Slide 48

Slide 48 text

Backup to multiple filesystems Get notified when something goes wrong Cleans up older backups Easy installation Cost: one postcard Benefits

Slide 49

Slide 49 text

Drawbacks Only use this for small to medium sized apps Can consume quite a bit a of disk space while backing up Application has credentials to access the backups No restore options

Slide 50

Slide 50 text

https://docs.spatie.be/laravel-backup

Slide 51

Slide 51 text

New major release: v4 Use Laravel 5.3’s native notiﬁcation capabilities Easier passing of arguments to the db dumpers PHP 7 ETA: somewhere in october Future plans

Slide 52

Slide 52 text

Do not rely solely on backups of your service provider Take care of backups yourself as well Many options available both free and paid Laravel Backup can help you out (cost: one postcard) Summary

Slide 53

Slide 53 text

QUESTIONS?

Slide 54

Slide 54 text

Thank you! https://speakerdeck.com/freekmurze/backing-up-with-laravel-tokyo https://murze.be https://spatie.be/opensource