Backing up with Laravel (unicorn)

Transcript

1. Backing up 

   with 

   Laravel
   

   View Slide

2. About me
   Freek Van der Herten
   Partner & developer at Spatie
   @freekmurze
   https://murze.be
   

   View Slide

3. About me
   phpantwerp.be
   Dries Vints
   @driesvints
   Frederick Vanbrabant
   @TheEdonian
   

   View Slide

4. Since 2003
   https://spatie.be
   Websites, applications and webshops
   Team
   4 developers, 1 manager
   Specialization
   Laravel development
   

   View Slide

5. OPEN SOURCE SOFTWARE
   

   View Slide

6. Open source software
   We use a lot of it
   Nginx, Laravel, Ubuntu, Elasticsearch, Memcached, Redis,
   Beanstalkd, MySQL, Composer, Glide, Valet, Flysystem, Fractal,
   Carbon, PHPUnit, NPM, React, Vue, Java, SSH, Sequel Pro,
   BackupPC …
   Basically everything listed in composer.json and package.json
   

   View Slide

7. ± 100 packages on Packagist
   Laravel, Framework agnostic PHP, JavaScript
   + 3 000 000 downloads in total, growing 350 000 a month
   We create a lot of it
   Open source software
   

   View Slide

8. http://github-awards.com/users?type=world&language=php
   Open source software
   

   View Slide

9. Learning
   Forced to write documentation and tests
   Showing the quality of our work
   Eat our own dogfood
   Attracting new clients
   Benefits
   Open source software
   

   View Slide

10. Fractalistic
    Uptime monitor
    Laravel Analytics
    Laravel permission
    Laravel Responsecache
    Open source software
    

    View Slide

11. Fractalistic
    $books = [

    ['id' => 1, 'title' => 'Hogfather', 'characters' => [...]],

    ['id' => 2, 'title' => 'Game Of Kill Everyone', 'characters' => [...]]

    ];

    

    $manager = new Manager();

    

    $resource = new Collection($books, new BookTransformer());

    

    $manager->parseIncludes('characters');

    

    $manager->createData($resource)->toArray();
    

    View Slide

12. Fractal::create()

    ->collection($books)

    ->transformWith(new BookTransformer())

    ->includeCharacters()

    ->toArray();

    Fractalistic
    

    View Slide

13. Uptime monitor
    

    View Slide

14. Retrieve data from Google Analytics
    Laravel Analytics
    Analytics::fetchMostVisitedPages();
    Analytics::fetchVisitorsAndPageViews();
    

    Analytics::getTopReferrers();
    

    View Slide

15. Associate users with roles and permissions
    Laravel Permission
    $role = Role::create(['name' => 'writer']);

    $role->givePermissionTo('edit articles');

    $user->assignRole('writer');

    $user->can('edit articles’);
    

    View Slide

16. Speed up your application 

    by caching the entire response
    Laravel Responsecache
    RESPONSE CACHE
    REQUESTS
    APP
    1
    2
    

    View Slide

17. https://spatie.be/opensource/laravel
    

    View Slide

18. https://spatie.be/opensource/php
    

    View Slide

19. Not free!
    

    View Slide

20. POSTCARDWARE
    

    View Slide

21. Spatie

    Samberstraat 69D
    2060 Antwerp

    Belgium
    

    View Slide

22. LET’S TALK BACKUPS
    

    View Slide

23. There is no 

    one size fits all solution
    for backups
    

    View Slide

24. Small sites
    Shared hosting
    Backups by hosting provider
    Did not do anything regarding backups ourselves
    Hosting
    Up until a few year ago
    

    View Slide

25. Bigger applications
    Learned basic server management through
    resources like serversforhackers.com
    Digital Ocean
    Forge / Ansible
    Hosting
    How we are currently doing it
    

    View Slide

26. HORROR STORIES
    

    View Slide

27. Horror Story #1
    Earlier today, our Cloud Operations team was alerted to some
    performance issues affecting the physical server that hosts your droplet.
    … The damage was serious enough that this droplet was lost and not able
    to be restored.
    One fine morning I got this mail from Digital Ocean
    Full story: https://murze.be/2016/02/today-digitalocean-lost-our-entire-server/
    

    View Slide

28. Horror Story #2
    This message is a follow-up to our previous emails. Despite the very best
    efforts of our engineering and datacenter operations teams, unfortunately
    the host machine that your cloud server resides on was not able to be
    recovered. At this point, there is complete data loss on the server.
    Horror can occur everywhere. A mail from Rackspace:
    Source: https://news.ycombinator.com/item?id=3982571
    

    View Slide

29. Ok, I’ll just use

    the backup service 

    from my cloud provider
    

    View Slide

30. Horror Story #3
    

    View Slide

31. Horror Story #3
    

    View Slide

32. Relying on DO backups 

    is not enough!
    Only weekly snapshots
    Potentially a lot of data could get lost
    All the backups in the same datacenter
    If data center is down backups can’t be accessed or restored
    

    View Slide

33. SOLUTIONS
    

    View Slide

34. DIY: A bash script
    Hosted service: https://ottomatik.io/
    Open source: BackupPC, Bacula
    Solutions
    Do not put your eggs in one basket
    

    View Slide

35. https://serversforhackers.com/video/backup-to-s3 interface
    Dumps the database with mysqldump
    Uploads dump + files using was cli tool to S3
    Use cron to frequently run script
    Downside: no notifications when something goes wrong

    

    A bash script
    

    View Slide

36. Backups as a service
    Not free
    Easy to use interface
    Bring your own storage
    Ottomatik.io
    

    View Slide

37. Open source software
    Installed onto a control server
    SSH’s into servers to be backed up
    Uses hard links to save disk space
    Some system administration knowledge is required to setup up
    BackupPC SERVERS
    BackupPC
    SSH
    

    View Slide

38. LARAVEL BACKUP
    

    View Slide

39. Backup files and databases to one or more filesystems
    Cleanup old backups
    Sending notifications
    Easily installed into any Laravel application
    Laravel Backup
    

    View Slide

40. View Slide

41. Backup files & databases
    FILES
    DATABASE
    SERVER
    S3
    DROPBOX
    DB DUMP
    .ZIP
    .SQL
    

    View Slide

42. Backups use a lot of storage
    Artisan task to clean up old backups
    Fully configurable
    Will never delete the youngest backup
    Grandfather-father-son rotation scheme
    Cleanup old backups
    

    View Slide

43. Grandfather-father-son
    rotation scheme
    FEW TIMES
    A DAY
    DAILY (16)
    WEEKLY (8)
    MONTHLY (4)
    YEARLY(2)
    KEEP

    EVERYTHING
    KEEP DAILY BACKUPS
    KEEP WEEKLY BACKUPS
    KEEP MONTHLY BACKUPS
    KEEP YEARLY BACKUPS
    

    View Slide

44. Detects when no backups were made in a certain amount of days
    Detects if there is too much storage used
    Fully configurable
    Monitor backups
    

    View Slide

45. On (un)successful events (backup failed, youngest backup is too
    old, backup uses too much storage …)
    Fully configurable
    Mail, Slack, Telegram, Pushover, …
    https://github.com/laravel-notification-channels
    Sending notifications
    

    View Slide

46. DEMO
    

    View Slide

47. Your Laravel app may break
    Server may be down
    Install the package in a Laravel app on a separate server
    Monitor the backups of all your apps from there
    Monitoring best practices
    

    View Slide

48. Monitoring best practices
    APPS
    BACKUPS
    MONITOR
    

    View Slide

49. Install a Laravel app on the same server
    Point the configuration the the database/directories you which to
    backup
    Profit!
    Backing up a non Laravel app
    

    View Slide

50. Backup to multiple filesystems
    Get notified when something goes wrong
    Cleans up older backups
    Easy installation
    Cost: one postcard
    Benefits
    

    View Slide

51. Drawbacks
    Only use this for small to medium sized apps
    Can consume quite a bit a of disk space while backing up
    Application has credentials to access the backups
    No restore options
    

    View Slide

52. https://docs.spatie.be/laravel-backup
    

    View Slide

53. Version 4
    PHP 7 / Laravel 5.3, 5.4
    Laravel 5.3’s native notification capabilities
    Version 3:
    PHP 5.6 / Laravel 5.1, 5.2

    Custom notification system: Mail, Log, Slack, Pushover,
    Telegram
    Requirements
    

    View Slide

54. Do not rely solely on backups of your service provider
    Take care of backups yourself as well
    Many options available both free and paid
    Laravel Backup can help you out (cost: one postcard)
    Summary
    

    View Slide

55. Thank you!
    https://joind.in/talk/42b1d
    https://speakerdeck.com/freekmurze/backing-up-with-laravel-unicorn
    https://spatie.be/opensource
    https://murze.be
    https://murze.be/newsletter/
    

    View Slide

56. QUESTIONS?
    

    View Slide