Containers @ Hyperspeed

Containers @ Hyperspeed

(talk given at container.camp london -- 2015-09-11)

Containers are awesome! But storing and moving them is painful! It's slow! Containers are huge! There's little deduplication! No cryptographic versioning! Images are lost! Worst, we must depend on other services -- or having fast connection to them...

No more! starship puts your containers on hyperspeed! Instead of old-school HTTP, starship uses shiny IPFS -- a new protocol, Git meets Bittorrent. With IPFS, starship (a) deduplicates all data between containers, (b) uses p2p download from fastest hosts, and (c) can limit connections to private cloud(s) or local networks.

A9670c143716320893863524a0efbaff?s=128

Juan Batiz-Benet

September 11, 2015
Tweet

Transcript

  1. 2.
  2. 3.

    IP UDP TCP SCTP Ethernet CSMA WIFI TP Coax Satellite

    Fiber Radio PPP HTTP SMTP FTP NFS Skype Pigeons Chat
  3. 4.
  4. 5.
  5. 6.
  6. 11.
  7. 25.
  8. 26.
  9. 27.
  10. 28.
  11. 29.
  12. 30.
  13. 31.
  14. 32.
  15. 33.
  16. 34.
  17. 35.
  18. 36.
  19. 37.
  20. 38.
  21. 39.
  22. 40.
  23. 41.
  24. 42.
  25. 43.
  26. 44.
  27. 45.
  28. 46.
  29. 47.
  30. 48.
  31. 49.
  32. 50.
  33. 51.
  34. 52.
  35. 53.
  36. 54.
  37. 55.
  38. 56.
  39. 58.
  40. 59.
  41. 63.
  42. 66.
  43. 67.
  44. 68.
  45. 69.
  46. 70.
  47. 71.
  48. 74.
  49. 79.
  50. 81.

    routing network exchange merkledag naming applications IPNS DNS Namecoin Bitswap

    HTTP BitTorrent Kad DHT Chord mDNS Gossip Delegated FTP TOR QUIC TOR TCP uTP WebRTC WebSockets Git Bitcoin VLC Etherpad Chat I2P I2P CJDNS UDT
  51. 82.
  52. 83.
  53. 84.
  54. 85.
  55. 86.
  56. 87.
  57. 88.
  58. 89.
  59. 90.
  60. 91.
  61. 92.
  62. 93.
  63. 94.
  64. 95.
  65. 96.
  66. 97.
  67. 98.
  68. 113.
  69. 114.
  70. 115.
  71. 116.

    1. as a docker/registry storage driver 2. as a docker/docker

    graph driver 3. standalone (load/save) dkr-shp
  72. 117.

    jbenet @ earth : ~ > dkr-shp shp - containers

    at hyperspeed shp (starship) uses IPFS, a p2p merkle-dag transport, to move around container images. This is dkr-shp, the docker-specific tool for shp. COMMANDS dkr-shp init initialize a shp station dkr-shp start start the shp station dkr-shp push <docker-id> publish given docker image on this host dkr-shp pull <ipfs-hash> pull down docker image, add it to docker
  73. 118.

    jbenet @ earth : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp
  74. 119.

    jbenet @ earth : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79
  75. 120.

    jbenet @ earth : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79 jbenet @ earth : ~ > docker images | grep ipfs jbenet/go-ipfs latest c1d9bf8d8844 4 hours ago 29.76 MB thelinuxkid/ipfs-reg latest 8bd6f334dca4 2 days ago 254.2 MB
  76. 121.

    jbenet @ earth : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79 jbenet @ earth : ~ > docker images | grep ipfs jbenet/go-ipfs latest c1d9bf8d8844 4 hours ago 29.76 MB thelinuxkid/ipfs-reg latest 8bd6f334dca4 2 days ago 254.2 MB jbenet @ earth : ~ > dkr-shp push c1d9bf8d8844 published as /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844
  77. 122.

    jbenet @ earth : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79 jbenet @ earth : ~ > docker images | grep ipfs jbenet/go-ipfs latest c1d9bf8d8844 4 hours ago 29.76 MB thelinuxkid/ipfs-reg latest 8bd6f334dca4 2 days ago 254.2 MB jbenet @ earth : ~ > dkr-shp push c1d9bf8d8844 published as /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844 jbenet @ earth : ~ > dkr-shp push jbenet/go-ipfs published as /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/jbenet/go-ipfs
  78. 123.

    jbenet @ mars : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp
  79. 124.

    jbenet @ mars : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ mars : ~ > dkr-shp start started shp station (registry) eac13c827004cf7a
  80. 125.

    jbenet @ mars : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ mars : ~ > dkr-shp start started shp station (registry) eac13c827004cf7a jbenet @ mars : ~ > dkr-shp pull c1d9bf8d8844 pulled /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844
  81. 126.

    jbenet @ mars : ~ > dkr-shp init initialized shp

    station (registry) at /var/lib/shp jbenet @ mars : ~ > dkr-shp start started shp station (registry) eac13c827004cf7a jbenet @ mars : ~ > dkr-shp pull c1d9bf8d8844 pulled /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844 jbenet @ mars : ~ > dkr-shp peers /ip4/104.236.32.22/tcp/4006/ipfs/QmNxzC53nTdTJaFivaev5mu2zFJmzcAAfQXJqTLarB8omm /ip4/162.243.139.61/tcp/4006/ipfs/QmNaVrdSKnZ8qpUhhGJeJuo7AjMHJBDE2xSWMvBau34g8P /ip4/104.236.176.52/tcp/4001/ipfs/QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z /ip4/128.199.219.111/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu /ip4/178.62.158.247/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd /ip4/178.62.61.185/tcp/4001/ipfs/QmSoLMeWqB7YGVLJN3pNLQpmmEk35v6wYtsMGLzSr5QBU3
  82. 129.

    jbenet @ mars : ~ > dkr-shp addrs /ip4/127.0.0.1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip4/172.17.0.25/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH

    /ip6/::1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH jbenet @ mars : ~ > dkr-shp connect $addr jbenet @ mars : ~ > # setup in a completely private network jbenet @ mars : ~ > dkr-shp init --net QmPzugFx3ysr8BujxZ3dkBiRYnuS2D63ndkAdq3W7J initialized shp station (registry) at /var/lib/shp