Containers @ Hyperspeed

(talk given at container.camp london -- 2015-09-11)

Containers are awesome! But storing and moving them is painful! It's slow! Containers are huge! There's little deduplication! No cryptographic versioning! Images are lost! Worst, we must depend on other services -- or having fast connection to them...

No more! starship puts your containers on hyperspeed! Instead of old-school HTTP, starship uses shiny IPFS -- a new protocol, Git meets Bittorrent. With IPFS, starship (a) deduplicates all data between containers, (b) uses p2p download from fastest hosts, and (c) can limit connections to private cloud(s) or local networks.

Transcript

1. ipfs.io @juanbenet Protocol Labs Containers @ Hyperspeed!
3. IP UDP TCP SCTP Ethernet CSMA WIFI TP Coax Satellite Fiber Radio PPP HTTP SMTP FTP NFS Skype Pigeons Chat
7. 1. containers 2. hyperspeed 3. starship
8. 1. containers 2. hyperspeed 3. starship 4. car
9. 1. containers 2. hyperspeed 3. starship 4. car
10. 1. containers 2. hyperspeed 3. starship
12. represent a program
13. have everything needed to run
14. have everything needed to run (sort of)
15. containers have flavors
16. just a bunch of files
17. just a bunch of files & standard protocols to run them
18. protocols!
19. protocols! runC Open Container Initiative
20. some containers are small
21. some containers are big
22. some containers are big ~ 50 MB ~ 300 MB
23. some containers are HUGE
24. some containers are HUGE ~ 5 GB ~ 10 GB
57. 1. containers 2. hyperspeed 3. starship
60. What is IPFS?
61. A protocol to upgrade the web
62. A protocol to upgrade the web
64. 2,387,990,609
65. 2,387,990,609 477.6 PB
72. A protocol to upgrade the web
73. Permanent Distributed Safer Smarter Offline Faster
75. hyperlinks
76. hyperlinks media
77. a hypermedia transport protocol HTTP
78. a hypermedia transport protocol HTTP IPFS
80. SFS web DHT +
81. routing network exchange merkledag naming applications IPNS DNS Namecoin Bitswap HTTP BitTorrent Kad DHT Chord mDNS Gossip Delegated FTP TOR QUIC TOR TCP uTP WebRTC WebSockets Git Bitcoin VLC Etherpad Chat I2P I2P CJDNS UDT
88. cvs/svn
89. cvs/svn
90. cvs/svn
99. just a bunch of files
100. just a bunch of files
101. just a bunch of files
102. just a bunch of files
103. just a bunch of files
104. just a bunch of files
105. just a bunch of files
106. just a bunch of files (tar) (tar) (tar)
107. just a bunch of files (tar) (tar) (tar)
108. 1. containers 2. hyperspeed 3. starship
109. shp - starship container hypertransport
110. shp - starship container hypertransport
111. shp - starship container hypertransport
112. shp - starship container hypertransport
115. DEMO
116. 1. as a docker/registry storage driver 2. as a docker/docker graph driver 3. standalone (load/save) dkr-shp
117. jbenet @ earth : ~ > dkr-shp shp - containers at hyperspeed shp (starship) uses IPFS, a p2p merkle-dag transport, to move around container images. This is dkr-shp, the docker-specific tool for shp. COMMANDS dkr-shp init initialize a shp station dkr-shp start start the shp station dkr-shp push <docker-id> publish given docker image on this host dkr-shp pull <ipfs-hash> pull down docker image, add it to docker
118. jbenet @ earth : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp
119. jbenet @ earth : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79
120. jbenet @ earth : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79 jbenet @ earth : ~ > docker images | grep ipfs jbenet/go-ipfs latest c1d9bf8d8844 4 hours ago 29.76 MB thelinuxkid/ipfs-reg latest 8bd6f334dca4 2 days ago 254.2 MB
121. jbenet @ earth : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79 jbenet @ earth : ~ > docker images | grep ipfs jbenet/go-ipfs latest c1d9bf8d8844 4 hours ago 29.76 MB thelinuxkid/ipfs-reg latest 8bd6f334dca4 2 days ago 254.2 MB jbenet @ earth : ~ > dkr-shp push c1d9bf8d8844 published as /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844
122. jbenet @ earth : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ earth : ~ > dkr-shp start started shp station (registry) 951f267861f99a79 jbenet @ earth : ~ > docker images | grep ipfs jbenet/go-ipfs latest c1d9bf8d8844 4 hours ago 29.76 MB thelinuxkid/ipfs-reg latest 8bd6f334dca4 2 days ago 254.2 MB jbenet @ earth : ~ > dkr-shp push c1d9bf8d8844 published as /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844 jbenet @ earth : ~ > dkr-shp push jbenet/go-ipfs published as /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/jbenet/go-ipfs
123. jbenet @ mars : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp
124. jbenet @ mars : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ mars : ~ > dkr-shp start started shp station (registry) eac13c827004cf7a
125. jbenet @ mars : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ mars : ~ > dkr-shp start started shp station (registry) eac13c827004cf7a jbenet @ mars : ~ > dkr-shp pull c1d9bf8d8844 pulled /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844
126. jbenet @ mars : ~ > dkr-shp init initialized shp station (registry) at /var/lib/shp jbenet @ mars : ~ > dkr-shp start started shp station (registry) eac13c827004cf7a jbenet @ mars : ~ > dkr-shp pull c1d9bf8d8844 pulled /ipfs/QmNsF3nR6kTz3PnKWH1uZP97ESYsLP9uJVqV9EpNfzRKSk/c1d9bf8d8844 jbenet @ mars : ~ > dkr-shp peers /ip4/104.236.32.22/tcp/4006/ipfs/QmNxzC53nTdTJaFivaev5mu2zFJmzcAAfQXJqTLarB8omm /ip4/162.243.139.61/tcp/4006/ipfs/QmNaVrdSKnZ8qpUhhGJeJuo7AjMHJBDE2xSWMvBau34g8P /ip4/104.236.176.52/tcp/4001/ipfs/QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z /ip4/128.199.219.111/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu /ip4/178.62.158.247/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd /ip4/178.62.61.185/tcp/4001/ipfs/QmSoLMeWqB7YGVLJN3pNLQpmmEk35v6wYtsMGLzSr5QBU3
127. jbenet @ mars : ~ > dkr-shp addrs /ip4/127.0.0.1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip4/172.17.0.25/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip6/::1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH
128. jbenet @ mars : ~ > dkr-shp addrs /ip4/127.0.0.1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip4/172.17.0.25/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip6/::1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH jbenet @ mars : ~ > dkr-shp connect $addr
129. jbenet @ mars : ~ > dkr-shp addrs /ip4/127.0.0.1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip4/172.17.0.25/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH /ip6/::1/tcp/4001/ipfs/QmUm6cjdSw7GFUVCxDoos9S5ikAMcjtug7J5syw74AovLH jbenet @ mars : ~ > dkr-shp connect $addr jbenet @ mars : ~ > # setup in a completely private network jbenet @ mars : ~ > dkr-shp init --net QmPzugFx3ysr8BujxZ3dkBiRYnuS2D63ndkAdq3W7J initialized shp station (registry) at /var/lib/shp
130. ipfs.io @juanbenet Protocol Labs Containers @ Hyperspeed!