IPFS: Re-distributing the Web

Slides from my presentation at DTN Conf (http://dtn.is) on May 2015

Transcript

1. ipfs.io
   @IPFSbot
   Juan Benet
   @juanbenet
   2015-05-24
   The Permanent Web
   

   View Slide

2. WIFI
   SSID: /
   PASS:
   

   View Slide

3. WIFI
   SSID: /
   PASS:
   

   View Slide

4. ipfs.io
   @IPFSbot
   Juan Benet
   @juanbenet
   2015-05-24
   The Permanent Web
   

   View Slide

5. ipfs.io
   @IPFSbot
   Juan Benet
   @juanbenet
   2015-05-24
   The Permanent Web
   Distribute
   

   View Slide

6. ipfs.io
   @IPFSbot
   Juan Benet
   @juanbenet
   2015-05-24
   The Permanent Web
   Re-
   

   View Slide

7. 1. What is IPFS?
   2. IPFS Intro
   3. Layering
   4. Datastructures
   5. Demo
   

   View Slide

8. What is IPFS?
   

   View Slide

9. What is IPFS?
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. A protocol to upgrade the web
    

    View Slide

18. View Slide

19. Distributed
    

    View Slide

20. Distributed
    Offline
    

    View Slide

21. Permanent
    Distributed
    Offline
    

    View Slide

22. Permanent
    Distributed
    Smarter
    Offline
    

    View Slide

23. Permanent
    Distributed Safer
    Smarter
    Offline
    

    View Slide

24. Permanent
    Distributed Safer
    Smarter
    Offline
    Faster
    

    View Slide

25. View Slide

26. View Slide

27. View Slide

28. View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. 2026
    1991 2015
    

    View Slide

37. 2026
    1991 2015
    24 11
    

    View Slide

38. 2026
    1991 2015
    24 11
    11
    

    View Slide

39. 2026
    1991 2015
    24 11
    11
    

    View Slide

40. View Slide

41. View Slide

42. What is IPFS?
    

    View Slide

43. View Slide

44. hyperlinks
    

    View Slide

45. hyperlinks media
    

    View Slide

46. a hypermedia transport protocol
    HTTP
    

    View Slide

47. web
    

    View Slide

48. SFS
    web
    

    View Slide

49. SFS
    web
    

    View Slide

50. SFS
    web
    

    View Slide

51. SFS
    web
    DHT
    

    View Slide

52. SFS
    web
    DHT
    +
    

    View Slide

53. View Slide

54. View Slide

55. View Slide

56. View Slide

57. View Slide

58. What is IPFS?
    

    View Slide

59. 1. A Protocol Stack
    2. Tools + Programs
    3. A Live System
    

    View Slide

60. web
    DHT
    SFS
    

    View Slide

61. web
    DHT
    SFS
    routing
    network
    exchange
    merkledag
    naming
    applications
    The Stack
    

    View Slide

62. routing
    network
    exchange
    merkledag
    naming
    applications
    The Stack
    

    View Slide

63. routing
    network
    exchange
    merkledag
    naming
    applications
    The Stack
    Defining the Data
    Moving the Data
    Using the Data
    

    View Slide

64. routing
    network
    exchange
    merkledag
    naming
    applications
    The Stack
    Defining the Data
    Moving the Data
    Using the Data
    

    View Slide

65. routing
    network
    exchange
    merkledag
    naming
    applications
    IPNS
    Bitswap
    QUIC TCP
    uTP WebRTC WebSockets
    Git Bitcoin
    VLC
    Etherpad Chat
    UDT
    CJDNS
    Kad DHT mDNS
    

    View Slide

66. routing
    network
    exchange
    merkledag
    naming
    applications
    IPNS
    Bitswap
    QUIC TCP
    uTP WebRTC WebSockets
    UDT
    CJDNS
    Kad DHT mDNS
    Git Bitcoin
    VLC
    Etherpad Chat
    

    View Slide

67. routing
    network
    exchange
    merkledag
    naming
    applications
    IPNS
    DNS Namecoin
    Bitswap HTTP
    BitTorrent
    Kad DHT
    Chord mDNS
    Gossip Delegated
    FTP
    TOR
    QUIC TOR
    TCP
    uTP WebRTC WebSockets
    Git Bitcoin
    VLC
    Etherpad Chat
    I2P
    I2P
    CJDNS UDT
    

    View Slide

68. IP
    UDP
    TCP SCTP
    Ethernet
    CSMA WIFI
    TP
    Coax Satellite
    Fiber Radio
    PPP
    HTTP SMTP
    FTP
    NFS Skype
    Pigeons
    Chat
    

    View Slide

69. IP
    UDP
    TCP SCTP
    Ethernet
    CSMA WIFI
    TP
    Coax Satellite
    Fiber Radio
    PPP
    HTTP SMTP
    FTP
    NFS Skype
    Pigeons
    Chat
    

    View Slide

70. View Slide

71. View Slide

72. View Slide

73. View Slide

74. View Slide

75. View Slide

76. View Slide

77. View Slide

78. Content Addressing
    1. Motivation
    

    View Slide

79. http://10.20.30.40/foo/bar/baz.png
    location path
    http://example.com/foo/bar/baz.png
    1. Motivation
    

    View Slide

80. http://
    location path
    http://
    1. Motivation
    /ipns/example.com/foo/bar/baz.png
    /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
    content path
    

    View Slide

81. http://10.20.30.40/foo/bar/baz.png
    you
    1. Motivation
    /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
    

    View Slide

82. http://10.20.30.40/foo/bar/baz.png
    10.20.30.40
    you
    1. Motivation
    /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
    HTTP
    

    View Slide

83. http://10.20.30.40/foo/bar/baz.png
    10.20.30.40
    you
    1. Motivation
    /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
    IPFS
    

    View Slide

84. 1. Motivation
    p2p
    distribution
    /ipfs/
    

    View Slide

85. 1. Motivation
    p2p
    distribution
    /ipfs/
    

    View Slide

86. merkledag
    2. Overview
    the model
    IPFS nodes
    

    View Slide

87. merkledag
    2. Overview
    the model
    IPFS nodes
    (content) (programs)
    

    View Slide

88. in IPFS
    data forms a merkle dag
    nodes have
    links and data
    [ (hash, size, name), … ]
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    anything you want
    []byte
    links
    data
    2. Overview
    

    View Slide

89. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    1
    3
    5
    2 4
    suppose links:
    

    View Slide

90. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    suppose H(<1>) = QmW98pJrc6FZ
    We can resolve:
    /ipfs/QmW98pJrc6FZ as 1
    /ipfs/QmW98pJrc6FZ/foo as 2
    /ipfs/QmW98pJrc6FZ/bar as 3
    /ipfs/QmW98pJrc6FZ/foo/biff as 3
    /ipfs/QmW98pJrc6FZ/baz as 4
    1
    3
    5
    1
    2
    2
    3
    4
    4
    3
    suppose links:
    1
    

    View Slide

91. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    suppose H(<1>) = QmW98pJrc6FZ
    We can resolve:
    /ipfs/QmW98pJrc6FZ as 1
    /ipfs/QmW98pJrc6FZ/foo as 2
    /ipfs/QmW98pJrc6FZ/bar as 3
    /ipfs/QmW98pJrc6FZ/foo/biff as 3
    /ipfs/QmW98pJrc6FZ/baz as 4
    1
    3
    5
    1
    2
    2
    3
    4
    4
    3
    suppose links:
    1
    

    View Slide

92. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    suppose H(<1>) = QmW98pJrc6FZ
    We can resolve:
    /ipfs/QmW98pJrc6FZ as 1
    /ipfs/QmW98pJrc6FZ/foo as 2
    /ipfs/QmW98pJrc6FZ/bar as 3
    /ipfs/QmW98pJrc6FZ/foo/biff as 3
    /ipfs/QmW98pJrc6FZ/baz as 4
    1
    3
    5
    1
    2
    2
    3
    4
    4
    3
    suppose links:
    1
    

    View Slide

93. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    suppose H(<1>) = QmW98pJrc6FZ
    We can resolve:
    /ipfs/QmW98pJrc6FZ as 1
    /ipfs/QmW98pJrc6FZ/foo as 2
    /ipfs/QmW98pJrc6FZ/bar as 3
    /ipfs/QmW98pJrc6FZ/foo/biff as 3
    /ipfs/QmW98pJrc6FZ/baz as 4
    1
    3
    5
    1
    2
    2
    3
    4
    4
    3
    suppose links:
    1
    

    View Slide

94. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    suppose H(<1>) = QmW98pJrc6FZ
    We can resolve:
    /ipfs/QmW98pJrc6FZ as 1
    /ipfs/QmW98pJrc6FZ/foo as 2
    /ipfs/QmW98pJrc6FZ/bar as 3
    /ipfs/QmW98pJrc6FZ/foo/biff as 3
    /ipfs/QmW98pJrc6FZ/baz as 4
    1
    3
    5
    1
    2
    2
    3
    4
    4
    3
    suppose links:
    1
    

    View Slide

95. can resolve unix-style
    paths across dag links
    2. Overview
    QmbTJW4iGGBS 3987 foo
    QmbTQmPMEFgh 1020 bar
    QmZFJguywFUY 6787 baz
    QmbTQmPMEFgh 1020 biff
    this node has some data…
    this node also has some
    data. but no links!
    suppose H(<1>) = QmW98pJrc6FZ
    We can resolve:
    /ipfs/QmW98pJrc6FZ as 1
    /ipfs/QmW98pJrc6FZ/foo as 2
    /ipfs/QmW98pJrc6FZ/bar as 3
    /ipfs/QmW98pJrc6FZ/foo/biff as 3
    /ipfs/QmW98pJrc6FZ/baz as 4
    1
    3
    5
    1
    2
    2
    3
    4
    4
    3
    suppose links:
    1
    

    View Slide

96. any data structures are
    represented as dags
    2. Overview
    

    View Slide

97. 2. Overview
    unix files
    and dirs
    any data structures are
    represented as dags
    

    View Slide

98. blob, tree, commit
    unix files
    and dirs
    any data structures are
    represented as dags
    2. Overview
    

    View Slide

99. blob, tree, commit
    unix files
    and dirs
    any data structures are
    represented as dags
    blocks, txns, wallets
    2. Overview
    

    View Slide

100. blob, tree, commit
     unix files
     and dirs
     any data structures are
     represented as dags
     blocks, txns, wallets
     kv-stores
     2. Overview
     

     View Slide

101. blob, tree, commit
     unix files
     and dirs
     any data structures are
     represented as dags
     blocks, txns, wallets
     kv-stores
     2. Overview
     

     View Slide

102. unixfs as a dag
     2. Overview
     

     View Slide

103. files are dag nodes
     unixfs as a dag
     2. Overview
     

     View Slide

104. files are dag nodes
     BIG files may be
     split into many
     unixfs as a dag
     2. Overview
     

     View Slide

105. directories are
     also dag nodes
     files are dag nodes
     BIG files may be
     split into many
     unixfs as a dag
     2. Overview
     

     View Slide

106. which link
     to others
     directories are
     also dag nodes
     files are dag nodes
     BIG files may be
     split into many
     unixfs as a dag
     2. Overview
     

     View Slide

107. ipfs paths
     /ipfs//
     

     View Slide

108. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     

     View Slide

109. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     XLYkgq61DYaQ8Nhk
     

     View Slide

110. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     foo aCpDMGv
     XLYkgq61DYaQ8Nhk
     data1
     

     View Slide

111. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     foo aCpDMGv
     XLYkgq61DYaQ8Nhk
     data1
     

     View Slide

112. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     foo aCpDMGv
     XLYkgq61DYaQ8Nhk
     data1
     aCpDMGvV2BGHeYER
     

     View Slide

113. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     foo aCpDMGv
     XLYkgq61DYaQ8Nhk
     data1
     bar.txt UEnRQAw
     aCpDMGvV2BGHeYER
     data2
     

     View Slide

114. ipfs paths
     /ipfs//
     /ipfs/XLYkgq61DYaQ8Nhk/foo/bar.txt
     foo aCpDMGv
     XLYkgq61DYaQ8Nhk
     data1
     bar.txt UEnRQAw
     aCpDMGvV2BGHeYER
     data2
     UEnRQAwe3N8SzbUt
     data3
     

     View Slide

115. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     

     View Slide

116. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     

     View Slide

117. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     
     

     View Slide

118. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     tx
     
     data2
     parent
     

     View Slide

119. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     tx
     
     data2
     
     parent
     

     View Slide

120. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     tx
     
     data2
     
     data3
     15
     parent
     14
     

     View Slide

121. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     tx
     
     data2
     
     data3
     15
     
     parent
     14
     

     View Slide

122. ipfs paths: blockchain
     /ipfs/XLYkgq61DYaQ8Nhk/parent/tx/15
     parent
     XLYkgq61DYaQ8Nhk
     data1
     tx
     
     data2
     
     data3
     15
     
     data4
     input0
     parent
     14
     output0
     

     View Slide

123. an IPFS node
     2. Overview
     

     View Slide

124. has pki based
     identity
     an IPFS node
     2. Overview
     

     View Slide

125. has pki based
     identity
     connects
     to others
     an IPFS node
     2. Overview
     

     View Slide

126. can store part
     of the dag
     has pki based
     identity
     connects
     to others
     an IPFS node
     2. Overview
     

     View Slide

127. has pki based
     identity
     connects
     to others
     can get more
     from peers
     an IPFS node
     can store part
     of the dag
     2. Overview
     

     View Slide

128. has pki based
     identity
     connects
     to others
     can be run
     as a server
     can get more
     from peers
     an IPFS node
     can store part
     of the dag
     2. Overview
     

     View Slide

129. has pki based
     identity
     connects
     to others
     can be run
     as a server
     or embedded
     in apps
     app

     

     can get more
     from peers
     an IPFS node
     can store part
     of the dag
     2. Overview
     

     View Slide

130. 2. Overview
     IPFS nodes form p2p networks
     transport agnostic and with NAT traversal
     

     View Slide

131. they find each other and content
     with routing systems, like DHTs
     2. Overview
     

     View Slide

132. and exchange dag nodes
     like files in http or bittorrent
     2. Overview
     

     View Slide

133. and exchange dag nodes
     like files in http or bittorrent
     2. Overview
     

     View Slide

134. and exchange dag nodes
     like files in http or bittorrent
     2. Overview
     

     View Slide

135. DEMO
     

     View Slide

136. 4 ways to install IPFS
     0. wget from an ifs link
     1. npm install -g go-ipfs
     2. go get github.com/ipfs/go-ipfs/cmd/ipfs
     3. install packages on the website
     

     View Slide

137. in go-ipfs:
     ipfs node cli library
     http api
     http/ipfs
     gateway
     webui
     

     View Slide

138. #ipfs on
     

     View Slide