Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Blockchain and Distributed Consensus

Blockchain and Distributed Consensus

Second lesson for the Bitcoin and Blockchain Technology course of Milano-Bicocca and Politecnico di Milano

www.ametrano.net/bbt/

Ferdinando M. Ametrano

March 01, 2019
Tweet

More Decks by Ferdinando M. Ametrano

Other Decks in Technology

Transcript

  1. Bitcoin and
    Blockchain Technology
    Blockchain, Mining and Distributed Consensus
    v2019.04.03
    Comments, corrections, and questions: https://drive.google.com/open?id=1_rGy7wdI8iWx6w6LG_CGCmmLnAIFhncz
    © 2019 Digital Gold Institute

    View full-size slide

  2. Credit
    In the following, some slides are from:
    A. Narayanan, et al.,
    “Bitcoin and Cryptocurrency Technologies”,
    Princeton Univ Press (2016)
    http://bitcoinbook.cs.princeton.edu/
    © 2019 Digital Gold Institute 2/109

    View full-size slide

  3. Table of Contents
    1. Hash Functions
    2. Simplified Digital Currency
    3. Distributed Consensus
    4. Mining
    5. P2P Network
    6. Protocol Governance
    © 2019 Digital Gold Institute 3/109

    View full-size slide

  4. Hash Functions
    A hash function, e.g. SHA256, is a map from the set of input data
    to the output set of hash values:
    SHA256(“Hello, world!”) =
    315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3
    Small differences in the input data produce large differences in the
    result:
    SHA256(“Hello, world.”) =
    f8c3bf62a9aa3e6fc1619c250e48abe7519373d3edf41be62eb5dc45199af2ef
    © 2019 Digital Gold Institute 4/109

    View full-size slide

  5. Cryptographic Hash Functions (1/2)
    ▪ Arbitrary message size: h(x) can be applied to message x of any
    size
    ▪ Fixed output length: h(x) produces a hash value of fixed length
    (256 bits for SHA256)
    ▪ Efficiency: h(x) is relatively easy to compute
    © 2019 Digital Gold Institute 5/109

    View full-size slide

  6. Cryptographic Hash Functions (2/2)
    ▪ One-wayness (preimage resistance):
    given h(x), it is computationally infeasible to find x
    ▪ Weak collision resistance (second pre-image resistance):
    given x it is computationally infeasible to find y!=x such that
    h(y)=h(x)
    ▪ (Strong) Collision resistance:
    it is computationally infeasible to find (x, y!=x) such that
    h(y)=h(x)
    © 2019 Digital Gold Institute 6/109

    View full-size slide

  7. Computationally Infeasible Collisions
    ▪ The size of the possible hash function output values is smaller
    than the size of possible input data. Therefore, many input data
    must share the same hash function output value: collisions do
    exist...
    ▪ Try 2^130 randomly chosen inputs: 99.8% chance that two of
    them will collide. Anyway, to try them is computationally
    unfeasible
    ▪ For some hash functions computationally feasible ways to find
    collisions have been found; this is not the case for “good”
    functions
    © 2019 Digital Gold Institute 7/109

    View full-size slide

  8. Merkle–Damgård Hash Construction
    A hash function is usually based on a one-way compression
    function; it is as resistant to collisions as its compression function
    256 bits
    Init.
    Value
    cf
    512 bits
    Msg
    (block1)
    cf
    512 bits
    Msg
    (block2)
    Length Padding
    cf
    512 bits
    Msg
    (blockN)
    256 bits
    Hash
    Msg
    (block1)
    Msg
    (block2)
    Msg
    (blockN)
    ….
    © 2019 Digital Gold Institute 8/109

    View full-size slide

  9. Hash as Message Digest
    ▪ If we know h(x) = h(y), then it is safe to assume that x = y
    ▪ Useful because h(x) can be much smaller than x, e.g. often
    used as download integrity check
    © 2019 Digital Gold Institute 9/109

    View full-size slide

  10. Puzzle Friendliness
    Given and a target set , to find from high min-entropy
    distribution such that
    ℎ || ∈
    there is no solving strategy better than trying random values of
    Min-entropy measures how unlikely you are to guess a value on
    your first try. If the probability of guessing right is p, then min-
    entropy is defined as −log2
    .
    For example, for a fair coin toss (i.e. the status of a single bit),
    you'd have p = 0.5, giving a min-entropy of 1. A uniformly random
    256-bit string would have −log2
    2−256 = 256 bits of min entropy
    © 2019 Digital Gold Institute 10/109

    View full-size slide

  11. Hash Puzzle: Partial Hash Inversion
    ▪ Find nonce so that ℎ || is small, e.g. starts with zero
    ▪ The smaller the target space, the harder to find a solution
    ▪ For good hash functions, the problem can only be solved by
    brute force trials
    ▪ It is trivial to verify the solution: it is just one hash function
    evaluation
    © 2019 Digital Gold Institute 11/109

    View full-size slide

  12. Hash Pointer
    The cryptographic hash of the data can be used as pointer to the
    data itself
    With hash pointer one:
    ▪ gets the data back
    ▪ verifies that data has not changed
    (data) H( )
    © 2019 Digital Gold Institute 12/109

    View full-size slide

  13. Hash Pointers Data Structures
    Hash pointers can be used in any pointer-based data structure with
    no cycles
    Binary Tree → Merkle Tree
    Linked list → Blockchain
    © 2019 Digital Gold Institute 13/109

    View full-size slide

  14. Blockchain: Hash Pointer Linked List
    data2
    prev: H( )
    data1
    prev: H( )
    data0
    Genesis Block
    H( )
    tamper evident data structure
    © 2019 Digital Gold Institute 14/109

    View full-size slide

  15. Merkle Tree: Hash Pointer Binary Tree
    H( )
    H( ) H( )
    H( ) H( ) H( ) H( )
    H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( )
    data0 data1 data2 data3 data4 data5 data6 data7
    © 2019 Digital Gold Institute
    inclusion evident data structure
    15/109

    View full-size slide

  16. Merkle Tree: Hash Pointer Binary Tree
    H( )
    H( ) H( )
    H( ) H( )
    H( ) H( )
    data3
    Data3 inclusion proof consists
    only of 3 hash values
    ▪ O(log n) inclusion proof
    If sorted:
    ▪ O(log n) non-inclusion proof
    © 2019 Digital Gold Institute
    inclusion evident data structure
    16/109

    View full-size slide

  17. Homework
    ▪ Find the nonce that appended to your name obtains a SHA256
    hash value starting with 7 zeros
    [As in FirstnameLastnameNonce, test at https://emn178.github.io/online-tools/sha256.html]
    ▪ How many numbers in the range 0, result into hash values
    starting with one or more zeros?
    ▪ Produce the histogram with the frequency of hash values
    starting with 1, 2, 3, 4, 5, 6, and 7 zeros in the 0, range
    Use the Python3 script hash_puzzle.py at github.com/dginst/bbt/py-scripts/
    © 2019 Digital Gold Institute 17/109

    View full-size slide

  18. Table of Contents
    1. Hash Functions
    2. Simplified Digital Currency
    3. Distributed Consensus
    4. Mining
    5. P2P Network
    6. Protocol Governance
    © 2019 Digital Gold Institute 18/109

    View full-size slide

  19. Hash value of this data is
    the coin “serial number”
    Central Bank Coin Issued to Carol
    Coin Creation
    signed by p
    CentralBanker
    Spendable by P
    Carol
    Owner of the spendable
    coin
    P Public Key
    p private key
    © 2019 Digital Gold Institute 19/109

    View full-size slide

  20. Carol Spends the Coin To Alice
    Coin Creation
    signed by p
    CentralBanker
    Spendable by P
    Carol
    Spend H( )
    signed by p
    Carol
    Spendable by P
    Alice
    © 2019 Digital Gold Institute
    New coin with different
    “serial number”

    P Public Key
    p private key
    This spent coin is now
    unspendable
    Owner of the spendable
    coin
    20/109

    View full-size slide

  21. Alice Spends the Coin To Bob
    Spend H( )
    signed by p
    Alice
    Spendable by P
    Bob
    © 2019 Digital Gold Institute
    P Public Key
    p private key
    This spent coin is now
    unspendable
    This spent coin is now
    unspendable
    New coin with another
    different “serial number”
    Owner of the spendable
    coin
    Spend H( )
    signed by p
    Carol
    Spendable by P
    Alice
    Coin Creation
    signed by p
    CentralBanker
    Spendable by P
    Carol


    21/109

    View full-size slide

  22. Double Spending Attack
    For the network both transactions are equal.
    Two alternative new coins: which one is valid?
    Spend H( )
    signed by p
    Alice
    Spendable by P
    Alice
    © 2019 Digital Gold Institute
    P Public Key
    p private key
    This spent coin is now
    unspendable
    This spent coin is now
    unspendable
    Spend H( )
    signed by p
    Alice
    Spendable by P
    Bob
    Spend H( )
    signed by p
    Carol
    Spendable by P
    Alice
    Coin Creation
    signed by p
    CentralBanker
    Spendable by P
    Carol


    22/109

    View full-size slide

  23. Hash Chain Signed By Central Banker
    Valid Chain Head H( )
    signed by p
    CentralBanker
    © 2019 Digital Gold Institute
    P Public Key
    p private key
    Coin Creation
    signed by p
    CentralBanker
    Spendable by P
    Carol

    Spend H( )
    signed by p
    Alice
    Spendable by P
    Bob
    Spend H( )
    signed by p
    Carol
    Spendable by P
    Alice
    Owner of the spendable
    coin
    Double-spend attack can be solved
    resorting to a central authority


    23/109

    View full-size slide

  24. Alice Pays Bob
    ▪ When Alice wants to pay Bob, she broadcasts the transaction to
    the network
    ▪ Bob is not required to be a network node or to actively monitor
    the network
    ▪ Bob will discover the transaction next time he will monitor the
    network
    © 2019 Digital Gold Institute 24/109

    View full-size slide

  25. Coinbase Transaction
    TransID:71 Coin Creation
    signed by p
    CentralBanker
    Output num value Pay to:
    0 0.8 P0
    1 3.2 P1
    2 1.1 P2
    coinID 71(0)
    coinID 71(1)
    coinID 71(2)
    © 2019 Digital Gold Institute
    P Public Key, p private key
    25/109

    View full-size slide

  26. Spending Transaction
    TransID:72 Spend 42(0), 71(2)
    signed by p
    42(0)
    , p2
    71(2)
    Output num value Pay to:
    0 0.5 P3
    1 2.1 P4
    2 1.7 P5
    Valid only if 42(0) and 71(2) are:
    ▪ valid
    ▪ not already spent
    ▪ signed by their owners
    Valid only if:
    Input coins >= output coins
    i.e. 42(0)+71(2) >= 72(0)+72(1)+72(2)
    coinID 72(0)
    coinID 72(1)
    coinID 72(2)
    © 2019 Digital Gold Institute
    P Public Key, p private key
    26/109

    View full-size slide

  27. Immutable Coins
    ▪ Coins cannot be transferred, subdivided, or combined
    ▪ Coins can only be entirely consumed as input to a transaction
    (TxIn) that creates new output coins (TxO)
    ▪ The set of current unspent coins is referred to as UTxO
    (Unspent Transaction Outputs)
    © 2019 Digital Gold Institute 27/109

    View full-size slide

  28. Transaction Bundling
    ▪ Transactions are bundled in blocks for convenience: shorter
    chain, single unit of validation work, faster history validation
    ▪ In a block transactions are packaged in a Merkle Tree: for now,
    we neglect that
    © 2019 Digital Gold Institute
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    28/109

    View full-size slide

  29. Remove the Central Banker
    Crucial questions:
    ▪ Can the digital currency be decentralized?
    ▪ Can it be functional without a trusted central party?
    © 2019 Digital Gold Institute 29/109

    View full-size slide

  30. Table of Contents
    1. Hash Functions
    2. Simplified Digital Currency
    3. Distributed Consensus
    4. Mining
    5. P2P Network
    6. Protocol Governance
    © 2019 Digital Gold Institute 30/109

    View full-size slide

  31. Consensus Needed By Bitcoin
    Who has authority over which transactions are valid?
    Who maintains the ledger?
    Who creates new bitcoins?
    Certainty is needed about:
    ▪ Past transactions
    ▪ Coin generation
    ▪ Spendable (i.e. still unspent) coins (UTXO)
    ▪ Who owns coins (i.e. how to spend them)
    © 2019 Digital Gold Institute 31/109

    View full-size slide

  32. Sybil Attack
    ▪ In P2P networks identities can be forged: a single agent can
    present itself as multiple nodes
    ▪ Reputation systems are easy to be subverted
    ▪ Distributed consensus cannot rely on node identity
    ▪ Moreover, pseudonymity is a goal of Bitcoin
    © 2019 Digital Gold Institute 32/109

    View full-size slide

  33. Distributed Consensus
    Correct nodes must decide about a value for a variable or a course
    of action:
    ▪ once the nodes reach a decision, that must be final
    (correctness)
    ▪ the network will make progress if some fraction of the network
    is non-faulty (liveness)
    The technical requirements of consensus are:
    ▪ every correct node eventually decides some value (termination)
    ▪ all correct nodes decide on the same value (agreement)
    ▪ if all correct nodes propose the same value, then any correct
    node decides that value (integrity)
    © 2019 Digital Gold Institute 33/109

    View full-size slide

  34. Distributed Consensus: Impossibility Result
    Consensus in distributed systems is hard because:
    ▪ Nodes may be faulty for technical malfunction (crash failure) or
    arbitrary malicious activity (Byzantine failure)
    ▪ Not all pairs of nodes are connected
    ▪ There is latency in communication between nodes
    ▪ There is no notion of global time
    Fischer, Lynch, and Paterson have proved that reaching consensus
    in an asynchronous network cannot be guaranteed, even on
    something as simple as a binary value, when even just a single
    node may be faulty or malicious
    © 2019 Digital Gold Institute 34/109

    View full-size slide

  35. The Byzantine Generals' Problem
    ▪ Generals can communicate using messengers, cannot have a
    summit
    ▪ There are traitors amongst them
    ▪ Must decide unanimously whether to attack
    ▪ Success (i.e. fault tolerance) is achieved if the loyal generals
    can agree on their strategy, whatever it might be
    © 2019 Digital Gold Institute 35/109

    View full-size slide

  36. Practical Byzantine Fault Tolerance
    ▪ No network is unboundedly asynchronous
    ▪ Practical Byzantine Fault Tolerance (PBFT) consensus can be
    achieved when a predominant majority of nodes are honest
    [Lamport, Marshall, Pease], even in the case of Byzantine
    failures
    ▪ Practically, networks being often effectively asynchronous, in
    some way distributed consensus is always implicitly probabilistic
    © 2019 Digital Gold Institute 36/109

    View full-size slide

  37. Simplified Consensus Algorithm
    1. New transactions are broadcasted to all nodes
    2. Each node collects new transactions into a block
    3. In each round, a random node is picked (as in a lottery or raffle,
    without verifying identities) and this node proposes the next
    block in the chain
    4. Other nodes accept the block only if it includes valid
    transactions (spend unspent coins, have valid signatures)
    5. Nodes express their implicit acceptance of the block by including
    its hash in the next block they create
    © 2019 Digital Gold Institute 37/109

    View full-size slide

  38. How To Pick a Random Node?
    (while preventing Sybil attacks)
    ▪ Let nodes compete for the right to create a block
    ▪ Nodes must provide Proof-of-Something
    Selection chances increase proportionally to a resource that cannot
    be faked:
    ▪ computing power: Proof-of-Work
    ▪ coin ownership: Proof-of-Stake
    © 2019 Digital Gold Institute 38/109

    View full-size slide

  39. Proof-of-Something Trade-offs
    Proof-of-Work (PoW):
    ▪ Must be incentivized someway, as it has operational costs
    ▪ Maintaining alternative transaction histories reduces the chance
    of block finalization
    Proof-of-Stake (PoS):
    ▪ Does not have significant operational costs
    ▪ Maintaining alternative transaction histories has no intrinsic
    penalty (nothing is at stake), some penalization mechanism
    must be introduced ex-post
    © 2019 Digital Gold Institute 39/109

    View full-size slide

  40. Proof-of-Work is the Most Secure
    ▪ Even if one party controls all PoW, an attack always has an
    energy cost
    ▪ In non-PoW systems, zero-cost attacks are always theoretically
    possible: one must rely on the soundness of an after-the-fact
    punishment
    © 2019 Digital Gold Institute 40/109

    View full-size slide

  41. Proof-of-Work
    Proof-of-Work types:
    ▪ Challenge-Response protocol: e.g. CAPTCHA
    ▪ Solution-Verification protocol: e.g. partial has inversion
    Nakamoto uses the latter, borrowing the idea from Hashcash
    (Adam Back 1997, 2002)
    © 2019 Digital Gold Institute 41/109

    View full-size slide

  42. Partial Hash Inversion
    Find nonce so that h(prev_hash || {Tx, …, Tx} || nonce) is very small
    ▪ The smaller the target space, the harder to find a solution
    ▪ For good hash functions, the problem can only be solved by
    brute force trials
    ▪ It is always trivial to verify the solution: it is just one hash
    function evaluation
    prev_hash
    nonce
    Tx
    Tx
    © 2019 Digital Gold Institute 42/109

    View full-size slide

  43. Partial Hash Inversion Difficulty
    ▪ Goal: 10 minutes average time between blocks
    ▪ Protocol enforces hash puzzle difficulty recalculation every 2016
    blocks (about two weeks): the bitcoin software automatically
    takes care of it
    © 2019 Digital Gold Institute 43/109

    View full-size slide

  44. Mining difficulty over time
    bitcoinwisdom.com
    © 2019 Digital Gold Institute 44/109

    View full-size slide

  45. Time to find a block
    10 minutes
    2016 blocks
    ~2 weeks
    bitcoinwisdom.com
    © 2019 Digital Gold Institute 45/109

    View full-size slide

  46. The Reference Chain
    ▪ At any moment the reference chain is the longest valid one
    ▪ More precisely: the reference chain is the one with higher
    cumulative difficulty
    ▪ Usually the two conditions are equivalent, but the former could
    be gamed lowering difficulty
    © 2019 Digital Gold Institute 46/109

    View full-size slide

  47. Blockchain Forks and Height
    ▪ Genesis Block
    ▪ Orphan Block
    ▪ Main Chain Block
    The reference chain here has height 8 (i.e. nine blocks)
    © 2019 Digital Gold Institute 47/109

    View full-size slide

  48. Orphan Blocks and Orphan Chains
    Orphan blocks are quite common: most common heuristic suggests
    to wait six confirmations before considering a transaction settled
    Orphaned
    Blocks
    History of orphan blocks
    97
    102
    101
    101
    98
    98
    98
    99
    99
    100
    100
    100
    103 104 105
    6 confirmations
    Wait 6 confirmations for important transactions
    © 2019 Digital Gold Institute 48/109

    View full-size slide

  49. Transaction Confirmation
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    CA
    → B transaction
    first heard: 0 confirmations
    prev_hash
    nonce
    CA
    → B
    Tx
    Tx
    1 confirmation
    © 2019 Digital Gold Institute 49/109

    View full-size slide

  50. Double Spending Attack
    For the network both
    transactions are equal.
    Which one is valid?
    © 2019 Digital Gold Institute
    CA
    → B transaction
    first heard: 0 confirmations
    prev_hash
    nonce
    CA
    → A
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    CA
    → B
    Tx
    Tx
    1 confirmation
    50/109

    View full-size slide

  51. Multiple Transaction Confirmations
    Double-spend success chances decrease exponentially with the
    number of confirmations
    © 2019 Digital Gold Institute
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    CA
    → B transaction
    first heard: 0 confirmations
    prev_hash
    nonce
    CA
    → B
    Tx
    Tx
    1 confirmation
    prev_hash
    nonce
    Tx
    Tx
    Tx
    3 confirmations
    prev_hash
    nonce
    Tx
    Tx
    Tx
    2 confirmations
    prev_hash
    nonce
    CA
    → A
    Tx
    Tx
    51/109

    View full-size slide

  52. Incentive For Honesty
    ▪ Can we go beyond distributed consensus protocol?
    ▪ Can we utilize the fact that the currency has value?
    © 2019 Digital Gold Institute
    prev_hash
    nonce
    Tx
    Tx
    Tx
    prev_hash
    nonce
    Tx
    Tx
    Tx
    CA
    → B transaction
    first heard: 0 confirmations
    prev_hash
    nonce
    CA
    → B
    Tx
    Tx
    1 confirmation
    prev_hash
    nonce
    Tx
    Tx
    Tx
    3 confirmations
    prev_hash
    nonce
    Tx
    Tx
    Tx
    2 confirmations
    prev_hash
    nonce
    CA
    → A
    Tx
    Tx
    Can we reward the nodes
    that created these blocks?
    Can we penalize the node
    that created this block?
    52/109

    View full-size slide

  53. Incentive 1: Block Reward
    ▪ The block creator includes a special coinbase transaction in the
    block, choosing at will the recipient address of this transaction
    ▪ Value is fixed: originally 50BTC, halves every 210,000 blocks
    (about 4 years)
    ▪ Block creator collects the reward only if the block ends up on
    long-term consensus branch!
    © 2019 Digital Gold Institute 53/109

    View full-size slide

  54. Incentive 2: Transaction Fees
    ▪ Transaction creators can choose to make output value less than
    input value: the remainder is a transaction fee and goes to the
    block creator
    ▪ Necessary to avoid spam attack with many payment-to-self or
    dust transactions
    ▪ Necessary as blocks are filled up to MAX_BLOCK_SIZE capacity
    © 2019 Digital Gold Institute 54/109

    View full-size slide

  55. Attacking the Blockchain
    An attacker wishing to change a given block would have to:
    ▪ apply a computational power equivalent to all the computational
    power spent from that block onward
    ▪ outrun the legitimate Bitcoin network that in the meantime
    would keep adding blocks to the original chain
    The attacker would need 51% of total computational power
    (actually less than 51%…)
    © 2019 Digital Gold Institute 55/109

    View full-size slide

  56. 51% Attack Effects
    ▪ Change the block reward?
    ▪ Steal coins from existing address?
    ▪ Remove transactions from the P2P network?
    ▪ High barrier of entry: buy/build hardware, setup farm, etc.
    ▪ Rational economic agents are better off mining coins than
    attacking
    © 2019 Digital Gold Institute



    56/109

    View full-size slide

  57. 51% Attack Effects
    ▪ Prevent transactions from being confirmed?
    ▪ Censor (kind of DoS attack against) specific addresses?
    ▪ Reverse recent transactions that have been confirmed, double-
    spending those coins?
    ▪ Erode confidence in Bitcoin, badly affecting its price?
    ▪ The use of off-chain financial market instruments (e.g.
    derivatives) could make the attack economically rationale
    ▪ Networks with total hash rate significantly smaller than other
    networks using same hashing algorithm are very fragile
    © 2019 Digital Gold Institute



    ✓ ✓
    57/109

    View full-size slide

  58. Nakamoto Consensus
    Solves the double spending problem reaching decentralized
    consensus about the unspent (i.e. still spendable) coin database:
    ▪ cryptographic protection against invalid transactions
    ▪ sequential transaction ordering (mining) against double spend
    transactions
    ▪ auditable append-only ledger (blockchain) as tamper evident
    tool
    ▪ economic incentives (coin creation) to make honest mining
    behavior more profitable than alternatives
    ▪ probabilistic finality: consensus happens probabilistically over
    long time scales (about 6 blocks, i.e. 1 hour)
    © 2019 Digital Gold Institute 58/109

    View full-size slide

  59. Blockchain as Log-file of DB Updates
    ▪ Blockchain is not meant to directly "store" or "distribute" data,
    but to guarantee that mutually distrusting parties ultimately will
    see the same data
    ▪ It is not a database, but a protocol for syncing databases
    ▪ It represents the log-file of sequential update operations to be
    applied to an empty database, in order to reach the data status
    agreed upon with distributed consensus (unspent coins)
    © 2019 Digital Gold Institute 59/109

    View full-size slide

  60. Blockchain and Consensus
    ▪ Blockchain is append-only and auditable
    ▪ Blockchain itself is tamper-evident, but it
    − does not solve the double-spending problem
    − does not guarantee ledger immutability
    ▪ It becomes tamper-resistant only because of consensus:
    centralized or distributed
    ▪ Bitcoin is hard money backed by thermodynamics (proof-of-
    work)
    © 2019 Digital Gold Institute 60/109

    View full-size slide

  61. The ratio of total work divided by estimate of hashrate at that time. Thus it's the amount of time it
    would take for an attacker with 100% of the hashrate to rewrite the entire blockchain.
    Proof-of-Work Equivalent Days
    © 2019 Digital Gold Institute http://bitcoin.sipa.be/powdays-ever.png 61/109

    View full-size slide

  62. Blockchain and Distributed Consensus
    ▪ Blockchain is just one of the components used to elegantly
    create the digital scarcity of an intrinsic native digital asset
    ▪ A native digital asset is crucial for reaching distributed
    consensus
    ▪ Blockchain is an idiosyncratic technology for distributed
    consensus, it does not have general purpose applicability, for
    centralized consensus better use databases
    © 2019 Digital Gold Institute 62/109

    View full-size slide

  63. Table of Contents
    1. Hash Functions
    2. Simplified Digital Currency
    3. Distributed Consensus
    4. Mining
    5. P2P Network
    6. Protocol Governance
    © 2019 Digital Gold Institute 63/109

    View full-size slide

  64. How to Mine
    1. Join the network
    2. Listen for transactions and validate them (to also forward them
    would be nice)
    3. Listen for new blocks and validate them (to also forward them
    would be nice)
    4. Assemble a new valid block, finding the nonce that make it valid
    5. Broadcast the new block and hope the network will accept it,
    building the next block on the top of it
    © 2019 Digital Gold Institute 64/109

    View full-size slide

  65. Mining Economics
    Profit = block reward + Tx fees - electricity cost – hardware cost
    Hash Rate
    Difficulty
    Millions of
    TeraHash/Second
    2015 2016
    2014
    Last
    Halving
    Fast
    recovery
    reward depends on
    global hash rate
    © 2019 Digital Gold Institute 65/109

    View full-size slide

  66. Mining Hardware
    Driven by the search of lower power consumption and higher
    hashing rate:
    ▪ CPUs (Computer Processing Unit) were used in 2009
    ▪ GPUs (Graphical Processing Unit) proved to perform better in
    2010
    Moved later to special purpose energy-efficient hardware:
    ▪ FPGAs (Field Programmable Gate Array) were programmed for
    SHA256 hashing and surpassed GPU in 2011
    ▪ ASICs (Application Specific Integrated Circuit), designed and
    manufactured for the specific purpose of SHA256 computations,
    were introduced in 2013 for Bitcoin and are now the standard
    © 2019 Digital Gold Institute 66/109

    View full-size slide

  67. Solving Hash Puzzles is Probabilistic
    For a single miner:
    ▪ Probability of solving next block:
    = ℎℎ
    ℎℎ
    ▪ Mean time to find a block:
    10

    ▪ 0.01% of the hash rate → one block every 69 days
    © 2019 Digital Gold Institute 67/109

    View full-size slide

  68. Mining Pools
    ▪ Statistical variance kills small miners: join a pool!
    ▪ Pool manager assembles the block with the coinbase going to
    himself
    ▪ Pool participants all attempt to mine the same block
    ▪ The pool manager distributes revenues to members based on
    how much work they have performed
    ▪ Mining pool protocols (API for fetching blocks and submitting
    shares): Stratum, Getwork, Getblockshare
    © 2019 Digital Gold Institute 68/109

    View full-size slide

  69. Mining Shares
    ▪ Shares measure the work performed by each pool participant
    ▪ Shares are “near-valid solutions”: hash value is low, but not
    enough to be a valid nonce at the current difficulty level
    ▪ If a participant would try to steal, changing the pool coinbase
    address, the resulting different block data would prevent the
    generation of valid mining shares, i.e. solo mining
    © 2019 Digital Gold Institute 69/109

    View full-size slide

  70. Near-valid Solutions: Mining Shares
    4AA087F0A52ED2093FA816E53B9B6317F9B8C1227A61F9481AFED67301F2E3FB
    D3E51477DCAB108750A5BC9093F6510759CC880BB171A5B77FB4A34ACA27DEDD
    00000000008534FF68B98935D090DF5669E3403BD16F1CDFD41CF17D6B474255
    BB34ECA3DBB52EFF4B104EBBC0974841EF2F3A59EBBC4474A12F9F595EB81F4B
    00000000002F891C1E232F687E41515637F7699EA0F462C2564233FE082BB0AF
    0090488133779E7E98177AF1C765CF02D01AB4848DF555533B6C4CFCA201CBA1
    460BEFA43B7083E502D36D9D08D64AFB99A100B3B80D4EA4F7B38E18174A0BFB
    000000000000000078FB7E1F7E2E4854B8BC71412197EB1448911FA77BAE808A
    652F374601D149AC47E01E7776138456181FA4F9D0EEDD8C4FDE3BEF6B1B7ECE
    785526402143A291CFD60DA09CC80DD066BC723FD5FD20F9B50D614313529AF3
    000000000041EE593434686000AF77F54CDE839A6CE30957B14EDEC10B15C9E5
    9C20B06B01A0136F192BD48E0F372A4B9E6BA6ABC36F02FCED22FD9780026A8F
    © 2019 Digital Gold Institute 70/109

    View full-size slide

  71. Mining Pool History
    ▪ First pools appeared in late-2010 (GPU era)
    ▪ By 2014: around 90% of mining became pool-based
    ▪ June 2014: GHash.io exceeded 50%. Participants voluntarily left
    GHash.io; Ghash.io does not exist anymore
    © 2019 Digital Gold Institute 71/109

    View full-size slide

  72. Mining pools (as of August 2014)
    ▪ Click the chart for update
    © 2019 Digital Gold Institute 72/109

    View full-size slide

  73. Mining pools (as of May 2018)
    A dynamically changing
    oligopoly
    © 2019 Digital Gold Institute 73/109

    View full-size slide

  74. Game-theoretic Analysis of Mining
    Strategic issues
    1. transactions to be included in
    a block
    Default behavior
    1. any transaction above
    minimum fee; if block is full
    transactions with highest
    fees
    2. block to mine on top of 2. head of the valid chain with
    higher cumulated difficulty
    3. criteria for choosing between
    equivalent valid blocks
    3. first block heard
    4. when to announce new
    blocks
    4. immediately after finding
    them
    © 2019 Digital Gold Institute 74/109

    View full-size slide

  75. Game-theoretic Analysis of Mining
    ▪ Assume a miner control 0 < α < 1 of hashing power
    ▪ Can he profit from a non-default strategy?
    There is an ongoing research…
    © 2019 Digital Gold Institute 75/109

    View full-size slide

  76. Table of Contents
    1. Hash Functions
    2. Simplified Digital Currency
    3. Distributed Consensus
    4. Mining
    5. P2P Network
    6. Protocol Governance
    © 2019 Digital Gold Institute 76/109

    View full-size slide

  77. Bitcoin: A Peer-to-Peer Network
    ▪ Ad-hoc network with random topology
    ▪ Open to anyone, low barrier to entry
    ▪ New nodes can join at any time
    ▪ Nodes can leave the network: non-responding nodes are
    forgotten after 3 hours
    ▪ All nodes are equal, but not all nodes must be mining node
    ▪ Ad-hoc protocol, it runs on TCP port 8333
    ▪ Optional: to be a connectable node, port 8333 must be open
    ▪ Accepts Remote Procedure Call (RPC) on port 8332
    © 2019 Digital Gold Institute 77/109

    View full-size slide

  78. Bitcoin Network: Connectable Nodes
    © 2019 Digital Gold Institute
    https://bitnodes.earn.com/
    78/109

    View full-size slide

  79. Joining the Bitcoin P2P network
    © 2019 Digital Gold Institute
    1
    6
    4
    7
    3
    5
    2
    8
    Hello World! I’m
    ready to Bitcoin!
    getaddr()
    1, 7
    getaddr()
    getaddr()
    now I will download and
    verify the blockchain
    79/109

    View full-size slide

  80. Transaction Propagation
    © 2019 Digital Gold Institute
    1
    6
    4
    7
    3
    5
    2
    8
    New Tx!
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    Already
    heard that!
    80/109

    View full-size slide

  81. Transaction Propagation
    Nodes relay a new transaction if:
    ▪ it tries to spend unspent coins
    ▪ it has valid signatures
    ▪ it does not create coins out of nothing (TxIns >= TxOuts)
    ▪ it is valid with respect to the running protocol client
    (Non-mandatory) sanity checks:
    ▪ transaction has not been already relayed (avoid infinite loops)
    ▪ transaction does not conflict with others just relayed (do not
    help double-spend attempts)
    ▪ script is standard (more details in the lesson about transactions)
    © 2019 Digital Gold Institute 81/109

    View full-size slide

  82. Nodes may differ on transaction pool
    © 2019 Digital Gold Institute
    1
    6
    4
    7
    3
    5
    2
    8
    A→B
    A→B
    A→B
    A→B
    A→B
    A→B
    New Tx!
    A→C
    A→C
    A→C
    A→B
    A→C
    A→C
    A→B
    A→C
    82/109

    View full-size slide

  83. Race Conditions
    ▪ Transactions or blocks may conflict: default behavior is to accept
    the first one heard
    ▪ Network position matters
    ▪ Miners may implement other logic!
    © 2019 Digital Gold Institute 83/109

    View full-size slide

  84. Block Propagation
    Nodes relay a new block if:
    ▪ it includes valid transactions only
    ▪ it meets the difficulty target
    ▪ it is valid with respect to the running protocol client
    (Non-mandatory) sanity check, in order not to help forks:
    ▪ it builds on the chain with most accumulated difficulty
    © 2019 Digital Gold Institute 84/109

    View full-size slide

  85. Block Propagation Time
    © 2019 Digital Gold Institute
    Source: Yonatan Sompolinsky and Aviv Zohar: “Accelerating Bitcoin’s Transaction Processing” 2014
    85/109

    View full-size slide

  86. Full Nodes and Software Diversity
    © 2019 Digital Gold Institute
    https://coin.dance/nodes/all
    86/109

    View full-size slide

  87. Bitcoin Core Software
    ▪ Direct continuation of Nakamoto’s 2009 open source (MIT license)
    code: https://github.com/bitcoin/bitcoin
    ▪ Most widely used Bitcoin software: reference implementation
    ▪ Even those who do not use Core, they basically follow its lead on
    rules: de facto Bitcoin rule book
    ▪ Multiple implementations are problematic for consensus: if they differ,
    which one is right?
    See https://blog.bitmex.com/bitcoin-cores-competition/
    https://bitcoincore.org
    “I don't believe a second, compatible implementation of Bitcoin will ever
    be a good idea. So much of the design depends on all nodes getting
    exactly identical results in lockstep that a second implementation would
    be a menace to the network. The MIT license is compatible with all other
    licenses and commercial uses, so there is no need to rewrite it from a
    licensing standpoint.” (Satoshi Nakamoto)
    https://bitcointalk.org/index.php?topic=195.msg1611#msg1611
    © 2019 Digital Gold Institute 87/109

    View full-size slide

  88. Bitcoin Testnet
    Alternative Bitcoin network for testing purposes: there have been three
    generations, the current one is Testnet3
    Compared to mainnet, testnet has:
    ▪ smaller blockchain size: about 24GB vs 222GB (as of March 2019)
    ▪ lower mining difficulty
    ▪ no market value
    ▪ default protocol listen port is 18333, instead of 8333
    ▪ default RPC connection port is 18332, instead of 8332
    ▪ different addresses (more details in the lesson about addresses)
    ▪ scripts can be non-standard (more details in the lesson about
    transactions)
    © 2019 Digital Gold Institute 88/109

    View full-size slide

  89. Homework
    Create a testnet wallet (https://en.bitcoin.it/wiki/testnet)
    ▪ bitcoin core https://bitcoincore.org/en/download is the best
    approach (beware that downloading mainnet, while
    wholeheartedly encouraged, requires much more hard drive
    space and it is not necessary for this assignment)
    ▪ electrum https://electrum.org/ is an easier approach: it does
    not require full blockchain download as it relies on servers
    Get free testnet bitcoins from a faucet listed at
    https://en.bitcoin.it/wiki/testnet, then send some to
    tb1q9u20ha7fg2nxm8d2usre2v6fgq44dsgnupy5ze
    and provide the transaction ID (even better: check your
    transaction ID at https://blockstream.info/testnet and send the
    corresponding link)
    © 2019 Digital Gold Institute 89/109

    View full-size slide

  90. Table of Contents
    1. Hash Functions
    2. Simplified Digital Currency
    3. Distributed Consensus
    4. Mining
    5. P2P Network
    6. Protocol Governance
    © 2019 Digital Gold Institute 90/109

    View full-size slide

  91. Some Bitcoin Protocol Limits
    ▪ 1MB blocksize (SegWit: 4MB block weight)
    ▪ 20,000 signature operations per block (SegWit: 80,000)
    ▪ 7 transactions/sec (VISA: 2,000-10,000 up to 60,000)
    ▪ One signature algorithm (ECDSA)
    ▪ Two hash functions (SHA256, RIPEMD)
    ▪ 100M satoshis per bitcoin
    © 2019 Digital Gold Institute 91/109

    View full-size slide

  92. Bitcoin Improvement Proposals
    ▪ “formal” proposals for changes to Bitcoin
    ▪ include technical spec and rationale
    ▪ each BIP has an assigned number
    ▪ each BIP has a champion to evangelize and coordinate
    ▪ also: informational BIPs, process-oriented BIPs
    © 2019 Digital Gold Institute 92/109

    View full-size slide

  93. Major Protocol Upgrades
    • P2SH (BIP16; 2012)
    • CSV/CLTV: (BIP65, BIP112; 2014-2015)
    • SegWit (BIP141, BIP143, BIP144, BIP145, BIP147, BIP148,
    BIP173; 2015-2017)
    © 2019 Digital Gold Institute 93/109

    View full-size slide

  94. Extend Rules: the Hard-fork Change
    © 2019 Digital Gold Institute
    1
    6
    4
    7
    3
    5
    2
    8
    I found a nifty
    new block!
    Block 24
    Block 24
    Block 24
    Block 24
    Block 24
    Block 23
    Block 23
    Block 23
    Block 23
    Block 23
    Block 23
    Block 23
    Block 23
    24
    24
    24
    24
    That’s crazy
    talk!!
    That’s crazy
    talk!!
    PROBLEM: Old nodes
    will never catch up
    94/109

    View full-size slide

  95. Hard Fork
    © 2019 Digital Gold Institute
    Valid under A rules
    Invalid under B rules
    Valid under B rules
    Invalid under A rules
    “Bitcoin”
    “A-coin”
    “B-coin”
    95/109

    View full-size slide

  96. After a Hard Fork
    If fork was meant to start an alternative coin (altcoin):
    ▪ altcoin goes its separate way
    ▪ branches coexist
    If fork reflected a fight over future of Bitcoin:
    ▪ branches fight for market share
    ▪ branches fight to be seen as “the real Bitcoin”
    ▪ probably one branch wins, one melts away
    © 2019 Digital Gold Institute 96/109

    View full-size slide

  97. Restrict Rules: the Soft-fork Change
    We can add new features while restricting rules!
    ▪ Need majority of nodes to enforce new rules
    ▪ Old nodes are not forced to upgrade: they do approve blocks
    created according to new rules
    ▪ Old nodes might mine blocks invalid according to the new rules:
    this is an economic incentive for them to join the soft-fork
    © 2019 Digital Gold Institute 97/109

    View full-size slide

  98. Major Protocol Bugs and Their Fixes
    • OP_CHECKMULTISIG extra value from stack: no fix (Jan 2009)
    • ~184 billion BTC created: CVE-2010-5139 (Aug 2010)
    • 24 block hard-fork in LevelDB upgrade: BIP50, CVE-2013-3220
    (Mar 2013)
    • unbounded monetary supply: BIP42 (Apr 2014)
    • potential coin inflation: CVE-2018-17144 (Sep 2018)
    © 2019 Digital Gold Institute 98/109

    View full-size slide

  99. Unbounded Monetary Supply Fixed by BIP42
    © 2019 Digital Gold Institute https://github.com/bitcoin/bips/tree/master/bip-0042 99/109

    View full-size slide

  100. Bitcoin Protocol/Network Governance
    P2P networks are open to anyone, with low barrier to enter
    ▪ Who has the authority to maintain the protocol (data formats,
    transaction/block validity, etc.)?
    ▪ Who can change the rules of the system and/or update
    software?
    ▪ What if a change is controversial?
    ▪ If there is a negotiation about rule-setting, who controls the
    outcome
    © 2019 Digital Gold Institute 100/109

    View full-size slide

  101. Bitcoin Foundation
    ▪ Founded 2012
    ▪ Used to pay some core developers
    ▪ Talk to governments as “voice of Bitcoin”
    Miserably failed!
    Bitcoin does not want representative or central governance
    © 2019 Digital Gold Institute 101/109

    View full-size slide

  102. Bitcoin Ecosystem
    ▪ Core Developers
    ▪ Users and Investors
    ▪ Miners
    ▪ Industry
    Centralization vs. decentralization: competing paradigms that
    underlie many digital technologies (e.g. e-mail is a decentralized
    protocol, dominated by centralized webmail service providers)
    © 2019 Digital Gold Institute 102/109

    View full-size slide

  103. Bitcoin Ecosystem: Core Developers
    ▪ They write the rulebook: as almost everybody uses their code,
    everybody follows their rules
    ▪ Their proposed changes must be accepted by miners, industry,
    and users
    ▪ Anyone can fork the software at any time, not just “core” devs
    © 2019 Digital Gold Institute 103/109

    View full-size slide

  104. Bitcoin Ecosystem: Miners
    ▪ Mining is a decentralized transaction ordering service: miners
    sequence transactions, writing a transaction history consistent
    with their consensus rules
    ▪ Miners can ignore developers’ changes
    ▪ Miners could force their changes
    Hard to move forward without some serious dev support
    ▪ Investors and industry can ignore miners’ rules and adopt a
    different chain
    © 2019 Digital Gold Institute 104/109

    View full-size slide

  105. Bitcoin Ecosystem: Users and Investors
    ▪ They determine whether bitcoin has any value
    ▪ In case of hard-fork, they decide which branch wins
    If they do not like a rule change:
    ▪ Centralized currency: they have the right to exit
    ▪ Bitcoin: they have the right to fork the rules
    Right to fork is more empowering than right to exit

    community retains more power
    © 2019 Digital Gold Institute 105/109

    View full-size slide

  106. Bitcoin Ecosystem: Industry
    Exchanges, Payment Processors, Merchants, etc.
    ▪ They serve users and investors, facilitating adoption
    ▪ They sustain ecosystem growth
    ▪ They handle most transactions
    Industry can influence and/or is influenced by users and investors:
    one party hardly lives without the other party
    © 2019 Digital Gold Institute 106/109

    View full-size slide

  107. Bitcoin: Protocol Governance and Evolution
    Governance is a dynamic distributed process involving multiple agents
    Bitcoin is the ledger of not-previously-spent, validly signed transactions
    contained in the chain of blocks that begins with the genesis block,
    follows the 21-million coin creation schedule, and has the most
    cumulative double-SHA256-proof-of-work (Gavin Andresen)
    http://gavinandresen.ninja/a-definition-of-bitcoin
    Close… but no cigar!
    Bitcoin is what
    the relevant economic majority decide Bitcoin to be
    © 2019 Digital Gold Institute 107/109

    View full-size slide

  108. Bibliography
    ▪ A. Narayanan, et al., “Bitcoin and Cryptocurrency Technologies”
    http://bitcoinbook.cs.princeton.edu/
    − chapter 1 «Introduction to Cryptography & Cryptocurrencies»
    − chapter 2 «How Bitcoin Achieves Decentralization»
    − chapter 5 «Bitcoin Mining»
    − chapter 7 «Community, Politics, and Regulation»
    − chapter 8 «Alternative Mining Puzzles»
    ▪ Pedro Franco, “Understanding Bitcoin”, Wiley
    − chapter 9 «Mining»
    ▪ Andreas Antonopoulos, “Mastering Bitcoin” 2nd edition, O'Reilly
    https://github.com/bitcoinbook/bitcoinbook
    − chapter 10 «Mining and Consensus»
    © 2019 Digital Gold Institute 108/109

    View full-size slide

  109. Takeaways
    ▪ Hash Functions generate unique identifiers for any given input
    ▪ Distributed Consensus is the hard part of a decentralized
    solution/currency
    ▪ Proof-of-Work mining is what makes blockchain immutable or at
    least tamper-resistant
    ▪ P2P networking makes blockchain solutions really distributed
    ▪ Distributed protocol governance is a dynamic process involving
    multiple agents
    ▪ Bitcoin is what the economic relevant majority decide
    Bitcoin to be
    © 2019 Digital Gold Institute 109/109

    View full-size slide

  110. Ferdinando M. Ametrano
    Executive Director
    [email protected]
    Paolo Mazzocchi
    Chief Operating Officer
    [email protected]
    www.github.com/dginst
    www.facebook.com/DigitalGoldInstitute
    www.twitter.com/DigitalGoldInst
    www.dgi.io/feed.xml
    [email protected]
    www.dgi.io
    www.linkedin.com/company/digital-gold-institute
    "Scarcity in the Digital Domain"

    View full-size slide