Blockchain and Distributed Consensus

Second lesson for the Bitcoin and Blockchain Technology course of Milano-Bicocca and Politecnico di Milano

www.ametrano.net/bbt/

1. Bitcoin and Blockchain Technology Blockchain, Mining and Distributed Consensus v2019.04.03 Comments, corrections, and questions: https://drive.google.com/open?id=1_rGy7wdI8iWx6w6LG_CGCmmLnAIFhncz © 2019 Digital Gold Institute
2. Credit In the following, some slides are from: A. Narayanan, et al., “Bitcoin and Cryptocurrency Technologies”, Princeton Univ Press (2016) http://bitcoinbook.cs.princeton.edu/ © 2019 Digital Gold Institute 2/109
3. Table of Contents 1. Hash Functions 2. Simplified Digital Currency 3. Distributed Consensus 4. Mining 5. P2P Network 6. Protocol Governance © 2019 Digital Gold Institute 3/109
4. Hash Functions A hash function, e.g. SHA256, is a map from the set of input data to the output set of hash values: SHA256(“Hello, world!”) = 315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3 Small differences in the input data produce large differences in the result: SHA256(“Hello, world.”) = f8c3bf62a9aa3e6fc1619c250e48abe7519373d3edf41be62eb5dc45199af2ef © 2019 Digital Gold Institute 4/109
5. Cryptographic Hash Functions (1/2) ▪ Arbitrary message size: h(x) can be applied to message x of any size ▪ Fixed output length: h(x) produces a hash value of fixed length (256 bits for SHA256) ▪ Efficiency: h(x) is relatively easy to compute © 2019 Digital Gold Institute 5/109
6. Cryptographic Hash Functions (2/2) ▪ One-wayness (preimage resistance): given h(x), it is computationally infeasible to find x ▪ Weak collision resistance (second pre-image resistance): given x it is computationally infeasible to find y!=x such that h(y)=h(x) ▪ (Strong) Collision resistance: it is computationally infeasible to find (x, y!=x) such that h(y)=h(x) © 2019 Digital Gold Institute 6/109
7. Computationally Infeasible Collisions ▪ The size of the possible hash function output values is smaller than the size of possible input data. Therefore, many input data must share the same hash function output value: collisions do exist... ▪ Try 2^130 randomly chosen inputs: 99.8% chance that two of them will collide. Anyway, to try them is computationally unfeasible ▪ For some hash functions computationally feasible ways to find collisions have been found; this is not the case for “good” functions © 2019 Digital Gold Institute 7/109
8. Merkle–Damgård Hash Construction A hash function is usually based on a one-way compression function; it is as resistant to collisions as its compression function 256 bits Init. Value cf 512 bits Msg (block1) cf 512 bits Msg (block2) Length Padding cf 512 bits Msg (blockN) 256 bits Hash Msg (block1) Msg (block2) Msg (blockN) …. © 2019 Digital Gold Institute 8/109
9. Hash as Message Digest ▪ If we know h(x) = h(y), then it is safe to assume that x = y ▪ Useful because h(x) can be much smaller than x, e.g. often used as download integrity check © 2019 Digital Gold Institute 9/109
10. Puzzle Friendliness Given and a target set , to find from high min-entropy distribution such that ℎ || ∈ there is no solving strategy better than trying random values of Min-entropy measures how unlikely you are to guess a value on your first try. If the probability of guessing right is p, then min- entropy is defined as −log2 . For example, for a fair coin toss (i.e. the status of a single bit), you'd have p = 0.5, giving a min-entropy of 1. A uniformly random 256-bit string would have −log2 2−256 = 256 bits of min entropy © 2019 Digital Gold Institute 10/109
11. Hash Puzzle: Partial Hash Inversion ▪ Find nonce so that ℎ || is small, e.g. starts with zero ▪ The smaller the target space, the harder to find a solution ▪ For good hash functions, the problem can only be solved by brute force trials ▪ It is trivial to verify the solution: it is just one hash function evaluation © 2019 Digital Gold Institute 11/109
12. Hash Pointer The cryptographic hash of the data can be used as pointer to the data itself With hash pointer one: ▪ gets the data back ▪ verifies that data has not changed (data) H( ) © 2019 Digital Gold Institute 12/109
13. Hash Pointers Data Structures Hash pointers can be used in any pointer-based data structure with no cycles Binary Tree → Merkle Tree Linked list → Blockchain © 2019 Digital Gold Institute 13/109
14. Blockchain: Hash Pointer Linked List data2 prev: H( ) data1 prev: H( ) data0 Genesis Block H( ) tamper evident data structure © 2019 Digital Gold Institute 14/109
15. Merkle Tree: Hash Pointer Binary Tree H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) H( ) data0 data1 data2 data3 data4 data5 data6 data7 © 2019 Digital Gold Institute inclusion evident data structure 15/109
16. Merkle Tree: Hash Pointer Binary Tree H( ) H( ) H( ) H( ) H( ) H( ) H( ) data3 Data3 inclusion proof consists only of 3 hash values ▪ O(log n) inclusion proof If sorted: ▪ O(log n) non-inclusion proof © 2019 Digital Gold Institute inclusion evident data structure 16/109
17. Homework ▪ Find the nonce that appended to your name obtains a SHA256 hash value starting with 7 zeros [As in FirstnameLastnameNonce, test at https://emn178.github.io/online-tools/sha256.html] ▪ How many numbers in the range 0, result into hash values starting with one or more zeros? ▪ Produce the histogram with the frequency of hash values starting with 1, 2, 3, 4, 5, 6, and 7 zeros in the 0, range Use the Python3 script hash_puzzle.py at github.com/dginst/bbt/py-scripts/ © 2019 Digital Gold Institute 17/109
18. Table of Contents 1. Hash Functions 2. Simplified Digital Currency 3. Distributed Consensus 4. Mining 5. P2P Network 6. Protocol Governance © 2019 Digital Gold Institute 18/109
19. Hash value of this data is the coin “serial number” Central Bank Coin Issued to Carol Coin Creation signed by p CentralBanker Spendable by P Carol Owner of the spendable coin P Public Key p private key © 2019 Digital Gold Institute 19/109
20. Carol Spends the Coin To Alice Coin Creation signed by p CentralBanker Spendable by P Carol Spend H( ) signed by p Carol Spendable by P Alice © 2019 Digital Gold Institute New coin with different “serial number” ✗ P Public Key p private key This spent coin is now unspendable Owner of the spendable coin 20/109
21. Alice Spends the Coin To Bob Spend H( ) signed by p Alice Spendable by P Bob © 2019 Digital Gold Institute P Public Key p private key This spent coin is now unspendable This spent coin is now unspendable New coin with another different “serial number” Owner of the spendable coin Spend H( ) signed by p Carol Spendable by P Alice Coin Creation signed by p CentralBanker Spendable by P Carol ✗ ✗ 21/109
22. Double Spending Attack For the network both transactions are equal. Two alternative new coins: which one is valid? Spend H( ) signed by p Alice Spendable by P Alice © 2019 Digital Gold Institute P Public Key p private key This spent coin is now unspendable This spent coin is now unspendable Spend H( ) signed by p Alice Spendable by P Bob Spend H( ) signed by p Carol Spendable by P Alice Coin Creation signed by p CentralBanker Spendable by P Carol ✗ ✗ 22/109
23. Hash Chain Signed By Central Banker Valid Chain Head H( ) signed by p CentralBanker © 2019 Digital Gold Institute P Public Key p private key Coin Creation signed by p CentralBanker Spendable by P Carol ✗ Spend H( ) signed by p Alice Spendable by P Bob Spend H( ) signed by p Carol Spendable by P Alice Owner of the spendable coin Double-spend attack can be solved resorting to a central authority ✗ ✓ 23/109
24. Alice Pays Bob ▪ When Alice wants to pay Bob, she broadcasts the transaction to the network ▪ Bob is not required to be a network node or to actively monitor the network ▪ Bob will discover the transaction next time he will monitor the network © 2019 Digital Gold Institute 24/109
25. Coinbase Transaction TransID:71 Coin Creation signed by p CentralBanker Output num value Pay to: 0 0.8 P0 1 3.2 P1 2 1.1 P2 coinID 71(0) coinID 71(1) coinID 71(2) © 2019 Digital Gold Institute P Public Key, p private key 25/109
26. Spending Transaction TransID:72 Spend 42(0), 71(2) signed by p 42(0) , p2 71(2) Output num value Pay to: 0 0.5 P3 1 2.1 P4 2 1.7 P5 Valid only if 42(0) and 71(2) are: ▪ valid ▪ not already spent ▪ signed by their owners Valid only if: Input coins >= output coins i.e. 42(0)+71(2) >= 72(0)+72(1)+72(2) coinID 72(0) coinID 72(1) coinID 72(2) © 2019 Digital Gold Institute P Public Key, p private key 26/109
27. Immutable Coins ▪ Coins cannot be transferred, subdivided, or combined ▪ Coins can only be entirely consumed as input to a transaction (TxIn) that creates new output coins (TxO) ▪ The set of current unspent coins is referred to as UTxO (Unspent Transaction Outputs) © 2019 Digital Gold Institute 27/109
28. Transaction Bundling ▪ Transactions are bundled in blocks for convenience: shorter chain, single unit of validation work, faster history validation ▪ In a block transactions are packaged in a Merkle Tree: for now, we neglect that © 2019 Digital Gold Institute prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx 28/109
29. Remove the Central Banker Crucial questions: ▪ Can the digital currency be decentralized? ▪ Can it be functional without a trusted central party? © 2019 Digital Gold Institute 29/109
30. Table of Contents 1. Hash Functions 2. Simplified Digital Currency 3. Distributed Consensus 4. Mining 5. P2P Network 6. Protocol Governance © 2019 Digital Gold Institute 30/109
31. Consensus Needed By Bitcoin Who has authority over which transactions are valid? Who maintains the ledger? Who creates new bitcoins? Certainty is needed about: ▪ Past transactions ▪ Coin generation ▪ Spendable (i.e. still unspent) coins (UTXO) ▪ Who owns coins (i.e. how to spend them) © 2019 Digital Gold Institute 31/109
32. Sybil Attack ▪ In P2P networks identities can be forged: a single agent can present itself as multiple nodes ▪ Reputation systems are easy to be subverted ▪ Distributed consensus cannot rely on node identity ▪ Moreover, pseudonymity is a goal of Bitcoin © 2019 Digital Gold Institute 32/109
33. Distributed Consensus Correct nodes must decide about a value for a variable or a course of action: ▪ once the nodes reach a decision, that must be final (correctness) ▪ the network will make progress if some fraction of the network is non-faulty (liveness) The technical requirements of consensus are: ▪ every correct node eventually decides some value (termination) ▪ all correct nodes decide on the same value (agreement) ▪ if all correct nodes propose the same value, then any correct node decides that value (integrity) © 2019 Digital Gold Institute 33/109
34. Distributed Consensus: Impossibility Result Consensus in distributed systems is hard because: ▪ Nodes may be faulty for technical malfunction (crash failure) or arbitrary malicious activity (Byzantine failure) ▪ Not all pairs of nodes are connected ▪ There is latency in communication between nodes ▪ There is no notion of global time Fischer, Lynch, and Paterson have proved that reaching consensus in an asynchronous network cannot be guaranteed, even on something as simple as a binary value, when even just a single node may be faulty or malicious © 2019 Digital Gold Institute 34/109
35. The Byzantine Generals' Problem ▪ Generals can communicate using messengers, cannot have a summit ▪ There are traitors amongst them ▪ Must decide unanimously whether to attack ▪ Success (i.e. fault tolerance) is achieved if the loyal generals can agree on their strategy, whatever it might be © 2019 Digital Gold Institute 35/109
36. Practical Byzantine Fault Tolerance ▪ No network is unboundedly asynchronous ▪ Practical Byzantine Fault Tolerance (PBFT) consensus can be achieved when a predominant majority of nodes are honest [Lamport, Marshall, Pease], even in the case of Byzantine failures ▪ Practically, networks being often effectively asynchronous, in some way distributed consensus is always implicitly probabilistic © 2019 Digital Gold Institute 36/109
37. Simplified Consensus Algorithm 1. New transactions are broadcasted to all nodes 2. Each node collects new transactions into a block 3. In each round, a random node is picked (as in a lottery or raffle, without verifying identities) and this node proposes the next block in the chain 4. Other nodes accept the block only if it includes valid transactions (spend unspent coins, have valid signatures) 5. Nodes express their implicit acceptance of the block by including its hash in the next block they create © 2019 Digital Gold Institute 37/109
38. How To Pick a Random Node? (while preventing Sybil attacks) ▪ Let nodes compete for the right to create a block ▪ Nodes must provide Proof-of-Something Selection chances increase proportionally to a resource that cannot be faked: ▪ computing power: Proof-of-Work ▪ coin ownership: Proof-of-Stake © 2019 Digital Gold Institute 38/109
39. Proof-of-Something Trade-offs Proof-of-Work (PoW): ▪ Must be incentivized someway, as it has operational costs ▪ Maintaining alternative transaction histories reduces the chance of block finalization Proof-of-Stake (PoS): ▪ Does not have significant operational costs ▪ Maintaining alternative transaction histories has no intrinsic penalty (nothing is at stake), some penalization mechanism must be introduced ex-post © 2019 Digital Gold Institute 39/109
40. Proof-of-Work is the Most Secure ▪ Even if one party controls all PoW, an attack always has an energy cost ▪ In non-PoW systems, zero-cost attacks are always theoretically possible: one must rely on the soundness of an after-the-fact punishment © 2019 Digital Gold Institute 40/109
41. Proof-of-Work Proof-of-Work types: ▪ Challenge-Response protocol: e.g. CAPTCHA ▪ Solution-Verification protocol: e.g. partial has inversion Nakamoto uses the latter, borrowing the idea from Hashcash (Adam Back 1997, 2002) © 2019 Digital Gold Institute 41/109
42. Partial Hash Inversion Find nonce so that h(prev_hash || {Tx, …, Tx} || nonce) is very small ▪ The smaller the target space, the harder to find a solution ▪ For good hash functions, the problem can only be solved by brute force trials ▪ It is always trivial to verify the solution: it is just one hash function evaluation prev_hash nonce Tx Tx © 2019 Digital Gold Institute 42/109
43. Partial Hash Inversion Difficulty ▪ Goal: 10 minutes average time between blocks ▪ Protocol enforces hash puzzle difficulty recalculation every 2016 blocks (about two weeks): the bitcoin software automatically takes care of it © 2019 Digital Gold Institute 43/109
44. Mining difficulty over time bitcoinwisdom.com © 2019 Digital Gold Institute 44/109
45. Time to find a block 10 minutes 2016 blocks ~2 weeks bitcoinwisdom.com © 2019 Digital Gold Institute 45/109
46. The Reference Chain ▪ At any moment the reference chain is the longest valid one ▪ More precisely: the reference chain is the one with higher cumulative difficulty ▪ Usually the two conditions are equivalent, but the former could be gamed lowering difficulty © 2019 Digital Gold Institute 46/109
47. Blockchain Forks and Height ▪ Genesis Block ▪ Orphan Block ▪ Main Chain Block The reference chain here has height 8 (i.e. nine blocks) © 2019 Digital Gold Institute 47/109
48. Orphan Blocks and Orphan Chains Orphan blocks are quite common: most common heuristic suggests to wait six confirmations before considering a transaction settled Orphaned Blocks History of orphan blocks 97 102 101 101 98 98 98 99 99 100 100 100 103 104 105 6 confirmations Wait 6 confirmations for important transactions © 2019 Digital Gold Institute 48/109
49. Transaction Confirmation prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx CA → B transaction first heard: 0 confirmations prev_hash nonce CA → B Tx Tx 1 confirmation © 2019 Digital Gold Institute 49/109
50. Double Spending Attack For the network both transactions are equal. Which one is valid? © 2019 Digital Gold Institute CA → B transaction first heard: 0 confirmations prev_hash nonce CA → A Tx Tx prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx prev_hash nonce CA → B Tx Tx 1 confirmation 50/109
51. Multiple Transaction Confirmations Double-spend success chances decrease exponentially with the number of confirmations © 2019 Digital Gold Institute prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx CA → B transaction first heard: 0 confirmations prev_hash nonce CA → B Tx Tx 1 confirmation prev_hash nonce Tx Tx Tx 3 confirmations prev_hash nonce Tx Tx Tx 2 confirmations prev_hash nonce CA → A Tx Tx 51/109
52. Incentive For Honesty ▪ Can we go beyond distributed consensus protocol? ▪ Can we utilize the fact that the currency has value? © 2019 Digital Gold Institute prev_hash nonce Tx Tx Tx prev_hash nonce Tx Tx Tx CA → B transaction first heard: 0 confirmations prev_hash nonce CA → B Tx Tx 1 confirmation prev_hash nonce Tx Tx Tx 3 confirmations prev_hash nonce Tx Tx Tx 2 confirmations prev_hash nonce CA → A Tx Tx Can we reward the nodes that created these blocks? Can we penalize the node that created this block? 52/109
53. Incentive 1: Block Reward ▪ The block creator includes a special coinbase transaction in the block, choosing at will the recipient address of this transaction ▪ Value is fixed: originally 50BTC, halves every 210,000 blocks (about 4 years) ▪ Block creator collects the reward only if the block ends up on long-term consensus branch! © 2019 Digital Gold Institute 53/109
54. Incentive 2: Transaction Fees ▪ Transaction creators can choose to make output value less than input value: the remainder is a transaction fee and goes to the block creator ▪ Necessary to avoid spam attack with many payment-to-self or dust transactions ▪ Necessary as blocks are filled up to MAX_BLOCK_SIZE capacity © 2019 Digital Gold Institute 54/109
55. Attacking the Blockchain An attacker wishing to change a given block would have to: ▪ apply a computational power equivalent to all the computational power spent from that block onward ▪ outrun the legitimate Bitcoin network that in the meantime would keep adding blocks to the original chain The attacker would need 51% of total computational power (actually less than 51%…) © 2019 Digital Gold Institute 55/109
56. 51% Attack Effects ▪ Change the block reward? ▪ Steal coins from existing address? ▪ Remove transactions from the P2P network? ▪ High barrier of entry: buy/build hardware, setup farm, etc. ▪ Rational economic agents are better off mining coins than attacking © 2019 Digital Gold Institute ✗ ✗ ✗ 56/109
57. 51% Attack Effects ▪ Prevent transactions from being confirmed? ▪ Censor (kind of DoS attack against) specific addresses? ▪ Reverse recent transactions that have been confirmed, double- spending those coins? ▪ Erode confidence in Bitcoin, badly affecting its price? ▪ The use of off-chain financial market instruments (e.g. derivatives) could make the attack economically rationale ▪ Networks with total hash rate significantly smaller than other networks using same hashing algorithm are very fragile © 2019 Digital Gold Institute ✓ ✓ ✓ ✓ ✓ 57/109
58. Nakamoto Consensus Solves the double spending problem reaching decentralized consensus about the unspent (i.e. still spendable) coin database: ▪ cryptographic protection against invalid transactions ▪ sequential transaction ordering (mining) against double spend transactions ▪ auditable append-only ledger (blockchain) as tamper evident tool ▪ economic incentives (coin creation) to make honest mining behavior more profitable than alternatives ▪ probabilistic finality: consensus happens probabilistically over long time scales (about 6 blocks, i.e. 1 hour) © 2019 Digital Gold Institute 58/109
59. Blockchain as Log-file of DB Updates ▪ Blockchain is not meant to directly "store" or "distribute" data, but to guarantee that mutually distrusting parties ultimately will see the same data ▪ It is not a database, but a protocol for syncing databases ▪ It represents the log-file of sequential update operations to be applied to an empty database, in order to reach the data status agreed upon with distributed consensus (unspent coins) © 2019 Digital Gold Institute 59/109
60. Blockchain and Consensus ▪ Blockchain is append-only and auditable ▪ Blockchain itself is tamper-evident, but it − does not solve the double-spending problem − does not guarantee ledger immutability ▪ It becomes tamper-resistant only because of consensus: centralized or distributed ▪ Bitcoin is hard money backed by thermodynamics (proof-of- work) © 2019 Digital Gold Institute 60/109
61. The ratio of total work divided by estimate of hashrate at that time. Thus it's the amount of time it would take for an attacker with 100% of the hashrate to rewrite the entire blockchain. Proof-of-Work Equivalent Days © 2019 Digital Gold Institute http://bitcoin.sipa.be/powdays-ever.png 61/109
62. Blockchain and Distributed Consensus ▪ Blockchain is just one of the components used to elegantly create the digital scarcity of an intrinsic native digital asset ▪ A native digital asset is crucial for reaching distributed consensus ▪ Blockchain is an idiosyncratic technology for distributed consensus, it does not have general purpose applicability, for centralized consensus better use databases © 2019 Digital Gold Institute 62/109
63. Table of Contents 1. Hash Functions 2. Simplified Digital Currency 3. Distributed Consensus 4. Mining 5. P2P Network 6. Protocol Governance © 2019 Digital Gold Institute 63/109
64. How to Mine 1. Join the network 2. Listen for transactions and validate them (to also forward them would be nice) 3. Listen for new blocks and validate them (to also forward them would be nice) 4. Assemble a new valid block, finding the nonce that make it valid 5. Broadcast the new block and hope the network will accept it, building the next block on the top of it © 2019 Digital Gold Institute 64/109
65. Mining Economics Profit = block reward + Tx fees - electricity cost – hardware cost Hash Rate Difficulty Millions of TeraHash/Second 2015 2016 2014 Last Halving Fast recovery reward depends on global hash rate © 2019 Digital Gold Institute 65/109
66. Mining Hardware Driven by the search of lower power consumption and higher hashing rate: ▪ CPUs (Computer Processing Unit) were used in 2009 ▪ GPUs (Graphical Processing Unit) proved to perform better in 2010 Moved later to special purpose energy-efficient hardware: ▪ FPGAs (Field Programmable Gate Array) were programmed for SHA256 hashing and surpassed GPU in 2011 ▪ ASICs (Application Specific Integrated Circuit), designed and manufactured for the specific purpose of SHA256 computations, were introduced in 2013 for Bitcoin and are now the standard © 2019 Digital Gold Institute 66/109
67. Solving Hash Puzzles is Probabilistic For a single miner: ▪ Probability of solving next block: = ℎℎ ℎℎ ▪ Mean time to find a block: 10 ▪ 0.01% of the hash rate → one block every 69 days © 2019 Digital Gold Institute 67/109
68. Mining Pools ▪ Statistical variance kills small miners: join a pool! ▪ Pool manager assembles the block with the coinbase going to himself ▪ Pool participants all attempt to mine the same block ▪ The pool manager distributes revenues to members based on how much work they have performed ▪ Mining pool protocols (API for fetching blocks and submitting shares): Stratum, Getwork, Getblockshare © 2019 Digital Gold Institute 68/109
69. Mining Shares ▪ Shares measure the work performed by each pool participant ▪ Shares are “near-valid solutions”: hash value is low, but not enough to be a valid nonce at the current difficulty level ▪ If a participant would try to steal, changing the pool coinbase address, the resulting different block data would prevent the generation of valid mining shares, i.e. solo mining © 2019 Digital Gold Institute 69/109
70. Near-valid Solutions: Mining Shares 4AA087F0A52ED2093FA816E53B9B6317F9B8C1227A61F9481AFED67301F2E3FB D3E51477DCAB108750A5BC9093F6510759CC880BB171A5B77FB4A34ACA27DEDD 00000000008534FF68B98935D090DF5669E3403BD16F1CDFD41CF17D6B474255 BB34ECA3DBB52EFF4B104EBBC0974841EF2F3A59EBBC4474A12F9F595EB81F4B 00000000002F891C1E232F687E41515637F7699EA0F462C2564233FE082BB0AF 0090488133779E7E98177AF1C765CF02D01AB4848DF555533B6C4CFCA201CBA1 460BEFA43B7083E502D36D9D08D64AFB99A100B3B80D4EA4F7B38E18174A0BFB 000000000000000078FB7E1F7E2E4854B8BC71412197EB1448911FA77BAE808A 652F374601D149AC47E01E7776138456181FA4F9D0EEDD8C4FDE3BEF6B1B7ECE 785526402143A291CFD60DA09CC80DD066BC723FD5FD20F9B50D614313529AF3 000000000041EE593434686000AF77F54CDE839A6CE30957B14EDEC10B15C9E5 9C20B06B01A0136F192BD48E0F372A4B9E6BA6ABC36F02FCED22FD9780026A8F © 2019 Digital Gold Institute 70/109
71. Mining Pool History ▪ First pools appeared in late-2010 (GPU era) ▪ By 2014: around 90% of mining became pool-based ▪ June 2014: GHash.io exceeded 50%. Participants voluntarily left GHash.io; Ghash.io does not exist anymore © 2019 Digital Gold Institute 71/109
72. Mining pools (as of August 2014) ▪ Click the chart for update © 2019 Digital Gold Institute 72/109
73. Mining pools (as of May 2018) A dynamically changing oligopoly © 2019 Digital Gold Institute 73/109
74. Game-theoretic Analysis of Mining Strategic issues 1. transactions to be included in a block Default behavior 1. any transaction above minimum fee; if block is full transactions with highest fees 2. block to mine on top of 2. head of the valid chain with higher cumulated difficulty 3. criteria for choosing between equivalent valid blocks 3. first block heard 4. when to announce new blocks 4. immediately after finding them © 2019 Digital Gold Institute 74/109
75. Game-theoretic Analysis of Mining ▪ Assume a miner control 0 < α < 1 of hashing power ▪ Can he profit from a non-default strategy? There is an ongoing research… © 2019 Digital Gold Institute 75/109
76. Table of Contents 1. Hash Functions 2. Simplified Digital Currency 3. Distributed Consensus 4. Mining 5. P2P Network 6. Protocol Governance © 2019 Digital Gold Institute 76/109
77. Bitcoin: A Peer-to-Peer Network ▪ Ad-hoc network with random topology ▪ Open to anyone, low barrier to entry ▪ New nodes can join at any time ▪ Nodes can leave the network: non-responding nodes are forgotten after 3 hours ▪ All nodes are equal, but not all nodes must be mining node ▪ Ad-hoc protocol, it runs on TCP port 8333 ▪ Optional: to be a connectable node, port 8333 must be open ▪ Accepts Remote Procedure Call (RPC) on port 8332 © 2019 Digital Gold Institute 77/109
78. Bitcoin Network: Connectable Nodes © 2019 Digital Gold Institute https://bitnodes.earn.com/ 78/109
79. Joining the Bitcoin P2P network © 2019 Digital Gold Institute 1 6 4 7 3 5 2 8 Hello World! I’m ready to Bitcoin! getaddr() 1, 7 getaddr() getaddr() now I will download and verify the blockchain 79/109
80. Transaction Propagation © 2019 Digital Gold Institute 1 6 4 7 3 5 2 8 New Tx! A→B A→B A→B A→B A→B A→B A→B A→B A→B A→B A→B Already heard that! 80/109
81. Transaction Propagation Nodes relay a new transaction if: ▪ it tries to spend unspent coins ▪ it has valid signatures ▪ it does not create coins out of nothing (TxIns >= TxOuts) ▪ it is valid with respect to the running protocol client (Non-mandatory) sanity checks: ▪ transaction has not been already relayed (avoid infinite loops) ▪ transaction does not conflict with others just relayed (do not help double-spend attempts) ▪ script is standard (more details in the lesson about transactions) © 2019 Digital Gold Institute 81/109
82. Nodes may differ on transaction pool © 2019 Digital Gold Institute 1 6 4 7 3 5 2 8 A→B A→B A→B A→B A→B A→B New Tx! A→C A→C A→C A→B A→C A→C A→B A→C 82/109
83. Race Conditions ▪ Transactions or blocks may conflict: default behavior is to accept the first one heard ▪ Network position matters ▪ Miners may implement other logic! © 2019 Digital Gold Institute 83/109
84. Block Propagation Nodes relay a new block if: ▪ it includes valid transactions only ▪ it meets the difficulty target ▪ it is valid with respect to the running protocol client (Non-mandatory) sanity check, in order not to help forks: ▪ it builds on the chain with most accumulated difficulty © 2019 Digital Gold Institute 84/109
85. Block Propagation Time © 2019 Digital Gold Institute Source: Yonatan Sompolinsky and Aviv Zohar: “Accelerating Bitcoin’s Transaction Processing” 2014 85/109
86. Full Nodes and Software Diversity © 2019 Digital Gold Institute https://coin.dance/nodes/all 86/109
87. Bitcoin Core Software ▪ Direct continuation of Nakamoto’s 2009 open source (MIT license) code: https://github.com/bitcoin/bitcoin ▪ Most widely used Bitcoin software: reference implementation ▪ Even those who do not use Core, they basically follow its lead on rules: de facto Bitcoin rule book ▪ Multiple implementations are problematic for consensus: if they differ, which one is right? See https://blog.bitmex.com/bitcoin-cores-competition/ https://bitcoincore.org “I don't believe a second, compatible implementation of Bitcoin will ever be a good idea. So much of the design depends on all nodes getting exactly identical results in lockstep that a second implementation would be a menace to the network. The MIT license is compatible with all other licenses and commercial uses, so there is no need to rewrite it from a licensing standpoint.” (Satoshi Nakamoto) https://bitcointalk.org/index.php?topic=195.msg1611#msg1611 © 2019 Digital Gold Institute 87/109
88. Bitcoin Testnet Alternative Bitcoin network for testing purposes: there have been three generations, the current one is Testnet3 Compared to mainnet, testnet has: ▪ smaller blockchain size: about 24GB vs 222GB (as of March 2019) ▪ lower mining difficulty ▪ no market value ▪ default protocol listen port is 18333, instead of 8333 ▪ default RPC connection port is 18332, instead of 8332 ▪ different addresses (more details in the lesson about addresses) ▪ scripts can be non-standard (more details in the lesson about transactions) © 2019 Digital Gold Institute 88/109
89. Homework Create a testnet wallet (https://en.bitcoin.it/wiki/testnet) ▪ bitcoin core https://bitcoincore.org/en/download is the best approach (beware that downloading mainnet, while wholeheartedly encouraged, requires much more hard drive space and it is not necessary for this assignment) ▪ electrum https://electrum.org/ is an easier approach: it does not require full blockchain download as it relies on servers Get free testnet bitcoins from a faucet listed at https://en.bitcoin.it/wiki/testnet, then send some to tb1q9u20ha7fg2nxm8d2usre2v6fgq44dsgnupy5ze and provide the transaction ID (even better: check your transaction ID at https://blockstream.info/testnet and send the corresponding link) © 2019 Digital Gold Institute 89/109
90. Table of Contents 1. Hash Functions 2. Simplified Digital Currency 3. Distributed Consensus 4. Mining 5. P2P Network 6. Protocol Governance © 2019 Digital Gold Institute 90/109
91. Some Bitcoin Protocol Limits ▪ 1MB blocksize (SegWit: 4MB block weight) ▪ 20,000 signature operations per block (SegWit: 80,000) ▪ 7 transactions/sec (VISA: 2,000-10,000 up to 60,000) ▪ One signature algorithm (ECDSA) ▪ Two hash functions (SHA256, RIPEMD) ▪ 100M satoshis per bitcoin © 2019 Digital Gold Institute 91/109
92. Bitcoin Improvement Proposals ▪ “formal” proposals for changes to Bitcoin ▪ include technical spec and rationale ▪ each BIP has an assigned number ▪ each BIP has a champion to evangelize and coordinate ▪ also: informational BIPs, process-oriented BIPs © 2019 Digital Gold Institute 92/109
93. Major Protocol Upgrades • P2SH (BIP16; 2012) • CSV/CLTV: (BIP65, BIP112; 2014-2015) • SegWit (BIP141, BIP143, BIP144, BIP145, BIP147, BIP148, BIP173; 2015-2017) © 2019 Digital Gold Institute 93/109
94. Extend Rules: the Hard-fork Change © 2019 Digital Gold Institute 1 6 4 7 3 5 2 8 I found a nifty new block! Block 24 Block 24 Block 24 Block 24 Block 24 Block 23 Block 23 Block 23 Block 23 Block 23 Block 23 Block 23 Block 23 24 24 24 24 That’s crazy talk!! That’s crazy talk!! PROBLEM: Old nodes will never catch up 94/109
95. Hard Fork © 2019 Digital Gold Institute Valid under A rules Invalid under B rules Valid under B rules Invalid under A rules “Bitcoin” “A-coin” “B-coin” 95/109
96. After a Hard Fork If fork was meant to start an alternative coin (altcoin): ▪ altcoin goes its separate way ▪ branches coexist If fork reflected a fight over future of Bitcoin: ▪ branches fight for market share ▪ branches fight to be seen as “the real Bitcoin” ▪ probably one branch wins, one melts away © 2019 Digital Gold Institute 96/109
97. Restrict Rules: the Soft-fork Change We can add new features while restricting rules! ▪ Need majority of nodes to enforce new rules ▪ Old nodes are not forced to upgrade: they do approve blocks created according to new rules ▪ Old nodes might mine blocks invalid according to the new rules: this is an economic incentive for them to join the soft-fork © 2019 Digital Gold Institute 97/109
98. Major Protocol Bugs and Their Fixes • OP_CHECKMULTISIG extra value from stack: no fix (Jan 2009) • ~184 billion BTC created: CVE-2010-5139 (Aug 2010) • 24 block hard-fork in LevelDB upgrade: BIP50, CVE-2013-3220 (Mar 2013) • unbounded monetary supply: BIP42 (Apr 2014) • potential coin inflation: CVE-2018-17144 (Sep 2018) © 2019 Digital Gold Institute 98/109
99. Unbounded Monetary Supply Fixed by BIP42 © 2019 Digital Gold Institute https://github.com/bitcoin/bips/tree/master/bip-0042 99/109
100. Bitcoin Protocol/Network Governance P2P networks are open to anyone, with low barrier to enter ▪ Who has the authority to maintain the protocol (data formats, transaction/block validity, etc.)? ▪ Who can change the rules of the system and/or update software? ▪ What if a change is controversial? ▪ If there is a negotiation about rule-setting, who controls the outcome © 2019 Digital Gold Institute 100/109
101. Bitcoin Foundation ▪ Founded 2012 ▪ Used to pay some core developers ▪ Talk to governments as “voice of Bitcoin” Miserably failed! Bitcoin does not want representative or central governance © 2019 Digital Gold Institute 101/109
102. Bitcoin Ecosystem ▪ Core Developers ▪ Users and Investors ▪ Miners ▪ Industry Centralization vs. decentralization: competing paradigms that underlie many digital technologies (e.g. e-mail is a decentralized protocol, dominated by centralized webmail service providers) © 2019 Digital Gold Institute 102/109
103. Bitcoin Ecosystem: Core Developers ▪ They write the rulebook: as almost everybody uses their code, everybody follows their rules ▪ Their proposed changes must be accepted by miners, industry, and users ▪ Anyone can fork the software at any time, not just “core” devs © 2019 Digital Gold Institute 103/109
104. Bitcoin Ecosystem: Miners ▪ Mining is a decentralized transaction ordering service: miners sequence transactions, writing a transaction history consistent with their consensus rules ▪ Miners can ignore developers’ changes ▪ Miners could force their changes Hard to move forward without some serious dev support ▪ Investors and industry can ignore miners’ rules and adopt a different chain © 2019 Digital Gold Institute 104/109
105. Bitcoin Ecosystem: Users and Investors ▪ They determine whether bitcoin has any value ▪ In case of hard-fork, they decide which branch wins If they do not like a rule change: ▪ Centralized currency: they have the right to exit ▪ Bitcoin: they have the right to fork the rules Right to fork is more empowering than right to exit ↓ community retains more power © 2019 Digital Gold Institute 105/109
106. Bitcoin Ecosystem: Industry Exchanges, Payment Processors, Merchants, etc. ▪ They serve users and investors, facilitating adoption ▪ They sustain ecosystem growth ▪ They handle most transactions Industry can influence and/or is influenced by users and investors: one party hardly lives without the other party © 2019 Digital Gold Institute 106/109
107. Bitcoin: Protocol Governance and Evolution Governance is a dynamic distributed process involving multiple agents Bitcoin is the ledger of not-previously-spent, validly signed transactions contained in the chain of blocks that begins with the genesis block, follows the 21-million coin creation schedule, and has the most cumulative double-SHA256-proof-of-work (Gavin Andresen) http://gavinandresen.ninja/a-definition-of-bitcoin Close… but no cigar! Bitcoin is what the relevant economic majority decide Bitcoin to be © 2019 Digital Gold Institute 107/109
108. Bibliography ▪ A. Narayanan, et al., “Bitcoin and Cryptocurrency Technologies” http://bitcoinbook.cs.princeton.edu/ − chapter 1 «Introduction to Cryptography & Cryptocurrencies» − chapter 2 «How Bitcoin Achieves Decentralization» − chapter 5 «Bitcoin Mining» − chapter 7 «Community, Politics, and Regulation» − chapter 8 «Alternative Mining Puzzles» ▪ Pedro Franco, “Understanding Bitcoin”, Wiley − chapter 9 «Mining» ▪ Andreas Antonopoulos, “Mastering Bitcoin” 2nd edition, O'Reilly https://github.com/bitcoinbook/bitcoinbook − chapter 10 «Mining and Consensus» © 2019 Digital Gold Institute 108/109
109. Takeaways ▪ Hash Functions generate unique identifiers for any given input ▪ Distributed Consensus is the hard part of a decentralized solution/currency ▪ Proof-of-Work mining is what makes blockchain immutable or at least tamper-resistant ▪ P2P networking makes blockchain solutions really distributed ▪ Distributed protocol governance is a dynamic process involving multiple agents ▪ Bitcoin is what the economic relevant majority decide Bitcoin to be © 2019 Digital Gold Institute 109/109
110. Ferdinando M. Ametrano Executive Director ferdinando@dgi.io Paolo Mazzocchi Chief Operating Officer paolo@dgi.io www.github.com/dginst www.facebook.com/DigitalGoldInstitute www.twitter.com/DigitalGoldInst www.dgi.io/feed.xml info@dgi.io www.dgi.io www.linkedin.com/company/digital-gold-institute "Scarcity in the Digital Domain"