Goals
• Get comfortable with bytes
• Write simpler, more efficient programs
Slide 7
Slide 7 text
Plan
• Bottom up
• History
• Math
• Code
Slide 8
Slide 8 text
1095 AD
Slide 9
Slide 9 text
Date: Aprilis 11, 1095
From: Emperor Alexios Komnenos
To: Robert II, Count of Flanders
Bob,
Looks like we’re gonna war with the
Turks. Could you see if the Pope
could help us crusade against ’em?
Thanks!
Alex
Letters!
• Human readable
• Signed
Slide 10
Slide 10 text
Date: Aprilis 11, 1095
From: Emperor Alexios Komnenos
To: Robert II, Count of Flanders
Bob,
Looks like we’re gonna war with the
Turks. Could you see if the Pope
could help us crusade against ’em?
Thanks!
Alex
Letters!
• Slow, dangerous to transmit
• Awkward to store
• Requires Literacy!
Slide 11
Slide 11 text
1854
Slide 12
Slide 12 text
Telegraphs
NYC
SF
Slide 13
Slide 13 text
Telegraphs
NYC
SF
Slide 14
Slide 14 text
Morse Code
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
Slide 15
Slide 15 text
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
▬ • •
Slide 16
Slide 16 text
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
▬ • • ▬ ▬ ▬
Slide 17
Slide 17 text
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
▬ • • ▬ ▬ ▬ ▬ •
Slide 18
Slide 18 text
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
▬ • • ▬ ▬ ▬ ▬ • • • ▬
Slide 19
Slide 19 text
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
▬ • • ▬ ▬ ▬ ▬ • • • ▬ ▬
Slide 20
Slide 20 text
A • ▬ G ▬ ▬ • M ▬ ▬ S • • • Y ▬ • ▬ ▬ 4 • • • • ▬
B ▬ • • • H • • • • N ▬ • T ▬ Z ▬ ▬ • • 5 • • • • •
C ▬ • ▬ • I • • O ▬ ▬ ▬ U • • ▬ 0 ▬ ▬ ▬ ▬ ▬ 6 ▬ • • • •
D ▬ • • J • ▬ ▬ ▬ P • ▬ ▬ • V • • • ▬ 1 • ▬ ▬ ▬ ▬ 7 ▬ ▬ • • •
E • K ▬ • ▬ Q ▬ ▬ • ▬ W • ▬ ▬ 2 • • ▬ ▬ ▬ 8 ▬ ▬ ▬ • •
F • • ▬ • L • ▬ • • R • ▬ • X ▬ • • ▬ 3 • • • ▬ ▬ 9 ▬ ▬ ▬ ▬ •
▬ • • ▬ ▬ ▬ ▬ • • • ▬ ▬
Slide 21
Slide 21 text
Morse Code
NYC
SF
Slide 22
Slide 22 text
Morse Code
• Telegraphs!
• 15 words per minute
• Wireless in 1898
Slide 23
Slide 23 text
Morse Code
• Very limited characters
• No lowercase
• Limited punctuation*
• Average message: 12 words
* For example, morse code doesn’t have an asterisk symbol.
Binary Refresher!
• Decimal lets you represent any integer with a sequence of digits:
0, 1, 2, 3, 4, 5, 6, 7, 8, 9
• Binary lets you represent any integer with a sequence of bits: 0, 1
Slide 39
Slide 39 text
Binary Refresher!
• Typically prefixed with “0b”, like 0b11001101
Layering
• Given a wire that transmits a single bit, we can use binary to
encode any integer!
• This works because the sender and recipient agree on how to
interpret the sequence
• That interpretation is called an encoding
Slide 43
Slide 43 text
Bytes
• Though binary supports any number of bits, we like 8-bit integers
• An 8-bit integer is called a byte
• 256 values from 0 to 255
Slide 44
Slide 44 text
1967
Slide 45
Slide 45 text
ASCII
• American Standard Code for Information Interchange
• A table of characters
• Interpret a sequence of bytes as a string of characters!
Slide 46
Slide 46 text
ASCII
• Work started in 1960
• Only uses 7 bits: in 1967 bits were very expensive!
• That means there’s 27 = 128 characters
Slide 47
Slide 47 text
ASCII
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
Slide 48
Slide 48 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68
Slide 49
Slide 49 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68 111
Slide 50
Slide 50 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68 111 110
Slide 51
Slide 51 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68 111 110 117
Slide 52
Slide 52 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68 111 110 117 116
Slide 53
Slide 53 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68 111 110 117 116
Slide 54
Slide 54 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
68 111 110 117 116
Slide 55
Slide 55 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
111 110 117 116
0 1 0 0 0 1 0 0
Slide 56
Slide 56 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
110 117 116
0 1 0 0 0 1 0 0 0 1 1 0 1 1 1 1
Slide 57
Slide 57 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
117 116
0 1 0 0 0 1 0 0 0 1 1 0 1 1 1 1 0 1 1 0 1 1 1 0
Slide 58
Slide 58 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
116
0 1 0 0 0 1 0 0 0 1 1 0 1 1 1 1 0 1 1 0 1 1 1 0 0 1 1 1 0 1 0 1
Slide 59
Slide 59 text
0 NULL 16 DLE 32 SP 48 0 64 @ 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74 J 90 Z 106 j 122 z
11 VT 27 ESC 43 + 59 ; 75 K 91 [ 107 k 123 {
12 FF 28 FS 44 , 60 < 76 L 92 \ 108 l 124 |
13 CR 29 GS 45 - 61 = 77 M 93 ] 109 m 125 }
14 SO 30 RS 46 . 62 > 78 N 94 ^ 110 n 126 ~
15 SI 31 US 47 / 63 ? 79 O 95 _ 111 o 127 DEL
0 1 0 0 0 1 0 0 0 1 1 0 1 1 1 1 0 1 1 0 1 1 1 0 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 0
Slide 60
Slide 60 text
1980’s
Slide 61
Slide 61 text
boss, what should
I be working on?
Slide 62
Slide 62 text
boss, what should
I be working on?
resume polishing
Slide 63
Slide 63 text
boss, what should
I be working on?
Slide 64
Slide 64 text
boss, what should
I be working on?
résumé polishing
Slide 65
Slide 65 text
Charset Hell
• ASCII is English only
• But non-English people also use computers!
Slide 66
Slide 66 text
Charset Hell
• Operating systems were installed for a specific character set and
wouldn’t work with any others
• Documents couldn’t mix Greek, French, and Russian characters
• If you see ISO-8859-1, run away!
Slide 67
Slide 67 text
1991
Slide 68
Slide 68 text
Unicode
• Support all languages in a single system
• A code point is a universal ID for a character
Slide 69
Slide 69 text
UTF-16
• 16-bit Unicode Transformation Format
• 2 bytes per code point
• This is Java’s char type
UTF-16 characters
• Max code point is 65,535
• Code point for is 127,849
• 127,849 > 65,535
Slide 78
Slide 78 text
Java’s char is broken!
• There’s a system called “surrogate pairs” which is like multidex for
code points
• It splits a single code point across 2 chars
• It’s an incredible pain
Slide 79
Slide 79 text
char[] s = "Café ".toCharArray();
C a f é
0 1 2 3 4 5 6
Slide 80
Slide 80 text
String s = “Café ";
for (int i = 0, size = s.length(); i < size; i++) {
char c = s.charAt(i);
System.out.printf("The character at %d is '%c'%n", i, c);
}
Slide 81
Slide 81 text
String s = “Café ";
for (int i = 0, size = s.length(); i < size; i++) {
char c = s.charAt(i);
System.out.printf("The character at %d is '%c'%n", i, c);
}
The character at 0 is 'C'
The character at 1 is 'a'
The character at 2 is 'f'
The character at 3 is 'é'
The character at 4 is ' '
The character at 5 is ' '
The character at 6 is ' '
Slide 82
Slide 82 text
String s = “Café ";
for (int i = 0, size = s.length(); i < size; ) {
int c = s.codePointAt(i);
System.out.printf("The code point at %d is '%c'%n", i, c);
i += Character.charCount(c);
}
Slide 83
Slide 83 text
String s = “Café ";
for (int i = 0, size = s.length(); i < size; ) {
int c = s.codePointAt(i);
System.out.printf("The code point at %d is '%c'%n", i, c);
i += Character.charCount(c);
}
The code point at 0 is 'C'
The code point at 1 is 'a'
The code point at 2 is 'f'
The code point at 3 is 'é'
The code point at 4 is ' '
The code point at 5 is ' '
Slide 84
Slide 84 text
1998
Slide 85
Slide 85 text
UTF-8
• 8-bit Unicode Transformation Format
• Variable number of bytes per code point
• This is how modern apps transmit & store text
Pictures
• Just a 2D array of colors
• Given 3 bytes per pixel:
• 64 × 64 icon is 12,288 bytes
• 1080 × 1920 picture is 5.9 MiB
• Compression is important!
Slide 127
Slide 127 text
Pictures
• Android can use fewer bits per pixel
• ARGB_8888: alpha, red, green, and blue get 8 bits each
• RGB_565: red gets 5 bits, geen gets 6, blue gets 5 bits
Slide 128
Slide 128 text
RGB_565
ARGB_8888
Slide 129
Slide 129 text
RGB_454
ARGB_8888
Slide 130
Slide 130 text
RGB_232
ARGB_8888
Slide 131
Slide 131 text
/** https://en.wikipedia.org/wiki/BMP_file_format */
public void encode(BufferedSink sink) throws IOException {
int height = pixels.length;
int width = pixels[0].length;
int bytesPerPixel = 3;
int rowByteCountWithoutPadding = (bytesPerPixel * width);
int rowByteCount = ((rowByteCountWithoutPadding + 3) / 4) * 4;
int pixelDataSize = rowByteCount * height;
int bmpHeaderSize = 14;
int dibHeaderSize = 40;
// BMP Header
sink.writeUtf8("BM"); // ID.
sink.writeIntLe(bmpHeaderSize + dibHeaderSize + pixelDataSize); // File size.
sink.writeShortLe(0); // Unused.
sink.writeShortLe(0); // Unused.
sink.writeIntLe(bmpHeaderSize + dibHeaderSize); // Offset of pixel data.
// DIB Header
sink.writeIntLe(dibHeaderSize);
Slide 132
Slide 132 text
int dibHeaderSize = 40;
// BMP Header
sink.writeUtf8("BM"); // ID.
sink.writeIntLe(bmpHeaderSize + dibHeaderSize + pixelDataSize); // File size.
sink.writeShortLe(0); // Unused.
sink.writeShortLe(0); // Unused.
sink.writeIntLe(bmpHeaderSize + dibHeaderSize); // Offset of pixel data.
// DIB Header
sink.writeIntLe(dibHeaderSize);
sink.writeIntLe(width);
sink.writeIntLe(height);
sink.writeShortLe(1); // Color plane count.
sink.writeShortLe(bytesPerPixel * Byte.SIZE);
sink.writeIntLe(0); // No compression.
sink.writeIntLe(16); // Size of bitmap data including padding.
sink.writeIntLe(2835); // Horizontal print resolution in pixels/meter. (72 dpi).
sink.writeIntLe(2835); // Vertical print resolution in pixels/meter. (72 dpi).
sink.writeIntLe(0); // Palette color count.
sink.writeIntLe(0); // 0 important colors.
// Pixel data.
Slide 133
Slide 133 text
sink.writeIntLe(0); // Palette color count.
sink.writeIntLe(0); // 0 important colors.
// Pixel data.
for (int y = height - 1; y >= 0; y--) {
int[] row = pixels[y];
for (int x = 0; x < width; x++) {
int pixel = row[x];
sink.writeByte((pixel & 0x0000ff)); // Blue.
sink.writeByte((pixel & 0x00ff00) >>> 8); // Green.
sink.writeByte((pixel & 0xff0000) >>> 16); // Red.
}
// Padding for 4-byte alignment.
for (int p = rowByteCountWithoutPadding; p < rowByteCount; p++) {
sink.writeByte(0);
}
}
}
Slide 134
Slide 134 text
Pictures on Bytes
• This bitmap writer is 50 lines of code
• Decoders are more difficult!
• Good specs make it easy
Slide 135
Slide 135 text
sink.writeIntLe(0); // Palette color count.
sink.writeIntLe(0); // 0 important colors.
// Pixel data.
for (int y = height - 1; y >= 0; y--) {
int[] row = pixels[y];
for (int x = 0; x < width; x++) {
int pixel = row[x];
sink.writeByte((pixel & 0x0000ff)); // Blue.
sink.writeByte((pixel & 0x00ff00) >>> 8); // Green.
sink.writeByte((pixel & 0xff0000) >>> 16); // Red.
}
// Padding for 4-byte alignment.
for (int p = rowByteCountWithoutPadding; p < rowByteCount; p++) {
sink.writeByte(0);
}
}
}
Slide 136
Slide 136 text
sink.writeIntLe(0); // Palette color count.
sink.writeIntLe(0); // 0 important colors.
// Pixel data.
for (int y = height - 1; y >= 0; y--) {
int[] row = pixels[y];
for (int x = 0; x < width; x++) {
int pixel = row[x];
sink.writeByte((pixel & 0x0000ff)); // Blue.
sink.writeByte((pixel & 0x00ff00) >>> 8); // Green.
sink.writeByte((pixel & 0xff0000) >>> 16); // Red.
}
// Padding for 4-byte alignment.
for (int p = rowByteCountWithoutPadding; p < rowByteCount; p++) {
sink.writeByte(0);
}
}
}
Slide 137
Slide 137 text
(pixel & 0xff0000) >>> 16
• Shifting and masking lets you access the bits within an integer
• & and | operators treat each int like a 32-element boolean array!
• <<, >> and >>> operators slide bits left and right
Conference App!
• Each talk has an ID, date, room, title,
and speaker
Slide 162
Slide 162 text
JSON is Good Stuff
{
"id": 72017,
"date": "2016-09-30T18:30:00Z",
"room": "RIGHT",
"title": "Decoding the Secrets of Binary Data",
"speaker": "Jesse Wilson"
}
Slide 163
Slide 163 text
JSON is Good Stuff
{
"id": 72017,
"date": "2016-09-30T18:30:00Z",
"room": "RIGHT",
"title": "Decoding the Secrets of Binary Data",
"speaker": "Jesse Wilson"
}
• A nice format that builds on UTF-8
• Easy to read & write
Slide 164
Slide 164 text
JSON is Self-Delimiting
{
"id": 72017,
"date": "2016-09-30T18:30:00Z",
"room": "RIGHT",
"title": "Decoding the Secrets of Binary Data",
"speaker": "Jesse Wilson"
}
• A JSON document has both structure and data
• Uses escape sequences like \" to be completely unambiguous
Slide 165
Slide 165 text
"2016-09-30T18:30:00Z"
• System.currentTimeMillis() returns milliseconds since
January 1, 1970 at 00:00:00 UTC
• A date that’s 8 bytes in memory is 22 bytes in JSON!
Slide 166
Slide 166 text
• Space: a simple message like this one is ~128 bytes
• Time: bigger sequences take longer to decode
JSON Space & Time
Slide 167
Slide 167 text
• Google’s “small, fast, simple” structured data format
• Upon closer inspection, it’s not that different from JSON!
• But it has a schema
Protocol Buffers
Slide 168
Slide 168 text
{
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
"id": 72017,
"date": "2016-09-30T18:30:00Z",
"room": "RIGHT",
"title": "Decoding the Secrets of Binary Data",
"speaker": "Jesse Wilson"
}
"id"
"date"
"room"
"title"
"speaker"
72017
"2016-09-30T18:30:00Z",
"RIGHT",
"Decoding the Secrets of Binary Data",
"Jesse Wilson",
Slide 169
Slide 169 text
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
Slide 170
Slide 170 text
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
000 enum, int32, int64...
001 fixed64
010 string, message
101 fixed32
Length Mode
Slide 171
Slide 171 text
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
000 enum, int32, int64...
001 fixed64
010 string, message
101 fixed32
Length Mode
1 0 1
Slide 172
Slide 172 text
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
000 enum, int32, int64...
001 fixed64
010 string, message
101 fixed32
Length Mode
1 0 1
0 0 1
Slide 173
Slide 173 text
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
000 enum, int32, int64...
001 fixed64
010 string, message
101 fixed32
Length Mode
0 0 0
1 0 1
0 0 1
Slide 174
Slide 174 text
message Talk {
optional fixed32 id = 1;
optional fixed64 date = 2;
optional Room room = 3;
optional string title = 4;
optional string speaker = 5;
enum Room {
UP = 1; RIGHT = 2; DOWN = 3; LEFT = 4;
}
}
000 enum, int32, int64...
001 fixed64
010 string, message
101 fixed32
Length Mode
0 0 0
1 0 1
0 1 0
0 0 1
511901
0d 11 18 22
2a
4465636f64696e6720746865205365637265747
4a657373652057696c736f6e
00 00
00
01
57
7D
37
EE
40 02 23
0c
3206f662042696e6172792044617461
• Protocol Buffers are small and fast
• ~1 byte for each field name
• Compact encoding for numbers and enums
• Strings stay the same length!
• This message is 67 bytes in protocol buffers, vs. 128 for JSON
Slide 204
Slide 204 text
Java I/O
Slide 205
Slide 205 text
• Mutable!
• equals() doesn’t work like it should: can’t be a map key!
• Doesn’t implement Comparable
byte[] is bad
Slide 206
Slide 206 text
• Range is -128..127 but you usually want 0..255
• But unsigned when you call InputStream.read()
byte is signed
Slide 207
Slide 207 text
• Range is -128..127 but you usually want 0..255
• But unsigned when you call InputStream.read()
byte is signed
Slide 208
Slide 208 text
Okio
Slide 209
Slide 209 text
Okio
• Square’s library for efficient I/O
• Used by OkHttp, Retrofit, Wire, Moshi
Slide 210
Slide 210 text
• Immutable!
• equals() works like it should
• Implements Comparable
ByteString is good
class Talk implements Parcelable {
public static final Parcelable.Creator CREATOR = new Parcelable.Creator() {
@Override public Talk createFromParcel(Parcel in) {
int id = in.readInt();
long date = in.readLong();
Room room = Room.values()[in.readInt()];
String title = in.readString();
String speaker = in.readString();
return new Talk(id, date, room, title, speaker);
}
};
…
@Override public void writeToParcel(Parcel out, int flags) {
out.writeInt(id);
out.writeLong(date);
out.writeInt(room.ordinal());
out.writeString(title);
out.writeString(speaker);
}
}
• Everything is bytes
• Java Strings are UTF-16. Encoded text is usually UTF-8
• Hex is handy
• Don’t be afraid of shifting and masking
• Integers are big or little-endian
• Java’s I/O APIs are trouble
6 tips
Slide 218
Slide 218 text
• You can encode almost anything
• Okio makes it easy
No Black Boxes