Upgrade to Pro — share decks privately, control downloads, hide ads and more …

In the beginning was TXT

Avatar for Markus Wein Markus Wein
October 02, 2014
Programming
0
110

In the beginning was TXT

A very short overview of the history of encodings, given at Vienna.rb on 2014-10-02

Avatar for Markus Wein

Markus Wein

October 02, 2014
Tweet

More Decks by Markus Wein

See All by Markus Wein
Command Line Productivity
Avatar for Markus Wein cypher
1
130
A crash intro to deliberate practice
Avatar for Markus Wein cypher
0
110
Keeping Your PostgreSQL Data Save
Avatar for Markus Wein cypher
0
120
Ghost in the State Machine
Avatar for Markus Wein cypher
2
320
n Things You Didn't Know About PostgreSQL (Rubyslava & PyVo 2014 Edition)
Avatar for Markus Wein cypher
1
240
How to Become a Better Developer
Avatar for Markus Wein cypher
2
1.8k
An Introduction to Rust
Avatar for Markus Wein cypher
1
8.2k
How to Become a Better Developer
Avatar for Markus Wein cypher
1
240
A Very Short Overview of Vagrant
Avatar for Markus Wein cypher
0
7.9k

Other Decks in Programming

See All in Programming
TVerのWeb内製化 - 開発スピードと品質を両立させるまでの道のり
Avatar for TVer Inc. techtver
PRO
1
470
flutter_kaigi_2025.pdf
Avatar for Kyohei Ito kyoheig3
1
290
Flutterアプリ運用の現場で役立った監視Tips 5選
Avatar for Takuma Osada ostk0069
1
420
AIの弱点、やっぱりプログラミングは人間が(も)勉強しよう / YAPC AI and Programming
Avatar for Naoki Kishida kishida
9
4.4k
JEP 496 と JEP 497 から学ぶ耐量子計算機暗号入門 / Learning Post-Quantum Crypto Basics from JEP 496 & 497
Avatar for mackey0225 mackey0225
2
240
なぜ強調表示できず ** が表示されるのか — Perlで始まったMarkdownの歴史と日本語文書における課題
Avatar for hkws kwahiro
11
5.7k
Register is more than clipboard
Avatar for Satoru Kitaguchi satorunooshie
1
470
CSC509 Lecture 13
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
250
ビルドプロセスをデバッグしよう！
Avatar for Yuta Tomiyama yt8492
0
310
Eloquentを使ってどこまでコードの治安を保てるのか？を新人が考察してみた
Avatar for ito_kohhh itokoh0405
0
3.1k
詳細の決定を遅らせつつ実装を早くする
Avatar for shimabox shimabox
1
1k
Vueで学ぶデータ構造入門 リンクリストとキューでリアクティビティを捉える / Vue Data Structures: Linked Lists and Queues for Reactivity
Avatar for konkarin konkarin
1
190

Featured

See All Featured
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.5k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
65
8k
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
62k
How Fast Is Fast Enough? [PerfNow 2025]
Avatar for Tammy Everts tammyeverts
3
320
Building Adaptive Systems
Avatar for Chris Keathley keathley
44
2.8k
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
Why Our Code Smells
Avatar for Brandon Keepers bkeepers
PRO
340
57k
Fantastic passwords and where to find them - at NoRuKo
Avatar for Phil Nash philnash
52
3.5k
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
46
7.8k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k
Side Projects
Avatar for Sacha Greif sachag
455
43k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
9
970

Transcript

  1. In the beginning was TXT

  2. !

  3. EBCDIC

  4. Source: http://en.wikipedia.org/wiki/EBCDIC

  5. ASCII

  6. "#$%&

  7. None

  8. ä, ö, or å and Ø?

  9. Latin-1 ISO/IEC 8859-1

  10. Latin-*

  11. Windows code pages

  12. Then came the €

  13. (

  14. None

  15. Shift-JIS

  16. This sucks

  17. Unicode!

  18. Unicode!

  19. ✈️ (planes!)

  20. Basic Multilingual Plane

  21. Code Points

  22. U+0041 (LATIN SMALL LETTER A)

  23. Source: http://codepoints.net/U+0041

  24. Grapheme

  25. a a a a a a a

  26. Composite characters

  27. U+0065 U+0301 or U+00E9

  28. e+´ => é é

  29. ´ != ´

  30. Unicode… is not an encoding

  31. UTF-32

  32. UCS-2/UTF-16

  33. UTF-8

  34. Source: http://en.wikipedia.org/wiki/File:UnicodeGrow2b.png

  35. What does it look like?

  36. Codepoint Char ASCII Latin-1 ISO-8859-15 UTF-8 UTF-16 U+0041 A 0x41

    0x41 0x41 0x41 0x00 0x41 U+00C4 Ä - 0xc4 0xc4 0xc3 0x84 0x00 0xc4 U+20AC € - - 0xa4 0xe3 0x82 0xac 0x20 0xac U+C218 ࣻ - - - 0xec 0x88 0x98 0xc2 0x18 Encoding comparison Source: http://perlgeek.de/en/article/encodings-and-unicode

  37. Remember: Just because someone claims it’s UTF-8, doesn’t mean it

    is