Upgrade to Pro — share decks privately, control downloads, hide ads and more …

In the beginning was TXT

Avatar for Markus Wein Markus Wein
October 02, 2014
Programming
140
0

In the beginning was TXT

A very short overview of the history of encodings, given at Vienna.rb on 2014-10-02

Avatar for Markus Wein

Markus Wein

October 02, 2014

More Decks by Markus Wein

See All by Markus Wein
Command Line Productivity
Avatar for Markus Wein cypher
1
180
A crash intro to deliberate practice
Avatar for Markus Wein cypher
0
150
Keeping Your PostgreSQL Data Save
Avatar for Markus Wein cypher
0
160
Ghost in the State Machine
Avatar for Markus Wein cypher
2
350
n Things You Didn't Know About PostgreSQL (Rubyslava & PyVo 2014 Edition)
Avatar for Markus Wein cypher
1
290
How to Become a Better Developer
Avatar for Markus Wein cypher
2
1.8k
An Introduction to Rust
Avatar for Markus Wein cypher
1
8.4k
How to Become a Better Developer
Avatar for Markus Wein cypher
1
280
A Very Short Overview of Vagrant
Avatar for Markus Wein cypher
0
8.1k

Other Decks in Programming

See All in Programming
言葉の格闘技のススメ～紙とペンと言葉から始める、キャリアの描き方～
Avatar for 進捗ゼミ progresscicada
2
160
freeeにおけるEvalsの実践例の紹介
Avatar for freee freee
PRO
0
130
仕様駆動開発の消費期限
Avatar for watany watany
20
8.7k
AI Readyの正体はデータマネジメントだ メダリオン2.0の最前線
Avatar for freee freee
PRO
0
320
そこに3びきプロダクトがいるじゃろう——生成AI時代における“価値が届かない理由”の構造
Avatar for kosuke T kosuket
0
520
2年かけて Deno に DOMMatrix を実装した話 / How I implemented DOMMatrix in Deno over two years
Avatar for petamoriken / 森建 petamoriken
0
210
楽しそうなつよつよエンジニアと目が死んでる僕/A brilliant engineer having a blast, and dead-eyed me.
Avatar for Ryusei Ohkura 3l4l5
2
200
yield再入門 #phpcon
Avatar for hideki kinjyo o0h
PRO
0
1.2k
ソフトウェア設計に溶けるインフラ ― AWS CDK のインフラ認識論
Avatar for Kenji Kono konokenj
3
810
Japan Community Day at Kubecon + CloudNativeCon Japan 2026: Learning Container Privilege Control by Building My Own Low-Level Container Runtime
Avatar for ternbusty ternbusty
1
160
How I Won Prize Money at a Hackathon Using Codex and Symphony Alpha
Avatar for yasei no otoko yasei_no_otoko
0
110
改善しないと、タスクが回らない。 “てんこ盛りポジション” を引き継いだ情シスの、入社3ヶ月の業務改善録
Avatar for Kurumi Watabe krm963
0
280

Featured

See All Featured
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
32
3k
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8.3k
Building Flexible Design Systems
Avatar for Yesenia Perez-Cruz yeseniaperezcruz
330
40k
Deep Space Network (abreviated)
Avatar for Tony Rice tonyrice
0
260
B2B Lead Gen: Tactics, Traps & Triumph
Avatar for Sophie Logan marketingsoph
0
220
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
570
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
370
The World Runs on Bad Software
Avatar for Brandon Keepers bkeepers
PRO
72
12k
エンジニアに許された特別な時間の終わり
Avatar for watany watany
108
250k
Statistics for Hackers
Avatar for Jake VanderPlas jakevdp
799
230k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
3k
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
Avatar for Akash Hashmi akashhashmi
0
420

Transcript

  1. In the beginning was TXT

  2. !

  3. EBCDIC

  4. Source: http://en.wikipedia.org/wiki/EBCDIC

  5. ASCII

  6. "#$%&

  7. None

  8. ä, ö, or å and Ø?

  9. Latin-1 ISO/IEC 8859-1

  10. Latin-*

  11. Windows code pages

  12. Then came the €

  13. (

  14. None

  15. Shift-JIS

  16. This sucks

  17. Unicode!

  18. Unicode!

  19. ✈️ (planes!)

  20. Basic Multilingual Plane

  21. Code Points

  22. U+0041 (LATIN SMALL LETTER A)

  23. Source: http://codepoints.net/U+0041

  24. Grapheme

  25. a a a a a a a

  26. Composite characters

  27. U+0065 U+0301 or U+00E9

  28. e+´ => é é

  29. ´ != ´

  30. Unicode… is not an encoding

  31. UTF-32

  32. UCS-2/UTF-16

  33. UTF-8

  34. Source: http://en.wikipedia.org/wiki/File:UnicodeGrow2b.png

  35. What does it look like?

  36. Codepoint Char ASCII Latin-1 ISO-8859-15 UTF-8 UTF-16 U+0041 A 0x41

    0x41 0x41 0x41 0x00 0x41 U+00C4 Ä - 0xc4 0xc4 0xc3 0x84 0x00 0xc4 U+20AC € - - 0xa4 0xe3 0x82 0xac 0x20 0xac U+C218 ࣻ - - - 0xec 0x88 0x98 0xc2 0x18 Encoding comparison Source: http://perlgeek.de/en/article/encodings-and-unicode

  37. Remember: Just because someone claims it’s UTF-8, doesn’t mean it

    is