Upgrade to Pro — share decks privately, control downloads, hide ads and more …

In the beginning was TXT

Avatar for Markus Wein Markus Wein
October 02, 2014
Programming
0
120

In the beginning was TXT

A very short overview of the history of encodings, given at Vienna.rb on 2014-10-02

Avatar for Markus Wein

Markus Wein

October 02, 2014
Tweet

More Decks by Markus Wein

See All by Markus Wein
Command Line Productivity
Avatar for Markus Wein cypher
1
160
A crash intro to deliberate practice
Avatar for Markus Wein cypher
0
120
Keeping Your PostgreSQL Data Save
Avatar for Markus Wein cypher
0
140
Ghost in the State Machine
Avatar for Markus Wein cypher
2
340
n Things You Didn't Know About PostgreSQL (Rubyslava & PyVo 2014 Edition)
Avatar for Markus Wein cypher
1
250
How to Become a Better Developer
Avatar for Markus Wein cypher
2
1.8k
An Introduction to Rust
Avatar for Markus Wein cypher
1
8.3k
How to Become a Better Developer
Avatar for Markus Wein cypher
1
240
A Very Short Overview of Vagrant
Avatar for Markus Wein cypher
0
8k

Other Decks in Programming

See All in Programming
Codexに役割を持たせる 他のAIエージェントと組み合わせる実務Tips
Avatar for o8n o8n
3
1.1k
encoding/json/v2のUnmarshalはこう変わった：内部実装で見る設計改善
Avatar for Hiroki Kurasawa kurakura0916
0
350
The Past, Present, and Future of Enterprise Java
Avatar for ivargrimstad ivargrimstad
0
430
New in Go 1.26 Implementing go fix in product development
Avatar for sunecosuri sunecosuri
0
380
Codex の「自走力」を高める
Avatar for yorifuji yorifuji
0
1.1k
atmaCup #23でAIコーディングを活用した話
Avatar for ML_Bear ml_bear
4
750
AIとペアプロして処理時間を97%削減した話 #pyconshizu
Avatar for Kashun Yoshida kashewnuts
1
210
Agent Skills Workshop - AIへの頼み方を仕組み化する
Avatar for Gota gotalab555
15
8.2k
LangChain4jとは一味違うLangChain4j-CDI
Avatar for Kenji Kazumura kazumura
1
160
2026年は Rust 置き換えが流行る！ / 20260220-niigata-5min-tech
Avatar for girigiribauer girigiribauer
0
230
AI活用のコスパを最大化する方法
Avatar for Ochtum ochtum
0
130
今更考える「単一責任原則」 / Thinking about the Single Responsibility Principle
Avatar for philomagi tooppoo
3
1.6k

Featured

See All Featured
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
22k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1370
200k
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
2
540
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
Avatar for Marc Duiker marcduiker
35
2.4k
SEO for Brand Visibility & Recognition
Avatar for Aleyda Solis aleyda
0
4.3k
Visualization
Avatar for Eitan Lees eitanlees
150
17k
WENDY [Excerpt]
Avatar for Tessa Abrams tessaabrams
9
36k
Lightning talk: Run Django tests with GitHub Actions
Avatar for Sarah Abderemane sabderemane
0
140
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
120
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
463
34k
Scaling GitHub
Avatar for Zach Holman holman
464
140k
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
Avatar for soudai sone soudai
PRO
62
51k

Transcript

  1. In the beginning was TXT

  2. !

  3. EBCDIC

  4. Source: http://en.wikipedia.org/wiki/EBCDIC

  5. ASCII

  6. "#$%&

  7. None

  8. ä, ö, or å and Ø?

  9. Latin-1 ISO/IEC 8859-1

  10. Latin-*

  11. Windows code pages

  12. Then came the €

  13. (

  14. None

  15. Shift-JIS

  16. This sucks

  17. Unicode!

  18. Unicode!

  19. ✈️ (planes!)

  20. Basic Multilingual Plane

  21. Code Points

  22. U+0041 (LATIN SMALL LETTER A)

  23. Source: http://codepoints.net/U+0041

  24. Grapheme

  25. a a a a a a a

  26. Composite characters

  27. U+0065 U+0301 or U+00E9

  28. e+´ => é é

  29. ´ != ´

  30. Unicode… is not an encoding

  31. UTF-32

  32. UCS-2/UTF-16

  33. UTF-8

  34. Source: http://en.wikipedia.org/wiki/File:UnicodeGrow2b.png

  35. What does it look like?

  36. Codepoint Char ASCII Latin-1 ISO-8859-15 UTF-8 UTF-16 U+0041 A 0x41

    0x41 0x41 0x41 0x00 0x41 U+00C4 Ä - 0xc4 0xc4 0xc3 0x84 0x00 0xc4 U+20AC € - - 0xa4 0xe3 0x82 0xac 0x20 0xac U+C218 ࣻ - - - 0xec 0x88 0x98 0xc2 0x18 Encoding comparison Source: http://perlgeek.de/en/article/encodings-and-unicode

  37. Remember: Just because someone claims it’s UTF-8, doesn’t mean it

    is