One thing to rule it all: entropy Piotr Migdał 4th Offtopicarium 5 Jan 2014, Słomczyn near Warsaw (or: the beauty of not knowing) 

No content

No content

About • Thermodynamics • Information theory • Statistics • ... 

Why things happen? • Because they have to • Because the everything else is extremely unlikely 

Entropy ~ logarithm of the number of possibilities higher entropy: more likely it will happen in large systems unlikely things became impossible (throwing 3 tails in a row vs throwing 1,000,000 tails in a row) 

http://ensemble.va.com.au/Treister/HEXEN2/HEXEN_2.html 

Shannon entropy S = X x p(x) log p(x) log2 1 = 0 log2 1 2 = 1 log2 1 4 = 2 log2 2 = 1 log2 4 = 2 = X x p(x) log 1 p ( x ) (the general concept for ANY probability) log2 8 = 3 log2 ) in bits ln = loge ) in nits log1 0 ) in bans 

Examples • A coin: • A die: • Dying one day: 6 1 6 log2(6) ⇡ 2 . 6 1 2 log2(2) + 1 2 log2(2) = 1 0 log2(0) + 1 log2(1) = 0 S = log( n ) For n equaly probable events: 

20 questions Is it alive? Is it a ring? 1 2 log2(2) + 1 2 log2(2) = 1 0 . 999 log2 1 0.999 + 0 . 001 log2 1 0.001 ⇡ 0 . 01 

Sequence compression 01001010001101100101111... A - 00 C - 01 G - 10 T - 11 effective number of characters 1 bit per char p(0) = p(1) = 1/2 ACTAGATACTG... 2 bit per char p(A) = p(C) = p(G) = p(T) = 1/4 00011100100011011110... 

Sequence compression zzzhzzhzzhhzzhzzzhzzzzhzhzz p(z) = 3/4 p(h) = 1/4 zz - 0 zh - 10 h - 11 0.81 bit per char 010011011110110100010100 

Heat dS = dQ T 

Phase space and phase transitions position momentum (“velocity”) ice water in low temperature 

Phase space and phase transitions position momentum (“velocity”) freezing water by looking at it (when you know the actual state, you can extract more energy) ice water in high temperature 

Other uses • Economic diversity (inequality) • Population diversity S2 = log p2 1 + . . . + p2 n (collision entropy) (purity) pi = ni n (Gini coefficient) 1/(p2 1 + . . . + p2 n ) 

gzip + poetry = awesome http://jvns.ca/blog/2013/10/24/day-16-gzip-plus-poetry-equals-awesome/ 

Mutual information yes no yes no mutual information = entropy 1 2 1 2 1 2 1 2 Bob Alice 

Mutual information yes no yes no Bob mutual information: from 1 bit to 0 bit to 1 bit Alice p p 1 p 1 p 1 2 1 2 1 2 1 2 

mutual information: bounded by the smaller entropy 

Off-topic: http://blog.xkcd.com/2010/05/03/color-survey-results/ 

Good to know • Cleaning you room DOES NOT decrease its entropy (macroscoping order in a negligible contribution, but usually you heat it a bit) • Every known sequence has entropy 0 • Unit of information, but typically used for lack of information 01001010001101100101111... 

Homework • What is entropy: • of a shuffled card deck? • word usage? • your bookshelf? • mutual information between countries and religions? 

Thank you! [email protected] 

Jokes & comments • Natural logarithm and alternative medicine • Sauron and wedding ring • Beauty of not knowing