Tweet
Tweet

Slide 1

Slide 1 text

TERMINAL WHISPERING Do more than print text with your terminal | | | Thomas Ballinger @ballingt [email protected] [email protected]

Slide 2

Slide 2 text

TERMINALS ARE COOL

Slide 3

Slide 3 text

WHAT ARE TERMINALS?

Slide 4

Slide 4 text

a p p l i c a t i o n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s

Slide 5

Slide 5 text

a p p l i c a t i o n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t y p e w r i t e r < - - - - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s

Slide 6

Slide 6 text

a p p l i c a t i o n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s

Slide 7

Slide 7 text

a p p l i c a t i o n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s

Slide 8

Slide 8 text

a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) Λ | | | r e a d ( ) r e t u r n s b y t e s | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 9

Slide 9 text

a p p l i c a t i o n - - - - - - - p r i n t ( ) - - + Λ | | | i n p u t ( ) | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 10

Slide 10 text

What bytes can we write and what happens? a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) Λ | | | r e a d ( ) r e t u r n s b y t e s | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 11

Slide 11 text

WHAT BYTES CAN WE SEND? ASCII bytes s y s . s t d o u t . b u f f e r . w r i t e ( b ' s o m e b y t e s ' ) control characters ([ c h r ( x ) f o r x i n r a n g e ( 3 2 ) ] e.g. ' \ a \ n \ r \ v \ b \ f \ t ' )

Slide 12

Slide 12 text

i m p o r t t i m e , s y s d e f r e p o r t _ p r o g r e s s ( r a t i o , w i d t h = 5 0 ) : f i l l e d = ' = ' * i n t ( r a t i o * w i d t h ) r e s t = ' - ' * ( w i d t h - i n t ( r a t i o * w i d t h ) ) s y s . s t d e r r . w r i t e ( ' \ r | ' + f i l l e d + r e s t + ' | ' ) s y s . s t d e r r . f l u s h ( ) f o r i i n r a n g e ( 1 0 1 ) : r e p o r t _ p r o g r e s s ( i / 1 0 0 . 0 , 5 0 ) t i m e . s l e e p ( . 0 1 )

Slide 13

Slide 13 text

WHAT BYTES CAN WE SEND? ASCII bytes s y s . s t d o u t . b u f f e r . w r i t e ( b ' s o m e b y t e s ' ) control characters ([ c h r ( x ) f o r x i n r a n g e ( 3 2 ) ] or ' \ a \ n \ r \ v \ b \ f \ t ' ) if terminal using an encoding, several bytes to make a character: s y s . s t d o u t . b u f f e r . w r i t e ( b ' \ x e 2 \ x 8 8 \ x 8 2 ' ) s y s . s t d o u t . w r i t e ( u ' ∂ ' ) (via . e n c o d i n g )

Slide 14

Slide 14 text

WHAT BYTES CAN WE SEND? ASCII bytes s y s . s t d o u t . b u f f e r . w r i t e ( b ' s o m e b y t e s ' ) control characters ([ c h r ( x ) f o r x i n r a n g e ( 3 2 ) ] or ' \ a \ n \ v \ b \ f \ t ' ) several bytes may encode a single character: s y s . s t d o u t . b u f f e r . w r i t e ( b ' \ x e 2 \ x 8 8 \ x 8 2 ' ) s y s . s t d o u t . w r i t e ( u ' ∂ ' ) (via . e n c o d i n g ) ANSI escape sequences (' \ x 1 b [ 3 1 m ' )

Slide 15

Slide 15 text

ANSI ESCAPE SEQUENCES Sequence Meaning \ x 1 b [ 2 3 A move cursor 23 rows up \ x 1 b [ 2 J clear entire screen \ x 1 b [ ? 2 5 l hide the cursor \ x 1 b [ 1 m start writing in bold \ x 1 b [ 3 1 m start writing in red See for more. Wikipedia entry on ANSI escape sequences

Slide 16

Slide 16 text

COMPATIBILITY CONCERNS termcap (then later terminfo) curses tput ANSI standardization efforts

Slide 17

Slide 17 text

e c h o " I n p u t p l e a s e " t p u t c u u 1 t p u t c u u 1 t p u t e l r e a d a k e y t p u t c l e a r code sample from http://aplawrence.com/Basics/tput_stty.html

Slide 18

Slide 18 text

Blessings - Eric Rose - see "Designing Poetic APIs" from last year f r o m b l e s s i n g s i m p o r t T e r m i n a l t = T e r m i n a l ( ) p r i n t t . b o l d ( ' H i t h e r e ! ' ) p r i n t t . b o l d _ r e d _ o n _ b r i g h t _ g r e e n ( ' I t h u r t s m y e y e s ! ' ) w i t h t . l o c a t i o n ( 0 , t . h e i g h t - 1 ) : p r i n t ' T h i s i s a t t h e b o t t o m . '

Slide 19

Slide 19 text

SECRETS UNLOCKED BY WRITING BYTES TO THE TERMINAL Colored and styled text (bold, underline, blink...) Moving the cursor anywhere Alternate screen Hide cursor etc. (scroll regions, save/restore cursor position)

Slide 20

Slide 20 text

What happens when the user types at the keyboard? a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) Λ | | | r e a d ( ) r e t u r n s b y t e s | | | t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 21

Slide 21 text

What happens when the user types at the keyboard? buffering, line editing discipline data not passed to application until the user hits return backspace, delete word, reprint line, delete line - not much else (try cat to see this behavior) echoing back characters

Slide 22

Slide 22 text

a p p l i c a t i o n - - - - - - - - m a y b e c o p p e r ? Λ | | | s i g n a l o v e r m e t a l w i r e | | | t e l e t y p e w r i t e r < - - - - - - - - - - - + Λ | | i n k | | | V | p a p e r | f i n g e r s p r e s s i n g k e y s

Slide 23

Slide 23 text

+ - - - - a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) | Λ | g e t | | & r e a d ( ) r e t u r n s b y t e s | s e t | | | | | + - - - ⚒ t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 24

Slide 24 text

Relevant man pages raw termios ioctl fcntl stty

Slide 25

Slide 25 text

Turning off echo, using stty $ s t t y - e c h o $ e c h o " e c h o i s t u r n e d o f f " $ s t t y e c h o

Slide 26

Slide 26 text

Turning off echo in Python > > > i m p o r t g e t p a s s > > > g e t p a s s . g e t p a s s ( ) P a s s w o r d : ' s e c r e t '

Slide 27

Slide 27 text

Turning off echo in Python from https://docs.python.org/2/library/termios.html d e f g e t p a s s ( p r o m p t = " P a s s w o r d : " ) : i m p o r t t e r m i o s , s y s f d = s y s . s t d i n . f i l e n o ( ) o l d = t e r m i o s . t c g e t a t t r ( f d ) n e w = t e r m i o s . t c g e t a t t r ( f d ) n e w [ 3 ] = n e w [ 3 ] & ~ t e r m i o s . E C H O # l f l a g s t r y : t e r m i o s . t c s e t a t t r ( f d , t e r m i o s . T C S A D R A I N , n e w ) p a s s w d = r a w _ i n p u t ( p r o m p t ) f i n a l l y : t e r m i o s . t c s e t a t t r ( f d , t e r m i o s . T C S A D R A I N , o l d ) r e t u r n p a s s w d

Slide 28

Slide 28 text

i m p o r t s y s , t e r m i o s c l a s s N o E c h o : d e f _ _ i n i t _ _ ( s e l f , i n _ s t r e a m ) : s e l f . f d = i n _ s t r e a m . f i l e n o ( ) d e f _ _ e n t e r _ _ ( s e l f ) : s e l f . o r i g = t e r m i o s . t c g e t a t t r ( s e l f . f d ) n e w = t e r m i o s . t c g e t a t t r ( s e l f . f d ) n e w [ 3 ] = n e w [ 3 ] & ~ t e r m i o s . E C H O t e r m i o s . t c s e t a t t r ( f d , t e r m i o s . T C S A D R A I N , n e w ) d e f _ _ e x i t _ _ ( s e l f , e x c _ t y p e , e x c _ v a l u e , t r a c e b a c k ) : t e r m i o s . t c s e t a t t r ( s e l f . f d ) , t e r m i o s . T C S A D R A I N , s e l f . o r i g )

Slide 29

Slide 29 text

w i t h N o E c h o ( s y s . s t d i n ) : h = s y s . s t d i n . r e a d l i n e ( ) p r i n t ( h )

Slide 30

Slide 30 text

SECRET TECHNIQUES MADE POSSIBLE WITH THESE DIALS turn off echo send keys immediately current terminal size nonblocking input

Slide 31

Slide 31 text

What happens when the user types at the keyboard buffering, line editing discipline data not passed to application until the user hits return backspace, delete word, reprint line, delete line - not much else to see this, try cat (it doesn't use readline) echoing back characters mapping bytes to keys

Slide 32

Slide 32 text

EXAMPLE BYTES/KEY MAPPINGS with my particular setup Bytes read Key presses b ' a ' a b ' \ x 1 b [ 2 3 ~ ' F11 b ' \ x 1 b [ Z ' Shift-Tab b ' \ x 1 b [ 1 ; 1 0 B ' Meta-Shift-Down b ' \ x 1 b [ 1 ~ ' Home

Slide 33

Slide 33 text

What happens when the user types at the keyboard buffering, line editing discipline data not passed to application until the user hits return backspace, delete word, reprint line, delete line - not much else to see this, try cat (it doesn't use readline) echoing back characters mapping bytes to keys bytes sent without being typed (cursor query)

Slide 34

Slide 34 text

> > > p r i n t ( " \ x 1 b [ 6 n " ) + - - - - a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) | Λ | g e t | | & r e a d ( ) r e t u r n s b y t e s | s e t | | | | | + - - - ⚒ t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 35

Slide 35 text

> > > p r i n t ( " \ x 1 b [ 6 n " )

Slide 36

Slide 36 text

> > > p r i n t ( " \ x 1 b [ 6 n " ) > > > 1 R

Slide 37

Slide 37 text

> > > p r i n t ( " \ x 1 b [ 6 n " ) > > > E S C [ 3 ; 1 R

Slide 38

Slide 38 text

SIGNALS

Slide 39

Slide 39 text

How can the terminal notify the application? + - - - - a p p l i c a t i o n - - - - - - - w r i t e ( b y t e s ) | Λ ⚡ | g e t | ⚡ | & r e a d ( ) ⚡s i g n a l s ! | s e t | ⚡ | | | ⚡ | + - - - ⚒ t e r m i n a l < - - - - - - - - - - - - - - - - - + Λ | | + - - G U I s t u f f - - + | | k e y p r e s s e s V | s c r e e n k e y b o a r d

Slide 40

Slide 40 text

Unix signals triggered from the terminal SIGINT SIGTSTP SIGWINCH tons of others, see The TTY demystified

Slide 41

Slide 41 text

COOKED, RAW AND CBREAK MODES

Slide 42

Slide 42 text

FANCY TERMINAL TRICKS IN PRACTICE

Slide 43

Slide 43 text

TOOLS Blessings - Eric Rose - see "Designing Poetic APIs" from last year Urwid - Ian Ward - a few examples Click - Armin Ronacher Clint - Kenneth Reitz Python Prompt Toolkit - Jonathan Slenders Curtsies - me!

Slide 44

Slide 44 text

BLESSINGS

Slide 45

Slide 45 text

URWID

Slide 46

Slide 46 text

CLICK, CLINT

Slide 47

Slide 47 text

CURTSIES

Slide 48

Slide 48 text

PYTHON PROMPT TOOLKIT

Slide 49

Slide 49 text

TOOLS Blessings - Eric Rose - see "Designing Poetic APIs" from last year Urwid - Ian Ward - a few examples Click - Armin Ronacher Clint - Kenneth Reitz Curtsies - me! Python Prompt Toolkit - Jonathan Slenders

Slide 50

Slide 50 text

RANDOM COOL STUFF termcast asciinema tests I've written

Slide 51

Slide 51 text

$ t e l n e t t e r m c a s t . o r g https://github.com/doy/python-termcast-server https://github.com/doy/python-termcast-client

Slide 52

Slide 52 text

demo of displaying test failures

Slide 53

Slide 53 text

FURTHER READING: Terminals, shells, tty, console The TTY Demystified FUN VIDEOS http://utcc.utoronto.ca/~cks/space/blog/unix/TypingEOFEffe Loading 4K BASIC with a Teletype Teletype Model 35ASR How The Teleprinter Works- 1940 IMAGE CREDITS Wordcloud credit: https://www.jasondavies.com/wordcloud

Slide 54

Slide 54 text

THANKS FOR LISTENING! I'll be tweeting terminal tips from @ballingt I'll announce when the slides are up on Twitter as well I'll be at the first day of sprints, happy to work on bpython if anyone wants