Tweet
Tweet

Slide 1

Slide 1 text

BASH for Control Freaks... Job control freaks. Steven Lembark Workhorse Computing [email protected]

Slide 2

Slide 2 text

Bourne Again BASH v5 is a general-purpose tool. Today we will look at: Job Control: Running background jobs.

Slide 3

Slide 3 text

Control Freaks Welcome BASH processes can be managed as “jobs”. Processes can be Put in the background.

Slide 4

Slide 4 text

Control Freaks Welcome BASH processes can be managed as “jobs”. Processes can be Put in the background. Brought to the foreground.

Slide 5

Slide 5 text

Control Freaks Welcome BASH processes can be managed as “jobs”. Processes can be Put in the background. Brought to the foreground. Stopped & Started.

Slide 6

Slide 6 text

Control Freaks Welcome BASH processes can be managed as “jobs”. Processes can be Put in the background. Brought to the foreground. Stopped & Started. Listed & Tracked.

Slide 7

Slide 7 text

Control Freaks Welcome man 1 bash; /JOB CONTROL JOB CONTROL Job control refers to the ability to selectively stop (suspend) the execution of processes and continue (resume) their execution at a later point. A user typically employs this facility via an interactive interface supplied jointly by the operating system kernel’s terminal driver and bash.

Slide 8

Slide 8 text

Control Freaks Welcome ‘process’ is anyexecutable run from the shell. Single command. Pipeline Executable

Slide 9

Slide 9 text

Step 1: Backgrounding You have one session available. And two big tables to restore from external tables in s3. $ psql > select count(1) from foo_s3archive; count ----------- 1000000000 (1 row) > select count(1) from bar_s3archive; count ----------- 2000000000 (1 row)

Slide 10

Slide 10 text

Step 1: Backgrounding You have one session available. And two big tables to restore from s3. Q:How can we restore them in parallel? $ psql > select count(1) from foo_s3archive; count ----------- 1000000000 (1 row) > select count(1) from bar_s3archive; count ----------- 2000000000 (1 row)

Slide 11

Slide 11 text

Step 1: Backgrounding You have one session available. And two big tables to restore from s3. Q:How can we restore them in parallel? A:By running two copies of psql. $ psql > select count(1) from foo_s3archive; count ----------- 1000000000 (1 row) > select count(1) from bar_s3archive; count ----------- 2000000000 (1 row)

Slide 12

Slide 12 text

Step 1: Backgrounding Safe copies take two steps. $ cat recover_foo.psql; create table foo_tmp like foo; insert into foo_tmp ( select * from foo_external ) ; insert into foo ( select * from foo_tmp ); drop table foo_tmp;

Slide 13

Slide 13 text

Step 1: Backgrounding Start by creating the destination tables. *_tmp tables avoids corruption if the extenal connecion has issues. $ cat recover_foo.psql; create table foo_tmp like foo; insert into foo_tmp ( select * from foo_external ) ; insert into foo ( select * from foo_tmp ) ; drop table foo_tmp;

Slide 14

Slide 14 text

Step 1: Backgrounding Now run the first script. This is going to take a while. $ psql > \i recover_foo

Slide 15

Slide 15 text

Step 1: Backgrounding Background the running insert. Hitting ctrl-Z (“^Z”) in BASH sends a SIGTSTP to the running process. The process is not dead, but also not running. $ psql > \i recover_foo ^Z [1]+ Stopped psql

Slide 16

Slide 16 text

Step 1: Backgrounding You can see the backgrounded process with “jobs”. This shows that job #1 is stopped. Not running, but also not dead. $ jobs [1]+ Stopped psql

Slide 17

Slide 17 text

Step 1: Backgrounding Now background the job with “bg”. This shows job #1 is running ‘&’. It also gives us a prompt. $ jobs [1]+ Stopped psql $ bg [1]+ psql & $

Slide 18

Slide 18 text

Step 1: Backgrounding Run another psql. Background it also. Job #1 is still running. Job #2 is ready to be started in the background. $ psql > \i recover_bar ^Z [2]+ Stopped psql $ jobs [1]- Running psql & [2]+ Stopped psql

Slide 19

Slide 19 text

Step 1: Backgrounding Run another psql. Background it also. Job #1 is still running. Job #2 is ready to be started in the background. $ psql > \i recover_bar ^Z [2]+ Stopped psql $ jobs [1]- Running psql & [2]+ Stopped psql $ bg [2]+ psql &

Slide 20

Slide 20 text

Step 1: Backgrounding Viola! Two running psql jobs! Catch: You don’t know what they are doing. $ jobs [1]- Running psql & [2]+ Running psql &

Slide 21

Slide 21 text

Step 1: Backgrounding Better way: \o redirects output. Allows a tail to check the query results. $ psql > \o /var/tmp/recover_foo.out > \i recover_foo.psql ^Z [1]+ Stopped psql $ bg [1]+ psql & $ psql > \o /var/tmp/recover_bar.out > \i recover_bar.psql ^Z [2]+ Stopped psql $ bg [2]+ psql & $ tail -f \ /var/tmp/recover-{foo,bar}.out;

Slide 22

Slide 22 text

Step 2: Foregrounding Say job #1 finishes... Use job control to foreground psql. The ‘%’ prefix tells bash to work on a job number. Hit Enter for a prompt. $ tail -f ... INSERT 0 1000000000 $ jobs [1]- Running psql & [2]+ Running psql & $ fg %1 psql >

Slide 23

Slide 23 text

Step 2: Foregrounding You can start another psql job. Or just exit. $ tail -f ... ... INSERT 0 2000000000 $ jobs [1]- Running psql & [2]+ Running psql & $ fg %1 psql > exit

Slide 24

Slide 24 text

Step 2: Foregrounding This leaves job #2 still running. $ jobs [2]+ Running psql &

Slide 25

Slide 25 text

Step 2: Foregrounding This leaves job #2 still running. Notice: The job number doesn’t change! Even if %1 exits, the one running job is still “%2”. $ jobs [2]+ Running psql &

Slide 26

Slide 26 text

bg and SIGHUP Background jobs ignore HUPS. If you loose your connection the job keeps running. Catch: It lacks a terminal. $ jobs [2]+ Running psql &

Slide 27

Slide 27 text

bg and SIGHUP Background jobs ignore HUPS. If you loose your connection the job keeps running. Catch: It lacks a terminal. Your new process has no jobs. $ jobs

Slide 28

Slide 28 text

bg and SIGHUP Background jobs ignore HUPS. If you loose your connection jobs usually keep running. Note: Systems may deliver more than SIGHUP to kill orphaned proc’s! $ jobs

Slide 29

Slide 29 text

bg and SIGHUP Background jobs ignore HUPS. If you loose your connection jobs usually keep running. Catch: They have no controlling termainal! $ jobs

Slide 30

Slide 30 text

bg and SIGHUP Few fixes if you get disconnected: ”reptr” is a utility for re- attaching to terminals.

Slide 31

Slide 31 text

bg and SIGHUP Few fixes if you get disconnected: End your SQL script with an “exit”. Eventually psql exits on its own.

Slide 32

Slide 32 text

bg and SIGHUP Few fixes if you get disconnected: Put “\q” into the buffer before backgrounding the job. This will cause an exit once the “\i” completes.

Slide 33

Slide 33 text

bg and SIGHUP Few fixes if you get disconnected: psql, like most programs, will exit gracefully on a TERM. Use “pkill -TERM psql” to locate the job and stop it that way. $ pgrep -a psql; 54741 psql $ kill -TERM 54741; # or just $ pkill -TERM psql;

Slide 34

Slide 34 text

%1 %ce %?ce %1, %2... list jobs by number. %XX uses the job name. %?XX looks for ‘XX’ anywhere in the command. $ psql … ^Z $ fg %vs; # psql somescript & $ fg %?somescript;

Slide 35

Slide 35 text

Shortcuts Foreground doesn’t require “fg”, just the ‘%’ specifier. “%1” is job #1. “%ps” is a job running ”psql”. “%?somescript” searches the entire commandl line. $ psql … ^Z # these both work $ %1 $ %ps # psql somescript & $ %?somescript;

Slide 36

Slide 36 text

Shortcuts %% is the last stopped job. %+ is the last backgrounded job. $ psql … ^Z # fg the last job stopped $ fg %% # psql somescript & $ %?somescript;

Slide 37

Slide 37 text

Bedside reading As always: man bash; Learning the bash Shell, 3rd Edition https://www.oreilly.com/library/view/learning-the-bash/0596009658/ Lots of alternatives online: https://duckduckgo.com/?q=learning+bash+shell&t=ffab&ia=web