Heads up Processes Daemons Pimping our daemon What’s a daemon Daemons are processes on a system which principal characteristics are: Aren’t directly controlled by the user
Heads up Processes Daemons Pimping our daemon What’s a daemon Daemons are processes on a system which principal characteristics are: Aren’t directly controlled by the user Run in background
Heads up Processes Daemons Pimping our daemon What’s a daemon Daemons are processes on a system which principal characteristics are: Aren’t directly controlled by the user Run in background Are spawned tipically by init manager
Heads up Processes Daemons Pimping our daemon What’s a daemon Daemons are processes on a system which principal characteristics are: Aren’t directly controlled by the user Run in background Are spawned tipically by init manager Tipically log data into logfiles
Heads up Processes Daemons Pimping our daemon What’s a daemon Daemons are processes on a system which principal characteristics are: Aren’t directly controlled by the user Run in background Are spawned tipically by init manager Tipically log data into logfiles Are detached from terminal
Heads up Processes Daemons Pimping our daemon Process What is a process? Is an instance of a computer program that is being executed How processes are created?
Heads up Processes Daemons Pimping our daemon Process What is a process? Is an instance of a computer program that is being executed How processes are created? fork in UNIX
Heads up Processes Daemons Pimping our daemon Process What is a process? Is an instance of a computer program that is being executed How processes are created? fork in UNIX, CreateProcess on the rest
Heads up Processes Daemons Pimping our daemon Process What is a process? Is an instance of a computer program that is being executed How processes are created? fork in UNIX, CreateProcess on the rest What is shared between processes?
Heads up Processes Daemons Pimping our daemon Process What is a process? Is an instance of a computer program that is being executed How processes are created? fork in UNIX, CreateProcess on the rest What is shared between processes? Libraries?
Heads up Processes Daemons Pimping our daemon Process What is a process? Is an instance of a computer program that is being executed How processes are created? fork in UNIX, CreateProcess on the rest What is shared between processes? Libraries? Memory?
Heads up Processes Daemons Pimping our daemon What is a process consisted of? An image of the executable machine code. Virtual memory which includes the executable code, process-specific data, a call stack, and a heap to hold intermediate computation data generated during run time.
Heads up Processes Daemons Pimping our daemon What is a process consisted of? An image of the executable machine code. Virtual memory which includes the executable code, process-specific data, a call stack, and a heap to hold intermediate computation data generated during run time. Operating system descriptors of resources, such as file descriptors.
Heads up Processes Daemons Pimping our daemon What is a process consisted of? An image of the executable machine code. Virtual memory which includes the executable code, process-specific data, a call stack, and a heap to hold intermediate computation data generated during run time. Operating system descriptors of resources, such as file descriptors. Security attributes, such as the process owner and the process’ set of permissions.
Heads up Processes Daemons Pimping our daemon What is a process consisted of? An image of the executable machine code. Virtual memory which includes the executable code, process-specific data, a call stack, and a heap to hold intermediate computation data generated during run time. Operating system descriptors of resources, such as file descriptors. Security attributes, such as the process owner and the process’ set of permissions. Processor state, such as the content of registers, physical memory addressing, etc. The state is typically stored in CPU registers when the process is executing, and in memory otherwise.
Heads up Processes Daemons Pimping our daemon Executing a program puts("Hello from process #{Process.pid}") exec(’uname -r’) puts("Bye from process") Hello from process 23191 3.14-2-amd64
Heads up Processes Daemons Pimping our daemon Executing programs Kernel.exec : Replaces the current process image Kernel.system : Executes cmd in a subshell Kernel.‘ ( backticks )
Heads up Processes Daemons Pimping our daemon Executing programs Kernel.exec : Replaces the current process image Kernel.system : Executes cmd in a subshell Kernel.‘ ( backticks ) : Executes cmd in a subshell IO.popen
Heads up Processes Daemons Pimping our daemon Executing programs Kernel.exec : Replaces the current process image Kernel.system : Executes cmd in a subshell Kernel.‘ ( backticks ) : Executes cmd in a subshell IO.popen : Runs command as a subprocess Open3.popen3 Process.spawn Process.daemon IO.popen4 ( JRuby )
Heads up Processes Daemons Pimping our daemon Creating a new process puts("I am process #{Process.pid}") fork puts("I am process #{Process.pid}") I am process 23823 I am process 23823 I am process 23825
Heads up Processes Daemons Pimping our daemon Executing a new program in a new process puts("I am process #{Process.pid}") pid = fork do puts("I am process #{Process.pid}, my parent is #{Process.ppid}") end puts("I am process #{Process.pid}, I am waiting for process #{pid}") Process.wait(pid)
Heads up Processes Daemons Pimping our daemon Executing a new program in a new process puts("I am process #{Process.pid}") pid = fork do puts("I am process #{Process.pid}, my parent is #{Process.ppid}") end puts("I am process #{Process.pid}, I am waiting for process #{pid}") Process.wait(pid) I am process 11135 I am process 11135, I am waiting for process 11137 I am process 11137, my parent is 11135
Heads up Processes Daemons Pimping our daemon Trying to become a daemon def print_process_info puts("PID: #{Process.pid} PPID: #{Process.ppid} SID: #{Process.getsid} PGRP: #{Process.getpgrp}") end print_process_info exit if fork print_process_info Process.getsid print_process_info exit if fork print_process_info
Heads up Processes Daemons Pimping our daemon Signals trap(:KILL) do puts(’I am not going to die!’) end puts(’Trying to kill myself’) Process.kill(:KILL, Process.pid) puts(’I should be alive’)
Heads up Processes Daemons Pimping our daemon Signals trap(:KILL) do puts(’I am not going to die!’) end puts(’Trying to kill myself’) Process.kill(:KILL, Process.pid) puts(’I should be alive’) Trying to k i l l myself K i l l e d
Heads up Processes Daemons Pimping our daemon Steps to become a UNIX daemon Dissociate from the controlling tty Become a session leader Become a process group leader
Heads up Processes Daemons Pimping our daemon Steps to become a UNIX daemon Dissociate from the controlling tty Become a session leader Become a process group leader Execute as a background task by forking and exiting (once or twice), required sometimes to become a session leader.
Heads up Processes Daemons Pimping our daemon Steps to become a UNIX daemon Dissociate from the controlling tty Become a session leader Become a process group leader Execute as a background task by forking and exiting (once or twice), required sometimes to become a session leader. Setting the root directory (/) as the current working directory so that the process does not keep any directory in use.
Heads up Processes Daemons Pimping our daemon Steps to become a UNIX daemon Dissociate from the controlling tty Become a session leader Become a process group leader Execute as a background task by forking and exiting (once or twice), required sometimes to become a session leader. Setting the root directory (/) as the current working directory so that the process does not keep any directory in use. Changing the umask to 0 to allow operating system calls to provide their own permission masks and not to depend on the umask of the caller
Heads up Processes Daemons Pimping our daemon Steps to become a UNIX daemon Dissociate from the controlling tty Become a session leader Become a process group leader Execute as a background task by forking and exiting (once or twice), required sometimes to become a session leader. Setting the root directory (/) as the current working directory so that the process does not keep any directory in use. Changing the umask to 0 to allow operating system calls to provide their own permission masks and not to depend on the umask of the caller Closing all inherited files at the time of execution that are left open by the parent process, including file descriptors 0, 1 and 2.
Heads up Processes Daemons Pimping our daemon Steps to become a UNIX daemon Dissociate from the controlling tty Become a session leader Become a process group leader Execute as a background task by forking and exiting (once or twice), required sometimes to become a session leader. Setting the root directory (/) as the current working directory so that the process does not keep any directory in use. Changing the umask to 0 to allow operating system calls to provide their own permission masks and not to depend on the umask of the caller Closing all inherited files at the time of execution that are left open by the parent process, including file descriptors 0, 1 and 2. Using a logfile, the console, or /dev/null as stdin, stdout, and stderr
Heads up Processes Daemons Pimping our daemon Daemonizing in Ruby 1.9 Process.daemon Detach the process from controlling terminal and run in the background as system daemon. Unless the argument nochdir is true, it changes the current working directory to /. Unless the argument noclose is true, daemon() will redirect standard input, standard output and standard error to /dev/null. Return zero on success, or raise one of Errno::*.
Heads up Processes Daemons Pimping our daemon proc daemon From https://github.com/ruby/ruby/blob/master/process.c s t a t i c VALUE proc daemon ( i n t argc , VALUE argv ) { VALUE nochdir , n o c l o s e ; i n t n ; r b s e c u r e (2) ; r b s c a n a r g s ( argc , argv , ”02” , &nochdir , &n o c l o s e ) ; p r e f o r k () ; b e f o r e f o r k ( ) ; n = daemon (RTEST( n oc h di r ) , RTEST( n o c l o s e ) ) ; a f t e r f o r k () ; i f ( n < 0) r b s y s f a i l ( ”daemon” ) ; return INT2FIX ( n ) ; } #d e f i n e daemon ( nochdir , n o c l o s e ) rb daemon ( nochdir , n o c l o s e ) #e n d i f
Heads up Processes Daemons Pimping our daemon rb process From https://github.com/ruby/ruby/blob/master/process.c s t a t i c i n t rb daemon ( i n t nochdir , i n t n o c l o s e ) { i n t n , e r r = 0 ; switch ( r b f o r k (0 , 0 , 0 , Qnil ) ) { case −1: r b s y s f a i l ( ”daemon” ) ; case 0: break ; d e f a u l t : e x i t (EXIT SUCCESS) ; } p r o c s e t s i d () ; switch ( r b f o r k (0 , 0 , 0 , Qnil ) ) { case −1: return −1; case 0: break ; d e f a u l t : e x i t (EXIT SUCCESS) ; } i f ( ! noch d ir ) e r r = c h d i r ( ”/” ) ; i f ( ! n o c l o s e && ( n = open ( ”/ dev / n u l l ” , O RDWR, 0) ) != −1) { ( void ) dup2 (n , 0) ; ( void ) dup2 (n , 1) ; ( void ) dup2 (n , 2) ; i f ( n > 2) ( void ) c l o s e ( n ) ; } return e r r ; }
Heads up Processes Daemons Pimping our daemon rb fork From https://github.com/ruby/ruby/blob/master/process.c r b p i d t r b f o r k ( i n t ∗status , i n t (∗ chfunc ) ( void ∗) , void ∗charg , VALUE f d s ) { i f ( chfunc ) { s t r u c t c h f u n c w r a p p e r t warg ; warg . chfunc = chfunc ; warg . arg = charg ; return r b f o r k e r r ( status , chfunc wrapper , &warg , fds , NULL, 0) ; } e l s e { return r b f o r k e r r ( status , NULL, NULL, fds , NULL, 0) ; } }
Heads up Processes Daemons Pimping our daemon rb fork err From https://github.com/ruby/ruby/blob/master/process.c r b p i d t r b f o r k e r r ( i n t ∗status , i n t (∗ chfunc ) ( void ∗ , char ∗ , s i z e t ) , void ∗charg , VALUE fds , char ∗errmsg , s i z e t e r r m s g b u f l e n ) { r b p i d t pid ; i n t err , s t a t e = 0 ; #d e f i n e p r e f o r k () ( d e f i n e \ r b i o f l u s h ( r b s t d o u t ) , \ r b i o f l u s h ( r b s t d e r r ) r b s t d e r r \ ) p r e f o r k () ; // STUFF f o r ( ; b e f o r e f o r k ( ) , ( pid = f o r k () ) < 0; p r e f o r k () ) { a f t e r f o r k () ; // STUFF } i f ( ! pid ) { f o r k e d c h i l d = 1 ; i f ( chfunc ) { i f ( !(∗ chfunc ) ( charg , errmsg , e r r m s g b u f l e n ) ) e x i t (EXIT SUCCESS) ; #i f EXIT SUCCESS == 127 e x i t ( EXIT FAILURE ) ; #e l s e e x i t (127) ; #e n d i f } } a f t e r f o r k () ; return pid ; }
Heads up Processes Daemons Pimping our daemon proc setsid From https://github.com/ruby/ruby/blob/master/process.c s t a t i c VALUE p r o c s e t s i d ( void ) { r b p i d t pid ; r b s e c u r e (2) ; pid = s e t s i d ( ) ; i f ( pid < 0) r b s y s f a i l (0) ; return PIDT2NUM( pid ) ; } #d e f i n e s e t s i d () r u b y s e t s i d ()
Heads up Processes Daemons Pimping our daemon ruby setsid.c From https://github.com/ruby/ruby/blob/master/process.c s t a t i c r b p i d t r u b y s e t s i d ( void ) { r b p i d t pid ; i n t r e t ; pid = g e t p i d ( ) ; #i f d e f i n e d (SETPGRP VOID) r e t = s e t p g r p () ; /∗ I f ‘ p i d t s e t p g r p ( void ) ’ i s e q u i v a l e n t to s e t s i d () , ∗ ‘ r e t ’ w i l l be the same v a l u e as ‘ pid ’ , and f o l l o w i n g open () w i l l f a i l . ∗ In Linux , ‘ i n t s e t p g r p ( void ) ’ i s e q u i v a l e n t to s e t p g i d (0 , 0) . ∗/ #e l s e r e t = s e t p g r p (0 , pid ) ; #e n d i f i f ( r e t == −1) return −1; i f (( fd = open ( ”/ dev / t t y ” , O RDWR) ) >= 0) { i o c t l ( fd , TIOCNOTTY, NULL) ; c l o s e ( fd ) ; } return pid ; } #e n d i f
Heads up Processes Daemons Pimping our daemon proc setpgrp.c From https://github.com/ruby/ruby/blob/master/process.c s t a t i c VALUE p r o c s e t p g r p ( void ) { r b s e c u r e (2) ; /∗ check f o r p o s i x s e t p g i d () f i r s t ; t h i s matches the p o s i x ∗/ /∗ getpgrp ( ) above . I t appears that c o n f i g u r e w i l l s e t SETPGRP VOID ∗/ /∗ even though s e t p g r p (0 ,0) would be p r e f e r r e d . The p o s i x c a l l a v o i d s ∗/ /∗ t h i s c o n f u s i o n . ∗/ #i f d e f HAVE SETPGID i f ( s e t p g i d (0 ,0) < 0) r b s y s f a i l (0) ; #e l i f d e f i n e d (HAVE SETPGRP) && d e f i n e d (SETPGRP VOID) i f ( s e t p g r p () < 0) r b s y s f a i l (0) ; #e n d i f return INT2FIX (0) ; } #e n d i f
Heads up Processes Daemons Pimping our daemon How to daemonize a program in Ruby def daemonize_app exit if fork Process.setsid exit if fork Dir.chdir("/") STDIN.reopen("/dev/null") STDOUT.reopen("/dev/null", "a") STDERR.reopen("/dev/null", "a") end
Heads up Processes Daemons Pimping our daemon Checking it live $ l s −l / proc /2206/ fd t o t a l 0 lrwx−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 0 −> / dev / n u l l lrwx−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 1 −> / dev / n u l l lrwx−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 2 −> / dev / n u l l l r −x−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 3 −> pipe : [ 5 9 5 6 6 3 ] l −wx−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 4 −> pipe : [ 5 9 5 6 6 3 ] l r −x−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 5 −> pipe : [ 5 9 5 6 6 4 ] l −wx−−−−−− 1 s e l u s e l u 64 Sep 21 16:52 6 −> pipe : [ 5 9 5 6 6 4 ] $ ps x f −o pid , ppid , pgid , sid , command | grep −e [ r ] uby −e [P] ID PID PPID PGID SID COMMAND 2206 1 2203 2203 ruby −e Process . daemon ; s l e e p 3600
Heads up Processes Daemons Pimping our daemon Daemonize in daemons gem From https://github.com/ghazel/daemons/blob/master/lib/daemons/daemonize.rb def daemonize ( l o g f i l e n a m e = nil , app name = nil) srand s a f e f o r k and e x i t unless s e s s i d = Process . s e t s i d r a i s e Daemons . RuntimeException . new ( ’ cannot detach from c o n t r o l l i n g t e r m i n a l ’ ) end # Prevent the possibility of acquiring a controlling terminal trap ’SIGHUP ’ , ’IGNORE ’ e x i t if pid = s a f e f o r k $0 = app name if app name Dir . c h d i r ”/” F i l e . umask 0000 ObjectSpace . e a c h o b j e c t ( IO ) do | i o | unless [ STDIN , STDOUT, STDERR ] . i n c l u d e ?( i o ) begin i o . c l o d e unless i o . c l o s e d ? rescue : : Exception end end end r e d i r e c t i o ( l o g f i l e n a m e ) return s e s s i d end
Heads up Processes Daemons Pimping our daemon TCP preforking daemon r e q u i r e ’ socket ’ Process . daemon socket = TCPServer . open ( ’ 0 . 0 . 0 . 0 ’ , 8080) wpids = [ ] 5. times do wpids < < f o r k do loop do connection = socket . accept connection . puts ” H e l l o from #{Process . pid}” connection . c l o s e end end end [ : INT , : QUIT ] . each do | s i g n a l | S i g n a l . trap ( s i g n a l ) do wpids . each { | wpid | Process . k i l l ( s i g n a l , wpid ) } end end Process . w a i t a l l $ nc l o c a l h o s t 8080; nc l o c a l h o s t 8080 Hello from 8349 Hello from 8361
Heads up Processes Daemons Pimping our daemon Our robust daemon Process . daemon socket = TCPServer . open ( ’ 0 . 0 . 0 . 0 ’ , 8080) wpids = [ ] 5. times do wpids < < f o r k do run = true [ : INT , : QUIT ] . each do | s i g n a l | S i g n a l . trap ( s i g n a l , lambda { run = false }) end while run connection = socket . accept connection . puts ( ” H e l l o from #{Process . pid}” ) s l e e p 10 connection . puts ( ”Goodbye from #{Process . pid}” ) connection . c l o s e end end end [ : INT , : QUIT ] . each do | s i g n a l | S i g n a l . trap ( s i g n a l ) do wpids . each { | wpid | Process . k i l l ( s i g n a l , wpid ) } end end Process . w a i t a l l $ nc l o c a l h o s t 8080; k i l l a l l −INT ruby Hello from 8328 Goodbye from 8328
Heads up Processes Daemons Pimping our daemon Our robust daemon with logging Process . daemon socket = TCPServer . open ( ’ 0 . 0 . 0 . 0 ’ , 8080) wpids = [ ] l o g g e r = Logger . new ( ’ /tmp/ l o g g e r . log ’ ) l o g g e r . l e v e l = Logger : : INFO 5 . times do wpids < < f o r k do run = true [ : INT , : QUIT ] . each do | s i g n a l | S i g n a l . trap ( s i g n a l , lambda { run = false }) end while run connection = socket . accept connection . puts ( ” H e l l o from #{Process . pid}” ) l o g g e r . i n f o ( ” Connection i n #{Process . pid}” ) s l e e p 10 connection . puts ( ”Goodbye from #{Process . pid}” ) connection . c l o s e end end end [ : INT , : QUIT ] . each do | s i g n a l | S i g n a l . trap ( s i g n a l ) do wpids . each { | wpid | Process . k i l l ( s i g n a l , wpid ) } end end Process . w a i t a l l l o g g e r . c l o s e $ t a i l −f /tmp/ l o g g e r . log # L o g f i l e c r e a t e d on 2014−09−21 19:46:25 +0200 by l o g g e r . rb /v1 . 2 . 7 I , [2014−09−21T19 :47: 12. 696949 #9452] INFO − − : Connection i n 9452 I , [2014−09−21T19 :47: 22. 702008 #9449] INFO − − : Connection i n 9449
Heads up Processes Daemons Pimping our daemon Logs rotation with a signal From https://github.com/kennethkalmer/daemon-kit/blob/master/lib/daemon kit/application.rb c o n f i g u r a t i o n . trap ( ”HUP” ) { DaemonKit : : A p p l i c a t i o n . r e o p e n l o g s } module DaemonKit class A p p l i c a t i o n def r e o p e n l o g s nr = 0 a p p e n d f l a g s = F i l e : :WRONLY | F i l e : : APPEND DaemonKit . l o g g e r . i n f o ” Rotating l o g s ” if DaemonKit . l o g g e r ObjectSpace . e a c h o b j e c t ( F i l e ) do | fp | next if fp . c l o s e d ? next unless ( fp . sync && fp . path [ 0 . . 0 ] == ”/” ) next unless ( fp . f c n t l ( F c n t l : : F GETFL) & a p p e n d f l a g s ) == a p p e n d f l a g s begin a , b = fp . stat , F i l e . s t a t ( fp . path ) next if a . ino == b . ino && a . dev == b . dev rescue Errno : : ENOENT end open arg = ’ a ’ if fp . r e s p o n d t o ? ( : e x t e r n a l e n c o d i n g ) && enc = fp . e x t e r n a l e n c o d i n g open arg < < ”:#{enc . t o s}” enc = fp . i n t e r n a l e n c o d i n g and open arg < < ”:#{enc . t o s}” end DaemonKit . l o g g e r . i n f o ” Rotating path : #{fp . path}” if DaemonKit . l o g g e r fp . reopen ( fp . path , open arg ) fp . sync = true nr += 1 end # each_object nr end end end
Heads up Processes Daemons Pimping our daemon The end Further info: http://codeincomplete.com/posts/2014/9/15/ruby daemons/ Working with unix processes by Jesse Storimer lib/unicorn/launcher.rb in Unicorn gem daemons, dante and daemon-kit gems Linux System Programming: Talking Directly to the Kernel and C Library Understanding the Linux Kernel
Heads up Processes Daemons Pimping our daemon The end Further info: http://codeincomplete.com/posts/2014/9/15/ruby daemons/ Working with unix processes by Jesse Storimer lib/unicorn/launcher.rb in Unicorn gem daemons, dante and daemon-kit gems Linux System Programming: Talking Directly to the Kernel and C Library Understanding the Linux Kernel Questions?
Heads up Processes Daemons Pimping our daemon The end Further info: http://codeincomplete.com/posts/2014/9/15/ruby daemons/ Working with unix processes by Jesse Storimer lib/unicorn/launcher.rb in Unicorn gem daemons, dante and daemon-kit gems Linux System Programming: Talking Directly to the Kernel and C Library Understanding the Linux Kernel Questions? Thanks!
josacar
mot_techtalk
sr2mg4
ivargrimstad
ogiwarat
dalinaum
nacatl
skanehira
psj59129
loic425
hschwentner
lhespanha
lauravandoore
searls
cromwellryan
denniskardys
ammeep
tanoku
aki_iinuma
destraynor
erikaheidi
eitanlees
maggiecrowley
mraible