Some bash tips -- 9 -- Pause and restart a process

This blog is part of a bash tips list I find useful to use on every script -- the whole list can be found here.

One day, when I was developing this super scheduler able to schedule and parallelize hundreds of jobs, the project manager came to me and asked me:
- "Would it be possible to have a PAUSE and RESTART button on the website to PAUSE and RESTART a job ?"

I assume that in these situations, we have to do a king of poker face not to show the "whaaaaaaaaaat ? what does this guy want ?" you have in mind at this moment and then start working on this. Having a button on a web page sending a PAUSE or a RESTART action to a script is easy but then... how to do PAUSE and RESTART a process at OS level !?

And, as I will show below, it is surpringly easy as it is a feature already provided by the system ! as we already saw this previously, we use the kill command to send signals to processes; so let's have a look at what is already existing (I won't show all the posible signals below, just the one we want):

$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ

Hey, look at these 2 signals in orange, SIGCONT for . . . continue and SIGSTOP for . . . stop !

Let's try them out ! firstly, in a first session, let's simulate a process running which will first show its own PID and then a loop showing the date and an increment number every 2 seconds:

$ echo "pid is "$$; for i in {1..100}; do echo "$(date) -- $i"; sleep 2; done
pid is 7   <== pid of the shell, very small as I did this example on my laptop
Mon Sep  6 22:47:31 AEST 2021 -- 1
Mon Sep  6 22:47:33 AEST 2021 -- 2
Mon Sep  6 22:47:35 AEST 2021 -- 3
Mon Sep  6 22:47:37 AEST 2021 -- 4

Then, in another session, let's pause, wait and restart this PID (look at the timestamps):

$ date; kill -STOP 7
Mon Sep  6 22:47:39 AEST 2021
$ date; kill -CONT 7
Mon Sep  6 22:48:01 AEST 2021
$

Now look at the whole output of the process we have paused and restarted:

$ echo "pid is "$$; for i in {1..100}; do echo "$(date) -- $i"; sleep 2; done
pid is 7
Mon Sep  6 22:47:30 AEST 2021 -- 1
Mon Sep  6 22:47:33 AEST 2021 -- 2
Mon Sep  6 22:47:35 AEST 2021 -- 3
Mon Sep  6 22:47:37 AEST 2021 -- 4   <== I paused    at 22:47:39
Mon Sep  6 22:48:01 AEST 2021 -- 5   <== I restarted at 22:48:01
Mon Sep  6 22:48:04 AEST 2021 -- 6
Mon Sep  6 22:48:06 AEST 2021 -- 7
Mon Sep  6 22:48:08 AEST 2021 -- 8
^C <== end of the example, I CTRL_C it to stop
$

We did it, we have paused and restarted a process, congratulations !

Before wrapping up this blog, note that you can indifferently use the name of the kill signal with SIG or not and also the signal number meaning the below are synonyms:

• kill -18, kill -SIGCONT, kill -CONT
• kill -19, kill -SIGSTOP, kill -STOP

Another interesting thing to know is that you can know which status a process is in from the operating system, check the below the orange status you can get using a simple ps (below my example has PID = 8):

$ ps -f 8
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
fred         8     7  0 11:48 tty1     S      0:00 -bash
$ kill -19 8  <== pause the process
$ ps -f 8
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
fred         8     7  0 11:48 tty1     T      0:00 -bash
$ kill -18 8  <== restart the process
$ ps -f 8
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
fred         8     7  0 11:48 tty1     S      0:00 -bash
$

This is documented in man ps:

PROCESS STATE CODES
       Here are the different values that the s, stat and state output specifiers (header "STAT" or "S") will display to describe the state of a process:
               D    uninterruptible sleep (usually IO)
               I    Idle kernel thread
               R    running or runnable (on run queue)
               S    interruptible sleep (waiting for an event to complete)
               T    stopped by job control signal
               t    stopped by debugger during the tracing
               W    paging (not valid since the 2.6.xx kernel)
               X    dead (should never be seen)
               Z    defunct ("zombie") process, terminated but not reaped by its parent

You can see that the process is in S status when the process is "doing something" and in a T status when the process is paused.

That's all for this one, enjoy your new powers !

---

< Previous bash tip / Next bash tip >