Some bash tips -- 3 -- Exit codes management in pipelines

This blog is part of a bash tips list I find useful to use on every script -- the whole list of it can be found here.

Return codes (aka exit codes) are heavily used in any language to know if a command has succeded or has failed. In shell script, the return code of the last executed command is accessible in the $? variable and is 0 if the command has completed successfully and different from 0 if the command has failed like we can see in the below example:

$ echo "blabla" > /tmp/iexist
$ cat /tmp/iexist
blabla
$ echo $?
0
$ cat /tmp/idontexist
cat: /tmp/idontexist: No such file or directory
$ echo $?
1
$

A return code can easily be tested and different actions triggered or a message shown depending on the success or the failure of a command:

$ cat /tmp/iexist
blabla
$ if [ $? -eq 0 ]; then echo "Success"; else echo "Failure"; fi
Success
$ cat /tmp/idontexist
cat: /tmp/idontexist: No such file or directory
$ if [ $? -eq 0 ]; then echo "Success"; else echo "Failure"; fi
Failure
$

In the above example, we can clearly see (and test) that the cat command succeeds when a file exists and fails when a file does not exist.

So far so good but now comes the scenario where you also need to sort the output of the file you just cat:

$ cat /tmp/iexist | sort
blabla
$ if [ $? -eq 0 ]; then   echo "Success"; else   echo "Failure"; fi
Success
$ cat /tmp/idontexist | sort
cat: /tmp/idontexist: No such file or directory
$ if [ $? -eq 0 ]; then   echo "Success"; else   echo "Failure"; fi
Success   <== ?? what ??
$

We can see that in this scenario, $? returns 0 even if the file I cat does not exist ! This is because $? returns the last command return code which in my case is 0 because this is the return code of the sort command !

So how to get the return code of the cat and not the sort ? well, bash has a mechanism for this, the return codes are in the PIPESTATUS array which you can check instead of $? in a multiple pipes command line scenario:

$ cat /tmp/iexist | sort
blabla
$ cecho ${PIPESTATUS[0]}":"${PIPESTATUS[1]}
0:0
$ cat /tmp/idontexist | sort
cat: /tmp/idontexist: No such file or directory
$ echo ${PIPESTATUS[0]}":"${PIPESTATUS[1]}
1:0
$

Here, ${PIPESTATUS[0]} (the first element of the PIPESTATUS array) contains the return code of the first command which is the cat and ${PIPESTATUS[1]} the return code of the sort command.

PIPESTATUS is a special array, you can print it as shown in the more complex example below:

$ cat /tmp/iexist | tac | sort | grep "something" | sort | tac
$ echo ${PIPESTATUS[@]}
0 0 0 1 0 0
      ^
      |
grep grepped nothing so it returned 1
$

Alternatively, you can aso use the set -o pipefail directive on top of our script to return the rightmost non zero exit code of a pipeline; you won't know exactly which one has failed but you will know that something in the pipeline has failed -- which may be enough in some cases:

$ cat /tmp/idontexist | sort
cat: /tmp/idontexist: No such file or directory
$ echo $?
0  <== default behavior
$ set -o pipefail
$ cat /tmp/idontexist | sort
cat: /tmp/idontexist: No such file or directory
$ echo $?
1  <== You don't know which part has failed but the overall result of the pipe is failure
$

PIPESTATUS and pipefail are good to be known to write robust scripts -- enjoy !

---

< Previous bash tip / Next bash tip >