bash "wc -l" command output differs if call or through tee

When I issued two equivalent commands in Bash I got different output (from "wc -l" command), see below:

root@devel:~# ls /usr/bin -lha | tee >(wc -l) >(head) > /dev/null
total 76M
drwxr-xr-x  2 root root      20K Nov 11 18:58 .
drwxr-xr-x 10 root root     4.0K Oct  8 15:31 ..
-rwxr-xr-x  1 root root      51K Feb 22  2017 [
-rwxr-xr-x  1 root root       96 Jan 19  2017 2to3-3.5
-rwxr-xr-x  1 root root      23K Mar 22  2017 addpart
lrwxrwxrwx  1 root root       26 May 10  2017 addr2line -> x86_64-linux-gnu-    addr2line
lrwxrwxrwx  1 root root        6 Dec 13  2016 apropos -> whatis
-rwxr-xr-x  1 root root      15K Sep 13 19:47 apt
-rwxr-xr-x  1 root root      79K Sep 13 19:47 apt-cache
137
root@devel:~# ls /usr/bin -lha | wc -l
648

what am I missing?

it's strange, but when I call it this way it gets even stranger output:

root@devel:~# ls /usr/bin -lha | tee >(wc) >(wc) > /dev/null
648    6121   39179
648    6121   39179
root@devel:~# ls /usr/bin -lha | tee >(wc) >(wc) > /dev/null
648    6121   39179
648    6121   39179
root@devel:~# ls /usr/bin -lha | tee >(wc) >(wc -l) > /dev/null
648
root@devel:~#     648    6121   39179

seems like commands running asynchronously and ends in different time... or what it can be?

1 answer

  • answered 2017-11-13 08:24 Bruce

    Simple answer:

    how to fix:

    ls /usr/bin -lha | tee --output-error=exit-nopipe >(wc -l) >(head) > /dev/null

    Detals:

    The command head only prints the head of input, so it can finish its job as long as it gets enough input, then exits without waiting for all input.

    So let's replace command head with the simple "head".

    ls /usr/bin -lha | tee >(wc -l) >(read l; echo $l) > /dev/null

    The simple "head" will read only one line, then exit, which causes that the pipe file gets closed immediately before tee finishes transferring all data to it.

    So no doubt, you'll get same result with the simple "head". wc still prints wrong number.

    The root reason of your issue, I think you can conclude yourself, is that one of the output file of tee is closed early, tee hits a write error, and then stops writing to other output files.

    After understanding the root reason, I think it would be very easy for you to understand the following section in man page.

    MODE determines behavior with write errors on the outputs:
       'warn' diagnose errors writing to any output
    
       'warn-nopipe'
              diagnose errors writing to any output not a pipe
    
       'exit' exit on error writing to any output
    
       'exit-nopipe'
              exit on error writing to any output not a pipe
    
       The  default MODE for the -p option is 'warn-nopipe'.  The default operation when --output-error
       is not specified, is to exit immediately on error writing to a pipe, and diagnose errors writing
       to non pipe outputs.
    

    Some extra words

    Actually if you replace <(wc -l) with a regular file in your problematic command line, you will find the file size will always be 16384 or 20480, which are all multiples of 4096.

    4096 is the value of PIPE_BUF for most UNIX system. If you know what PIPE_BUF is, you will easily understand why the file size is always multiple of 4096.