hi Ralph, thank you for that clarification. the function works a little faster with them. file_info(){ echo -n ${1:=/dev/stdin}$'\t'; ( tee < "${1}" >( file --mime-type -b -e compress -e tar -e elf - >&3; cat >/dev/null ) >( md5sum
&3 ) >( sha1sum >&3 ) >/dev/null; ) 3>&1 | tr '\n' '\t'; echo; }
pv big.tar > /dev/null 1,71GiO 0:00:00 [5,32GiB/s] [==============================================================================================================>] 100% time ( for i in $( seq 24 ); do file_info big.tar ; done ) big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc - 6989542fabd98b04086524d1106b7907 - ... big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc - 6989542fabd98b04086524d1106b7907 - real 3m2,712s user 0m14,988s sys 1m13,303s file_info(){ echo -n ${1:=/dev/stdin}$'\t'; ( trap "" pipe; tee < "${1}" >( md5sum >&3 ) >( sha1sum >&3 ) | file --mime-type -b -e compress -e tar -e elf - ) 3>&1 | tr '\n' '\t'; echo; } pv big.tar > /dev/null 1,71GiO 0:00:00 [5,37GiB/s] [==============================================================================================================>] 100% time ( for i in $( seq 24 ); do file_info big.tar ; done ) big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc - 6989542fabd98b04086524d1106b7907 - ... big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc - 6989542fabd98b04086524d1106b7907 - real 2m36,013s user 0m9,349s sys 0m50,257s Le jeu. 19 déc. 2019 à 11:59, Ralph Corderoy <ralph@inputplus.co.uk> a écrit :
Hi Pascal,
file_info(){ echo -n ${1:=/dev/stdin}$'\t' ( tee < "${1}" \ >( file --mime-type -b -e compress -e tar -e elf - >&3 ) \ >( md5sum >&3 ) \ >( sha1sum >&3 ) \ >/dev/null ) 3>&1 | tr '\n' '\t' echo }
it no longer works because the data flow is quickly interrupted by tee which does not consume all the data.
You're missing the reason why. tee(1) receives a SIGPIPE because it writes to a pipe that's closed. Adding a cat(1) is a waste of CPU, as is discarding tee's stdout instead of using it for one of the workers.
Examine these differences.
$ seq 31415 | wc 31415 31415 177384 $ seq 31415 | tee >(sed q) >(wc) > >(tr -d 42 | wc); sleep 1 1 14139 14109 62130 12773 12774 65536 $ seq 31415 | (trap '' pipe; tee >(sed q) >(wc) > >(tr -d 42 | wc)); sleep 1 1 31415 31369 142504 31415 31415 177384 $
Note the output of the commands can be in any order, and intermingle if they're long enough.
tee(1) has -p and --output-error but they're not as specific as stating SIGPIPE is expected for just one worker.
-- Cheers, Ralph.