it depends a little bit on the context in which the function will be played : - with an SSD disk, the playback of a large file is very fast, - with a lot of available memory, the file is cached at its first reading and subsequent readings are almost instantaneous. if these two conditions are met, indeed, I am not sure that the function brings a real benefit. otherwise, there is a real interest in using the function. the tests below were performed on a large file placed on a USB 3.0 key. ll -hgG elementaryos-5.1-stable.20191202.iso -rw-r--r-- 1 1.4G Dec 17 14:22 elementaryos-5.1-stable.20191202.iso here the large file is kept in cache : pv elementaryos-5.1-stable.20191202.iso > /dev/null 1.37GiB 0:00:35 [39.4MiB/s] [================================>] 100% pv elementaryos-5.1-stable.20191202.iso > /dev/null 1.37GiB 0:00:00 [4.51GiB/s] [================================>] 100% time ( file --mime-type -b -e compress -e tar -e elf elementaryos-5.1-stable.20191202.iso; md5sum elementaryos-5.1-stable.20191202.iso; sha1sum elementaryos-5.1-stable.20191202.iso ) application/octet-stream d1addd17377aa73700680ff937d7a0f4 elementaryos-5.1-stable.20191202.iso cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f elementaryos-5.1-stable.20191202.iso real 0m4.680s user 0m4.298s sys 0m0.318s file_info(){ echo -n ${1:=/dev/stdin}$'\t'; ( tee < "${1}" >( file --mime-type -b -e compress -e tar -e elf - >&3; cat >/dev/null ) >( md5sum
&3 ) >( sha1sum >&3 ) >/dev/null; ) 3>&1 | tr '\n' '\t'; echo; }
time ( file_info elementaryos-5.1-stable.20191202.iso )
elementaryos-5.1-stable.20191202.iso application/octet-stream
cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f -
d1addd17377aa73700680ff937d7a0f4 -
real 0m3.435s
user 0m0.113s
sys 0m1.176s
now the cache is cleared between each access (eg. limited available memory)
:
function flush { sync && sysctl -q vm.drop_caches=3; }
flush; pv elementaryos-5.1-stable.20191202.iso > /dev/null
1.37GiB 0:00:35 [39.4MiB/s] [================================>] 100%
flush; pv elementaryos-5.1-stable.20191202.iso > /dev/null
1.37GiB 0:00:35 [39.4MiB/s] [================================>] 100%
flush; time ( file --mime-type -b -e compress -e tar -e elf
elementaryos-5.1-stable.20191202.iso; flush; md5sum
elementaryos-5.1-stable.20191202.iso; flush; sha1sum
elementaryos-5.1-stable.20191202.iso )
application/octet-stream
d1addd17377aa73700680ff937d7a0f4 elementaryos-5.1-stable.20191202.iso
cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f
elementaryos-5.1-stable.20191202.iso
real 1m11.054s
user 0m8.092s
sys 0m1.441s
flush; time ( file_info elementaryos-5.1-stable.20191202.iso )
elementaryos-5.1-stable.20191202.iso application/octet-stream
cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f -
d1addd17377aa73700680ff937d7a0f4 -
real 0m35.217s
user 0m0.316s
sys 0m2.146s
Le mer. 18 déc. 2019 à 16:42, Andy Pieters
On Wed, 18 Dec 2019 at 15:32, Pascal via arch-general
wrote: that's awesome, it works ! it was so simple with cat taking over and consuming the data until the
end !
(I added a redirect to /dev/null to cat) big thank you.
I'm interested in the amount of effort you put into this. Isn't the overhead of the pipes etc going to negate any memory/speed improvements you may get from only opening the file once or is there another consideration at play here?