it depends a little bit on the context in which the function will be played : - with an SSD disk, the playback of a large file is very fast, - with a lot of available memory, the file is cached at its first reading and subsequent readings are almost instantaneous. if these two conditions are met, indeed, I am not sure that the function brings a real benefit. otherwise, there is a real interest in using the function. the tests below were performed on a large file placed on a USB 3.0 key. ll -hgG elementaryos-5.1-stable.20191202.iso -rw-r--r-- 1 1.4G Dec 17 14:22 elementaryos-5.1-stable.20191202.iso here the large file is kept in cache : pv elementaryos-5.1-stable.20191202.iso > /dev/null 1.37GiB 0:00:35 [39.4MiB/s] [================================>] 100% pv elementaryos-5.1-stable.20191202.iso > /dev/null 1.37GiB 0:00:00 [4.51GiB/s] [================================>] 100% time ( file --mime-type -b -e compress -e tar -e elf elementaryos-5.1-stable.20191202.iso; md5sum elementaryos-5.1-stable.20191202.iso; sha1sum elementaryos-5.1-stable.20191202.iso ) application/octet-stream d1addd17377aa73700680ff937d7a0f4 elementaryos-5.1-stable.20191202.iso cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f elementaryos-5.1-stable.20191202.iso real 0m4.680s user 0m4.298s sys 0m0.318s file_info(){ echo -n ${1:=/dev/stdin}$'\t'; ( tee < "${1}" >( file --mime-type -b -e compress -e tar -e elf - >&3; cat >/dev/null ) >( md5sum
&3 ) >( sha1sum >&3 ) >/dev/null; ) 3>&1 | tr '\n' '\t'; echo; }
time ( file_info elementaryos-5.1-stable.20191202.iso ) elementaryos-5.1-stable.20191202.iso application/octet-stream cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f - d1addd17377aa73700680ff937d7a0f4 - real 0m3.435s user 0m0.113s sys 0m1.176s now the cache is cleared between each access (eg. limited available memory) : function flush { sync && sysctl -q vm.drop_caches=3; } flush; pv elementaryos-5.1-stable.20191202.iso > /dev/null 1.37GiB 0:00:35 [39.4MiB/s] [================================>] 100% flush; pv elementaryos-5.1-stable.20191202.iso > /dev/null 1.37GiB 0:00:35 [39.4MiB/s] [================================>] 100% flush; time ( file --mime-type -b -e compress -e tar -e elf elementaryos-5.1-stable.20191202.iso; flush; md5sum elementaryos-5.1-stable.20191202.iso; flush; sha1sum elementaryos-5.1-stable.20191202.iso ) application/octet-stream d1addd17377aa73700680ff937d7a0f4 elementaryos-5.1-stable.20191202.iso cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f elementaryos-5.1-stable.20191202.iso real 1m11.054s user 0m8.092s sys 0m1.441s flush; time ( file_info elementaryos-5.1-stable.20191202.iso ) elementaryos-5.1-stable.20191202.iso application/octet-stream cbe37d55c44db5bb0ab0ae6f5bf0eb96209bb16f - d1addd17377aa73700680ff937d7a0f4 - real 0m35.217s user 0m0.316s sys 0m2.146s Le mer. 18 déc. 2019 à 16:42, Andy Pieters <arch-general@andypieters.me.uk> a écrit :
On Wed, 18 Dec 2019 at 15:32, Pascal via arch-general <arch-general@archlinux.org> wrote:
that's awesome, it works ! it was so simple with cat taking over and consuming the data until the
end !
(I added a redirect to /dev/null to cat) big thank you.
I'm interested in the amount of effort you put into this. Isn't the overhead of the pipes etc going to negate any memory/speed improvements you may get from only opening the file once or is there another consideration at play here?